From asolano at icai.es Mon Apr 1 01:14:11 2013 From: asolano at icai.es (=?UTF-8?Q?Alfredo_Solano_Mart=C3=ADnez?=) Date: Mon, 1 Apr 2013 01:14:11 +0200 Subject: [Python-Dev] A bit about the GIL Message-ID: Hi, I know this may be tiresome by now, so feel free to ignore, but I'd like to share with the list an idea about the GIL, more specifically the reference counting mechanism. Simply put, make the reference counter a sharded one. That is, separate it into several subcounters, in this case one for each thread. The logic would then be something like this: - when increasing the refcount, a thread writes only to its own subcounter, creating one first if necessary. - similarly, when decreasing the refcount, there is no need to access other subcounters until that subcounter reaches zero. - when a subcounter gets to zero, delete it, and read the other subcounters to check if it was the last one. - delete the object only if there are no more subcounters. Contention could then be reduced to a minimum, since a thread only needs to read other subcounters when its own reaches zero or wants the total value. Depending on the implementation it might help with false sharing too, as subcounters may or may not be in the same cache-line. Unfortunately, in a crude test of mine there is already a severe performance degradation, and that is without rwlocks. I've used a basic linked list, and changed the INCREF/DECREF macros to functions to accommodate the extra logic so it may not be the best approach (too many dereferences). Does this makes sense to anyone? Thanks, Alfredo PS: At the very least, it might be another reason to keep the GIL. From rdmurray at bitdance.com Mon Apr 1 03:49:19 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Sun, 31 Mar 2013 21:49:19 -0400 Subject: [Python-Dev] A bit about the GIL In-Reply-To: References: Message-ID: <20130401014919.E4BE5250BC3@webabinitio.net> On Mon, 01 Apr 2013 01:14:11 +0200, =?UTF-8?Q?Alfredo_Solano_Mart=C3=ADnez?= wrote: > Simply put, make the reference counter a sharded one. That is, separate it > into several subcounters, in this case one for each thread. It seems to me this has a family resemblance to some of the stuff Trent is doing in his parallel Python work. --David From ncoghlan at gmail.com Mon Apr 1 04:06:49 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 1 Apr 2013 12:06:49 +1000 Subject: [Python-Dev] A bit about the GIL In-Reply-To: <20130401014919.E4BE5250BC3@webabinitio.net> References: <20130401014919.E4BE5250BC3@webabinitio.net> Message-ID: On Mon, Apr 1, 2013 at 11:49 AM, R. David Murray wrote: > On Mon, 01 Apr 2013 01:14:11 +0200, > =?UTF-8?Q?Alfredo_Solano_Mart=C3=ADnez?= wrote: > > Simply put, make the reference counter a sharded one. That is, separate > it > > into several subcounters, in this case one for each thread. > > It seems to me this has a family resemblance to some of the > stuff Trent is doing in his parallel Python work. > It's also a thread for python-ideas, not python-dev. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Mon Apr 1 11:32:12 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 1 Apr 2013 11:32:12 +0200 Subject: [Python-Dev] A bit about the GIL References: Message-ID: <20130401113212.15d626d4@pitrou.net> Hello, On Mon, 1 Apr 2013 01:14:11 +0200 Alfredo Solano Mart?nez wrote: > > The logic would then be something like this: > - when increasing the refcount, a thread writes only to its own subcounter, > creating one first if necessary. > - similarly, when decreasing the refcount, there is no need to access other > subcounters until that subcounter reaches zero. > - when a subcounter gets to zero, delete it, and read the other subcounters > to check if it was the last one. But then you will decrement another subcounter, right? Meaning you need a lock around all increments / decrements. > Unfortunately, in a crude test of mine there is already a severe performance > degradation, and that is without rwlocks. Yes, there will be. Given how often INCREF and DECREF are called, any complication of their implementation will significantly decrease single-threaded performance. See also http://mail.python.org/pipermail/python-ideas/2009-November/006599.html Regards Antoine. From grigory.v.p at gmail.com Mon Apr 1 22:51:21 2013 From: grigory.v.p at gmail.com (Grigory Petrov) Date: Tue, 2 Apr 2013 00:51:21 +0400 Subject: [Python-Dev] What code in Python creates .exe launchers for 'entry_points' in 'setup.py'? Message-ID: Hello. On Windows, 'setuptools' and 'distribute' package systems allows to add 'entry_points' definition into 'setup.py' python distribution script. For each entry in this definition, some kind of bootstrapper '.exe' file will be created and placed into 'Scripts' python dir. For example, if i install 'pip' via 'setup.py', a file named 'pip.exe' will be created and placed into 'C:\Python27\Scripts'. But what python code is responsible for creation of this bootstrapper executables? I have searched python 2.7.3 source code for some time, but can't find a place. Of course i can spend lots of time to debug 'setup.py' installation (really lots of source code), but maybe someone just knows the place and can pinpoint it? -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric at netwok.org Mon Apr 1 22:58:03 2013 From: eric at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Mon, 01 Apr 2013 16:58:03 -0400 Subject: [Python-Dev] What code in Python creates .exe launchers for 'entry_points' in 'setup.py'? In-Reply-To: References: Message-ID: <5159F4DB.1080105@netwok.org> Hello, Le 01/04/2013 16:51, Grigory Petrov a ?crit : > But what python code is responsible for creation of this bootstrapper > executables? I have searched python 2.7.3 source code for some time, but > can't find a place. Setuptools is not in the standard library, so you would need to search the setuptools or distribute codebases. Regards From dholth at gmail.com Mon Apr 1 23:22:00 2013 From: dholth at gmail.com (Daniel Holth) Date: Mon, 1 Apr 2013 17:22:00 -0400 Subject: [Python-Dev] What code in Python creates .exe launchers for 'entry_points' in 'setup.py'? In-Reply-To: <5159F4DB.1080105@netwok.org> References: <5159F4DB.1080105@netwok.org> Message-ID: The new "distlib" also provides that feature On Apr 1, 2013 4:58 PM, "?ric Araujo" wrote: > Hello, > > Le 01/04/2013 16:51, Grigory Petrov a ?crit : > > But what python code is responsible for creation of this bootstrapper > > executables? I have searched python 2.7.3 source code for some time, but > > can't find a place. > > Setuptools is not in the standard library, so you would need to search > the setuptools or distribute codebases. > > Regards > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/dholth%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kristjan at ccpgames.com Mon Apr 1 22:20:34 2013 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Mon, 1 Apr 2013 20:20:34 +0000 Subject: [Python-Dev] relative import circular problem Message-ID: I just ran into the issue described in http://stackoverflow.com/questions/6351805/cyclic-module-dependencies-and-relative-imports-in-python. This is unfortunate, because we have been trying to move towards relative imports in order to aid flexibility in package and library design. The relative import syntax (from foo import bar) is a getattr type of lookup (i.e. import foo, then get attr from it). This is in contrast with absolute import import foo.bar (get the module foo.bar from sys.modules or import it) bar = foo.bar the latter works with partially initialized modules, but not the former, rendering two sibling modules unable to import each other using the relative syntax. as far as I know, relative imports are only supported using the former (import from) syntax. Are there any plans to alleviate this by allowing proper relative imports? After all, relative imports and packages go hand in hand. K -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Tue Apr 2 00:15:59 2013 From: barry at python.org (Barry Warsaw) Date: Mon, 1 Apr 2013 18:15:59 -0400 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: <20130401181559.0ea23b5b@anarchist> On Apr 01, 2013, at 08:20 PM, Kristj?n Valur J?nsson wrote: >The relative import syntax > > (from foo import bar) is a getattr type of lookup (i.e. import foo, then get > attr from it). > >This is in contrast with absolute import > > import foo.bar (get the module foo.bar from sys.modules or import it) > > bar = foo.bar I always thought of both syntaxes as absolute imports because they both name the full dotted path to the module you want. This contrasts with true PEP 328 relative imports such as: from ..foo import bar >the latter works with partially initialized modules, but not the former, >rendering two sibling modules unable to import each other using the relative >syntax. > >as far as I know, relative imports are only supported using the former >(import from) syntax. Are there any plans to alleviate this by allowing >proper relative imports? After all, relative imports and packages go hand in >hand. I personally dislike PEP 328 relative imports, since they seem fragile, but that's a different discussion. -Barry From brett at python.org Tue Apr 2 00:37:53 2013 From: brett at python.org (Brett Cannon) Date: Mon, 1 Apr 2013 18:37:53 -0400 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: On Mon, Apr 1, 2013 at 4:20 PM, Kristj?n Valur J?nsson < kristjan at ccpgames.com> wrote: > I just ran into the issue described in > http://stackoverflow.com/questions/6351805/cyclic-module-dependencies-and-relative-imports-in-python > . > > This is unfortunate, because we have been trying to move towards relative > imports in order to aid flexibility in package and library design. > > The relative import syntax > > (from foo import bar) is a getattr type of lookup (i.e. import foo, then > get attr from it). > > This is in contrast with absolute import > > import foo.bar (get the module foo.bar from sys.modules or import it) > > bar = foo.bar > Or ``import foo.bar as bar`` > > > the latter works with partially initialized modules, but not the former, > rendering two sibling modules unable to import each other using the > relative syntax. > Clarification on terminology: the ``from .. import`` syntax is in no way relative. Relative imports use leading dots to specify relative offsets from your current position (i.e. as Barry said). It's more of a syntax for facilitating binding long names (e.g. foo.bar) to shorter names (bar). It's just unfortunate that it can lead to circular import issues when people start pulling in objects off of modules instead of modules alone. > > > as far as I know, relative imports are only supported using the former > (import from) syntax. Are there any plans to alleviate this by allowing > proper relative imports? After all, relative imports and packages go hand > in hand. > No, there are no plans to either tweak ``from ... import`` statements nor introduce a new syntax to deal help alleviate circular imports. -Brett > > > K > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Apr 2 00:52:56 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 2 Apr 2013 08:52:56 +1000 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: On Tue, Apr 2, 2013 at 6:20 AM, Kristj?n Valur J?nsson < kristjan at ccpgames.com> wrote: > I just ran into the issue described in > http://stackoverflow.com/questions/6351805/cyclic-module-dependencies-and-relative-imports-in-python > . > > This is unfortunate, because we have been trying to move towards relative > imports in order to aid flexibility in package and library design. > > The relative import syntax > > (from foo import bar) is a getattr type of lookup (i.e. import foo, then > get attr from it). > > This is in contrast with absolute import > > import foo.bar (get the module foo.bar from sys.modules or import it) > > bar = foo.bar > > > > the latter works with partially initialized modules, but not the former, > rendering two sibling modules unable to import each other using the > relative syntax. > This is really a quality-of-implementation issue in the import system rather than a core language design problem. It's just that those of us with the knowledge and ability to fix it aren't inclined to do so because circular imports usually (although not quite always) indicate a need to factor some common code out into a third support module imported by both of the original modules. At that point, the code is cleaner and more decoupled, and the uneven circular import support ceases to be a problem for that application. If you're interested in digging further, see http://bugs.python.org/issue992389 (this should also be a *lot* easier to fix now we're using importlib than it ever would have been while we were still relying on import.c) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Apr 2 02:44:48 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 2 Apr 2013 10:44:48 +1000 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> Message-ID: On Mon, Apr 1, 2013 at 12:28 AM, Mark Dickinson wrote: > As written, int_check would do the wrong thing for bools, too: I > definitely want int(True) to be 1, not True. > > For (2) and (4), it's not so clear. Are there use-cases for an __index__ > return value that's not directly of type int? I can't think of any offhand. > int() and operator.index() are both type coercion calls to produce true Python integers - they will never return a subclass, and this is both deliberate and consistent with all the other builtin types that accept an instance of themselves as input to the constructor. Passing a subclass instance to the base class constructor is the way you convert a subclass to an ordinary instance of the base class: >>> for base in (str, bytes, bytearray, int, float, complex, dict, tuple, list, set, frozenset): ... class subclass(base): pass ... print("'type(base(subclass()))' is", type(base(subclass()))) ... 'type(base(subclass()))' is 'type(base(subclass()))' is 'type(base(subclass()))' is 'type(base(subclass()))' is 'type(base(subclass()))' is 'type(base(subclass()))' is 'type(base(subclass()))' is 'type(base(subclass()))' is 'type(base(subclass()))' is 'type(base(subclass()))' is 'type(base(subclass()))' is There's code in the slot wrappers so that if you return a non-int object from either __int__ or __index__, then the interpreter will complain about it, and if you return a subclass, it will be stripped back to just the base class. If the language and library reference aren't clear on this, it's a documentation issue. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From dholth at gmail.com Tue Apr 2 02:47:08 2013 From: dholth at gmail.com (Daniel Holth) Date: Mon, 1 Apr 2013 20:47:08 -0400 Subject: [Python-Dev] PEP 4XX: pyzaa "Improving Python ZIP Application Support" Message-ID: https://docs.google.com/document/d/1MKXgPzhWD5wIUpoSQX7dxmqgTZVO6l9iZZis8dnri78/edit?usp=sharing PEP: 4XX Title: Improving Python ZIP Application Support Author: Daniel Holth Status: Draft Type: Standards Track Python-Version: 3.4 Created: 30 March 2013 Post-History: 30 March 2013, 1 April 2013 Improving Python ZIP Application Support Python has had the ability to execute directories or ZIP-format archives as scripts since version 2.6. When invoked with a zip file or directory as its first argument the interpreter adds that directory to sys.path and executes the __main__ module. These archives provide a great way to publish software that needs to be distributed as a single file script but is complex enough to need to be written as a collection of modules. This feature is not as popular as it should be, mainly because no one?s heard of it because it wasn?t promoted as part of Python 2.6, but also because Windows users don?t have a file extension (other than .py) to associate with the launcher. This PEP proposes to fix these problems by re-publicising the feature, defining the .pyz and .pyzw extensions as ?Python ZIP Applications? and ?Windowed Python ZIP Applications?, and providing some simple tooling to manage the format. A New Python ZIP Application Extension The Python 3.4 installer will associate .pyz and .pyzw ?Python ZIP Applications? with the platform launcher so they can be executed. A .pyz archive is a console application and a .pyzw archive is a windowed application, indicating whether the console should appear when running the app. Why not use .zip or .py? Users expect a .zip file would be opened with an archive tool, and users expect .py to be opened with a text editor. Both would be confusing for this use case. For UNIX users, .pyz applications should be prefixed with a #! line pointing to the correct Python interpreter and an optional explanation. #!/usr/bin/env python3 # This is a Python application stored in a ZIP archive. (binary contents of archive) As background, ZIP archives are defined with a footer containing relative offsets from the end of the file. They remain valid when concatenated to the end of any other file. This feature is completely standard and is how self-extracting ZIP archives and the bdist_wininst installer format work. Minimal Tooling: The pyzaa Module This PEP also proposes including a simple application for working with these archives: The Python Zip Application Archiver ?pyzaa? (rhymes with ?huzzah? or ?pizza?). ?pyzaa? can archive or extract these files, compile bytecode, and can write the __main__ module if it is not present. Usage python -m pyzaa (pack | compile) python -m pyzaa pack [-o path/name] [-m module.submodule:callable] [-c] [-w] [-p interpreter] directory: ZIP the contents of directory as directory.pyz or [-w] directory.pyzw. Adds the executable flag to the archive. -c compile .pyc files and add them to the archive -p interpreter include #!interpreter as the first line of the archive -o path/name archive is written to path/name.pyz[w] instead of dirname. The extension is added if not specified. -m module.submodule:callable __main__.py is written as ?import module.submodule; module.submodule.callable()? pyzaa pack will warn if the directory contains C extensions or if it doesn?t contain __main__.py. python -m pyzaa compile arcname.pyz[w] The Python files in arcname.pyz[w] are compiled and appended to the ZIP file. A standard ZIP utility or Python?s zipfile module can unpack the archives. FAQ Q. Isn?t pyzaa just a very thin wrapper over zipfile and compileall? A. Yes. Q. How does this compete with existing sdist/bdist formats? A. There is some overlap, but .pyz files are especially interesting as a way to distribute an installer. They may also prove useful as a way to deliver applications when users shouldn?t be asked to perform virtualenv + ?pip install?. References [1] http://bugs.python.org/issue1739468 ?Allow interpreter to execute a zip file? [2] http://bugs.python.org/issue17359 ?Feature is not documented? Copyright This document has been placed into the public domain. From v+python at g.nevcal.com Tue Apr 2 04:25:25 2013 From: v+python at g.nevcal.com (Glenn Linderman) Date: Mon, 01 Apr 2013 19:25:25 -0700 Subject: [Python-Dev] PEP 4XX: pyzaa "Improving Python ZIP Application Support" In-Reply-To: References: Message-ID: <515A4195.4080706@g.nevcal.com> On 4/1/2013 5:47 PM, Daniel Holth wrote: > users expect .py to be opened with a text editor. This user expects .py to be executed as an executable script, and thinks that is the default after an installation of Python on Windows. Windows has a separate option, Edit, to use to edit things. But, I'm glad to see you write the PEP. I have an even thinner method of doing this, using .py extensions, that I've been using for several years now with Python 3, and wondered why it wasn't more popular. My equivalent of pyzaa, though, only performs the pack operation, and requires a bit of cooperation from the application (as a convenient way of storing the application-specific parameters, I build the invocation of pyzaa-equivalent into the application itself using a non-documented command-line option, and build to a different directory, to avoid overwriting application.py). Feel free to incorporate all or parts of that idea if it makes sense to you and sounds convenient. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dickinsm at gmail.com Tue Apr 2 09:07:45 2013 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 2 Apr 2013 08:07:45 +0100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> Message-ID: On Tue, Apr 2, 2013 at 1:44 AM, Nick Coghlan wrote: > int() and operator.index() are both type coercion calls to produce true > Python integers - they will never return a subclass, and this is both > deliberate and consistent with all the other builtin types that accept an > instance of themselves as input to the constructor. > That's good to hear. > There's code in the slot wrappers so that if you return a non-int object > from either __int__ or __index__, then the interpreter will complain about > it, and if you return a subclass, it will be stripped back to just the base > class. > Can you point me to that code? All I could find was PyLong_Check calls (I was looking for PyLong_CheckExact). Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From dickinsm at gmail.com Tue Apr 2 09:18:29 2013 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 2 Apr 2013 08:18:29 +0100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> Message-ID: On Tue, Apr 2, 2013 at 8:07 AM, Mark Dickinson wrote: > On Tue, Apr 2, 2013 at 1:44 AM, Nick Coghlan wrote: > >> There's code in the slot wrappers so that if you return a non-int object >> from either __int__ or __index__, then the interpreter will complain about >> it, and if you return a subclass, it will be stripped back to just the base >> class. >> > > Can you point me to that code? All I could find was PyLong_Check calls (I > was looking for PyLong_CheckExact). > And indeed: iwasawa:Objects mdickinson$ /opt/local/bin/python3.3 Python 3.3.0 (default, Sep 29 2012, 08:16:19) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> class A: ... def __int__(self): ... return True ... def __index__(self): ... return False ... >>> a = A() >>> int(a) True >>> import operator; operator.index(a) False Which means I have to do int(int(a)) to get the actual integer value. Grr. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at hotpy.org Tue Apr 2 10:33:21 2013 From: mark at hotpy.org (Mark Shannon) Date: Tue, 02 Apr 2013 09:33:21 +0100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> Message-ID: <515A97D1.9040506@hotpy.org> On 02/04/13 01:44, Nick Coghlan wrote: > On Mon, Apr 1, 2013 at 12:28 AM, Mark Dickinson > wrote: > > As written, int_check would do the wrong thing for bools, too: I > definitely want int(True) to be 1, not True. > > For (2) and (4), it's not so clear. Are there use-cases for an > __index__ return value that's not directly of type int? I can't > think of any offhand. > > > int() and operator.index() are both type coercion calls to produce true > Python integers - they will never return a subclass, and this is both > deliberate and consistent with all the other builtin types that accept > an instance of themselves as input to the constructor. Passing a > subclass instance to the base class constructor is the way you convert a > subclass to an ordinary instance of the base class: Unfortunately, that is not true :( >>> class Int(int): ... def __int__(self): ... return self ... >>> type(int(Int())) Hence my original question: what *should* the semantics be? > > >>> for base in (str, bytes, bytearray, int, float, complex, dict, > tuple, list, set, frozenset): > ... class subclass(base): pass > ... print("'type(base(subclass()))' is", type(base(subclass()))) > ... > 'type(base(subclass()))' is > 'type(base(subclass()))' is > 'type(base(subclass()))' is > 'type(base(subclass()))' is > 'type(base(subclass()))' is > 'type(base(subclass()))' is > 'type(base(subclass()))' is > 'type(base(subclass()))' is > 'type(base(subclass()))' is > 'type(base(subclass()))' is > 'type(base(subclass()))' is > > There's code in the slot wrappers so that if you return a non-int object > from either __int__ or __index__, then the interpreter will complain > about it, and if you return a subclass, it will be stripped back to just > the base class. > > If the language and library reference aren't clear on this, it's a > documentation issue. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | > Brisbane, Australia From dickinsm at gmail.com Tue Apr 2 10:53:41 2013 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 2 Apr 2013 09:53:41 +0100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <515A97D1.9040506@hotpy.org> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> Message-ID: On Tue, Apr 2, 2013 at 9:33 AM, Mark Shannon wrote: > > Hence my original question: what *should* the semantics be? > > I like Nick's answer to that: int *should* always return something of exact type int. Otherwise you're always left wondering whether you have to do "int(int(x))", or perhaps even "int(int(int(x)))", to be absolutely sure of getting an int. The question is whether / how to fix the current behaviour, given that it doesn't conform to those ideal semantics. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Tue Apr 2 10:58:58 2013 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 2 Apr 2013 10:58:58 +0200 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> Message-ID: On Tue, Apr 2, 2013 at 10:53 AM, Mark Dickinson wrote: > On Tue, Apr 2, 2013 at 9:33 AM, Mark Shannon wrote: >> >> >> Hence my original question: what *should* the semantics be? >> > > I like Nick's answer to that: int *should* always return something of exact > type int. Otherwise you're always left wondering whether you have to do > "int(int(x))", or perhaps even "int(int(int(x)))", to be absolutely sure of > getting an int. > > The question is whether / how to fix the current behaviour, given that it > doesn't conform to those ideal semantics. > > Mark My 2 cents here is that which one is called seems to be truly random. Try looking into what builtin functions call (for example list.pop calls __int__, who knew) From solipsis at pitrou.net Tue Apr 2 11:01:20 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 2 Apr 2013 11:01:20 +0200 Subject: [Python-Dev] Semantics of __int__(), __index__() References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> Message-ID: <20130402110120.1a1f6c62@pitrou.net> Le Tue, 2 Apr 2013 09:53:41 +0100, Mark Dickinson a ?crit : > On Tue, Apr 2, 2013 at 9:33 AM, Mark Shannon wrote: > > > > > Hence my original question: what *should* the semantics be? > > > > > I like Nick's answer to that: int *should* always return something of > exact type int. Otherwise you're always left wondering whether you > have to do "int(int(x))", or perhaps even "int(int(int(x)))", to be > absolutely sure of getting an int. Agreed. Antoine. From dickinsm at gmail.com Tue Apr 2 11:02:20 2013 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 2 Apr 2013 10:02:20 +0100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> Message-ID: On Tue, Apr 2, 2013 at 9:58 AM, Maciej Fijalkowski wrote: > > My 2 cents here is that which one is called seems to be truly random. > Try looking into what builtin functions call (for example list.pop > calls __int__, who knew) > That sounds like a clear bug to me. It should definitely be using __index__. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From dickinsm at gmail.com Tue Apr 2 11:19:06 2013 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 2 Apr 2013 10:19:06 +0100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> Message-ID: On Tue, Apr 2, 2013 at 10:02 AM, Mark Dickinson wrote: > On Tue, Apr 2, 2013 at 9:58 AM, Maciej Fijalkowski wrote: > >> >> My 2 cents here is that which one is called seems to be truly random. >> Try looking into what builtin functions call (for example list.pop >> calls __int__, who knew) >> > > That sounds like a clear bug to me. It should definitely be using > __index__. > Ah, and I see it *is* using `__index__` in Python 3; just not in Python 2.7. It may be one of those Python 2 bugs that's not worth fixing because the fix would do more harm (in the form of breakage of existing code) than good. -- Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From kristjan at ccpgames.com Tue Apr 2 11:28:17 2013 From: kristjan at ccpgames.com (=?utf-8?B?S3Jpc3Rqw6FuIFZhbHVyIErDs25zc29u?=) Date: Tue, 2 Apr 2013 09:28:17 +0000 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: Right, as I explained in my reply to Barry, I was imprecise. But the ?from X import Y? is the only way to invoke relative imports, where X can have leading dots. This syntax places the constraint on X that Y is actually an attribute of X at this time, where ?import X.Y? does not. So, even without the leading dots issue, they are not equivalent. You run into the same circular dependency problem without using relative imports if trying to use the ?from X import Y? where X is an absolute name. K From: bcannon at gmail.com [mailto:bcannon at gmail.com] On Behalf Of Brett Cannon Sent: 1. apr?l 2013 22:38 To: Kristj?n Valur J?nsson Cc: python-dev at python.org Subject: Re: [Python-Dev] relative import circular problem the latter works with partially initialized modules, but not the former, rendering two sibling modules unable to import each other using the relative syntax. Clarification on terminology: the ``from .. import`` syntax is in no way relative. Relative imports use leading dots to specify relative offsets from your current position (i.e. as Barry said). It's more of a syntax for facilitating binding long names (e.g. foo.bar) to shorter names (bar). It's just unfortunate that it can lead to circular import issues when people start pulling in objects off of modules instead of modules alone. as far as I know, relative imports are only supported using the former (import from) syntax. Are there any plans to alleviate this by allowing proper relative imports? After all, relative imports and packages go hand in hand. No, there are no plans to either tweak ``from ... import`` statements nor introduce a new syntax to deal help alleviate circular imports. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kristjan at ccpgames.com Tue Apr 2 12:02:12 2013 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Tue, 2 Apr 2013 10:02:12 +0000 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: It certainly affects the quality, yes. I also understand why it happens: When importing X.Y, Y isn't actually put into X's dict until it is fully initialized. It is, however put temporarily in sys.modules["X.Y"] hence, "import X.Y" on a partially initialized submodule Y will work, whereas "from X import Y" won't. Fixing this within the "from X import Y" mechanism would add an additional strain on the already complex import protocol (as defined in pep 302), I think. Which is why I wonder if the relative import syntax ought to be allowed for "import X" since that syntax does not involve a getattr. ("from X import Y" necessarily means strictly a getattr, since Y can both be any attribute of X, not just a submodule) As for ways around this: Note that this is a language design question, not a software architecture one. It is possible to work around these issues, but it is not always nice. Python is one of those languages that allow cyclic module dependencies and it is a very nice way to separate code by functionality, if not by dependency. It is one of the good things about Python and we should try to make sure that we allow such architectural freedom to continue to work. Also, relative imports are apparently falling into favor, having only marginally been accepted at the time of pep 328, so we should perhaps find a way for these two things to co-exist :) I'm not sure that http://bugs.python.org/issue992389 warrants a fix. This issue is about general attributes of a module. In the general case, this is probably unfixable. But access to a partially constructed module hierarchy through the import mechanism ought to be possible. K From: Nick Coghlan [mailto:ncoghlan at gmail.com] Sent: 1. apr?l 2013 22:53 To: Kristj?n Valur J?nsson Cc: python-dev at python.org Subject: Re: [Python-Dev] relative import circular problem with partially initialized modules, but not the former, rendering two sibling modules unable to import each other using the relative syntax. This is really a quality-of-implementation issue in the import system rather than a core language design problem. It's just that those of us with the knowledge and ability to fix it aren't inclined to do so because circular imports usually (although not quite always) indicate a need to factor some common code out into a third support module imported by both of the original modules. At that point, the code is cleaner and more decoupled, and the uneven circular import support ceases to be a problem for that application. If you're interested in digging further, see http://bugs.python.org/issue992389 (this should also be a *lot* easier to fix now we're using importlib than it ever would have been while we were -------------- next part -------------- An HTML attachment was scrubbed... URL: From kristjan at ccpgames.com Tue Apr 2 11:24:37 2013 From: kristjan at ccpgames.com (=?utf-8?B?S3Jpc3Rqw6FuIFZhbHVyIErDs25zc29u?=) Date: Tue, 2 Apr 2013 09:24:37 +0000 Subject: [Python-Dev] relative import circular problem In-Reply-To: <20130401181559.0ea23b5b@anarchist> References: <20130401181559.0ea23b5b@anarchist> Message-ID: > -----Original Message----- > From: Python-Dev [mailto:python-dev- > bounces+kristjan=ccpgames.com at python.org] On Behalf Of Barry Warsaw > Sent: 1. apr?l 2013 22:16 > To: python-dev at python.org > Subject: Re: [Python-Dev] relative import circular problem > > On Apr 01, 2013, at 08:20 PM, Kristj?n Valur J?nsson wrote: > > >The relative import syntax > > > > (from foo import bar) is a getattr type of lookup (i.e. import foo, > > then get attr from it). > > > >This is in contrast with absolute import > > > > import foo.bar (get the module foo.bar from sys.modules or import > > it) > > > > bar = foo.bar > > I always thought of both syntaxes as absolute imports because they both > name the full dotted path to the module you want. This contrasts with true > PEP > 328 relative imports such as: > > from ..foo import bar > Right, the example I gave was not a relative import but an absolute one, relative imports always starting with a dot. I should have been more clear. However, relative imports can _only_ be performed using the "from X import Y syntax" (http://www.python.org/dev/peps/pep-0328/#id3) and so, when one wishes to use relative imports, the parent module must be fully initialized with the child module as an attribute. importing with the "import X.Y" does not appear to have this restriction. > I personally dislike PEP 328 relative imports, since they seem fragile, but that's a different discussion. Yes. I find them very useful. In our environment, we tend to write packages that then go into different places. Using relative imports allows us the freedom to move packages around and rename them, for example, make a package a sub-package of another, and do this differently for different products, while still sharing code. A package can thus refer to internals of itself, without knowing its own name, or absolute path. This is really useful. Consider this a property of Python-based "products", with a custom assembly of source code, as opposed to publicly available packages that always install into the sys.path Cheers, K From solipsis at pitrou.net Tue Apr 2 13:46:06 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 2 Apr 2013 13:46:06 +0200 Subject: [Python-Dev] relative import circular problem References: Message-ID: <20130402134606.5efb3552@pitrou.net> Le Tue, 2 Apr 2013 09:28:17 +0000, Kristj?n Valur J?nsson a ?crit : > Right, as I explained in my reply to Barry, I was imprecise. > But the ?from X import Y? is the only way to invoke relative imports, > where X can have leading dots. This syntax places the constraint on X > that Y is actually an attribute of X at this time, where ?import X.Y? > does not. So, even without the leading dots issue, they are not > equivalent. You run into the same circular dependency problem > without using relative imports if trying to use the ?from X import Y? > where X is an absolute name. I agree with Kristjan, it is rather annoying. And it also makes explicit relative imports harder to sell to people used to the implicit relative import style, since the latter works well in such circumstances. Regards Antoine. From Steve.Dower at microsoft.com Tue Apr 2 19:20:11 2013 From: Steve.Dower at microsoft.com (Steve Dower) Date: Tue, 2 Apr 2013 17:20:11 +0000 Subject: [Python-Dev] PEP 4XX: pyzaa "Improving Python ZIP Application Support" In-Reply-To: References: Message-ID: <99ec8f3bb91b41a897359beffb3da225@BLUPR03MB035.namprd03.prod.outlook.com> > python -m pyzaa pack [-o path/name] [-m module.submodule:callable] [-c] [-w] [-p interpreter] directory: > > ZIP the contents of directory as directory.pyz or [-w] directory.pyzw. Adds the executable flag to the archive. > > ... > > -p interpreter include #!interpreter as the first line of the archive What happens when -p is omitted? I'd hope it would add the interpreter used to create the zip (or at least the major version), but that may not be ideal for some reason that I haven't thought of yet. Everything else looks great. I'm really looking forward to this. Cheers, Steve From brett at python.org Tue Apr 2 19:28:55 2013 From: brett at python.org (Brett Cannon) Date: Tue, 2 Apr 2013 13:28:55 -0400 Subject: [Python-Dev] PEP 4XX: pyzaa "Improving Python ZIP Application Support" In-Reply-To: <99ec8f3bb91b41a897359beffb3da225@BLUPR03MB035.namprd03.prod.outlook.com> References: <99ec8f3bb91b41a897359beffb3da225@BLUPR03MB035.namprd03.prod.outlook.com> Message-ID: On Tue, Apr 2, 2013 at 1:20 PM, Steve Dower wrote: > > python -m pyzaa pack [-o path/name] [-m module.submodule:callable] [-c] > [-w] [-p interpreter] directory: > > > > ZIP the contents of directory as directory.pyz or [-w] > directory.pyzw. Adds the executable flag to the archive. > > > > ... > > > > -p interpreter include #!interpreter as the first line of the archive > > What happens when -p is omitted? I'd hope it would add the interpreter > used to create the zip (or at least the major version), but that may not be > ideal for some reason that I haven't thought of yet. > Question is whether ``/usr/bin/python3.3`` is better or ``/usr/bin/env python3.3``. I vote for the latter since it gets you the right thing without having to care about whether the interpreter moved or is being hidden by a user-installed interpreter. -Brett > > Everything else looks great. I'm really looking forward to this. > > Cheers, > Steve > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric at trueblade.com Tue Apr 2 23:16:30 2013 From: eric at trueblade.com (Eric V. Smith) Date: Tue, 02 Apr 2013 17:16:30 -0400 Subject: [Python-Dev] [Python-checkins] cpython (3.3): Whatsnew typo In-Reply-To: <515B4A68.1020400@trueblade.com> References: <3XXG2s6bp2zQXD@mail.python.org> <515B4A68.1020400@trueblade.com> Message-ID: <515B4AAE.9010009@trueblade.com> On 4/2/2013 5:15 PM, Eric V. Smith wrote: > On 10/3/2012 8:59 PM, jesus.cea wrote: >> >> -Solaris and derived platforms have a new class :class:`select.devpoll` >> -for high performance asyncronous sockets via :file:`/dev/poll`. >> +Solaris and derivatives platforms have a new class :class:`select.devpoll` >> +for high performance asynchronous sockets via :file:`/dev/poll`. > > That should be "Solaris and derivative platforms ...". > Ignore that. My mailer went haywire and was showing ancient emails as recent and unread. Sorry for the noise. Eric. From eric at trueblade.com Tue Apr 2 23:15:20 2013 From: eric at trueblade.com (Eric V. Smith) Date: Tue, 02 Apr 2013 17:15:20 -0400 Subject: [Python-Dev] [Python-checkins] cpython (3.3): Whatsnew typo In-Reply-To: <3XXG2s6bp2zQXD@mail.python.org> References: <3XXG2s6bp2zQXD@mail.python.org> Message-ID: <515B4A68.1020400@trueblade.com> On 10/3/2012 8:59 PM, jesus.cea wrote: > > -Solaris and derived platforms have a new class :class:`select.devpoll` > -for high performance asyncronous sockets via :file:`/dev/poll`. > +Solaris and derivatives platforms have a new class :class:`select.devpoll` > +for high performance asynchronous sockets via :file:`/dev/poll`. That should be "Solaris and derivative platforms ...". From greg.ewing at canterbury.ac.nz Wed Apr 3 01:12:34 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 03 Apr 2013 12:12:34 +1300 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: <20130401181559.0ea23b5b@anarchist> Message-ID: <515B65E2.3040109@canterbury.ac.nz> Kristj?n Valur J?nsson wrote: > However, relative imports can _only_ be performed using the "from X import Y syntax" This seems like a legitimate complaint on its own, regardless of the circular import issue. The principle of least surprise suggests that relative imports should be possible using either syntax. I'm guessing the reason that 'import' doesn't support relative imports is that it's not clear what name to bind the imported module to. There are a couple of ways that this could be resolved. One would be to use the name resulting from stripping off the leading dots, so that import .foo would bind the module to the name 'foo'. Another would be to always require an 'as' clause in this case, so that you would have to write' import .foo as foo -- Greg From 2281570025 at qq.com Wed Apr 3 04:42:23 2013 From: 2281570025 at qq.com (=?ISO-8859-1?B?aU1hdGg=?=) Date: Wed, 3 Apr 2013 10:42:23 +0800 Subject: [Python-Dev] could gevent be apart the standard Python library in future ? Message-ID: Hi, It will be my first post here. could gevent be apart the standard Python library in future ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan_ml at behnel.de Wed Apr 3 07:26:22 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 03 Apr 2013 07:26:22 +0200 Subject: [Python-Dev] PEP 4XX: pyzaa "Improving Python ZIP Application Support" In-Reply-To: References: <99ec8f3bb91b41a897359beffb3da225@BLUPR03MB035.namprd03.prod.outlook.com> Message-ID: Brett Cannon, 02.04.2013 19:28: > On Tue, Apr 2, 2013 at 1:20 PM, Steve Dower wrote: > >>> python -m pyzaa pack [-o path/name] [-m module.submodule:callable] [-c] >> [-w] [-p interpreter] directory: >>> >>> ZIP the contents of directory as directory.pyz or [-w] >> directory.pyzw. Adds the executable flag to the archive. >>> >>> ... >>> >>> -p interpreter include #!interpreter as the first line of the archive >> >> What happens when -p is omitted? I'd hope it would add the interpreter >> used to create the zip (or at least the major version), but that may not be >> ideal for some reason that I haven't thought of yet. > > Question is whether ``/usr/bin/python3.3`` is better or ``/usr/bin/env > python3.3``. I vote for the latter since it gets you the right thing > without having to care about whether the interpreter moved or is being > hidden by a user-installed interpreter. It can't work properly from within a virtualenv when you write "/usr/bin/python", so using "/usr/bin/env" instead is actually required. Stefan From fijall at gmail.com Wed Apr 3 08:04:38 2013 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 3 Apr 2013 08:04:38 +0200 Subject: [Python-Dev] could gevent be apart the standard Python library in future ? In-Reply-To: References: Message-ID: On Wed, Apr 3, 2013 at 4:42 AM, iMath <2281570025 at qq.com> wrote: > Hi, > It will be my first post here. > > > > could gevent be apart the standard Python library in future ? Hi. Such question generally belongs to python-ideas. To start with you need to restart the question whether greenlets module should be a part of the standard library (and please do it on the python-ideas mailing list). Cheers and welcome, fijal From stefan.bucur at gmail.com Wed Apr 3 12:47:28 2013 From: stefan.bucur at gmail.com (Stefan Bucur) Date: Wed, 3 Apr 2013 12:47:28 +0200 Subject: [Python-Dev] Are undocumented exceptions considered bugs? In-Reply-To: References: Message-ID: On Sun, Mar 24, 2013 at 2:09 AM, Devin Jeanpierre wrote: > On Sat, Mar 23, 2013 at 11:21 AM, Nick Coghlan wrote: > > In this specific case, the error message is > > confusing-but-not-really-wrong, due to the "two-types-in-one" nature > > of Python 2.x strings - 8-bit strings are used as both text sequences > > (generally not containing NUL characters) and also as arbitrary binary > > data, including encoded text (quite likely to contain NUL bytes). > > With your terminology, three types: char, non-NUL-text, encoded-text > (e.g. what happens with ord('ab')) > > That's pretty silly, considering that these are all one Python type, > and TypeError is raised into Python code. Obviously it can't change, > because of historical reasons, but documenting it would be > straightforward and helpful. These are not errors you can just infer > will happen, you need to see it via documentation, reading the source, > or experimentation (and re experimentation, then you have to establish > whether or not this was an accident or deliberate). > Thanks for your answers, guys, and sorry for replying so late (had a research paper submission deadline in the mean time...). Filing a bug report for this issue sounds like a good idea. I have just submitted http://bugs.python.org/issue17624 Stefan -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Wed Apr 3 13:17:06 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 3 Apr 2013 21:17:06 +1000 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <20130402110120.1a1f6c62@pitrou.net> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> Message-ID: On 2 Apr 2013 19:04, "Antoine Pitrou" wrote: > > Le Tue, 2 Apr 2013 09:53:41 +0100, > Mark Dickinson a ?crit : > > On Tue, Apr 2, 2013 at 9:33 AM, Mark Shannon wrote: > > > > > > > > Hence my original question: what *should* the semantics be? > > > > > > > > I like Nick's answer to that: int *should* always return something of > > exact type int. Otherwise you're always left wondering whether you > > have to do "int(int(x))", or perhaps even "int(int(int(x)))", to be > > absolutely sure of getting an int. > > Agreed. Perhaps we should start emitting a DeprecationWarning for int subclasses returned from __int__ and __index__ in 3.4? (I like the idea of an explicit error over implicit conversion to the base type, so deprecation of subtypes makes sense as a way forward. We should check the other type coercion methods, too.) Cheers, Nick. > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From dickinsm at gmail.com Wed Apr 3 14:12:06 2013 From: dickinsm at gmail.com (Mark Dickinson) Date: Wed, 3 Apr 2013 13:12:06 +0100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> Message-ID: On Wed, Apr 3, 2013 at 12:17 PM, Nick Coghlan wrote: > Perhaps we should start emitting a DeprecationWarning for int subclasses > returned from __int__ and __index__ in 3.4? > +1. Sounds good to me. > (I like the idea of an explicit error over implicit conversion to the base > type, so deprecation of subtypes makes sense as a way forward. We should > check the other type coercion methods, too.) > Agreed on both points. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From hrvoje.niksic at avl.com Wed Apr 3 14:47:31 2013 From: hrvoje.niksic at avl.com (Hrvoje Niksic) Date: Wed, 03 Apr 2013 14:47:31 +0200 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> Message-ID: <515C24E3.5010703@avl.com> On 04/03/2013 01:17 PM, Nick Coghlan wrote: > > > > > > > I like Nick's answer to that: int *should* always return something of > > > exact type int. Otherwise you're always left wondering whether you > > > have to do "int(int(x))", or perhaps even "int(int(int(x)))", to be > > > absolutely sure of getting an int. > > > > Agreed. > > Perhaps we should start emitting a DeprecationWarning for int subclasses > returned from __int__ and __index__ in 3.4? Why would one want to be absolutely sure of getting an int? It seems like a good feature that an __int__ implementation can choose to return an int subclass with additional (and optional) information. After all, int subclass instances should be usable everywhere where ints are, including in C code. I can imagine numpy and similar projects would be making use of this ability already -- just think of uses for numpy's subclasses of "float". If one wants to break the abstraction and be absolutely positively sure of getting an int and not a subclass thereof, they can write something like (0).__add__(obj). But I suspect this will be extremely rare. From robert.kern at gmail.com Wed Apr 3 15:11:18 2013 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 03 Apr 2013 14:11:18 +0100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <515C24E3.5010703@avl.com> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> Message-ID: On 2013-04-03 13:47, Hrvoje Niksic wrote: > On 04/03/2013 01:17 PM, Nick Coghlan wrote: >> > > > >> > > I like Nick's answer to that: int *should* always return something of >> > > exact type int. Otherwise you're always left wondering whether you >> > > have to do "int(int(x))", or perhaps even "int(int(int(x)))", to be >> > > absolutely sure of getting an int. >> > >> > Agreed. >> >> Perhaps we should start emitting a DeprecationWarning for int subclasses >> returned from __int__ and __index__ in 3.4? > > Why would one want to be absolutely sure of getting an int? > > It seems like a good feature that an __int__ implementation can choose to return > an int subclass with additional (and optional) information. After all, int > subclass instances should be usable everywhere where ints are, including in C > code. I can imagine numpy and similar projects would be making use of this > ability already -- just think of uses for numpy's subclasses of "float". We don't. [~] |1> type(float(np.float64(1.0))) float [~] |2> type(int(np.int32(1))) int -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From barry at python.org Wed Apr 3 16:16:12 2013 From: barry at python.org (Barry Warsaw) Date: Wed, 3 Apr 2013 10:16:12 -0400 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> Message-ID: <20130403101612.27157a4d@anarchist> On Apr 03, 2013, at 09:17 PM, Nick Coghlan wrote: >Perhaps we should start emitting a DeprecationWarning for int subclasses >returned from __int__ and __index__ in 3.4? I definitely agree with doing this for __int__(), since it's intimately tied to int(), which is clearly a type conversion operation. It's analogous to all the other built-in types-as-functions, so int() calls __int__() which must return a concrete integer. __index__() is a bit trickier because it is not tied directly to type conversion. In this case, int subclasses could be valid, and as Hrvoje later points out, returning int-subclasses from __index__() should still work for all valid use cases. (Bug 17576 would still be a bug in this scenario, since obj.__index__() still needs to be called by operator.index() even when it's an int subclass.) >(I like the idea of an explicit error over implicit conversion to the base >type, so deprecation of subtypes makes sense as a way forward. We should >check the other type coercion methods, too.) +1 -Barry From graffatcolmingov at gmail.com Wed Apr 3 16:24:29 2013 From: graffatcolmingov at gmail.com (Ian Cordasco) Date: Wed, 3 Apr 2013 10:24:29 -0400 Subject: [Python-Dev] [Announcement] New mailing list for code quality tools including Flake8, Pyflakes and Pep8 Message-ID: Hello, There's a new mailing-list related to Python code-quality tools. Are you concerned about the evolution of various code checkers? Do you have questions or suggestions? Subscribe here: http://mail.python.org/mailman/listinfo/code-quality Best regards, Ian From ncoghlan at gmail.com Wed Apr 3 17:14:28 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 4 Apr 2013 01:14:28 +1000 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <20130403101612.27157a4d@anarchist> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <20130403101612.27157a4d@anarchist> Message-ID: On 4 Apr 2013 00:18, "Barry Warsaw" wrote: > > On Apr 03, 2013, at 09:17 PM, Nick Coghlan wrote: > > >Perhaps we should start emitting a DeprecationWarning for int subclasses > >returned from __int__ and __index__ in 3.4? > > I definitely agree with doing this for __int__(), since it's intimately tied > to int(), which is clearly a type conversion operation. It's analogous to all > the other built-in types-as-functions, so int() calls __int__() which must > return a concrete integer. > > __index__() is a bit trickier because it is not tied directly to type > conversion. In this case, int subclasses could be valid, and as Hrvoje later > points out, returning int-subclasses from __index__() should still work for > all valid use cases. Implementing __index__ just means "This type can be converted to a Python integer without losing information". Aside from that extra "without information loss" qualification, it's the same as __int__. Cheers, Nick. > > (Bug 17576 would still be a bug in this scenario, since obj.__index__() still > needs to be called by operator.index() even when it's an int subclass.) > > >(I like the idea of an explicit error over implicit conversion to the base > >type, so deprecation of subtypes makes sense as a way forward. We should > >check the other type coercion methods, too.) > > +1 > > -Barry > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Wed Apr 3 17:21:22 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 03 Apr 2013 08:21:22 -0700 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <20130403101612.27157a4d@anarchist> Message-ID: <515C48F2.9000607@stoneleaf.us> On 04/03/2013 08:14 AM, Nick Coghlan wrote: > > On 4 Apr 2013 00:18, "Barry Warsaw" > wrote: >> >> __index__() is a bit trickier because it is not tied directly to type >> conversion. In this case, int subclasses could be valid, and as Hrvoje later >> points out, returning int-subclasses from __index__() should still work for >> all valid use cases. > > Implementing __index__ just means "This type can be converted to a Python integer without losing information". Aside > from that extra "without information loss" qualification, it's the same as __int__. How is that possible? Whether int or int subclass, if I'm implementing __index__ it means my type is not an int subclass, and when I return an int I most certainly have lost information from the original type. -- ~Ethan~ From solipsis at pitrou.net Wed Apr 3 17:41:59 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 3 Apr 2013 17:41:59 +0200 Subject: [Python-Dev] Semantics of __int__(), __index__() References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <20130403101612.27157a4d@anarchist> <515C48F2.9000607@stoneleaf.us> Message-ID: <20130403174159.155b423f@pitrou.net> Le Wed, 03 Apr 2013 08:21:22 -0700, Ethan Furman a ?crit : > On 04/03/2013 08:14 AM, Nick Coghlan wrote: > > > > On 4 Apr 2013 00:18, "Barry Warsaw" > > wrote: > >> > >> __index__() is a bit trickier because it is not tied directly to > >> type conversion. In this case, int subclasses could be valid, and > >> as Hrvoje later points out, returning int-subclasses from > >> __index__() should still work for all valid use cases. > > > > Implementing __index__ just means "This type can be converted to a > > Python integer without losing information". Aside from that extra > > "without information loss" qualification, it's the same as __int__. > > How is that possible? Whether int or int subclass, if I'm > implementing __index__ it means my type is not an int subclass, and > when I return an int I most certainly have lost information from the > original type. Without losing information about the numeric value. Regards Antoine. From steve at pearwood.info Wed Apr 3 17:49:31 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 04 Apr 2013 02:49:31 +1100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <515C24E3.5010703@avl.com> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> Message-ID: <515C4F8B.2030007@pearwood.info> On 03/04/13 23:47, Hrvoje Niksic wrote: > On 04/03/2013 01:17 PM, Nick Coghlan wrote: >> > > > >> > > I like Nick's answer to that: int *should* always return something of >> > > exact type int. Otherwise you're always left wondering whether you >> > > have to do "int(int(x))", or perhaps even "int(int(int(x)))", to be >> > > absolutely sure of getting an int. >> > >> > Agreed. >> >> Perhaps we should start emitting a DeprecationWarning for int subclasses >> returned from __int__ and __index__ in 3.4? > > Why would one want to be absolutely sure of getting an int? > > It seems like a good feature that an __int__ implementation can choose to return an int subclass with additional (and optional) information. After all, int subclass instances should be usable everywhere where ints are, including in C code. I agree with Hrvoje here, and I have code that's probably going to be impacted by any change in behaviour. In OO terms, an instance of an int subclass *is* an int, and we shouldn't care whether __int__ returns a subclass or a builtin. I think that before any change is made, even mere DeprecationWarning, there needs to be a very strong reason justifying restricting __int__ to return a built-in int. To put it another way, I think it is perfectly reasonable for __int__ to enforce the constraint ``isinstance(result, int)`` but *not* to enforce the constraint ``type(result) is int``. We rarely, if ever, write explicit type checks like the second. Why should __int__ etc. implicitly do so? This issue doesn't just apply to ints. For example: py> class MyStr(str): ... def __str__(self): ... return self ... py> s = MyStr('spam') py> type(str(s)) This seems perfectly reasonable to me: __str__ is expected to return a string, and it does. If I absolutely must have a built-in str, I can do this: py> type(''.join(s)) But I can't really think of any cases where I am going to insist on a built-in str, instead of accepting anything that passes either the weaker isinstance check. Even if there are such cases, surely they are going to be rare and unusual. (I note that in OO terms, we don't really have good terminology for "an instance of a class, excluding instances of any subclasses". That's because in OO terms we shouldn't care whether the type of the instance is the superclass or subclass, at least not in general.) -1 on forcing __int__, __str__, __float__ etc. to return instances of the built-in types. -- Steven From steve at pearwood.info Wed Apr 3 18:04:11 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 04 Apr 2013 03:04:11 +1100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <20130403101612.27157a4d@anarchist> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <20130403101612.27157a4d@anarchist> Message-ID: <515C52FB.2070705@pearwood.info> On 04/04/13 01:16, Barry Warsaw wrote: > On Apr 03, 2013, at 09:17 PM, Nick Coghlan wrote: > >> Perhaps we should start emitting a DeprecationWarning for int subclasses >> returned from __int__ and __index__ in 3.4? > > I definitely agree with doing this for __int__(), since it's intimately tied > to int(), which is clearly a type conversion operation. It's analogous to all > the other built-in types-as-functions, so int() calls __int__() which must > return a concrete integer. Why must it? I think that's the claim which must be justified, not just taken as a given. When we call n = int(something), what's the use-case for caring that n is an instance of built-in int but not of a subclass, and is that use-case so compelling that it must be enforced for all uses of int() etc.? As I see it, what I expect is this: n = int(something) assert isinstance(n, int) not this: n = int(something) assert type(n) is int I haven't cared about checking type identity since "Unifying types and classes" way back in Python 2.2, and I don't see why we should enforce a more restrictive rule now. -- Steven From tseaver at palladion.com Wed Apr 3 18:25:32 2013 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 03 Apr 2013 12:25:32 -0400 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <515C4F8B.2030007@pearwood.info> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> <515C4F8B.2030007@pearwood.info> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/03/2013 11:49 AM, Steven D'Aprano wrote: > -1 on forcing __int__, __str__, __float__ etc. to return instances of > the built-in types. - -1 as well, for the reasons Steven lists. The only quasi-legitimate reason I know of for checking 'type(x) is int' rather than 'isinstance(x, int)' is speed. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlFcV/wACgkQ+gerLs4ltQ6OBwCg0YMyUdiji82TwYQZTx85F9cJ wmMAoKBL13C+a4MN640jL5X+X+G9RP5b =8q3C -----END PGP SIGNATURE----- From guido at python.org Wed Apr 3 18:36:50 2013 From: guido at python.org (Guido van Rossum) Date: Wed, 3 Apr 2013 09:36:50 -0700 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <515C52FB.2070705@pearwood.info> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <20130403101612.27157a4d@anarchist> <515C52FB.2070705@pearwood.info> Message-ID: I always intended for int() and str() to case subclasses to the built-in base class, and I don't want to change that rule. Consider a subclass of int() that overrides __repr__() and __str__() to print something fancy (maybe it defaults to hex; maybe it's an enum :-). I want to be able to say repr(int(x)) and get the standard decimal representation. Same with strings. If int() or str() were allowed to return a subclass instance, this wouldn't work, and I'd have to resort to draconian measures. (Betcha the first few solutions you come up with don't even work. :-) There are plenty of other use cases where a trivial subclass of one of the built-in types is used as some kind of "flag" -- e.g. maybe for values that ought to be serialized differently, or to enable some kind of type checking -- but it should always be possible to convert such values to the underlying base class to remove the special treatment. -- --Guido van Rossum (python.org/~guido -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Wed Apr 3 19:41:27 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 04 Apr 2013 04:41:27 +1100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <20130403101612.27157a4d@anarchist> <515C52FB.2070705@pearwood.info> Message-ID: <515C69C7.2080107@pearwood.info> On 04/04/13 03:36, Guido van Rossum wrote: > Consider a subclass of int() that overrides __repr__() and __str__() to > print something fancy (maybe it defaults to hex; maybe it's an enum :-). I > want to be able to say repr(int(x)) and get the standard decimal > representation. Same with strings. If int() or str() were allowed to return > a subclass instance, this wouldn't work, and I'd have to resort to > draconian measures. int and str currently are allowed to return subclass instances. > (Betcha the first few solutions you come up with don't > even work. :-) Well, I'm always game to learn something. Challenge accepted. # Force a string subclass s to a built-in string. ''.join(s) # Force an int subclass n to a built-in int. (0).__add__(n) # And similarly for float subclass. 0.0.__add__(x) -- Steven From barry at python.org Wed Apr 3 19:44:06 2013 From: barry at python.org (Barry Warsaw) Date: Wed, 3 Apr 2013 13:44:06 -0400 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <20130403101612.27157a4d@anarchist> Message-ID: <20130403134406.56bb37ad@anarchist> On Apr 04, 2013, at 01:14 AM, Nick Coghlan wrote: >Implementing __index__ just means "This type can be converted to a Python >integer without losing information". Aside from that extra "without >information loss" qualification, it's the same as __int__. Hmm. object.__index__(self) Called to implement operator.index(). Also called whenever Python needs an integer object (such as in slicing, or in the built-in bin(), hex() and oct() functions). Must return an integer. operator.index(a) operator.__index__(a) Return a converted to an integer. Equivalent to a.__index__(). bin(x) Convert an integer number to a binary string. The result is a valid Python expression. If x is not a Python int object, it has to define an __index__() method that returns an integer. Certainly, in slicing an int subclass will work fine: >>> class myint(int): ... pass ... >>> range(10)[myint(2):myint(3)] range(2, 3) Same goes for hex/oct/bin: >>> hex(myint(4)) '0x4' >>> bin(myint(4)) '0b100' >>> oct(myint(4)) '0o4' Also, hex/oct/bin aren't types, they're functions, so this doesn't seem equivalent to int or str for me. The latter two are much more equivalent to the built-in tuple, dict, and list types. So I don't think there's any inconsistency in allowing int subclasses to be returned from __index__(), and nothing should break, so I see no reason to change the status quo here. >>> class anint(int): ... def __index__(self): ... return myint(self) ... >>> from operator import index >>> type(index(anint(7))) Aside: It seems a bit odd to me that bin/hex/oct are defined to use __index__() instead of __int__(), but that's probably not worth arguing about. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Wed Apr 3 19:46:49 2013 From: barry at python.org (Barry Warsaw) Date: Wed, 3 Apr 2013 13:46:49 -0400 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <515C52FB.2070705@pearwood.info> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <20130403101612.27157a4d@anarchist> <515C52FB.2070705@pearwood.info> Message-ID: <20130403134649.68bd21a3@anarchist> On Apr 04, 2013, at 03:04 AM, Steven D'Aprano wrote: >On 04/04/13 01:16, Barry Warsaw wrote: >> the other built-in types-as-functions, so int() calls __int__() which must >> return a concrete integer. >Why must it? I think that's the claim which must be justified, not just taken >as a given. When we call n = int(something), what's the use-case for caring >that n is an instance of built-in int but not of a subclass, and is that >use-case so compelling that it must be enforced for all uses of int() etc.? It's a consistency-of-implementation issue. Where built-in types are callable, they return concrete instances of themselves. This is true for e.g. list, tuple, dict, bytes, str, and should also be true of int. -Barry From barry at python.org Wed Apr 3 19:56:04 2013 From: barry at python.org (Barry Warsaw) Date: Wed, 3 Apr 2013 13:56:04 -0400 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <20130403134649.68bd21a3@anarchist> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <20130403101612.27157a4d@anarchist> <515C52FB.2070705@pearwood.info> <20130403134649.68bd21a3@anarchist> Message-ID: <20130403135604.489f022e@anarchist> On Apr 03, 2013, at 01:46 PM, Barry Warsaw wrote: >It's a consistency-of-implementation issue. Where built-in types are >callable, they return concrete instances of themselves. This is true for >e.g. list, tuple, dict, bytes, str, and should also be true of int. Well, I guess it's consistent for some of those built-in types . -Barry From ethan at stoneleaf.us Wed Apr 3 19:50:21 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 03 Apr 2013 10:50:21 -0700 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <20130403134649.68bd21a3@anarchist> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <20130403101612.27157a4d@anarchist> <515C52FB.2070705@pearwood.info> <20130403134649.68bd21a3@anarchist> Message-ID: <515C6BDD.9090500@stoneleaf.us> On 04/03/2013 10:46 AM, Barry Warsaw wrote: > On Apr 04, 2013, at 03:04 AM, Steven D'Aprano wrote: > >> On 04/04/13 01:16, Barry Warsaw wrote: > >>> the other built-in types-as-functions, so int() calls __int__() which must >>> return a concrete integer. > >> Why must it? I think that's the claim which must be justified, not just taken >> as a given. When we call n = int(something), what's the use-case for caring >> that n is an instance of built-in int but not of a subclass, and is that >> use-case so compelling that it must be enforced for all uses of int() etc.? > > It's a consistency-of-implementation issue. Where built-in types are > callable, they return concrete instances of themselves. This is true for > e.g. list, tuple, dict, bytes, str, and should also be true of int. +1 From tseaver at palladion.com Wed Apr 3 21:21:59 2013 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 03 Apr 2013 15:21:59 -0400 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <515C6BDD.9090500@stoneleaf.us> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <20130403101612.27157a4d@anarchist> <515C52FB.2070705@pearwood.info> <20130403134649.68bd21a3@anarchist> <515C6BDD.9090500@stoneleaf.us> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/03/2013 01:50 PM, Ethan Furman wrote: > On 04/03/2013 10:46 AM, Barry Warsaw wrote: >> On Apr 04, 2013, at 03:04 AM, Steven D'Aprano wrote: >> >>> On 04/04/13 01:16, Barry Warsaw wrote: >> >>>> the other built-in types-as-functions, so int() calls __int__() >>>> which must return a concrete integer. >> >>> Why must it? I think that's the claim which must be justified, not >>> just taken as a given. When we call n = int(something), what's >>> the use-case for caring that n is an instance of built-in int but >>> not of a subclass, and is that use-case so compelling that it must >>> be enforced for all uses of int() etc.? >> >> It's a consistency-of-implementation issue. Where built-in types >> are callable, they return concrete instances of themselves. This is >> true for e.g. list, tuple, dict, bytes, str, and should also be true >> of int. Given that requirement, we still don't have to mandate that __int__ return an actual instance of the int type: the coercion could happen inside int() (as it would for any non-subclass). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlFcgVcACgkQ+gerLs4ltQ4ScwCfScssK/Cv74lPitQxbygmk5h/ RGoAnj2yUEgmEgorJi8GZh0GEB/iJrN1 =0I+y -----END PGP SIGNATURE----- From ethan at stoneleaf.us Wed Apr 3 21:36:58 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 03 Apr 2013 12:36:58 -0700 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <20130403101612.27157a4d@anarchist> <515C52FB.2070705@pearwood.info> <20130403134649.68bd21a3@anarchist> <515C6BDD.9090500@stoneleaf.us> Message-ID: <515C84DA.7020004@stoneleaf.us> On 04/03/2013 12:21 PM, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 04/03/2013 01:50 PM, Ethan Furman wrote: >> On 04/03/2013 10:46 AM, Barry Warsaw wrote: >>> On Apr 04, 2013, at 03:04 AM, Steven D'Aprano wrote: >>> >>>> On 04/04/13 01:16, Barry Warsaw wrote: >>> >>>>> the other built-in types-as-functions, so int() calls __int__() >>>>> which must return a concrete integer. >>> >>>> Why must it? I think that's the claim which must be justified, not >>>> just taken as a given. When we call n = int(something), what's >>>> the use-case for caring that n is an instance of built-in int but >>>> not of a subclass, and is that use-case so compelling that it must >>>> be enforced for all uses of int() etc.? >>> >>> It's a consistency-of-implementation issue. Where built-in types >>> are callable, they return concrete instances of themselves. This is >>> true for e.g. list, tuple, dict, bytes, str, and should also be true >>> of int. > > Given that requirement, we still don't have to mandate that __int__ > return an actual instance of the int type: the coercion could happen > inside int() (as it would for any non-subclass). I don't understand. A non-int could only become an int via __int__, which int() calls. What magic is there in int() to turn any arbitrary object into an integer? --> class NonInt(): ... def __str__(self): ... return "Not an integer!" ... --> ni = NonInt() --> ni <__main__.NonInt object at 0x267a090> --> str(ni) 'Not an integer!' --> int(ni) Traceback (most recent call last): File "", line 1, in TypeError: int() argument must be a string or a number, not 'NonInt' -- ~Ethan~ From francismb at email.de Wed Apr 3 22:01:21 2013 From: francismb at email.de (francis) Date: Wed, 03 Apr 2013 22:01:21 +0200 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <51588AF7.6000801@email.de> References: <51583A56.8070404@hotpy.org> <51588AF7.6000801@email.de> Message-ID: <515C8A91.4040705@email.de> On 03/31/2013 09:13 PM, francis wrote: > why is printing new ? read twice before you write :-) From python-dev at masklinn.net Wed Apr 3 22:01:48 2013 From: python-dev at masklinn.net (Xavier Morel) Date: Wed, 3 Apr 2013 22:01:48 +0200 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <20130403134649.68bd21a3@anarchist> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <20130403101612.27157a4d@anarchist> <515C52FB.2070705@pearwood.info> <20130403134649.68bd21a3@anarchist> Message-ID: <1A568864-1086-47DE-B293-CB3F66B9C8EF@masklinn.net> On 2013-04-03, at 19:46 , Barry Warsaw wrote: > On Apr 04, 2013, at 03:04 AM, Steven D'Aprano wrote: > >> On 04/04/13 01:16, Barry Warsaw wrote: > >>> the other built-in types-as-functions, so int() calls __int__() which must >>> return a concrete integer. > >> Why must it? I think that's the claim which must be justified, not just taken >> as a given. When we call n = int(something), what's the use-case for caring >> that n is an instance of built-in int but not of a subclass, and is that >> use-case so compelling that it must be enforced for all uses of int() etc.? > > It's a consistency-of-implementation issue. Where built-in types are > callable, they return concrete instances of themselves. This is true for > e.g. list, tuple, dict, bytes, str, and should also be true of int. FWIW unless I missed something it's true for none of bytes, str or float, though it's true for complex (for some reason): types = (int, float, complex, bytes, str) Obj = type('Obj', (), { '__{0.__name__}__'.format(t): (lambda t: lambda self: type('my_{0.__name__}'.format(t), (t,), {})())(t) for t in types }) obj = Obj() for t in types: print("{} = {} ? {}".format(t, type(t(obj)), type(t(obj)) is t)) > python3 test.py = ? False = ? False = ? True = ? False = ? False bool can not be subclassed so the question doesn't make sense for it Broadly speaking (complex doesn't fit it), if there's a dedicated dunder method in the data model, the only check on what it returns is that it's a subtype of the conversion type. list, tuple and dict use non-dedicated conversion methods (iteration or a fallback thereof) so they don't have this occasion and have no choice but to instantiate "themselves" From cs at zip.com.au Thu Apr 4 00:07:14 2013 From: cs at zip.com.au (Cameron Simpson) Date: Thu, 4 Apr 2013 09:07:14 +1100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <515C24E3.5010703@avl.com> References: <515C24E3.5010703@avl.com> Message-ID: <20130403220714.GA77099@cskk.homeip.net> On 03Apr2013 14:47, Hrvoje Niksic wrote: | On 04/03/2013 01:17 PM, Nick Coghlan wrote: | Why would one want to be absolutely sure of getting an int? So that arithmetic can be relied upon? If a subclass can override the add methods etc it can look like an int, be a subclass instance of an int, and yet not act like an int in all ways. If I go int(x), I personally want a real int out the end of it. -- Cameron Simpson Q: What's the difference between a psychotic and a neurotic? A: A psychotic doesn't believe that 2 + 2 = 4. A neurotic knows it's true, but it bothers him. From asolano at icai.es Thu Apr 4 03:21:36 2013 From: asolano at icai.es (=?UTF-8?Q?Alfredo_Solano_Mart=C3=ADnez?=) Date: Thu, 4 Apr 2013 03:21:36 +0200 Subject: [Python-Dev] [Announcement] New mailing list for code quality tools including Flake8, Pyflakes and Pep8 In-Reply-To: References: Message-ID: Hi, Are you planning to cover the code quality of the interpreter itself too? I've been recently reading through the cert.org secure coding practice recommendations and was wondering if there has is any ongoing effort to perform static analysis on the cpython codebase. Thanks, Alfredo On Wed, Apr 3, 2013 at 4:24 PM, Ian Cordasco wrote: > Hello, > > There's a new mailing-list related to Python code-quality tools. > > Are you concerned about the evolution of various code checkers? > Do you have questions or suggestions? > > Subscribe here: > http://mail.python.org/mailman/listinfo/code-quality > > Best regards, > Ian > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/asolano%40icai.es From tjreedy at udel.edu Thu Apr 4 04:18:57 2013 From: tjreedy at udel.edu (Terry Jan Reedy) Date: Wed, 03 Apr 2013 22:18:57 -0400 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <515C84DA.7020004@stoneleaf.us> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <20130403101612.27157a4d@anarchist> <515C52FB.2070705@pearwood.info> <20130403134649.68bd21a3@anarchist> <515C6BDD.9090500@stoneleaf.us> <515C84DA.7020004@stoneleaf.us> Message-ID: On 4/3/2013 3:36 PM, Ethan Furman wrote: > On 04/03/2013 12:21 PM, Tres Seaver wrote: >> Given that requirement, we still don't have to mandate that __int__ >> return an actual instance of the int type: the coercion could happen >> inside int() (as it would for any non-subclass). > > I don't understand. A non-int could only become an int via __int__, > which int() calls. What magic is there in int() to turn any arbitrary > object into an integer? From the 2.7 manual: " object.__complex__(self) object.__int__(self) object.__long__(self) object.__float__(self) Called to implement the built-in functions complex(), int(), long(), and float(). Should return a value of the appropriate type." I have always understood 'value of the appropriate type' to mean just what Guido says he intended it to mean. Changing to 'instance of the class or subclass thereof' is a change. From steve at pearwood.info Thu Apr 4 04:46:25 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 04 Apr 2013 13:46:25 +1100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <20130403220714.GA77099@cskk.homeip.net> References: <515C24E3.5010703@avl.com> <20130403220714.GA77099@cskk.homeip.net> Message-ID: <515CE981.7060803@pearwood.info> On 04/04/13 09:07, Cameron Simpson wrote: > On 03Apr2013 14:47, Hrvoje Niksic wrote: > | On 04/03/2013 01:17 PM, Nick Coghlan wrote: > | Why would one want to be absolutely sure of getting an int? > > So that arithmetic can be relied upon? If a subclass can override > the add methods etc it can look like an int, be a subclass instance > of an int, and yet not act like an int in all ways. Python is generally a consenting adults language. If you don't trust the subclass to actually behave like an int in ways that matter, why are you using it? Ultimately, something may have monkey-patched builtins int without your knowledge, so even calling int() doesn't guarantee anything about the object you get back. Isn't it funny how we have differences in opinion on just when and how we're allowed to shoot ourselves in the foot? I personally hate the idea that global constants aren't constant, and can be rebound by anything, but others apparently don't care. You hate the idea that int() might return an instance of a subclass, and I think that's a feature, not a bug. -- Steven From ericsnowcurrently at gmail.com Thu Apr 4 06:55:58 2013 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Wed, 3 Apr 2013 22:55:58 -0600 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <515C24E3.5010703@avl.com> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> Message-ID: On Wed, Apr 3, 2013 at 6:47 AM, Hrvoje Niksic wrote: > It seems like a good feature that an __int__ implementation can choose to > return an int subclass with additional (and optional) information. After > all, int subclass instances should be usable everywhere where ints are, > including in C code. Unless you want to try to use the concrete C-API in CPython. In my experience the concrete API is not very subclass friendly. -eric From cf.natali at gmail.com Thu Apr 4 06:57:14 2013 From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=) Date: Thu, 4 Apr 2013 06:57:14 +0200 Subject: [Python-Dev] [Announcement] New mailing list for code quality tools including Flake8, Pyflakes and Pep8 In-Reply-To: References: Message-ID: > Are you planning to cover the code quality of the interpreter itself > too? I've been recently reading through the cert.org secure coding > practice recommendations and was wondering if there has is any ongoing > effort to perform static analysis on the cpython codebase. AFAICT CPython already benefits from Coverity scans (I guess the Python-security guys receive those notifications). Note that this only covers the C codebase. cf From ericsnowcurrently at gmail.com Thu Apr 4 06:57:15 2013 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Wed, 3 Apr 2013 22:57:15 -0600 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: On Mon, Apr 1, 2013 at 4:52 PM, Nick Coghlan wrote: > This is really a quality-of-implementation issue in the import system rather > than a core language design problem. It's just that those of us with the > knowledge and ability to fix it aren't inclined to do so because circular > imports usually (although not quite always) indicate a need to factor some > common code out into a third support module imported by both of the original > modules. At that point, the code is cleaner and more decoupled, and the > uneven circular import support ceases to be a problem for that application. +1 -eric From ncoghlan at gmail.com Thu Apr 4 07:37:27 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 4 Apr 2013 15:37:27 +1000 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> Message-ID: On 4 Apr 2013 14:58, "Eric Snow" wrote: > > On Wed, Apr 3, 2013 at 6:47 AM, Hrvoje Niksic wrote: > > It seems like a good feature that an __int__ implementation can choose to > > return an int subclass with additional (and optional) information. After > > all, int subclass instances should be usable everywhere where ints are, > > including in C code. > > Unless you want to try to use the concrete C-API in CPython. In my > experience the concrete API is not very subclass friendly. Using it with subclasses is an outright bug (except as part of a subclass implementation). Cheers, Nick. > > -eric > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From olemis at gmail.com Thu Apr 4 07:56:34 2013 From: olemis at gmail.com (Olemis Lang) Date: Thu, 4 Apr 2013 00:56:34 -0500 Subject: [Python-Dev] [Announcement] New mailing list for code quality tools including Flake8, Pyflakes and Pep8 In-Reply-To: References: Message-ID: On 4/3/13, Charles-Fran?ois Natali wrote: >> Are you planning to cover the code quality of the interpreter itself >> too? I've been recently reading through the cert.org secure coding >> practice recommendations and was wondering if there has is any ongoing >> effort to perform static analysis on the cpython codebase. > > AFAICT CPython already benefits from Coverity scans (I guess the > Python-security guys receive those notifications). Note that this only > covers the C codebase. > ... but the question seems to be ? is there anything similar that could be used for Python code ? ? -- Regards, Olemis. Apache? Bloodhound contributor http://issues.apache.org/bloodhound Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: From cf.natali at gmail.com Thu Apr 4 10:18:58 2013 From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=) Date: Thu, 4 Apr 2013 10:18:58 +0200 Subject: [Python-Dev] Slides from today's parallel/async Python talk In-Reply-To: <0FE526DD-12B6-4A30-A68C-CBDB390B54C5@snakebite.org> References: <20130314020540.GB22505@snakebite.org> <5141C0B5.6060904@python.org> <20130314182352.GC24307@snakebite.org> <20130314222337.GG24307@snakebite.org> <20130321141725.5a1e9d0d@pitrou.net> <0FE526DD-12B6-4A30-A68C-CBDB390B54C5@snakebite.org> Message-ID: Just a quick implementation question (didn't have time to read through all your emails :-) async.submit_work(func, args, kwds, callback=None, errback=None) How do you implement arguments passing and return value? e.g. let's say I pass a list as argument: how do you iterate on the list from the worker thread without modifying the backing objects for refcounts (IIUC you use a per-thread heap and don't do any refcounting). Same thing for return value, how do you pass it to the callback? cf From hrvoje.niksic at avl.com Thu Apr 4 11:39:35 2013 From: hrvoje.niksic at avl.com (Hrvoje Niksic) Date: Thu, 04 Apr 2013 11:39:35 +0200 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> Message-ID: <515D4A57.7010806@avl.com> Eric Snow: > On Wed, Apr 3, 2013 at 6:47 AM, Hrvoje Niksic wrote: >> It seems like a good feature that an __int__ implementation can choose to >> return an int subclass with additional (and optional) information. After >> all, int subclass instances should be usable everywhere where ints are, >> including in C code. > > Unless you want to try to use the concrete C-API in CPython. In my > experience the concrete API is not very subclass friendly. Nick: > Using it with subclasses is an outright bug (except as part of > a subclass implementation). This is true for mutable objects like dicts and lists where calling things like PyDict_SetItem will happily circumvent the object. But for ints and floats, all that the C code really cares about is the object's intrinsic value as returned by PyLong_AS_LONG and friends, which is constant and unchangeable by subclasses. The typical reason to subclass int is to add more information or new methods on the instance, not to break basic arithmetic. Doing anything else breaks subtitutability and is well outside the realm of "consenting adults". Someone who wants to change basic arithmetic is free to implement an independent int type. Hrvoje From kristjan at ccpgames.com Thu Apr 4 11:36:11 2013 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Thu, 4 Apr 2013 09:36:11 +0000 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: > -----Original Message----- > From: Eric Snow [mailto:ericsnowcurrently at gmail.com] > Sent: 4. apr?l 2013 04:57 > > imported by both of the original modules. At that point, the code is > > cleaner and more decoupled, and the uneven circular import support > ceases to be a problem for that application. > > +1 I tried to make the point in an earlier mail that I don't think that we ought to let our personal opinions de-jour on software architecture put a constraint on our language features. There can be good and valid reasons to put things that depend on each other in separate modules, for example when trying to separate modules by functionality or simply when splitting a long file into two. Notice that cyclic dependencies are allowed _within_ a module file. Imagine if we decided that we could only refer to objects _previously_ declared within a .py file, because it encouraged the good design practice of factoring out common dependencies. Dependency graphs of software entities can sadly not always be reduced into a DAG, and we should, IMHO, by no means force people to keep each cygle in the dependency graph within a single .py module. K From graffatcolmingov at gmail.com Thu Apr 4 14:45:20 2013 From: graffatcolmingov at gmail.com (Ian Cordasco) Date: Thu, 4 Apr 2013 08:45:20 -0400 Subject: [Python-Dev] [Announcement] New mailing list for code quality tools including Flake8, Pyflakes and Pep8 In-Reply-To: References: Message-ID: On Wed, Apr 3, 2013 at 9:21 PM, Alfredo Solano Mart?nez wrote: > Hi, > > Are you planning to cover the code quality of the interpreter itself > too? I've been recently reading through the cert.org secure coding > practice recommendations and was wondering if there has is any ongoing > effort to perform static analysis on the cpython codebase. Hey Alfredo, We do not currently have any tools to do that, but it would definitely be something interesting to discuss and maybe design on the list. I'm sure there are static analysis tools for the C part and I'm sure we as a community could come up with a "super tool" to check both the C and Python parts of CPython. -- Ian From jtaylor.debian at googlemail.com Thu Apr 4 14:46:05 2013 From: jtaylor.debian at googlemail.com (Julian Taylor) Date: Thu, 04 Apr 2013 14:46:05 +0200 Subject: [Python-Dev] How to fix the incorrect shared library extension on linux for 3.2 and newer? In-Reply-To: References: Message-ID: <515D760D.50608@googlemail.com> The values on macos for these variables still look wrong in 3.3.1rc1: ./configure --prefix=/Users/jtaylor/tmp/py3.3.1 --enable-shared on macosx-10.8-x86_64 sys.version_info(major=3, minor=3, micro=1, releaselevel='candidate', serial=1) SO .so EXT_SUFFIX .so SHLIB_SUFFIX 0 the only correct one here is EXT_SUFFIX, SHLIB_SUFFIX should be .dylib (libpython is a .dylib) and .SO possibly too given for what it was used in the past. 3.3.0 also returns wrong values SO .so EXT_SUFFIX None SHLIB_SUFFIX "" From ncoghlan at gmail.com Thu Apr 4 15:01:46 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 4 Apr 2013 23:01:46 +1000 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: On 4 Apr 2013 19:37, "Kristj?n Valur J?nsson" wrote: > > > > > -----Original Message----- > > From: Eric Snow [mailto:ericsnowcurrently at gmail.com] > > Sent: 4. apr?l 2013 04:57 > > > imported by both of the original modules. At that point, the code is > > > cleaner and more decoupled, and the uneven circular import support > > ceases to be a problem for that application. > > > > +1 > > I tried to make the point in an earlier mail that I don't think that we ought to let our personal opinions de-jour on software architecture put a constraint on our language features. > There can be good and valid reasons to put things that depend on each other in separate modules, for example when trying to separate modules by functionality or simply when splitting a long file into two. > Notice that cyclic dependencies are allowed _within_ a module file. Imagine if we decided that we could only refer to objects _previously_ declared within a .py file, because it encouraged the good design practice of factoring out common dependencies. > Dependency graphs of software entities can sadly not always be reduced into a DAG, and we should, IMHO, by no means force people to keep each cygle in the dependency graph within a single .py module. That's why it's just a reason none of the current import system devs are inclined to work on it, rather than a reason for rejecting proposals and tested patches that give improved behaviour. Cheers, Nick. > > K > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From oscar.j.benjamin at gmail.com Thu Apr 4 16:23:38 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 4 Apr 2013 15:23:38 +0100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <515D4A57.7010806@avl.com> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> <515D4A57.7010806@avl.com> Message-ID: On 4 April 2013 10:39, Hrvoje Niksic wrote: > >> On Wed, Apr 3, 2013 at 6:47 AM, Hrvoje Niksic >> wrote: >>> >>> It seems like a good feature that an __int__ implementation can choose to >>> return an int subclass with additional (and optional) information. After >>> all, int subclass instances should be usable everywhere where ints are, >>> including in C code. [SNIP] > > The typical reason to subclass int is to add more information or new methods > on the instance, not to break basic arithmetic. Doing anything else breaks > subtitutability and is well outside the realm of "consenting adults". > Someone who wants to change basic arithmetic is free to implement an > independent int type. The reason for calling int(obj) is to get an object that is precisely of type int. When I call this I do not want any modified or additional methods or data attached to the resulting object. The example given at the start of the thread makes minimal modifications in an int subclass but still allows the result of int(obj) to be unpickleable: >>> class Int1(int): ... def __int__(self): ... return self ... >>> n = Int1(4) >>> n 4 >>> import pickle >>> ni = int(n) >>> pickle.dumps(ni) ... snip ... File "q:\tools\Python27\lib\pickle.py", line 562, in save_tuple save(element) File "q:\tools\Python27\lib\pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "q:\tools\Python27\lib\pickle.py", line 401, in save_reduce save(args) File "q:\tools\Python27\lib\pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "q:\tools\Python27\lib\pickle.py", line 562, in save_tuple save(element) File "q:\tools\Python27\lib\pickle.py", line 291, in save issc = issubclass(t, TypeType) RuntimeError: maximum recursion depth exceeded while calling a Python object I don't know whether that's a bug in pickle, but I think it's fair to say that there are times when someone wants an object that is precisely of type int. They should be able to rely on int(obj) returning an int or raising an error. This is true similarly for __index__. The entire purpose of __index__ is to permit APIs like list.__getitem__ to work extensibly with any int-like object. This is achieved by allowing any object to advertise its convertibility from an int-like object to an int with the same numeric value. Anyone who calls operator.index(obj) is explicitly stating that they do not want any property of obj apart from its integer value and that they want that as an int. That it should be an object of type int is explicitly stated in PEP 357 (http://www.python.org/dev/peps/pep-0357/): """ 2) The __index__ special method will have the signature def __index__(self): return obj where obj must be either an int or a long. """ Oscar From asolano at icai.es Thu Apr 4 16:34:22 2013 From: asolano at icai.es (=?UTF-8?Q?Alfredo_Solano_Mart=C3=ADnez?=) Date: Thu, 4 Apr 2013 16:34:22 +0200 Subject: [Python-Dev] [Announcement] New mailing list for code quality tools including Flake8, Pyflakes and Pep8 In-Reply-To: References: Message-ID: On Thu, Apr 4, 2013 at 2:45 PM, Ian Cordasco wrote: > Hey Alfredo, > > We do not currently have any tools to do that, but it would definitely > be something interesting to discuss and maybe design on the list. I'm > sure there are static analysis tools for the C part and I'm sure we as > a community could come up with a "super tool" to check both the C and > Python parts of CPython. As cf said it seems they're using Coverity for the C part, but the idea of checking the cpython interpreter with python itself is intriguing. Suscribed. Alfredo From solipsis at pitrou.net Thu Apr 4 16:42:12 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 4 Apr 2013 16:42:12 +0200 Subject: [Python-Dev] [Announcement] New mailing list for code quality tools including Flake8, Pyflakes and Pep8 References: Message-ID: <20130404164212.565f2571@pitrou.net> Le Thu, 4 Apr 2013 06:57:14 +0200, Charles-Fran?ois Natali a ?crit : > > Are you planning to cover the code quality of the interpreter itself > > too? I've been recently reading through the cert.org secure coding > > practice recommendations and was wondering if there has is any > > ongoing effort to perform static analysis on the cpython codebase. > > AFAICT CPython already benefits from Coverity scans (I guess the > Python-security guys receive those notifications). Note that this only > covers the C codebase. Correction: the security@ address doesn't receive any coverity notifications. Perhaps someone checks the (private) coverity builds from time to time, but I don't think there's anything automatic. Regards Antoine. From rosuav at gmail.com Thu Apr 4 16:47:45 2013 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 5 Apr 2013 01:47:45 +1100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> <515D4A57.7010806@avl.com> Message-ID: On Fri, Apr 5, 2013 at 1:23 AM, Oscar Benjamin wrote: > The reason for calling int(obj) is to get an object that is precisely > of type int. When I call this I do not want any modified or additional > methods or data attached to the resulting object. There's something I'm fundamentally not understanding about this debate, and that is: How is it that calling a class can logically return anything other than an instance of that class? Taking it to a user-defined type: class Foo: pass class Bar(Foo): pass Is there any argument that I can pass to Foo() to get back a Bar()? Would anyone expect there to be one? Sure, I could override __new__ to do stupid things, but in terms of logical expectations, I'd expect that Foo(x) will return a Foo object, not a Bar object. Why should int be any different? What have I missed here? ChrisA From g.brandl at gmx.net Thu Apr 4 16:59:24 2013 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 04 Apr 2013 16:59:24 +0200 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> <515D4A57.7010806@avl.com> Message-ID: Am 04.04.2013 16:47, schrieb Chris Angelico: > On Fri, Apr 5, 2013 at 1:23 AM, Oscar Benjamin > wrote: >> The reason for calling int(obj) is to get an object that is precisely >> of type int. When I call this I do not want any modified or additional >> methods or data attached to the resulting object. > > There's something I'm fundamentally not understanding about this > debate, and that is: How is it that calling a class can logically > return anything other than an instance of that class? Taking it to a > user-defined type: > > class Foo: > pass > > class Bar(Foo): > pass > > Is there any argument that I can pass to Foo() to get back a Bar()? > Would anyone expect there to be one? Sure, I could override __new__ to > do stupid things, but in terms of logical expectations, I'd expect > that Foo(x) will return a Foo object, not a Bar object. Why should int > be any different? What have I missed here? I think the issue that the constructors for basic classes like int() are often seen as built-in functions that "cast" values to the respective type (or alternatively, functions that call the respective __method__ on the argument, like len()), not class constructors. FWIW, I agree with you that the "class constructor" view is the right one and would prefer exact int/str/... instances returned. Georg From guido at python.org Thu Apr 4 16:59:49 2013 From: guido at python.org (Guido van Rossum) Date: Thu, 4 Apr 2013 07:59:49 -0700 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> <515D4A57.7010806@avl.com> Message-ID: On Thu, Apr 4, 2013 at 7:47 AM, Chris Angelico wrote: > On Fri, Apr 5, 2013 at 1:23 AM, Oscar Benjamin > wrote: > > The reason for calling int(obj) is to get an object that is precisely > > of type int. When I call this I do not want any modified or additional > > methods or data attached to the resulting object. > > There's something I'm fundamentally not understanding about this > debate, and that is: How is it that calling a class can logically > return anything other than an instance of that class? Taking it to a > user-defined type: > > class Foo: > pass > > class Bar(Foo): > pass > > Is there any argument that I can pass to Foo() to get back a Bar()? > Would anyone expect there to be one? Sure, I could override __new__ to > do stupid things, but in terms of logical expectations, I'd expect > that Foo(x) will return a Foo object, not a Bar object. Why should int > be any different? What have I missed here? > A class can define a __new__ method that returns a different object. E.g. (python 3): >>> class C: ... def __new__(cls): return 42 ... >>> C() 42 >>> -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From python-dev at masklinn.net Thu Apr 4 17:01:25 2013 From: python-dev at masklinn.net (Xavier Morel) Date: Thu, 4 Apr 2013 17:01:25 +0200 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> <515D4A57.7010806@avl.com> Message-ID: <6E74FA58-816D-49B9-BB73-98619A6DAF5F@masklinn.net> On 2013-04-04, at 16:47 , Chris Angelico wrote: > Sure, I could override __new__ to do stupid things Or to do perfectly logical and sensible things, such as implementing "cluster classes" or using the base class as a factory of sorts. > in terms of logical expectations, I'd expect > that Foo(x) will return a Foo object, not a Bar object. The problem is the expectation of what "a Foo object" is: type-wise, any Bar object is also a Foo object. I would not expect Foo() to return an object of a completely unrelated type, but returning an object of a subtype? That does not seem outlandish. From rosuav at gmail.com Thu Apr 4 17:01:53 2013 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 5 Apr 2013 02:01:53 +1100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> <515D4A57.7010806@avl.com> Message-ID: On Fri, Apr 5, 2013 at 1:59 AM, Guido van Rossum wrote: > On Thu, Apr 4, 2013 at 7:47 AM, Chris Angelico wrote: >> Is there any argument that I can pass to Foo() to get back a Bar()? >> Would anyone expect there to be one? Sure, I could override __new__ to >> do stupid things, but in terms of logical expectations, I'd expect >> that Foo(x) will return a Foo object, not a Bar object. Why should int >> be any different? What have I missed here? > > > A class can define a __new__ method that returns a different object. E.g. > (python 3): > Right, I'm aware it's possible. But who would expect it of a class? ChrisA From python-dev at masklinn.net Thu Apr 4 17:16:20 2013 From: python-dev at masklinn.net (Xavier Morel) Date: Thu, 4 Apr 2013 17:16:20 +0200 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> <515D4A57.7010806@avl.com> Message-ID: On 2013-04-04, at 17:01 , Chris Angelico wrote: > On Fri, Apr 5, 2013 at 1:59 AM, Guido van Rossum wrote: >> On Thu, Apr 4, 2013 at 7:47 AM, Chris Angelico wrote: >>> Is there any argument that I can pass to Foo() to get back a Bar()? >>> Would anyone expect there to be one? Sure, I could override __new__ to >>> do stupid things, but in terms of logical expectations, I'd expect >>> that Foo(x) will return a Foo object, not a Bar object. Why should int >>> be any different? What have I missed here? >> >> >> A class can define a __new__ method that returns a different object. E.g. >> (python 3): >> > > Right, I'm aware it's possible. But who would expect it of a class? Given it's one of the use cases for __new__ and immutable types have to be initialized through __new__ anyway, why would it be unexpected? From guido at python.org Thu Apr 4 17:17:01 2013 From: guido at python.org (Guido van Rossum) Date: Thu, 4 Apr 2013 08:17:01 -0700 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: On Thu, Apr 4, 2013 at 2:36 AM, Kristj?n Valur J?nsson < kristjan at ccpgames.com> wrote: [...] > There can be good and valid reasons to put things that depend on each > other in separate modules, for example when trying to separate modules by > functionality or simply when splitting a long file into two. > Notice that cyclic dependencies are allowed _within_ a module file. > Imagine if we decided that we could only refer to objects _previously_ > declared within a .py file, because it encouraged the good design practice > of factoring out common dependencies. > Dependency graphs of software entities can sadly not always be reduced > into a DAG, and we should, IMHO, by no means force people to keep each > cygle in the dependency graph within a single .py module. > IMO in both cases (cyclic dependencies within modules and between modules) Python's approach is consistently derived from a specific implementation strategy: names are available after they are defined. This works (at least it doesn't raise NameError :-): def f(): return g() def g(): return f() f() Because by the time f() is called, g() is defined. Note that this doesn't work: def f(): return g() f() def g(): return f() IOW there is no magic. If you think of putting objects in a dictionary, the semantics are clear. (Except for the shenanigans that the compiler carries out for determining whether something is a local variable reference or not, but that's also explainable.) This does not work: a = b b = a because the first line references b which is not yet defined. Similarly, if module A imports module B and B imports A, that works because imports are first satisfied from sys.modules, and when A is loaded, it is entered into sys.modules before its code is executed. But it may be that if A is loaded and starts importing B, B is then loaded and imports A, which finds A in the state it reached when it decided to import B. The rules here are more complicated because of what "from A import X" means (it depends on whether X is a submodule or not) but there are rules, and they are totally well-defined. The final piece of this puzzle is that, given A and B mutually importing each other, the top-level app code could start by importing A or B, and the execution order will be different then. (You can force this by putting A and B in a package whose __init__.py imports them in the desired order.) I don't really see what we could change to avoid breaking code in any particular case -- the burden is up to the library to do it right. I don't see a reason to forbid any of this either. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Thu Apr 4 17:18:27 2013 From: guido at python.org (Guido van Rossum) Date: Thu, 4 Apr 2013 08:18:27 -0700 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> <515D4A57.7010806@avl.com> Message-ID: On Thu, Apr 4, 2013 at 8:01 AM, Chris Angelico wrote: > On Fri, Apr 5, 2013 at 1:59 AM, Guido van Rossum wrote: > > On Thu, Apr 4, 2013 at 7:47 AM, Chris Angelico wrote: > >> Is there any argument that I can pass to Foo() to get back a Bar()? > >> Would anyone expect there to be one? Sure, I could override __new__ to > >> do stupid things, but in terms of logical expectations, I'd expect > >> that Foo(x) will return a Foo object, not a Bar object. Why should int > >> be any different? What have I missed here? > > > > > > A class can define a __new__ method that returns a different object. E.g. > > (python 3): > > > > Right, I'm aware it's possible. But who would expect it of a class? > If it's documented you could expect it. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Thu Apr 4 17:19:10 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 4 Apr 2013 17:19:10 +0200 Subject: [Python-Dev] Semantics of __int__(), __index__() References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> <515D4A57.7010806@avl.com> Message-ID: <20130404171910.73acce92@pitrou.net> Le Fri, 5 Apr 2013 01:47:45 +1100, Chris Angelico a ?crit : > > class Foo: > pass > > class Bar(Foo): > pass > > Is there any argument that I can pass to Foo() to get back a Bar()? > Would anyone expect there to be one? Sure, I could override __new__ to > do stupid things, but in terms of logical expectations, I'd expect > that Foo(x) will return a Foo object, not a Bar object. >>> OSError(errno.ENOENT, "couldn't find that file") FileNotFoundError(2, "couldn't find that file") Regards Antoine. From ethan at stoneleaf.us Thu Apr 4 17:16:26 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 04 Apr 2013 08:16:26 -0700 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> <515D4A57.7010806@avl.com> Message-ID: <515D994A.2000202@stoneleaf.us> On 04/04/2013 08:01 AM, Chris Angelico wrote: > On Fri, Apr 5, 2013 at 1:59 AM, Guido van Rossum wrote: >> On Thu, Apr 4, 2013 at 7:47 AM, Chris Angelico wrote: >>> Is there any argument that I can pass to Foo() to get back a Bar()? >>> Would anyone expect there to be one? Sure, I could override __new__ to >>> do stupid things, but in terms of logical expectations, I'd expect >>> that Foo(x) will return a Foo object, not a Bar object. Why should int >>> be any different? What have I missed here? >> >> >> A class can define a __new__ method that returns a different object. E.g. >> (python 3): > > Right, I'm aware it's possible. But who would expect it of a class? FTR I'm in the int() should return an int camp, but to answer your question: my dbf module has a Table class, but it returns either a Db3Table, FpTable, VfpTable, or ClpTable depending on arguments (if creating a new one) or the type of the table in the existing dbf file. -- ~Ethan~ From brett at python.org Thu Apr 4 17:44:18 2013 From: brett at python.org (Brett Cannon) Date: Thu, 4 Apr 2013 11:44:18 -0400 Subject: [Python-Dev] [Announcement] New mailing list for code quality tools including Flake8, Pyflakes and Pep8 In-Reply-To: <20130404164212.565f2571@pitrou.net> References: <20130404164212.565f2571@pitrou.net> Message-ID: On Thu, Apr 4, 2013 at 10:42 AM, Antoine Pitrou wrote: > Le Thu, 4 Apr 2013 06:57:14 +0200, > Charles-Fran?ois Natali a ?crit : > > > Are you planning to cover the code quality of the interpreter itself > > > too? I've been recently reading through the cert.org secure coding > > > practice recommendations and was wondering if there has is any > > > ongoing effort to perform static analysis on the cpython codebase. > > > > AFAICT CPython already benefits from Coverity scans (I guess the > > Python-security guys receive those notifications). Note that this only > > covers the C codebase. > > Correction: the security@ address doesn't receive any coverity > notifications. Perhaps someone checks the (private) coverity builds from > time to time, but I don't think there's anything automatic. > Christian Heimes has a daily build set up and checks the results periodically. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nad at acm.org Thu Apr 4 17:48:02 2013 From: nad at acm.org (Ned Deily) Date: Thu, 04 Apr 2013 08:48:02 -0700 Subject: [Python-Dev] How to fix the incorrect shared library extension on linux for 3.2 and newer? References: <515D760D.50608@googlemail.com> Message-ID: In article <515D760D.50608 at googlemail.com>, Julian Taylor wrote: > The values on macos for these variables still look wrong in 3.3.1rc1: > > ./configure --prefix=/Users/jtaylor/tmp/py3.3.1 --enable-shared > on macosx-10.8-x86_64 > > sys.version_info(major=3, minor=3, micro=1, releaselevel='candidate', > serial=1) > SO .so > EXT_SUFFIX .so > SHLIB_SUFFIX 0 > > > the only correct one here is EXT_SUFFIX, SHLIB_SUFFIX should be .dylib > (libpython is a .dylib) and .SO possibly too given for what it was used > in the past. > > 3.3.0 also returns wrong values > SO .so > EXT_SUFFIX None > SHLIB_SUFFIX "" I get a SHLIB_SUFFIX of '.so'. --enable-shared configurations are particularly error-prone; for instance, you either need to test using an installed version or be very careful to override the dynamic loading path. Please see the discussion at http://bugs.python.org/issue16754 and either report your concerns there or open an new issue. -- Ned Deily, nad at acm.org From shibturn at gmail.com Thu Apr 4 18:26:04 2013 From: shibturn at gmail.com (Richard Oudkerk) Date: Thu, 04 Apr 2013 17:26:04 +0100 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: On 04/04/2013 4:17pm, Guido van Rossum wrote: > I don't really see what we could change to avoid breaking code in any > particular case -- the burden is up to the library to do it right. I > don't see a reason to forbid any of this either. How about having a form of relative import which only works for submodules. For instance, instead of from . import moduleX write import .moduleX which is currently a SyntaxError. I think this could be implemented as moduleX = importlib.import_module('.moduleX', __package__) -- Richard From guido at python.org Thu Apr 4 19:24:34 2013 From: guido at python.org (Guido van Rossum) Date: Thu, 4 Apr 2013 10:24:34 -0700 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: Redirecting to python-ideas. On Thu, Apr 4, 2013 at 9:26 AM, Richard Oudkerk wrote: > On 04/04/2013 4:17pm, Guido van Rossum wrote: >> >> I don't really see what we could change to avoid breaking code in any >> particular case -- the burden is up to the library to do it right. I >> don't see a reason to forbid any of this either. > > > How about having a form of relative import which only works for submodules. > For instance, instead of > > from . import moduleX > > write > > import .moduleX > > which is currently a SyntaxError. I think this could be implemented as > > moduleX = importlib.import_module('.moduleX', __package__) We considered that when relative import was designed and rejected it, because it violates the expectation that after "import " you can use exactly "" in your code. -- --Guido van Rossum (python.org/~guido) From trent at snakebite.org Thu Apr 4 22:04:41 2013 From: trent at snakebite.org (Trent Nelson) Date: Thu, 4 Apr 2013 16:04:41 -0400 Subject: [Python-Dev] Slides from today's parallel/async Python talk In-Reply-To: References: <20130314020540.GB22505@snakebite.org> <5141C0B5.6060904@python.org> <20130314182352.GC24307@snakebite.org> <20130314222337.GG24307@snakebite.org> <20130321141725.5a1e9d0d@pitrou.net> <0FE526DD-12B6-4A30-A68C-CBDB390B54C5@snakebite.org> Message-ID: <20130404200440.GA12213@snakebite.org> Hi Charles-Fran?ois, On Thu, Apr 04, 2013 at 01:18:58AM -0700, Charles-Fran?ois Natali wrote: > Just a quick implementation question (didn't have time to read through > all your emails :-) > > async.submit_work(func, args, kwds, callback=None, errback=None) > > How do you implement arguments passing and return value? > > e.g. let's say I pass a list as argument: how do you iterate on the > list from the worker thread without modifying the backing objects for > refcounts (IIUC you use a per-thread heap and don't do any > refcounting). Correct, nothing special is done for the arguments (apart from incref'ing them in the main thread before kicking off the parallel thread (then decref'ing them in the main thread once we're sure the parallel thread has finished)). > Same thing for return value, how do you pass it to the > callback? For submit_work(), you can't :-) In fact, an exception is raised if the func() or callback() or errback() attempts to return a non-None value. It's worth noting that I eventually plan to have the map/reduce-type functionality (similar to what multiprocessing offers) available via a separate 'parallel' fa?ade. This will be geared towards programs that are predominantly single-threaded, but have lots of data that can be processed in parallel at various points. Now, with that being said, there are a few options available at the moment if you want to communicate stuff from parallel threads back to the main thread. Originally, you could do something like this: d = async.dict() def foo(): d['foo'] = async.rdtsc() def bar(): d['bar'] = async.rdtsc() async.submit_work(foo) async.submit_work(bar) But I recently identified a few memory-management flaws with that approach (I'm still on the fence with this issue... initially I was going to drop all support, but I've since had ideas to address the memory issues, so, we'll see). There's also this option: d = dict() @async.call_from_main_thread_and_wait def store(k, v): d[str(k)] = str(v) def foo(): store('foo', async.rdtsc()) def bar(): store('bar', async.rdtsc()) async.submit_work(foo) async.submit_work(bar) (Not a particularly performant option though; the main-thread instantly becomes the bottleneck.) Post-PyCon, I've been working on providing new interlocked data types that are specifically designed to bridge the parallel/main- thread divide: xl = async.xlist() def foo(): xl.push(async.rdtsc()) def bar(): xl.push(async.rdtsc()) async.submit_work(foo) async.submit_work(bar) while True: x = xl.pop() if not x: break process(x) What's interesting about xlist() is that it takes ownership of the parallel objects being pushed onto it. That is, it basically clones them, using memory allocated from its own internal heap (allowing the parallel-thread's context heap to be freed, which is desirable). The push/pop operations are interlocked at the C level, which obviates the need for any explicit locking. I've put that work on hold for now though; I want to finish the async client/server stuff (it's about 60-70% done) first. Once that's done, I'll tackle the parallel.*-type fa?ade. Trent. From pje at telecommunity.com Thu Apr 4 22:28:38 2013 From: pje at telecommunity.com (PJ Eby) Date: Thu, 4 Apr 2013 16:28:38 -0400 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: On Thu, Apr 4, 2013 at 11:17 AM, Guido van Rossum wrote: > I don't really see what we could change to avoid breaking code in any > particular case Actually, the problem has *nothing* to do with circularity per se; it's that "import a.b" and "from a import b" behave differently in terms of how they obtain the module 'a.b'... And "from a import b" will *always* fail if 'a.b' is part of a cycle with the current module, whereas "import a.b" will *always* succeed. The workaround is to tell people to always use "import a.b" in the case of circular imports; it's practically a FAQ, at least to me. ;-) The problem with "from import" is that it always tries to getattr(a,'b'), even if 'a.b' is in sys.modules. In contrast, a plain import will simply fetch a.b from sys.modules first. In the case of a normal import, this is no problem, because a.b is set to sys.modules['a.b'] at the end of the module loading process. But if the import is circular, then the module is in sys.modules['a.b'], but *not* yet bound to a.b. So the "from import" fails. So, this is actually an implementation quirk that could be fixed in a (somewhat) straightforward manner: by making "from a import b" succeed if 'a.b' is in sys.modules, the same way "import a.b" does. It would require a little bit of discussion to hash out the exact way to do it, but it could be done. From guido at python.org Thu Apr 4 22:42:08 2013 From: guido at python.org (Guido van Rossum) Date: Thu, 4 Apr 2013 13:42:08 -0700 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: Thanks for the insight. That could indeed be done and I would encourage someone to come up with a fix. FWIW there is another difference between the two forms -- "from a import b" allows b to be any attribute of a, whereas "import a.b" requires b to be a submodule of a. I don't want to compromise on the latter. I do think it would be fine if "from a import b" returned the attribute 'b' of module 'a' if it exists, and otherwise look for module 'a.b' in sys.modules. On Thu, Apr 4, 2013 at 1:28 PM, PJ Eby wrote: > On Thu, Apr 4, 2013 at 11:17 AM, Guido van Rossum wrote: >> I don't really see what we could change to avoid breaking code in any >> particular case > > Actually, the problem has *nothing* to do with circularity per se; > it's that "import a.b" and "from a import b" behave differently in > terms of how they obtain the module 'a.b'... > > And "from a import b" will *always* fail if 'a.b' is part of a cycle > with the current module, whereas "import a.b" will *always* succeed. > > The workaround is to tell people to always use "import a.b" in the > case of circular imports; it's practically a FAQ, at least to me. ;-) > > The problem with "from import" is that it always tries to > getattr(a,'b'), even if 'a.b' is in sys.modules. In contrast, a plain > import will simply fetch a.b from sys.modules first. > > In the case of a normal import, this is no problem, because a.b is set > to sys.modules['a.b'] at the end of the module loading process. But > if the import is circular, then the module is in sys.modules['a.b'], > but *not* yet bound to a.b. So the "from import" fails. > > So, this is actually an implementation quirk that could be fixed in a > (somewhat) straightforward manner: by making "from a import b" succeed > if 'a.b' is in sys.modules, the same way "import a.b" does. It would > require a little bit of discussion to hash out the exact way to do it, > but it could be done. -- --Guido van Rossum (python.org/~guido) From pje at telecommunity.com Thu Apr 4 23:00:01 2013 From: pje at telecommunity.com (PJ Eby) Date: Thu, 4 Apr 2013 17:00:01 -0400 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: On Thu, Apr 4, 2013 at 4:42 PM, Guido van Rossum wrote: > I do think it would be fine if "from a import b" returned the > attribute 'b' of module 'a' if it exists, and otherwise look for > module 'a.b' in sys.modules. Technically, it already does that -- but inside of __import__, not in the IMPORT_FROM opcode. But then *after* doing that check-and-fallback, __import__ doesn't assign a.b, because it assumes the recursive import it called has already done this... which means that when __import__ returns, the IMPORT_FROM opcode tries and fails to do the getattr. This could be fixed in one of two ways. Either: 1. Change importlib._bootstrap._handle_fromlist() to set a.b if it successfully imports 'a.b' (inside its duplicate handling for what IMPORT_FROM does), or 2. Change the IMPORT_FROM opcode to handle the fallback itself While the latter involves a bit of C coding, it has fewer potential side-effects on the import system as a whole, and simply ensures that if "import" would succeed, then so would "from...import" targeting the same module. (There might be other fixes I haven't thought of, but really, changing IMPORT_FROM to fallback to a sys.modules check is probably by far the least-invasive way to handle it.) From tim.delaney at aptare.com Thu Apr 4 22:50:58 2013 From: tim.delaney at aptare.com (Tim Delaney) Date: Fri, 5 Apr 2013 07:50:58 +1100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <515D994A.2000202@stoneleaf.us> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> <515D4A57.7010806@avl.com> <515D994A.2000202@stoneleaf.us> Message-ID: On 5 April 2013 02:16, Ethan Furman wrote: > On 04/04/2013 08:01 AM, Chris Angelico wrote: > >> On Fri, Apr 5, 2013 at 1:59 AM, Guido van Rossum >> wrote: >> >>> On Thu, Apr 4, 2013 at 7:47 AM, Chris Angelico wrote: >>> >>>> Is there any argument that I can pass to Foo() to get back a Bar()? >>>> Would anyone expect there to be one? Sure, I could override __new__ to >>>> do stupid things, but in terms of logical expectations, I'd expect >>>> that Foo(x) will return a Foo object, not a Bar object. Why should int >>>> be any different? What have I missed here? >>>> >>> >>> >>> A class can define a __new__ method that returns a different object. E.g. >>> (python 3): >>> >> >> Right, I'm aware it's possible. But who would expect it of a class? >> > > FTR I'm in the int() should return an int camp, but to answer your > question: my dbf module has a Table class, but it returns either a > Db3Table, FpTable, VfpTable, or ClpTable depending on arguments (if > creating a new one) or the type of the table in the existing dbf file. > I fall into: 1. int(), float(), str() etc should return that exact class (and operator.index() should return exactly an int). 2. It could sometimes be useful for __int__() and __index__() to return a subclass of int. So, for the int constructor, I would have the following logic (assume appropriate try/catch): def __new__(cls, obj): i = obj.__int__() if type(i) is int: return i return i._internal_value Tim Delaney -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Thu Apr 4 23:14:05 2013 From: guido at python.org (Guido van Rossum) Date: Thu, 4 Apr 2013 14:14:05 -0700 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> <515D4A57.7010806@avl.com> <515D994A.2000202@stoneleaf.us> Message-ID: On Thu, Apr 4, 2013 at 1:50 PM, Tim Delaney wrote: > I fall into: > > 1. int(), float(), str() etc should return that exact class (and > operator.index() should return exactly an int). > > 2. It could sometimes be useful for __int__() and __index__() to return a > subclass of int. > > So, for the int constructor, I would have the following logic (assume > appropriate try/catch): > > def __new__(cls, obj): > i = obj.__int__() > > if type(i) is int: > return i > > return i._internal_value CPython can solve this in C using an unsafe cast, and the code that checks for allowable subclasses of int actually ensures such a cast will work. But it still feels wrong; __int__ should be expected to do the work. -- --Guido van Rossum (python.org/~guido) From cs at zip.com.au Thu Apr 4 23:45:36 2013 From: cs at zip.com.au (Cameron Simpson) Date: Fri, 5 Apr 2013 08:45:36 +1100 Subject: [Python-Dev] relative import circular problem In-Reply-To: <515B65E2.3040109@canterbury.ac.nz> References: <515B65E2.3040109@canterbury.ac.nz> Message-ID: <20130404214536.GA90996@cskk.homeip.net> On 03Apr2013 12:12, Greg Ewing wrote: | Kristj?n Valur J?nsson wrote: | >However, relative imports can _only_ be performed using the "from X import Y syntax" | | This seems like a legitimate complaint on its own, [...] | There are a couple of ways that this could be resolved. One | would be to use the name resulting from stripping off the | leading dots, so that | import .foo | would bind the module to the name 'foo'. +0 from me (I'd have been +0.5 but for Guido's explaination about "import X" generally letting you use "X" exactly as phrased in the import). | Another would be | to always require an 'as' clause in this case, so that you | would have to write' | import .foo as foo And a big +1 for this. Cheers, -- Cameron Simpson The Force. It surrounds us; It enfolds us; It gets us dates on Saturday Nights. - Obi Wan Kenobi, Famous Jedi Knight and Party Animal. From brett at python.org Fri Apr 5 00:38:10 2013 From: brett at python.org (Brett Cannon) Date: Thu, 4 Apr 2013 18:38:10 -0400 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: On Thu, Apr 4, 2013 at 5:00 PM, PJ Eby wrote: > On Thu, Apr 4, 2013 at 4:42 PM, Guido van Rossum wrote: > > I do think it would be fine if "from a import b" returned the > > attribute 'b' of module 'a' if it exists, and otherwise look for > > module 'a.b' in sys.modules. > > Technically, it already does that -- but inside of __import__, not in > the IMPORT_FROM opcode. > > But then *after* doing that check-and-fallback, __import__ doesn't > assign a.b, because it assumes the recursive import it called has > already done this... It's an unfortunate side-effect of having loaders set sys.modules for new modules not also set them as an attribute on their parent package immediately as well (or you could argue it's a side-effect of not passing in a module instead of a name to load_module() but that's another discussion). > which means that when __import__ returns, the > IMPORT_FROM opcode tries and fails to do the getattr. > > This could be fixed in one of two ways. Either: > > 1. Change importlib._bootstrap._handle_fromlist() to set a.b if it > successfully imports 'a.b' (inside its duplicate handling for what > IMPORT_FROM does), or > It's three lines, one of which is 'else:'. Just did it. > 2. Change the IMPORT_FROM opcode to handle the fallback itself > While the latter involves a bit of C coding, it has fewer potential > side-effects on the import system as a whole, and simply ensures that > if "import" would succeed, then so would "from...import" targeting the > same module. > > (There might be other fixes I haven't thought of, but really, changing > IMPORT_FROM to fallback to a sys.modules check is probably by far the > least-invasive way to handle it.) > This is my preference as well. The change would be small: I think all you need to do is if the getattr() fails then fall back to sys.modules. Although if it were me and I was casting backwards-compatibility to the wind I would rip out the whole fromlist part of __import__() and let the bytecode worry about the fromlist, basically making the import opcode call importlib.import_module(). -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Fri Apr 5 00:47:25 2013 From: guido at python.org (Guido van Rossum) Date: Thu, 4 Apr 2013 15:47:25 -0700 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: +1 on Brett and PJE just doing this. On Thu, Apr 4, 2013 at 3:38 PM, Brett Cannon wrote: > > > > On Thu, Apr 4, 2013 at 5:00 PM, PJ Eby wrote: >> >> On Thu, Apr 4, 2013 at 4:42 PM, Guido van Rossum wrote: >> > I do think it would be fine if "from a import b" returned the >> > attribute 'b' of module 'a' if it exists, and otherwise look for >> > module 'a.b' in sys.modules. >> >> Technically, it already does that -- but inside of __import__, not in >> the IMPORT_FROM opcode. >> >> But then *after* doing that check-and-fallback, __import__ doesn't >> assign a.b, because it assumes the recursive import it called has >> already done this... > > > It's an unfortunate side-effect of having loaders set sys.modules for new > modules not also set them as an attribute on their parent package > immediately as well (or you could argue it's a side-effect of not passing in > a module instead of a name to load_module() but that's another discussion). > >> >> which means that when __import__ returns, the >> IMPORT_FROM opcode tries and fails to do the getattr. >> >> This could be fixed in one of two ways. Either: >> >> 1. Change importlib._bootstrap._handle_fromlist() to set a.b if it >> successfully imports 'a.b' (inside its duplicate handling for what >> IMPORT_FROM does), or > > > It's three lines, one of which is 'else:'. Just did it. > >> >> 2. Change the IMPORT_FROM opcode to handle the fallback itself >> >> >> While the latter involves a bit of C coding, it has fewer potential >> side-effects on the import system as a whole, and simply ensures that >> if "import" would succeed, then so would "from...import" targeting the >> same module. >> >> (There might be other fixes I haven't thought of, but really, changing >> IMPORT_FROM to fallback to a sys.modules check is probably by far the >> least-invasive way to handle it.) > > > This is my preference as well. The change would be small: I think all you > need to do is if the getattr() fails then fall back to sys.modules. Although > if it were me and I was casting backwards-compatibility to the wind I would > rip out the whole fromlist part of __import__() and let the bytecode worry > about the fromlist, basically making the import opcode call > importlib.import_module(). > -- --Guido van Rossum (python.org/~guido) From brett at python.org Fri Apr 5 02:03:11 2013 From: brett at python.org (Brett Cannon) Date: Thu, 4 Apr 2013 20:03:11 -0400 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: On Apr 4, 2013 6:47 PM, "Guido van Rossum" wrote: > > +1 on Brett and PJE just doing this. I'll file a bug when I get home. -brett > > On Thu, Apr 4, 2013 at 3:38 PM, Brett Cannon wrote: > > > > > > > > On Thu, Apr 4, 2013 at 5:00 PM, PJ Eby wrote: > >> > >> On Thu, Apr 4, 2013 at 4:42 PM, Guido van Rossum wrote: > >> > I do think it would be fine if "from a import b" returned the > >> > attribute 'b' of module 'a' if it exists, and otherwise look for > >> > module 'a.b' in sys.modules. > >> > >> Technically, it already does that -- but inside of __import__, not in > >> the IMPORT_FROM opcode. > >> > >> But then *after* doing that check-and-fallback, __import__ doesn't > >> assign a.b, because it assumes the recursive import it called has > >> already done this... > > > > > > It's an unfortunate side-effect of having loaders set sys.modules for new > > modules not also set them as an attribute on their parent package > > immediately as well (or you could argue it's a side-effect of not passing in > > a module instead of a name to load_module() but that's another discussion). > > > >> > >> which means that when __import__ returns, the > >> IMPORT_FROM opcode tries and fails to do the getattr. > >> > >> This could be fixed in one of two ways. Either: > >> > >> 1. Change importlib._bootstrap._handle_fromlist() to set a.b if it > >> successfully imports 'a.b' (inside its duplicate handling for what > >> IMPORT_FROM does), or > > > > > > It's three lines, one of which is 'else:'. Just did it. > > > >> > >> 2. Change the IMPORT_FROM opcode to handle the fallback itself > >> > >> > >> While the latter involves a bit of C coding, it has fewer potential > >> side-effects on the import system as a whole, and simply ensures that > >> if "import" would succeed, then so would "from...import" targeting the > >> same module. > >> > >> (There might be other fixes I haven't thought of, but really, changing > >> IMPORT_FROM to fallback to a sys.modules check is probably by far the > >> least-invasive way to handle it.) > > > > > > This is my preference as well. The change would be small: I think all you > > need to do is if the getattr() fails then fall back to sys.modules. Although > > if it were me and I was casting backwards-compatibility to the wind I would > > rip out the whole fromlist part of __import__() and let the bytecode worry > > about the fromlist, basically making the import opcode call > > importlib.import_module(). > > > > > > -- > --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Apr 5 01:56:09 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 5 Apr 2013 09:56:09 +1000 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> <515D4A57.7010806@avl.com> Message-ID: On 5 Apr 2013 01:07, "Chris Angelico" wrote: > > On Fri, Apr 5, 2013 at 1:59 AM, Guido van Rossum wrote: > > On Thu, Apr 4, 2013 at 7:47 AM, Chris Angelico wrote: > >> Is there any argument that I can pass to Foo() to get back a Bar()? > >> Would anyone expect there to be one? Sure, I could override __new__ to > >> do stupid things, but in terms of logical expectations, I'd expect > >> that Foo(x) will return a Foo object, not a Bar object. Why should int > >> be any different? What have I missed here? > > > > > > A class can define a __new__ method that returns a different object. E.g. > > (python 3): > > > > Right, I'm aware it's possible. But who would expect it of a class? Python 3.3 does it for OSError to map errno values to the appropriate subclasses. That's mainly to aid migration to the new exception structure, though (see PEP 3151). For a clean slate API design you would use a separate factory function or class method to do the conversion. Cheers, Nick. > > ChrisA > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Fri Apr 5 03:15:15 2013 From: brett at python.org (Brett Cannon) Date: Thu, 4 Apr 2013 21:15:15 -0400 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: http://bugs.python.org/issue17636 On Thu, Apr 4, 2013 at 8:03 PM, Brett Cannon wrote: > > On Apr 4, 2013 6:47 PM, "Guido van Rossum" wrote: > > > > +1 on Brett and PJE just doing this. > > I'll file a bug when I get home. > > -brett > > > > > On Thu, Apr 4, 2013 at 3:38 PM, Brett Cannon wrote: > > > > > > > > > > > > On Thu, Apr 4, 2013 at 5:00 PM, PJ Eby wrote: > > >> > > >> On Thu, Apr 4, 2013 at 4:42 PM, Guido van Rossum > wrote: > > >> > I do think it would be fine if "from a import b" returned the > > >> > attribute 'b' of module 'a' if it exists, and otherwise look for > > >> > module 'a.b' in sys.modules. > > >> > > >> Technically, it already does that -- but inside of __import__, not in > > >> the IMPORT_FROM opcode. > > >> > > >> But then *after* doing that check-and-fallback, __import__ doesn't > > >> assign a.b, because it assumes the recursive import it called has > > >> already done this... > > > > > > > > > It's an unfortunate side-effect of having loaders set sys.modules for > new > > > modules not also set them as an attribute on their parent package > > > immediately as well (or you could argue it's a side-effect of not > passing in > > > a module instead of a name to load_module() but that's another > discussion). > > > > > >> > > >> which means that when __import__ returns, the > > >> IMPORT_FROM opcode tries and fails to do the getattr. > > >> > > >> This could be fixed in one of two ways. Either: > > >> > > >> 1. Change importlib._bootstrap._handle_fromlist() to set a.b if it > > >> successfully imports 'a.b' (inside its duplicate handling for what > > >> IMPORT_FROM does), or > > > > > > > > > It's three lines, one of which is 'else:'. Just did it. > > > > > >> > > >> 2. Change the IMPORT_FROM opcode to handle the fallback itself > > >> > > >> > > >> While the latter involves a bit of C coding, it has fewer potential > > >> side-effects on the import system as a whole, and simply ensures that > > >> if "import" would succeed, then so would "from...import" targeting the > > >> same module. > > >> > > >> (There might be other fixes I haven't thought of, but really, changing > > >> IMPORT_FROM to fallback to a sys.modules check is probably by far the > > >> least-invasive way to handle it.) > > > > > > > > > This is my preference as well. The change would be small: I think all > you > > > need to do is if the getattr() fails then fall back to sys.modules. > Although > > > if it were me and I was casting backwards-compatibility to the wind I > would > > > rip out the whole fromlist part of __import__() and let the bytecode > worry > > > about the fromlist, basically making the import opcode call > > > importlib.import_module(). > > > > > > > > > > > -- > > --Guido van Rossum (python.org/~guido) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Fri Apr 5 04:04:03 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 05 Apr 2013 13:04:03 +1100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> <515D4A57.7010806@avl.com> Message-ID: <515E3113.7020503@pearwood.info> On 05/04/13 01:23, Oscar Benjamin wrote: > The reason for calling int(obj) is to get an object that is precisely > of type int. When I call this I do not want any modified or additional > methods or data attached to the resulting object. When I call int(), I'm expecting an int. That includes well-behaved subclasses of int that continue to behave like ints in all the ways that matter. It's curious to see the (d)evolution of thinking on type checking in Python circles. Once upon a time, type checking was discouraged, duck-typing was encouraged, and the philosophy was that an int is anything that behaves like an int. Now type-checking is tolerated or even encouraged, provided that you use isinstance, and an int is anything that inherits from the builtin (or the ABC). And this proposed change in behaviour continues the move away from Python's former emphasis on duck-typing. Now an int is only the builtin. Oscar, a question: what harm does it do to you if the int you receive has additional methods or data attached? I can appreciate Guido's concern that (say) a subclass might mess with the repr of the number, and he doesn't want that. (I don't find that argument compelling, but it does make sense.) But I don't understand why you would care if the int you receive happens to have an additional method or data that you don't expect. [...] > I think it's fair to > say that there are times when someone wants an object that is > precisely of type int. They should be able to rely on int(obj) > returning an int or raising an error. The first part of that is correct, but I disagree on the second. The 90-10 rule applies here. 90% of the time, we shouldn't care whether we receive an actual builtin int, only that we receive something that quacks like an int. In practice, that generally means something that inherits from int. IMO, int() and __int__ should support the common case, and not enforce the uncommon case > This is true similarly for __index__. I have no argument with the idea that __index__ should guarantee to return a builtin int. I thought it already did make that guarantee. -- Steven From larry at hastings.org Fri Apr 5 06:26:43 2013 From: larry at hastings.org (Larry Hastings) Date: Thu, 04 Apr 2013 21:26:43 -0700 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <515C52FB.2070705@pearwood.info> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <20130403101612.27157a4d@anarchist> <515C52FB.2070705@pearwood.info> Message-ID: <515E5283.6060200@hastings.org> On 04/03/2013 09:04 AM, Steven D'Aprano wrote: > On 04/04/13 01:16, Barry Warsaw wrote: >> It's analogous to all >> the other built-in types-as-functions, so int() calls __int__() which >> must >> return a concrete integer. > > Why must it? I think that's the claim which must be justified, not > just taken > as a given. Principle Of Least Surprise. My observation of the thread is that the majority of people feel "of course __int__ needs to return a real int". People are surprised that you can return subclasses. Nick thought there was an explicit check to prevent it! Why is this so surprising? I think it's because int, str, and float are all classes. Therefore, as callables they are constructors. When you call a constructor, you expect to get an instance of that specific class. Yes, it's always been possible with new-style classes to return a subclass from the constructor. But calling a constructor and getting a subclass is always surprising behavior. Also, permitting subclasses means the interface becomes conceptually far more complicated. We would need to restrict it to subclasses that don't hijack the representation. It's plausible, for example, to write a subclass of str that uses a novel approach for storing the string data, say as a rope. It could overload all the magic methods and behave identically to a str in every important way. But if its __str__ returned self instead of a real str object, that means that PyUnicode_AS_UNICODE would blissfully return the bypassed internal string value, whatever it is, and Python breaks. So we can't enforce it programmatically, we'd need to document it as convention. But explaining that is complicated, and If The Implementation Is Hard To Explain, It's A Bad Idea. > When we call n = int(something), what's the use-case for caring that n > is an > instance of built-in int but not of a subclass, and is that use-case so > compelling that it must be enforced for all uses of int() etc.? I'm much more interested in your counter use-case. When is it appealing or necessary to you to return a subclass of int from __int__()? In your postings so far you've said that this makes sense to you, but you haven't said why you need it. In lieu of a compelling use case, my vote is firmly against surprise and complexity. //arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan_ml at behnel.de Fri Apr 5 08:22:12 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 05 Apr 2013 08:22:12 +0200 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> <515D4A57.7010806@avl.com> <515D994A.2000202@stoneleaf.us> Message-ID: Guido van Rossum, 04.04.2013 23:14: > On Thu, Apr 4, 2013 at 1:50 PM, Tim Delaney wrote: >> I fall into: >> >> 1. int(), float(), str() etc should return that exact class (and >> operator.index() should return exactly an int). >> >> 2. It could sometimes be useful for __int__() and __index__() to return a >> subclass of int. >> >> So, for the int constructor, I would have the following logic (assume >> appropriate try/catch): >> >> def __new__(cls, obj): >> i = obj.__int__() >> >> if type(i) is int: >> return i >> >> return i._internal_value > > CPython can solve this in C using an unsafe cast, and the code that > checks for allowable subclasses of int actually ensures such a cast > will work. But it still feels wrong; __int__ should be expected to do > the work. +1, that's why it's called "__int__" (even if it returns a PyLong in Py3 ;) Stefan From cf.natali at gmail.com Fri Apr 5 08:53:01 2013 From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=) Date: Fri, 5 Apr 2013 08:53:01 +0200 Subject: [Python-Dev] Slides from today's parallel/async Python talk In-Reply-To: <20130404200440.GA12213@snakebite.org> References: <20130314020540.GB22505@snakebite.org> <5141C0B5.6060904@python.org> <20130314182352.GC24307@snakebite.org> <20130314222337.GG24307@snakebite.org> <20130321141725.5a1e9d0d@pitrou.net> <0FE526DD-12B6-4A30-A68C-CBDB390B54C5@snakebite.org> <20130404200440.GA12213@snakebite.org> Message-ID: Hello, >> async.submit_work(func, args, kwds, callback=None, errback=None) >> >> How do you implement arguments passing and return value? >> >> e.g. let's say I pass a list as argument: how do you iterate on the >> list from the worker thread without modifying the backing objects for >> refcounts (IIUC you use a per-thread heap and don't do any >> refcounting). > > Correct, nothing special is done for the arguments (apart from > incref'ing them in the main thread before kicking off the parallel > thread (then decref'ing them in the main thread once we're sure the > parallel thread has finished)). IIUC you incref the argument from the main thread before publishing it to the worker thread: but what about containers like list? How do you make sure the refcounts of the elements don't get deallocated while the worker thread iterates? More generally, how do you deal with non-local objects? BTW I don't know if you did, but you could probably have a look at Go's goroutines and Erlang processes. cf From kristjan at ccpgames.com Fri Apr 5 13:18:08 2013 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Fri, 5 Apr 2013 11:18:08 +0000 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: +1. I was thinking along the same lines. Allowing relative imports in "import module [as X]" statements. If 'module' consists of pure dots, then "as X" is required. Otherwise, if "as X" is not present, strip the leading dot(s) when assigning the local name. K > -----Original Message----- > From: Python-Dev [mailto:python-dev- > bounces+kristjan=ccpgames.com at python.org] On Behalf Of Richard > Oudkerk > Sent: 4. apr?l 2013 16:26 > To: python-dev at python.org > Subject: Re: [Python-Dev] relative import circular problem > > > How about having a form of relative import which only works for > submodules. For instance, instead of > > from . import moduleX > > write > > import .moduleX > > which is currently a SyntaxError. I think this could be implemented as > > moduleX = importlib.import_module('.moduleX', __package__) > > -- > From kristjan at ccpgames.com Fri Apr 5 13:23:50 2013 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Fri, 5 Apr 2013 11:23:50 +0000 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: > -----Original Message----- > From: PJ Eby [mailto:pje at telecommunity.com] > Sent: 4. apr?l 2013 20:29 > To: Guido van Rossum > Cc: Kristj?n Valur J?nsson; Nick Coghlan; python-dev at python.org > Subject: Re: [Python-Dev] relative import circular problem > > So, this is actually an implementation quirk that could be fixed in a > (somewhat) straightforward manner: by making "from a import b" succeed if > 'a.b' is in sys.modules, the same way "import a.b" does. It would require a > little bit of discussion to hash out the exact way to do it, but it could be done. Yes, except that "from a import b" is not only used to import modules. It is pretty much defined to mean "b = getattr(a, 'b')". Consider this module: #foo.py # pull helper.helper from its implementation place from .helper import helper now, "from foo import helper" is expected to get the helper() function, not the helper module, but changing the semantics of the import statement would be "surprising". K From kristjan at ccpgames.com Fri Apr 5 13:26:46 2013 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Fri, 5 Apr 2013 11:26:46 +0000 Subject: [Python-Dev] relative import circular problem In-Reply-To: References: Message-ID: And I should learn to read the entire thread before I start responding. Cheers! K > -----Original Message----- > From: Python-Dev [mailto:python-dev- > bounces+kristjan=ccpgames.com at python.org] On Behalf Of Guido van > Rossum > Sent: 4. apr?l 2013 22:47 > To: Brett Cannon > Cc: PJ Eby; Nick Coghlan; python-dev at python.org > Subject: Re: [Python-Dev] relative import circular problem > > +1 on Brett and PJE just doing this. > From trent at snakebite.org Fri Apr 5 15:12:23 2013 From: trent at snakebite.org (Trent Nelson) Date: Fri, 5 Apr 2013 09:12:23 -0400 Subject: [Python-Dev] Slides from today's parallel/async Python talk In-Reply-To: References: <20130314020540.GB22505@snakebite.org> <5141C0B5.6060904@python.org> <20130314182352.GC24307@snakebite.org> <20130314222337.GG24307@snakebite.org> <20130321141725.5a1e9d0d@pitrou.net> <0FE526DD-12B6-4A30-A68C-CBDB390B54C5@snakebite.org> <20130404200440.GA12213@snakebite.org> Message-ID: <20130405131223.GA13840@snakebite.org> On Thu, Apr 04, 2013 at 11:53:01PM -0700, Charles-Fran?ois Natali wrote: > Hello, > > >> async.submit_work(func, args, kwds, callback=None, errback=None) > >> > >> How do you implement arguments passing and return value? > >> > >> e.g. let's say I pass a list as argument: how do you iterate on the > >> list from the worker thread without modifying the backing objects for > >> refcounts (IIUC you use a per-thread heap and don't do any > >> refcounting). > > > > Correct, nothing special is done for the arguments (apart from > > incref'ing them in the main thread before kicking off the parallel > > thread (then decref'ing them in the main thread once we're sure the > > parallel thread has finished)). > > IIUC you incref the argument from the main thread before publishing it > to the worker thread: but what about containers like list? How do you > make sure the refcounts of the elements don't get deallocated while > the worker thread iterates? Ah, so, all of my examples were missing async.run(). They should have looked like this: async.submit_work(foo) async.submit_work(bar) async.run() async.run() is called from the main thread, with the GIL held, and it blocks until all parallel threads (well, parallel contexts, to be exact) have completed. The parallel 'work' doesn't actually start until async.run() is called either. (That's completely untrue at the moment; async.submit_work(foo) will execute foo() in a parallel thread immediately. Fixing that is on the todo list.) With only parallel threads running, no main-thread objects could ever be deallocated*, as no decref'ing is ever done. [*]: unless you went out of your way to delete/deallocate main thread objects via the @async.call_from_main_thread facility. At the moment, that's firmly in the category of "Don't Do That". (And, thinking about it a little more, I guess I could augment the ceval loop in such a way that in order for the main thread to run things scheduled via @async.call_from_main_thread, all parallel threads need to be suspended. Or I could just freeze/thaw them (although I don't know if there are POSIX counterparts to those Windows methods). That would definitely impede performance, but it would assure data integrity. Perhaps it should be enabled by default, with the option to disable it for consenting adults.) > More generally, how do you deal with non-local objects? Read-only ops against non-local (main-thread) objects from parallel threads are free, which is nice. Things get tricky when you try to mutate main-thread objects from parallel threads. That's where all the context persistence, interlocked data types, object protection etc stuff comes in. Is... that what you mean by how do I deal with non-local objects? I took a guess ;-) Regards, Trent. From status at bugs.python.org Fri Apr 5 18:07:30 2013 From: status at bugs.python.org (Python tracker) Date: Fri, 5 Apr 2013 18:07:30 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20130405160730.5EFE5568EF@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2013-03-29 - 2013-04-05) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3905 (+20) closed 25508 (+47) total 29413 (+67) Open issues with patches: 1729 Issues opened (48) ================== #17425: Update OpenSSL versions in Windows builds http://bugs.python.org/issue17425 reopened by pitrou #17574: pysetup failing with "OSError: [Errno 18] Invalid cross-device http://bugs.python.org/issue17574 opened by vaab #17575: HTTPConnection.send http://bugs.python.org/issue17575 opened by dspublic at freemail.hu #17576: PyNumber_Index() is not int-subclass friendly (or operator.ind http://bugs.python.org/issue17576 opened by barry #17577: Add lookahead/peek wrapper to itertools http://bugs.python.org/issue17577 opened by pconnell #17580: ctypes: ARM hardfloat argument corruption calling functions wi http://bugs.python.org/issue17580 opened by bivab #17582: xml.etree.ElementTree does not preserve whitespaces in attribu http://bugs.python.org/issue17582 opened by piro #17583: IDLE HOWTO http://bugs.python.org/issue17583 opened by Amit.Saha #17585: IDLE - regression with exit() and quit() http://bugs.python.org/issue17585 opened by roger.serwy #17588: runpy cannot run Unicode path on Windows http://bugs.python.org/issue17588 opened by Drekin #17589: Make documentation about macros in C API explicit about rvalu http://bugs.python.org/issue17589 opened by larry #17590: mingw: translate gcc internal defines to python platform speci http://bugs.python.org/issue17590 opened by rpetrov #17592: mingw: configure MACHDEP and platform for build http://bugs.python.org/issue17592 opened by rpetrov #17593: mailbox.py tries to link even on filesystem wihch does not sup http://bugs.python.org/issue17593 opened by dominik-stadler #17594: mingw: preset configure defaults http://bugs.python.org/issue17594 opened by rpetrov #17595: mingw: configure largefile support for windows builds http://bugs.python.org/issue17595 opened by rpetrov #17596: mingw: add wincrypt.h in Python/random.c http://bugs.python.org/issue17596 opened by rpetrov #17597: mingw: add $srcdir/PC to CPPFLAGS http://bugs.python.org/issue17597 opened by rpetrov #17598: mingw: init system calls http://bugs.python.org/issue17598 opened by rpetrov #17599: mingw: detect REPARSE_DATA_BUFFER http://bugs.python.org/issue17599 opened by rpetrov #17600: mingw: build-in windows modules (winreg) http://bugs.python.org/issue17600 opened by rpetrov #17601: mingw: determine if pwdmodule should be used http://bugs.python.org/issue17601 opened by rpetrov #17602: mingw: default sys.path calculations for windows platforms http://bugs.python.org/issue17602 opened by rpetrov #17603: AC_LIBOBJ replacement of fileblocks http://bugs.python.org/issue17603 opened by rpetrov #17604: mingw: use main() to start execution http://bugs.python.org/issue17604 opened by rpetrov #17605: mingw-meta: build interpeter core http://bugs.python.org/issue17605 opened by rpetrov #17606: xml.sax.saxutils.XMLGenerator cannot output with the correct e http://bugs.python.org/issue17606 opened by neoecos #17607: missed peephole optimization (unnecessary jump at end of funct http://bugs.python.org/issue17607 opened by Neal.Norwitz #17611: Move unwinding of stack for "pseudo exceptions" from interpret http://bugs.python.org/issue17611 opened by Mark.Shannon #17612: hooks/mail.py requires [smtp] host to be set, despite a commen http://bugs.python.org/issue17612 opened by eric.smith #17613: IDLE "AttributeError: 'NoneType' object has no attribute 'inde http://bugs.python.org/issue17613 opened by rhettinger #17615: String comparison performance regression http://bugs.python.org/issue17615 opened by Neil.Hodgson #17616: wave.Wave_read and wave.Wave_write can be context managers http://bugs.python.org/issue17616 opened by Claudiu.Popa #17618: base85 encoding http://bugs.python.org/issue17618 opened by pitrou #17620: Python interactive console doesn't use sys.stdin for input http://bugs.python.org/issue17620 opened by Drekin #17621: Create a lazy import loader mixin http://bugs.python.org/issue17621 opened by brett.cannon #17626: set's __isub__ doesn't support non-sets. http://bugs.python.org/issue17626 opened by Roy.Wellington #17627: Datetime and time doesn't update timezone in a running Win app http://bugs.python.org/issue17627 opened by cm #17628: str==str: compare the first and last character before calling http://bugs.python.org/issue17628 opened by haypo #17630: Create a pure Python zipfile importer http://bugs.python.org/issue17630 opened by brett.cannon #17631: inspect getsource does not display full text of lambda http://bugs.python.org/issue17631 opened by Alexey.Spiridonov #17633: zipimport's handling of namespace packages is incorrect http://bugs.python.org/issue17633 opened by pconnell #17634: Win32: shutil.copy leaks file handles to child processes http://bugs.python.org/issue17634 opened by akrpic77 #17635: Doc of multiprocessing.connection mentions answerChallenge ins http://bugs.python.org/issue17635 opened by Antony.Lee #17636: Modify IMPORT_FROM to fallback on sys.modules http://bugs.python.org/issue17636 opened by brett.cannon #17637: Mention "What's New" in devguide's patch guidelines http://bugs.python.org/issue17637 opened by brett.cannon #17638: test_ssl failure http://bugs.python.org/issue17638 opened by neologix #17640: from distutils.util import byte_compile hangs http://bugs.python.org/issue17640 opened by Dmitry.Sivachenko Most recent 15 issues with no replies (15) ========================================== #17640: from distutils.util import byte_compile hangs http://bugs.python.org/issue17640 #17635: Doc of multiprocessing.connection mentions answerChallenge ins http://bugs.python.org/issue17635 #17631: inspect getsource does not display full text of lambda http://bugs.python.org/issue17631 #17621: Create a lazy import loader mixin http://bugs.python.org/issue17621 #17620: Python interactive console doesn't use sys.stdin for input http://bugs.python.org/issue17620 #17618: base85 encoding http://bugs.python.org/issue17618 #17616: wave.Wave_read and wave.Wave_write can be context managers http://bugs.python.org/issue17616 #17604: mingw: use main() to start execution http://bugs.python.org/issue17604 #17603: AC_LIBOBJ replacement of fileblocks http://bugs.python.org/issue17603 #17602: mingw: default sys.path calculations for windows platforms http://bugs.python.org/issue17602 #17601: mingw: determine if pwdmodule should be used http://bugs.python.org/issue17601 #17600: mingw: build-in windows modules (winreg) http://bugs.python.org/issue17600 #17599: mingw: detect REPARSE_DATA_BUFFER http://bugs.python.org/issue17599 #17598: mingw: init system calls http://bugs.python.org/issue17598 #17597: mingw: add $srcdir/PC to CPPFLAGS http://bugs.python.org/issue17597 Most recent 15 issues waiting for review (15) ============================================= #17633: zipimport's handling of namespace packages is incorrect http://bugs.python.org/issue17633 #17628: str==str: compare the first and last character before calling http://bugs.python.org/issue17628 #17616: wave.Wave_read and wave.Wave_write can be context managers http://bugs.python.org/issue17616 #17613: IDLE "AttributeError: 'NoneType' object has no attribute 'inde http://bugs.python.org/issue17613 #17611: Move unwinding of stack for "pseudo exceptions" from interpret http://bugs.python.org/issue17611 #17606: xml.sax.saxutils.XMLGenerator cannot output with the correct e http://bugs.python.org/issue17606 #17604: mingw: use main() to start execution http://bugs.python.org/issue17604 #17603: AC_LIBOBJ replacement of fileblocks http://bugs.python.org/issue17603 #17602: mingw: default sys.path calculations for windows platforms http://bugs.python.org/issue17602 #17601: mingw: determine if pwdmodule should be used http://bugs.python.org/issue17601 #17600: mingw: build-in windows modules (winreg) http://bugs.python.org/issue17600 #17599: mingw: detect REPARSE_DATA_BUFFER http://bugs.python.org/issue17599 #17598: mingw: init system calls http://bugs.python.org/issue17598 #17597: mingw: add $srcdir/PC to CPPFLAGS http://bugs.python.org/issue17597 #17596: mingw: add wincrypt.h in Python/random.c http://bugs.python.org/issue17596 Top 10 most discussed issues (10) ================================= #17583: IDLE HOWTO http://bugs.python.org/issue17583 15 msgs #17613: IDLE "AttributeError: 'NoneType' object has no attribute 'inde http://bugs.python.org/issue17613 15 msgs #6743: Add function compatible with print to pprint module http://bugs.python.org/issue6743 11 msgs #14146: IDLE: source line in editor doesn't highlight when debugging http://bugs.python.org/issue14146 10 msgs #17585: IDLE - regression with exit() and quit() http://bugs.python.org/issue17585 10 msgs #17615: String comparison performance regression http://bugs.python.org/issue17615 10 msgs #17628: str==str: compare the first and last character before calling http://bugs.python.org/issue17628 10 msgs #17546: Document the circumstances where the locals() dict get updated http://bugs.python.org/issue17546 9 msgs #17611: Move unwinding of stack for "pseudo exceptions" from interpret http://bugs.python.org/issue17611 9 msgs #17206: Py_XDECREF() expands its argument multiple times http://bugs.python.org/issue17206 7 msgs Issues closed (47) ================== #1274: doctest fails to run file based tests with 8bit paths http://bugs.python.org/issue1274 closed by terry.reedy #6419: Broken test_kqueue.py on OpenBSD http://bugs.python.org/issue6419 closed by neologix #6649: idlelib/rpc.py missing exit status on exithook http://bugs.python.org/issue6649 closed by roger.serwy #6698: IDLE no longer opens only an edit window when configured to do http://bugs.python.org/issue6698 closed by roger.serwy #6822: Error calling .storlines from ftplib http://bugs.python.org/issue6822 closed by python-dev #8900: IDLE crashes when using keyboard shortcuts to open a file. http://bugs.python.org/issue8900 closed by roger.serwy #8913: Document that datetime.__format__ is datetime.strftime http://bugs.python.org/issue8913 closed by ezio.melotti #10044: small int optimization http://bugs.python.org/issue10044 closed by mark.dickinson #13163: `port` and `host` are confused in `_get_socket http://bugs.python.org/issue13163 closed by r.david.murray #13271: When -h is used with argparse, default values that fail should http://bugs.python.org/issue13271 closed by Joshua.Chia #14135: check for locale changes in test.regrtest http://bugs.python.org/issue14135 closed by brett.cannon #14254: IDLE - shell restart or closing during readline does not exit http://bugs.python.org/issue14254 closed by roger.serwy #15940: Time module: effect of locale http://bugs.python.org/issue15940 closed by terry.reedy #16757: Faster _PyUnicode_FindMaxChar() http://bugs.python.org/issue16757 closed by python-dev #17357: Add missing verbosity message to importlib http://bugs.python.org/issue17357 closed by brett.cannon #17365: Remove Python 2 code from test_print http://bugs.python.org/issue17365 closed by asvetlov #17435: threading.Timer.__init__() should use immutable argument defau http://bugs.python.org/issue17435 closed by r.david.murray #17483: Can not tell urlopen not to check the hostname for https conne http://bugs.python.org/issue17483 closed by pitrou #17492: Increase test coverage for random (up to 99%) http://bugs.python.org/issue17492 closed by r.david.murray #17526: inspect.findsource raises undocumented error for code objects http://bugs.python.org/issue17526 closed by ezio.melotti #17533: test_xpickle fails with "cannot import name precisionbigmemtes http://bugs.python.org/issue17533 closed by python-dev #17539: Use the builtins module in the unittest.mock.patch example http://bugs.python.org/issue17539 closed by ezio.melotti #17540: logging formatter support 'style' key in dictionary config http://bugs.python.org/issue17540 closed by python-dev #17549: Some exceptions not highlighted in exceptions documentation. http://bugs.python.org/issue17549 closed by ezio.melotti #17568: re: Infinite loop with repeated empty alternative http://bugs.python.org/issue17568 closed by ezio.melotti #17572: strptime exception context http://bugs.python.org/issue17572 closed by ezio.melotti #17578: Update devguide for 3.2 in security-fix-only mode http://bugs.python.org/issue17578 closed by ned.deily #17579: socket module in 2.7.4 raises error instead of gaierror in 2.7 http://bugs.python.org/issue17579 closed by barry #17581: _ssl fails building on OS X Tiger http://bugs.python.org/issue17581 closed by pitrou #17584: collections.Iterator __subclasshook__ does not check if next() http://bugs.python.org/issue17584 closed by rhettinger #17586: fix typo in contextlib.rst http://bugs.python.org/issue17586 closed by ned.deily #17587: Have all core library modules imported by default http://bugs.python.org/issue17587 closed by r.david.murray #17591: mingw: use header in lowercase http://bugs.python.org/issue17591 closed by pitrou #17608: configparser not honouring section but not variable case http://bugs.python.org/issue17608 closed by lukasz.langa #17609: IDLE: Remove config option 'editor on startup' and utilize com http://bugs.python.org/issue17609 closed by Todd.Rovito #17610: Qsort function misuse in typeobject.c http://bugs.python.org/issue17610 closed by python-dev #17614: IDLE - quickly closing a large file triggers a traceback http://bugs.python.org/issue17614 closed by roger.serwy #17617: struct.calcsize() incorrect http://bugs.python.org/issue17617 closed by mark.dickinson #17619: MS WINDOWS: input() swallows KeyboardInterrupt in Python 3.3 http://bugs.python.org/issue17619 closed by sbt #17622: Python sets wrong pid in window properties http://bugs.python.org/issue17622 closed by r.david.murray #17623: Typo in Doc/whatsnew/3.3.rst http://bugs.python.org/issue17623 closed by r.david.murray #17624: Confusing TypeError in urllib.urlopen http://bugs.python.org/issue17624 closed by r.david.murray #17625: IDLE regression -- Search and Replace Window doesn't automatic http://bugs.python.org/issue17625 closed by python-dev #17629: Expose string width to Python http://bugs.python.org/issue17629 closed by benjamin.peterson #17632: IDLE: Add clear-window option to the Shell menu. http://bugs.python.org/issue17632 closed by roger.serwy #17639: symlinking .py files creates unexpected sys.path http://bugs.python.org/issue17639 closed by gvanrossum #1387483: sys.path[0] when executed thru a symbolic link http://bugs.python.org/issue1387483 closed by kristjan.jonsson From g.rodola at gmail.com Fri Apr 5 19:15:45 2013 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Fri, 5 Apr 2013 19:15:45 +0200 Subject: [Python-Dev] Is file.readlines(sizehint=N) broken on 2.7? Message-ID: with open('test-xxx', 'w') as f: f.write('aaa\nbbb\nccc') with open('test-xxx', 'r') as f: print(f.readlines(1)) On Python 3.3 I get: ['aaa\n'] ...while on Python 2.7: ['aaa\n', 'bbb\n', 'ccc'] Is this a bug or I'm missing something? --- Giampaolo https://code.google.com/p/pyftpdlib/ https://code.google.com/p/psutil/ https://code.google.com/p/pysendfile/ From songofacandy at gmail.com Fri Apr 5 19:27:07 2013 From: songofacandy at gmail.com (INADA Naoki) Date: Sat, 6 Apr 2013 02:27:07 +0900 Subject: [Python-Dev] Is file.readlines(sizehint=N) broken on 2.7? In-Reply-To: References: Message-ID: The builtin open() was replaced with io.open(). It's difference between file.readlines() and io.IOBase.readlines(). On Sat, Apr 6, 2013 at 2:15 AM, Giampaolo Rodol? wrote: > with open('test-xxx', 'w') as f: > f.write('aaa\nbbb\nccc') > with open('test-xxx', 'r') as f: > print(f.readlines(1)) > > On Python 3.3 I get: > > ['aaa\n'] > > ...while on Python 2.7: > > ['aaa\n', 'bbb\n', 'ccc'] > > > Is this a bug or I'm missing something? > > > --- Giampaolo > https://code.google.com/p/pyftpdlib/ > https://code.google.com/p/psutil/ > https://code.google.com/p/pysendfile/ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/songofacandy%40gmail.com > -- INADA Naoki -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.rodola at gmail.com Fri Apr 5 20:24:43 2013 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Fri, 5 Apr 2013 20:24:43 +0200 Subject: [Python-Dev] Is file.readlines(sizehint=N) broken on 2.7? In-Reply-To: References: Message-ID: 2013/4/5 INADA Naoki : > The builtin open() was replaced with io.open(). > It's difference between file.readlines() and io.IOBase.readlines(). Should that justify this difference in behavior? Apparently on 2.X sizehint does not have any effect as far as I can see. --- Giampaolo https://code.google.com/p/pyftpdlib/ https://code.google.com/p/psutil/ https://code.google.com/p/pysendfile/ From guido at python.org Fri Apr 5 20:50:12 2013 From: guido at python.org (Guido van Rossum) Date: Fri, 5 Apr 2013 11:50:12 -0700 Subject: [Python-Dev] Is file.readlines(sizehint=N) broken on 2.7? In-Reply-To: References: Message-ID: It's a *hint*, remember? :-) It does work, it's just rounded up to a fairly large number in some implementations. On Fri, Apr 5, 2013 at 11:24 AM, Giampaolo Rodol? wrote: > 2013/4/5 INADA Naoki : >> The builtin open() was replaced with io.open(). >> It's difference between file.readlines() and io.IOBase.readlines(). > > Should that justify this difference in behavior? > Apparently on 2.X sizehint does not have any effect as far as I can see. > > --- Giampaolo > https://code.google.com/p/pyftpdlib/ > https://code.google.com/p/psutil/ > https://code.google.com/p/pysendfile/ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) From rdmurray at bitdance.com Fri Apr 5 21:10:36 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 05 Apr 2013 15:10:36 -0400 Subject: [Python-Dev] Is file.readlines(sizehint=N) broken on 2.7? In-Reply-To: References: Message-ID: <20130405191036.BA1FD250BD1@webabinitio.net> On Fri, 05 Apr 2013 20:24:43 +0200, =?ISO-8859-1?Q?Giampaolo_Rodol=E0?= wrote: > 2013/4/5 INADA Naoki : > > The builtin open() was replaced with io.open(). > > It's difference between file.readlines() and io.IOBase.readlines(). > > Should that justify this difference in behavior? Yes. The 'file' documentation for readlines says documents that the argument is advisory and may be rounded up to an internal buffer size. io's readlines, on the other hand, says no more lines will be read if the total size of bytes read so far exceeds the hint. So, different semantics. > Apparently on 2.X sizehint does not have any effect as far as I can see. Have you played with large enough hints/small enough buffers? --David From g.rodola at gmail.com Fri Apr 5 21:31:23 2013 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Fri, 5 Apr 2013 21:31:23 +0200 Subject: [Python-Dev] Is file.readlines(sizehint=N) broken on 2.7? In-Reply-To: <20130405191036.BA1FD250BD1@webabinitio.net> References: <20130405191036.BA1FD250BD1@webabinitio.net> Message-ID: 2013/4/5 R. David Murray : > On Fri, 05 Apr 2013 20:24:43 +0200, =?ISO-8859-1?Q?Giampaolo_Rodol=E0?= wrote: >> 2013/4/5 INADA Naoki : >> > The builtin open() was replaced with io.open(). >> > It's difference between file.readlines() and io.IOBase.readlines(). >> >> Should that justify this difference in behavior? > > Yes. The 'file' documentation for readlines says documents that the > argument is advisory and may be rounded up to an internal buffer size. > > io's readlines, on the other hand, says no more lines will be read if > the total size of bytes read so far exceeds the hint. > > So, different semantics. I see. >> Apparently on 2.X sizehint does not have any effect as far as I can see. > > Have you played with large enough hints/small enough buffers? Right, it seems it tends to change around 8196 bytes. Ok then, nevermind. --- Giampaolo https://code.google.com/p/pyftpdlib/ https://code.google.com/p/psutil/ https://code.google.com/p/pysendfile/ From eliben at gmail.com Sat Apr 6 05:41:30 2013 From: eliben at gmail.com (Eli Bendersky) Date: Fri, 5 Apr 2013 20:41:30 -0700 Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default): Fix typo In-Reply-To: <3ZhvcM0Hp2zRmp@mail.python.org> References: <3ZhvcM0Hp2zRmp@mail.python.org> Message-ID: On Fri, Apr 5, 2013 at 1:40 AM, andrew.svetlov wrote: > http://hg.python.org/cpython/rev/6cf485ffd325 > changeset: 83110:6cf485ffd325 > parent: 83106:94fb906e5899 > parent: 83109:9610ede72ed2 > user: Andrew Svetlov > date: Fri Apr 05 11:40:01 2013 +0300 > summary: > Fix typo > > files: > Doc/howto/argparse.rst | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > > diff --git a/Doc/howto/argparse.rst b/Doc/howto/argparse.rst > --- a/Doc/howto/argparse.rst > +++ b/Doc/howto/argparse.rst > @@ -668,7 +668,7 @@ > So far, we have been working with two methods of an > :class:`argparse.ArgumentParser` instance. Let's introduce a third one, > :meth:`add_mutually_exclusive_group`. It allows for us to specify options > that > -conflict with each other. Let's also change the rest of the program make > the > +conflict with each other. Let's also change the rest of the program to > make the > new functionality makes more sense: > Andrew, while you're at it you can also fix the rest of the sentence, which makes no sense ;-) Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew.svetlov at gmail.com Sat Apr 6 16:25:20 2013 From: andrew.svetlov at gmail.com (Andrew Svetlov) Date: Sat, 6 Apr 2013 17:25:20 +0300 Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default): Fix typo In-Reply-To: References: <3ZhvcM0Hp2zRmp@mail.python.org> Message-ID: Do you mean something like: ?Let's also change the rest of the program to make the new functionality:? ??? On Sat, Apr 6, 2013 at 6:41 AM, Eli Bendersky wrote: > > On Fri, Apr 5, 2013 at 1:40 AM, andrew.svetlov > wrote: >> >> http://hg.python.org/cpython/rev/6cf485ffd325 >> changeset: 83110:6cf485ffd325 >> parent: 83106:94fb906e5899 >> parent: 83109:9610ede72ed2 >> user: Andrew Svetlov >> date: Fri Apr 05 11:40:01 2013 +0300 >> summary: >> Fix typo >> >> files: >> Doc/howto/argparse.rst | 2 +- >> 1 files changed, 1 insertions(+), 1 deletions(-) >> >> >> diff --git a/Doc/howto/argparse.rst b/Doc/howto/argparse.rst >> --- a/Doc/howto/argparse.rst >> +++ b/Doc/howto/argparse.rst >> @@ -668,7 +668,7 @@ >> So far, we have been working with two methods of an >> :class:`argparse.ArgumentParser` instance. Let's introduce a third one, >> :meth:`add_mutually_exclusive_group`. It allows for us to specify options >> that >> -conflict with each other. Let's also change the rest of the program make >> the >> +conflict with each other. Let's also change the rest of the program to >> make the >> new functionality makes more sense: > > > > Andrew, while you're at it you can also fix the rest of the sentence, which > makes no sense ;-) > > Eli > -- Thanks, Andrew Svetlov From eliben at gmail.com Sat Apr 6 16:41:20 2013 From: eliben at gmail.com (Eli Bendersky) Date: Sat, 6 Apr 2013 07:41:20 -0700 Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default): Fix typo In-Reply-To: References: <3ZhvcM0Hp2zRmp@mail.python.org> Message-ID: It currently says: "Let's also change the rest of the program to make the new functionality makes more sense" This can be changed to: "Let's also change the rest of the program so that the new functionality makes more sense". Eli On Sat, Apr 6, 2013 at 7:25 AM, Andrew Svetlov wrote: > Do you mean something like: > ?Let's also change the rest of the program to make the new functionality:? > ??? > > > On Sat, Apr 6, 2013 at 6:41 AM, Eli Bendersky wrote: > > > > On Fri, Apr 5, 2013 at 1:40 AM, andrew.svetlov < > python-checkins at python.org> > > wrote: > >> > >> http://hg.python.org/cpython/rev/6cf485ffd325 > >> changeset: 83110:6cf485ffd325 > >> parent: 83106:94fb906e5899 > >> parent: 83109:9610ede72ed2 > >> user: Andrew Svetlov > >> date: Fri Apr 05 11:40:01 2013 +0300 > >> summary: > >> Fix typo > >> > >> files: > >> Doc/howto/argparse.rst | 2 +- > >> 1 files changed, 1 insertions(+), 1 deletions(-) > >> > >> > >> diff --git a/Doc/howto/argparse.rst b/Doc/howto/argparse.rst > >> --- a/Doc/howto/argparse.rst > >> +++ b/Doc/howto/argparse.rst > >> @@ -668,7 +668,7 @@ > >> So far, we have been working with two methods of an > >> :class:`argparse.ArgumentParser` instance. Let's introduce a third one, > >> :meth:`add_mutually_exclusive_group`. It allows for us to specify > options > >> that > >> -conflict with each other. Let's also change the rest of the program > make > >> the > >> +conflict with each other. Let's also change the rest of the program to > >> make the > >> new functionality makes more sense: > > > > > > > > Andrew, while you're at it you can also fix the rest of the sentence, > which > > makes no sense ;-) > > > > Eli > > > > > > -- > Thanks, > Andrew Svetlov > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Sat Apr 6 16:58:25 2013 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 6 Apr 2013 10:58:25 -0400 Subject: [Python-Dev] give IDLE its own NEWS section? Message-ID: I noticed IDLE changes were being put under the "library" section in Misc/NEWS. How about creating a IDLE section? -- Regards, Benjamin From andrew.svetlov at gmail.com Sat Apr 6 17:56:04 2013 From: andrew.svetlov at gmail.com (Andrew Svetlov) Date: Sat, 6 Apr 2013 18:56:04 +0300 Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default): Fix typo In-Reply-To: References: <3ZhvcM0Hp2zRmp@mail.python.org> Message-ID: Done, thanks On Sat, Apr 6, 2013 at 5:41 PM, Eli Bendersky wrote: > It currently says: > > "Let's also change the rest of the program to make the new functionality > makes more sense" > > This can be changed to: > > "Let's also change the rest of the program so that the new functionality > makes more sense". > > Eli > > > > > On Sat, Apr 6, 2013 at 7:25 AM, Andrew Svetlov > wrote: >> >> Do you mean something like: >> ?Let's also change the rest of the program to make the new functionality:? >> ??? >> > > > > >> >> On Sat, Apr 6, 2013 at 6:41 AM, Eli Bendersky wrote: >> > >> > On Fri, Apr 5, 2013 at 1:40 AM, andrew.svetlov >> > >> > wrote: >> >> >> >> http://hg.python.org/cpython/rev/6cf485ffd325 >> >> changeset: 83110:6cf485ffd325 >> >> parent: 83106:94fb906e5899 >> >> parent: 83109:9610ede72ed2 >> >> user: Andrew Svetlov >> >> date: Fri Apr 05 11:40:01 2013 +0300 >> >> summary: >> >> Fix typo >> >> >> >> files: >> >> Doc/howto/argparse.rst | 2 +- >> >> 1 files changed, 1 insertions(+), 1 deletions(-) >> >> >> >> >> >> diff --git a/Doc/howto/argparse.rst b/Doc/howto/argparse.rst >> >> --- a/Doc/howto/argparse.rst >> >> +++ b/Doc/howto/argparse.rst >> >> @@ -668,7 +668,7 @@ >> >> So far, we have been working with two methods of an >> >> :class:`argparse.ArgumentParser` instance. Let's introduce a third >> >> one, >> >> :meth:`add_mutually_exclusive_group`. It allows for us to specify >> >> options >> >> that >> >> -conflict with each other. Let's also change the rest of the program >> >> make >> >> the >> >> +conflict with each other. Let's also change the rest of the program to >> >> make the >> >> new functionality makes more sense: >> > >> > >> > >> > Andrew, while you're at it you can also fix the rest of the sentence, >> > which >> > makes no sense ;-) >> > >> > Eli >> > >> >> >> >> -- >> Thanks, >> Andrew Svetlov > > -- Thanks, Andrew Svetlov From g.brandl at gmx.net Sat Apr 6 20:53:38 2013 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 06 Apr 2013 20:53:38 +0200 Subject: [Python-Dev] give IDLE its own NEWS section? In-Reply-To: References: Message-ID: Am 06.04.2013 16:58, schrieb Benjamin Peterson: > I noticed IDLE changes were being put under the "library" section in > Misc/NEWS. How about creating a IDLE section? There is also Lib/idlelib/NEWS.txt. It also contains IDLE news items and is shown in a dialog box from within IDLE. IMO it's fine for changelog entries to go there exclusively. Georg From tjreedy at udel.edu Fri Apr 5 19:34:10 2013 From: tjreedy at udel.edu (Terry Jan Reedy) Date: Fri, 05 Apr 2013 13:34:10 -0400 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: <515E3113.7020503@pearwood.info> References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> <515D4A57.7010806@avl.com> <515E3113.7020503@pearwood.info> Message-ID: On 4/4/2013 10:04 PM, Steven D'Aprano wrote: > When I call int(), I'm expecting an int. We agree so far..., > That includes well-behaved subclasses of int that continue to behave > like ints in all the ways that matter. but not here. I currently expect an actual int instance for 3 reasons. 1. As I read the doc, that is the currently documented 'should'. 2. I believe class constructors should, generally, return an instance of the class, in the narrow sense, and that factory functions should, generally, be used to return instances of multiple classes. The multiple classes would typically, or at least often, all be subclasses of some baseclass. 3. Most apropos to your next paragraph: *because* Python is duck-typed, I would not replace a function arg that might be an int subclass with int(arg) unless (I thought) the difference would make a difference. Lets consider cases: 1. int(non-scalar-number): this is usually an error, except for bytes or unicode strings that represents a number in standard base-[2-32] notation. int has builtin knowledge of these two builtin classes. One can add an __int__ method to string subclasses that represent integers with non-standard notation. A common use is int(input(prompt)) or int(other-external-input). 2. int(rational): for floats, Fractions, and Decimals, this returns the integral part, truncating toward 0. Decimal and float have __int__ methods. Fractions, to my surprise, does not, so int must use __floor__ or __round__ as a backup. I believe we have no disagreement that int() should return an int for these cases. Here is a possible use for input checking. def fib(n): "return fibonnaci(integral input); return type == input type" if int(n) != n or n < 0: raise TypeError('fib input must be a count') # let int() or < exception propagate # the input check is needed to prevent infinite looping return fib-of-n Because of duck-typing, there is no need to replace n with int(n). The return type will be the input type. 3. int(int-subclass-instance): If the int subclass instances behave like an int in all ways that matter in the context, there is no reason to specifically do. In other words, this use should be very rare, such as wanting the int repr. I am also not sure, without checking the doc for the definition of the bit operations, if all int subclasses would dependably substitute in bit manipulations. So unless you can give a good reason otherwise, I think the subclass .__int__ class should assume that the programmer might actually specifically want an int instance. In the example above, int() is part of a general case 2 input check that non-negative subclass instances should pass whether they return themselves or an int. > It's curious to see the (d)evolution of thinking on type checking in > Python circles. Once upon a time, type checking was discouraged, > duck-typing was encouraged, and the philosophy was that an int is > anything that behaves like an int. I always differentiated 'int' as a type/class int object from 'integer' as anything that behaves like an 'int'. For me, and the function outlined above, that includes integer-values rationals along with int subclass instances. > Now type-checking is tolerated or > even encouraged, provided that you use isinstance, I seem to have missed the encouragement. > and an int is anything that inherits from the builtin (or the ABC). > And this proposed change in behaviour To conform to how come read the doc.. > continues the move away from Python's former emphasis on duck-typing. For the reason explained above, I do not see this issue in such apocalyptic terms ;-) -- Terry Jan Reedy From benjamin at python.org Sat Apr 6 21:20:33 2013 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 6 Apr 2013 15:20:33 -0400 Subject: [Python-Dev] give IDLE its own NEWS section? In-Reply-To: References: Message-ID: Having gotten positive input from Todd, I've now done the move. 2013/4/6 Benjamin Peterson : > I noticed IDLE changes were being put under the "library" section in > Misc/NEWS. How about creating a IDLE section? > > -- > Regards, > Benjamin -- Regards, Benjamin From storchaka at gmail.com Sat Apr 6 21:20:53 2013 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sat, 06 Apr 2013 22:20:53 +0300 Subject: [Python-Dev] give IDLE its own NEWS section? In-Reply-To: References: Message-ID: On 06.04.13 17:58, Benjamin Peterson wrote: > I noticed IDLE changes were being put under the "library" section in > Misc/NEWS. How about creating a IDLE section? http://bugs.python.org/issue17221 http://bugs.python.org/issue17506 From benjamin at python.org Sat Apr 6 22:04:38 2013 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 6 Apr 2013 16:04:38 -0400 Subject: [Python-Dev] [RELEASE] Python 2.7.4 Message-ID: I'm thrilled to announce the release of Python 2.7.4. 2.7.4 is the latest maintenance release in the Python 2.7 series. It includes hundreds of bugfixes to the core language and standard library. Downloads are at http://python.org/download/releases/2.7.4/ As always, please report bugs to http://bugs.python.org/ Several regressions found in the release candidate have been fixed. Many thanks to those who tested the preview release. There has recently been a lot of discussion about XML-based denial of service attacks. Specifically, certain XML files can cause XML parsers, including ones in the Python stdlib, to consume gigabytes of RAM and swamp the CPU. 2.7.4 does not include any changes in Python XML code to address these issues. Interested parties should examine the defusedxml package on PyPI: https://pypi.python.org/pypi/defusedxml This is a production release. Best wishes, Benjamin Peterson 2.7 Release Manager (on behalf of all of Python 2.7's contributors) From dickinsm at gmail.com Sat Apr 6 22:34:19 2013 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 6 Apr 2013 21:34:19 +0100 Subject: [Python-Dev] Semantics of __int__(), __index__() In-Reply-To: References: <51583A56.8070404@hotpy.org> <515A97D1.9040506@hotpy.org> <20130402110120.1a1f6c62@pitrou.net> <515C24E3.5010703@avl.com> <515D4A57.7010806@avl.com> <515E3113.7020503@pearwood.info> Message-ID: On Fri, Apr 5, 2013 at 6:34 PM, Terry Jan Reedy wrote: > 2. int(rational): for floats, Fractions, and Decimals, this returns the > integral part, truncating toward 0. Decimal and float have __int__ methods. > Fractions, to my surprise, does not, so int must use __floor__ or __round__ > as a backup. > It uses __trunc__, which is supposed to be the unambiguous "Yes I really want to throw away the fractional part and risk losing information" replacement for __int__. int() will try __int__ first, and then __trunc__, as per PEP 3141. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From georg at python.org Sat Apr 6 22:43:11 2013 From: georg at python.org (Georg Brandl) Date: Sat, 06 Apr 2013 22:43:11 +0200 Subject: [Python-Dev] [RELEASED] Python 3.2.4 and Python 3.3.1 Message-ID: <516088DF.5050303@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On behalf of the Python development team, I am pleased to announce the final releases of Python 3.2.4 and 3.3.1. Python 3.2.4 is the final regular maintenance release for the Python 3.2 series, while Python 3.3.1 is the first maintenance release for the 3.3 series. Both releases include hundreds of bugfixes. There has recently been a lot of discussion about XML-based denial of service attacks. Specifically, certain XML files can cause XML parsers, including ones in the Python stdlib, to consume gigabytes of RAM and swamp the CPU. These releases do not include any changes in Python XML code to address these issues. Interested parties should examine the defusedxml package on PyPI: https://pypi.python.org/pypi/defusedxml To download Python 3.2.4 or Python 3.3.1, visit: http://www.python.org/download/releases/3.2.4/ or http://www.python.org/download/releases/3.3.1/ respectively. As always, please report bugs to http://bugs.python.org/ Enjoy! - -- Georg Brandl, Release Manager georg at python.org (on behalf of the entire python-dev team and all contributors) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) iEYEARECAAYFAlFgiN8ACgkQN9GcIYhpnLAXxQCdHAd2lECpYfmYM4Wbd3I01es4 898AoKBDvHtgecD/PeVRKUrdQRSWGPJg =K8RQ -----END PGP SIGNATURE----- From benjamin at python.org Sat Apr 6 23:02:17 2013 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 6 Apr 2013 17:02:17 -0400 Subject: [Python-Dev] The end of 2.7 Message-ID: Per my last message, 2.7.4 has at long last been released. I apologize for the long interval between 2.7.3 and 2.7.4. To create more determinism in the future, I will be soon updating PEP 373 with approximate dates of future 2.7 bugfix releases. I will be aiming for 6 month intervals. This means we need to talk about how many more 2.7 releases there are going to be. At the release of 2.7.0, I thought we promised 5 years of bugfix maintenance, but my memory may be fuddled. At any rate, 2.7.0 was released in July 2010, which currently puts us within a few months of 3 years of maintenance. Over the past year, I've been happy to see a lot of movement towards 3 including the porting of important codebases like Twisted and Django. However, there's also no doubt that 2.x is still widely used. Obviously, there will be people who would be happy if we kept maintaining 2.7 until 2025, but I think at this juncture 5 total years of maintenance is reasonable. This means there will be approximately 4 more 2.7 releases. Thoughts? Regards, Benjamin From g.brandl at gmx.net Sat Apr 6 23:11:41 2013 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 06 Apr 2013 23:11:41 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: Message-ID: Am 06.04.2013 23:02, schrieb Benjamin Peterson: > Per my last message, 2.7.4 has at long last been released. I apologize > for the long interval between 2.7.3 and 2.7.4. To create more > determinism in the future, I will be soon updating PEP 373 with > approximate dates of future 2.7 bugfix releases. I will be aiming for > 6 month intervals. > > This means we need to talk about how many more 2.7 releases there are > going to be. At the release of 2.7.0, I thought we promised 5 years of > bugfix maintenance, but my memory may be fuddled. At any rate, 2.7.0 > was released in July 2010, which currently puts us within a few months > of 3 years of maintenance. Over the past year, I've been happy to see > a lot of movement towards 3 including the porting of important > codebases like Twisted and Django. However, there's also no doubt that > 2.x is still widely used. Obviously, there will be people who would be > happy if we kept maintaining 2.7 until 2025, but I think at this > juncture 5 total years of maintenance is reasonable. This means there > will be approximately 4 more 2.7 releases. > > Thoughts? I agree that keeping to 5 years of official maintenance releases is reasonable at present. However, in 2015 I can well imagine offers from group(s) in the community to maintain the 2.7 branch with fixes ported from 3.x. At that point, we will have to decide how to treat releases from this "backports" branch. Georg From solipsis at pitrou.net Sat Apr 6 23:41:06 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 6 Apr 2013 23:41:06 +0200 Subject: [Python-Dev] The end of 2.7 References: Message-ID: <20130406234106.0a102746@pitrou.net> On Sat, 6 Apr 2013 17:02:17 -0400 Benjamin Peterson wrote: > Obviously, there will be people who would be > happy if we kept maintaining 2.7 until 2025, but I think at this > juncture 5 total years of maintenance is reasonable. This means there > will be approximately 4 more 2.7 releases. > > Thoughts? That's quite fine with me. Regards Antoine. From christian at python.org Sun Apr 7 00:05:01 2013 From: christian at python.org (Christian Heimes) Date: Sun, 07 Apr 2013 00:05:01 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: Message-ID: <51609C0D.8000106@python.org> Am 06.04.2013 23:11, schrieb Georg Brandl: > Am 06.04.2013 23:02, schrieb Benjamin Peterson: >> Per my last message, 2.7.4 has at long last been released. I apologize >> for the long interval between 2.7.3 and 2.7.4. To create more >> determinism in the future, I will be soon updating PEP 373 with >> approximate dates of future 2.7 bugfix releases. I will be aiming for >> 6 month intervals. >> >> This means we need to talk about how many more 2.7 releases there are >> going to be. At the release of 2.7.0, I thought we promised 5 years of >> bugfix maintenance, but my memory may be fuddled. At any rate, 2.7.0 >> was released in July 2010, which currently puts us within a few months >> of 3 years of maintenance. Over the past year, I've been happy to see >> a lot of movement towards 3 including the porting of important >> codebases like Twisted and Django. However, there's also no doubt that >> 2.x is still widely used. Obviously, there will be people who would be >> happy if we kept maintaining 2.7 until 2025, but I think at this >> juncture 5 total years of maintenance is reasonable. This means there >> will be approximately 4 more 2.7 releases. >> >> Thoughts? > > I agree that keeping to 5 years of official maintenance releases is > reasonable at present. > > However, in 2015 I can well imagine offers from group(s) in the community > to maintain the 2.7 branch with fixes ported from 3.x. At that point, > we will have to decide how to treat releases from this "backports" branch. Five years official releases sounds fine to me, too. Martin, how long are you going to build official Windows binaries for Python 2.7? Christian From greg at krypto.org Sun Apr 7 00:21:10 2013 From: greg at krypto.org (Gregory P. Smith) Date: Sat, 6 Apr 2013 15:21:10 -0700 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <51609C0D.8000106@python.org> References: <51609C0D.8000106@python.org> Message-ID: I agree with Benjamin though is it really necessary to do two 2.7 releases a year for the last two years? that's rather rapid (but as the release manager its your call). A few of us (sorry I forgot who all was there though I think Martin was?) had a discussion at PyCon a few weeks ago and seemed to think that a state of affairs where a 2.7.5 release one year-ish from now would be fine as the last _binary_ release but that continuing to make a 2.7.6 and beyond as source only releases was reasonable. Regardless, the 5 years of 2.7 supported releases plan still makes sense regardless of release binaries being available for windows and mac or not. -gps On Sat, Apr 6, 2013 at 3:05 PM, Christian Heimes wrote: > Am 06.04.2013 23:11, schrieb Georg Brandl: > > Am 06.04.2013 23:02, schrieb Benjamin Peterson: > >> Per my last message, 2.7.4 has at long last been released. I apologize > >> for the long interval between 2.7.3 and 2.7.4. To create more > >> determinism in the future, I will be soon updating PEP 373 with > >> approximate dates of future 2.7 bugfix releases. I will be aiming for > >> 6 month intervals. > >> > >> This means we need to talk about how many more 2.7 releases there are > >> going to be. At the release of 2.7.0, I thought we promised 5 years of > >> bugfix maintenance, but my memory may be fuddled. At any rate, 2.7.0 > >> was released in July 2010, which currently puts us within a few months > >> of 3 years of maintenance. Over the past year, I've been happy to see > >> a lot of movement towards 3 including the porting of important > >> codebases like Twisted and Django. However, there's also no doubt that > >> 2.x is still widely used. Obviously, there will be people who would be > >> happy if we kept maintaining 2.7 until 2025, but I think at this > >> juncture 5 total years of maintenance is reasonable. This means there > >> will be approximately 4 more 2.7 releases. > >> > >> Thoughts? > > > > I agree that keeping to 5 years of official maintenance releases is > > reasonable at present. > > > > However, in 2015 I can well imagine offers from group(s) in the community > > to maintain the 2.7 branch with fixes ported from 3.x. At that point, > > we will have to decide how to treat releases from this "backports" > branch. > > Five years official releases sounds fine to me, too. > > Martin, how long are you going to build official Windows binaries for > Python 2.7? > > Christian > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/greg%40krypto.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Sun Apr 7 00:37:41 2013 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 6 Apr 2013 18:37:41 -0400 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> Message-ID: 2013/4/6 Gregory P. Smith : > I agree with Benjamin though is it really necessary to do two 2.7 releases a > year for the last two years? that's rather rapid (but as the release > manager its your call). What I like about 6 months is that its short enough, so we don't have feel bad about not taking a certain change; it can just be pushed to the next no-too-far-away release. A year is quite a while to wait for a fix to be released. It's also a nice timeframe for some distributions (looking at you, Ubuntu). -- Regards, Benjamin From nad at acm.org Sun Apr 7 03:23:29 2013 From: nad at acm.org (Ned Deily) Date: Sat, 06 Apr 2013 18:23:29 -0700 Subject: [Python-Dev] The end of 2.7 References: <51609C0D.8000106@python.org> Message-ID: In article , Benjamin Peterson wrote: > 2013/4/6 Gregory P. Smith : > What I like about 6 months is that its short enough, so we don't have > feel bad about not taking a certain change; it can just be pushed to > the next no-too-far-away release. A year is quite a while to wait for > a fix to be released. It's also a nice timeframe for some > distributions (looking at you, Ubuntu). +1 to both a 6-month release interval and to soon announcing a date for the last maintenance release that is no later than 2015. As Georg points out, though, there undoubtedly will be pushback on that so we should make sure we have a good story for what happens after that date and we start communicating it in plenty of time. An important of that is emphasizing what will be available on Python 3.x by then and figuring out what to do about any important missing pieces, e.g. key third-party components not yet ported. -- Ned Deily, nad at acm.org From raymond.hettinger at gmail.com Sun Apr 7 03:54:12 2013 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Sat, 6 Apr 2013 18:54:12 -0700 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: Message-ID: <7DF6FD0E-D18C-412F-8149-9A2AED35548C@gmail.com> On Apr 6, 2013, at 2:02 PM, Benjamin Peterson wrote: > we need to talk about how many more 2.7 releases there are > going to be. At the release of 2.7.0, I thought we promised 5 years of > bugfix maintenance, but my memory may be fuddled. I don't we need to make any "promises" beyond 5 years, but I also think it is likely the 2.7 will end-up being a long-term maintenance version of Python. At this year's Pycon keynote, I surveyed the crowd (approx 2500 people) and all almost everyone indicated that they had tried out Python 3.x and almost no one was using it in production or writing code for it. That indicates that Python 2.7 will continue to be important for a good while. In addition, the other implementations of Python (Jython, PyPy, GAE, and IronPython) are all at or nearly at Python 2.7. So, continued support will be needed for their users as well. After 2.7.4, I expect that the pace of real bug fixes will slow down, but that we'll continue to improve docs, add docstrings, update IDLE, etc. IMO, it is premature to utter the phrase "the end of 2.7". Better to say, "2.7 is stable and is expected to only have minor updates". Future point releases probably ought to occur "on their own schedule" whenever there are a sufficient number of changes to warrant a release, or an important security fix, or whenever the release managers have time. Raymond ------ PYTHON 2.7 I'm not dead! CART DRIVER 'Ere. He says he's not dead. LARGE MAN Yes he is. PYTHON 2.7 I'm not! CART DRIVER He isn't. LARGE MAN He will be soon. He's very ill. PYTHON 2.7 I'm getting better! LARGE MAN You're not. You'll be stone dead in a few minutes. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Sun Apr 7 04:42:28 2013 From: tjreedy at udel.edu (Terry Jan Reedy) Date: Sat, 06 Apr 2013 22:42:28 -0400 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: Message-ID: On 4/6/2013 5:11 PM, Georg Brandl wrote: > Am 06.04.2013 23:02, schrieb Benjamin Peterson: >> Per my last message, 2.7.4 has at long last been released. I apologize >> for the long interval between 2.7.3 and 2.7.4. To create more >> determinism in the future, I will be soon updating PEP 373 with >> approximate dates of future 2.7 bugfix releases. I will be aiming for >> 6 month intervals. In 6 months, there will be a bunch more IDLE fixes (there are already some that were too late for today's releases), so that will be good from that standpoint. Some people will continue teaching with 2.7 for who knows how long. I expect Idle to be considerably polished within 2 years. >> This means we need to talk about how many more 2.7 releases there are >> going to be. At the release of 2.7.0, I thought we promised 5 years of >> bugfix maintenance, but my memory may be fuddled. At any rate, 2.7.0 >> was released in July 2010, which currently puts us within a few months >> of 3 years of maintenance. Over the past year, I've been happy to see >> a lot of movement towards 3 including the porting of important >> codebases like Twisted and Django. However, there's also no doubt that >> 2.x is still widely used. Obviously, there will be people who would be >> happy if we kept maintaining 2.7 until 2025, but I think at this >> juncture 5 total years of maintenance is reasonable. This means there >> will be approximately 4 more 2.7 releases. >> >> Thoughts? > I agree that keeping to 5 years of official maintenance releases is > reasonable at present. I do not remember if there was any promise of security fixes after 5 years. > However, in 2015 I can well imagine offers from group(s) in the community > to maintain the 2.7 branch with fixes ported from 3.x. I can imagine that. And I can imagine no volunteers ;-). I think that volunteering after the mid-2015 5-year release is too late in a sense. Anybody who thinks they will want to prolong maintenance should start working *now* to test bugfix patches on 2.7 and re-write as necessary and earn core-developer status. I think this should be suggested/publicized now. Unless Benjamin volunteers to continue doing releases, at least one new volunteer needs to learn how to do them, perhaps by working with him on some the the remaining releases he does do. > At that point, we will have to decide how to treat releases from this "backports" branch. If there are at least a couple of people with 2.7 branch push privileges, who understand and agree to follow 'bugfixes only', with due consideration of back-compatibility, then I see no reason for such releases not to be official PSF releases. If some people take up 2.7 after the final 2.7 release and work independently us, then it is out of our hands. (And they will have to call their releases something other than 'Python 2.7.z') -- Terry Jan Reedy From senthil at uthcode.com Sun Apr 7 04:47:50 2013 From: senthil at uthcode.com (Senthil Kumaran) Date: Sat, 6 Apr 2013 19:47:50 -0700 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: Message-ID: On Sat, Apr 6, 2013 at 2:02 PM, Benjamin Peterson wrote: > juncture 5 total years of maintenance is reasonable. This means there > will be approximately 4 more 2.7 releases. That's good. From the subject of the email, I though you were announcing "This is the end of 2.7.x releases". 2 more year with 6 month cycle seem to be a good one. Thank you! Senthil From ncoghlan at gmail.com Sun Apr 7 05:38:40 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 7 Apr 2013 13:38:40 +1000 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: Message-ID: On Sun, Apr 7, 2013 at 7:02 AM, Benjamin Peterson wrote: > Per my last message, 2.7.4 has at long last been released. I apologize > for the long interval between 2.7.3 and 2.7.4. To create more > determinism in the future, I will be soon updating PEP 373 with > approximate dates of future 2.7 bugfix releases. I will be aiming for > 6 month intervals. > > This means we need to talk about how many more 2.7 releases there are > going to be. At the release of 2.7.0, I thought we promised 5 years of > bugfix maintenance, but my memory may be fuddled. At any rate, 2.7.0 > was released in July 2010, which currently puts us within a few months > of 3 years of maintenance. Over the past year, I've been happy to see > a lot of movement towards 3 including the porting of important > codebases like Twisted and Django. However, there's also no doubt that > 2.x is still widely used. Obviously, there will be people who would be > happy if we kept maintaining 2.7 until 2025, but I think at this > juncture 5 total years of maintenance is reasonable. This means there > will be approximately 4 more 2.7 releases. > > Thoughts? This aligns well with what I've been telling people for the past couple of years, so +1 from me. Commercial Linux distros will also offer 2.7 support out beyond 2015, and it seems to me that the intersection between "needs Python 2.7 support beyond 2015" and "is willing to pay for that support" is likely to be pretty high. If the demand is there on the Windows side, then I expect companies like ActiveState and Enthought will also be happy to oblige. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From martin at v.loewis.de Sun Apr 7 07:11:26 2013 From: martin at v.loewis.de (martin at v.loewis.de) Date: Sun, 07 Apr 2013 07:11:26 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: Message-ID: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> Quoting Benjamin Peterson : > This means we need to talk about how many more 2.7 releases there are > going to be. At the release of 2.7.0, I thought we promised 5 years of > bugfix maintenance, but my memory may be fuddled. I'd like to promote the idea to abandon 2.7 bug fix releases earlier than that, e.g. along with the release of 3.4. My recollection is that "we" didn't actually promise any specific time frame; I recall that Guido said that Python 2.7 would be supported "indefinitely", which is not "infinitely" [1]. The Whats New says [2] """It?s very likely the 2.7 release will have a longer period of maintenance compared to earlier 2.x versions.""" which explicitly refuses to set a date. Of course, individual committers may have promised a more specific policy publicly in the past. Since Christian asked: I'll likely continue to make binary releases for Windows as along as Benjamin declares releases to be bug fix releases. However, it will become increasingly difficult for users to actually use these releases to build extension modules since Microsoft decided to take VS 2008 Express offline (VS 2008 remains available to MSDN subscribers; getting it from a store might also be difficult in 2014). I wonder whether the burden of maintaining three branches for bug fixes (2.7, 3.3, default) and three more for security fixes (2.6, 3.1, 3.2) is really sustainable for committers. I wouldn't want to back off wrt. security fixes, and 2.6 will soon fall out of the promised 5 years (after the initial release). However, stopping to accept bug fixes for 2.7 would IMO significantly reduce the load for committers - it would certainly continue to get security fixes, and (for the time being) "indefinitely" so. Wrt. to the 3.x migration rate: I think this is a self-fulfilling prophecy. Migration rate will certainly increase once we announce an end of 2.7, and then again when the end is actually reached. I'm doubtful with respect to a community-managed ongoing 2.7 bug fix release (i.e. I doubt that it will happen); the same was discussed for a next 2.x feature release, and it hasn't happened. OTOH, it is very likely that people will publish their own patches to 2.7 throughout the net, just as the Linux distributions already do. It may even happen that some volunteer offers to publish a combined repository for such patches, with various members of the community having write access to such a repository (but no formal releases coming out of that). Regards, Martin [1] http://mail.python.org/pipermail/python-dev/2009-November/093651.html [2] http://docs.python.org/dev/whatsnew/2.7.html#the-future-for-python-2-x From fijall at gmail.com Sun Apr 7 09:13:21 2013 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sun, 7 Apr 2013 09:13:21 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> Message-ID: On Sun, Apr 7, 2013 at 7:11 AM, wrote: > > Quoting Benjamin Peterson : > >> This means we need to talk about how many more 2.7 releases there are >> going to be. At the release of 2.7.0, I thought we promised 5 years of >> bugfix maintenance, but my memory may be fuddled. > > > I'd like to promote the idea to abandon 2.7 bug fix releases earlier > than that, e.g. along with the release of 3.4. My recollection is > that "we" didn't actually promise any specific time frame; I recall > that Guido said that Python 2.7 would be supported "indefinitely", > which is not "infinitely" [1]. The Whats New says [2] > > """It?s very likely the 2.7 release will have a longer period of > maintenance compared to earlier 2.x versions.""" > > which explicitly refuses to set a date. Of course, individual committers > may have promised a more specific policy publicly in the past. > > Since Christian asked: I'll likely continue to make binary releases > for Windows as along as Benjamin declares releases to be bug fix > releases. However, it will become increasingly difficult for users > to actually use these releases to build extension modules since > Microsoft decided to take VS 2008 Express offline (VS 2008 remains > available to MSDN subscribers; getting it from a store might > also be difficult in 2014). > > I wonder whether the burden of maintaining three branches for bug > fixes (2.7, 3.3, default) and three more for security fixes > (2.6, 3.1, 3.2) is really sustainable for committers. I wouldn't > want to back off wrt. security fixes, and 2.6 will soon fall out > of the promised 5 years (after the initial release). However, > stopping to accept bug fixes for 2.7 would IMO significantly reduce > the load for committers - it would certainly continue to get > security fixes, and (for the time being) "indefinitely" so. > > Wrt. to the 3.x migration rate: I think this is a self-fulfilling > prophecy. Migration rate will certainly increase once we announce > an end of 2.7, and then again when the end is actually reached. > > I'm doubtful with respect to a community-managed ongoing 2.7 bug > fix release (i.e. I doubt that it will happen); the same was > discussed for a next 2.x feature release, and it hasn't happened. > OTOH, it is very likely that people will publish their own patches > to 2.7 throughout the net, just as the Linux distributions already > do. It may even happen that some volunteer offers to publish a > combined repository for such patches, with various members of the > community having write access to such a repository (but no formal > releases coming out of that). Martin, you guys are shooting yourself in a foot. Almost noone uses python 3 in production, even at pycon, which is the more progressive crowd. There is a giant group of people using python that are not as vocal. While I bet some are using Python 3, Python 2 is incredibly popular for the "long tail" of libraries and applications. How much is 2.7 a burden? There are no major changes and it's pretty cool to consider it "done". For what is worth, we'll maintain the stdlib part of 2.7 past 2 years. It would be cool if python-dev participated in that. Cheers, fijal From regebro at gmail.com Sun Apr 7 09:38:02 2013 From: regebro at gmail.com (Lennart Regebro) Date: Sun, 7 Apr 2013 09:38:02 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> Message-ID: On Sun, Apr 7, 2013 at 7:11 AM, wrote: > Wrt. to the 3.x migration rate: I think this is a self-fulfilling > prophecy. Migration rate will certainly increase once we announce > an end of 2.7, and then again when the end is actually reached. Well... People are in general *stuck* on Python 2. They are not staying because they want to. So I'm not so sure migration rate will increase because an end is announced or reached. //Lennart From martin at v.loewis.de Sun Apr 7 09:44:42 2013 From: martin at v.loewis.de (martin at v.loewis.de) Date: Sun, 07 Apr 2013 09:44:42 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> Message-ID: <20130407094442.Horde.ZcTy9dzVyRcjtYctD-yamg1@webmail.df.eu> > Martin, you guys are shooting yourself in a foot. Almost noone uses > python 3 in production, even at pycon, which is the more progressive > crowd. There is a giant group of people using python that are not as > vocal. While I bet some are using Python 3, Python 2 is incredibly > popular for the "long tail" of libraries and applications. How much is > 2.7 a burden? There are no major changes and it's pretty cool to > consider it "done". Indeed - hence I think it is just fine to stop applying bug fixes to it, as well. People for whom it works fine today apparently don't run into any significant bugs. They can happily continue to use it as-is for ten or more years. It will not go away just when we reduce changes to security fixes. It will remain available for download, the documentation will keep being online, people can continue to ask questions about it on python-list, and continue to get answers. Stopping to apply bug fixes does not really *end* Python 2.7. It's only that people who *do* run into bugs don't have the option anymore that we will eventually publish a fixed release. Their options reduce to - port to 3.x (particularly interesting if Python 3.x *already* fixed it) - find a work-around - maintain a bug fix locally - do something else entirely (like abandoning Python) People deserve to know our plans, so we really need to agree on them and then announce them (see PEP 404). However, people (IMO) have no right to expect us to maintain Python 2.7 until they migrate to 3.x. If we would do that, they will never migrate. Regards, Martin From fijall at gmail.com Sun Apr 7 09:52:13 2013 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sun, 7 Apr 2013 09:52:13 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <20130407094442.Horde.ZcTy9dzVyRcjtYctD-yamg1@webmail.df.eu> References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> <20130407094442.Horde.ZcTy9dzVyRcjtYctD-yamg1@webmail.df.eu> Message-ID: On Sun, Apr 7, 2013 at 9:44 AM, wrote: >> Martin, you guys are shooting yourself in a foot. Almost noone uses >> python 3 in production, even at pycon, which is the more progressive >> crowd. There is a giant group of people using python that are not as >> vocal. While I bet some are using Python 3, Python 2 is incredibly >> popular for the "long tail" of libraries and applications. How much is >> 2.7 a burden? There are no major changes and it's pretty cool to >> consider it "done". > > > Indeed - hence I think it is just fine to stop applying bug fixes to it, > as well. People for whom it works fine today apparently don't run into any > significant bugs. They can happily continue to use it as-is for ten or more > years. It will not go away just when we reduce changes to security fixes. > It will remain available for download, the documentation will keep being > online, people can continue to ask questions about it on python-list, and > continue to get answers. No, they manage to work around issues. It doesn't mean there are no bugs or bugfixes won't help. But I'm not going to argue with you, I don't think you can be convinced about anything here. > > Stopping to apply bug fixes does not really *end* Python 2.7. > > It's only that people who *do* run into bugs don't have the option anymore > that we will eventually publish a fixed release. Their options reduce to > - port to 3.x (particularly interesting if Python 3.x *already* fixed it) > - find a work-around > - maintain a bug fix locally > - do something else entirely (like abandoning Python) > > People deserve to know our plans, so we really need to agree on them and > then announce them (see PEP 404). However, people (IMO) have no right to > expect us to maintain Python 2.7 until they migrate to 3.x. If we would do > that, they will never migrate. > > Regards, > Martin As far as I remember python 3 was supposed to be a better language, not just "the maintained version". It's such a bad idea to force people to go through porting because 2.x is not maintained any more. If they never migrate on the premises of python 3 being a better language what does it say about python 3? I cannot of course tell you what you should do in your free time though, if you don't feel like doing anything in that area, fine. We'll maintain the stdlib of Python 2.7 past the 2 year mark though. Cheers, fijal From martin at v.loewis.de Sun Apr 7 09:51:17 2013 From: martin at v.loewis.de (martin at v.loewis.de) Date: Sun, 07 Apr 2013 09:51:17 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> Message-ID: <20130407095117.Horde.eT3bmz8gGAZstl32oR712Q9@webmail.df.eu> Quoting Lennart Regebro : > On Sun, Apr 7, 2013 at 7:11 AM, wrote: >> Wrt. to the 3.x migration rate: I think this is a self-fulfilling >> prophecy. Migration rate will certainly increase once we announce >> an end of 2.7, and then again when the end is actually reached. > > Well... People are in general *stuck* on Python 2. They are not > staying because they want to. So I'm not so sure migration rate will > increase because an end is announced or reached. I assume you say that because people rely on libraries that haven't been ported (correct me if there are other reasons to be stuck). With an announced end-of-life, I'm certain that migration rate will increase, because people will now urge their suppliers, pointing to the announcement. With Benjamin's proposed schedule, they would still have two years for their suppliers to act. Even under my proposed schedule, there would be plenty of time. Also, this is all free software (at least most of it). Nobody can *really* be stuck on a not-ported dependency, as they could always port it themselves, and even fork if the developer refuses to integrate the port (and you know that this actually happens). Regards, Martin From stefan_ml at behnel.de Sun Apr 7 09:58:57 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 07 Apr 2013 09:58:57 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> <20130407094442.Horde.ZcTy9dzVyRcjtYctD-yamg1@webmail.df.eu> Message-ID: Maciej Fijalkowski, 07.04.2013 09:52: > As far as I remember python 3 was supposed to be a better language, > not just "the maintained version". It's such a bad idea to force > people to go through porting because 2.x is not maintained any more. > If they never migrate on the premises of python 3 being a better > language what does it say about python 3? Nothing. Most people simply don't do the switch all by themselves before you can convince that that what they have in their hands is actually a dead parrot. Stefan From steve at pearwood.info Sun Apr 7 10:01:44 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 07 Apr 2013 18:01:44 +1000 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <20130407094442.Horde.ZcTy9dzVyRcjtYctD-yamg1@webmail.df.eu> References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> <20130407094442.Horde.ZcTy9dzVyRcjtYctD-yamg1@webmail.df.eu> Message-ID: <516127E8.2080508@pearwood.info> On 07/04/13 17:44, martin at v.loewis.de wrote: >> Martin, you guys are shooting yourself in a foot. Almost noone uses >> python 3 in production, even at pycon, which is the more progressive >> crowd. There is a giant group of people using python that are not as >> vocal. While I bet some are using Python 3, Python 2 is incredibly >> popular for the "long tail" of libraries and applications. How much is >> 2.7 a burden? There are no major changes and it's pretty cool to >> consider it "done". > > Indeed - hence I think it is just fine to stop applying bug fixes to it, > as well. People for whom it works fine today apparently don't run into any > significant bugs. They can happily continue to use it as-is for ten or more > years. It will not go away just when we reduce changes to security fixes. > It will remain available for download, the documentation will keep being > online, people can continue to ask questions about it on python-list, and > continue to get answers. +1 On the python-list@ mailing list, we occasionally get posts from people still using Python 2.3, and regularly from people on 2.5. > Stopping to apply bug fixes does not really *end* Python 2.7. > > It's only that people who *do* run into bugs don't have the option anymore > that we will eventually publish a fixed release. Their options reduce to > - port to 3.x (particularly interesting if Python 3.x *already* fixed it) > - find a work-around > - maintain a bug fix locally > - do something else entirely (like abandoning Python) Or, if they have paid support from a vendor like Red Hat, hassle the vendor for a fix. Speaking of 2.3, as I understand it Red Hat still offer paid support for 2.3, which won't expire for a few more years, and security fixes only for some years beyond that. [By memory, which may not be entirely accurate.] -- Steven From regebro at gmail.com Sun Apr 7 10:10:17 2013 From: regebro at gmail.com (Lennart Regebro) Date: Sun, 7 Apr 2013 10:10:17 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <20130407095117.Horde.eT3bmz8gGAZstl32oR712Q9@webmail.df.eu> References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> <20130407095117.Horde.eT3bmz8gGAZstl32oR712Q9@webmail.df.eu> Message-ID: On Sun, Apr 7, 2013 at 9:51 AM, wrote: > Quoting Lennart Regebro : >> Well... People are in general *stuck* on Python 2. They are not >> staying because they want to. So I'm not so sure migration rate will >> increase because an end is announced or reached. > > I assume you say that because people rely on libraries that haven't > been ported (correct me if there are other reasons to be stuck). Company policies that mean you are using old distros with no support for Python 3 and not being allowed to install it from source is also a reason, but yes, the main one being libraries/frameworks, yes. > With an announced end-of-life, I'm certain that migration rate will > increase, because people will now urge their suppliers, pointing > to the announcement. The suppliers are often people who are maintaining an open source library of some sort. When I see questions on stackoverflow about support for X on Python 3 I sometimes take a quick look of the state of libraries, check out their mailing list etc. It's *always* a problem of that the maintainers themselves are stuck on Python 2.7 or earlier together with porting being problematic. I think Python 3.3 with the u'' literal is much more important for increased adoption there than the end of life of 2.7 as it often makes porting much easier. But even so sometimes API's needs to be changed, etc, so it takes a big concerted effort of both the maintainers, and the few people that are interested in porting it to Python 3. And when you get one new person asking for Python 3 support every 6 months, that's just not enough people. That's the hangup IMO. Ending Python 2.7 will make no difference there either good or bad, I think. We need to find other ways of improving adoption. As for the company policies, in theory it sounds like a good argument that ending Python 2.7 would be incentive for these to change. But these are often slow moving companies that are happy using outdated software, and are clearly using it already, or they would be on distros that *did* support Python 3 already. :-) > Nobody can *really* be stuck on a not-ported dependency, as they > could always port it themselves, and even fork if the developer > refuses to integrate the port (and you know that this actually > happens). Yes, but time/money/knowledge is at a premium. Also self-confidence. A lot of people probably think porting is much harder than it is. :-) //Lennart From steve at pearwood.info Sun Apr 7 10:11:03 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 07 Apr 2013 18:11:03 +1000 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> <20130407094442.Horde.ZcTy9dzVyRcjtYctD-yamg1@webmail.df.eu> Message-ID: <51612A17.2000408@pearwood.info> On 07/04/13 17:52, Maciej Fijalkowski wrote: > If they never migrate on the premises of python 3 being a better > language what does it say about python 3? Very little. People stick with languages for all sorts of reasons, including: - It's what I know - I don't like change - That's what the client insists on - That's what my boss insists on - That's what the vendor provides - That's what my web host provides - I really love language X, but I need this *one* library that's only available on language Y, so I'm stuck I stuck with Python 1.5 for seven or eight years, and didn't migrate to Python 2 until 2.3. Why? Because that's what came standard on my Linux distro of choice. -- Steven From fijall at gmail.com Sun Apr 7 10:12:54 2013 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sun, 7 Apr 2013 10:12:54 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <20130407095117.Horde.eT3bmz8gGAZstl32oR712Q9@webmail.df.eu> References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> <20130407095117.Horde.eT3bmz8gGAZstl32oR712Q9@webmail.df.eu> Message-ID: On Sun, Apr 7, 2013 at 9:51 AM, wrote: > > Quoting Lennart Regebro : > >> On Sun, Apr 7, 2013 at 7:11 AM, wrote: >>> >>> Wrt. to the 3.x migration rate: I think this is a self-fulfilling >>> prophecy. Migration rate will certainly increase once we announce >>> an end of 2.7, and then again when the end is actually reached. >> >> >> Well... People are in general *stuck* on Python 2. They are not >> staying because they want to. So I'm not so sure migration rate will >> increase because an end is announced or reached. > > > I assume you say that because people rely on libraries that haven't > been ported (correct me if there are other reasons to be stuck). > I'm stuck because I can't tell my users "oh, we didn't improve pypy for the last year/6 months/3 months, because we were busy upgrading sources you'll never see to python 3" Cheers, fijal From regebro at gmail.com Sun Apr 7 10:12:51 2013 From: regebro at gmail.com (Lennart Regebro) Date: Sun, 7 Apr 2013 10:12:51 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> <20130407095117.Horde.eT3bmz8gGAZstl32oR712Q9@webmail.df.eu> Message-ID: On Sun, Apr 7, 2013 at 10:10 AM, Lennart Regebro wrote: > That's the hangup IMO. Ending Python 2.7 will make no difference there > either good or bad, I think. We need to find other ways of improving > adoption. And to be clear: I am therefore not arguing *not* to end it. I just don't think that doing so will increase Python 3 adoption. I think that's a red herring. I have little opinion on whether to announce an official end or not, nor when. I think the burden of maintaining many branches is a much better argument, and that it therefore probably should be decided by the maintainers. //Lennart From stefan_ml at behnel.de Sun Apr 7 10:32:12 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 07 Apr 2013 10:32:12 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> <20130407095117.Horde.eT3bmz8gGAZstl32oR712Q9@webmail.df.eu> Message-ID: Maciej Fijalkowski, 07.04.2013 10:12: > On Sun, Apr 7, 2013 at 9:51 AM, wrote: >> Quoting Lennart Regebro: >>> On Sun, Apr 7, 2013 at 7:11 AM, wrote: >>>> Wrt. to the 3.x migration rate: I think this is a self-fulfilling >>>> prophecy. Migration rate will certainly increase once we announce >>>> an end of 2.7, and then again when the end is actually reached. >>> >>> Well... People are in general *stuck* on Python 2. They are not >>> staying because they want to. So I'm not so sure migration rate will >>> increase because an end is announced or reached. >> >> I assume you say that because people rely on libraries that haven't >> been ported (correct me if there are other reasons to be stuck). > > I'm stuck because I can't tell my users "oh, we didn't improve pypy > for the last year/6 months/3 months, because we were busy upgrading > sources you'll never see to python 3" Why not? It's not like many people *see* PyPy's sources ever in their life, but my guess is that most of your users will eventually end up *using* those upgraded sources anyway. So those upgrades will also be an improvement for most of them. Stefan From ncoghlan at gmail.com Sun Apr 7 10:33:11 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 7 Apr 2013 18:33:11 +1000 Subject: [Python-Dev] Updates to PEP 1 (PEP process meta-PEP) Message-ID: I just pushed two modifications to PEP 1, both triggered by the in-progress packaging PEP updates (see diff at http://hg.python.org/peps/rev/0a8e456973ed). The first change is the one I raised at the language summit, allowing pronouncement on PEPs that don't immediately affect the language definition or the standard library to occur on a list other than python-dev. This is done by giving the "Discussions-To" header a bit more force. The other change is closely related, but wasn't specifically discussed at the language summit (I only noticed the need for it when making the edits for the change above). This second change was to adjust the wording to make the "Python-Version" header optional for Standards track PEPs that don't directly affect the language reference or standard library. In effect, the new rule is that a Standards track PEP with a specific Python-Version set still has to be discussed and accepted or rejected on python-dev. It's only non-version-specific interoperability standards (like the packaging PEPs) that are fair game for another list. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From fijall at gmail.com Sun Apr 7 10:37:33 2013 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sun, 7 Apr 2013 10:37:33 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> <20130407095117.Horde.eT3bmz8gGAZstl32oR712Q9@webmail.df.eu> Message-ID: On Sun, Apr 7, 2013 at 10:32 AM, Stefan Behnel wrote: > Maciej Fijalkowski, 07.04.2013 10:12: >> On Sun, Apr 7, 2013 at 9:51 AM, wrote: >>> Quoting Lennart Regebro: >>>> On Sun, Apr 7, 2013 at 7:11 AM, wrote: >>>>> Wrt. to the 3.x migration rate: I think this is a self-fulfilling >>>>> prophecy. Migration rate will certainly increase once we announce >>>>> an end of 2.7, and then again when the end is actually reached. >>>> >>>> Well... People are in general *stuck* on Python 2. They are not >>>> staying because they want to. So I'm not so sure migration rate will >>>> increase because an end is announced or reached. >>> >>> I assume you say that because people rely on libraries that haven't >>> been ported (correct me if there are other reasons to be stuck). >> >> I'm stuck because I can't tell my users "oh, we didn't improve pypy >> for the last year/6 months/3 months, because we were busy upgrading >> sources you'll never see to python 3" > > Why not? It's not like many people *see* PyPy's sources ever in their life, > but my guess is that most of your users will eventually end up *using* > those upgraded sources anyway. So those upgrades will also be an > improvement for most of them. > > Stefan Some of them, maybe. Most people absolutely don't care. Most of my users are people who want this 10% speed improvement rather than sources upgraded to a different, supposedly better, language. Cheers, fijal From stefan_ml at behnel.de Sun Apr 7 10:42:51 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 07 Apr 2013 10:42:51 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> <20130407095117.Horde.eT3bmz8gGAZstl32oR712Q9@webmail.df.eu> Message-ID: Maciej Fijalkowski, 07.04.2013 10:37: > On Sun, Apr 7, 2013 at 10:32 AM, Stefan Behnel wrote: >> Maciej Fijalkowski, 07.04.2013 10:12: >>> On Sun, Apr 7, 2013 at 9:51 AM, wrote: >>>> Quoting Lennart Regebro: >>>>> On Sun, Apr 7, 2013 at 7:11 AM, wrote: >>>>>> Wrt. to the 3.x migration rate: I think this is a self-fulfilling >>>>>> prophecy. Migration rate will certainly increase once we announce >>>>>> an end of 2.7, and then again when the end is actually reached. >>>>> >>>>> Well... People are in general *stuck* on Python 2. They are not >>>>> staying because they want to. So I'm not so sure migration rate will >>>>> increase because an end is announced or reached. >>>> >>>> I assume you say that because people rely on libraries that haven't >>>> been ported (correct me if there are other reasons to be stuck). >>> >>> I'm stuck because I can't tell my users "oh, we didn't improve pypy >>> for the last year/6 months/3 months, because we were busy upgrading >>> sources you'll never see to python 3" >> >> Why not? It's not like many people *see* PyPy's sources ever in their life, >> but my guess is that most of your users will eventually end up *using* >> those upgraded sources anyway. So those upgrades will also be an >> improvement for most of them. > > Some of them, maybe. > > Most people absolutely don't care. Most of my users are people who > want this 10% speed improvement rather than sources upgraded to a > different, supposedly better, language. My guess is that they don't care because they don't have a choice anyway. If they want to use PyPy (because they care about this 10% speedup), then they have to stick to Python 2 as of now. Extrapolating from that that they wouldn't prefer writing Python 3 code if they could is a fallacy. Stefan From fijall at gmail.com Sun Apr 7 10:45:55 2013 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sun, 7 Apr 2013 10:45:55 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> <20130407095117.Horde.eT3bmz8gGAZstl32oR712Q9@webmail.df.eu> Message-ID: On Sun, Apr 7, 2013 at 10:42 AM, Stefan Behnel wrote: > Maciej Fijalkowski, 07.04.2013 10:37: >> On Sun, Apr 7, 2013 at 10:32 AM, Stefan Behnel wrote: >>> Maciej Fijalkowski, 07.04.2013 10:12: >>>> On Sun, Apr 7, 2013 at 9:51 AM, wrote: >>>>> Quoting Lennart Regebro: >>>>>> On Sun, Apr 7, 2013 at 7:11 AM, wrote: >>>>>>> Wrt. to the 3.x migration rate: I think this is a self-fulfilling >>>>>>> prophecy. Migration rate will certainly increase once we announce >>>>>>> an end of 2.7, and then again when the end is actually reached. >>>>>> >>>>>> Well... People are in general *stuck* on Python 2. They are not >>>>>> staying because they want to. So I'm not so sure migration rate will >>>>>> increase because an end is announced or reached. >>>>> >>>>> I assume you say that because people rely on libraries that haven't >>>>> been ported (correct me if there are other reasons to be stuck). >>>> >>>> I'm stuck because I can't tell my users "oh, we didn't improve pypy >>>> for the last year/6 months/3 months, because we were busy upgrading >>>> sources you'll never see to python 3" >>> >>> Why not? It's not like many people *see* PyPy's sources ever in their life, >>> but my guess is that most of your users will eventually end up *using* >>> those upgraded sources anyway. So those upgrades will also be an >>> improvement for most of them. >> >> Some of them, maybe. >> >> Most people absolutely don't care. Most of my users are people who >> want this 10% speed improvement rather than sources upgraded to a >> different, supposedly better, language. > > My guess is that they don't care because they don't have a choice anyway. > If they want to use PyPy (because they care about this 10% speedup), then > they have to stick to Python 2 as of now. Extrapolating from that that they > wouldn't prefer writing Python 3 code if they could is a fallacy. > > Stefan > You're completely missing what I said. I'm not arguing against providing pypy3k and we're working on it. Then users can choose. I'm arguing against me porting PyPy *source* to python3, which does not affect the language they users are using. Those are two drastically different scenarios. One is visible to users (and our plan is to support both pypy and pypy3k for the forseeable future) and the other is what developers see with really 0 visibility to the user. Cheers, fijal From stefan_ml at behnel.de Sun Apr 7 11:05:57 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 07 Apr 2013 11:05:57 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> <20130407095117.Horde.eT3bmz8gGAZstl32oR712Q9@webmail.df.eu> Message-ID: Maciej Fijalkowski, 07.04.2013 10:45: > On Sun, Apr 7, 2013 at 10:42 AM, Stefan Behnel wrote: >> Maciej Fijalkowski, 07.04.2013 10:37: >>> On Sun, Apr 7, 2013 at 10:32 AM, Stefan Behnel wrote: >>>> Maciej Fijalkowski, 07.04.2013 10:12: >>>>> On Sun, Apr 7, 2013 at 9:51 AM, wrote: >>>>>> Quoting Lennart Regebro: >>>>>>> On Sun, Apr 7, 2013 at 7:11 AM, wrote: >>>>>>>> Wrt. to the 3.x migration rate: I think this is a self-fulfilling >>>>>>>> prophecy. Migration rate will certainly increase once we announce >>>>>>>> an end of 2.7, and then again when the end is actually reached. >>>>>>> >>>>>>> Well... People are in general *stuck* on Python 2. They are not >>>>>>> staying because they want to. So I'm not so sure migration rate will >>>>>>> increase because an end is announced or reached. >>>>>> >>>>>> I assume you say that because people rely on libraries that haven't >>>>>> been ported (correct me if there are other reasons to be stuck). >>>>> >>>>> I'm stuck because I can't tell my users "oh, we didn't improve pypy >>>>> for the last year/6 months/3 months, because we were busy upgrading >>>>> sources you'll never see to python 3" >>>> >>>> Why not? It's not like many people *see* PyPy's sources ever in their life, >>>> but my guess is that most of your users will eventually end up *using* >>>> those upgraded sources anyway. So those upgrades will also be an >>>> improvement for most of them. >>> >>> Some of them, maybe. >>> >>> Most people absolutely don't care. Most of my users are people who >>> want this 10% speed improvement rather than sources upgraded to a >>> different, supposedly better, language. >> >> My guess is that they don't care because they don't have a choice anyway. >> If they want to use PyPy (because they care about this 10% speedup), then >> they have to stick to Python 2 as of now. Extrapolating from that that they >> wouldn't prefer writing Python 3 code if they could is a fallacy. > > You're completely missing what I said. I'm not arguing against > providing pypy3k and we're working on it. Then users can choose. I'm > arguing against me porting PyPy *source* to python3, which does not > affect the language they users are using. Those are two drastically > different scenarios. One is visible to users (and our plan is to > support both pypy and pypy3k for the forseeable future) and the other > is what developers see with really 0 visibility to the user. Then I don't see why that code would have to be changed at all. AFAIK, most of PyPy isn't even written in (real) Python but in RPython, which is essentially "the subset of Python 2.x that PyPy can translate statically". Unless you deliberately and arbitrarily want to change RPython to be "the subset of Python 3.x that PyPy can translate statically", the eventual death of Python 2 shouldn't affect that code. Stefan From ncoghlan at gmail.com Sun Apr 7 11:19:07 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 7 Apr 2013 19:19:07 +1000 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <516127E8.2080508@pearwood.info> References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> <20130407094442.Horde.ZcTy9dzVyRcjtYctD-yamg1@webmail.df.eu> <516127E8.2080508@pearwood.info> Message-ID: On Sun, Apr 7, 2013 at 6:01 PM, Steven D'Aprano wrote: > Or, if they have paid support from a vendor like Red Hat, hassle the vendor > for a fix. Speaking of 2.3, as I understand it Red Hat still offer paid > support for 2.3, which won't expire for a few more years, and security fixes > only for some years beyond that. > > [By memory, which may not be entirely accurate.] Correct, the system Python in RHEL 4 is 2.3 and that is still a supported platform if you pay for the Extended Lifecycle Support (until early next year you can still get extended support for RHEL *3*, and I believe the system Python in that is 2.2). RHEL 5 ships with 2.4 and RHEL 6 with 2.6 and those are both still in their regular support period. The system Python version in the upcoming RHEL 7 release hasn't been formally announced yet, but it doesn't take a genius to figure out what it is going to be when the system Python in Fedora is currently still Python 2.7. I'll also note that regular support for RHEL 6 doesn't end until 2020, and extended support in 2023, so even Python *2.6* should have commercial support available for another decade, let alone 2.7. These supported versions are also (or will also be) available for free through CentOS, ScientificLinux and other Red Hat derivatives. It actually isn't the bug fixes which I consider particularly important in 2.7 - it's the build fixes (including the cross-compilation support). As the IDLE team ramp up their efforts, there may also be benefit in getting updated versions of IDLE into the hands of beginners (remember, a huge amount of training is enterprise focused, which isn't likely to switch to 3.x until after Red Hat does, and that migration has barely started on the Fedora side - having the installer and package management system written in Python means that migration is likely to open up some fairly major cans of worms). Python 2.7 is a mature, stable platform for software development. Python 3.3 is *better* in most respects, but if you've already worked around (or aren't affected by) the text model issues in 2.x, then most of the cool features in 3.x are available as backported modules on PyPI. With the lifting of the language moratorium. 3.3 has taken us further down the "carrot" path for migration (adding the more efficient Unicode representation, the "yield from" notation, more memory efficient objects, import system enhancements and the improved exception hierarchy to the pre-existing carrot that is chained exceptions). 3.4 will likely add even more carrots, as we take further advantage of the cleaner code base that was gained in the 3.x migration. I think it's quite reasonable for individual committers to decide how much we want to worry about Python 2.7. I know I've fixed some bugs in 3.x and then left the issue open as still applicable to 2.7 (usually because the bug affected some part of the code that changed in the transition, so the backport isn't straightforward). I consider that a reasonable thing for me, or any other committer, to do: leave issues open as applicable to 2.7 rather than doing the backport immediately. If someone else cares enough to backport it, great, otherwise the issue is still there to indicate that the problem still exists in 2.7. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From asolano at icai.es Sun Apr 7 12:08:05 2013 From: asolano at icai.es (=?UTF-8?Q?Alfredo_Solano_Mart=C3=ADnez?=) Date: Sun, 7 Apr 2013 12:08:05 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> <20130407094442.Horde.ZcTy9dzVyRcjtYctD-yamg1@webmail.df.eu> <516127E8.2080508@pearwood.info> Message-ID: I think the question average python users have is "What's in it for me?". While the guts have undergone lots of changes, from the outside it is mostly perceived as the unicode-by-default and the print function. As per Bret's talk at pycon [1], speed is roughly the same, which is great, considering all the new stuff, but not a compelling reason to change for Joe Programmer. Joe will probably consider PyPy because "moar speez" is an easy sell, though. In a way, python3 is victim of the success of python2. Inertia is a powerful force, and honestly, most of the time python2 just works (easy to write, easy to modify, fast enough). Maybe it's just a marketing problem, and more examples of "things you can only do with python3" are needed. Or maybe for all the good changes already there, it still needs a killer feature, i.e. the proverbial elevator pitch, that sets it apart from its older brother. Alfredo [1] http://pyvideo.org/video/1730/python-33-trust-me-its-better-than-27 From tshepang at gmail.com Sun Apr 7 11:46:25 2013 From: tshepang at gmail.com (Tshepang Lekhonkhobe) Date: Sun, 7 Apr 2013 11:46:25 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> Message-ID: On Sun, Apr 7, 2013 at 9:13 AM, Maciej Fijalkowski wrote: > For what is worth, we'll maintain the stdlib part of 2.7 past 2 years. You mean 2 years beyond 2015 (assuming that will be end-of-bugfix date)? PS: I only noticed you were talking about PyPy because I recognized your name; others won't. From solipsis at pitrou.net Sun Apr 7 12:41:00 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 7 Apr 2013 12:41:00 +0200 Subject: [Python-Dev] The end of 2.7 References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> <20130407094442.Horde.ZcTy9dzVyRcjtYctD-yamg1@webmail.df.eu> <51612A17.2000408@pearwood.info> Message-ID: <20130407124100.098b927a@pitrou.net> On Sun, 07 Apr 2013 18:11:03 +1000 Steven D'Aprano wrote: > On 07/04/13 17:52, Maciej Fijalkowski wrote: > > If they never migrate on the premises of python 3 being a better > > language what does it say about python 3? > > Very little. People stick with languages for all sorts of reasons, > including: > > - It's what I know > - I don't like change > - That's what the client insists on > - That's what my boss insists on > - That's what the vendor provides > - That's what my web host provides > - I really love language X, but I need this *one* library that's only > available on language Y, so I'm stuck - It takes time to migrate a codebase. Regards Antoine. From martin at v.loewis.de Sun Apr 7 12:45:48 2013 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 07 Apr 2013 12:45:48 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> Message-ID: <51614E5C.9080103@v.loewis.de> Am 07.04.13 00:37, schrieb Benjamin Peterson: > What I like about 6 months is that its short enough, so we don't have > feel bad about not taking a certain change; it can just be pushed to > the next no-too-far-away release. A year is quite a while to wait for > a fix to be released. It's also a nice timeframe for some > distributions (looking at you, Ubuntu). This means that we will see two-digit micro-releases, right (assuming that there will be a few security releases)? Regards, Martin From martin at v.loewis.de Sun Apr 7 13:49:27 2013 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 07 Apr 2013 13:49:27 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <20130407071126.Horde.Zl7wQ72lXZ4_mr2onM8BNg8@webmail.df.eu> Message-ID: <51615D47.2020206@v.loewis.de> Am 07.04.13 11:46, schrieb Tshepang Lekhonkhobe: > On Sun, Apr 7, 2013 at 9:13 AM, Maciej Fijalkowski wrote: >> For what is worth, we'll maintain the stdlib part of 2.7 past 2 years. > > You mean 2 years beyond 2015 (assuming that will be end-of-bugfix date)? No, I think he means "beyond 2 years from now", i.e. "beyond 2015", i.e. "after python-dev stops maintaining it", without any specific end of that support. Regards, Martin From skip at pobox.com Sun Apr 7 14:10:45 2013 From: skip at pobox.com (Skip Montanaro) Date: Sun, 7 Apr 2013 07:10:45 -0500 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> Message-ID: I started writing this last night before the flurry of messages which arrived overnight. I thought originally, "Oh, Skip, you're being too harsh." But now I'm not so sure. I think you are approaching the issue of 2.7's EOL incorrectly. Of those discussing the end of Python 2.7, how many of you still use it in your day-to-day work? Have any of you yet to move to Python 3? It sounds like many people at PyCon are still 2.x users. Where I work (a trading firm that uses Python as just one of many different pieces of technology, not a company where Python is the core technology upon which the firm is based) we are only just now migrating from 2.4 to 2.7. I can't imagine we'll have migrated to Python 3 in two years. It's not like we haven't seen this coming, but you can only justify moving so fast with technology that already works, especially if, like Python, you use it with lots of other packages (most/all of which themselves have to be ported to Python 3) and in-house software. I think the discussion should focus on who's left on 2.x and why, not, "yeah, releases every six months for the next couple years ought to do it." Just my 2?. Skip From benjamin at python.org Sun Apr 7 15:38:05 2013 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 7 Apr 2013 09:38:05 -0400 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <51614E5C.9080103@v.loewis.de> References: <51609C0D.8000106@python.org> <51614E5C.9080103@v.loewis.de> Message-ID: 2013/4/7 "Martin v. L?wis" : > Am 07.04.13 00:37, schrieb Benjamin Peterson: >> What I like about 6 months is that its short enough, so we don't have >> feel bad about not taking a certain change; it can just be pushed to >> the next no-too-far-away release. A year is quite a while to wait for >> a fix to be released. It's also a nice timeframe for some >> distributions (looking at you, Ubuntu). > > This means that we will see two-digit micro-releases, right (assuming > that there will be a few security releases)? With the current proposal, 2.7.8 in 2015 would be the last non-security release. -- Regards, Benjamin From benjamin at python.org Sun Apr 7 15:44:27 2013 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 7 Apr 2013 09:44:27 -0400 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> Message-ID: 2013/4/7 Skip Montanaro : > I started writing this last night before the flurry of messages which > arrived overnight. I thought originally, "Oh, Skip, you're being too > harsh." But now I'm not so sure. I think you are approaching the > issue of 2.7's EOL incorrectly. Of those discussing the end of Python > 2.7, how many of you still use it in your day-to-day work? Have any of > you yet to move to Python 3? It sounds like many people at PyCon are > still 2.x users. > > Where I work (a trading firm that uses Python as just one of many > different pieces of technology, not a company where Python is the core > technology upon which the firm is based) we are only just now > migrating from 2.4 to 2.7. I can't imagine we'll have migrated to > Python 3 in two years. It's not like we haven't seen this coming, but > you can only justify moving so fast with technology that already > works, especially if, like Python, you use it with lots of other > packages (most/all of which themselves have to be ported to Python 3) > and in-house software. > > I think the discussion should focus on who's left on 2.x and why, not, > "yeah, releases every six months for the next couple years ought to do > it." This thread is about setting CPython release schedules, so that the discussion focuses on that is unavoidable. :) I don't think the bug fix releases of CPython are critically important to the life of a Python version. Every 2.x version has survived much longer than Python-dev has done bugfixes on it. As has been noted on this thread, there will be commercial and apparently PyPy support for 2.7 long after cpython stops bug fixing it. -- Regards, Benjamin From tismer at stackless.com Sun Apr 7 16:00:39 2013 From: tismer at stackless.com (Christian Tismer) Date: Sun, 07 Apr 2013 16:00:39 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <7DF6FD0E-D18C-412F-8149-9A2AED35548C@gmail.com> References: <7DF6FD0E-D18C-412F-8149-9A2AED35548C@gmail.com> Message-ID: <51617C07.3000501@stackless.com> On 07.04.13 03:54, Raymond Hettinger wrote: > > On Apr 6, 2013, at 2:02 PM, Benjamin Peterson > wrote: > >> we need to talk about how many more 2.7 releases there are >> going to be. At the release of 2.7.0, I thought we promised 5 years of >> bugfix maintenance, but my memory may be fuddled. > > I don't we need to make any "promises" beyond 5 years, > but I also think it is likely the 2.7 will end-up being a > long-term maintenance version of Python. > > At this year's Pycon keynote, I surveyed the crowd (approx 2500 people) > and all almost everyone indicated that they had tried out Python 3.x > and almost no one was using it in production or writing code for it. > That indicates that Python 2.7 will continue to be important for a good > while. > > In addition, the other implementations of Python (Jython, PyPy, GAE, > and IronPython) are all at or nearly at Python 2.7. So, continued > support will be needed for their users as well. > > After 2.7.4, I expect that the pace of real bug fixes will slow down, > but that we'll continue to improve docs, add docstrings, update IDLE, etc. > > IMO, it is premature to utter the phrase "the end of 2.7". > Better to say, "2.7 is stable and is expected to only have minor updates". > > Future point releases probably ought to occur "on their own schedule" > whenever there are a sufficient number of changes to warrant a release, > or an important security fix, or whenever the release managers have time. > Raimond, although I think you are right, I don't think this statement helps Python to move forward. Your vision is realistic, nevertheless I think it is important to tell everybody that 2013 is the year to move to Python 3. If enough people like you, Alex, Dabeaz, etc are claiming that loudly enough, it will eventually happen! cheers - chris -- Christian Tismer :^) Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/ 14482 Potsdam : PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From tismer at stackless.com Sun Apr 7 15:53:57 2013 From: tismer at stackless.com (Christian Tismer) Date: Sun, 07 Apr 2013 15:53:57 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> Message-ID: <51617A75.8040506@stackless.com> Hi Skip, On 07.04.13 14:10, Skip Montanaro wrote: > I started writing this last night before the flurry of messages which > arrived overnight. I thought originally, "Oh, Skip, you're being too > harsh." But now I'm not so sure. I think you are approaching the > issue of 2.7's EOL incorrectly. Of those discussing the end of Python > 2.7, how many of you still use it in your day-to-day work? Have any of > you yet to move to Python 3? It sounds like many people at PyCon are > still 2.x users. > > Where I work (a trading firm that uses Python as just one of many > different pieces of technology, not a company where Python is the core > technology upon which the firm is based) we are only just now > migrating from 2.4 to 2.7. I can't imagine we'll have migrated to > Python 3 in two years. It's not like we haven't seen this coming, but > you can only justify moving so fast with technology that already > works, especially if, like Python, you use it with lots of other > packages (most/all of which themselves have to be ported to Python 3) > and in-house software. > > I think the discussion should focus on who's left on 2.x and why, not, > "yeah, releases every six months for the next couple years ought to do > it." > when I read this, I was slightly shocked. You know what? """ We are pleased to announce the release of*Python 2.4, final*on November 30, 2004. """ I know that companies try to save (time? money?) something by not upgrading software, and this is extremely annoying. In my own project, which is for a customer, I just managed to do the complete transition from Python 2.7 to 3.3. Well, this was relatively simple because there is just my boss to be convinced, and myself, because honestly the 3.3 support is still not as good as needed. But I think every employee (including you) can quite easily put some pressure on his company by claiming that Python 2.x is a dead end, and everybody is about to move on to 3.x. This does not have to be true, I just recognize that by claiming it and doing it with your projects, the movement becomes a reality. Just say that we all need to move on and cannot care about companies that ignore this necessity. I agree it is hard to push things forward, when certain tools are just supporting 2.x. My way to get over this is ranting, and porting some things, and claiming it was a cake walk. A lie, but it helped. my 2.01 cent -- chris -- Christian Tismer :^) Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/ 14482 Potsdam : PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg at krypto.org Sun Apr 7 16:58:15 2013 From: greg at krypto.org (Gregory P. Smith) Date: Sun, 7 Apr 2013 07:58:15 -0700 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <51617A75.8040506@stackless.com> References: <51609C0D.8000106@python.org> <51617A75.8040506@stackless.com> Message-ID: On Sun, Apr 7, 2013 at 6:53 AM, Christian Tismer wrote: > Hi Skip, > > > On 07.04.13 14:10, Skip Montanaro wrote: > > I started writing this last night before the flurry of messages which > arrived overnight. I thought originally, "Oh, Skip, you're being too > harsh." But now I'm not so sure. I think you are approaching the > issue of 2.7's EOL incorrectly. Of those discussing the end of Python > 2.7, how many of you still use it in your day-to-day work? Have any of > you yet to move to Python 3? It sounds like many people at PyCon are > still 2.x users. > > Where I work (a trading firm that uses Python as just one of many > different pieces of technology, not a company where Python is the core > technology upon which the firm is based) we are only just now > migrating from 2.4 to 2.7. I can't imagine we'll have migrated to > Python 3 in two years. It's not like we haven't seen this coming, but > you can only justify moving so fast with technology that already > works, especially if, like Python, you use it with lots of other > packages (most/all of which themselves have to be ported to Python 3) > and in-house software. > > I think the discussion should focus on who's left on 2.x and why, not, > "yeah, releases every six months for the next couple years ought to do > it." > > > > when I read this, I was slightly shocked. You know what? > """ > We are pleased to announce the release of *Python 2.4, final* on November > 30, 2004. > """ > > I know that companies try to save (time? money?) something by not upgrading > software, and this is extremely annoying. > You're not looking at it from the users perspective. They see: "we are pleased to announce that RHEL 4 will be supported until the year 3325" and continue to use everything that it ships with and only that. its their own loss for not investing in maintaining infrastructure of their own rather than investing in a support contract from their vendor but it is a valid choice none the less. it has nothing to do with what python-dev chooses to do release wise. I think this thread has already settled the question that Benjamin set out to ask: it doesn't matter when we stop issuing bug fix releases of 2.7, users will exist long enough for even today's deniers to hate them for using an old version. If Benjamin wants to see bug fix releases made for two years, great! If not, no big deal either. We as python-dev are a bunch of volunteers and it is up to each one of us if we'll bother to continue back porting fixes or investigating bugs on 2.7 or not. I strongly suspect that many of us will only continue to do so as long as 2.7 is relevant to our day jobs. We don't need to close the 2.7 branch to commits and bug fixes. Ever. But most of us will stop caring about making changes to it at some point. For me that point is after 3.4. my 3 cents, -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.brandl at gmx.net Sun Apr 7 17:04:03 2013 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 07 Apr 2013 17:04:03 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> Message-ID: Am 07.04.2013 14:10, schrieb Skip Montanaro: > I started writing this last night before the flurry of messages which > arrived overnight. I thought originally, "Oh, Skip, you're being too > harsh." But now I'm not so sure. I think you are approaching the > issue of 2.7's EOL incorrectly. Of those discussing the end of Python > 2.7, how many of you still use it in your day-to-day work? Have any of > you yet to move to Python 3? It sounds like many people at PyCon are > still 2.x users. > > Where I work (a trading firm that uses Python as just one of many > different pieces of technology, not a company where Python is the core > technology upon which the firm is based) we are only just now > migrating from 2.4 to 2.7. I can't imagine we'll have migrated to > Python 3 in two years. You won't have to. You've been using 2.4 for 4.5 years after its last maintenance release (which was in December 2008), so by analogy you'll have until 2019 to migrate away from Python 2.7. <0.5wink> Georg From stephen at xemacs.org Sun Apr 7 17:25:12 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 08 Apr 2013 00:25:12 +0900 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> Message-ID: <87a9pa8isn.fsf@uwakimon.sk.tsukuba.ac.jp> Skip Montanaro writes: > It sounds like many people at PyCon are still 2.x users. I suspect we're all still 2.x users at some level. But the question is not "where are the users?" It's "where do the development resources come from?" Pretty clearly, the python-dev crowd has voted with their keyboards. You don't see a lot of complaints from committers about this policy. I gather the general feeling is that at this point supporting Python 2.x is just work that somebody else benefits from. 2.x's EOL was discussed in the past (the thread about "why no 2.8?"), and what we observe is nobody coming forward to maintain Python 2 for the fun of it. People not only work on Python 3 for the fun of it, but they even port packages to Python 3 for the fun of it![1] > Where I work [...] it's not like we haven't seen this coming, but > you can only justify moving so fast with technology that already > works, But by the same token, you should be able to see quite a ways in advance when it's going to stop working, and then you can decide how you want to pay for what you'd been getting for free. As far as I can see, this is a win-win situation for Python 2 users. Stick with Python 2, which you get for free and has evolved into a robust powerful language embedded in a very rich ecosystem of add-on packages. It's open source, so you can maintain it yourself if necessary -- but it mostly *won't* be necessary. Or migrate to Python 3, which you get for free, is a better language, and whose ecosystem is advancing at a good clip. And it is much more fun to work with in many ways. Sorry-no-free-ponies-here-ly y'rs, Footnotes: [1] FVO "fun" including "people who have done me a good turn will be happy to see this done". From rdmurray at bitdance.com Sun Apr 7 17:48:28 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Sun, 07 Apr 2013 11:48:28 -0400 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <87a9pa8isn.fsf@uwakimon.sk.tsukuba.ac.jp> References: <51609C0D.8000106@python.org> <87a9pa8isn.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130407154829.3CE6D250BC8@webabinitio.net> On Mon, 08 Apr 2013 00:25:12 +0900, "Stephen J. Turnbull" wrote: > 2.x's EOL was discussed in the past (the thread about "why no 2.8?"), > and what we observe is nobody coming forward to maintain Python 2 for > the fun of it. People not only work on Python 3 for the fun of it, > but they even port packages to Python 3 for the fun of it![1] Indeed. As one of the people who regularly makes commits to Python, I can say that not applying bug fixes to 2.7 will be a big relief. Having to patch 2.7 roughly doubles the time it takes to commit a fix (much more if the fix doesn't apply cleanly), and I find myself more and more likely to say "well, it's been that way in Python2 for a long while, fixing it there is more likely to break things than it is to improve things, so let's not backport". Or, as gps said, just leaving the issue open to see if anyone else is willing to put in the effort to backport it. I am likely to continue to consider backporting fixes (I mostly do stdlib stuff) until Benjamin stops issuing bugfix releases, but the bar for a fix getting backported will continue to rise, and by the time of 3.4 my behavior may well be almost indistinguishable from those who are deciding to stop backporting fixes at the 3.4 boundary :) As others have pointed out, we are not talking about the end of 2.7, just of the end of python-dev doing 2.7 bugfix releases. 2.7 will live on longer than even 2.3/2.4 did, I expect, and I personally have no problem with that. My primary customers *are* using Python3, by the way. But I and they still use Python2 for lots of things, and will probably do so for a while yet. So I can also speak from a customer/consultant perspective and say that I have no problem with the impending end of 2.7 bugfix releases. In fact (except for IDLE, which I don't use myself but I really want to see improved), I would be fine if this *had* been the last 2.7 bugfix release :) --David From martin at v.loewis.de Sun Apr 7 18:52:07 2013 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 07 Apr 2013 18:52:07 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> <51617A75.8040506@stackless.com> Message-ID: <5161A437.4060100@v.loewis.de> Am 07.04.13 16:58, schrieb Gregory P. Smith: > We don't need to close the 2.7 branch to commits and bug fixes. Ever. I wouldn't want this to happen, actually. People making changes to the 2.7 branch will want to see them released some day. The expectation is on the release people to actually make the releases. I personally want to see a fixed date when I can stop making Windows releases of 2.7, and uninstall Visual Studio 2008. So when we (Benjamin specifically) announce an end to bug fixing 2.7, I'd really like to see the branch closed for bug fixing. It may well be that another clone of cpython is established that gets bugs fixed (and perhaps even new features), but I would rather that branch not be hg.python.org/cpython. Regards, Martin From stefan_ml at behnel.de Sun Apr 7 19:16:54 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 07 Apr 2013 19:16:54 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <51617A75.8040506@stackless.com> References: <51609C0D.8000106@python.org> <51617A75.8040506@stackless.com> Message-ID: Christian Tismer, 07.04.2013 15:53: > But I think every employee (including you) can quite easily put some pressure > on his company by claiming that Python 2.x is a dead end, and everybody is > about to move on to 3.x. > This does not have to be true, I just recognize that by claiming it and > doing it with your projects, the movement becomes a reality. Just say > that we all need to move on and cannot care about companies that ignore > this necessity. > > I agree it is hard to push things forward, when certain tools are just > supporting 2.x. My way to get over this is ranting, and porting some > things, and claiming it was a cake walk. A lie, but it helped. +1, although I'd rather call it a self-fulfilling prophecy than a lie. Stefan From guido at python.org Sun Apr 7 20:02:05 2013 From: guido at python.org (Guido van Rossum) Date: Sun, 7 Apr 2013 11:02:05 -0700 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> Message-ID: I have spent many years in industry working for large companies that have big, successful internal Python code bases, with dependencies on large numbers of external packages. From talking about colleagues about migrating to new language versions, several issues come forward. They all conspire to make it hard to move forward, but not impossible. - A single external package that doesn't yet support the new version can block migration until it has been ported or been replaced by something else. - Many third party packages change their API as they move forward (sometimes they are forced by changes in the language). This means trouble in the backwards compatibility department. - A migration takes a lot of effort. You have to put several engineers on it full-time who could otherwise be developing new features; and it disturbs the development activities of many other engineers, who will be asked to fix tests, decide whether something is still used, and so on. - There are probably parts of the codebase that depend extensively on some feature that doesn't work in the new version, e.g. str/unicode equivalency. Yes, that code is probably buggy, but it is heavily used, and rewriting it means changing interfaces between different components, which in turn requires more rewrites. - There are probably different departments within the company that move at different speeds. It is extremely difficult to get everyone to switch at the same time. So you have to come up with some kind of gradual transition plan, which will probably make the effort even more work, by requiring backwards compatibility for some internal interfaces. - Even if everything goes extremely smoothly, you can still count on some interruption of production -- outages, performance degradations, all the things that end users notice and gripe about. The prospect of this makes managers *very* uncomfortable. But, despite all this, migrations happen all the time, and I am sure that Python 3 will prevail as time progresses. For many *users*, Python 3 may be a distraction. But for most *developers*, maintaining 2.7 is a distraction. By and large, users of 2.7 don't need new features, they just need it to keep working. And it does, of course. (At the risk of a flawed analogy: Windows XP is still the best version of Windows for hardware built when XP was current.) But perhaps we could change the focus for 2.7 development a bit: instead of fixing bugs (or bickering about whether something is a bug fix or a new feature) we could limit changes to ensuring that it works on newer platforms. Martin mentioned that building 2.7 for Windows with the same toolchain that was used for the 2.7.0 release is getting more and more problematic. I'm not sure, but I could imagine similar problems for future versions of OS X and even Linux (though the Linux distributions typically take care of issues themselves). There's not much of a point in fixing bugs that always existed in 2.7, since must 2.7 users are by now used to working around these. However, I do see a point in supporting builds targeting newer OS versions. This won't be much of a relief for Martin, but it might be one of the best ways to interpret "support" of Python 2.7 for the next several years. I would also support having a 3rd party doing this and sell the binaries for a small fee (ActiveState started out this way) -- but it would still behoove us to have the necessary build files in the core repo. Some final words: if any of the alternate Python implementations (IronPython, Jython, PyPy, even Cython) are feeling down on Python 3 because they do not have enough volunteers to help with the port (even though at least for IronPython and Jython, the new str/bytes model is much more suitable than the old), they should apply to the PSF for funding. I believe PyPy is already in the process of doing so. -- --Guido van Rossum (python.org/~guido) From rdmurray at bitdance.com Sun Apr 7 20:09:34 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Sun, 07 Apr 2013 14:09:34 -0400 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <20130407154829.3CE6D250BC8@webabinitio.net> References: <51609C0D.8000106@python.org> <87a9pa8isn.fsf@uwakimon.sk.tsukuba.ac.jp> <20130407154829.3CE6D250BC8@webabinitio.net> Message-ID: <20130407180934.E7885250BCA@webabinitio.net> On Sun, 07 Apr 2013 11:48:28 -0400, "R. David Murray" wrote: > (much more if the fix doesn't apply cleanly), and I find myself more and > more likely to say "well, it's been that way in Python2 for a long while, > fixing it there is more likely to break things than it is to improve > things, so let's not backport". Or, as gps said, just leaving the issue Having sent this, I noticed that it is actually a significant point that no one else has raised out, and is worth emphasising. When we fix bugs, there is always a backward compatibility estimation that goes into the fix, and whether to even make the fix in the bug fix release: what are the chances that fixing this bug will break currently working code, versus the chances that currently broken code will start working correctly? If the chance of breakage outweighs the good done, we don't apply the fix to the maintenance release. The longer that a maintenance release is in the field, the higher the probability that fixing a bug will break existing working code. So even *without* a set maintenance end date, the number of bug fixes *should* decline over time. Five years is a long time. By that point, even regardless of any maintenance commitment concerns, it is probably best to stop fixing anything except security bugs *anyway*, as a service to the user community :) --David From martin at v.loewis.de Sun Apr 7 21:50:08 2013 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 07 Apr 2013 21:50:08 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> Message-ID: <5161CDF0.8010007@v.loewis.de> > But perhaps we could change the focus for 2.7 development a bit: > instead of fixing bugs (or bickering about whether something is a bug > fix or a new feature) we could limit changes to ensuring that it works > on newer platforms. Martin mentioned that building 2.7 for Windows > with the same toolchain that was used for the 2.7.0 release is getting > more and more problematic. For Windows, I don't see a way to achieve this. With the current setup of the Microsoft C runtime library, we need to continue to build with VS 2008 "forever", in the 2.7 branch. While it would be possible to include project files for VS 2012 (and fix the few places where the code doesn't work on VS 2012), this wouldn't help: If we release 2.7.5 (say) built with VS 2012 (say), then existing third-party extension modules may break (depending how precisely they use the CRT). Likewise, if a user choses to rebuild Python with VS 2012 themselves, they really ought to rebuild all extension modules that they use as well. They probably won't recognize this requirement, and then debug difficult-to-understand issues. IOW, to update the tool chain, we would really have to call it python28.dll (or start with a new approach of calling it python27vs11.dll, which would create distinct universes where each extension needs to be built for each universe). So I believe that extension building is becoming more and more painful on Windows for Python 2.7 as time passes (and it is already way more painful than it is on Linux), and I see no way to do much about that. The "stable ABI" would have been a solution, but it's too late now for 2.7. Regards, Martin From barry at python.org Sun Apr 7 23:29:02 2013 From: barry at python.org (Barry Warsaw) Date: Sun, 7 Apr 2013 17:29:02 -0400 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: Message-ID: <20130407172902.73722e29@limelight.wooz.org> On Apr 06, 2013, at 05:02 PM, Benjamin Peterson wrote: >This means we need to talk about how many more 2.7 releases there are >going to be. I'm all for putting stakes in the ground and clearly describing the future life of Python 2.7, rather than the current indefinite status quo. We talked about this at Pycon, and a final maintenance release of 2.7 when 3.4 is released, plus a few years of security-only source-only releases after that seems entirely reasonable. I would like to make a definitive statement as to 2.7's EOL because I think that will spur more people to work on porting. Just as 3.3 makes porting easier than 3.2, I expect that as more people see the writing on the wall, and we've reached the Python 3 tipping point (e.g. "wos" now means "wall of superpowers :) we'll get even more feedback on porting difficulties that can be alleviated in 3.4. (Example, I was against re-adding the u-prefix, but I was wrong!) Thankfully, after October, I won't have to worry about 2.6 any more. -Barry From barry at python.org Sun Apr 7 23:41:21 2013 From: barry at python.org (Barry Warsaw) Date: Sun, 7 Apr 2013 17:41:21 -0400 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <7DF6FD0E-D18C-412F-8149-9A2AED35548C@gmail.com> References: <7DF6FD0E-D18C-412F-8149-9A2AED35548C@gmail.com> Message-ID: <20130407174121.79043841@limelight.wooz.org> On Apr 06, 2013, at 06:54 PM, Raymond Hettinger wrote: >At this year's Pycon keynote, I surveyed the crowd (approx 2500 people) >and all almost everyone indicated that they had tried out Python 3.x >and almost no one was using it in production or writing code for it. >That indicates that Python 2.7 will continue to be important for a good >while. Now that porting has reached the top of the food chain (e.g. Twisted, Django) I think these numbers will change. Some from porters, but also from new projects which can start with a clean slate and avoid endless UncodeErrors and rafts of other problems. This will produce downward pressure on lagging libraries to adopt Python 3 or get left behind, and that should increase the momentum. Python 3 *is* being used in production, but today it's limited to new code bases and ports where all the dependencies are already there. Now we're identifying key bottlenecks, such as (for us) Xapian, and places in the language or libraries where more help is needed. Some bottlenecks have already been fixed (e.g. for us, dbus and OAuth, where the most popular library is already abandoned upstream for 4 years, but there is thankfully a great replacement that's Python 3 compatible). I talked to someone at Pycon who was still using Python 1.5, which is probably older than some of the people on this list ;). -Barry From steve at pearwood.info Mon Apr 8 00:44:15 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 08 Apr 2013 08:44:15 +1000 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <20130407174121.79043841@limelight.wooz.org> References: <7DF6FD0E-D18C-412F-8149-9A2AED35548C@gmail.com> <20130407174121.79043841@limelight.wooz.org> Message-ID: <5161F6BF.50308@pearwood.info> On 08/04/13 07:41, Barry Warsaw wrote: > I talked to someone at Pycon who was still using Python 1.5, which is probably > older than some of the people on this list ;). Awesome! :-) -- Steven From stephen at xemacs.org Mon Apr 8 02:27:23 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 08 Apr 2013 09:27:23 +0900 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <20130407172902.73722e29@limelight.wooz.org> References: <20130407172902.73722e29@limelight.wooz.org> Message-ID: <874nfh989g.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > I would like to make a definitive statement as to 2.7's EOL because > I think that will spur more people to work on porting. I have to agree with the people who say that it's not a major spur. Internal support for existing Python 2.7 installations is by now quite a bit less than a full-time job (at least the part corresponding to what python-dev does for a version in late maintenance releases[1]). I don't see how it makes the choice between sticking with 2.7 vs. contributing ports to 3.x more stark than it already is. Of course it does free up core developer time, especially the release engineers. In maintaining 2.7 past 2015, are core developers really doing anything that a business can't do cheaply and with maximum social benefit?[2] Footnotes: [1] Maybe the Windows build process brings it close to that order of magnitude, if users are building internally. But this should be a saleable product. [2] It has been proved that open source development styles *do* provide benefits *beyond* what business can profitably provide, because of network externalities among developers and users, especially the ability of user-developers to contribute. But these benefits mostly disappear when the developer-vs-user externality disappears because the spec is fixed, and the implementation 99.44% corresponds to the spec. From ncoghlan at gmail.com Mon Apr 8 02:59:18 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 8 Apr 2013 10:59:18 +1000 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> <51617A75.8040506@stackless.com> Message-ID: On Mon, Apr 8, 2013 at 12:58 AM, Gregory P. Smith wrote: > You're not looking at it from the users perspective. They see: > > "we are pleased to announce that RHEL 4 will be supported until the year > 3325" > > and continue to use everything that it ships with and only that. its their > own loss for not investing in maintaining infrastructure of their own rather > than investing in a support contract from their vendor but it is a valid > choice none the less. it has nothing to do with what python-dev chooses to > do release wise. Right, people pay companies like Red Hat* good money to support ancient versions of open source software. Upstream doesn't want to do that (because it's tedious and not at all interesting), and *they don't have to*. If people want things that volunteers aren't interested in providing, then they have the option to pay to get the software they want on the platforms they want (note that in the later parts of a supported product's life, even we don't support running ancient versions of RHEL directly on new hardware - we only support running it as a VM inside a supported hypervisor that supports the new hardware). * In case anyone in the thread isn't already aware of my multiple perspectives on this issue, note that I work on internal tools development for Red Hat these days. > We don't need to close the 2.7 branch to commits and bug fixes. Ever. But > most of us will stop caring about making changes to it at some point. +1000 > For > me that point is after 3.4. In terms of most of the stuff I work on that isn't a new feature, it's either obscure enough, different enough between 2.x and 3.x or close enough to the "new feature" line that I don't really care about getting it changed in 2.7. So for me personally, the "stop worrying about fixing 2.7" point is mostly passed already (fixing aspects of the 2.7 *ecosystem* is still thoroughly on my radar, but that's about improving external tools, not CPython itself). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Mon Apr 8 02:59:18 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 8 Apr 2013 10:59:18 +1000 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> <51617A75.8040506@stackless.com> Message-ID: On Mon, Apr 8, 2013 at 12:58 AM, Gregory P. Smith wrote: > You're not looking at it from the users perspective. They see: > > "we are pleased to announce that RHEL 4 will be supported until the year > 3325" > > and continue to use everything that it ships with and only that. its their > own loss for not investing in maintaining infrastructure of their own rather > than investing in a support contract from their vendor but it is a valid > choice none the less. it has nothing to do with what python-dev chooses to > do release wise. Right, people pay companies like Red Hat* good money to support ancient versions of open source software. Upstream doesn't want to do that (because it's tedious and not at all interesting), and *they don't have to*. If people want things that volunteers aren't interested in providing, then they have the option to pay to get the software they want on the platforms they want (note that in the later parts of a supported product's life, even we don't support running ancient versions of RHEL directly on new hardware - we only support running it as a VM inside a supported hypervisor that supports the new hardware). * In case anyone in the thread isn't already aware of my multiple perspectives on this issue, note that I work on internal tools development for Red Hat these days. > We don't need to close the 2.7 branch to commits and bug fixes. Ever. But > most of us will stop caring about making changes to it at some point. +1000 > For > me that point is after 3.4. In terms of most of the stuff I work on that isn't a new feature, it's either obscure enough, different enough between 2.x and 3.x or close enough to the "new feature" line that I don't really care about getting it changed in 2.7. So for me personally, the "stop worrying about fixing 2.7" point is mostly passed already (fixing aspects of the 2.7 *ecosystem* is still thoroughly on my radar, but that's about improving external tools, not CPython itself). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From dholth at gmail.com Mon Apr 8 05:01:13 2013 From: dholth at gmail.com (Daniel Holth) Date: Sun, 7 Apr 2013 23:01:13 -0400 Subject: [Python-Dev] PEP 4XX: pyzaa "Improving Python ZIP Application Support" In-Reply-To: References: <99ec8f3bb91b41a897359beffb3da225@BLUPR03MB035.namprd03.prod.outlook.com> Message-ID: On Wed, Apr 3, 2013 at 1:26 AM, Stefan Behnel wrote: > Brett Cannon, 02.04.2013 19:28: >> On Tue, Apr 2, 2013 at 1:20 PM, Steve Dower wrote: >> >>>> python -m pyzaa pack [-o path/name] [-m module.submodule:callable] [-c] >>> [-w] [-p interpreter] directory: >>>> >>>> ZIP the contents of directory as directory.pyz or [-w] >>> directory.pyzw. Adds the executable flag to the archive. >>>> >>>> ... >>>> >>>> -p interpreter include #!interpreter as the first line of the archive >>> >>> What happens when -p is omitted? I'd hope it would add the interpreter >>> used to create the zip (or at least the major version), but that may not be >>> ideal for some reason that I haven't thought of yet. >> >> Question is whether ``/usr/bin/python3.3`` is better or ``/usr/bin/env >> python3.3``. I vote for the latter since it gets you the right thing >> without having to care about whether the interpreter moved or is being >> hidden by a user-installed interpreter. > > It can't work properly from within a virtualenv when you write > "/usr/bin/python", so using "/usr/bin/env" instead is actually required. > > Stefan Pushed as Draft PEP 441, tooling prototyped (with less than awesome CLI) at https://bitbucket.org/dholth/pyzaa or https://crate.io/packages/pyzaa Thanks, Daniel Holth From tjreedy at udel.edu Sun Apr 7 22:23:37 2013 From: tjreedy at udel.edu (Terry Jan Reedy) Date: Sun, 07 Apr 2013 16:23:37 -0400 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> Message-ID: On 4/7/2013 2:02 PM, Guido van Rossum wrote: > There's not much of a point in fixing bugs that always existed in 2.7, I has been suggested that backporting bugfix patches from current 3.x to 2.7 will make it easier to port from the atest 2.7.x to 3.x. I have no idea how true that is. > since must 2.7 users are by now used to working around these. An exception is Idle, where the workaround is to grit one teeth and perhaps yell 'Idle is broken' or to switch to something else. With PEP343 accepted, this is rapidly being worked on. NEWS for 2.7.5 already has 8 Idle items. Most of the credit goes to Roger Serwy. tjr From nad at acm.org Sun Apr 7 23:54:14 2013 From: nad at acm.org (Ned Deily) Date: Sun, 07 Apr 2013 14:54:14 -0700 Subject: [Python-Dev] The end of 2.7 References: <51609C0D.8000106@python.org> Message-ID: In article , Guido van Rossum wrote: [...] > But perhaps we could change the focus for 2.7 development a bit: > instead of fixing bugs (or bickering about whether something is a bug > fix or a new feature) we could limit changes to ensuring that it works > on newer platforms. Martin mentioned that building 2.7 for Windows > with the same toolchain that was used for the 2.7.0 release is getting > more and more problematic. I'm not sure, but I could imagine similar > problems for future versions of OS X and even Linux (though the Linux > distributions typically take care of issues themselves). That's an excellent point. In the OS X world, my sense is that OS and build tool updates are adopted more quickly by users than in less homogenous platform environments so we've tried to be be pragmatic about supporting new releases of Xcode built tools in Python maintenance branches like 2.7. For example, in 2.7.4 there were some significant changes to building Python and to Distutils to support extension module building with the latest Xcode releases. (That's not to say we are always as timely as we should be.) I don't think there has been any major disagreements that doing those kinds of limited changes for platform support are in scope. There have also been some changes to better support cross-building for platforms that have become more strategically important, like ARM. These have been more controversial but a good case can be made for considering such changes as pragmatic long-term investments. It might be a good idea to have a more formal policy in place going forward for 2.7.x. Beyond build tools is the issue of the components that Python depends on at runtime, primarily third-party libraries. Many of these are also under active development, with schedules and compatibility criteria that differ from those we use. In some cases, security issues will force the rapid adoption of new versions, in other cases, widespread adoption on the platforms we support will force this. Again, pragmatically, we'll need to continue to track these components and support newer versions as necessary in order for maintained versions of Python to remain viable. (And we need to step up the activity in some areas. Tcl/Tk 8.6 is now official and starting to be adopted. Tkinter and IDLE are intimately dependent on Tk and, AFAIK, we don't really have anyone closely following the changes there and their implications for Python. Thankfully, Serhiy has been doing some work with 8.6 already.) -- Ned Deily, nad at acm.org From fuzzyman at gmail.com Mon Apr 8 11:52:30 2013 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 8 Apr 2013 10:52:30 +0100 Subject: [Python-Dev] [Python-checkins] cpython (3.3): Process DEFAULT values in mock side_effect that returns iterator. In-Reply-To: <3ZkGFs6HVrzRNj@mail.python.org> References: <3ZkGFs6HVrzRNj@mail.python.org> Message-ID: On 7 April 2013 14:44, andrew.svetlov wrote: > http://hg.python.org/cpython/rev/18fd64f1de2d > changeset: 83179:18fd64f1de2d > branch: 3.3 > user: Andrew Svetlov > date: Sun Apr 07 16:42:24 2013 +0300 > summary: > Process DEFAULT values in mock side_effect that returns iterator. > > Patch by Michael Ford. > > files: > Lib/unittest/mock.py | 2 ++ > Lib/unittest/test/testmock/testmock.py | 4 ++++ > 2 files changed, 6 insertions(+), 0 deletions(-) > > > diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py > --- a/Lib/unittest/mock.py > +++ b/Lib/unittest/mock.py > @@ -904,6 +904,8 @@ > result = next(effect) > if _is_exception(result): > raise result > + if result is DEFAULT: > + result = self.return_value > return result > > ret_val = effect(*args, **kwargs) > diff --git a/Lib/unittest/test/testmock/testmock.py > b/Lib/unittest/test/testmock/testmock.py > --- a/Lib/unittest/test/testmock/testmock.py > +++ b/Lib/unittest/test/testmock/testmock.py > @@ -906,6 +906,10 @@ > self.assertRaises(StopIteration, mock) > self.assertIs(mock.side_effect, this_iter) > > + def test_side_effect_iterator_default(self): > + mock = Mock(return_value=2) > + mock.side_effect = iter([1, DEFAULT]) > + self.assertEqual([mock(), mock()], [1, 2]) > > def test_assert_has_calls_any_order(self): > mock = Mock() > > -- > Repository URL: http://hg.python.org/cpython > > This was committed without a NEWS entry. Michael > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Mon Apr 8 16:25:11 2013 From: brett at python.org (Brett Cannon) Date: Mon, 8 Apr 2013 10:25:11 -0400 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: Message-ID: On Sat, Apr 6, 2013 at 5:02 PM, Benjamin Peterson wrote: > Per my last message, 2.7.4 has at long last been released. I apologize > for the long interval between 2.7.3 and 2.7.4. To create more > determinism in the future, I will be soon updating PEP 373 with > approximate dates of future 2.7 bugfix releases. I will be aiming for > 6 month intervals. > > This means we need to talk about how many more 2.7 releases there are > going to be. At the release of 2.7.0, I thought we promised 5 years of > bugfix maintenance, but my memory may be fuddled. At any rate, 2.7.0 > was released in July 2010, which currently puts us within a few months > of 3 years of maintenance. Over the past year, I've been happy to see > a lot of movement towards 3 including the porting of important > codebases like Twisted and Django. However, there's also no doubt that > 2.x is still widely used. Obviously, there will be people who would be > happy if we kept maintaining 2.7 until 2025, but I think at this > juncture 5 total years of maintenance is reasonable. This means there > will be approximately 4 more 2.7 releases. > > Thoughts? > Since this has ended up with roughly 50 responses, I'm going to try and summarize where things stand for my own benefit. First off, core devs almost all seem fine with declaring an end date to maintaining 2.7 and seeing these last releases happen every 6 months (since Benjamin volunteered and I think Martin and Ned said they are fine with that as well and it's really their call). The question for EOL seems to be whether to do one more release after 3.4 goes out in early 2014 or to see 2.7 through until early 2015. The other question seems to be whether we should lock down the branch so people don't think we will continue to accept patches and such (much like Georg has done with the 3.2 pre-commit hook). So those two points -- where to draw the line and whether to mothball the branch -- seem to be the only open questions. For me, I think a possible compromise might work out. What if we say python-dev will see patches backported until the release following 3.4, but after that the last two releases (which sees us into 2015 as Benjamin originally proposed) will only be patched by contributions from the community? IOW we make it very clear that python-dev considers themselves off the hook after 3.4 in terms of feeling obliged to backport any of their own code, but we are willing to examine and commit patches as provided by external contributors as long as they apply to all applicable branches. E.g. if someone wants something fixed in 2.7 after 3.4 is out they need to supply the patches for both 2.7 and 3.4 so that python-dev is doing nothing more than acting and gatekeepers on the repo and doing the commits on behalf of the patch writer. And then once the final 2.7 release is out we lock it down to make it abundantly clear that python-dev is entirely done with Python 2. -------------- next part -------------- An HTML attachment was scrubbed... URL: From me+python at ixokai.io Mon Apr 8 17:42:06 2013 From: me+python at ixokai.io (Stephen Hansen) Date: Mon, 8 Apr 2013 08:42:06 -0700 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <51617A75.8040506@stackless.com> References: <51609C0D.8000106@python.org> <51617A75.8040506@stackless.com> Message-ID: On Sun, Apr 7, 2013 at 6:53 AM, Christian Tismer wrote: > On 07.04.13 14:10, Skip Montanaro wrote: > > Where I work (a trading firm that uses Python as just one of many > different pieces of technology, not a company where Python is the core > technology upon which the firm is based) we are only just now > migrating from 2.4 to 2.7. I can't imagine we'll have migrated to > Python 3 in two years. It's not like we haven't seen this coming, but > you can only justify moving so fast with technology that already > works, especially if, like Python, you use it with lots of other > packages (most/all of which themselves have to be ported to Python 3) > and in-house software. > > I think the discussion should focus on who's left on 2.x and why, not, > "yeah, releases every six months for the next couple years ought to do > it." > > > > when I read this, I was slightly shocked. You know what? > """ > We are pleased to announce the release of *Python 2.4, final* on November > 30, 2004. > """ > > I know that companies try to save (time? money?) something by not upgrading > software, and this is extremely annoying. > I'm in the same boat as Skip (just now moving from 2.4 to 2.7), and Python *is* a core technology for us. It has nothing really to do with saving time or money, its about priorities. The transition from 2.3 to 2.4 was actually fairly painful (don't ask me why, I don't even remember anymore), but we got stuck on 2.4 not by any specific decision -- it simply worked, and our time was always focused upon solving problems and improving our software itself. Could we have solved our problems easier if we upgraded Python and had new tools? Some, yes. (Some features we have added had me actually walking through third party code bases and backporting it -- converting with to try/finally is an amusing big one for example) For one thing, even with this relatively ancient Python, we almost never ran into bugs. It just worked and worked fine, so when we looked at our development plan the list of feature requests and issues for various customers (especially those that were potential new clients) overrode "infrastructure" upgrades as priorities. However, in a huge system that has many tens of thousands of lines of code, doing a platform upgrade is just a serious endeavor -- and its often not even Python's fault itself, but the reality that it means we're going to be upgrading *everything* and involves a much more involved QA cycle and often runs into third party software. We are finally upgrading now because the time to work around certain bugs in both Python and third-party libraries that no longer support 2.4 are enough for us to say, okay, we finally really do need to get this done. Migration to Python 3 ... IF it ever happens is more of a question then when. That's not a indictment of Python 3 or a problem with the current plan (for what its worth, the bugfix every 6 months until 5 years is up seems totally reasonable). Any new product we do, I'd seriously consider starting from Python 3. (Though PyPy supporting Py3 would help that argument a lot) The case for migrating existing products is a lot harder to make. But I think every employee (including you) can quite easily put some > pressure > on his company by claiming that Python 2.x is a dead end, and everybody is > about to move on to 3.x. > This does not have to be true, I just recognize that by claiming it and > doing it > with your projects, the movement becomes a reality. Just say that we all > need to > move on and cannot care about companies that ignore this necessity. > The thing is, 2.7 works. Some third-party libraries we rely upon have no clear sign for when they will be ported (such as wxPython), and though we are transitioning away from certain others (omniORB for Apache Thrift for example), that process itself is planned to be a gradual thing for the next year, at least. My concern is for the health of my company, and happiness of my customers; I love Python and am an advocate for it, but in my day job, pushing things forward is just about at the bottom of my list of concerns. (Though, our migration to 2.7 is actually part of a long term strategic plan to embrace pypy) And now I go back to lurking. --Stephen -------------- next part -------------- An HTML attachment was scrubbed... URL: From breamoreboy at yahoo.co.uk Mon Apr 8 18:57:28 2013 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 08 Apr 2013 17:57:28 +0100 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> <51617A75.8040506@stackless.com> Message-ID: On 08/04/2013 16:42, Stephen Hansen wrote: > > The thing is, 2.7 works. Some third-party libraries we rely upon have no > clear sign for when they will be ported (such as wxPython), and though > we are transitioning away from certain others (omniORB for Apache Thrift > for example), that process itself is planned to be a gradual thing for > the next year, at least. > From http://wiki.wxpython.org/ProjectPhoenix "WooHoo! Phoenix runs on Python 3!! " > > --Stephen > -- If you're using GoogleCrap? please read this http://wiki.python.org/moin/GoogleGroupsPython. Mark Lawrence From solipsis at pitrou.net Mon Apr 8 19:03:31 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 8 Apr 2013 19:03:31 +0200 Subject: [Python-Dev] [Python-checkins] cpython (3.3): Process DEFAULT values in mock side_effect that returns iterator. References: <3ZkGFs6HVrzRNj@mail.python.org> Message-ID: <20130408190331.21a77e05@pitrou.net> On Mon, 8 Apr 2013 10:52:30 +0100 Michael Foord wrote: > On 7 April 2013 14:44, andrew.svetlov wrote: > > > http://hg.python.org/cpython/rev/18fd64f1de2d > > changeset: 83179:18fd64f1de2d > > branch: 3.3 > > user: Andrew Svetlov > > date: Sun Apr 07 16:42:24 2013 +0300 > > summary: > > Process DEFAULT values in mock side_effect that returns iterator. > > > > Patch by Michael Ford. > > [...] > > This was committed without a NEWS entry. And I wonder who that Michael Ford is :-) Regards Antoine. From skip at pobox.com Mon Apr 8 20:14:33 2013 From: skip at pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2013 13:14:33 -0500 Subject: [Python-Dev] Fwd: SourceForge Project Upgrade Notification In-Reply-To: References: Message-ID: Obviously SourceForge doesn't think the current release interval is short enough. (Emphasis mine.) :-) Skip ---------- Forwarded message ---------- From: SourceForge.net Date: Mon, Apr 8, 2013 at 1:09 PM Subject: SourceForge Project Upgrade Notification To: noreply at in.sf.net Dear SourceForge Project Developer, We wanted to let you know that we've upgraded your project, Python, to the new SourceForge developer platform. The next time you log in, things will look a little different. *We've noticed that your project has been idle for a while. This would be a great time to come back and push out a * *new version.* Visit your project at http://sourceforge.net/projects/python/ The new platform, codenamed "Allura", is a rewrite and redesign of the Open Source development tool set. It provides tighter integration between the various tools to give you a better developer experience. Allura is a project in incubation at the Apache Software Foundation, where people from many different companies are collaborating to make it even better. You can see a quick overview of the platform at http://sourceforge.net/create/#feature-holder and you can see ongoing updates to the platform on our blog, at http://sourceforge.net/blog/tag/updates/ Please let us know if there's anything we can do to make your SourceForge experience better. There's lots of ways to get in touch. You can email us at communityteam at sourceforge.net, on IRC at #sourceforge on Freenode, on our Facebook page at http://facebook.com/SourceForgeNet, on our Google Plus community at https://plus.google.com/communities/108980648585760076156, or via Twitter at @sourceforge. -- SourceForge.net has sent this mailing to you as a registered user of the SourceForge.net site to convey important information regarding your SourceForge.net account or your use of SourceForge.net services. If you have concerns about this mailing please contact our Support team per: http://sourceforge.net/support -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.rodola at gmail.com Mon Apr 8 21:48:23 2013 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Mon, 8 Apr 2013 21:48:23 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> <51617A75.8040506@stackless.com> Message-ID: 2013/4/8 Stephen Hansen : > On Sun, Apr 7, 2013 at 6:53 AM, Christian Tismer > wrote: >> >> On 07.04.13 14:10, Skip Montanaro wrote: >> >> Where I work (a trading firm that uses Python as just one of many >> different pieces of technology, not a company where Python is the core >> technology upon which the firm is based) we are only just now >> migrating from 2.4 to 2.7. I can't imagine we'll have migrated to >> Python 3 in two years. It's not like we haven't seen this coming, but >> you can only justify moving so fast with technology that already >> works, especially if, like Python, you use it with lots of other >> packages (most/all of which themselves have to be ported to Python 3) >> and in-house software. >> >> I think the discussion should focus on who's left on 2.x and why, not, >> "yeah, releases every six months for the next couple years ought to do >> it." >> >> >> when I read this, I was slightly shocked. You know what? >> """ >> We are pleased to announce the release of Python 2.4, final on November >> 30, 2004. >> """ >> >> I know that companies try to save (time? money?) something by not >> upgrading >> software, and this is extremely annoying. > > > I'm in the same boat as Skip (just now moving from 2.4 to 2.7), and Python > *is* a core technology for us. It has nothing really to do with saving time > or money, its about priorities. The transition from 2.3 to 2.4 was actually > fairly painful (don't ask me why, I don't even remember anymore), but we got > stuck on 2.4 not by any specific decision -- it simply worked, and our time > was always focused upon solving problems and improving our software itself. > > Could we have solved our problems easier if we upgraded Python and had new > tools? Some, yes. (Some features we have added had me actually walking > through third party code bases and backporting it -- converting with to > try/finally is an amusing big one for example) > > For one thing, even with this relatively ancient Python, we almost never ran > into bugs. It just worked and worked fine, so when we looked at our > development plan the list of feature requests and issues for various > customers (especially those that were potential new clients) overrode > "infrastructure" upgrades as priorities. > > However, in a huge system that has many tens of thousands of lines of code, > doing a platform upgrade is just a serious endeavor -- and its often not > even Python's fault itself, but the reality that it means we're going to be > upgrading *everything* and involves a much more involved QA cycle and often > runs into third party software. We are finally upgrading now because the > time to work around certain bugs in both Python and third-party libraries > that no longer support 2.4 are enough for us to say, okay, we finally really > do need to get this done. > > Migration to Python 3 ... IF it ever happens is more of a question then > when. > > That's not a indictment of Python 3 or a problem with the current plan (for > what its worth, the bugfix every 6 months until 5 years is up seems totally > reasonable). > > Any new product we do, I'd seriously consider starting from Python 3. > (Though PyPy supporting Py3 would help that argument a lot) The case for > migrating existing products is a lot harder to make. You might also think about rewriting the code so that it "kind of" works on both 2.7 *and* 3.3. By that I mean that your code on Python 3 should not necessarily work but neither it should raise SyntaxError. If from the start you use: - six - "except exception as:" - "__future__ module?s from __future__ import division, print_statement, unicode_literals" - fix warnings signaled by "python -3 app.py" ...and other similar tricks, your ported code is likely to look nicer and more modern and a future porting to Python 3 should be a lot less painful. At least, if the circumstances are right, I personally might see some value in doing so. --- Giampaolo https://code.google.com/p/pyftpdlib/ https://code.google.com/p/psutil/ https://code.google.com/p/pysendfile/ From doko at ubuntu.com Mon Apr 8 22:03:16 2013 From: doko at ubuntu.com (Matthias Klose) Date: Mon, 08 Apr 2013 22:03:16 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> Message-ID: <51632284.8020704@ubuntu.com> Am 07.04.2013 20:02, schrieb Guido van Rossum: > But perhaps we could change the focus for 2.7 development a bit: > instead of fixing bugs (or bickering about whether something is a bug > fix or a new feature) we could limit changes to ensuring that it works > on newer platforms. Martin mentioned that building 2.7 for Windows > with the same toolchain that was used for the 2.7.0 release is getting > more and more problematic. I'm not sure, but I could imagine similar > problems for future versions of OS X and even Linux (though the Linux > distributions typically take care of issues themselves). I would like this new focus :-) Note that for 2.7.4 we did backport the bsddb module to build with recent db-5.x versions, the embedded libffi library to build on new platforms. I would like to see a backport for #17536 too, a change to support new web browsers in an updated runtime environment. I would like to continue to backport cross build changes to 2.7.x, before all these people with Raspberry Pi's get too much annoyed about slow native builds. Support for new targets should be allowed after a review. > There's not much of a point in fixing bugs that always existed in 2.7, > since must 2.7 users are by now used to working around these. However, > I do see a point in supporting builds targeting newer OS versions. The upcoming Ubuntu 13.04 release uses mostly Python 3.3.1 for the desktop images, but still ships with Python 2.7.4 too on the images. For now most third party modules and extensions still have to be available for both versions. For now these binary packages (in the sense of a package in a Linux distribution) are built for Python 2 and 3 from the same source package, so keeping the build procedures and support about the same way helps with this approach. Of course I can patch things locally, but would prefer to push these changes upstream. At the language summit I was surprised to hear about a common subset of backports for other vendor branches. Matthias From skip at pobox.com Mon Apr 8 22:40:26 2013 From: skip at pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2013 15:40:26 -0500 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> <51617A75.8040506@stackless.com> Message-ID: > If from the start you use: > - six ... There's the rub. We are not blessed with Guido's time machine where I work. Much of the Python code we run was written long before six was a gleam in anybody's eye. Heck, some of it was probably written before some active members of python-dev graduated from high school. :-) I'm really amazed at how many people seem to have the impression that porting to Python 3 should be no big deal. Please go back and read Guido's post in this thread from yesterday. He identified many barriers to moving between versions. This is not really a Python-specific problem. All large organizations encounter this, and wind up supporting lots of legacy code, long after its original authors are gone. Go to monster.com and search for COBOL or Ada. As I wrote in my previous message, we are only now moving from 2.4 to 2.7. If moving to Python 3 wasn't going to be much more difficult, I think we would have attempted that. 2.7 Seemed like the better step though, especially considering its compatibility with 2.4 and the fact that it has a lot of things backported from Python 3 to ease the eventual transition to Python 3. Skip From terri at zone12.com Mon Apr 8 22:41:31 2013 From: terri at zone12.com (Terri Oda) Date: Mon, 08 Apr 2013 14:41:31 -0600 Subject: [Python-Dev] Python accepted as a GSoC 2013 mentoring organization! Message-ID: <51632B7B.1020004@zone12.com> Congratulations all; we've been accepted as a mentoring organization for Google Summer of Code 2013! Students can check out our ideas page here: http://wiki.python.org/moin/SummerOfCode/2013 If you're interested in mentoring with the PSF and aren't already on the 2013 mentors mailing list, please get in touch with me ASAP. Terri From tseaver at palladion.com Mon Apr 8 23:08:38 2013 From: tseaver at palladion.com (Tres Seaver) Date: Mon, 08 Apr 2013 17:08:38 -0400 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> <51617A75.8040506@stackless.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/08/2013 04:40 PM, Skip Montanaro wrote: > I'm really amazed at how many people seem to have the impression that > porting to Python 3 should be no big deal. FWIW, the effort of porting the "modern" bits of the Zope ecosystem (the ones I still use in Pyramid apps today, meaning the component architecture, the ZODB, and a few others) soaked up basically all of my FLOSS time between the two Santa Clara PyCons. To be fair, some of that effort went into improving test coverage, docs, etc., to ensure that the apps running against the ported librarties wouldn't break, even on Python2: but is was *not* a trivial effort. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlFjMdYACgkQ+gerLs4ltQ62mACfSxdVNlTpSusR5MGMmuIw7lhf 3yIAoIJd6P8KoewUAjJnViuziWQWPHb8 =Bpul -----END PGP SIGNATURE----- From barry at python.org Tue Apr 9 00:05:23 2013 From: barry at python.org (Barry Warsaw) Date: Mon, 8 Apr 2013 18:05:23 -0400 Subject: [Python-Dev] The end of 2.7 In-Reply-To: References: <51609C0D.8000106@python.org> <51617A75.8040506@stackless.com> Message-ID: <20130408180523.220f9557@anarchist> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On Apr 08, 2013, at 05:08 PM, Tres Seaver wrote: >FWIW, the effort of porting the "modern" bits of the Zope ecosystem (the >ones I still use in Pyramid apps today, meaning the component >architecture, the ZODB, and a few others) soaked up basically all of my >FLOSS time between the two Santa Clara PyCons. > >To be fair, some of that effort went into improving test coverage, docs, >etc., to ensure that the apps running against the ported librarties >wouldn't break, even on Python2: but is was *not* a trivial effort. I've ported a ton of stuff, most of it not written by me. The actual job of porting (not counting convincing your manager to let you do it) will either be easy and quick or painful and difficult . It's often hard to know before you start. It almost always comes down to bytes vs. strings, IME. Sometimes, the code you're porting has a clear model and you just have to understand it, and then the porting goes fairly smoothly. Often, the model isn't clear or there *is* no distinction, in which case your life will suck. It's important to realize that everyone doing porting work now is also making Python 3 better by helping to find the pain points in the language and stdlib. The u-prefix is a perfect example of this. .format() on bytes (issue3982) and some of the discussions around issue17445 are two examples where we're at least identifying additional pain, if not yet fixing it. Python 3.3 is easier to port to than 3.2 is. I hope that we'll be able to take all of our experiences and funnel that into 3.4 to make it a better porting target still. Then, even though people will still be using Python 2 when Orlijn is BDFL, we'll at least be making progress. We also want to make 3.4 and future Python 3 releases so compelling that starting new projects in Python 3 will be a no-brainer, and we want to make sure that the batteries, both in the stdlib and 3rd party are up to the task. Eventually Python 2 programmers will be like today's COBOL programmers (which is good for future employment prospects :), but there's more Python 3 code out there waiting to be written than there is existing Python 2 code today. :) Don't worry about what you *can't* port! - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQIcBAEBCAAGBQJRYz8jAAoJEBJutWOnSwa/lsQP/jyIYBzfl9eDXw81Ar4t+Uls 7JOYfQqQXhMXL2f1XLHivCplhso0YBZn2ZepXJ9kLJ4SnO7bPZbo+esD99IoAlhG QXjlvUnrSsc9mQG16HeuQLHjRmiZPg/mT+ACPuv3/ixxZggaKga7Jdcr2Xpo4KIS 89WzG2MWYhVGr3o28AAgSkSN/4fbmXRzjwmox0QjyhPdM6CWbTLO5KkF1cOLIXi/ UyYmlZAwSqVxR6cVRsGVwLStUy417UwZnc0h71IJxgRwgXOZDXzVKPvy/JqbVsK7 451aWB/mTkiaKkp+Qi0ebjZLLmSyEITS8DOurZKIy4Qfppyqy+patwazsY113zAg wdexiFfFc2W4zLFflxTScqULFJulYyL7KzSsQBHPcou1G5nDUVXBXoM+AC71EBtd TE+tyAhZWE38/mfDtOBCJCkJzevzJWF4tpGVgm1Gd5hSU7M8Yb2NQcZLYY0UxzZK weDItJUYTSYEQh6j9nBlZGaX9v62SnXPlZaV2s9/bbVFsHekwYQcEAFlpGUqeUxi KFFzS1GUl8IE8ZcHJDVW6U3YXzBSUKdSBa55IHw5hD+I730WAj8wq8aH7yFhyyzv jyFRdhtEFhapnA2zpxogFGVLPRjl9iOphfVnUR6wNuh7q+226GsqHyaMbkWBM9Cb sg39ITeWH6C0jMMxiGfx =ZAcF -----END PGP SIGNATURE----- From a.cavallo at cavallinux.eu Tue Apr 9 00:32:01 2013 From: a.cavallo at cavallinux.eu (Antonio Cavallo) Date: Mon, 8 Apr 2013 23:32:01 +0100 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <20130408180523.220f9557@anarchist> References: <51609C0D.8000106@python.org> <51617A75.8040506@stackless.com> <20130408180523.220f9557@anarchist> Message-ID: <1E510580-5329-4850-B11B-84E50D7D870C@cavallinux.eu> > It almost always comes down to bytes vs. strings, IME. Cool, next time I have to port an extension written in C/C++ I'll be looking only for bytes vs. strings problems. I knew it was easy. Thanks From a.cavallo at cavallinux.eu Tue Apr 9 12:24:27 2013 From: a.cavallo at cavallinux.eu (a.cavallo at cavallinux.eu) Date: Tue, 09 Apr 2013 12:24:27 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <1E510580-5329-4850-B11B-84E50D7D870C@cavallinux.eu> References: <51609C0D.8000106@python.org> <51617A75.8040506@stackless.com> <20130408180523.220f9557@anarchist> <1E510580-5329-4850-B11B-84E50D7D870C@cavallinux.eu> Message-ID: <0c1301bf54972f55e00939093cebf84b@cavallinux.eu> ;) there's the missing bit. btw apologies if that looked offensive: it wasn't intended. I disagree on "bilingual extension modules are easier". While #ifdef can sort some issues (compiling ones mostly) it won't be much of a help if a module crash (and not much help from testing either). In that case debugging it involves a lot of steps as gatering the core dumps (if available), having a readily build python debug version, a debugger and restoring the crashing system in a similar state. All these steps might not be possible at all (imagine a secured production server). I'm not saying it is not possible but the caused downtime can quickly escalate (think of it in days terms more than hours). These are hidden costs to a company and it is hard to convince anyone to agressively port something to 3.x if it is reliably working on let's say 2.x: especially under time pressure conditions. On the bright side there's some success moving into 2.7: and we can all make sure the move to 3.x will be as small as possible in case in future time/policy constraints are relaxed. Thanks, Antonio On 2013-04-09 00:48, Barry Warsaw wrote: > On Apr 08, 2013, at 11:32 PM, Antonio Cavallo wrote: > >>Cool, next time I have to port an extension written in C/C++ I'll be >> looking >>only for bytes vs. strings problems. I knew it was easy. > > Since I didn't see a smiley, I'll assume that wasn't sarcastic. ;) > > In some ways bilingual extension modules are easier because of > #ifdef, but > the general principle still holds. If you have a clear bytes v. > strings > story, it's not really that difficult to port extension modules > either, at > least IME. > > -Barry From albl500 at york.ac.uk Tue Apr 9 12:37:09 2013 From: albl500 at york.ac.uk (Alex Leach) Date: Tue, 09 Apr 2013 11:37:09 +0100 Subject: [Python-Dev] Accelerating extension module compilation [distutils] Message-ID: Hi, Apologies if this is the wrong place to ask, but thought this question would be relevant to Python core and extension module devs.. This the right place? I've been using distutils to compile C++ extensions / bindings written with Boost.Python, and have been implementing some (often frowned-upon) monkey-patching magic to speed up the compilation process. I was wondering if other Python devs would appreciate the benefit from a distutils-integrated patch of the same functionality, using less-frowned upon programming techniques. More specifically, I have felt it useful during development to incorporate the following functionality into a setupext.py file:- 1. Parallel compilation, by monkey-patching distutils.ccompiler.CCompiler.compile. (A basic solution was provided on StackOverflow[1].) 2. Create a "unity-build"[2]. 3. Only re-compile objects whose source-code / included files have changed. I'll happily share my code (hacks), but before getting too technical with the discussion, I was just wondering whether any of these would be considered useful enough / easy enough to implement without breaking backwards-compatibility, to incorporate into core distutils. Any thoughts? Thanks for your time. Kind regards, Alex [1] - http://stackoverflow.com/questions/11013851/speeding-up-build-process-with-distutils [2] - http://stackoverflow.com/questions/543697/include-all-cpp-files-into-a-single-compilation-unit -- Using Opera's mail client: http://www.opera.com/mail/ From andrew.svetlov at gmail.com Tue Apr 9 14:43:08 2013 From: andrew.svetlov at gmail.com (Andrew Svetlov) Date: Tue, 9 Apr 2013 15:43:08 +0300 Subject: [Python-Dev] [Python-checkins] cpython (3.3): Process DEFAULT values in mock side_effect that returns iterator. In-Reply-To: <20130408190331.21a77e05@pitrou.net> References: <3ZkGFs6HVrzRNj@mail.python.org> <20130408190331.21a77e05@pitrou.net> Message-ID: My bad, sorry On Mon, Apr 8, 2013 at 8:03 PM, Antoine Pitrou wrote: > On Mon, 8 Apr 2013 10:52:30 +0100 > Michael Foord wrote: >> On 7 April 2013 14:44, andrew.svetlov wrote: >> >> > http://hg.python.org/cpython/rev/18fd64f1de2d >> > changeset: 83179:18fd64f1de2d >> > branch: 3.3 >> > user: Andrew Svetlov >> > date: Sun Apr 07 16:42:24 2013 +0300 >> > summary: >> > Process DEFAULT values in mock side_effect that returns iterator. >> > >> > Patch by Michael Ford. >> > > [...] >> >> This was committed without a NEWS entry. > > And I wonder who that Michael Ford is :-) > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov From ronaldoussoren at mac.com Wed Apr 10 17:30:23 2013 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 10 Apr 2013 17:30:23 +0200 Subject: [Python-Dev] How to fix the incorrect shared library extension on linux for 3.2 and newer? In-Reply-To: <515D760D.50608@googlemail.com> References: <515D760D.50608@googlemail.com> Message-ID: On 4 Apr, 2013, at 14:46, Julian Taylor wrote: > The values on macos for these variables still look wrong in 3.3.1rc1: > > ./configure --prefix=/Users/jtaylor/tmp/py3.3.1 --enable-shared > on macosx-10.8-x86_64 > > sys.version_info(major=3, minor=3, micro=1, releaselevel='candidate', serial=1) > SO .so > EXT_SUFFIX .so > SHLIB_SUFFIX 0 > > > the only correct one here is EXT_SUFFIX, SHLIB_SUFFIX should be .dylib (libpython is a .dylib) and .SO possibly too given for what it was used in the past. SO is explicitly defined as being the same as EXT_SUFFIX (in Makefile.pre.in for 3.3), and is gone in default. I'm not sure that SHLIB_SUFFIX is supposed to be because it isn't used other than to calculate the suffix to use for extensions and that shouldn't change on OSX for backward compatiblity reasons, and if it were changed I'd much rather see it changes to something like '.pyext' instead of '.dylib'. But that ship has long sailed, the very limited advantages of using a unique filename suffix for Python extensions isn't worth the very real breakage of actually changing it :-) Oh, and at least setup.py assumes that sysconfig.get_config_var('EXT_SUFFIX').endswith(sysconfig.get_config_var('SHLIB_SUFFIX')). BTW. This is a problem for a lot of the information you can retrieve with sysconfig.get_config_var(), a large subset of the information is only useful during the build/installation of Python itself and as none of them are actually documented using sysconfig.get_config_var() is somewhat of a black art. > > 3.3.0 also returns wrong values > SO .so > EXT_SUFFIX None > SHLIB_SUFFIX "" Could you file an issue on the tracker about this? Ronald > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com From solipsis at pitrou.net Wed Apr 10 23:46:45 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 10 Apr 2013 23:46:45 +0200 Subject: [Python-Dev] cpython (3.3): don't run frame if it has no stack (closes #17669) References: <3ZmHqG4hwyzShm@mail.python.org> Message-ID: <20130410234645.0c4d45cf@pitrou.net> On Wed, 10 Apr 2013 23:01:46 +0200 (CEST) benjamin.peterson wrote: > http://hg.python.org/cpython/rev/35cb75b9d653 > changeset: 83238:35cb75b9d653 > branch: 3.3 > parent: 83235:172f825d7fc9 > user: Benjamin Peterson > date: Wed Apr 10 17:00:56 2013 -0400 > summary: > don't run frame if it has no stack (closes #17669) Wouldn't it be better with a test? Regards Antoine. > > files: > Misc/NEWS | 2 ++ > Objects/genobject.c | 2 +- > 2 files changed, 3 insertions(+), 1 deletions(-) > > > diff --git a/Misc/NEWS b/Misc/NEWS > --- a/Misc/NEWS > +++ b/Misc/NEWS > @@ -12,6 +12,8 @@ > Core and Builtins > ----------------- > > +- Issue #17669: Fix crash involving finalization of generators using yield from. > + > - Issue #17619: Make input() check for Ctrl-C correctly on Windows. > > - Issue #17610: Don't rely on non-standard behavior of the C qsort() function. > diff --git a/Objects/genobject.c b/Objects/genobject.c > --- a/Objects/genobject.c > +++ b/Objects/genobject.c > @@ -178,7 +178,7 @@ > PyObject *yf = NULL; > PyFrameObject *f = gen->gi_frame; > > - if (f) { > + if (f && f->f_stacktop) { > PyObject *bytecode = f->f_code->co_code; > unsigned char *code = (unsigned char *)PyBytes_AS_STRING(bytecode); > > From benjamin at python.org Wed Apr 10 23:57:52 2013 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 10 Apr 2013 17:57:52 -0400 Subject: [Python-Dev] cpython (3.3): don't run frame if it has no stack (closes #17669) In-Reply-To: <20130410234645.0c4d45cf@pitrou.net> References: <3ZmHqG4hwyzShm@mail.python.org> <20130410234645.0c4d45cf@pitrou.net> Message-ID: 2013/4/10 Antoine Pitrou : > On Wed, 10 Apr 2013 23:01:46 +0200 (CEST) > benjamin.peterson wrote: >> http://hg.python.org/cpython/rev/35cb75b9d653 >> changeset: 83238:35cb75b9d653 >> branch: 3.3 >> parent: 83235:172f825d7fc9 >> user: Benjamin Peterson >> date: Wed Apr 10 17:00:56 2013 -0400 >> summary: >> don't run frame if it has no stack (closes #17669) > > Wouldn't it be better with a test? Yes, if I could come up with a non-fragile one. -- Regards, Benjamin From ncoghlan at gmail.com Thu Apr 11 12:28:55 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 11 Apr 2013 20:28:55 +1000 Subject: [Python-Dev] cpython (3.3): don't run frame if it has no stack (closes #17669) In-Reply-To: <20130410234645.0c4d45cf@pitrou.net> References: <3ZmHqG4hwyzShm@mail.python.org> <20130410234645.0c4d45cf@pitrou.net> Message-ID: On 11 Apr 2013 07:49, "Antoine Pitrou" wrote: > > On Wed, 10 Apr 2013 23:01:46 +0200 (CEST) > benjamin.peterson wrote: > > http://hg.python.org/cpython/rev/35cb75b9d653 > > changeset: 83238:35cb75b9d653 > > branch: 3.3 > > parent: 83235:172f825d7fc9 > > user: Benjamin Peterson > > date: Wed Apr 10 17:00:56 2013 -0400 > > summary: > > don't run frame if it has no stack (closes #17669) > > Wouldn't it be better with a test? Benjamin said much the same thing on the issue, but persuading the interpreter to create a frame without a stack that then gets exposed to this code path isn't straightforward :P Cheers, Nick. > > Regards > > Antoine. > > > > > > > files: > > Misc/NEWS | 2 ++ > > Objects/genobject.c | 2 +- > > 2 files changed, 3 insertions(+), 1 deletions(-) > > > > > > diff --git a/Misc/NEWS b/Misc/NEWS > > --- a/Misc/NEWS > > +++ b/Misc/NEWS > > @@ -12,6 +12,8 @@ > > Core and Builtins > > ----------------- > > > > +- Issue #17669: Fix crash involving finalization of generators using yield from. > > + > > - Issue #17619: Make input() check for Ctrl-C correctly on Windows. > > > > - Issue #17610: Don't rely on non-standard behavior of the C qsort() function. > > diff --git a/Objects/genobject.c b/Objects/genobject.c > > --- a/Objects/genobject.c > > +++ b/Objects/genobject.c > > @@ -178,7 +178,7 @@ > > PyObject *yf = NULL; > > PyFrameObject *f = gen->gi_frame; > > > > - if (f) { > > + if (f && f->f_stacktop) { > > PyObject *bytecode = f->f_code->co_code; > > unsigned char *code = (unsigned char *)PyBytes_AS_STRING(bytecode); > > > > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Thu Apr 11 12:38:06 2013 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 11 Apr 2013 12:38:06 +0200 Subject: [Python-Dev] cpython (3.3): don't run frame if it has no stack (closes #17669) In-Reply-To: References: <3ZmHqG4hwyzShm@mail.python.org> <20130410234645.0c4d45cf@pitrou.net> Message-ID: On Thu, Apr 11, 2013 at 12:28 PM, Nick Coghlan wrote: > > On 11 Apr 2013 07:49, "Antoine Pitrou" wrote: >> >> On Wed, 10 Apr 2013 23:01:46 +0200 (CEST) >> benjamin.peterson wrote: >> > http://hg.python.org/cpython/rev/35cb75b9d653 >> > changeset: 83238:35cb75b9d653 >> > branch: 3.3 >> > parent: 83235:172f825d7fc9 >> > user: Benjamin Peterson >> > date: Wed Apr 10 17:00:56 2013 -0400 >> > summary: >> > don't run frame if it has no stack (closes #17669) >> >> Wouldn't it be better with a test? > > Benjamin said much the same thing on the issue, but persuading the > interpreter to create a frame without a stack that then gets exposed to this > code path isn't straightforward :P > > Cheers, > Nick. Maybe it's worth understanding in which circumstances this holds? Please write a test, we'll run into similar issue at some point Im'm sure. Cheers, fijal From benjamin at python.org Thu Apr 11 14:10:17 2013 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 11 Apr 2013 08:10:17 -0400 Subject: [Python-Dev] cpython (3.3): don't run frame if it has no stack (closes #17669) In-Reply-To: References: <3ZmHqG4hwyzShm@mail.python.org> <20130410234645.0c4d45cf@pitrou.net> Message-ID: 2013/4/11 Maciej Fijalkowski : > On Thu, Apr 11, 2013 at 12:28 PM, Nick Coghlan wrote: >> >> On 11 Apr 2013 07:49, "Antoine Pitrou" wrote: >>> >>> On Wed, 10 Apr 2013 23:01:46 +0200 (CEST) >>> benjamin.peterson wrote: >>> > http://hg.python.org/cpython/rev/35cb75b9d653 >>> > changeset: 83238:35cb75b9d653 >>> > branch: 3.3 >>> > parent: 83235:172f825d7fc9 >>> > user: Benjamin Peterson >>> > date: Wed Apr 10 17:00:56 2013 -0400 >>> > summary: >>> > don't run frame if it has no stack (closes #17669) >>> >>> Wouldn't it be better with a test? >> >> Benjamin said much the same thing on the issue, but persuading the >> interpreter to create a frame without a stack that then gets exposed to this >> code path isn't straightforward :P >> >> Cheers, >> Nick. > > Maybe it's worth understanding in which circumstances this holds? > Please write a test, we'll run into similar issue at some point Im'm > sure. Probably not. It's related to cyclic GC. -- Regards, Benjamin From storchaka at gmail.com Thu Apr 11 21:35:53 2013 From: storchaka at gmail.com (Serhiy Storchaka) Date: Thu, 11 Apr 2013 22:35:53 +0300 Subject: [Python-Dev] cpython: Add fast-path in PyUnicode_DecodeCharmap() for pure 8 bit encodings: In-Reply-To: <3Zlg8C6fgrzSyL@mail.python.org> References: <3Zlg8C6fgrzSyL@mail.python.org> Message-ID: On 09.04.13 23:29, victor.stinner wrote: > http://hg.python.org/cpython/rev/53879d380313 > changeset: 83216:53879d380313 > parent: 83214:b7f2d28260b4 > user: Victor Stinner > date: Tue Apr 09 21:53:09 2013 +0200 > summary: > Add fast-path in PyUnicode_DecodeCharmap() for pure 8 bit encodings: > cp037, cp500 and iso8859_1 codecs I deliberately specialized only most typical case in order to reduce maintaining cost. Further optimization of two not the most popular encodings probably not worth additional 25 lines of code. From victor.stinner at gmail.com Thu Apr 11 22:57:38 2013 From: victor.stinner at gmail.com (Victor Stinner) Date: Thu, 11 Apr 2013 22:57:38 +0200 Subject: [Python-Dev] cpython: Add fast-path in PyUnicode_DecodeCharmap() for pure 8 bit encodings: In-Reply-To: References: <3Zlg8C6fgrzSyL@mail.python.org> Message-ID: 2013/4/11 Serhiy Storchaka : > On 09.04.13 23:29, victor.stinner wrote: >> >> http://hg.python.org/cpython/rev/53879d380313 >> changeset: 83216:53879d380313 >> parent: 83214:b7f2d28260b4 >> user: Victor Stinner >> date: Tue Apr 09 21:53:09 2013 +0200 >> summary: >> Add fast-path in PyUnicode_DecodeCharmap() for pure 8 bit encodings: >> cp037, cp500 and iso8859_1 codecs > > I deliberately specialized only most typical case in order to reduce > maintaining cost. Further optimization of two not the most popular encodings > probably not worth additional 25 lines of code. I did the commit while I was trying to avoid usage of PyUnicode_READ_CHAR() and PyUnicode_READ() in unicodeobject.c (slow macros). I was surprised that PyUnicode_DecodeCharmap() has a fast path for Py_UCS2 mapping but not Py_UCS1 mapping. After implementing the fast-path, I realized that only a very few codecs use it. So what do you suggest? Revert the commit to restore the following "hack" (to only have one fast-path)? '\ufffe' ## Widen to UCS2 for optimization The Py_UCS1 fast-path has a small advantage: decoding cannot fail (no need to call an error handler). Victor From guido at python.org Thu Apr 11 23:11:21 2013 From: guido at python.org (Guido van Rossum) Date: Thu, 11 Apr 2013 14:11:21 -0700 Subject: [Python-Dev] casefolding in pathlib (PEP 428) Message-ID: Hey Antoine, Some of my Dropbox colleagues just drew my attention to the occurrence of case folding in pathlib.py. Basically, case folding as an approach to comparing pathnames is fatally flawed. The issues include: - most OSes these days allow the mounting of both case-sensitive and case-insensitive filesystems simultaneously - the case-folding algorithm on some filesystems is burned into the disk when the disk is formatted - case folding requires domain knowledge, e.g. turkish dotless I - normalization is a mess, even on OSX, where it's better defined than elsewhere One or more of them may reply-all to this message with more details. -- --Guido van Rossum (python.org/~guido) From phd at phdru.name Thu Apr 11 23:18:00 2013 From: phd at phdru.name (Oleg Broytman) Date: Fri, 12 Apr 2013 01:18:00 +0400 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: References: Message-ID: <20130411211800.GA25631@iskra.aviel.ru> On Thu, Apr 11, 2013 at 02:11:21PM -0700, Guido van Rossum wrote: > - the case-folding algorithm on some filesystems is burned into the > disk when the disk is formatted Into the partition, I guess, not the physical disc? Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From solipsis at pitrou.net Thu Apr 11 23:27:06 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 11 Apr 2013 23:27:06 +0200 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: References: Message-ID: <20130411232706.51790b7a@pitrou.net> On Thu, 11 Apr 2013 14:11:21 -0700 Guido van Rossum wrote: > Hey Antoine, > > Some of my Dropbox colleagues just drew my attention to the occurrence > of case folding in pathlib.py. Basically, case folding as an approach > to comparing pathnames is fatally flawed. The issues include: > > - most OSes these days allow the mounting of both case-sensitive and > case-insensitive filesystems simultaneously > > - the case-folding algorithm on some filesystems is burned into the > disk when the disk is formatted The problem is that: - if you always make the comparison case-sensitive, you'll get false negatives - if you make the comparison case-insensitive under Windows, you'll get false positives My assumption was that, globally, the number of false positives in case (2) is much less than the number of false negatives in case (1). On the other hand, one could argue that all comparisons should be case-sensitive *and* the proper way to test for "identical" paths is to access the filesystem. Which makes me think, perhaps concrete paths should get a "samefile" method as in os.path.samefile(). Hmm, I think I'm tending towards the latter right now. Regards Antoine. From robertc at robertcollins.net Thu Apr 11 23:29:44 2013 From: robertc at robertcollins.net (Robert Collins) Date: Fri, 12 Apr 2013 09:29:44 +1200 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: <20130411211800.GA25631@iskra.aviel.ru> References: <20130411211800.GA25631@iskra.aviel.ru> Message-ID: On 12 April 2013 09:18, Oleg Broytman wrote: > On Thu, Apr 11, 2013 at 02:11:21PM -0700, Guido van Rossum wrote: >> - the case-folding algorithm on some filesystems is burned into the >> disk when the disk is formatted > > Into the partition, I guess, not the physical disc? CDROMs - Joliet IIRC - so yes, physical disc. -Rob -- Robert Collins Distinguished Technologist HP Cloud Services From phd at phdru.name Thu Apr 11 23:46:00 2013 From: phd at phdru.name (Oleg Broytman) Date: Fri, 12 Apr 2013 01:46:00 +0400 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: References: <20130411211800.GA25631@iskra.aviel.ru> Message-ID: <20130411214600.GA27003@iskra.aviel.ru> On Fri, Apr 12, 2013 at 09:29:44AM +1200, Robert Collins wrote: > On 12 April 2013 09:18, Oleg Broytman wrote: > > On Thu, Apr 11, 2013 at 02:11:21PM -0700, Guido van Rossum wrote: > >> - the case-folding algorithm on some filesystems is burned into the > >> disk when the disk is formatted > > > > Into the partition, I guess, not the physical disc? > > CDROMs - Joliet IIRC - so yes, physical disc. Ah, I've completely forgotten about that one. I was thinking in terms of filesystems. Thank you for reminding! Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From guido at python.org Fri Apr 12 00:42:00 2013 From: guido at python.org (Guido van Rossum) Date: Thu, 11 Apr 2013 15:42:00 -0700 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: <20130411232706.51790b7a@pitrou.net> References: <20130411232706.51790b7a@pitrou.net> Message-ID: On Thu, Apr 11, 2013 at 2:27 PM, Antoine Pitrou wrote: > On Thu, 11 Apr 2013 14:11:21 -0700 > Guido van Rossum wrote: >> Hey Antoine, >> >> Some of my Dropbox colleagues just drew my attention to the occurrence >> of case folding in pathlib.py. Basically, case folding as an approach >> to comparing pathnames is fatally flawed. The issues include: >> >> - most OSes these days allow the mounting of both case-sensitive and >> case-insensitive filesystems simultaneously >> >> - the case-folding algorithm on some filesystems is burned into the >> disk when the disk is formatted > > The problem is that: > - if you always make the comparison case-sensitive, you'll get false > negatives > - if you make the comparison case-insensitive under Windows, you'll get > false positives > > My assumption was that, globally, the number of false positives in case > (2) is much less than the number of false negatives in case (1). > > On the other hand, one could argue that all comparisons should be > case-sensitive *and* the proper way to test for "identical" paths is to > access the filesystem. Which makes me think, perhaps concrete paths > should get a "samefile" method as in os.path.samefile(). > > Hmm, I think I'm tending towards the latter right now. Python on OSX has been using (1) for a decade now without major problems. Perhaps it would be best if the code never called lower() or upper() (not even indirectly via os.path.normcase()). Then any case-folding and path-normalization bugs are the responsibility of the application, and we won't have to worry about how to fix the stdlib without breaking backwards compatibility if we ever figure out how to fix this (which I somehow doubt we ever will anyway :-). Some other issues to be mindful of: - On Linux, paths are really bytes; on Windows (at least NTFS), they are really (16-bit) Unicode; on Mac, they are UTF-8 in a specific normal form (except on some external filesystems). - On Windows, short names are still supported, making the number of ways to spell the path for any given file even larger. -- --Guido van Rossum (python.org/~guido) From cs at zip.com.au Fri Apr 12 01:09:07 2013 From: cs at zip.com.au (Cameron Simpson) Date: Fri, 12 Apr 2013 09:09:07 +1000 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: References: Message-ID: <20130411230907.GA91680@cskk.homeip.net> On 11Apr2013 14:11, Guido van Rossum wrote: | Some of my Dropbox colleagues just drew my attention to the occurrence | of case folding in pathlib.py. Basically, case folding as an approach | to comparing pathnames is fatally flawed. The issues include: | | - most OSes these days allow the mounting of both case-sensitive and | case-insensitive filesystems simultaneously | | - the case-folding algorithm on some filesystems is burned into the | disk when the disk is formatted | | - case folding requires domain knowledge, e.g. turkish dotless I | | - normalization is a mess, even on OSX, where it's better defined than elsewhere Yes, but what's the use case? Specificly, _why_ are you comparing pathnames? To my mind case folding is just one mode of filename conflict. Surely there are others (forbidden characters in some domains, like colons; names significant only to a certain number of characters; an so forth). Thus: what specific problem are you case-folding to address? On a personal basis I would normally address this kind of thing with stat(), avoiding any app knowledge about pathname rules: does this path exist, or are these paths referencing the same file? But of course that doesn't solve the wider issue with Dropbox, where the rules differ per platform and where work can take place disparately on separate hosts. Imagining Dropbox, I'd guess there's a file tree in the backing store. What is its policy? Does it allow multiple files differing only by case? I can imagine that would be bad when the tree is presented on a case insensitive platform (eg Windows, default MacOSX). Taking the view that DropBox should avoid that situation (in what are doubtless several forms), does Dropbox pre-emptively prevent making files with specific names based on what is already in the store, or resolve them to the same object (hard link locally, or simply and less confusingly and more portably, diverting opens to the existing name like a CI filesystem would)? What about offline? That suggests that the forbidden modes should known to the Dropbox app too. Is this the use case for comparing filenames instead of just doing a stat() to the local filesystem or to the remote backing store (via a virtual stat, as it were)? What does Dropbox do if the local app is disabled and a user runs riot in the Dropbox directory, making conflicting names: allowed by the local FS but conflicting in the backing store or on other hosts? What does Dropbox do if a user makes conflicting files independently on different hosts, and then syncs? I just feel you've got a name conflist issue to resolve (and how that's done is partly just policy), and pathlib which offers some facilities related to that kind of thing. But a mismatch between what you actually need to do and what pathlib offers. Fixing your problem isn't necessarily a bugfix for pathlib. I think we need to know the wider context. Cheers, -- Cameron Simpson I had a *bad* day. I had to subvert my principles and kowtow to an idiot. Television makes these daily sacrifices possible. It deadens the inner core of my being. - Martin Donovan, _Trust_ From guido at python.org Fri Apr 12 01:23:47 2013 From: guido at python.org (Guido van Rossum) Date: Thu, 11 Apr 2013 16:23:47 -0700 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: <20130411230907.GA91680@cskk.homeip.net> References: <20130411230907.GA91680@cskk.homeip.net> Message-ID: On Thu, Apr 11, 2013 at 4:09 PM, Cameron Simpson wrote: > On 11Apr2013 14:11, Guido van Rossum wrote: > | Some of my Dropbox colleagues just drew my attention to the occurrence > | of case folding in pathlib.py. Basically, case folding as an approach > | to comparing pathnames is fatally flawed. The issues include: > | > | - most OSes these days allow the mounting of both case-sensitive and > | case-insensitive filesystems simultaneously > | > | - the case-folding algorithm on some filesystems is burned into the > | disk when the disk is formatted > | > | - case folding requires domain knowledge, e.g. turkish dotless I > | > | - normalization is a mess, even on OSX, where it's better defined than elsewhere > > Yes, but what's the use case? Specificly, _why_ are you comparing pathnames? Um, this isn't about Dropbox. This is a warning against the inclusion of any behavior depending on case folding in pathlib, based on experience with case folding at Dropbox (both in the client and in the server). > To my mind case folding is just one mode of filename conflict. > Surely there are others (forbidden characters in some domains, like > colons; names significant only to a certain number of characters; > an so forth). Of course. > Thus: what specific problem are you case-folding to address? Why Dropbox is folding case really doesn't matter. But we have to deal with it because users expect unreasonable things, such as having two files, "readme" and "README", on a Linux box, and then syncing both files to a box running Windows or OSX. (There are many other edge cases, most not involving Linux at all.) We can't always os os.stat() because some of this logic runs on a box where the files don't exist (e.g. the server, or the Linux box in the above example). > On a personal basis I would normally address this kind of thing > with stat(), avoiding any app knowledge about pathname rules: does > this path exist, or are these paths referencing the same file? But > of course that doesn't solve the wider issue with Dropbox, where > the rules differ per platform and where work can take place disparately > on separate hosts. You seem to be completely misunderstanding me. I am not asking for help solving our problem. I am giving advice to avoid baking the wrong solution to this class of problems into a new stdlib module. > Imagining Dropbox, I'd guess there's a file tree in the backing store. > What is its policy? Does it allow multiple files differing only by case? > I can imagine that would be bad when the tree is presented on a case > insensitive platform (eg Windows, default MacOSX). You got the basic idea, but we can't just refuse to sync files that might be problematic on some other box. Suppose someone is using Dropbox just as a backup service for their Linux box. They shouldn't have to worry about case clashes not being backed up. > Taking the view that DropBox should avoid that situation (in what > are doubtless several forms), does Dropbox pre-emptively prevent > making files with specific names based on what is already in the > store, or resolve them to the same object (hard link locally, or > simply and less confusingly and more portably, diverting opens to > the existing name like a CI filesystem would)? We have lots of different solutions based on the specific situations. > What about offline? That suggests that the forbidden modes should > known to the Dropbox app too. Is this the use case for comparing > filenames instead of just doing a stat() to the local filesystem > or to the remote backing store (via a virtual stat, as it were)? Again, please don't try to solve our problem for us. > What does Dropbox do if the local app is disabled and a user runs > riot in the Dropbox directory, making conflicting names: allowed > by the local FS but conflicting in the backing store or on other > hosts? > > What does Dropbox do if a user makes conflicting files independently > on different hosts, and then syncs? > > I just feel you've got a name conflist issue to resolve (and how > that's done is partly just policy), and pathlib which offers some > facilities related to that kind of thing. But a mismatch between > what you actually need to do and what pathlib offers. > > Fixing your problem isn't necessarily a bugfix for pathlib. > > I think we need to know the wider context. My reasoning is as follows. If pathlib supports functionality for checking whether two paths spelled differently point to the same file, users are going to rely on that functionality. But if the implementation is based on knowing how and when to case fold, it will definitely have bugs. So I am proposing to either remove that functionality, or to implement it by consulting the filesystem. Which of course has its own set of issues, for example if the file doesn't exist yet, but there are ways to deal with that too. -- --Guido van Rossum (python.org/~guido) From cs at zip.com.au Fri Apr 12 01:30:55 2013 From: cs at zip.com.au (Cameron Simpson) Date: Fri, 12 Apr 2013 09:30:55 +1000 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: References: Message-ID: <20130411233055.GA14408@cskk.homeip.net> On 11Apr2013 16:23, Guido van Rossum wrote: | On Thu, Apr 11, 2013 at 4:09 PM, Cameron Simpson wrote: | > On 11Apr2013 14:11, Guido van Rossum wrote: | > | Some of my Dropbox colleagues just drew my attention to the occurrence | > | of case folding in pathlib.py. Basically, case folding as an approach | > | to comparing pathnames is fatally flawed. [...] | > | > Yes, but what's the use case? Specificly, _why_ are you comparing pathnames? | | Um, this isn't about Dropbox. This is a warning against the inclusion | of any behavior depending on case folding in pathlib, based on | experience with case folding at Dropbox (both in the client and in the | server). Ah. That wasn't so apparent to me. I took you to have tripped over a specific problem that pathlib appeared to be missolving. I've always viewed path normalisation and its ilk as hazard prone and very context dependent, so I tend not to do it if I can help it. | You seem to be completely misunderstanding me. I am not asking for | help solving our problem. I am giving advice to avoid baking the wrong | solution to this class of problems into a new stdlib module. Ok, fine. [...snip lots of stuff now not relevant...] | My reasoning is as follows. If pathlib supports functionality for | checking whether two paths spelled differently point to the same file, | users are going to rely on that functionality. But if the | implementation is based on knowing how and when to case fold, it will | definitely have bugs. So I am proposing to either remove that | functionality, or to implement it by consulting the filesystem. Which | of course has its own set of issues, for example if the file doesn't | exist yet, but there are ways to deal with that too. Personally I'd be for removing it, or making the doco quite blunt about the many possible shortcomings of guessing whether two paths are the same thing without being able to stat() them. I'll back out now. Cheers, -- Cameron Simpson Having been erased, The document you're seeking Must now be retyped. - Haiku Error Messages http://www.salonmagazine.com/21st/chal/1998/02/10chal2.html From Nikolaus at rath.org Fri Apr 12 07:04:45 2013 From: Nikolaus at rath.org (Nikolaus Rath) Date: Thu, 11 Apr 2013 22:04:45 -0700 Subject: [Python-Dev] Destructors and Closing of File Objects Message-ID: <87a9p41gr6.fsf@vostro.rath.org> [ Note: I already asked this on http://stackoverflow.com/questions/15917502 but didn't get any satisfactory answers] Hello, The description of tempfile.NamedTemporaryFile() says: ,---- | If delete is true (the default), the file is deleted as soon as it is | closed. `---- In some circumstances, this means that the file is not deleted after the program ends. For example, when running the following test under py.test, the temporary file remains: ,---- | from __future__ import division, print_function, absolute_import | import tempfile | import unittest2 as unittest | class cache_tests(unittest.TestCase): | def setUp(self): | self.dbfile = tempfile.NamedTemporaryFile() | def test_get(self): | self.assertEqual('foo', 'foo') `---- In some way this makes sense, because this program never explicitly closes the file object. The only other way for the object to get closed would presumably be in the __del__ destructor, but here the language references states that "It is not guaranteed that __del__() methods are called for objects that still exist when the interpreter exits." So everything is consistent with the documentation so far. However, I'm confused about the implications of this. If it is not guaranteed that file objects are closed on interpreter exit, can it possibly happen that some data that was successfully written to a (buffered) file object is lost even though the program exits gracefully, because it was still in the file objects buffer and the file object never got closed? Somehow that seems very unlikely and un-pythonic to me, and the open() documentation doesn't contain any such warnings either. So I (tentatively) conclude that file objects are, after all, guaranteed to be closed. But how does this magic happen, and why can't NamedTemporaryFile() use the same magic to ensure that the file is deleted? Best, -Nikolaus -- ?Time flies like an arrow, fruit flies like a Banana.? PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C From brian at python.org Fri Apr 12 07:44:28 2013 From: brian at python.org (Brian Curtin) Date: Fri, 12 Apr 2013 00:44:28 -0500 Subject: [Python-Dev] Destructors and Closing of File Objects In-Reply-To: <87a9p41gr6.fsf@vostro.rath.org> References: <87a9p41gr6.fsf@vostro.rath.org> Message-ID: On Fri, Apr 12, 2013 at 12:04 AM, Nikolaus Rath wrote: > [ Note: I already asked this on > http://stackoverflow.com/questions/15917502 but didn't get any > satisfactory answers] Sorry, but that's not a reason to repost your question to this list. If you have to ask somewhere else, it would be python-list, aka, comp.lang.python. From dirkjan at ochtman.nl Fri Apr 12 07:53:10 2013 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Fri, 12 Apr 2013 07:53:10 +0200 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: <20130411232706.51790b7a@pitrou.net> References: <20130411232706.51790b7a@pitrou.net> Message-ID: On Thu, Apr 11, 2013 at 11:27 PM, Antoine Pitrou wrote: > Hmm, I think I'm tending towards the latter right now. You might also want to look at what Mercurial does. As a cross-platform filesystem-oriented tool, it has some interesting issues in the department of casefolding. Cheers, Dirkjan From solipsis at pitrou.net Fri Apr 12 10:39:54 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 12 Apr 2013 10:39:54 +0200 Subject: [Python-Dev] casefolding in pathlib (PEP 428) References: <20130411232706.51790b7a@pitrou.net> Message-ID: <20130412103954.4c96451a@pitrou.net> Le Thu, 11 Apr 2013 15:42:00 -0700, Guido van Rossum a ?crit : > On Thu, Apr 11, 2013 at 2:27 PM, Antoine Pitrou > wrote: > > On Thu, 11 Apr 2013 14:11:21 -0700 > > Guido van Rossum wrote: > >> Hey Antoine, > >> > >> Some of my Dropbox colleagues just drew my attention to the > >> occurrence of case folding in pathlib.py. Basically, case folding > >> as an approach to comparing pathnames is fatally flawed. The > >> issues include: > >> > >> - most OSes these days allow the mounting of both case-sensitive > >> and case-insensitive filesystems simultaneously > >> > >> - the case-folding algorithm on some filesystems is burned into the > >> disk when the disk is formatted > > > > The problem is that: > > - if you always make the comparison case-sensitive, you'll get false > > negatives > > - if you make the comparison case-insensitive under Windows, you'll > > get false positives > > > > My assumption was that, globally, the number of false positives in > > case (2) is much less than the number of false negatives in case > > (1). > > > > On the other hand, one could argue that all comparisons should be > > case-sensitive *and* the proper way to test for "identical" paths > > is to access the filesystem. Which makes me think, perhaps concrete > > paths should get a "samefile" method as in os.path.samefile(). > > > > Hmm, I think I'm tending towards the latter right now. > > Python on OSX has been using (1) for a decade now without major > problems. > > Perhaps it would be best if the code never called lower() or upper() > (not even indirectly via os.path.normcase()). Then any case-folding > and path-normalization bugs are the responsibility of the application, > and we won't have to worry about how to fix the stdlib without > breaking backwards compatibility if we ever figure out how to fix this > (which I somehow doubt we ever will anyway :-). Ok, I've taken a look at the code. Right now lower() is used for two purposes: 1. comparisons (__eq__ and __ne__) 2. globbing and matching While (1) could be dropped, for (2) I think we want glob("*.py") to find "SETUP.PY" under Windows. Anything else will probably be surprising to users of that platform. > - On Linux, paths are really bytes; on Windows (at least NTFS), they > are really (16-bit) Unicode; on Mac, they are UTF-8 in a specific > normal form (except on some external filesystems). pathlib is just relying on Python 3's sane handling of unicode paths (thanks to PEP 383). Bytes paths are never used internally. > - On Windows, short names are still supported, making the number of > ways to spell the path for any given file even larger. They are still supported but I doubt they are still relied on (long filenames appeared in Windows 95!). I think in common situations we can ignore their existence. Specialized tools like Mercurial may have to know that they exist, in order to manage potential collisions (but Mercurial isn't really the target audience for pathlib, and I don't think they would be interested in such an abstraction). Regards Antoine. From p.f.moore at gmail.com Fri Apr 12 12:06:30 2013 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 12 Apr 2013 11:06:30 +0100 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: <20130412103954.4c96451a@pitrou.net> References: <20130411232706.51790b7a@pitrou.net> <20130412103954.4c96451a@pitrou.net> Message-ID: On 12 April 2013 09:39, Antoine Pitrou wrote: > Ok, I've taken a look at the code. Right now lower() is used for two > purposes: > > 1. comparisons (__eq__ and __ne__) > 2. globbing and matching > > While (1) could be dropped, for (2) I think we want glob("*.py") to find > "SETUP.PY" under Windows. Anything else will probably be surprising to > users of that platform. If glob("*.py") failed to find SETUP.PY on Windows, that would be a usability disaster. Too many tools still exist that mangle filename case for anything else to be acceptable. For an easy example, the standard Windows ssh client, putty, is distributed as PUTTY.EXE. shutil.which('putty') needs to find that file if it's to be of any practical use. For comparisons, I think naive Windows users would expect __eq__ comparisons to work case insensitively, but Windows users with any level of understanding of cross-platform portability issues would be comfortable with the idea that this is risky. Having said that, currently there aren't any "pathname comparisons" as such, just string comparisons which "clearly" need application handling. In all honesty, I don't think that equality comparison for path *objects* (as opposed to "pathnames" as strings) is necessarily even well defined. If someone has two path objects and tries to compare them for equality, my first question would be whether that's really what they want to do... (But case-sensitive comparison, with copious warnings, is probably a reasonable practical compromise). Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeanpierreda at gmail.com Fri Apr 12 14:06:37 2013 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Fri, 12 Apr 2013 08:06:37 -0400 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: <20130412103954.4c96451a@pitrou.net> References: <20130411232706.51790b7a@pitrou.net> <20130412103954.4c96451a@pitrou.net> Message-ID: On Fri, Apr 12, 2013 at 4:39 AM, Antoine Pitrou wrote: > Ok, I've taken a look at the code. Right now lower() is used for two > purposes: > > 1. comparisons (__eq__ and __ne__) > 2. globbing and matching > > While (1) could be dropped, for (2) I think we want glob("*.py") to find > "SETUP.PY" under Windows. Anything else will probably be surprising to > users of that platform. OT, but, why is .lower() used for case folding in these use-cases instead of .casefold()? -- Devin From solipsis at pitrou.net Fri Apr 12 14:17:43 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 12 Apr 2013 14:17:43 +0200 Subject: [Python-Dev] casefolding in pathlib (PEP 428) References: <20130411232706.51790b7a@pitrou.net> <20130412103954.4c96451a@pitrou.net> Message-ID: <20130412141743.173ff933@pitrou.net> Le Fri, 12 Apr 2013 08:06:37 -0400, Devin Jeanpierre a ?crit : > On Fri, Apr 12, 2013 at 4:39 AM, Antoine Pitrou > wrote: > > Ok, I've taken a look at the code. Right now lower() is used for two > > purposes: > > > > 1. comparisons (__eq__ and __ne__) > > 2. globbing and matching > > > > While (1) could be dropped, for (2) I think we want glob("*.py") to > > find "SETUP.PY" under Windows. Anything else will probably be > > surprising to users of that platform. > > OT, but, why is .lower() used for case folding in these use-cases > instead of .casefold()? Because the code was written before str.casefold() was introduced. But, actually, if we want to approximate Windows' casefolding behaviour, str.lower() may be better. Regards Antoine. From ronaldoussoren at mac.com Fri Apr 12 14:43:42 2013 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Fri, 12 Apr 2013 14:43:42 +0200 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: <20130412103954.4c96451a@pitrou.net> References: <20130411232706.51790b7a@pitrou.net> <20130412103954.4c96451a@pitrou.net> Message-ID: <2CD9807E-1A2E-45C8-A9BA-443CD21B33EC@mac.com> On 12 Apr, 2013, at 10:39, Antoine Pitrou wrote: >> >> >> Perhaps it would be best if the code never called lower() or upper() >> (not even indirectly via os.path.normcase()). Then any case-folding >> and path-normalization bugs are the responsibility of the application, >> and we won't have to worry about how to fix the stdlib without >> breaking backwards compatibility if we ever figure out how to fix this >> (which I somehow doubt we ever will anyway :-). > > Ok, I've taken a look at the code. Right now lower() is used for two > purposes: > > 1. comparisons (__eq__ and __ne__) > 2. globbing and matching > > While (1) could be dropped, for (2) I think we want glob("*.py") to find > "SETUP.PY" under Windows. Anything else will probably be surprising to > users of that platform. Globbing necessarily accesses the filesystem and could in theory do the right thing, except for the minor detail of there not being an easy way to determine of the names in a particular folder are compared case sensitive or not. > >> - On Linux, paths are really bytes; on Windows (at least NTFS), they >> are really (16-bit) Unicode; on Mac, they are UTF-8 in a specific >> normal form (except on some external filesystems). > > pathlib is just relying on Python 3's sane handling of unicode paths > (thanks to PEP 383). Bytes paths are never used internally. At least for OSX the kernel will normalize names for you, at least for HFS+, and therefore two names that don't compare equal with '==' can refer to the same file (for example the NFKD and NFKC forms of L?we). Isn't unicode fun :-) Ronald > >> - On Windows, short names are still supported, making the number of >> ways to spell the path for any given file even larger. > > They are still supported but I doubt they are still relied on (long > filenames appeared in Windows 95!). I think in common situations we can > ignore their existence. Specialized tools like Mercurial may have to > know that they exist, in order to manage potential collisions (but > Mercurial isn't really the target audience for pathlib, and I don't > think they would be interested in such an abstraction). > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com From eliben at gmail.com Fri Apr 12 14:55:00 2013 From: eliben at gmail.com (Eli Bendersky) Date: Fri, 12 Apr 2013 05:55:00 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library Message-ID: Hello python-dev, We're happy to present the revised PEP 435, collecting valuable feedback from python-ideas discussions as well as in-person discussions and decisions made during the latest PyCon language summit. We believe the proposal is now better than the original one, providing both a wider set of features and more convenient ways to use those features. Link to the PEP: http://www.python.org/dev/peps/pep-0435/ [it's also pasted fully below for convenience]. Reference implementation is available as the recently released flufl.enum version 4.0 - you can get it either from PyPi or https://launchpad.net/flufl.enum. flufl.enum 4.0 was developed in parallel with revising PEP 435. Comments welcome, Barry and Eli ---------------------------------- PEP: 435 Title: Adding an Enum type to the Python standard library Version: $Revision$ Last-Modified: $Date$ Author: Barry Warsaw , Eli Bendersky Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 2013-02-23 Python-Version: 3.4 Post-History: 2013-02-23 Abstract ======== This PEP proposes adding an enumeration type to the Python standard library. Specifically, it proposes moving the existing ``flufl.enum`` package by Barry Warsaw into the standard library. Much of this PEP is based on the "using" [1]_ document from the documentation of ``flufl.enum``. An enumeration is a set of symbolic names bound to unique, constant values. Within an enumeration, the values can be compared by identity, and the enumeration itself can be iterated over. Decision ======== TODO: update decision here once pronouncement is made. Status of discussions ===================== The idea of adding an enum type to Python is not new - PEP 354 [2]_ is a previous attempt that was rejected in 2005. Recently a new set of discussions was initiated [3]_ on the ``python-ideas`` mailing list. Many new ideas were proposed in several threads; after a lengthy discussion Guido proposed adding ``flufl.enum`` to the standard library [4]_. During the PyCon 2013 language summit the issue was discussed further. It became clear that many developers want to see an enum that subclasses ``int``, which can allow us to replace many integer constants in the standard library by enums with friendly string representations, without ceding backwards compatibility. An additional discussion among several interested core developers led to the proposal of having ``IntEnum`` as a special case of ``Enum``. The key dividing issue between ``Enum`` and ``IntEnum`` is whether comparing to integers is semantically meaningful. For most uses of enumerations, it's a **feature** to reject comparison to integers; enums that compare to integers lead, through transitivity, to comparisons between enums of unrelated types, which isn't desirable in most cases. For some uses, however, greater interoperatiliby with integers is desired. For instance, this is the case for replacing existing standard library constants (such as ``socket.AF_INET``) with enumerations. This PEP is an attempt to formalize this decision as well as discuss a number of variations that were discussed and can be considered for inclusion. Motivation ========== *[Based partly on the Motivation stated in PEP 354]* The properties of an enumeration are useful for defining an immutable, related set of constant values that have a defined sequence but no inherent semantic meaning. Classic examples are days of the week (Sunday through Saturday) and school assessment grades ('A' through 'D', and 'F'). Other examples include error status values and states within a defined process. It is possible to simply define a sequence of values of some other basic type, such as ``int`` or ``str``, to represent discrete arbitrary values. However, an enumeration ensures that such values are distinct from any others including, importantly, values within other enumerations, and that operations without meaning ("Wednesday times two") are not defined for these values. It also provides a convenient printable representation of enum values without requiring tedious repetition while defining them (i.e. no ``GREEN = 'green'``). Module and type name ==================== We propose to add a module named ``enum`` to the standard library. The main type exposed by this module is ``Enum``. Hence, to import the ``Enum`` type user code will run:: >>> from enum import Enum Proposed semantics for the new enumeration type =============================================== Creating an Enum ---------------- Enumerations are created using the class syntax, which makes them easy to read and write. An alternative creation method is described in `Convenience API`_. To define an enumeration, derive from the ``Enum`` class and add attributes with assignment to their integer values:: >>> from enum import Enum >>> class Colors(Enum): ... red = 1 ... green = 2 ... blue = 3 Enumeration values have nice, human readable string representations:: >>> print(Colors.red) Colors.red ...while their repr has more information:: >>> print(repr(Colors.red)) The enumeration value names are available through the class members:: >>> for member in Colors.__members__: ... print(member) red green blue Let's say you wanted to encode an enumeration value in a database. You might want to get the enumeration class object from an enumeration value:: >>> cls = Colors.red.enum >>> print(cls.__name__) Colors Enums also have a property that contains just their item name:: >>> print(Colors.red.name) red >>> print(Colors.green.name) green >>> print(Colors.blue.name) blue The str and repr of the enumeration class also provides useful information:: >>> print(Colors) >>> print(repr(Colors)) The ``Enum`` class supports iteration. Iteration is defined as the sorted order of the item values:: >>> class FiveColors(Enum): ... pink = 4 ... cyan = 5 ... green = 2 ... blue = 3 ... red = 1 >>> [v.name for v in FiveColors] ['red', 'green', 'blue', 'pink', 'cyan'] Enumeration values are hashable, so they can be used in dictionaries and sets:: >>> apples = {} >>> apples[Colors.red] = 'red delicious' >>> apples[Colors.green] = 'granny smith' >>> apples {: 'granny smith', : 'red delicious'} To programmatically access enumeration values, use ``getattr``:: >>> getattr(Colors, 'red') Comparisons ----------- Enumeration values are compared by identity:: >>> Colors.red is Colors.red True >>> Colors.blue is Colors.blue True >>> Colors.red is not Colors.blue True >>> Colors.blue is Colors.red False Ordered comparisons between enumeration values are *not* supported. Enums are not integers (but see `IntEnum`_ below):: >>> Colors.red < Colors.blue Traceback (most recent call last): ... NotImplementedError >>> Colors.red <= Colors.blue Traceback (most recent call last): ... NotImplementedError >>> Colors.blue > Colors.green Traceback (most recent call last): ... NotImplementedError >>> Colors.blue >= Colors.green Traceback (most recent call last): ... NotImplementedError Equality comparisons are defined though:: >>> Colors.blue == Colors.blue True >>> Colors.green != Colors.blue True Comparisons against non-enumeration values will always compare not equal:: >>> Colors.green == 2 False >>> Colors.blue == 3 False >>> Colors.green != 3 True >>> Colors.green == 'green' False Extending enumerations by subclassing ------------------------------------- You can extend previously defined Enums by subclassing:: >>> class MoreColors(Colors): ... pink = 4 ... cyan = 5 When extended in this way, the base enumeration's values are identical to the same named values in the derived class:: >>> Colors.red is MoreColors.red True >>> Colors.blue is MoreColors.blue True However, these are not doing comparisons against the integer equivalent values, because if you define an enumeration with similar item names and integer values, they will not be identical:: >>> class OtherColors(Enum): ... red = 1 ... blue = 2 ... yellow = 3 >>> Colors.red is OtherColors.red False >>> Colors.blue is not OtherColors.blue True These enumeration values are not equal, nor do they and hence may exist in the same set, or as distinct keys in the same dictionary:: >>> Colors.red == OtherColors.red False >>> len(set((Colors.red, OtherColors.red))) 2 You may not define two enumeration values with the same integer value:: >>> class Bad(Enum): ... cartman = 1 ... stan = 2 ... kyle = 3 ... kenny = 3 # Oops! ... butters = 4 Traceback (most recent call last): ... ValueError: Conflicting enums with value '3': 'kenny' and 'kyle' You also may not duplicate values in derived enumerations:: >>> class BadColors(Colors): ... yellow = 4 ... chartreuse = 2 # Oops! Traceback (most recent call last): ... ValueError: Conflicting enums with value '2': 'green' and 'chartreuse' Enumeration values ------------------ The examples above use integers for enumeration values. Using integers is short and handy (and provided by default by the `Convenience API`_), but not strictly enforced. In the vast majority of use-cases, one doesn't care what the actual value of an enumeration is. But if the value *is* important, enumerations can have arbitrary values. The following example uses strings:: >>> class SpecialId(Enum): ... selector = '$IM($N)' ... adaptor = '~$IM' ... >>> SpecialId.selector >>> SpecialId.selector.value '$IM($N)' >>> a = SpecialId.adaptor >>> a == '~$IM' False >>> a == SpecialId.adaptor True >>> print(a) SpecialId.adaptor >>> print(a.value) ~$IM Here ``Enum`` is used to provide readable (and syntactically valid!) names for some special values, as well as group them together. While ``Enum`` supports this flexibility, one should only use it in very special cases. Code will be most readable when actual values of enumerations aren't important and enumerations are just used for their naming and comparison properties. IntEnum ------- A variation of ``Enum`` is proposed where the enumeration values also subclasses ``int`` - ``IntEnum``. These values can be compared to integers; by extension, enumerations of different types can also be compared to each other:: >>> from enum import IntEnum >>> class Shape(IntEnum): ... circle = 1 ... square = 2 ... >>> class Request(IntEnum): ... post = 1 ... get = 2 ... >>> Shape == 1 False >>> Shape.circle == 1 True >>> Shape.circle == Request.post True However they still can't be compared to ``Enum``:: >>> class Shape(IntEnum): ... circle = 1 ... square = 2 ... >>> class Colors(Enum): ... red = 1 ... green = 2 ... >>> Shape.circle == Colors.red False ``IntEnum`` values behave like integers in other ways you'd expect:: >>> int(Shape.circle) 1 >>> ['a', 'b', 'c'][Shape.circle] 'b' >>> [i for i in range(Shape.square)] [0, 1] For the vast majority of code, ``Enum`` is strongly recommended. Since ``IntEnum`` breaks some semantic promises of an enumeration (by being comparable to integers, and thus by transitivity to other unrelated enumerations), it should be used only in special cases where there's no other choice; for example, when integer constants are replaced with enumerations and backwards compatibility is required with code that still expects integers. Pickling -------- Enumerations created with the class syntax can also be pickled and unpickled:: >>> from enum.tests.fruit import Fruit >>> from pickle import dumps, loads >>> Fruit.tomato is loads(dumps(Fruit.tomato)) True Convenience API --------------- The ``Enum`` class is callable, providing the following convenience API:: >>> Animals = Enum('Animals', 'ant bee cat dog') >>> Animals >>> Animals.ant >>> Animals.ant.value 1 The semantics of this API resemble ``namedtuple``. The first argument of the call to ``Enum`` is the name of the enumeration. The second argument is a source of enumeration value names. It can be a whitespace-separated string of names, a sequence of names or a sequence of 2-tuples with key/value pairs. The last option enables assigning arbitrary values to enumerations; the others auto-assign increasing integers starting with 1. A new class derived from ``Enum`` is returned. In other words, the above assignment to ``Animals`` is equivalent to:: >>> class Animals(Enum): ... ant = 1 ... bee = 2 ... cat = 3 ... dog = 4 Examples of alternative name/value specifications:: >>> Enum('Animals', ['ant', 'bee', 'cat', 'dog']) >>> Enum('Animals', (('ant', 'one'), ('bee', 'two'), ('cat', 'three'), ('dog', 'four'))) The second argument can also be a dictionary mapping names to values:: >>> levels = dict(debug=10, info=20, warning=30, severe=40) >>> Enum('Levels', levels) Proposed variations =================== Some variations were proposed during the discussions in the mailing list. Here's some of the more popular ones. Not having to specify values for enums -------------------------------------- Michael Foord proposed (and Tim Delaney provided a proof-of-concept implementation) to use metaclass magic that makes this possible:: class Color(Enum): red, green, blue The values get actually assigned only when first looked up. Pros: cleaner syntax that requires less typing for a very common task (just listing enumeration names without caring about the values). Cons: involves much magic in the implementation, which makes even the definition of such enums baffling when first seen. Besides, explicit is better than implicit. Using special names or forms to auto-assign enum values ------------------------------------------------------- A different approach to avoid specifying enum values is to use a special name or form to auto assign them. For example:: class Color(Enum): red = None # auto-assigned to 0 green = None # auto-assigned to 1 blue = None # auto-assigned to 2 More flexibly:: class Color(Enum): red = 7 green = None # auto-assigned to 8 blue = 19 purple = None # auto-assigned to 20 Some variations on this theme: #. A special name ``auto`` imported from the enum package. #. Georg Brandl proposed ellipsis (``...``) instead of ``None`` to achieve the same effect. Pros: no need to manually enter values. Makes it easier to change the enum and extend it, especially for large enumerations. Cons: actually longer to type in many simple cases. The argument of explicit vs. implicit applies here as well. Use-cases in the standard library ================================= The Python standard library has many places where the usage of enums would be beneficial to replace other idioms currently used to represent them. Such usages can be divided to two categories: user-code facing constants, and internal constants. User-code facing constants like ``os.SEEK_*``, ``socket`` module constants, decimal rounding modes and HTML error codes could require backwards compatibility since user code may expect integers. ``IntEnum`` as described above provides the required semantics; being a subclass of ``int``, it does not affect user code that expects integers, while on the other hand allowing printable representations for enumeration values:: >>> import socket >>> family = socket.AF_INET >>> family == 2 True >>> print(family) SocketFamily.AF_INET Internal constants are not seen by user code but are employed internally by stdlib modules. These can be implemented with ``Enum``. Some examples uncovered by a very partial skim through the stdlib: ``binhex``, ``imaplib``, ``http/client``, ``urllib/robotparser``, ``idlelib``, ``concurrent.futures``, ``turtledemo``. In addition, looking at the code of the Twisted library, there are many use cases for replacing internal state constants with enums. The same can be said about a lot of networking code (especially implementation of protocols) and can be seen in test protocols written with the Tulip library as well. Differences from PEP 354 ======================== Unlike PEP 354, enumeration values are not defined as a sequence of strings, but as attributes of a class. This design was chosen because it was felt that class syntax is more readable. Unlike PEP 354, enumeration values require an explicit integer value. This difference recognizes that enumerations often represent real-world values, or must interoperate with external real-world systems. For example, to store an enumeration in a database, it is better to convert it to an integer on the way in and back to an enumeration on the way out. Providing an integer value also provides an explicit ordering. However, there is no automatic conversion to and from the integer values, because explicit is better than implicit. Unlike PEP 354, this implementation does use a metaclass to define the enumeration's syntax, and allows for extended base-enumerations so that the common values in derived classes are identical (a singleton model). While PEP 354 dismisses this approach for its complexity, in practice any perceived complexity, though minimal, is hidden from users of the enumeration. Unlike PEP 354, enumeration values should only be tested by identity comparison. This is to emphasize the fact that enumeration values are singletons, much like ``None``. Acknowledgments =============== This PEP describes the ``flufl.enum`` package by Barry Warsaw. ``flufl.enum`` is based on an example by Jeremy Hylton. It has been modified and extended by Barry Warsaw for use in the GNU Mailman [5]_ project. Ben Finney is the author of the earlier enumeration PEP 354. References ========== .. [1] http://pythonhosted.org/flufl.enum/docs/using.html .. [2] http://www.python.org/dev/peps/pep-0354/ .. [3] http://mail.python.org/pipermail/python-ideas/2013-January/019003.html .. [4] http://mail.python.org/pipermail/python-ideas/2013-February/019373.html .. [5] http://www.list.org Copyright ========= This document has been placed in the public domain. Todo ==== * Mark PEP 354 "superseded by" this one, if accepted .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian at python.org Fri Apr 12 15:00:28 2013 From: christian at python.org (Christian Heimes) Date: Fri, 12 Apr 2013 15:00:28 +0200 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: <2CD9807E-1A2E-45C8-A9BA-443CD21B33EC@mac.com> References: <20130411232706.51790b7a@pitrou.net> <20130412103954.4c96451a@pitrou.net> <2CD9807E-1A2E-45C8-A9BA-443CD21B33EC@mac.com> Message-ID: <5168056C.6040603@python.org> Am 12.04.2013 14:43, schrieb Ronald Oussoren: > At least for OSX the kernel will normalize names for you, at least for HFS+, > and therefore two names that don't compare equal with '==' can refer to the > same file (for example the NFKD and NFKC forms of L?we). > > Isn't unicode fun :-) Seriously, the OSX kernel normalizes unicode forms? It's a cool feature and makes sense for the user's POV but ... WTF? Perhaps we should use the platform's API for the job. Does OSX offer an API function to create a case folded and canonical form of a path? Windows has PathCchCanonicalizeEx(). Christian From christian at python.org Fri Apr 12 15:00:28 2013 From: christian at python.org (Christian Heimes) Date: Fri, 12 Apr 2013 15:00:28 +0200 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: <2CD9807E-1A2E-45C8-A9BA-443CD21B33EC@mac.com> References: <20130411232706.51790b7a@pitrou.net> <20130412103954.4c96451a@pitrou.net> <2CD9807E-1A2E-45C8-A9BA-443CD21B33EC@mac.com> Message-ID: <5168056C.6040603@python.org> Am 12.04.2013 14:43, schrieb Ronald Oussoren: > At least for OSX the kernel will normalize names for you, at least for HFS+, > and therefore two names that don't compare equal with '==' can refer to the > same file (for example the NFKD and NFKC forms of L?we). > > Isn't unicode fun :-) Seriously, the OSX kernel normalizes unicode forms? It's a cool feature and makes sense for the user's POV but ... WTF? Perhaps we should use the platform's API for the job. Does OSX offer an API function to create a case folded and canonical form of a path? Windows has PathCchCanonicalizeEx(). Christian From ronaldoussoren at mac.com Fri Apr 12 15:10:42 2013 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Fri, 12 Apr 2013 15:10:42 +0200 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: <5168056C.6040603@python.org> References: <20130411232706.51790b7a@pitrou.net> <20130412103954.4c96451a@pitrou.net> <2CD9807E-1A2E-45C8-A9BA-443CD21B33EC@mac.com> <5168056C.6040603@python.org> Message-ID: On 12 Apr, 2013, at 15:00, Christian Heimes wrote: > Am 12.04.2013 14:43, schrieb Ronald Oussoren: >> At least for OSX the kernel will normalize names for you, at least for HFS+, >> and therefore two names that don't compare equal with '==' can refer to the >> same file (for example the NFKD and NFKC forms of L?we). >> >> Isn't unicode fun :-) > > Seriously, the OSX kernel normalizes unicode forms? It's a cool feature > and makes sense for the user's POV but ... WTF? IIRC only for HFS+ filesystems, it is possible to access files on an NFS share where the filename encoding isn't UTF-8. > > Perhaps we should use the platform's API for the job. Does OSX offer an > API function to create a case folded and canonical form of a path? > Windows has PathCchCanonicalizeEx(). This would have to be done on a per path element case, because every directory in a file's path could be on a separate filesystem with different conventions (HFS+, HFS+ case sensitive, NFS mounted unix filesystem). I have found sample code that can determine if a directory is on a case sensitive filesystem (attached to , doesn't work in a 64-binary but I haven't check yet why is doesn't work there). I don'tknow if there is a function to determine the filesystem encoding, I guess assuming that the special casing is only needed for HFS+ variants could work but I'd have test that. Ronald From dirkjan at ochtman.nl Fri Apr 12 15:31:12 2013 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Fri, 12 Apr 2013 15:31:12 +0200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: Message-ID: On Fri, Apr 12, 2013 at 2:55 PM, Eli Bendersky wrote: > Ordered comparisons between enumeration values are *not* supported. Enums > are > not integers (but see `IntEnum`_ below):: > > >>> Colors.red < Colors.blue > Traceback (most recent call last): > ... > NotImplementedError > >>> Colors.red <= Colors.blue > Traceback (most recent call last): > ... > NotImplementedError > >>> Colors.blue > Colors.green > Traceback (most recent call last): > ... > NotImplementedError > >>> Colors.blue >= Colors.green > Traceback (most recent call last): > ... > NotImplementedError I like much of this PEP, but the exception type for this case seems odd to me. Wouldn't a TypeError be more appropriate here? Somewhat like this: >>> 'a' - 'b' Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for -: 'str' and 'str' Cheers, Dirkjan From rdmurray at bitdance.com Fri Apr 12 15:58:48 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 12 Apr 2013 09:58:48 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: Message-ID: <20130412135848.BD9DA250BD9@webabinitio.net> On Fri, 12 Apr 2013 05:55:00 -0700, Eli Bendersky wrote: > Link to the PEP: http://www.python.org/dev/peps/pep-0435/ [it's also pasted > fully below for convenience]. This looks great. There's just one bit I don't understand. I'm sure it was discussed in the python-ideas thread, but the discussion of it in the PEP does not provide any motivation for the decision. > The ``Enum`` class supports iteration. Iteration is defined as the > sorted order of the item values:: > > >>> class FiveColors(Enum): > ... pink = 4 > ... cyan = 5 > ... green = 2 > ... blue = 3 > ... red = 1 > >>> [v.name for v in FiveColors] > ['red', 'green', 'blue', 'pink', 'cyan'] [...] > Ordered comparisons between enumeration values are *not* supported. Enums > are > not integers (but see `IntEnum`_ below):: > > >>> Colors.red < Colors.blue > Traceback (most recent call last): > ... > NotImplementedError > >>> Colors.red <= Colors.blue > Traceback (most recent call last): > ... > NotImplementedError > >>> Colors.blue > Colors.green > Traceback (most recent call last): > ... > NotImplementedError > >>> Colors.blue >= Colors.green > Traceback (most recent call last): > ... > NotImplementedError This is the part that I don't understand. Enums *clearly* have an ordering, since the iteration order is defined and stable. Why should I not be allowed to compare values from the same Enum type? There are certainly use cases where that is very useful. To give you a concrete use case: consider response codes from a client server application constructed the way typical internet protocols are. We might have: class MyAppCode(Enum): ok = 200 command_complete = 201 status_response = 202 help_response = 203 service_ready = 204 signoff_accepted = 205 temporary_failure = 400 service_not_available = 401 server_error = 402 out_of_resources = 403 error = 500 syntax_error = 501 command_not_implemented = 502 access_denied = 503 resource_not_found = 504 It can be quite handy to be able to say things like: code = myapp.operation(opstring) if MyAppCode.temporary_failure < code < MyAppCode.error: myframework.requeue(opstring, code=code) return False elif code > MyAppCode.error: raise AppError(code) .... In talking to an existing internet protocol it would be natural to use IntEnum and this issue would not arise, but I have recently worked on an application that had *exactly* the above sort of enumeration used internally, when it would have been totally appropriate to use Enum rather than IntEnum. The ap has several places where an ordered comparison against the enum is used to check if a code is in the error range or not. --David From murman at gmail.com Fri Apr 12 16:03:48 2013 From: murman at gmail.com (Michael Urman) Date: Fri, 12 Apr 2013 09:03:48 -0500 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: Message-ID: > You may not define two enumeration values with the same integer value:: > > >>> class Bad(Enum): > ... cartman = 1 > ... stan = 2 > ... kyle = 3 > ... kenny = 3 # Oops! > ... butters = 4 > Traceback (most recent call last): > ... > ValueError: Conflicting enums with value '3': 'kenny' and 'kyle' > > You also may not duplicate values in derived enumerations:: > > >>> class BadColors(Colors): > ... yellow = 4 > ... chartreuse = 2 # Oops! > Traceback (most recent call last): > ... > ValueError: Conflicting enums with value '2': 'green' and 'chartreuse' > Is there a convenient way to change this behavior, namely to indicate that conflicts are acceptable in a given Enum? While I like the idea of catching mistaken collisions, I've seen far too many C/C++ scenarios where multiple names map to the same value. This does raise questions, as it's unclear whether each name should get its own representation, or whether it's better to let DupEnum.name1 is DupEnum.name2 be True. (For the latter behavior, would adding DupEnum.name2 = DupEnum.name1 after the class declaration work today?) Michael -- Michael Urman -------------- next part -------------- An HTML attachment was scrubbed... URL: From lele at metapensiero.it Fri Apr 12 15:53:32 2013 From: lele at metapensiero.it (Lele Gaifax) Date: Fri, 12 Apr 2013 15:53:32 +0200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library References: Message-ID: <87bo9jlusj.fsf@nautilus.nautilus> Eli Bendersky writes: > These enumeration values are not equal, nor do they and hence may exist > in the same set, or as distinct keys in the same dictionary:: I'm not a native speaker and I found the above difficult to parse: is there anything missing between "nor do they" and "and hence may exist"? ciao, lele. -- nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia. lele at metapensiero.it | -- Fortunato Depero, 1929. From sinistersnare at gmail.com Fri Apr 12 16:19:29 2013 From: sinistersnare at gmail.com (Davis Silverman) Date: Fri, 12 Apr 2013 10:19:29 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412135848.BD9DA250BD9@webabinitio.net> References: <20130412135848.BD9DA250BD9@webabinitio.net> Message-ID: I think the reason they are not supporting __lt__, __gt__,etc. is because ints are optional values for enums, therefore it wouldnt be a good idea to compare enums of different types in that way. example: >>>class MyEnum(Enum): >>> fir = 1 >>> sec = 2 >>> thir = "THIRD!" and doing >>> MyEnum.fir >= MyEnum.thir would give unexpected results, therefore not making it a great idea On Fri, Apr 12, 2013 at 9:58 AM, R. David Murray wrote: > On Fri, 12 Apr 2013 05:55:00 -0700, Eli Bendersky > wrote: > > Link to the PEP: http://www.python.org/dev/peps/pep-0435/ [it's also > pasted > > fully below for convenience]. > > This looks great. There's just one bit I don't understand. I'm sure > it was discussed in the python-ideas thread, but the discussion of it > in the PEP does not provide any motivation for the decision. > > > The ``Enum`` class supports iteration. Iteration is defined as the > > sorted order of the item values:: > > > > >>> class FiveColors(Enum): > > ... pink = 4 > > ... cyan = 5 > > ... green = 2 > > ... blue = 3 > > ... red = 1 > > >>> [v.name for v in FiveColors] > > ['red', 'green', 'blue', 'pink', 'cyan'] > > [...] > > > Ordered comparisons between enumeration values are *not* supported. > Enums > > are > > not integers (but see `IntEnum`_ below):: > > > > >>> Colors.red < Colors.blue > > Traceback (most recent call last): > > ... > > NotImplementedError > > >>> Colors.red <= Colors.blue > > Traceback (most recent call last): > > ... > > NotImplementedError > > >>> Colors.blue > Colors.green > > Traceback (most recent call last): > > ... > > NotImplementedError > > >>> Colors.blue >= Colors.green > > Traceback (most recent call last): > > ... > > NotImplementedError > > This is the part that I don't understand. Enums *clearly* have an > ordering, since the iteration order is defined and stable. Why should > I not be allowed to compare values from the same Enum type? There are > certainly use cases where that is very useful. > > To give you a concrete use case: consider response codes from a client > server application constructed the way typical internet protocols are. > We might have: > > class MyAppCode(Enum): > > ok = 200 > command_complete = 201 > status_response = 202 > help_response = 203 > service_ready = 204 > signoff_accepted = 205 > > temporary_failure = 400 > service_not_available = 401 > server_error = 402 > out_of_resources = 403 > > error = 500 > syntax_error = 501 > command_not_implemented = 502 > access_denied = 503 > resource_not_found = 504 > > > It can be quite handy to be able to say things like: > > code = myapp.operation(opstring) > if MyAppCode.temporary_failure < code < MyAppCode.error: > myframework.requeue(opstring, code=code) > return False > elif code > MyAppCode.error: > raise AppError(code) > .... > > In talking to an existing internet protocol it would be natural to use > IntEnum and this issue would not arise, but I have recently worked on > an application that had *exactly* the above sort of enumeration used > internally, when it would have been totally appropriate to use Enum rather > than IntEnum. The ap has several places where an ordered comparison > against the enum is used to check if a code is in the error range or not. > > --David > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/sinistersnare%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdmurray at bitdance.com Fri Apr 12 16:23:47 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 12 Apr 2013 10:23:47 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412135848.BD9DA250BD9@webabinitio.net> Message-ID: <20130412142348.35E49250BD7@webabinitio.net> On Fri, 12 Apr 2013 10:19:29 -0400, Davis Silverman wrote: > I think the reason they are not supporting __lt__, __gt__,etc. is because > ints are optional values for enums, therefore it wouldnt be a good idea to > compare enums of different types in that way. > > example: > > >>>class MyEnum(Enum): > >>> fir = 1 > >>> sec = 2 > >>> thir = "THIRD!" > > and doing > > >>> MyEnum.fir >= MyEnum.thir > would give unexpected results, therefore not making it a great idea That's why I included the bit about iterating the values. The ordering *is* defined. I find it much more surprising for that ordering to be inaccessible via the comparison operators. I think either the iteration order should be undefined (like a set or dict), or the comparison operations should work. I'd prefer the latter, because of the use case I outlined. --David From sinistersnare at gmail.com Fri Apr 12 16:34:18 2013 From: sinistersnare at gmail.com (Davis Silverman) Date: Fri, 12 Apr 2013 10:34:18 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412142348.35E49250BD7@webabinitio.net> References: <20130412135848.BD9DA250BD9@webabinitio.net> <20130412142348.35E49250BD7@webabinitio.net> Message-ID: i agree with that, and good point, i guess i misunderstood. enums could be a nice edition, i cant wait to see how this goes. --Sinistersnare On Fri, Apr 12, 2013 at 10:23 AM, R. David Murray wrote: > On Fri, 12 Apr 2013 10:19:29 -0400, Davis Silverman < > sinistersnare at gmail.com> wrote: > > I think the reason they are not supporting __lt__, __gt__,etc. is because > > ints are optional values for enums, therefore it wouldnt be a good idea > to > > compare enums of different types in that way. > > > > example: > > > > >>>class MyEnum(Enum): > > >>> fir = 1 > > >>> sec = 2 > > >>> thir = "THIRD!" > > > > and doing > > > > >>> MyEnum.fir >= MyEnum.thir > > would give unexpected results, therefore not making it a great idea > > That's why I included the bit about iterating the values. The ordering > *is* defined. I find it much more surprising for that ordering to > be inaccessible via the comparison operators. > > I think either the iteration order should be undefined (like a set > or dict), or the comparison operations should work. I'd prefer > the latter, because of the use case I outlined. > > --David > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/sinistersnare%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Fri Apr 12 16:24:59 2013 From: barry at python.org (Barry Warsaw) Date: Fri, 12 Apr 2013 10:24:59 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <87bo9jlusj.fsf@nautilus.nautilus> References: <87bo9jlusj.fsf@nautilus.nautilus> Message-ID: <20130412102459.7e3bd778@anarchist> On Apr 12, 2013, at 03:53 PM, Lele Gaifax wrote: >Eli Bendersky writes: > >> These enumeration values are not equal, nor do they and hence may exist >> in the same set, or as distinct keys in the same dictionary:: > >I'm not a native speaker and I found the above difficult to parse: is >there anything missing between "nor do they" and "and hence may exist"? Oops, yes. Good catch! I changed it to: Because ``Colors`` and ``OtherColors`` are unrelated enumerations, their values are not equal, and thus they may exist in the same set, or as distinct keys in the same dictionary:: -Barry From tjreedy at udel.edu Fri Apr 12 16:44:07 2013 From: tjreedy at udel.edu (Terry Jan Reedy) Date: Fri, 12 Apr 2013 10:44:07 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <87bo9jlusj.fsf@nautilus.nautilus> References: <87bo9jlusj.fsf@nautilus.nautilus> Message-ID: On 4/12/2013 9:53 AM, Lele Gaifax wrote: > Eli Bendersky writes: > >> These enumeration values are not equal, nor do they and hence may exist >> in the same set, or as distinct keys in the same dictionary:: > > I'm not a native speaker and I found the above difficult to parse: is > there anything missing between "nor do they" and "and hence may exist"? I am a native speaker and also do not understand the above quote. From barry at python.org Fri Apr 12 16:30:05 2013 From: barry at python.org (Barry Warsaw) Date: Fri, 12 Apr 2013 10:30:05 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: Message-ID: <20130412103005.15a0b935@anarchist> On Apr 12, 2013, at 09:03 AM, Michael Urman wrote: >> You also may not duplicate values in derived enumerations:: >> >> >>> class BadColors(Colors): >> ... yellow = 4 >> ... chartreuse = 2 # Oops! >> Traceback (most recent call last): >> ... >> ValueError: Conflicting enums with value '2': 'green' and 'chartreuse' >> > >Is there a convenient way to change this behavior, namely to indicate that >conflicts are acceptable in a given Enum? While I like the idea of catching >mistaken collisions, I've seen far too many C/C++ scenarios where multiple >names map to the same value. This does raise questions, as it's unclear >whether each name should get its own representation, or whether it's better >to let DupEnum.name1 is DupEnum.name2 be True. Currently there is not. This behavior is defined in the metaclass. >(For the latter behavior, would adding DupEnum.name2 = DupEnum.name1 after >the class declaration work today?) Yes, but the repr/str of the alias will show the original value. >>> from flufl.enum import Enum >>> Colors = Enum('Colors', 'red green blue') >>> Colors.black = Colors.blue >>> Colors.black >>> Colors.black is Colors.blue True -Barry From solipsis at pitrou.net Fri Apr 12 16:59:50 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 12 Apr 2013 16:59:50 +0200 Subject: [Python-Dev] casefolding in pathlib (PEP 428) References: <20130411232706.51790b7a@pitrou.net> <20130412103954.4c96451a@pitrou.net> <2CD9807E-1A2E-45C8-A9BA-443CD21B33EC@mac.com> Message-ID: <20130412165950.500f4f92@pitrou.net> Le Fri, 12 Apr 2013 14:43:42 +0200, Ronald Oussoren a ?crit : > > On 12 Apr, 2013, at 10:39, Antoine Pitrou wrote: > >> > >> > >> Perhaps it would be best if the code never called lower() or > >> upper() (not even indirectly via os.path.normcase()). Then any > >> case-folding and path-normalization bugs are the responsibility of > >> the application, and we won't have to worry about how to fix the > >> stdlib without breaking backwards compatibility if we ever figure > >> out how to fix this (which I somehow doubt we ever will anyway :-). > > > > Ok, I've taken a look at the code. Right now lower() is used for two > > purposes: > > > > 1. comparisons (__eq__ and __ne__) > > 2. globbing and matching > > > > While (1) could be dropped, for (2) I think we want glob("*.py") to > > find "SETUP.PY" under Windows. Anything else will probably be > > surprising to users of that platform. > > Globbing necessarily accesses the filesystem and could in theory do > the right thing, except for the minor detail of there not being an > easy way to determine of the names in a particular folder are > compared case sensitive or not. It's also much less efficient, since you have to stat() every potential match. e.g. when encountering "SETUP.PY", you would have to stat() (or, rather, lstat()) both "setup.py" and "SETUP.PY" to check if they have the same st_ino. > At least for OSX the kernel will normalize names for you, at least > for HFS+, and therefore two names that don't compare equal with '==' > can refer to the same file (for example the NFKD and NFKC forms of > L?we). I don't think differently normalized filenames are as common on OS X as differently cased filenames are on Windows, right? Regards Antoine. From barry at python.org Fri Apr 12 16:50:44 2013 From: barry at python.org (Barry Warsaw) Date: Fri, 12 Apr 2013 10:50:44 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412135848.BD9DA250BD9@webabinitio.net> References: <20130412135848.BD9DA250BD9@webabinitio.net> Message-ID: <20130412105044.1f1f7061@anarchist> On Apr 12, 2013, at 09:58 AM, R. David Murray wrote: >> The ``Enum`` class supports iteration. Iteration is defined as the >> sorted order of the item values:: >> >> >>> class FiveColors(Enum): >> ... pink = 4 >> ... cyan = 5 >> ... green = 2 >> ... blue = 3 >> ... red = 1 >> >>> [v.name for v in FiveColors] >> ['red', 'green', 'blue', 'pink', 'cyan'] > >[...] > >> Ordered comparisons between enumeration values are *not* supported. Enums >> are >> not integers (but see `IntEnum`_ below):: >> >> >>> Colors.red < Colors.blue >> Traceback (most recent call last): >> ... >> NotImplementedError >> >>> Colors.red <= Colors.blue >> Traceback (most recent call last): >> ... >> NotImplementedError >> >>> Colors.blue > Colors.green >> Traceback (most recent call last): >> ... >> NotImplementedError >> >>> Colors.blue >= Colors.green >> Traceback (most recent call last): >> ... >> NotImplementedError > >This is the part that I don't understand. Enums *clearly* have an >ordering, since the iteration order is defined and stable. Why should >I not be allowed to compare values from the same Enum type? There are >certainly use cases where that is very useful. Nick brought this up in private email, and my response was basically that iteration order for Enums should not be guaranteed, even if that happens to work in the current implementation. The reason why it works in the current implementation is purely to provide predictable reprs. Of course, we could sort the values for the repr and remove the sorted() call in EnumMetaclass.__iter__() but then we would have to add it back for IntEnums, since we *do* want to make explicit iteration order guarantees for IntEnums, as they also have full rich comparisons. I'm just not sure it's worth it from an implementation point of view. I will update the PEP to make this more clear. >In talking to an existing internet protocol it would be natural to use >IntEnum and this issue would not arise, but I have recently worked on >an application that had *exactly* the above sort of enumeration used >internally, when it would have been totally appropriate to use Enum rather >than IntEnum. The ap has several places where an ordered comparison >against the enum is used to check if a code is in the error range or not. Why Enums and not IntEnums? Enums will not have ordered comparisons, but IntEnums will. -Barry From barry at python.org Fri Apr 12 16:51:50 2013 From: barry at python.org (Barry Warsaw) Date: Fri, 12 Apr 2013 10:51:50 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412142348.35E49250BD7@webabinitio.net> References: <20130412135848.BD9DA250BD9@webabinitio.net> <20130412142348.35E49250BD7@webabinitio.net> Message-ID: <20130412105150.4314dc2e@anarchist> On Apr 12, 2013, at 10:23 AM, R. David Murray wrote: >I think either the iteration order should be undefined (like a set >or dict), or the comparison operations should work. I'd prefer >the latter, because of the use case I outlined. Order (both iteration and comparison) is officially undefined for Enums but defined for IntEnums. -Barry From barry at python.org Fri Apr 12 17:02:54 2013 From: barry at python.org (Barry Warsaw) Date: Fri, 12 Apr 2013 11:02:54 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: Message-ID: <20130412110254.0c4c7a44@anarchist> On Apr 12, 2013, at 03:31 PM, Dirkjan Ochtman wrote: >On Fri, Apr 12, 2013 at 2:55 PM, Eli Bendersky wrote: >> Ordered comparisons between enumeration values are *not* supported. Enums >> are >> not integers (but see `IntEnum`_ below):: >> >> >>> Colors.red < Colors.blue >> Traceback (most recent call last): >> ... >> NotImplementedError >> >>> Colors.red <= Colors.blue >> Traceback (most recent call last): >> ... >> NotImplementedError >> >>> Colors.blue > Colors.green >> Traceback (most recent call last): >> ... >> NotImplementedError >> >>> Colors.blue >= Colors.green >> Traceback (most recent call last): >> ... >> NotImplementedError > >I like much of this PEP, but the exception type for this case seems >odd to me. Wouldn't a TypeError be more appropriate here? > >Somewhat like this: > >>>> 'a' - 'b' >Traceback (most recent call last): > File "", line 1, in >TypeError: unsupported operand type(s) for -: 'str' and 'str' Interesting. I'm having a hard time articulating why, but NotImplementedError just feels more right to me in this case. -Barry From rdmurray at bitdance.com Fri Apr 12 17:23:19 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 12 Apr 2013 11:23:19 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412105044.1f1f7061@anarchist> References: <20130412135848.BD9DA250BD9@webabinitio.net> <20130412105044.1f1f7061@anarchist> Message-ID: <20130412152320.48E0B250BD2@webabinitio.net> On Fri, 12 Apr 2013 10:50:44 -0400, Barry Warsaw wrote: > Nick brought this up in private email, and my response was basically that > iteration order for Enums should not be guaranteed, even if that happens to > work in the current implementation. The reason why it works in the current > implementation is purely to provide predictable reprs. > > Of course, we could sort the values for the repr and remove the sorted() call > in EnumMetaclass.__iter__() but then we would have to add it back for > IntEnums, since we *do* want to make explicit iteration order guarantees for > IntEnums, as they also have full rich comparisons. I'm just not sure it's > worth it from an implementation point of view. > > I will update the PEP to make this more clear. OK. > >In talking to an existing internet protocol it would be natural to use > >IntEnum and this issue would not arise, but I have recently worked on > >an application that had *exactly* the above sort of enumeration used > >internally, when it would have been totally appropriate to use Enum rather > >than IntEnum. The ap has several places where an ordered comparison > >against the enum is used to check if a code is in the error range or not. > > Why Enums and not IntEnums? Enums will not have ordered comparisons, but > IntEnums will. To take advantage of their incommensurability with other Enums. It's not a big deal, though; I'm more concerned that the API be internally consistent. I presume that one could always define an Enum subclass and provide comparison methods if desired :) --David From rdmurray at bitdance.com Fri Apr 12 17:29:29 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 12 Apr 2013 11:29:29 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412110254.0c4c7a44@anarchist> References: <20130412110254.0c4c7a44@anarchist> Message-ID: <20130412152930.77D29250BD2@webabinitio.net> On Fri, 12 Apr 2013 11:02:54 -0400, Barry Warsaw wrote: > On Apr 12, 2013, at 03:31 PM, Dirkjan Ochtman wrote: > >On Fri, Apr 12, 2013 at 2:55 PM, Eli Bendersky wrote: > >> >>> Colors.blue >= Colors.green > >> Traceback (most recent call last): > >> ... > >> NotImplementedError > > > >I like much of this PEP, but the exception type for this case seems > >odd to me. Wouldn't a TypeError be more appropriate here? > > > >Somewhat like this: > > > >>>> 'a' - 'b' > >Traceback (most recent call last): > > File "", line 1, in > >TypeError: unsupported operand type(s) for -: 'str' and 'str' > > Interesting. I'm having a hard time articulating why, but NotImplementedError > just feels more right to me in this case. I think TypeError is more consistent with the rest of Python: >>> object() < object() Traceback (most recent call last): File "", line 1, in TypeError: unorderable types: object() < object() You get that automatically if you return NotImplemented from the comparison methods. I don't think you should be explicitly raising NotImplemented. --David From ronaldoussoren at mac.com Fri Apr 12 17:29:58 2013 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Fri, 12 Apr 2013 17:29:58 +0200 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: <20130412165950.500f4f92@pitrou.net> References: <20130411232706.51790b7a@pitrou.net> <20130412103954.4c96451a@pitrou.net> <2CD9807E-1A2E-45C8-A9BA-443CD21B33EC@mac.com> <20130412165950.500f4f92@pitrou.net> Message-ID: On 12 Apr, 2013, at 16:59, Antoine Pitrou wrote: > Le Fri, 12 Apr 2013 14:43:42 +0200, > Ronald Oussoren a ?crit : >> >> On 12 Apr, 2013, at 10:39, Antoine Pitrou wrote: >>>> >>>> >>>> Perhaps it would be best if the code never called lower() or >>>> upper() (not even indirectly via os.path.normcase()). Then any >>>> case-folding and path-normalization bugs are the responsibility of >>>> the application, and we won't have to worry about how to fix the >>>> stdlib without breaking backwards compatibility if we ever figure >>>> out how to fix this (which I somehow doubt we ever will anyway :-). >>> >>> Ok, I've taken a look at the code. Right now lower() is used for two >>> purposes: >>> >>> 1. comparisons (__eq__ and __ne__) >>> 2. globbing and matching >>> >>> While (1) could be dropped, for (2) I think we want glob("*.py") to >>> find "SETUP.PY" under Windows. Anything else will probably be >>> surprising to users of that platform. >> >> Globbing necessarily accesses the filesystem and could in theory do >> the right thing, except for the minor detail of there not being an >> easy way to determine of the names in a particular folder are >> compared case sensitive or not. > > It's also much less efficient, since you have to stat() every potential > match. e.g. when encountering "SETUP.PY", you would have to stat() (or, > rather, lstat()) both "setup.py" and "SETUP.PY" to check if they have > the same st_ino. I found a way to determine if names in a directory are stored case sensitive, at least on OSX. That way you'd only have to perform one call for the directory, or one call per path element that contains wildcard characters for glob.glob. That API is definitly platform specific. > >> At least for OSX the kernel will normalize names for you, at least >> for HFS+, and therefore two names that don't compare equal with '==' >> can refer to the same file (for example the NFKD and NFKC forms of >> L?we). > > I don't think differently normalized filenames are as common on OS X as > differently cased filenames are on Windows, right? The problem is more that HFS+ stores names with decomposed characters, which basicly means that accents are stored separate from their base characters. In most input the accented character will be one character, and hence a naieve comparison like this could fail to match: .> name = input() .> for fn in os.listdir('.'): .> if fn.lower() == name.lower(): .> print("Found {} in the current directory".format(name)) Ronald > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com From barry at python.org Fri Apr 12 17:30:43 2013 From: barry at python.org (Barry Warsaw) Date: Fri, 12 Apr 2013 11:30:43 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412152320.48E0B250BD2@webabinitio.net> References: <20130412135848.BD9DA250BD9@webabinitio.net> <20130412105044.1f1f7061@anarchist> <20130412152320.48E0B250BD2@webabinitio.net> Message-ID: <20130412113043.32fa834e@limelight> On Apr 12, 2013, at 11:23 AM, R. David Murray wrote: >To take advantage of their incommensurability with other Enums. It's >not a big deal, though; I'm more concerned that the API be internally >consistent. I presume that one could always define an Enum subclass >and provide comparison methods if desired :) Hey, no peeking! :) -Barry From ethan at stoneleaf.us Fri Apr 12 17:36:11 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 12 Apr 2013 08:36:11 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412105150.4314dc2e@anarchist> References: <20130412135848.BD9DA250BD9@webabinitio.net> <20130412142348.35E49250BD7@webabinitio.net> <20130412105150.4314dc2e@anarchist> Message-ID: <516829EB.4040301@stoneleaf.us> On 04/12/2013 07:51 AM, Barry Warsaw wrote: > On Apr 12, 2013, at 10:23 AM, R. David Murray wrote: > >> I think either the iteration order should be undefined (like a set >> or dict), or the comparison operations should work. I'd prefer >> the latter, because of the use case I outlined. > > Order (both iteration and comparison) is officially undefined for Enums but > defined for IntEnums. Huh?? From the PEP ============ The ``Enum`` class supports iteration. Iteration is defined as the sorted order of the item values:: >>> class FiveColors(Enum): ... pink = 4 ... cyan = 5 ... green = 2 ... blue = 3 ... red = 1 >>> [v.name for v in FiveColors] ['red', 'green', 'blue', 'pink', 'cyan'] -- ~Ethan~ From ethan at stoneleaf.us Fri Apr 12 17:31:29 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 12 Apr 2013 08:31:29 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412110254.0c4c7a44@anarchist> References: <20130412110254.0c4c7a44@anarchist> Message-ID: <516828D1.5050205@stoneleaf.us> On 04/12/2013 08:02 AM, Barry Warsaw wrote: > On Apr 12, 2013, at 03:31 PM, Dirkjan Ochtman wrote: > >> On Fri, Apr 12, 2013 at 2:55 PM, Eli Bendersky wrote: >>> Ordered comparisons between enumeration values are *not* supported. Enums >>> are >>> not integers (but see `IntEnum`_ below):: >>> >>> >>> Colors.red < Colors.blue >>> Traceback (most recent call last): >>> ... >>> NotImplementedError >>> >>> Colors.red <= Colors.blue >>> Traceback (most recent call last): >>> ... >>> NotImplementedError >>> >>> Colors.blue > Colors.green >>> Traceback (most recent call last): >>> ... >>> NotImplementedError >>> >>> Colors.blue >= Colors.green >>> Traceback (most recent call last): >>> ... >>> NotImplementedError >> >> I like much of this PEP, but the exception type for this case seems >> odd to me. Wouldn't a TypeError be more appropriate here? >> >> Somewhat like this: >> >>>>> 'a' - 'b' >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: unsupported operand type(s) for -: 'str' and 'str' > > Interesting. I'm having a hard time articulating why, but NotImplementedError > just feels more right to me in this case. NotImplemented makes it seem like we could implement it in a subclass -- is this true? Also, for a more direct comparison: --> 'a' < 1 Traceback (most recent call last): File "", line 1, in TypeError: unorderable types: str() < int() I would think this is the more appropriate exception and text to use. -- ~Ethan~ From barry at python.org Fri Apr 12 17:24:59 2013 From: barry at python.org (Barry Warsaw) Date: Fri, 12 Apr 2013 11:24:59 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412152930.77D29250BD2@webabinitio.net> References: <20130412110254.0c4c7a44@anarchist> <20130412152930.77D29250BD2@webabinitio.net> Message-ID: <20130412112459.728e8249@anarchist> On Apr 12, 2013, at 11:29 AM, R. David Murray wrote: >You get that automatically if you return NotImplemented from the >comparison methods. I don't think you should be explicitly raising >NotImplemented. Oh, that's much better. Thanks, I'll update the implementation, docs, and PEP. -Barry From barry at python.org Fri Apr 12 17:37:55 2013 From: barry at python.org (Barry Warsaw) Date: Fri, 12 Apr 2013 11:37:55 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412152930.77D29250BD2@webabinitio.net> References: <20130412110254.0c4c7a44@anarchist> <20130412152930.77D29250BD2@webabinitio.net> Message-ID: <20130412113755.21f7e425@anarchist> On Apr 12, 2013, at 11:29 AM, R. David Murray wrote: >You get that automatically if you return NotImplemented from the >comparison methods. I don't think you should be explicitly raising >NotImplemented. Oh darn, this doesn't work for Python 2.7. You don't care for PEP 435, but flufl.enum will have to raise the TypeError explicitly for consistency. -Barry From barry at python.org Fri Apr 12 17:38:59 2013 From: barry at python.org (Barry Warsaw) Date: Fri, 12 Apr 2013 11:38:59 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <516829EB.4040301@stoneleaf.us> References: <20130412135848.BD9DA250BD9@webabinitio.net> <20130412142348.35E49250BD7@webabinitio.net> <20130412105150.4314dc2e@anarchist> <516829EB.4040301@stoneleaf.us> Message-ID: <20130412113859.1e6dd328@anarchist> On Apr 12, 2013, at 08:36 AM, Ethan Furman wrote: >Huh?? Refresh the page :). -Barry From pelson.pub at gmail.com Fri Apr 12 17:57:49 2013 From: pelson.pub at gmail.com (Phil Elson) Date: Fri, 12 Apr 2013 16:57:49 +0100 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412152320.48E0B250BD2@webabinitio.net> References: <20130412135848.BD9DA250BD9@webabinitio.net> <20130412105044.1f1f7061@anarchist> <20130412152320.48E0B250BD2@webabinitio.net> Message-ID: 1) Is there limitation that EnumValues themselves must be immutable? 2) I'm most excited by the prospect of using these Enums as function defaults. I've not worked it through fully, but I'm guessing the following will work? >>> class Colors(Enum): ... red = (256, 0, 0) >>> def fill(color=Colors.red): ... pass ... >>> from inspect import signature >>> print(signature(fill)) (color=) 3) Enums are generally used for defining constants - Is there a case to be made for using capitals in the 435 as PEP8 suggests, or are enums a special case? (http://www.python.org/dev/peps/pep-0008/#constants) 4) Is there an easy way to create custom EnumValues subclasses? In particular it'd be nice to be able to change the __repr__ in some cases to hide the value itself, which is often not important. Sorry if this has already been discussed elsewhere (I've only recently signed up to the dev mailinglist). On 12 April 2013 16:23, R. David Murray wrote: > On Fri, 12 Apr 2013 10:50:44 -0400, Barry Warsaw wrote: > > Nick brought this up in private email, and my response was basically that > > iteration order for Enums should not be guaranteed, even if that happens > to > > work in the current implementation. The reason why it works in the > current > > implementation is purely to provide predictable reprs. > > > > Of course, we could sort the values for the repr and remove the sorted() > call > > in EnumMetaclass.__iter__() but then we would have to add it back for > > IntEnums, since we *do* want to make explicit iteration order guarantees > for > > IntEnums, as they also have full rich comparisons. I'm just not sure > it's > > worth it from an implementation point of view. > > > > I will update the PEP to make this more clear. > > OK. > > > >In talking to an existing internet protocol it would be natural to use > > >IntEnum and this issue would not arise, but I have recently worked on > > >an application that had *exactly* the above sort of enumeration used > > >internally, when it would have been totally appropriate to use Enum > rather > > >than IntEnum. The ap has several places where an ordered comparison > > >against the enum is used to check if a code is in the error range or > not. > > > > Why Enums and not IntEnums? Enums will not have ordered comparisons, but > > IntEnums will. > > To take advantage of their incommensurability with other Enums. It's > not a big deal, though; I'm more concerned that the API be internally > consistent. I presume that one could always define an Enum subclass > and provide comparison methods if desired :) > > --David > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/pelson.pub%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From status at bugs.python.org Fri Apr 12 18:06:46 2013 From: status at bugs.python.org (Python tracker) Date: Fri, 12 Apr 2013 18:06:46 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20130412160646.F2052568F8@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2013-04-05 - 2013-04-12) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3908 ( +3) closed 25569 (+61) total 29477 (+64) Open issues with patches: 1728 Issues opened (38) ================== #17642: IDLE add font resizing hot keys http://bugs.python.org/issue17642 opened by rhettinger #17643: Expose weakref callback for introspection purposes. http://bugs.python.org/issue17643 opened by mark.dickinson #17644: str.format() crashes http://bugs.python.org/issue17644 opened by poldnev #17646: traceback.py has a lot of code duplication http://bugs.python.org/issue17646 opened by isoschiz #17647: subprocess.communicate() should preserve colored output on Win http://bugs.python.org/issue17647 opened by Caitlin.Potter #17649: Python/Python-ast.c: No such file or directory http://bugs.python.org/issue17649 opened by pfg #17651: Errno checking replaced by concrete classes inherited from OSE http://bugs.python.org/issue17651 opened by atykhonov #17652: Add skip_on_windows decorator to test.support http://bugs.python.org/issue17652 opened by berker.peksag #17653: fix typo in socketserver.rst http://bugs.python.org/issue17653 opened by tshepang #17654: IDLE only customizes correctly for OS X when using framework b http://bugs.python.org/issue17654 opened by Todd.Rovito #17656: Python 2.7.4 breaks ZipFile extraction of zip files with unico http://bugs.python.org/issue17656 opened by Vhati #17658: pythonw.exe crashes on opening IDLE http://bugs.python.org/issue17658 opened by Acebulf #17659: First weekday http://bugs.python.org/issue17659 opened by IzidorMatusov #17660: mock.patch could whitelist builtins to not need create=True http://bugs.python.org/issue17660 opened by michael.foord #17661: documentation of '%r' links to the wrong repr http://bugs.python.org/issue17661 opened by twouters #17665: convert to idiomatic unittest code http://bugs.python.org/issue17665 opened by tshepang #17666: Extra gzip headers breaks _read_gzip_header http://bugs.python.org/issue17666 opened by maubp #17667: Windows: build with "build_pgo.bat -2" fails to optimize pytho http://bugs.python.org/issue17667 opened by anselm.kruis #17668: re.split loses characters matching ungrouped parts of a patter http://bugs.python.org/issue17668 opened by triquetra011 #17670: expandtabs() weirdness http://bugs.python.org/issue17670 opened by asolano #17671: io.BufferedRWPair can use uninitialized members http://bugs.python.org/issue17671 opened by amaury.forgeotdarc #17673: add `copy_from` argument to temporaryfile http://bugs.python.org/issue17673 opened by pitrou #17676: spwd uses -1 for empty attributes http://bugs.python.org/issue17676 opened by Alexqw #17679: sysconfig generation uses some env variables multiple times http://bugs.python.org/issue17679 opened by bkabrda #17681: Work with an extra field of gzip files http://bugs.python.org/issue17681 opened by serhiy.storchaka #17683: socket.getsockname() inconsistent return type with AF_UNIX http://bugs.python.org/issue17683 opened by giampaolo.rodola #17684: Skip tests in test_socket like testFDPassSeparate on OS X http://bugs.python.org/issue17684 opened by jramnani #17686: Doc using/unix broken link (http://linuxmafia.com/) http://bugs.python.org/issue17686 opened by hashimo #17689: Fix test discovery for test_tarfile.py http://bugs.python.org/issue17689 opened by zach.ware #17691: Fix test discovery for test_univnewlines.py http://bugs.python.org/issue17691 opened by zach.ware #17694: Enhance _PyUnicodeWriter API to control minimum buffer length http://bugs.python.org/issue17694 opened by haypo #17695: _sysconfigdata broken with universal builds on OSX http://bugs.python.org/issue17695 opened by ronaldoussoren #17697: Incorrect stacktrace from pdb http://bugs.python.org/issue17697 opened by donaldcallen #17700: Update Curses HOWTO for 3.4 http://bugs.python.org/issue17700 opened by akuchling #17701: Improving strftime documentation http://bugs.python.org/issue17701 opened by wolever #17702: os.environ converts key type from string to bytes in KeyError http://bugs.python.org/issue17702 opened by Robert.Tasarz #17703: Trash can mechanism segfault during interpreter finalization i http://bugs.python.org/issue17703 opened by lemburg #17704: ImportError: No module named '_curses' http://bugs.python.org/issue17704 opened by marco.buttu Most recent 15 issues with no replies (15) ========================================== #17704: ImportError: No module named '_curses' http://bugs.python.org/issue17704 #17695: _sysconfigdata broken with universal builds on OSX http://bugs.python.org/issue17695 #17694: Enhance _PyUnicodeWriter API to control minimum buffer length http://bugs.python.org/issue17694 #17691: Fix test discovery for test_univnewlines.py http://bugs.python.org/issue17691 #17689: Fix test discovery for test_tarfile.py http://bugs.python.org/issue17689 #17686: Doc using/unix broken link (http://linuxmafia.com/) http://bugs.python.org/issue17686 #17681: Work with an extra field of gzip files http://bugs.python.org/issue17681 #17676: spwd uses -1 for empty attributes http://bugs.python.org/issue17676 #17671: io.BufferedRWPair can use uninitialized members http://bugs.python.org/issue17671 #17667: Windows: build with "build_pgo.bat -2" fails to optimize pytho http://bugs.python.org/issue17667 #17665: convert to idiomatic unittest code http://bugs.python.org/issue17665 #17653: fix typo in socketserver.rst http://bugs.python.org/issue17653 #17640: from distutils.util import byte_compile hangs http://bugs.python.org/issue17640 #17631: inspect getsource does not display full text of lambda http://bugs.python.org/issue17631 #17621: Create a lazy import loader mixin http://bugs.python.org/issue17621 Most recent 15 issues waiting for review (15) ============================================= #17702: os.environ converts key type from string to bytes in KeyError http://bugs.python.org/issue17702 #17701: Improving strftime documentation http://bugs.python.org/issue17701 #17700: Update Curses HOWTO for 3.4 http://bugs.python.org/issue17700 #17691: Fix test discovery for test_univnewlines.py http://bugs.python.org/issue17691 #17689: Fix test discovery for test_tarfile.py http://bugs.python.org/issue17689 #17684: Skip tests in test_socket like testFDPassSeparate on OS X http://bugs.python.org/issue17684 #17683: socket.getsockname() inconsistent return type with AF_UNIX http://bugs.python.org/issue17683 #17679: sysconfig generation uses some env variables multiple times http://bugs.python.org/issue17679 #17670: expandtabs() weirdness http://bugs.python.org/issue17670 #17667: Windows: build with "build_pgo.bat -2" fails to optimize pytho http://bugs.python.org/issue17667 #17660: mock.patch could whitelist builtins to not need create=True http://bugs.python.org/issue17660 #17656: Python 2.7.4 breaks ZipFile extraction of zip files with unico http://bugs.python.org/issue17656 #17653: fix typo in socketserver.rst http://bugs.python.org/issue17653 #17652: Add skip_on_windows decorator to test.support http://bugs.python.org/issue17652 #17651: Errno checking replaced by concrete classes inherited from OSE http://bugs.python.org/issue17651 Top 10 most discussed issues (10) ================================= #17511: Idle find function closes after each find operation http://bugs.python.org/issue17511 13 msgs #14010: deeply nested filter segfaults http://bugs.python.org/issue14010 11 msgs #16427: Faster hash implementation http://bugs.python.org/issue16427 11 msgs #17668: re.split loses characters matching ungrouped parts of a patter http://bugs.python.org/issue17668 11 msgs #17618: base85 encoding http://bugs.python.org/issue17618 10 msgs #17656: Python 2.7.4 breaks ZipFile extraction of zip files with unico http://bugs.python.org/issue17656 10 msgs #17649: Python/Python-ast.c: No such file or directory http://bugs.python.org/issue17649 8 msgs #17703: Trash can mechanism segfault during interpreter finalization i http://bugs.python.org/issue17703 8 msgs #12181: SIGBUS error on OpenBSD (sparc64) http://bugs.python.org/issue12181 7 msgs #15518: Provide test coverage for filecmp.dircmp.report methods. http://bugs.python.org/issue15518 7 msgs Issues closed (57) ================== #5609: Create Unit Tests for nturl2path module http://bugs.python.org/issue5609 closed by orsenthil #6640: urlparse should parse mailto: URL headers as query parameters http://bugs.python.org/issue6640 closed by orsenthil #6696: Profile objects should be documented http://bugs.python.org/issue6696 closed by ezio.melotti #6743: Add function compatible with print to pprint module http://bugs.python.org/issue6743 closed by pitrou #12820: Tests for Lib/xml/dom/minicompat.py http://bugs.python.org/issue12820 closed by ezio.melotti #13056: test_multibytecodec.py:TestStreamWriter is skipped after PEP39 http://bugs.python.org/issue13056 closed by ezio.melotti #13126: find() slower than rfind() http://bugs.python.org/issue13126 closed by python-dev #13249: argparse.ArgumentParser() lists arguments in the wrong order http://bugs.python.org/issue13249 closed by asvetlov #14439: Easier error diagnosis when bootstrapping the runpy module in http://bugs.python.org/issue14439 closed by python-dev #15092: Using enum PyUnicode_Kind http://bugs.python.org/issue15092 closed by serhiy.storchaka #15194: libffi-3.0.11 update http://bugs.python.org/issue15194 closed by doko #15596: pickle: Faster serialization of Unicode strings http://bugs.python.org/issue15596 closed by pitrou #16153: PyUnicode_FromFormatV() must fail if the format string is inva http://bugs.python.org/issue16153 closed by haypo #16389: re._compiled_typed's lru_cache causes significant degradation http://bugs.python.org/issue16389 closed by serhiy.storchaka #16850: Add "e" mode to open(): close-and-exec (O_CLOEXEC) / O_NOINHER http://bugs.python.org/issue16850 closed by haypo #16887: IDLE - tabify/untabify applied when clicking Cancel http://bugs.python.org/issue16887 closed by roger.serwy #17093: Make importlib.abc more inheritance-friendly http://bugs.python.org/issue17093 closed by brett.cannon #17469: Fix sys.getallocatedblocks() when running on valgrind http://bugs.python.org/issue17469 closed by pitrou #17477: update the bsddb module do build with db 5.x versions http://bugs.python.org/issue17477 closed by doko #17484: add tests for getpass http://bugs.python.org/issue17484 closed by r.david.murray #17487: wave.Wave_read.getparams should be more user friendly http://bugs.python.org/issue17487 closed by r.david.murray #17502: unittest.mock: side_effect iterators ignore DEFAULT http://bugs.python.org/issue17502 closed by asvetlov #17566: Make importlib.abc.Loader.module_repr optional http://bugs.python.org/issue17566 closed by brett.cannon #17567: Clarify importlib.abc.PathEntryFinder.find_loader() docs http://bugs.python.org/issue17567 closed by brett.cannon #17585: IDLE - regression with exit() and quit() http://bugs.python.org/issue17585 closed by roger.serwy #17613: IDLE "AttributeError: 'NoneType' object has no attribute 'inde http://bugs.python.org/issue17613 closed by roger.serwy #17635: Doc of multiprocessing.connection mentions answerChallenge ins http://bugs.python.org/issue17635 closed by ezio.melotti #17637: Mention "What's New" in devguide's patch guidelines http://bugs.python.org/issue17637 closed by ezio.melotti #17638: test_ssl failure http://bugs.python.org/issue17638 closed by neologix #17641: ssl module doc unification http://bugs.python.org/issue17641 closed by giampaolo.rodola #17645: assert fails in _Py_Mangle http://bugs.python.org/issue17645 closed by pitrou #17648: test_urllib2 convert doctests to unittest http://bugs.python.org/issue17648 closed by orsenthil #17650: There is no exception correspond to errno EROFS http://bugs.python.org/issue17650 closed by georg.brandl #17655: Use writev() function in the io module http://bugs.python.org/issue17655 closed by haypo #17657: IDLE: about dialog should report the full version of TK http://bugs.python.org/issue17657 closed by roger.serwy #17662: socketmodule raises on import when compiled using Setup.dist o http://bugs.python.org/issue17662 closed by kristjan.jonsson #17663: re.sub not replacing all http://bugs.python.org/issue17663 closed by r.david.murray #17664: ssl.SSLError has errno value of None http://bugs.python.org/issue17664 closed by pitrou #17669: Segfault caused by weird combination of imports and yield from http://bugs.python.org/issue17669 closed by python-dev #17672: ssl unclean shutdown http://bugs.python.org/issue17672 closed by pitrou #17674: All examples for concurrent.futures fail with "BrokenProcessPo http://bugs.python.org/issue17674 closed by gjwebber #17675: show addresses in socket.__repr__ http://bugs.python.org/issue17675 closed by giampaolo.rodola #17677: spam http://bugs.python.org/issue17677 closed by neologix #17678: DeprecationWarning fix on cookiejar module since Python 3.3 http://bugs.python.org/issue17678 closed by orsenthil #17680: self is lost if methods are callable objects http://bugs.python.org/issue17680 closed by christian.heimes #17682: _io is missing in Setup.dist in 2.7.x http://bugs.python.org/issue17682 closed by doko #17685: Frozenset literal? http://bugs.python.org/issue17685 closed by mark.dickinson #17687: Undeclared symbol in _struct.c http://bugs.python.org/issue17687 closed by inducer #17688: Wrong signature for richcmpfunc in documentation http://bugs.python.org/issue17688 closed by asvetlov #17690: Fix test discovery for test_time.py http://bugs.python.org/issue17690 closed by ezio.melotti #17692: Fix test discovery for test_sqlite.py http://bugs.python.org/issue17692 closed by ezio.melotti #17693: Use _PyUnicodeWriter API for CJK decoders http://bugs.python.org/issue17693 closed by python-dev #17696: lookup fails for renamed functions http://bugs.python.org/issue17696 closed by r.david.murray #17698: Setting allow_reuse_address in BaseHTTPServer has no effect. http://bugs.python.org/issue17698 closed by r.david.murray #17699: Fix test_getpass on Windows http://bugs.python.org/issue17699 closed by r.david.murray #16432: Template strings documentation in Python 3 refers to % substit http://bugs.python.org/issue16432 closed by andrewsg #900112: cgi.fieldStorage doesn't grok standards env. variables http://bugs.python.org/issue900112 closed by orsenthil From guido at python.org Fri Apr 12 18:34:34 2013 From: guido at python.org (Guido van Rossum) Date: Fri, 12 Apr 2013 09:34:34 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412110254.0c4c7a44@anarchist> References: <20130412110254.0c4c7a44@anarchist> Message-ID: On Fri, Apr 12, 2013 at 8:02 AM, Barry Warsaw wrote: > Interesting. I'm having a hard time articulating why, but NotImplementedError > just feels more right to me in this case. To me, NotImplementedError means that a subclass didn't implement something it should have implemented. But that's not the case here, is it? It's not a bug in the class, it's a bug in the call site. So I agree it ought to be TypeError. -- --Guido van Rossum (python.org/~guido) From ethan at stoneleaf.us Fri Apr 12 18:29:15 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 12 Apr 2013 09:29:15 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412113755.21f7e425@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412152930.77D29250BD2@webabinitio.net> <20130412113755.21f7e425@anarchist> Message-ID: <5168365B.9020108@stoneleaf.us> On 04/12/2013 08:37 AM, Barry Warsaw wrote: > On Apr 12, 2013, at 11:29 AM, R. David Murray wrote: > >> You get that automatically if you return NotImplemented from the >> comparison methods. I don't think you should be explicitly raising >> NotImplemented. > > Oh darn, this doesn't work for Python 2.7. You don't care for PEP 435, but > flufl.enum will have to raise the TypeError explicitly for consistency. Yeah, 2.x has everything ordered; returning NotImplemented only works for the non-comparison methods (such as add, subtract, etc.); one of the nice things about 3.x. :) -- ~Ethan~ From guido at python.org Fri Apr 12 18:43:19 2013 From: guido at python.org (Guido van Rossum) Date: Fri, 12 Apr 2013 09:43:19 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412110254.0c4c7a44@anarchist> Message-ID: On Fri, Apr 12, 2013 at 9:34 AM, Guido van Rossum wrote: > To me, NotImplementedError means that a subclass didn't implement > something it should have implemented. But that's not the case here, is > it? It's not a bug in the class, it's a bug in the call site. So I > agree it ought to be TypeError. Seems I was late to this particular argument. :-) Anyway, as far as I can tell the PEP looks great. I personally think it would be nicer if regular Enums were ordered (as long as the underlying values are ordered), but I don't care enough to overrule the FLUFL. I do wonder about this passage in the PEP: > Let's say you wanted to encode an enumeration value in a database. You might > want to get the enumeration class object from an enumeration value:: > > >>> cls = Colors.red.enum > >>> print(cls.__name__) > Colors I don't understand what this has to do with storing enums in a database. But it reminded me that for the purpose of storing enums in a database, it would be nice to have two examples: one that stores the names and looks them up (do you really have to use getattr() for that?), and one that stores the values and looks them up (how do you do that at all?). Should the metaclass-based API used to create IntEnum be documented, so strongly motivated people can write their own crazy variants? -- --Guido van Rossum (python.org/~guido) From guido at python.org Fri Apr 12 19:05:32 2013 From: guido at python.org (Guido van Rossum) Date: Fri, 12 Apr 2013 10:05:32 -0700 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: <20130412103954.4c96451a@pitrou.net> References: <20130411232706.51790b7a@pitrou.net> <20130412103954.4c96451a@pitrou.net> Message-ID: On Fri, Apr 12, 2013 at 1:39 AM, Antoine Pitrou wrote: > Ok, I've taken a look at the code. Right now lower() is used for two > purposes: > > 1. comparisons (__eq__ and __ne__) > 2. globbing and matching > > While (1) could be dropped, for (2) I think we want glob("*.py") to find > "SETUP.PY" under Windows. Anything else will probably be surprising to > users of that platform. Yeah, I suppose so. But there are more crazy details. E.g. IIRC Windows silently ignores trailing dots in filenames. Do we want "*.py." to match SETUP.PY then? >> - On Linux, paths are really bytes; on Windows (at least NTFS), they >> are really (16-bit) Unicode; on Mac, they are UTF-8 in a specific >> normal form (except on some external filesystems). > > pathlib is just relying on Python 3's sane handling of unicode paths > (thanks to PEP 383). Bytes paths are never used internally. I suppose that just leaves Unicode normalization, discussed later in the thread. >> - On Windows, short names are still supported, making the number of >> ways to spell the path for any given file even larger. > > They are still supported but I doubt they are still relied on (long > filenames appeared in Windows 95!). I think in common situations we can > ignore their existence. Specialized tools like Mercurial may have to > know that they exist, in order to manage potential collisions (but > Mercurial isn't really the target audience for pathlib, and I don't > think they would be interested in such an abstraction). Actually, I've heard of code that dynamically falls back on short names when paths using long names exceed the system limit for path length (either 256 or 1024 IIRC). But short names pretty much require consulting the filesystem, so we can probably ignore them. I guess the bottom line is that, no matter how hard pathlib tries, apps cannot always rely on the predictions about filename validity or equivalence made by pathlib -- we'll have to document that it may be wrong, even though we have the moral obligation to make sure that it is right as often as possible. -- --Guido van Rossum (python.org/~guido) From ralf at systemexit.de Fri Apr 12 19:42:25 2013 From: ralf at systemexit.de (Ralf Schmitt) Date: Fri, 12 Apr 2013 19:42:25 +0200 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: (Guido van Rossum's message of "Fri, 12 Apr 2013 10:05:32 -0700") References: <20130411232706.51790b7a@pitrou.net> <20130412103954.4c96451a@pitrou.net> Message-ID: <87li8n3ata.fsf@brainbot.com> Guido van Rossum writes: > Actually, I've heard of code that dynamically falls back on short > names when paths using long names exceed the system limit for path > length (either 256 or 1024 IIRC). But short names pretty much require > consulting the filesystem, so we can probably ignore them. The limit is 260 characters. But longer paths can be handled by prepending \\?\ and using the unicode APIs. see http://msdn.microsoft.com/en-us/library/aa365247.aspx#maxpath we have the following code to handle the above insanity: ,---- | def prepend_magic_win32(path): | assert isinstance(path, unicode), "path must be of type unicode" | | if path.startswith(u"\\\\"): | if path.startswith(u"\\\\?\\"): | return path | else: | return u"\\\\?\\UNC\\" + path[2:] | else: | return u"\\\\?\\" + path `---- -- Cheers Ralf From solipsis at pitrou.net Fri Apr 12 19:50:58 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 12 Apr 2013 19:50:58 +0200 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: <87li8n3ata.fsf@brainbot.com> References: <20130411232706.51790b7a@pitrou.net> <20130412103954.4c96451a@pitrou.net> <87li8n3ata.fsf@brainbot.com> Message-ID: <20130412195058.589ae32f@pitrou.net> On Fri, 12 Apr 2013 19:42:25 +0200 Ralf Schmitt wrote: > Guido van Rossum writes: > > > Actually, I've heard of code that dynamically falls back on short > > names when paths using long names exceed the system limit for path > > length (either 256 or 1024 IIRC). But short names pretty much require > > consulting the filesystem, so we can probably ignore them. > > The limit is 260 characters. But longer paths can be handled by > prepending \\?\ and using the unicode APIs. > > see http://msdn.microsoft.com/en-us/library/aa365247.aspx#maxpath Indeed. I thought I might use them by default in pathlib but there are other pains: notably, extended paths (those starting with \\?\) can only be absolute. So pathlib supports *passing* them explicitly (kind of, there are very few tests for them) but it doesn't constructs them implicitly. (as Dirkjan pointed out, Mercurial also has domain-specific code to handle Windows paths quirks; this is where I took the idea of having a is_reserved() method for NUL, CON, etc.) Regards Antoine. > > we have the following code to handle the above insanity: > ,---- > | def prepend_magic_win32(path): > | assert isinstance(path, unicode), "path must be of type unicode" > | > | if path.startswith(u"\\\\"): > | if path.startswith(u"\\\\?\\"): > | return path > | else: > | return u"\\\\?\\UNC\\" + path[2:] > | else: > | return u"\\\\?\\" + path > `---- > From nelsonsteves at hushmail.com Fri Apr 12 19:32:09 2013 From: nelsonsteves at hushmail.com (nelsonsteves at hushmail.com) Date: Fri, 12 Apr 2013 13:32:09 -0400 Subject: [Python-Dev] Biggest Fake Conference in Computer Science Message-ID: <20130412173210.0031E10E2D5@smtp.hushmail.com> We are researchers from different parts of the world and conducted a study on the world?s biggest bogus computer science conference WORLDCOMP ( http://sites.google.com/site/worlddump1 ) organized by Prof. Hamid Arabnia from University of Georgia, USA. We submitted a fake paper to WORLDCOMP 2011 and again (the same paper with a modified title) to WORLDCOMP 2012. This paper had numerous fundamental mistakes. Sample statements from that paper include: (1). Binary logic is fuzzy logic and vice versa (2). Pascal developed fuzzy logic (3). Object oriented languages do not exhibit any polymorphism or inheritance (4). TCP and IP are synonyms and are part of OSI model (5). Distributed systems deal with only one computer (6). Laptop is an example for a super computer (7). Operating system is an example for computer hardware Also, our paper did not express any conceptual meaning. However, it was accepted both the times without any modifications (and without any reviews) and we were invited to submit the final paper and a payment of $500+ fee to present the paper. We decided to use the fee for better purposes than making Prof. Hamid Arabnia (Chairman of WORLDCOMP) rich. After that, we received few reminders from WORLDCOMP to pay the fee but we never responded. We MUST say that you should look at the above website if you have any thoughts to submit a paper to WORLDCOMP. DBLP and other indexing agencies have stopped indexing WORLDCOMP?s proceedings since 2011 due to its fakeness. See http://www.informatik.uni-trier.de/~ley/db/conf/icai/index.html for of one of the conferences of WORLDCOMP and notice that there is no listing after 2010. See http://sites.google.com/site/dumpconf for comments from well-known researchers about WORLDCOMP. If WORLDCOMP is not fake then why did DBLP suddenly stopped listing the proceedings after? The status of your WORLDCOMP papers can be changed from ?scientific? to ?other? (i.e., junk or non-technical) at any time. See the comments http://www.mail-archive.com/tccc at lists.cs.columbia.edu/msg05168.html of a respected researcher on this. Better not to have a paper than having it in WORLDCOMP and spoil the resume and peace of mind forever! Our study revealed that WORLDCOMP is a money making business, using University of Georgia mask, for Prof. Hamid Arabnia. He is throwing out a small chunk of that money (around 20 dollars per paper published in WORLDCOMP?s proceedings) to his puppet (Mr. Ashu Solo or A.M.G. Solo) who publicizes WORLDCOMP and also defends it at various forums, using fake/anonymous names. The puppet uses fake names and defames other conferences to divert traffic to WORLDCOMP. He also makes anonymous phone calls and threatens the critiques of WORLDCOMP (see Item 7 in Section 5 of http://sites.google.com/site/dumpconf ).That is, the puppet does all his best to get a maximum number of papers published at WORLDCOMP to get more money into his (and Prof. Hamid Arabnia?s) pockets. Monte Carlo Resort (the venue of WORLDCOMP until 2012) has refused to provide the venue for WORLDCOMP?13 because of the fears of their image being tarnished due to WORLDCOMP?s fraudulent activities. WORLDCOMP?13 will be held at a different resort. WORLDCOMP will not be held after 2013. The paper submission deadline for WORLDCOMP?13 was March 18 and it was extended to April 6 and now it is extended to April 20 (it may be extended again) but still there are no committee members, no reviewers, and there is no conference Chairman. The only contact details available on WORLDCOMP?s website is just an email address! Prof. Hamid Arabnia expends the deadline to get more papers (means, more registration fee into his pocket!). Let us make a direct request to Prof. Hamid arabnia: publish all reviews for all the papers (after blocking identifiable details) since 2000 conference. Reveal the names and affiliations of all the reviewers (for each year) and how many papers each reviewer had reviewed on average. We also request him to look at the Open Challenge at https://sites.google.com/site/moneycomp1 Sorry for posting to multiple lists. Spreading the word is the only way to stop this bogus conference. Please forward this message to other mailing lists and people. We are shocked with Prof. Hamid Arabnia and his puppet?s activities http://worldcomp-fake-bogus.blogspot.com Search Google using the keyword worldcomp fake for additional links. From rowen at uw.edu Fri Apr 12 20:21:09 2013 From: rowen at uw.edu (Russell E. Owen) Date: Fri, 12 Apr 2013 11:21:09 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library References: <20130412110254.0c4c7a44@anarchist> Message-ID: In article , Guido van Rossum wrote: > On Fri, Apr 12, 2013 at 9:34 AM, Guido van Rossum wrote: > > To me, NotImplementedError means that a subclass didn't implement > > something it should have implemented. But that's not the case here, is > > it? It's not a bug in the class, it's a bug in the call site. So I > > agree it ought to be TypeError. > > Seems I was late to this particular argument. :-) > > Anyway, as far as I can tell the PEP looks great. I personally think > it would be nicer if regular Enums were ordered (as long as the > underlying values are ordered), but I don't care enough to overrule > the FLUFL. I, too, would strongly prefer to see ordering within an enum. I use home-made enums heavily in my code and find ordering comparisons useful there. Using intEnum is certainly doable, but that opens up the door to comparing values from different Enums, which is not something I'd want to allow. I don't understand why order tests are seen as a bad thing, as long as the values have order (as they will in the common cases of string and int values). It seems the code must go out of its way to prohibit such tests. In any case, I'm very pleased to have this library. it will get rid of much boilerplate in my code. It seems very well designed, and I'm really glad to see it supports subclassing to add values! -- Russell From barry at python.org Fri Apr 12 20:56:24 2013 From: barry at python.org (Barry Warsaw) Date: Fri, 12 Apr 2013 14:56:24 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412135848.BD9DA250BD9@webabinitio.net> <20130412105044.1f1f7061@anarchist> <20130412152320.48E0B250BD2@webabinitio.net> Message-ID: <20130412145624.3f655300@anarchist> On Apr 12, 2013, at 04:57 PM, Phil Elson wrote: >1) Is there limitation that EnumValues themselves must be immutable? I'm not sure it makes sense to have mutable EnumValues, but yes, they are immutable in the sense that you cannot change their underlying value (well, without hackery). >2) I'm most excited by the prospect of using these Enums as function >defaults. I've not worked it through fully, but I'm guessing the following >will work? Sure, why not? :) >>> from flufl.enum import Enum >>> X = Enum('X', 'a b c') >>> def doit(default=X.b): ... pass ... >>> from inspect import signature >>> print(signature(doit)) (default=) >3) Enums are generally used for defining constants - Is there a case to be >made for using capitals in the 435 as PEP8 suggests, or are enums a special >case? (http://www.python.org/dev/peps/pep-0008/#constants) I dunno, I like that the PEP isn't shouting at me. :) >4) Is there an easy way to create custom EnumValues subclasses? In >particular it'd be nice to be able to change the __repr__ in some cases to >hide the value itself, which is often not important. Yes, although we aren't documenting it so we don't get locked into the API. If you look at the flufl.enum implementation, you'll see this is how we implement IntEnums. -Barry From sturla at molden.no Fri Apr 12 20:50:21 2013 From: sturla at molden.no (Sturla Molden) Date: Fri, 12 Apr 2013 20:50:21 +0200 Subject: [Python-Dev] The end of 2.7 In-Reply-To: <5161CDF0.8010007@v.loewis.de> References: <51609C0D.8000106@python.org> <5161CDF0.8010007@v.loewis.de> Message-ID: <5168576D.8070708@molden.no> On 07.04.2013 21:50, "Martin v. L?wis" wrote: > So I believe that extension building is becoming more and more > painful on Windows for Python 2.7 as time passes (and it is already > way more painful than it is on Linux), and I see no way to do much > about that. The "stable ABI" would have been a solution, but it's > too late now for 2.7. I think extension building for Python 2.7 on Windows for this reason is moving from VS2008 to GCC 4.7 (MinGW). When using VS, we are stuck with an old compiler (i.e. the .NET 3.5 SDK). With GCC, there is no such issue - we just link with whatever CRT is appropriate. Thus, providing link libraries for GCC/MinGW (both for the Python and the CRT DLL) somewhat alleviates the problem, unless using VS is mandatory. A long-term solution might be to expose the CRT used by the Python 2.7 DLL with DLL forwarding. That way, linking with the Python DLL's import library would also link the correct CRT. Sturla From murman at gmail.com Fri Apr 12 21:32:44 2013 From: murman at gmail.com (Michael Urman) Date: Fri, 12 Apr 2013 14:32:44 -0500 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412103005.15a0b935@anarchist> References: <20130412103005.15a0b935@anarchist> Message-ID: On Fri, Apr 12, 2013 at 9:30 AM, Barry Warsaw wrote: > On Apr 12, 2013, at 09:03 AM, Michael Urman wrote: > >(For the latter behavior, would adding DupEnum.name2 = DupEnum.name1 after > >the class declaration work today?) > > Yes, but the repr/str of the alias will show the original value. > That satisfies my concern. This gives an author the means to provide two names for a single value, and a way to choose which one is canonical. It's easy to imagine some corner cases related to persisting those values and then retrieving them with a later enum definition that changes the canonical name, but if you store raw values or names it should be easy enough to work around such things. Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From fabiosantosart at gmail.com Fri Apr 12 21:37:32 2013 From: fabiosantosart at gmail.com (=?ISO-8859-1?Q?F=E1bio_Santos?=) Date: Fri, 12 Apr 2013 20:37:32 +0100 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library Message-ID: On Fri, Apr 12, 2013 at 7:21 PM, Russell E. Owen wrote: > Using intEnum is certainly doable, but that opens up the door to > comparing values from different Enums, which is not something I'd want > to allow. I agree. Comparing values from different Enums could cause a lot of hard to find bugs. On the other hand, simply checking in `__cmp__` what Enum both values belong to (since they do have a `enum` attribute) should allow us to avoid cross-comparing instead of just raising a TypeError. It's not very intuitive behavior IMHO. Just my 2 cents. -- F?bio Santos From barry at python.org Fri Apr 12 21:26:57 2013 From: barry at python.org (Barry Warsaw) Date: Fri, 12 Apr 2013 15:26:57 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412110254.0c4c7a44@anarchist> Message-ID: <20130412152657.6ece1ca3@anarchist> On Apr 12, 2013, at 09:43 AM, Guido van Rossum wrote: >I do wonder about this passage in the PEP: > >> Let's say you wanted to encode an enumeration value in a database. You >> might want to get the enumeration class object from an enumeration value:: >> >> >>> cls = Colors.red.enum >> >>> print(cls.__name__) >> Colors > >I don't understand what this has to do with storing enums in a >database. Not much, really. It's just hold over text from the original motivation for exposing the enum class as an attribute on the values. In Mailman, I store these values in my database and they get reconstituted correctly by the ORM layer. Anyway, in this particular case, I think the motivation is unnecessary for describing the API, so I'll remove that from the PEP. >But it reminded me that for the purpose of storing enums in a database, it >would be nice to have two examples: one that stores the names and looks them >up (do you really have to use getattr() for that?), and one that stores the >values and looks them up (how do you do that at all?). It's going to be dependent on how you store and retrieve enum values. As an example, in my database layer I store the enum values in an integer column, with the ORM layer knowing which Enum subclass to use. So all I need to do to store the value is ``int(enum_value)`` and to get back the original enum value, I just do ``self._enum[int_value]`` where self._enum is the Enum subclass. To me, that's probably the most common way of doing it. If you store by name though, yes, you'd have to use ``getattr(self._enum, name)``. At one point Enums also supported getitem syntax for lookup by name, but consider this case: class Fruit(Enum): apple = 'red' banana = 'yellow' tangerine = 'orange' orange = 'reddish yellow' What should Fruit['orange'] return? In private email Nick pointed out that using getattr() for lookup by name works fine, and getitem for look up by value has been in the API since the beginning, so now Fruit['orange'] is documented to return Fruit.tangerine, i.e. lookup by value only. (Actually, in flufl.enum, lookup-by-name is still there but deprecated. We can just drop it for Python 3.4). >Should the metaclass-based API used to create IntEnum be documented, >so strongly motivated people can write their own crazy variants? I think you recommended against that in python-ideas :). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Fri Apr 12 21:33:02 2013 From: barry at python.org (Barry Warsaw) Date: Fri, 12 Apr 2013 15:33:02 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412110254.0c4c7a44@anarchist> Message-ID: <20130412153302.402eca6a@anarchist> On Apr 12, 2013, at 11:21 AM, Russell E. Owen wrote: >I, too, would strongly prefer to see ordering within an enum. I use >home-made enums heavily in my code and find ordering comparisons useful >there. This was all hashed out in gory detail on python-ideas. I feel strongly that base EnumValues should be unordered, especially because the underlying values can be of any type. What would you expect to happen in this case: class X(Enum): a = 1 b = 'hi' if X.a < myvalue < X.b: # whaa? I think for most use cases, IntEnums will fit the bill for those who want ordered comparisons, and it's also easy to subclass EnumValues to specialize the behavior (in fact, this is how IntEnums are implemented). So if you really want ordered-comparisons-with-untyped-enum-values, you can have them. :) -Barry From guido at python.org Fri Apr 12 21:56:03 2013 From: guido at python.org (Guido van Rossum) Date: Fri, 12 Apr 2013 12:56:03 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412152657.6ece1ca3@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412152657.6ece1ca3@anarchist> Message-ID: On Fri, Apr 12, 2013 at 12:26 PM, Barry Warsaw wrote: > On Apr 12, 2013, at 09:43 AM, Guido van Rossum wrote: > >>I do wonder about this passage in the PEP: >> >>> Let's say you wanted to encode an enumeration value in a database. You >>> might want to get the enumeration class object from an enumeration value:: >>> >>> >>> cls = Colors.red.enum >>> >>> print(cls.__name__) >>> Colors >> >>I don't understand what this has to do with storing enums in a >>database. > > Not much, really. It's just hold over text from the original motivation for > exposing the enum class as an attribute on the values. In Mailman, I store > these values in my database and they get reconstituted correctly by the ORM > layer. Anyway, in this particular case, I think the motivation is unnecessary > for describing the API, so I'll remove that from the PEP. Yeah, it looked like an editing mistake. :-) >>But it reminded me that for the purpose of storing enums in a database, it >>would be nice to have two examples: one that stores the names and looks them >>up (do you really have to use getattr() for that?), and one that stores the >>values and looks them up (how do you do that at all?). > > It's going to be dependent on how you store and retrieve enum values. > > As an example, in my database layer I store the enum values in an integer > column, with the ORM layer knowing which Enum subclass to use. So all I need > to do to store the value is ``int(enum_value)`` and to get back the original > enum value, I just do ``self._enum[int_value]`` where self._enum is the Enum > subclass. To me, that's probably the most common way of doing it. Agreed. I can't easily find that in the PEP though. It doesn't mention __getitem__ and I can't find any examples of using []. > If you store by name though, yes, you'd have to use > ``getattr(self._enum, name)``. At one point Enums also supported getitem > syntax for lookup by name, but consider this case: > > class Fruit(Enum): > apple = 'red' > banana = 'yellow' > tangerine = 'orange' > orange = 'reddish yellow' > > What should Fruit['orange'] return? In private email Nick pointed out that > using getattr() for lookup by name works fine, and getitem for look up by > value has been in the API since the beginning, so now Fruit['orange'] is > documented to return Fruit.tangerine, i.e. lookup by value only. (Actually, > in flufl.enum, lookup-by-name is still there but deprecated. We can just drop > it for Python 3.4). Yeah, getattr() is good enough, and it is documented in the PEP. >>Should the metaclass-based API used to create IntEnum be documented, >>so strongly motivated people can write their own crazy variants? > > I think you recommended against that in python-ideas :). I have changed my mind; I am now at least +0 on documenting the metaclass craziness. -- --Guido van Rossum (python.org/~guido) From ethan at stoneleaf.us Fri Apr 12 21:58:22 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 12 Apr 2013 12:58:22 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: Message-ID: <5168675E.8000807@stoneleaf.us> On 04/12/2013 12:37 PM, ? wrote: > On Fri, Apr 12, 2013 at 7:21 PM, Russell E. Owen wrote: >> Using intEnum is certainly doable, but that opens up the door to >> comparing values from different Enums, which is not something I'd want >> to allow. > > I agree. Comparing values from different Enums could cause a lot of > hard to find bugs. > > On the other hand, simply checking in `__cmp__` what Enum both values > belong to (since they do have a `enum` attribute) should allow us to > avoid cross-comparing instead of just raising a TypeError. It's not > very intuitive behavior IMHO. Something like this is what happens for Enums, but IntEnums are subtypes of int, and if we had something like: class Fruit(IntEnum): apples = 1 bananas = 2 oranges = 3 class Flowers(IntEnum): roses = 1 lillies = 2 violets = 3 --> Fruit.apples == 1 True --> Flowers.roses == 1 True --> Fruit.apples == 1 == Flowers.roses ??? This way lies madness. -- ~Ethan~ From rdmurray at bitdance.com Fri Apr 12 22:52:16 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 12 Apr 2013 16:52:16 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412153302.402eca6a@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> Message-ID: <20130412205216.7216B250BD2@webabinitio.net> On Fri, 12 Apr 2013 15:33:02 -0400, Barry Warsaw wrote: > On Apr 12, 2013, at 11:21 AM, Russell E. Owen wrote: > > >I, too, would strongly prefer to see ordering within an enum. I use > >home-made enums heavily in my code and find ordering comparisons useful > >there. > > This was all hashed out in gory detail on python-ideas. I feel strongly that > base EnumValues should be unordered, especially because the underlying values > can be of any type. What would you expect to happen in this case: > > class X(Enum): > a = 1 > b = 'hi' > > if X.a < myvalue < X.b: > # whaa? > > I think for most use cases, IntEnums will fit the bill for those who want > ordered comparisons, and it's also easy to subclass EnumValues to specialize > the behavior (in fact, this is how IntEnums are implemented). > > So if you really want ordered-comparisons-with-untyped-enum-values, you can > have them. :) You are right, the problem of comparison of disparate types makes ordering a non-starter. But by the same token that means you are going to have to be consistent and give up on having a sorted iteration and a stable repr: >>> import enum >>> class Foo(enum.Enum): ... aa = 1 ... bb = 2 ... cc = 'hi' >>> Foo Traceback (most recent call last): File "", line 1, in File "./enum.py", line 103, in __repr__ for k in sorted(cls._enums))) TypeError: unorderable types: str() < int() --David From eliben at gmail.com Fri Apr 12 23:06:55 2013 From: eliben at gmail.com (Eli Bendersky) Date: Fri, 12 Apr 2013 14:06:55 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412205216.7216B250BD2@webabinitio.net> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> Message-ID: On Fri, Apr 12, 2013 at 1:52 PM, R. David Murray wrote: > On Fri, 12 Apr 2013 15:33:02 -0400, Barry Warsaw wrote: > > On Apr 12, 2013, at 11:21 AM, Russell E. Owen wrote: > > > > >I, too, would strongly prefer to see ordering within an enum. I use > > >home-made enums heavily in my code and find ordering comparisons useful > > >there. > > > > This was all hashed out in gory detail on python-ideas. I feel strongly > that > > base EnumValues should be unordered, especially because the underlying > values > > can be of any type. What would you expect to happen in this case: > > > > class X(Enum): > > a = 1 > > b = 'hi' > > > > if X.a < myvalue < X.b: > > # whaa? > > > > I think for most use cases, IntEnums will fit the bill for those who want > > ordered comparisons, and it's also easy to subclass EnumValues to > specialize > > the behavior (in fact, this is how IntEnums are implemented). > > > > So if you really want ordered-comparisons-with-untyped-enum-values, you > can > > have them. :) > > You are right, the problem of comparison of disparate types makes ordering > a non-starter. But by the same token that means you are going to have to > be consistent and give up on having a sorted iteration and a stable repr: > > >>> import enum > >>> class Foo(enum.Enum): > ... aa = 1 > ... bb = 2 > ... cc = 'hi' > >>> Foo > Traceback (most recent call last): > File "", line 1, in > File "./enum.py", line 103, in __repr__ > for k in sorted(cls._enums))) > TypeError: unorderable types: str() < int() > I actually think that having values with different types within a single Enum is conceptually wrong and should be disallowed at creation time. With enums, you either care or don't care about their actual value. If you don't care (the most common use case of an enum, IMHO), then no problem here. If you do care, then it's probably for very specific reasons most of which are solved by IntEnum. I can't imagine why someone would need differently typed values in a single enum - this just seems like a completely inappropriate use of an enum to me. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From larry at hastings.org Fri Apr 12 23:15:13 2013 From: larry at hastings.org (Larry Hastings) Date: Fri, 12 Apr 2013 14:15:13 -0700 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: References: <20130411232706.51790b7a@pitrou.net> <20130412103954.4c96451a@pitrou.net> Message-ID: <51687961.8010602@hastings.org> On 04/12/2013 10:05 AM, Guido van Rossum wrote: > On Fri, Apr 12, 2013 at 1:39 AM, Antoine Pitrou wrote: >> I think we want glob("*.py") to find >> "SETUP.PY" under Windows. Anything else will probably be surprising to >> users of that platform. > Yeah, I suppose so. But there are more crazy details. E.g. IIRC > Windows silently ignores trailing dots in filenames. Do we want > "*.py." to match SETUP.PY then? Someone who is fresher than I am at Windows programming should answer this, but AFAICT Win32 provides no API that will tell you if a particular filename / volume is case sensitive. The VOLUME2 structure from GetVolumeInfo doesn't report anything, and FindFirstFileEx provides a special flag for you to tell the OS (!) whether or not you want case-sensitive globbing. The closest I can get with my cursory browsing of MSDN is that you could infer case-sensitivity from the filesystem reported by GetVolumeInfo, but I doubt even that would be perfect. My only suggestion: lob the problem back into the user's lap, perhaps with something like pathlib.cs['/'] = True pathlib.cs['/mnt/samba-share'] = False >> (long filenames appeared in Windows 95!). That wasn't their first appearance; I'm pretty sure Windows NT 3.1 supported long filenames in 1992, and though I don't remember specifically it's possible NT 3.1 also supported long and short filenames for the same file. Windows 95 was the first appearance of VFAT, the clever hack adding support for long and short filenames to FAT filesystems. /arry From rdmurray at bitdance.com Fri Apr 12 23:17:02 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 12 Apr 2013 17:17:02 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> Message-ID: <20130412211702.D393D250BD2@webabinitio.net> On Fri, 12 Apr 2013 14:06:55 -0700, Eli Bendersky wrote: > I actually think that having values with different types within a single > Enum is conceptually wrong and should be disallowed at creation time. With > enums, you either care or don't care about their actual value. If you don't > care (the most common use case of an enum, IMHO), then no problem here. If > you do care, then it's probably for very specific reasons most of which are > solved by IntEnum. I can't imagine why someone would need differently typed > values in a single enum - this just seems like a completely inappropriate > use of an enum to me. I'm sure someone will come up with one :) But seriously, even if you require all values to be of the same type, that doesn't solve the sorting problem: >>> class Foo(enum.Enum): ... aa = object() ... bb = object() ... >>> Foo Traceback (most recent call last): File "", line 1, in File "./enum.py", line 103, in __repr__ for k in sorted(cls._enums))) TypeError: unorderable types: object() < object() Now, you could *further* require that the type of enum values be sortable....but that point you really have no excuse for not allowing enum values to be compared :) --David From ethan at stoneleaf.us Fri Apr 12 23:19:53 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 12 Apr 2013 14:19:53 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> Message-ID: <51687A79.1070502@stoneleaf.us> On 04/12/2013 02:06 PM, Eli Bendersky wrote: > On Fri, Apr 12, 2013 at 1:52 PM, R. David Murray wrote: > >>> import enum > >>> class Foo(enum.Enum): > ... aa = 1 > ... bb = 2 > ... cc = 'hi' > >>> Foo > Traceback (most recent call last): > File "", line 1, in > File "./enum.py", line 103, in __repr__ > for k in sorted(cls._enums))) > TypeError: unorderable types: str() < int() > > > I actually think that having values with different types within a single Enum is conceptually wrong and should be > disallowed at creation time. With enums, you either care or don't care about their actual value. If you don't care (the > most common use case of an enum, IMHO), then no problem here. If you do care, then it's probably for very specific > reasons most of which are solved by IntEnum. I can't imagine why someone would need differently typed values in a single > enum - this just seems like a completely inappropriate use of an enum to me. +1 (on disallowing the mixed type enum, not the valueless enum being more common ;) From guido at python.org Fri Apr 12 23:34:16 2013 From: guido at python.org (Guido van Rossum) Date: Fri, 12 Apr 2013 14:34:16 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412211702.D393D250BD2@webabinitio.net> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> Message-ID: So, pragmatically, if e and f are values of the same enum class, couldn't e f (where is any comparison operator) defer to e.value f.value ? Or is the problem with being == and e and f being different enum values with the same underlying value? But that's already iffy, isn't it? (Barry's favorite solution for serializing to a database wouldn't work either.) And they could still be distinguished by using 'is' instead of '=='. On Fri, Apr 12, 2013 at 2:17 PM, R. David Murray wrote: > On Fri, 12 Apr 2013 14:06:55 -0700, Eli Bendersky wrote: >> I actually think that having values with different types within a single >> Enum is conceptually wrong and should be disallowed at creation time. With >> enums, you either care or don't care about their actual value. If you don't >> care (the most common use case of an enum, IMHO), then no problem here. If >> you do care, then it's probably for very specific reasons most of which are >> solved by IntEnum. I can't imagine why someone would need differently typed >> values in a single enum - this just seems like a completely inappropriate >> use of an enum to me. > > I'm sure someone will come up with one :) > > But seriously, even if you require all values to be of the same type, > that doesn't solve the sorting problem: > >>>> class Foo(enum.Enum): > ... aa = object() > ... bb = object() > ... >>>> Foo > Traceback (most recent call last): > File "", line 1, in > File "./enum.py", line 103, in __repr__ > for k in sorted(cls._enums))) > TypeError: unorderable types: object() < object() > > Now, you could *further* require that the type of enum values be > sortable....but that point you really have no excuse for not allowing > enum values to be compared :) > > --David > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) From eliben at gmail.com Fri Apr 12 23:54:18 2013 From: eliben at gmail.com (Eli Bendersky) Date: Fri, 12 Apr 2013 14:54:18 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412211702.D393D250BD2@webabinitio.net> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> Message-ID: On Fri, Apr 12, 2013 at 2:17 PM, R. David Murray wrote: > On Fri, 12 Apr 2013 14:06:55 -0700, Eli Bendersky > wrote: > > I actually think that having values with different types within a single > > Enum is conceptually wrong and should be disallowed at creation time. > With > > enums, you either care or don't care about their actual value. If you > don't > > care (the most common use case of an enum, IMHO), then no problem here. > If > > you do care, then it's probably for very specific reasons most of which > are > > solved by IntEnum. I can't imagine why someone would need differently > typed > > values in a single enum - this just seems like a completely inappropriate > > use of an enum to me. > > I'm sure someone will come up with one :) > > Which is precisely the reason to ban it :) > But seriously, even if you require all values to be of the same type, > that doesn't solve the sorting problem: > > >>> class Foo(enum.Enum): > ... aa = object() > ... bb = object() > ... > >>> Foo > Traceback (most recent call last): > File "", line 1, in > File "./enum.py", line 103, in __repr__ > for k in sorted(cls._enums))) > TypeError: unorderable types: object() < object() > > Now, you could *further* require that the type of enum values be > sortable....but that point you really have no excuse for not allowing > enum values to be compared :) > I'm actually not really in favor of enum values being comparable. I think this is more a C-ism and does not cleanly settle with my concept of what an enum is. For comparable enums and other C-derived properties, IntEnum is out there, so call it maybe ;-) Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Sat Apr 13 00:23:30 2013 From: barry at python.org (Barry Warsaw) Date: Fri, 12 Apr 2013 18:23:30 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412110254.0c4c7a44@anarchist> <20130412152657.6ece1ca3@anarchist> Message-ID: <20130412182330.5f676649@anarchist> On Apr 12, 2013, at 12:56 PM, Guido van Rossum wrote: >Agreed. I can't easily find that in the PEP though. It doesn't mention >__getitem__ and I can't find any examples of using []. Indeed, this looks like an omission in the PEP. flufl.enum's usage documentation definitely talks about this: http://pythonhosted.org/flufl.enum/docs/using.html specifically: http://pythonhosted.org/flufl.enum/docs/using.html#conversions Eli, care to add this to the PEP? >>>Should the metaclass-based API used to create IntEnum be documented, >>>so strongly motivated people can write their own crazy variants? >> >> I think you recommended against that in python-ideas :). > >I have changed my mind; I am now at least +0 on documenting the >metaclass craziness. It would be fine with me. I left it out of the flufl.enum docs and we left it out of the PEP after your original comments, but I'm pretty happy with the API and can't foresee us changing it (famous last words). FWIW, we use a special attribute called __value_factory__ on the Enum subclass to name the class used to create enum values. This is all there is to IntEnum: class IntEnum(Enum): """A specialized enumeration with values that are also integers.""" __value_factory__ = IntEnumValue and even the IntEnumValue class isn't that big. It can be even smaller in Python 3.4 because of the workarounds in flufl.enum for Python 2 compatibility, and deprecations. Eli, what do you think about documenting the extension API? Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Sat Apr 13 00:32:41 2013 From: barry at python.org (Barry Warsaw) Date: Fri, 12 Apr 2013 18:32:41 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412205216.7216B250BD2@webabinitio.net> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> Message-ID: <20130412183241.18d6749f@anarchist> On Apr 12, 2013, at 04:52 PM, R. David Murray wrote: >You are right, the problem of comparison of disparate types makes ordering >a non-starter. But by the same token that means you are going to have to >be consistent and give up on having a sorted iteration and a stable repr: Why do you make me cry? -Barry From barry at python.org Sat Apr 13 00:35:58 2013 From: barry at python.org (Barry Warsaw) Date: Fri, 12 Apr 2013 18:35:58 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412211702.D393D250BD2@webabinitio.net> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> Message-ID: <20130412183558.16b3f608@anarchist> On Apr 12, 2013, at 05:17 PM, R. David Murray wrote: >Now, you could *further* require that the type of enum values be >sortable....but that point you really have no excuse for not allowing >enum values to be compared :) I'd be more willing to give up on sorting for the base enum type's iteration and repr. It's not crucial functionality whereas I still don't want the base enum values to support ordered comparisons. -Barry From timothy.c.delaney at gmail.com Sat Apr 13 00:37:47 2013 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Sat, 13 Apr 2013 08:37:47 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412183241.18d6749f@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> Message-ID: On 13 April 2013 08:32, Barry Warsaw wrote: > On Apr 12, 2013, at 04:52 PM, R. David Murray wrote: > > >You are right, the problem of comparison of disparate types makes ordering > >a non-starter. But by the same token that means you are going to have to > >be consistent and give up on having a sorted iteration and a stable repr: > > Why do you make me cry? > Just using definition order as the stable iteration order would do the trick - no need for any comparisons at all. Subclasses (e.g. IntEnum) can then override it. You could then easily have a subclass that implemented comparisons defined based on iteration order. It makes sense not to have this in the base Enum class (it would be confusing). On a related note, I really would like to have the ordinal exposed if this were added. Tim Delaney -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Sat Apr 13 00:44:49 2013 From: barry at python.org (Barry Warsaw) Date: Fri, 12 Apr 2013 18:44:49 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> Message-ID: <20130412184449.6d1e9806@anarchist> On Apr 12, 2013, at 02:34 PM, Guido van Rossum wrote: >So, pragmatically, if e and f are values of the same enum class, >couldn't e f (where is any comparison operator) defer to >e.value f.value ? Or is the problem with being == and e >and f being different enum values with the same underlying value? But >that's already iffy, isn't it? (Barry's favorite solution for >serializing to a database wouldn't work either.) And they could still >be distinguished by using 'is' instead of '=='. If I'm parsing that correctly, yes, I think we don't want to defer to the enum.value for the base enum type because unrelated enumeration values should not compare equal. E.g. class Colors(Enum): red = 1 blue = 2 green = 3 class Animals(Enum): ant = 1 bee = 2 cat = 3 In this case, Animals.bee != Colors.blue. Of course, they would be == if they derived from IntEnums. While I personally recommend and use identity to compare enum types, it seems to be difficult to convince other users to also do so. We could enforce this by not implementing __eq__ and __ne__, but it seems worse to disallow this than to make it work. E.g. if shape.color is Colors.red: # works if shape.color == Colors.red: # throws an exception -Barry From tismer at stackless.com Sat Apr 13 00:58:30 2013 From: tismer at stackless.com (Christian Tismer) Date: Sat, 13 Apr 2013 00:58:30 +0200 Subject: [Python-Dev] Multiline Strings confusion it tutorial Message-ID: <51689196.2060001@stackless.com> I wanted to point a bling guy to the Python wiki: http://wiki.python.org/moin/BeginnersGuide/Programmers/SimpleExamples#preview and when reading a little bit, I found the entry on multiline strings. I found that example pretty contorted, because this is not a multiline string. Instead, there are multiple lines which define a single line string! Actually, the construct is even syntactically nothing else than a single line string which is handled by the parser, already. A multiline string is IMHO a string which value covers multiple lines, after whatever pre-processing was done. I don't think the given example is very helpful, but adds confusion. Where would I add such a complaint, usually? Or should I simply fix it? cheers - chris -- Christian Tismer :^) Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/ 14482 Potsdam : PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From guido at python.org Sat Apr 13 00:59:04 2013 From: guido at python.org (Guido van Rossum) Date: Fri, 12 Apr 2013 15:59:04 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412184449.6d1e9806@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> <20130412184449.6d1e9806@anarchist> Message-ID: On Fri, Apr 12, 2013 at 3:44 PM, Barry Warsaw wrote: > On Apr 12, 2013, at 02:34 PM, Guido van Rossum wrote: > >>So, pragmatically, if e and f are values of the same enum class, >>couldn't e f (where is any comparison operator) defer to >>e.value f.value ? Or is the problem with being == and e >>and f being different enum values with the same underlying value? But >>that's already iffy, isn't it? (Barry's favorite solution for >>serializing to a database wouldn't work either.) And they could still >>be distinguished by using 'is' instead of '=='. > > If I'm parsing that correctly, yes, I think we don't want to defer to the > enum.value for the base enum type because unrelated enumeration values should > not compare equal. > > E.g. > > class Colors(Enum): > red = 1 > blue = 2 > green = 3 > > class Animals(Enum): > ant = 1 > bee = 2 > cat = 3 > > In this case, Animals.bee != Colors.blue. No, my proposal was only meant for if the classes are the same. If the classes are different the comparison should always fail. > Of course, they would be == if they derived from IntEnums. > > While I personally recommend and use identity to compare enum types, it seems > to be difficult to convince other users to also do so. We could enforce this > by not implementing __eq__ and __ne__, but it seems worse to disallow this > than to make it work. E.g. > > if shape.color is Colors.red: > # works > > if shape.color == Colors.red: > # throws an exception Right, I *only* meant this as a way to differentiate between bee and wasp in: class Insect(Enum): wasp = 1 bee = 1 ant = 2 We'd have Insect.wasp == Insect.bee < Insect.ant but Insect.wasp is not Insect.bee. -- --Guido van Rossum (python.org/~guido) From rdmurray at bitdance.com Sat Apr 13 01:31:10 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 12 Apr 2013 19:31:10 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412183558.16b3f608@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> <20130412183558.16b3f608@anarchist> Message-ID: <20130412233111.2C991250BCA@webabinitio.net> On Fri, 12 Apr 2013 18:35:58 -0400, Barry Warsaw wrote: > On Apr 12, 2013, at 05:17 PM, R. David Murray wrote: > > >Now, you could *further* require that the type of enum values be > >sortable....but that point you really have no excuse for not allowing > >enum values to be compared :) > > I'd be more willing to give up on sorting for the base enum type's iteration > and repr. It's not crucial functionality whereas I still don't want the base > enum values to support ordered comparisons. That's fine with me. I'm just requesting that it be self-consistent. --David From steve at pearwood.info Sat Apr 13 01:56:10 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 13 Apr 2013 09:56:10 +1000 Subject: [Python-Dev] Multiline Strings confusion it tutorial In-Reply-To: <51689196.2060001@stackless.com> References: <51689196.2060001@stackless.com> Message-ID: <51689F1A.5060708@pearwood.info> On 13/04/13 08:58, Christian Tismer wrote: > I wanted to point a bling guy to the Python wiki: > > http://wiki.python.org/moin/BeginnersGuide/Programmers/SimpleExamples#preview [...] > Where would I add such a complaint, usually? > Or should I simply fix it? It's a wiki. You can fix it yourself, next time. I've done it this time. -- Steven From v+python at g.nevcal.com Sat Apr 13 02:13:02 2013 From: v+python at g.nevcal.com (Glenn Linderman) Date: Fri, 12 Apr 2013 17:13:02 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> <20130412184449.6d1e9806@anarchist> Message-ID: <5168A30E.9000202@g.nevcal.com> On 4/12/2013 3:59 PM, Guido van Rossum wrote: > class Insect(Enum): > wasp = 1 > bee = 1 > ant = 2 > > We'd have Insect.wasp == Insect.bee < Insect.ant but Insect.wasp is > not Insect.bee. can't define two names in the same enum to have the same value, per the PEP. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Sat Apr 13 01:55:15 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 12 Apr 2013 16:55:15 -0700 Subject: [Python-Dev] Multiline Strings confusion it tutorial In-Reply-To: <51689196.2060001@stackless.com> References: <51689196.2060001@stackless.com> Message-ID: <51689EE3.2030100@stoneleaf.us> On 04/12/2013 03:58 PM, Christian Tismer wrote: > I wanted to point a bling guy to the Python wiki: > > http://wiki.python.org/moin/BeginnersGuide/Programmers/SimpleExamples#preview > > and when reading a little bit, I found the entry on multiline strings. This entry? Defining multiline strings string = '''This is a string with embedded newlines. Also known as a tripled-quoted string. Whitespace at the beginning of lines is included, so the above line is indented but the others are not. ''' > I found that example pretty contorted, because this is not a multiline string. > Instead, there are multiple lines which define a single line string! > Actually, the construct is even syntactically nothing else than a single > line string which is handled by the parser, already. That string has four (4) embedded '\n's -- that makes it multiline. > A multiline string is IMHO a string which value covers multiple lines, after whatever > pre-processing was done. I don't think the given example is very helpful, > but adds confusion. And that string does cover multiple lines. > Where would I add such a complaint, usually? > Or should I simply fix it? Nothing broken here, move along, nothing to see... -- ~Ethan~ From guido at python.org Sat Apr 13 02:58:25 2013 From: guido at python.org (Guido van Rossum) Date: Fri, 12 Apr 2013 17:58:25 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5168A30E.9000202@g.nevcal.com> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> <20130412184449.6d1e9806@anarchist> <5168A30E.9000202@g.nevcal.com> Message-ID: Well, even better. :-) On Fri, Apr 12, 2013 at 5:13 PM, Glenn Linderman wrote: > On 4/12/2013 3:59 PM, Guido van Rossum wrote: > > class Insect(Enum): > wasp = 1 > bee = 1 > ant = 2 > > We'd have Insect.wasp == Insect.bee < Insect.ant but Insect.wasp is > not Insect.bee. > > > can't define two names in the same enum to have the same value, per the PEP. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) From steve at pearwood.info Sat Apr 13 04:07:27 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 13 Apr 2013 12:07:27 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412153302.402eca6a@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> Message-ID: <5168BDDF.1050707@pearwood.info> On 13/04/13 05:33, Barry Warsaw wrote: > On Apr 12, 2013, at 11:21 AM, Russell E. Owen wrote: > >> I, too, would strongly prefer to see ordering within an enum. I use >> home-made enums heavily in my code and find ordering comparisons useful >> there. > > This was all hashed out in gory detail on python-ideas. I feel strongly that > base EnumValues should be unordered, especially because the underlying values > can be of any type. What would you expect to happen in this case: > > class X(Enum): > a = 1 > b = 'hi' > > if X.a < myvalue < X.b: > # whaa? I would expect the same behaviour from enums that I get in Python 3 from non-enums. That is, if enums X.a and X.b happen to both be ints, or both strings, then comparisons should succeed, but if they are different types, I should get a TypeError. The above applies to related enums. If they are unrelated (e.g. Colours.red < Insects.ant) then I think TypeError is appropriate. -- Steven From guido at python.org Sat Apr 13 04:14:56 2013 From: guido at python.org (Guido van Rossum) Date: Fri, 12 Apr 2013 19:14:56 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5168BDDF.1050707@pearwood.info> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <5168BDDF.1050707@pearwood.info> Message-ID: On Fri, Apr 12, 2013 at 7:07 PM, Steven D'Aprano wrote: > On 13/04/13 05:33, Barry Warsaw wrote: >> >> On Apr 12, 2013, at 11:21 AM, Russell E. Owen wrote: >> >>> I, too, would strongly prefer to see ordering within an enum. I use >>> home-made enums heavily in my code and find ordering comparisons useful >>> there. >> >> >> This was all hashed out in gory detail on python-ideas. I feel strongly >> that >> base EnumValues should be unordered, especially because the underlying >> values >> can be of any type. What would you expect to happen in this case: >> >> class X(Enum): >> a = 1 >> b = 'hi' >> >> if X.a < myvalue < X.b: >> # whaa? > > > > I would expect the same behaviour from enums that I get in Python 3 from > non-enums. That is, if enums X.a and X.b happen to both be ints, or both > strings, then comparisons should succeed, but if they are different types, I > should get a TypeError. > > The above applies to related enums. If they are unrelated (e.g. Colours.red > < Insects.ant) then I think TypeError is appropriate. That's exactly what I'm trying to propose too, but Steven has the better words. :-) -- --Guido van Rossum (python.org/~guido) From steve at pearwood.info Sat Apr 13 04:51:43 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 13 Apr 2013 12:51:43 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5168A30E.9000202@g.nevcal.com> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> <20130412184449.6d1e9806@anarchist> <5168A30E.9000202@g.nevcal.com> Message-ID: <5168C83F.4050103@pearwood.info> On 13/04/13 10:13, Glenn Linderman wrote: > can't define two names in the same enum to have the same value, per the PEP. I think that's too strong a restriction. I would expect to be able to do this: class Insect(Enum): wsap = 1 # Oops, needed for backward compatibility, do not remove. wasp = 1 # Preferred. Use this in new code. bee = 2 ant = 3 Or at the very least: class Insect(Enum): wasp = wsap = 1 bee = 2 ant = 3 I googled on "C enum" and the very first hit includes a duplicate value: http://msdn.microsoft.com/en-AU/library/whbyts4t%28v=vs.80%29.aspx And two examples from asm-generic/errno.h: #define EWOULDBLOCK EAGAIN /* Operation would block */ #define EDEADLOCK EDEADLK What's the justification for this restriction? I have looked in the PEP, and didn't see one. -- Steven From eliben at gmail.com Sat Apr 13 04:58:20 2013 From: eliben at gmail.com (Eli Bendersky) Date: Fri, 12 Apr 2013 19:58:20 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130412182330.5f676649@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412152657.6ece1ca3@anarchist> <20130412182330.5f676649@anarchist> Message-ID: On Fri, Apr 12, 2013 at 3:23 PM, Barry Warsaw wrote: > On Apr 12, 2013, at 12:56 PM, Guido van Rossum wrote: > > >Agreed. I can't easily find that in the PEP though. It doesn't mention > >__getitem__ and I can't find any examples of using []. > > Indeed, this looks like an omission in the PEP. flufl.enum's usage > documentation definitely talks about this: > > http://pythonhosted.org/flufl.enum/docs/using.html > > specifically: > > http://pythonhosted.org/flufl.enum/docs/using.html#conversions > > Eli, care to add this to the PEP? > > Done. getattr did get a mention there, but now I made it more prominent and described __getitem__ access as well. > >>>Should the metaclass-based API used to create IntEnum be documented, > >>>so strongly motivated people can write their own crazy variants? > >> > >> I think you recommended against that in python-ideas :). > > > >I have changed my mind; I am now at least +0 on documenting the > >metaclass craziness. > > It would be fine with me. I left it out of the flufl.enum docs and we > left it > out of the PEP after your original comments, but I'm pretty happy with the > API > and can't foresee us changing it (famous last words). > > FWIW, we use a special attribute called __value_factory__ on the Enum > subclass > to name the class used to create enum values. This is all there is to > IntEnum: > > class IntEnum(Enum): > """A specialized enumeration with values that are also integers.""" > __value_factory__ = IntEnumValue > > and even the IntEnumValue class isn't that big. It can be even smaller in > Python 3.4 because of the workarounds in flufl.enum for Python 2 > compatibility, and deprecations. > > Eli, what do you think about documenting the extension API? > I don't have major objections... Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From storchaka at gmail.com Sat Apr 13 10:31:59 2013 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sat, 13 Apr 2013 11:31:59 +0300 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: Message-ID: On 12.04.13 15:55, Eli Bendersky wrote: > The enumeration value names are available through the class members:: > > >>> for member in Colors.__members__: > ... print(member) > red > green > blue This is unnecessary because enumerations are iterable. Colors.__members__ is equal to [v.name for v in Colors] and the latter looks more preferable, because it does not use the magic method. > The str and repr of the enumeration class also provides useful information:: > > >>> print(Colors) > > >>> print(repr(Colors)) > Does the enumeration's repr() use str() or repr() for the enumeration values? And same question for the enumeration's str(). > To programmatically access enumeration values, use ``getattr``:: > > >>> getattr(Colors, 'red') > How to get the enumeration value by its value? > Ordered comparisons between enumeration values are *not* supported. Enums > are > not integers (but see `IntEnum`_ below):: It's unexpected if values of the enumeration values have the natural order. And values of the enumeration values *should be* comparable ("Iteration is defined as the sorted order of the item values"). > Enumeration values > ------------------ There is some ambiguity in the term "enumeration values". On the one hand, it's the singleton instances of the enumeration class (Colors.red, Colors.gree, Colors.blue), and on the other hand it is their values (1, 2, 3). > But if the value *is* important, enumerations can have arbitrary values. Should enumeration values be hashable? At least they should be comparable ("Iteration is defined as the sorted order of the item values"). > ``IntEnum`` values behave like integers in other ways you'd expect:: > > >>> int(Shape.circle) > 1 > >>> ['a', 'b', 'c'][Shape.circle] > 'b' > >>> [i for i in range(Shape.square)] > [0, 1] What is ``isinstance(Shape.circle, int)``? Does PyLong_Check() return true for ``IntEnum`` values? > Enumerations created with the class syntax can also be pickled and > unpickled:: This does not apply to marshalling, I suppose? Perhaps this is worth to mention explicitly. There may be some errors of incompatibility. > The ``Enum`` class is callable, providing the following convenience API:: > > >>> Animals = Enum('Animals', 'ant bee cat dog') > >>> Animals > > >>> Animals.ant > > >>> Animals.ant.value > 1 > > The semantics of this API resemble ``namedtuple``. The first argument of > the call to ``Enum`` is the name of the enumeration. The second argument is > a source of enumeration value names. It can be a whitespace-separated > string > of names, a sequence of names or a sequence of 2-tuples with key/value > pairs. Why the enumeration starts from 1? It is not consistent with namedtuple, in which indices are zero-based, and I believe that in most practical cases the enumeration integer values are zero-based. > Use-cases in the standard library > ================================= The Python standard library has many places where named integer constants used as bitmasks (i.e. os.O_CREAT | os.O_WRONLY | os.O_TRUNC, select.POLLIN | select.POLLPRI, re.IGNORECASE | re.ASCII). The proposed PEP is not applicable to these cases. Whether it is planned expansion of Enum or additional EnumSet class to aid in these cases? From ben+python at benfinney.id.au Sat Apr 13 12:30:48 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 13 Apr 2013 20:30:48 +1000 Subject: [Python-Dev] Deciding against the CLA (was: Introducing Electronic Contributor Agreements) References: <87k3pmk07r.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <7wwqs667tz.fsf@benfinney.id.au> "Stephen J. Turnbull" writes: > Mark Lawrence writes: > > > People already use the bug tracker as an excuse not to contribute, > > wouldn't this requirement make the situation worse? > > A failure to sign the CLA is already a decision not to contribute to > the distribution As someone who cannot in good faith sign the CLA, that characterisation is far from accurate: I would very much like to contribute to the Python distribution, and so have not decided as you describe. Rather, I leave the matter of contribution undecided, while advocating (when opportunity arises) against the CLA. The decision that the current terms are unacceptable does not entail a decision not to contribute. (aside: good sigmonster, have a treat.) -- \ Lucifer: ?Just sign the Contract, sir, and the Piano is yours.? | `\ Ray: ?Sheesh! This is long! Mind if I sign it now and read it | _o__) later?? ?http://www.achewood.com/ | Ben Finney From eliben at gmail.com Sat Apr 13 14:43:32 2013 From: eliben at gmail.com (Eli Bendersky) Date: Sat, 13 Apr 2013 05:43:32 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: Message-ID: On Sat, Apr 13, 2013 at 1:31 AM, Serhiy Storchaka wrote: > On 12.04.13 15:55, Eli Bendersky wrote: > >> The enumeration value names are available through the class members:: >> >> >>> for member in Colors.__members__: >> ... print(member) >> red >> green >> blue >> > > This is unnecessary because enumerations are iterable. Colors.__members__ > is equal to [v.name for v in Colors] and the latter looks more > preferable, because it does not use the magic method. > > Right. Fixed (removed this part because it's redundant), thanks. > > The str and repr of the enumeration class also provides useful >> information:: >> >> >>> print(Colors) >> >> >>> print(repr(Colors)) >> >> > > Does the enumeration's repr() use str() or repr() for the enumeration > values? And same question for the enumeration's str(). > > str > > To programmatically access enumeration values, use ``getattr``:: >> >> >>> getattr(Colors, 'red') >> >> > > How to get the enumeration value by its value? > > I've updated the PEP since then. It also shows how to use __getitem__ syntax to access by value. > > Ordered comparisons between enumeration values are *not* supported. Enums >> are >> not integers (but see `IntEnum`_ below):: >> > > It's unexpected if values of the enumeration values have the natural > order. And values of the enumeration values *should be* comparable > ("Iteration is defined as the sorted order of the item values"). > > Enumeration values >> ------------------ >> > > There is some ambiguity in the term "enumeration values". On the one hand, > it's the singleton instances of the enumeration class (Colors.red, > Colors.gree, Colors.blue), and on the other hand it is their values (1, 2, > 3). > > I agree, but not sure how to resolve it. I hope it's clear enough from the context. > > But if the value *is* important, enumerations can have arbitrary values. >> > > Should enumeration values be hashable? > > At least they should be comparable ("Iteration is defined as the sorted > order of the item values"). > > See long discussion previously in this thread. > > ``IntEnum`` values behave like integers in other ways you'd expect:: >> >> >>> int(Shape.circle) >> 1 >> >>> ['a', 'b', 'c'][Shape.circle] >> 'b' >> >>> [i for i in range(Shape.square)] >> [0, 1] >> > > What is ``isinstance(Shape.circle, int)``? Does PyLong_Check() return true > for ``IntEnum`` values? > > Yes. IntEnumValue (the value class underlying IntEnum) subclasses int. > > Enumerations created with the class syntax can also be pickled and >> unpickled:: >> > > This does not apply to marshalling, I suppose? Perhaps this is worth to > mention explicitly. There may be some errors of incompatibility. No special provision has been made for marshalling. The ``Enum`` class is callable, providing the following convenience API:: > > >>> Animals = Enum('Animals', 'ant bee cat dog') > >>> Animals > > >>> Animals.ant > > >>> Animals.ant.value > 1 > > The semantics of this API resemble ``namedtuple``. The first argument of > the call to ``Enum`` is the name of the enumeration. The second argument > is > a source of enumeration value names. It can be a whitespace-separated > string > of names, a sequence of names or a sequence of 2-tuples with key/value > pairs. > Why the enumeration starts from 1? It is not consistent with namedtuple, in > which indices are zero-based, and I believe that in most practical cases > the enumeration integer values are zero-based. I don't know if there was a special reason for this. Perhaps backwards compatibility with existing flufl.enum APIs. Barry may know more about this. > Use-cases in the standard library > ================================= > The Python standard library has many places where named integer constants > used as bitmasks (i.e. os.O_CREAT | os.O_WRONLY | os.O_TRUNC, select.POLLIN > | select.POLLPRI, re.IGNORECASE | re.ASCII). The proposed PEP is not > applicable to these cases. Whether it is planned expansion of Enum or > additional EnumSet class to aid in these cases? It is applicable, in the sense that os.O_CREAT etc can be IntEnum values. Their bitset operation results will be simple integers. It's not planned to add a special enum for this - this was ruled against during the Pycon discussions. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From regebro at gmail.com Sat Apr 13 15:43:18 2013 From: regebro at gmail.com (Lennart Regebro) Date: Sat, 13 Apr 2013 15:43:18 +0200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: Message-ID: OK, so I finally got tie to read the PEP. I like it, I really have missed Enums, this is awesome. That's all folks! //Lennart From eliben at gmail.com Sat Apr 13 16:24:19 2013 From: eliben at gmail.com (Eli Bendersky) Date: Sat, 13 Apr 2013 07:24:19 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: Message-ID: On Sat, Apr 13, 2013 at 6:43 AM, Lennart Regebro wrote: > OK, so I finally got tie to read the PEP. I like it, I really have > missed Enums, this is awesome. > > That's all folks! > > //Lennart > More of these kinds of comments ;-) Thanks, Lennart. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eliben at gmail.com Sat Apr 13 16:25:33 2013 From: eliben at gmail.com (Eli Bendersky) Date: Sat, 13 Apr 2013 07:25:33 -0700 Subject: [Python-Dev] [Python-checkins] cpython (2.7): Issue #16447: Fix potential segfault when setting __name__ on a class. In-Reply-To: <3Znylg6gCJzS33@mail.python.org> References: <3Znylg6gCJzS33@mail.python.org> Message-ID: Test case? On Sat, Apr 13, 2013 at 7:19 AM, mark.dickinson wrote: > http://hg.python.org/cpython/rev/d5e5017309b1 > changeset: 83283:d5e5017309b1 > branch: 2.7 > user: Mark Dickinson > date: Sat Apr 13 15:19:05 2013 +0100 > summary: > Issue #16447: Fix potential segfault when setting __name__ on a class. > > files: > Lib/test/test_descr.py | 14 ++++++++++++++ > Misc/NEWS | 3 +++ > Objects/typeobject.c | 6 +++++- > 3 files changed, 22 insertions(+), 1 deletions(-) > > > diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py > --- a/Lib/test/test_descr.py > +++ b/Lib/test/test_descr.py > @@ -4136,6 +4136,20 @@ > C.__name__ = 'D.E' > self.assertEqual((C.__module__, C.__name__), (mod, 'D.E')) > > + def test_evil_type_name(self): > + # A badly placed Py_DECREF in type_set_name led to arbitrary code > + # execution while the type structure was not in a sane state, and > a > + # possible segmentation fault as a result. See bug #16447. > + class Nasty(str): > + def __del__(self): > + C.__name__ = "other" > + > + class C(object): > + pass > + > + C.__name__ = Nasty("abc") > + C.__name__ = "normal" > + > def test_subclass_right_op(self): > # Testing correct dispatch of subclass overloading __r__... > > diff --git a/Misc/NEWS b/Misc/NEWS > --- a/Misc/NEWS > +++ b/Misc/NEWS > @@ -17,6 +17,9 @@ > Core and Builtins > ----------------- > > +- Issue #16447: Fixed potential segmentation fault when setting __name__ > on a > + class. > + > - Issue #17610: Don't rely on non-standard behavior of the C qsort() > function. > > Library > diff --git a/Objects/typeobject.c b/Objects/typeobject.c > --- a/Objects/typeobject.c > +++ b/Objects/typeobject.c > @@ -225,6 +225,7 @@ > type_set_name(PyTypeObject *type, PyObject *value, void *context) > { > PyHeapTypeObject* et; > + PyObject *tmp; > > if (!(type->tp_flags & Py_TPFLAGS_HEAPTYPE)) { > PyErr_Format(PyExc_TypeError, > @@ -253,10 +254,13 @@ > > Py_INCREF(value); > > - Py_DECREF(et->ht_name); > + /* Wait until et is a sane state before Py_DECREF'ing the old > et->ht_name > + value. (Bug #16447.) */ > + tmp = et->ht_name; > et->ht_name = value; > > type->tp_name = PyString_AS_STRING(value); > + Py_DECREF(tmp); > > return 0; > } > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Sat Apr 13 16:27:00 2013 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 13 Apr 2013 10:27:00 -0400 Subject: [Python-Dev] [Python-checkins] cpython (2.7): Issue #16447: Fix potential segfault when setting __name__ on a class. In-Reply-To: References: <3Znylg6gCJzS33@mail.python.org> Message-ID: 2013/4/13 Eli Bendersky : > Test case? I see one. > > > On Sat, Apr 13, 2013 at 7:19 AM, mark.dickinson > wrote: >> >> http://hg.python.org/cpython/rev/d5e5017309b1 >> changeset: 83283:d5e5017309b1 >> branch: 2.7 >> user: Mark Dickinson >> date: Sat Apr 13 15:19:05 2013 +0100 >> summary: >> Issue #16447: Fix potential segfault when setting __name__ on a class. >> >> files: >> Lib/test/test_descr.py | 14 ++++++++++++++ >> Misc/NEWS | 3 +++ >> Objects/typeobject.c | 6 +++++- >> 3 files changed, 22 insertions(+), 1 deletions(-) >> >> >> diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py >> --- a/Lib/test/test_descr.py >> +++ b/Lib/test/test_descr.py >> @@ -4136,6 +4136,20 @@ >> C.__name__ = 'D.E' >> self.assertEqual((C.__module__, C.__name__), (mod, 'D.E')) >> >> + def test_evil_type_name(self): >> + # A badly placed Py_DECREF in type_set_name led to arbitrary code >> + # execution while the type structure was not in a sane state, and >> a >> + # possible segmentation fault as a result. See bug #16447. >> + class Nasty(str): >> + def __del__(self): >> + C.__name__ = "other" >> + >> + class C(object): >> + pass >> + >> + C.__name__ = Nasty("abc") >> + C.__name__ = "normal" >> + >> def test_subclass_right_op(self): >> # Testing correct dispatch of subclass overloading __r__... >> >> diff --git a/Misc/NEWS b/Misc/NEWS >> --- a/Misc/NEWS >> +++ b/Misc/NEWS >> @@ -17,6 +17,9 @@ >> Core and Builtins >> ----------------- >> >> +- Issue #16447: Fixed potential segmentation fault when setting __name__ >> on a >> + class. >> + >> - Issue #17610: Don't rely on non-standard behavior of the C qsort() >> function. >> >> Library >> diff --git a/Objects/typeobject.c b/Objects/typeobject.c >> --- a/Objects/typeobject.c >> +++ b/Objects/typeobject.c >> @@ -225,6 +225,7 @@ >> type_set_name(PyTypeObject *type, PyObject *value, void *context) >> { >> PyHeapTypeObject* et; >> + PyObject *tmp; >> >> if (!(type->tp_flags & Py_TPFLAGS_HEAPTYPE)) { >> PyErr_Format(PyExc_TypeError, >> @@ -253,10 +254,13 @@ >> >> Py_INCREF(value); >> >> - Py_DECREF(et->ht_name); >> + /* Wait until et is a sane state before Py_DECREF'ing the old >> et->ht_name >> + value. (Bug #16447.) */ >> + tmp = et->ht_name; >> et->ht_name = value; >> >> type->tp_name = PyString_AS_STRING(value); >> + Py_DECREF(tmp); >> >> return 0; >> } >> >> -- >> Repository URL: http://hg.python.org/cpython >> >> _______________________________________________ >> Python-checkins mailing list >> Python-checkins at python.org >> http://mail.python.org/mailman/listinfo/python-checkins >> > > > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > -- Regards, Benjamin From eliben at gmail.com Sat Apr 13 16:26:57 2013 From: eliben at gmail.com (Eli Bendersky) Date: Sat, 13 Apr 2013 07:26:57 -0700 Subject: [Python-Dev] [Python-checkins] cpython (2.7): Issue #16447: Fix potential segfault when setting __name__ on a class. In-Reply-To: References: <3Znylg6gCJzS33@mail.python.org> Message-ID: On Sat, Apr 13, 2013 at 7:25 AM, Eli Bendersky wrote: > Test case? > > Ugh, sorry. I missed it. Ignore my previous email please. Eli > > On Sat, Apr 13, 2013 at 7:19 AM, mark.dickinson < > python-checkins at python.org> wrote: > >> http://hg.python.org/cpython/rev/d5e5017309b1 >> changeset: 83283:d5e5017309b1 >> branch: 2.7 >> user: Mark Dickinson >> date: Sat Apr 13 15:19:05 2013 +0100 >> summary: >> Issue #16447: Fix potential segfault when setting __name__ on a class. >> >> files: >> Lib/test/test_descr.py | 14 ++++++++++++++ >> Misc/NEWS | 3 +++ >> Objects/typeobject.c | 6 +++++- >> 3 files changed, 22 insertions(+), 1 deletions(-) >> >> >> diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py >> --- a/Lib/test/test_descr.py >> +++ b/Lib/test/test_descr.py >> @@ -4136,6 +4136,20 @@ >> C.__name__ = 'D.E' >> self.assertEqual((C.__module__, C.__name__), (mod, 'D.E')) >> >> + def test_evil_type_name(self): >> + # A badly placed Py_DECREF in type_set_name led to arbitrary code >> + # execution while the type structure was not in a sane state, >> and a >> + # possible segmentation fault as a result. See bug #16447. >> + class Nasty(str): >> + def __del__(self): >> + C.__name__ = "other" >> + >> + class C(object): >> + pass >> + >> + C.__name__ = Nasty("abc") >> + C.__name__ = "normal" >> + >> def test_subclass_right_op(self): >> # Testing correct dispatch of subclass overloading __r__... >> >> diff --git a/Misc/NEWS b/Misc/NEWS >> --- a/Misc/NEWS >> +++ b/Misc/NEWS >> @@ -17,6 +17,9 @@ >> Core and Builtins >> ----------------- >> >> +- Issue #16447: Fixed potential segmentation fault when setting __name__ >> on a >> + class. >> + >> - Issue #17610: Don't rely on non-standard behavior of the C qsort() >> function. >> >> Library >> diff --git a/Objects/typeobject.c b/Objects/typeobject.c >> --- a/Objects/typeobject.c >> +++ b/Objects/typeobject.c >> @@ -225,6 +225,7 @@ >> type_set_name(PyTypeObject *type, PyObject *value, void *context) >> { >> PyHeapTypeObject* et; >> + PyObject *tmp; >> >> if (!(type->tp_flags & Py_TPFLAGS_HEAPTYPE)) { >> PyErr_Format(PyExc_TypeError, >> @@ -253,10 +254,13 @@ >> >> Py_INCREF(value); >> >> - Py_DECREF(et->ht_name); >> + /* Wait until et is a sane state before Py_DECREF'ing the old >> et->ht_name >> + value. (Bug #16447.) */ >> + tmp = et->ht_name; >> et->ht_name = value; >> >> type->tp_name = PyString_AS_STRING(value); >> + Py_DECREF(tmp); >> >> return 0; >> } >> >> -- >> Repository URL: http://hg.python.org/cpython >> >> _______________________________________________ >> Python-checkins mailing list >> Python-checkins at python.org >> http://mail.python.org/mailman/listinfo/python-checkins >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From storchaka at gmail.com Sat Apr 13 16:30:36 2013 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sat, 13 Apr 2013 17:30:36 +0300 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: Message-ID: On 13.04.13 15:43, Eli Bendersky wrote: > On Sat, Apr 13, 2013 at 1:31 AM, Serhiy Storchaka wrote: >> On 12.04.13 15:55, Eli Bendersky wrote: >> There is some ambiguity in the term "enumeration values". On the one hand, >> it's the singleton instances of the enumeration class (Colors.red, >> Colors.gree, Colors.blue), and on the other hand it is their values (1, 2, >> 3). >> >> > I agree, but not sure how to resolve it. I hope it's clear enough from the > context. May be use "enumeration items" or "enumeration members" if instances of the enumeration class have in mind? And left "enumeration names" and "enumeration values" for sets of corresponding attributes (.name and .value) of instances. >>> But if the value *is* important, enumerations can have arbitrary values. >> >> Should enumeration values be hashable? >> >> At least they should be comparable ("Iteration is defined as the sorted >> order of the item values"). >> >> > See long discussion previously in this thread. I think this requirements (hashability and comparability (for repr() and iteration)) should be mentioned explicitly. >> The Python standard library has many places where named integer constants >> used as bitmasks (i.e. os.O_CREAT | os.O_WRONLY | os.O_TRUNC, select.POLLIN >> | select.POLLPRI, re.IGNORECASE | re.ASCII). The proposed PEP is not >> applicable to these cases. Whether it is planned expansion of Enum or >> additional EnumSet class to aid in these cases? > > It is applicable, in the sense that os.O_CREAT etc can be IntEnum values. > Their bitset operation results will be simple integers. It's not planned to > add a special enum for this - this was ruled against during the Pycon > discussions. But IntEnum is useless in such cases because a resulting mask will be an integer an will lost its convenient printable representation. There is almost no benefit of IntEnum before int constant. From storchaka at gmail.com Sat Apr 13 16:45:48 2013 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sat, 13 Apr 2013 17:45:48 +0300 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5168A30E.9000202@g.nevcal.com> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> <20130412184449.6d1e9806@anarchist> <5168A30E.9000202@g.nevcal.com> Message-ID: On 13.04.13 03:13, Glenn Linderman wrote: > On 4/12/2013 3:59 PM, Guido van Rossum wrote: >> class Insect(Enum): >> wasp = 1 >> bee = 1 >> ant = 2 >> >> We'd have Insect.wasp == Insect.bee < Insect.ant but Insect.wasp is >> not Insect.bee. > > can't define two names in the same enum to have the same value, per the > PEP. For current flufl.enum implementations this requires values to be hashable. An alternative implementation can use comparability (which already required for repr() and iteration). From guido at python.org Sat Apr 13 17:33:42 2013 From: guido at python.org (Guido van Rossum) Date: Sat, 13 Apr 2013 08:33:42 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: Message-ID: [Eli] >> It is applicable, in the sense that os.O_CREAT etc can be IntEnum values. >> Their bitset operation results will be simple integers. It's not planned to >> add a special enum for this - this was ruled against during the Pycon >> discussions. On Sat, Apr 13, 2013 at 7:30 AM, Serhiy Storchaka wrote: > But IntEnum is useless in such cases because a resulting mask will be an > integer an will lost its convenient printable representation. There is > almost no benefit of IntEnum before int constant. If you really wanted that you could define an int subclass that does better and use that -- IntEnum is just one example of how you can override the value type of Enum. (And yes, I am now +1 on documenting this mechanism.) -- --Guido van Rossum (python.org/~guido) From brett at python.org Sat Apr 13 18:26:40 2013 From: brett at python.org (Brett Cannon) Date: Sat, 13 Apr 2013 12:26:40 -0400 Subject: [Python-Dev] Deciding against the CLA (was: Introducing Electronic Contributor Agreements) In-Reply-To: <7wwqs667tz.fsf@benfinney.id.au> References: <87k3pmk07r.fsf@uwakimon.sk.tsukuba.ac.jp> <7wwqs667tz.fsf@benfinney.id.au> Message-ID: On Sat, Apr 13, 2013 at 6:30 AM, Ben Finney wrote: > "Stephen J. Turnbull" writes: > >> Mark Lawrence writes: >> >> > People already use the bug tracker as an excuse not to contribute, >> > wouldn't this requirement make the situation worse? >> >> A failure to sign the CLA is already a decision not to contribute to >> the distribution > > As someone who cannot in good faith sign the CLA, that characterisation > is far from accurate: I would very much like to contribute to the Python > distribution, and so have not decided as you describe. > > Rather, I leave the matter of contribution undecided, while advocating > (when opportunity arises) against the CLA. > > The decision that the current terms are unacceptable does not entail a > decision not to contribute. Stephen said that it's a choice not to contribute and not that one wouldn't _like_ to contribute if the CLA wasn't there. Those are both distinctive choices to make. A desire to help is independent of whether you are willing to take the necessary step of signing the CLA in order to change that desire into an actual act of contributing (which is obviously fine; if you have moral issues with the CLA no one will hold it against you, we just can't legally risk accepting code without it). -Brett > > (aside: good sigmonster, have a treat.) > > -- > \ Lucifer: ?Just sign the Contract, sir, and the Piano is yours.? | > `\ Ray: ?Sheesh! This is long! Mind if I sign it now and read it | > _o__) later?? ?http://www.achewood.com/ | > Ben Finney > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org From scott+python-dev at scottdial.com Sat Apr 13 20:15:37 2013 From: scott+python-dev at scottdial.com (Scott Dial) Date: Sat, 13 Apr 2013 14:15:37 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5168C83F.4050103@pearwood.info> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> <20130412184449.6d1e9806@anarchist> <5168A30E.9000202@g.nevcal.com> <5168C83F.4050103@pearwood.info> Message-ID: <5169A0C9.9090200@scottdial.com> On 4/12/2013 10:51 PM, Steven D'Aprano wrote: > And two examples from asm-generic/errno.h: > > #define EWOULDBLOCK EAGAIN /* Operation would block */ > #define EDEADLOCK EDEADLK > That's actually even better of an example than you may have realized because historically EWOULDBLOCK != EAGAIN[1]. So, there very well may need to exist such code as: if : _EAGAIN = _EWOULDBLOCK = else: _EAGAIN = _EWOULDBLOCK = class Errno(Enum): EAGAIN = _EAGAIN EWOULDBLOCK = _EWOULDBLOCK I don't think it's all that uncommon that enum values that represent states of a system get merged or renamed over time, and this one is a great example of that. [1] http://www.gnu.org/savannah-checkouts/gnu/libc/manual/html_node/Error-Codes.html#index-EAGAIN-97 -- Scott Dial scott at scottdial.com From ben+python at benfinney.id.au Sun Apr 14 04:53:34 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 14 Apr 2013 12:53:34 +1000 Subject: [Python-Dev] [Announcement] New mailing list for code quality tools including Flake8, Pyflakes and Pep8 References: Message-ID: <7wmwt16cwh.fsf@benfinney.id.au> Ian Cordasco writes: > Are you concerned about the evolution of various code checkers? > Do you have questions or suggestions? > > Subscribe here: > http://mail.python.org/mailman/listinfo/code-quality Now available via Gmane also . Thanks guys! -- \ ?I have never imputed to Nature a purpose or a goal, or | `\ anything that could be understood as anthropomorphic.? ?Albert | _o__) Einstein, unsent letter, 1955 | Ben Finney From guido at python.org Sun Apr 14 06:08:34 2013 From: guido at python.org (Guido van Rossum) Date: Sat, 13 Apr 2013 21:08:34 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5169A0C9.9090200@scottdial.com> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> <20130412184449.6d1e9806@anarchist> <5168A30E.9000202@g.nevcal.com> <5168C83F.4050103@pearwood.info> <5169A0C9.9090200@scottdial.com> Message-ID: So clearly the ree expectation is that they compare equal (if the values are). On Apr 13, 2013 11:24 AM, "Scott Dial" wrote: > On 4/12/2013 10:51 PM, Steven D'Aprano wrote: > > And two examples from asm-generic/errno.h: > > > > #define EWOULDBLOCK EAGAIN /* Operation would block */ > > #define EDEADLOCK EDEADLK > > > > That's actually even better of an example than you may have realized > because historically EWOULDBLOCK != EAGAIN[1]. So, there very well may > need to exist such code as: > > if : > _EAGAIN = > _EWOULDBLOCK = > else: > _EAGAIN = _EWOULDBLOCK = > > class Errno(Enum): > EAGAIN = _EAGAIN > EWOULDBLOCK = _EWOULDBLOCK > > I don't think it's all that uncommon that enum values that represent > states of a system get merged or renamed over time, and this one is a > great example of that. > > [1] > > http://www.gnu.org/savannah-checkouts/gnu/libc/manual/html_node/Error-Codes.html#index-EAGAIN-97 > > -- > Scott Dial > scott at scottdial.com > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mail at timgolden.me.uk Sat Apr 13 21:26:24 2013 From: mail at timgolden.me.uk (Tim Golden) Date: Sat, 13 Apr 2013 20:26:24 +0100 Subject: [Python-Dev] casefolding in pathlib (PEP 428) In-Reply-To: <51687961.8010602@hastings.org> References: <20130411232706.51790b7a@pitrou.net> <20130412103954.4c96451a@pitrou.net> <51687961.8010602@hastings.org> Message-ID: <5169B160.8070704@timgolden.me.uk> On 12/04/2013 22:15, Larry Hastings wrote: > > On 04/12/2013 10:05 AM, Guido van Rossum wrote: >> On Fri, Apr 12, 2013 at 1:39 AM, Antoine Pitrou >> wrote: >>> I think we want glob("*.py") to find >>> "SETUP.PY" under Windows. Anything else will probably be surprising to >>> users of that platform. >> Yeah, I suppose so. But there are more crazy details. E.g. IIRC >> Windows silently ignores trailing dots in filenames. Do we want >> "*.py." to match SETUP.PY then? > > Someone who is fresher than I am at Windows programming should answer > this, but AFAICT Win32 provides no API that will tell you if a > particular filename / volume is case sensitive. I don't have web access at the moment to check but IIRC the GetVolumeInformation call does return an indicator of whether the volume is case-sensitive via the VOLUME_FLAG flag enum. At least, it claims to: I don't have access to a case-sensitive filesystem to check whether it's lying or not. TJG From ncoghlan at gmail.com Sun Apr 14 13:49:05 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 14 Apr 2013 21:49:05 +1000 Subject: [Python-Dev] [Python-checkins] cpython: Issue #17693: CJK encoders now use the new Unicode API (PEP 393) In-Reply-To: <3ZpCnM5YLlzRl3@mail.python.org> References: <3ZpCnM5YLlzRl3@mail.python.org> Message-ID: On Sun, Apr 14, 2013 at 10:06 AM, victor.stinner wrote: > http://hg.python.org/cpython/rev/d621bdaed7c3 > changeset: 83317:d621bdaed7c3 > user: Victor Stinner > date: Sun Apr 14 02:06:32 2013 +0200 > summary: > Issue #17693: CJK encoders now use the new Unicode API (PEP 393) I'm going to run with the wild theory that there's a reference link in here somewhere... :) results for 0ee785c9d1b4 on branch "default" ------------------------------ -------------- test_unittest leaked [-1, 2, 1] memory blocks, sum=2 test_codecencodings_cn leaked [16133, 16133, 16133] references, sum=48399 test_codecencodings_cn leaked [28992, 28994, 28994] memory blocks, sum=86980 test_codecencodings_hk leaked [378, 378, 378] references, sum=1134 test_codecencodings_hk leaked [414, 416, 416] memory blocks, sum=1246 test_codecencodings_iso2022 leaked [13861, 13861, 13861] references, sum=41583 test_codecencodings_iso2022 leaked [25698, 25700, 25700] memory blocks, sum=77098 test_codecencodings_jp leaked [39322, 39322, 39322] references, sum=117966 test_codecencodings_jp leaked [71335, 71337, 71337] memory blocks, sum=214009 test_codecencodings_kr leaked [9033, 9033, 9033] references, sum=27099 test_codecencodings_kr leaked [15844, 15846, 15846] memory blocks, sum=47536 test_codecencodings_tw leaked [3879, 3879, 3879] references, sum=11637 test_codecencodings_tw leaked [6562, 6564, 6564] memory blocks, sum=19690 test_codecs leaked [1296, 1296, 1296] references, sum=3888 test_codecs leaked [384, 386, 386] memory blocks, sum=1156 test_multibytecodec leaked [44, 44, 44] references, sum=132 test_multibytecodec leaked [21, 23, 23] memory blocks, sum=67 test_traceback leaked [2, 2, 2] references, sum=6 test_traceback leaked [4, 6, 6] memory blocks, sum=16 Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Sun Apr 14 13:50:42 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 14 Apr 2013 21:50:42 +1000 Subject: [Python-Dev] [Python-checkins] cpython: Issue #17693: CJK encoders now use the new Unicode API (PEP 393) In-Reply-To: References: <3ZpCnM5YLlzRl3@mail.python.org> Message-ID: On Sun, Apr 14, 2013 at 9:49 PM, Nick Coghlan wrote: > On Sun, Apr 14, 2013 at 10:06 AM, victor.stinner > wrote: >> http://hg.python.org/cpython/rev/d621bdaed7c3 >> changeset: 83317:d621bdaed7c3 >> user: Victor Stinner >> date: Sun Apr 14 02:06:32 2013 +0200 >> summary: >> Issue #17693: CJK encoders now use the new Unicode API (PEP 393) > > I'm going to run with the wild theory that there's a reference link in > here somewhere... :) Or, perhaps even a reference *leak*! I can't even blame autocorrect for that one, since I typed it at my desk on a real keyboard :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From victor.stinner at gmail.com Sun Apr 14 19:36:16 2013 From: victor.stinner at gmail.com (Victor Stinner) Date: Sun, 14 Apr 2013 19:36:16 +0200 Subject: [Python-Dev] [Python-checkins] cpython: Issue #17693: CJK encoders now use the new Unicode API (PEP 393) In-Reply-To: References: <3ZpCnM5YLlzRl3@mail.python.org> Message-ID: Hi, 2013/4/14 Nick Coghlan : > On Sun, Apr 14, 2013 at 10:06 AM, victor.stinner > wrote: >> http://hg.python.org/cpython/rev/d621bdaed7c3 >> changeset: 83317:d621bdaed7c3 >> user: Victor Stinner >> date: Sun Apr 14 02:06:32 2013 +0200 >> summary: >> Issue #17693: CJK encoders now use the new Unicode API (PEP 393) > > I'm going to run with the wild theory that there's a reference link in > here somewhere... :) > > results for 0ee785c9d1b4 on branch "default" > ------------------------------ > -------------- > > test_codecencodings_cn leaked [16133, 16133, 16133] references, sum=48399 > test_codecencodings_cn leaked [28992, 28994, 28994] memory blocks, sum=86980 > ... Oh thanks for the report. My laptop is too slow to run the test suite using -R. It should be fixed by: New changeset ffd4b72f7f95 by Victor Stinner in branch 'default': Issue #17693: Fix memory/reference leaks http://hg.python.org/cpython/rev/ffd4b72f7f95 Victor From steve at pearwood.info Mon Apr 15 03:34:39 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 15 Apr 2013 11:34:39 +1000 Subject: [Python-Dev] Deciding against the CLA In-Reply-To: <7wwqs667tz.fsf@benfinney.id.au> References: <87k3pmk07r.fsf@uwakimon.sk.tsukuba.ac.jp> <7wwqs667tz.fsf@benfinney.id.au> Message-ID: <516B592F.6040207@pearwood.info> On 13/04/13 20:30, Ben Finney wrote: > "Stephen J. Turnbull" writes: > >> Mark Lawrence writes: >> >> > People already use the bug tracker as an excuse not to contribute, >> > wouldn't this requirement make the situation worse? >> >> A failure to sign the CLA is already a decision not to contribute to >> the distribution > > As someone who cannot in good faith sign the CLA, that characterisation > is far from accurate: I would very much like to contribute to the Python > distribution, and so have not decided as you describe. Could you explain, briefly, why you cannot sign the CLA? -- Steven From stephen at xemacs.org Mon Apr 15 06:08:56 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 15 Apr 2013 13:08:56 +0900 Subject: [Python-Dev] Deciding against the CLA In-Reply-To: <7wwqs667tz.fsf@benfinney.id.au> References: <87k3pmk07r.fsf@uwakimon.sk.tsukuba.ac.jp> <7wwqs667tz.fsf@benfinney.id.au> Message-ID: <8761zo5tbb.fsf@uwakimon.sk.tsukuba.ac.jp> Ben Finney writes: > "Stephen J. Turnbull" writes: > > > Mark Lawrence writes: > > > > > People already use the bug tracker as an excuse not to > > > contribute, wouldn't this requirement make the situation > > > worse? > > > > A failure to sign the CLA is already a decision not to contribute > > to the distribution > > As someone who cannot in good faith sign the CLA, that > characterisation is far from accurate: I would very much like to > contribute to the Python distribution, and so have not decided as > you describe. > > Rather, I leave the matter of contribution undecided, Whatever. In fact, the consequence of your failure to sign the CLA is that your code doesn't get distributed with any of the current Python releases, is that correct? Back in context, I don't see how placing a reminder to sign the CLA on the page makes your decision at that instant harder. I suppose it might deter you from submitting code that by policy shouldn't be included in the distribution, but might be useful to third parties. Whether such deterrence is a good thing or a bad thing would depend on how likely it was to be independently invented by some who is willing to contribute code, and whether you would try to enforce your copyright in the event that it resembled your code (in which case there would be an obvious case for infringement, with the burden of proof on the individual who is willing to sign the CLA). (By the way, what is your problem of conscience with the PSF CLA? Are you afraid that the PSF's obligation to use an "open source license" is not enforceable? You don't like the choice of Initial Licenses? Something else?) From ben+python at benfinney.id.au Mon Apr 15 09:15:22 2013 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 15 Apr 2013 17:15:22 +1000 Subject: [Python-Dev] Deciding against the CLA References: <87k3pmk07r.fsf@uwakimon.sk.tsukuba.ac.jp> <7wwqs667tz.fsf@benfinney.id.au> <516B592F.6040207@pearwood.info> Message-ID: <7wtxn8gt85.fsf@benfinney.id.au> Steven D'Aprano writes: > On 13/04/13 20:30, Ben Finney wrote: > > "Stephen J. Turnbull" writes: > >> A failure to sign the CLA is already a decision not to contribute > >> to the distribution > > > > As someone who cannot in good faith sign the CLA, that > > characterisation is far from accurate: I would very much like to > > contribute to the Python distribution, and so have not decided as > > you describe. > > Could you explain, briefly, why you cannot sign the CLA? Because software freedom in a work is undermined when any recipient is granted special legal privilege in the work. As it currently stands, the Contributor Agreement grants special legal privilege in the work (the power to unilaterally re-license the work) to the PSF. By ?special privilege?, I mean that this power is granted specially to some but denied to all other recipients of the work. Hence to sign the Contributor Agreement as it currently stands is to undermine software freedom in the resulting work. -- \ ?Choose mnemonic identifiers. If you can't remember what | `\ mnemonic means, you've got a problem.? ?Larry Wall | _o__) | Ben Finney From david.k.lam1 at gmail.com Mon Apr 15 09:56:07 2013 From: david.k.lam1 at gmail.com (David Lam) Date: Mon, 15 Apr 2013 00:56:07 -0700 Subject: [Python-Dev] Sharing docstrings between the Python and C implementations of a module Message-ID: Recently I helped out on issue16954 which involved filling in docstrings for methods and classes in ElementTree.py While doing so, I tried to test my work in the interpreter like this... >>> from xml.etree.ElementTree import Element >>> help(Element) ...but found that help() showed nothing but empty strings! After some debugging, I found that the culprit was the `from _elementtree import *` near the bottom of the module. Not wanting to copy & paste docstrings around, I thought one solution might be to just reassign Element.__doc__ with the right docstring. But, it seems that you can't do that for C extensions: >>> from _elementtree import Element as cElement >>> cElement.__doc__ = 'correct docstring' Traceback (most recent call last): File "", line 1, in TypeError: can't set attributes of built-in/extension type 'xml.etree.ElementTree.Element' --- Q. Is there way to maintain the same docstring without resorting to copying and pasting it in two places? I tried to find an example in the source which addressed this, but found that the docstrings in similar cases to be largely duplicated. For instance, _datetimemodule.c, decimal_.c and _json.c all seem to exhibit this docstring copy and pastage. -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Mon Apr 15 09:50:37 2013 From: v+python at g.nevcal.com (Glenn Linderman) Date: Mon, 15 Apr 2013 00:50:37 -0700 Subject: [Python-Dev] Deciding against the CLA In-Reply-To: <7wtxn8gt85.fsf@benfinney.id.au> References: <87k3pmk07r.fsf@uwakimon.sk.tsukuba.ac.jp> <7wwqs667tz.fsf@benfinney.id.au> <516B592F.6040207@pearwood.info> <7wtxn8gt85.fsf@benfinney.id.au> Message-ID: <516BB14D.2080802@g.nevcal.com> On 4/15/2013 12:15 AM, Ben Finney wrote: > Steven D'Aprano writes: > >> On 13/04/13 20:30, Ben Finney wrote: >>> "Stephen J. Turnbull" writes: >>>> A failure to sign the CLA is already a decision not to contribute >>>> to the distribution >>> As someone who cannot in good faith sign the CLA, that >>> characterisation is far from accurate: I would very much like to >>> contribute to the Python distribution, and so have not decided as >>> you describe. >> Could you explain, briefly, why you cannot sign the CLA? > Because software freedom in a work is undermined when any recipient is > granted special legal privilege in the work. > > As it currently stands, the Contributor Agreement grants special legal > privilege in the work (the power to unilaterally re-license the work) to > the PSF. > > By ?special privilege?, I mean that this power is granted specially to > some but denied to all other recipients of the work. Hence to sign the > Contributor Agreement as it currently stands is to undermine software > freedom in the resulting work. > Easily curable by granting that right to all recipients of the contributions you make to PSF, no? Of course, the contributor's agreement has no particular need to include such a clause, but you can include it in a separately published version of the contributions, to keep freedom free... -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Mon Apr 15 12:05:32 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 15 Apr 2013 19:05:32 +0900 Subject: [Python-Dev] Deciding against the CLA In-Reply-To: <7wtxn8gt85.fsf@benfinney.id.au> References: <87k3pmk07r.fsf@uwakimon.sk.tsukuba.ac.jp> <7wwqs667tz.fsf@benfinney.id.au> <516B592F.6040207@pearwood.info> <7wtxn8gt85.fsf@benfinney.id.au> Message-ID: <87obdg3y8j.fsf@uwakimon.sk.tsukuba.ac.jp> Ben Finney writes: > As it currently stands, the Contributor Agreement grants special > legal privilege in the work (the power to unilaterally re-license > the work) to the PSF. "I hate to disagree, Sir, but that turns out to be incorrect."[0] First, it's not the Contributor Agreement, it's the approved Initial Licenses that grant the power to sublicense (the technical term in US law for the kind of "re-licensing" being discussed here). The AFL does so explicitly. I'm not sure about the Apache license, but it does so at least implicitly. (That's the main feature of a "permissive license".) I agree the wording is a little vague, but in fact the Contributor Agreement simply affirms that the Initial License has been granted and that it provides for sublicensing. It then *takes away* power from the PSF, by requiring it to use an open source license, which the Initial Licenses (AFL and Apache) do not. Second, although in theory the PSF might change its license, at the moment the current PSF license also (implicitly) permits some form of "re-licensing" because it requires only preservation of copyright notice (clause 2) and a list of changes (clause 3). In particular it implicitly grants the right to use a proprietary license for works derived from the contribution, which is denied to the PSF under the Contributor Agreement.[1] I think that is very unlikely to change. So the PSF Contributor Agreement grants no special privileges to the PSF not available in practice to any Python user. Footnotes: [0] IANAL, but I'm on pretty firm ground on this one. [1] Technically speaking, that proprietary license may not apply to the portion of code copied from Python. In practice, to the extent that proprietary original code is mixed with PSF Python, it can effectively prevent copying any part of the derived work. Cf. the infamous "non-permission" statement on O'Reilly's _The X Window System_ series. From fijall at gmail.com Mon Apr 15 12:17:39 2013 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 15 Apr 2013 12:17:39 +0200 Subject: [Python-Dev] Sharing docstrings between the Python and C implementations of a module In-Reply-To: References: Message-ID: On Mon, Apr 15, 2013 at 9:56 AM, David Lam wrote: > Recently I helped out on issue16954 which involved filling in docstrings > for methods and classes in ElementTree.py > > While doing so, I tried to test my work in the interpreter like this... > > >>> from xml.etree.ElementTree import Element > >>> help(Element) > > ...but found that help() showed nothing but empty strings! > > After some debugging, I found that the culprit was the > `from _elementtree import *` near the bottom of the module. > > Not wanting to copy & paste docstrings around, I thought one solution > might be to just reassign Element.__doc__ with the right docstring. > But, it seems that you can't do that for C extensions: > > >>> from _elementtree import Element as cElement > >>> cElement.__doc__ = 'correct docstring' > Traceback (most recent call last): > File "", line 1, in > TypeError: can't set attributes of built-in/extension type > 'xml.etree.ElementTree.Element' > > --- > > Q. Is there way to maintain the same docstring without > resorting to copying and pasting it in two places? > > I tried to find an example in the source which addressed this, but > found that the docstrings in similar cases to be largely duplicated. > For instance, _datetimemodule.c, decimal_.c and _json.c all seem to > exhibit this docstring copy and pastage. > Hi NumPy uses a hack to deal with this problem. It has a small C function that would mutate the docstring under your feet. Personally I would prefer some sort of tagging in C source that can copy-paste stuff instead, honestly. It does sound like a good idea to share docstrings. Seems also relatively trivial to write a test that checks that they stay the same. Cheers, fijal From ncoghlan at gmail.com Mon Apr 15 12:45:23 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 15 Apr 2013 20:45:23 +1000 Subject: [Python-Dev] Sharing docstrings between the Python and C implementations of a module In-Reply-To: References: Message-ID: On Mon, Apr 15, 2013 at 8:17 PM, Maciej Fijalkowski wrote: > On Mon, Apr 15, 2013 at 9:56 AM, David Lam wrote: >> I tried to find an example in the source which addressed this, but >> found that the docstrings in similar cases to be largely duplicated. >> For instance, _datetimemodule.c, decimal_.c and _json.c all seem to >> exhibit this docstring copy and pastage. >> > > Hi > > NumPy uses a hack to deal with this problem. It has a small C function > that would mutate the docstring under your feet. Personally I would > prefer some sort of tagging in C source that can copy-paste stuff > instead, honestly. It does sound like a good idea to share docstrings. > Seems also relatively trivial to write a test that checks that they > stay the same. It's actually even worse than that - if a subclass overrides a method, it has to completely duplicate the docstring, even if the original docstring was still valid. So, for example, ABCs can't provide docstrings for abstract methods. So yeah, we end up not only duplicating between the C and Python versions, but sometimes we end up duplicating between different subclasses as well (datetime.datetime, datetime.date and datetime.time are the worst offenders here). I like the idea of at least adding a test that checks the Python docstring and the C docstring are the same in the duplicated cases - that should be a lot easier than adjusting the build process to let the C version use the Python docstrings or vice-versa (even the argument clinic DSL in PEP 434 doesn't try to achieve that - it just tries to cut down on the duplication within the C code itself). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Mon Apr 15 12:48:09 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 15 Apr 2013 20:48:09 +1000 Subject: [Python-Dev] [Python-checkins] cpython: Issue #17693: CJK encoders now use the new Unicode API (PEP 393) In-Reply-To: References: <3ZpCnM5YLlzRl3@mail.python.org> Message-ID: On Mon, Apr 15, 2013 at 3:36 AM, Victor Stinner wrote: > Hi, > > 2013/4/14 Nick Coghlan : >> On Sun, Apr 14, 2013 at 10:06 AM, victor.stinner >> wrote: >>> http://hg.python.org/cpython/rev/d621bdaed7c3 >>> changeset: 83317:d621bdaed7c3 >>> user: Victor Stinner >>> date: Sun Apr 14 02:06:32 2013 +0200 >>> summary: >>> Issue #17693: CJK encoders now use the new Unicode API (PEP 393) >> >> I'm going to run with the wild theory that there's a reference link in >> here somewhere... :) >> >> results for 0ee785c9d1b4 on branch "default" >> ------------------------------ >> -------------- >> >> test_codecencodings_cn leaked [16133, 16133, 16133] references, sum=48399 >> test_codecencodings_cn leaked [28992, 28994, 28994] memory blocks, sum=86980 >> ... > > Oh thanks for the report. My laptop is too slow to run the test suite using -R. It's thanks to Antoine, really - I don't make a habit of running with -R either, I just watch for the leak reporting emails that look like real leaks rather than random noise :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From guido at python.org Mon Apr 15 16:14:50 2013 From: guido at python.org (Guido van Rossum) Date: Mon, 15 Apr 2013 07:14:50 -0700 Subject: [Python-Dev] Deciding against the CLA In-Reply-To: <7wtxn8gt85.fsf@benfinney.id.au> References: <87k3pmk07r.fsf@uwakimon.sk.tsukuba.ac.jp> <7wwqs667tz.fsf@benfinney.id.au> <516B592F.6040207@pearwood.info> <7wtxn8gt85.fsf@benfinney.id.au> Message-ID: Can we get this discussion off python-dev? It's not going to change, and this is not the forum to express your disagreement. On Mon, Apr 15, 2013 at 12:15 AM, Ben Finney wrote: > Steven D'Aprano writes: > >> On 13/04/13 20:30, Ben Finney wrote: >> > "Stephen J. Turnbull" writes: >> >> A failure to sign the CLA is already a decision not to contribute >> >> to the distribution >> > >> > As someone who cannot in good faith sign the CLA, that >> > characterisation is far from accurate: I would very much like to >> > contribute to the Python distribution, and so have not decided as >> > you describe. >> >> Could you explain, briefly, why you cannot sign the CLA? > > Because software freedom in a work is undermined when any recipient is > granted special legal privilege in the work. > > As it currently stands, the Contributor Agreement grants special legal > privilege in the work (the power to unilaterally re-license the work) to > the PSF. > > By ?special privilege?, I mean that this power is granted specially to > some but denied to all other recipients of the work. Hence to sign the > Contributor Agreement as it currently stands is to undermine software > freedom in the resulting work. > > -- > \ ?Choose mnemonic identifiers. If you can't remember what | > `\ mnemonic means, you've got a problem.? ?Larry Wall | > _o__) | > Ben Finney > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) From eliben at gmail.com Mon Apr 15 18:31:40 2013 From: eliben at gmail.com (Eli Bendersky) Date: Mon, 15 Apr 2013 09:31:40 -0700 Subject: [Python-Dev] Sharing docstrings between the Python and C implementations of a module In-Reply-To: References: Message-ID: On Mon, Apr 15, 2013 at 3:45 AM, Nick Coghlan wrote: > On Mon, Apr 15, 2013 at 8:17 PM, Maciej Fijalkowski > wrote: > > On Mon, Apr 15, 2013 at 9:56 AM, David Lam > wrote: > >> I tried to find an example in the source which addressed this, but > >> found that the docstrings in similar cases to be largely duplicated. > >> For instance, _datetimemodule.c, decimal_.c and _json.c all seem to > >> exhibit this docstring copy and pastage. > >> > > > > Hi > > > > NumPy uses a hack to deal with this problem. It has a small C function > > that would mutate the docstring under your feet. Personally I would > > prefer some sort of tagging in C source that can copy-paste stuff > > instead, honestly. It does sound like a good idea to share docstrings. > > Seems also relatively trivial to write a test that checks that they > > stay the same. > > It's actually even worse than that - if a subclass overrides a method, > it has to completely duplicate the docstring, even if the original > docstring was still valid. So, for example, ABCs can't provide > docstrings for abstract methods. > > So yeah, we end up not only duplicating between the C and Python > versions, but sometimes we end up duplicating between different > subclasses as well (datetime.datetime, datetime.date and datetime.time > are the worst offenders here). > > I like the idea of at least adding a test that checks the Python > docstring and the C docstring are the same in the duplicated cases - > that should be a lot easier than adjusting the build process to let > the C version use the Python docstrings or vice-versa (even the > argument clinic DSL in PEP 434 doesn't try to achieve that - it just > tries to cut down on the duplication within the C code itself). > Would it make sense to think about adding this in the scope of the argument clinic work, or is it too unrelated? This seems like a commonly needed thing for large parts of the stdlib (where the C accelerator overrides Python code). Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsbueno at python.org.br Mon Apr 15 18:45:34 2013 From: jsbueno at python.org.br (Joao S. O. Bueno) Date: Mon, 15 Apr 2013 13:45:34 -0300 Subject: [Python-Dev] Sharing docstrings between the Python and C implementations of a module In-Reply-To: References: Message-ID: On 15 April 2013 13:31, Eli Bendersky wrote: > > > > On Mon, Apr 15, 2013 at 3:45 AM, Nick Coghlan wrote: >> >> On Mon, Apr 15, 2013 at 8:17 PM, Maciej Fijalkowski >> wrote: >> > On Mon, Apr 15, 2013 at 9:56 AM, David Lam >> > wrote: >> >> I tried to find an example in the source which addressed this, but >> >> found that the docstrings in similar cases to be largely duplicated. >> >> For instance, _datetimemodule.c, decimal_.c and _json.c all seem to >> >> exhibit this docstring copy and pastage. >> >> >> > >> > Hi >> > >> > NumPy uses a hack to deal with this problem. It has a small C function >> > that would mutate the docstring under your feet. Personally I would >> > prefer some sort of tagging in C source that can copy-paste stuff >> > instead, honestly. It does sound like a good idea to share docstrings. >> > Seems also relatively trivial to write a test that checks that they >> > stay the same. >> >> It's actually even worse than that - if a subclass overrides a method, >> it has to completely duplicate the docstring, even if the original >> docstring was still valid. So, for example, ABCs can't provide >> docstrings for abstract methods. >> >> So yeah, we end up not only duplicating between the C and Python >> versions, but sometimes we end up duplicating between different >> subclasses as well (datetime.datetime, datetime.date and datetime.time >> are the worst offenders here). >> >> I like the idea of at least adding a test that checks the Python >> docstring and the C docstring are the same in the duplicated cases - >> that should be a lot easier than adjusting the build process to let >> the C version use the Python docstrings or vice-versa (even the >> argument clinic DSL in PEP 434 doesn't try to achieve that - it just >> tries to cut down on the duplication within the C code itself). > > > Would it make sense to think about adding this in the scope of the argument > clinic work, or is it too unrelated? This seems like a commonly needed thing > for large parts of the stdlib (where the C accelerator overrides Python > code). +1 It is a problem I was met with when building extensions as well - when one would like to keep the C parts to a minimum and dynamically populate de doc-strings from another source with a Python script, for example. > > Eli > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/jsbueno%40python.org.br > From skip at pobox.com Mon Apr 15 19:07:42 2013 From: skip at pobox.com (Skip Montanaro) Date: Mon, 15 Apr 2013 12:07:42 -0500 Subject: [Python-Dev] Sharing docstrings between the Python and C implementations of a module In-Reply-To: References: Message-ID: > Would it make sense to think about adding this in the scope of the argument > clinic work, or is it too unrelated? This seems like a commonly needed thing > for large parts of the stdlib (where the C accelerator overrides Python > code). Or maybe separate doc strings from both code bases altogether and insert them where appropriate as part of the build process? That said, I haven't any idea how you might accomplish that. Maybe this general problem should be thrown over the python-ideas to cook for awhile... Skip From alexandre at peadrop.com Mon Apr 15 20:37:05 2013 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Mon, 15 Apr 2013 11:37:05 -0700 Subject: [Python-Dev] Sharing docstrings between the Python and C implementations of a module In-Reply-To: References: Message-ID: On Mon, Apr 15, 2013 at 12:56 AM, David Lam wrote: > I tried to find an example in the source which addressed this, but > found that the docstrings in similar cases to be largely duplicated. > I find this annoying too. It would be nice to have a common way to share docstrings between C and Python implementations of the same interface. One roadblock though is functions in C modules often document their parameters in their docstring. >>> import _json >>> help(_json.scanstring) scanstring(...) scanstring(basestring, end, encoding, strict=True) -> (str, end) Scan the string s for a JSON string. End is the index of the character in s after the quote that started the JSON string. [...] Argument clinic will hopefully lift this roadblock soon. Perhaps, we could add something to the clinic DSL a way to fetch the docstring directly from the Python implementation. And as an extra, it would be easy to add verification step as well that checks the both implementations provide a similar interfaces once we have this in place. -------------- next part -------------- An HTML attachment was scrubbed... URL: From larry at hastings.org Tue Apr 16 01:21:40 2013 From: larry at hastings.org (Larry Hastings) Date: Mon, 15 Apr 2013 16:21:40 -0700 Subject: [Python-Dev] Sharing docstrings between the Python and C implementations of a module In-Reply-To: References: Message-ID: <516C8B84.9000207@hastings.org> On 04/15/2013 09:31 AM, Eli Bendersky wrote: > Would it make sense to think about adding this in the scope of the > argument clinic work, or is it too unrelated? This seems like a > commonly needed thing for large parts of the stdlib (where the C > accelerator overrides Python code). From my perspective, the C accelerator doesn't override the Python code; the Python code is in charge, and elects to call the C accelerator. To answer your question, Clinic could help but I think it'd kind of suck. We'd have to use Clinic to preprocess the .py file and copy the docstring there, which would mean slurping in the C file. Clumsy but workable. We can talk about it once I finish the revised implementation, which should make processes like this much easier. /arry From v+python at g.nevcal.com Tue Apr 16 01:51:13 2013 From: v+python at g.nevcal.com (Glenn Linderman) Date: Mon, 15 Apr 2013 16:51:13 -0700 Subject: [Python-Dev] Sharing docstrings between the Python and C implementations of a module In-Reply-To: <516C8B84.9000207@hastings.org> References: <516C8B84.9000207@hastings.org> Message-ID: <516C9271.8000700@g.nevcal.com> On 4/15/2013 4:21 PM, Larry Hastings wrote: > On 04/15/2013 09:31 AM, Eli Bendersky wrote: >> Would it make sense to think about adding this in the scope of the >> argument clinic work, or is it too unrelated? This seems like a >> commonly needed thing for large parts of the stdlib (where the C >> accelerator overrides Python code). > > From my perspective, the C accelerator doesn't override the Python > code; the Python code is in charge, and elects to call the C accelerator. > > To answer your question, Clinic could help but I think it'd kind of > suck. We'd have to use Clinic to preprocess the .py file and copy the > docstring there, which would mean slurping in the C file. Clumsy but > workable. We can talk about it once I finish the revised > implementation, which should make processes like this much easier. Since Clinic is mostly aimed at making C usage work with python, isn't the above backwards? Wouldn't it be better to have Clinic slurp in the Python code to find the docstrings, and then put them in the C code? Then no need to preprocess the python file, just read it to obtain the docstrings. You are already preprocessing the C code... -------------- next part -------------- An HTML attachment was scrubbed... URL: From benhoyt at gmail.com Tue Apr 16 04:04:24 2013 From: benhoyt at gmail.com (Ben Hoyt) Date: Tue, 16 Apr 2013 14:04:24 +1200 Subject: [Python-Dev] mimetypes broken on Windows Message-ID: Hi folks, The built-in mimetypes module is broken on Windows, and it has been since Python 2.7 alpha 1. On all Windows systems I've tried, guess_type() returns the wrong mime type for common types like .png and .jpg. For example (on Python 2.7.4 and 3.3.1): >>> import mimetypes >>> mimetypes.guess_type('f.png') ('image/x-png', None) >>> mimetypes.guess_type('f.jpg') ('image/pjpeg', None) These should be 'image/png' and 'image/jpeg', respectively. There's an open issue for this: http://bugs.python.org/issue15207. However, it hasn't gotten any love in the last few months, so per r.david.murray's comment, I'm posting it here. Dave Chambers, who opened the bug, has proposed a fix, which is significantly better (i.e., not totally broken for common types). However, as I mentioned in http://bugs.python.org/issue15207#msg177030, using the Windows registry for this at all is basically a bad idea, because: 1) Important keys like .jpg and .png aren't in the registry anyway. 2) Some that do exist are wrong in the Windows registry. This includes .zip, which is "application/x-zip-compressed" (at least in my registry) but should be "application/zip". 3) It makes the first call to guess_type() slow (~100ms), which isn't terrible, but with the above concerns, not worth it. 4) Perhaps most importantly: the keys in the Windows registry depend on what programs you have installed. And the users and programs can change registry keys at will. Obviously one can work around this bug, either by calling mimetypes.init(files=[]) before any calls to guess_type, or calling init() with your own mime types file. However, "broken out of the box" is going to cause a lot of people headaches. :-) So my proposal is simply to get rid of read_windows_registry() altogether, and fall back to the default type mapping in mimetypes.py on Windows systems. This is correct and fast, even if not complete. As always, folks can always use their own mimetypes file if they want. In summary: the current behaviour is buggy and broken, the behaviour proposed in Issue 15207 is problematic, getting this from the Windows registry is bad idea, and we should revert the whole registry thing. :-) If folks agree with my reasoning above, I can provide a patch to fix this, along with a patch to the Windows unit tests. -Ben P.S. Kind of proving my point about the fragility of using the registry, the Python 2.7.4 unit test test_registry_parsing in test_mimetypes.py fail on my machine. It's because I've installed some SQL server, and text/plain is my registry is mapped from .sql (instead of .txt), causing this: Traceback (most recent call last): File "C:\python27\lib\test\test_mimetypes.py", line 85, in test_registry_parsing eq(self.db.guess_type("foo.txt"), ("text/plain", None)) AssertionError: Tuples differ: (None, None) != ('text/plain', None) -------------- next part -------------- An HTML attachment was scrubbed... URL: From Nikolaus at rath.org Tue Apr 16 07:02:14 2013 From: Nikolaus at rath.org (Nikolaus Rath) Date: Mon, 15 Apr 2013 22:02:14 -0700 Subject: [Python-Dev] Destructors and Closing of File Objects References: <87a9p41gr6.fsf@vostro.rath.org> Message-ID: <877gk35aqx.fsf@vostro.rath.org> Brian Curtin writes: > On Fri, Apr 12, 2013 at 12:04 AM, Nikolaus Rath wrote: >> [ Note: I already asked this on >> http://stackoverflow.com/questions/15917502 but didn't get any >> satisfactory answers] > > Sorry, but that's not a reason to repost your question to this list. > If you have to ask somewhere else, it would be python-list, aka, > comp.lang.python. I figured it belonged here because the question is really about the internal implementation of file objects, which to me didn't seem like a question about using Python. But I'll give it a few days and send another mail there if still haven't found the answer by then. Best, -Nikolaus -- ?Time flies like an arrow, fruit flies like a Banana.? PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C From guido at python.org Tue Apr 16 07:13:23 2013 From: guido at python.org (Guido van Rossum) Date: Mon, 15 Apr 2013 22:13:23 -0700 Subject: [Python-Dev] Destructors and Closing of File Objects In-Reply-To: <877gk35aqx.fsf@vostro.rath.org> References: <87a9p41gr6.fsf@vostro.rath.org> <877gk35aqx.fsf@vostro.rath.org> Message-ID: You got your snswer 16 hours ago on S.O. On Monday, April 15, 2013, Nikolaus Rath wrote: > Brian Curtin > writes: > > On Fri, Apr 12, 2013 at 12:04 AM, Nikolaus Rath > > wrote: > >> [ Note: I already asked this on > >> http://stackoverflow.com/questions/15917502 but didn't get any > >> satisfactory answers] > > > > Sorry, but that's not a reason to repost your question to this list. > > If you have to ask somewhere else, it would be python-list, aka, > > comp.lang.python. > > I figured it belonged here because the question is really about the > internal implementation of file objects, which to me didn't seem like a > question about using Python. But I'll give it a few days and send > another mail there if still haven't found the answer by then. > > Best, > > -Nikolaus > > -- > ?Time flies like an arrow, fruit flies like a Banana.? > > PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Tue Apr 16 09:48:02 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 16 Apr 2013 16:48:02 +0900 Subject: [Python-Dev] Sharing docstrings between the Python and C implementations of a module In-Reply-To: References: Message-ID: <87haj6532l.fsf@uwakimon.sk.tsukuba.ac.jp> Skip Montanaro writes: > > Would it make sense to think about adding this in the scope of > > the argument clinic work, or is it too unrelated? This seems like > > a commonly needed thing for large parts of the stdlib (where the > > C accelerator overrides Python code). > > Or maybe separate doc strings from both code bases altogether and > insert them where appropriate as part of the build process? Experience with gettext vs other kinds of message catalogs for localization suggests that's a really painful approach to take. It's not entirely clear to me that this whole effort isn't a premature optimization. Eventually (next 5 to 15 years? long run, anyway) we'll probably localize, and *most* messages will be shared (from gettext .mo files) anyway. (Yes, I recognize that space is not the most important aspect of sharing docstrings, and that it's likely shared docstrings can be automatically shared by gettext. We should take care that that's so.) The other thing that occurs to me is that maybe something like gettext may be the way to deal with these issues anyway. From tjreedy at udel.edu Tue Apr 16 20:00:53 2013 From: tjreedy at udel.edu (Terry Jan Reedy) Date: Tue, 16 Apr 2013 14:00:53 -0400 Subject: [Python-Dev] mimetypes broken on Windows In-Reply-To: References: Message-ID: On 4/15/2013 10:04 PM, Ben Hoyt wrote: > Hi folks, > > The built-in mimetypes module is broken on Windows, and it has been > since Python 2.7 alpha 1. On all Windows systems I've tried, > guess_type() returns the wrong mime type for common types like .png and > .jpg. For example (on Python 2.7.4 and 3.3.1): > > >>> import mimetypes > >>> mimetypes.guess_type('f.png') > ('image/x-png', None) > >>> mimetypes.guess_type('f.jpg') > ('image/pjpeg', None) > > These should be 'image/png' and 'image/jpeg', respectively. > > There's an open issue for this: http://bugs.python.org/issue15207. > However, it hasn't gotten any love in the last few months, so per > r.david.murray's comment, I'm posting it here. > > Dave Chambers, who opened the bug, has proposed a fix, which is > significantly better (i.e., not totally broken for common types). > However, as I mentioned in http://bugs.python.org/issue15207#msg177030, > using the Windows registry for this at all is basically a bad idea, because: The actual mapping is fixed and more or less system independent while the windows registry is for volatile system and user dependent mappings. > 1) Important keys like .jpg and .png aren't in the registry anyway. > 2) Some that do exist are wrong in the Windows registry. This includes > .zip, which is "application/x-zip-compressed" (at least in my registry) > but should be "application/zip". > 3) It makes the first call to guess_type() slow (~100ms), which isn't > terrible, but with the above concerns, not worth it. > 4) Perhaps most importantly: the keys in the Windows registry depend on > what programs you have installed. And the users and programs can change > registry keys at will. And change what a given key is mapped to. > Obviously one can work around this bug, either by calling > mimetypes.init(files=[]) before any calls to guess_type, or calling > init() with your own mime types file. However, "broken out of the box" > is going to cause a lot of people headaches. :-) > > So my proposal is simply to get rid of read_windows_registry() > altogether, and fall back to the default type mapping in mimetypes.py on > Windows systems. This is correct and fast, even if not complete. As I basicallly agree, but am not sure what to do about back-compatibility considerations. But we do not have to reproduce buggy behavior. > always, folks can always use their own mimetypes file if they want. > > In summary: the current behaviour is buggy and broken, the behaviour > proposed in Issue 15207 is problematic, getting this from the Windows > registry is bad idea, and we should revert the whole registry thing. :-) > > If folks agree with my reasoning above, I can provide a patch to fix > this, along with a patch to the Windows unit tests. > > -Ben > > P.S. Kind of proving my point about the fragility of using the registry, > the Python 2.7.4 unit test test_registry_parsing in test_mimetypes.py > fail on my machine. It's because I've installed some SQL server, and > text/plain is my registry is mapped from .sql (instead of .txt), causing > this: > > Traceback (most recent call last): > File "C:\python27\lib\test\test_mimetypes.py", line 85, in > test_registry_parsing > eq(self.db.guess_type("foo.txt"), ("text/plain", None)) > AssertionError: Tuples differ: (None, None) != ('text/plain', None) From rdmurray at bitdance.com Tue Apr 16 22:10:37 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 16 Apr 2013 16:10:37 -0400 Subject: [Python-Dev] mimetypes broken on Windows In-Reply-To: References: Message-ID: <20130416201038.21A13250BD9@webabinitio.net> On Tue, 16 Apr 2013 14:00:53 -0400, Terry Jan Reedy wrote: > On 4/15/2013 10:04 PM, Ben Hoyt wrote: > > So my proposal is simply to get rid of read_windows_registry() > > altogether, and fall back to the default type mapping in mimetypes.py on > > Windows systems. This is correct and fast, even if not complete. As > > I basicallly agree, but am not sure what to do about back-compatibility > considerations. But we do not have to reproduce buggy behavior. I basically agree as well, but as a non-windows user I'm not willing to commit any change without approval from a committer who actually understands what's going on. My understanding is that referencing the windows registry is a relatively new feature (I'm not sure exactly how new), and that it is itself causing more backward compatibility problems than would likely be caused by removing it. But as I said, I'm not enough of a Windows expert to be comfortable making that decision. I'm glad this was brought up on python-dev; it's been nagging at me that this issue hasn't been getting resolved. --David From benhoyt at gmail.com Wed Apr 17 00:22:00 2013 From: benhoyt at gmail.com (Ben Hoyt) Date: Wed, 17 Apr 2013 10:22:00 +1200 Subject: [Python-Dev] mimetypes broken on Windows In-Reply-To: References: Message-ID: (Sorry if this reply doesn't thread as I intend -- I wasn't configured to get python-dev emails, so I'm replying to my original with copy-n-paste.) On Tue, 16 Apr 2013 14:00:53 -0400, Terry Jan Reedy wrote: > On 4/15/2013 10:04 PM, Ben Hoyt wrote: > > So my proposal is simply to get rid of read_windows_registry() > > altogether, and fall back to the default type mapping in mimetypes.py on > > Windows systems. This is correct and fast, even if not complete. As > > I basicallly agree, but am not sure what to do about back-compatibility > considerations. But we do not have to reproduce buggy behavior. Agreed. What we have is just plain wrong. Dave Chambers' fix is better, but still problematic. What we *could* do is implement Dave Chambers' fix in read_windows_registry(), but not call this by default. So a user would have to explicitly call it if they really want Windows registry. But I actually don't think even that's necessary. I honestly can't see how anyone will be "depending" on the current behaviour, as it's just plain buggy (.png and .jpg give the wrong mime type). So I don't think backwards-compatibility is an issue here. As R. David Murray mentioned, reading the registry is quite new (Python 2.7 alpha 1, I believe), and has caused several problems already. There's been encoding issues, and there's even a duplicate of issue 15207, "part 3" of http://bugs.python.org/issue10551 But yes, I would love to see a Windows Python committer chip in, even if it's just with "agreed, please provide a patch". -Ben From mail at timgolden.me.uk Wed Apr 17 09:28:23 2013 From: mail at timgolden.me.uk (Tim Golden) Date: Wed, 17 Apr 2013 08:28:23 +0100 Subject: [Python-Dev] mimetypes broken on Windows In-Reply-To: References: Message-ID: <516E4F17.9020705@timgolden.me.uk> On 16/04/2013 23:22, Ben Hoyt wrote: > But yes, I would love to see a Windows Python committer chip in, even > if it's just with "agreed, please provide a patch". I can chip in with an apology, at least. This has been on my to-do list for ages; but I have had absolutely minimal time to work on Python this last year. I'll set aside an hour later today to look over the different options and patches on offer and at least come back with an opinion on what should happen next, even if I have to ask someone else to apply the patch. Obviously should some other developer want to dive in, please do. Thanks for bringing it back to the table, Ben. TJG From mail at timgolden.me.uk Wed Apr 17 14:25:36 2013 From: mail at timgolden.me.uk (Tim Golden) Date: Wed, 17 Apr 2013 13:25:36 +0100 Subject: [Python-Dev] mimetypes broken on Windows In-Reply-To: <516E4F17.9020705@timgolden.me.uk> References: <516E4F17.9020705@timgolden.me.uk> Message-ID: <516E94C0.9070300@timgolden.me.uk> On 17/04/2013 08:28, Tim Golden wrote: > On 16/04/2013 23:22, Ben Hoyt wrote: >> But yes, I would love to see a Windows Python committer chip in, even >> if it's just with "agreed, please provide a patch". > > I can chip in with an apology, at least. This has been on my to-do list > for ages; but I have had absolutely minimal time to work on Python this > last year. I'll set aside an hour later today to look over the different > options and patches on offer and at least come back with an opinion on > what should happen next, even if I have to ask someone else to apply the > patch. I've responded over there for now --> http://bugs.python.org/issue15207#msg187158 TJG From fijall at gmail.com Thu Apr 18 13:41:45 2013 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 18 Apr 2013 13:41:45 +0200 Subject: [Python-Dev] libffi inclusion in python Message-ID: Hi libffi has bugs sometimes (like this http://bugs.python.org/issue17580). Now this is a thing that upstream fixes really quickly, but tracking down issues on bugs.python.org is annoying (they never get commited as quickly as the upstream). is there a good reason why cpython has it's own copy of libffi? I understand historical reasons, but PyPy gets along relying on the system library, so maybe we can kill the inclusion. Cheers, fijal From stefan_ml at behnel.de Thu Apr 18 13:56:09 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 18 Apr 2013 13:56:09 +0200 Subject: [Python-Dev] libffi inclusion in python In-Reply-To: References: Message-ID: Maciej Fijalkowski, 18.04.2013 13:41: > PyPy gets along relying on the system library Depends on what systems you want to support, I guess. Stefan From benjamin at python.org Thu Apr 18 14:47:17 2013 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 18 Apr 2013 08:47:17 -0400 Subject: [Python-Dev] libffi inclusion in python In-Reply-To: References: Message-ID: 2013/4/18 Maciej Fijalkowski : > Hi > > libffi has bugs sometimes (like this > http://bugs.python.org/issue17580). Now this is a thing that upstream > fixes really quickly, but tracking down issues on bugs.python.org is > annoying (they never get commited as quickly as the upstream). is > there a good reason why cpython has it's own copy of libffi? I > understand historical reasons, but PyPy gets along relying on the > system library, so maybe we can kill the inclusion. IIRC, it had (has?) some custom windows patches, which no one knows whether they're relevant or not. I personally would love to see all libraries that we have copies of in the repo killed. -- Regards, Benjamin From brett at python.org Thu Apr 18 17:02:37 2013 From: brett at python.org (Brett Cannon) Date: Thu, 18 Apr 2013 11:02:37 -0400 Subject: [Python-Dev] A decade as a core dev Message-ID: Today marks my 10 year anniversary as a core developer on Python. I wrote a blog post to mark the occasion (http://sayspy.blogspot.ca/2013/04/a-decade-of-commits.html), but I wanted to personally thank python-dev for the past decade (and whatever comes in the future). All of you taught me how to really program and for that I will be eternally grateful. And the friendships I have built through this list are priceless. From nad at acm.org Thu Apr 18 18:09:09 2013 From: nad at acm.org (Ned Deily) Date: Thu, 18 Apr 2013 09:09:09 -0700 Subject: [Python-Dev] libffi inclusion in python References: Message-ID: In article , Benjamin Peterson wrote: > 2013/4/18 Maciej Fijalkowski : > > libffi has bugs sometimes (like this > > http://bugs.python.org/issue17580). Now this is a thing that upstream > > fixes really quickly, but tracking down issues on bugs.python.org is > > annoying (they never get commited as quickly as the upstream). is > > there a good reason why cpython has it's own copy of libffi? I > > understand historical reasons, but PyPy gets along relying on the > > system library, so maybe we can kill the inclusion. > > IIRC, it had (has?) some custom windows patches, which no one knows > whether they're relevant or not. The cpython copy also has custom OS X patches. I've never looked at them so I don't have a feel for how much work would be involved in migrating to current upstream. If it's just a matter of supporting universal builds, it could be done with some Makefile hacking to do a lipo dance. Ronald, any additional thoughts? http://bugs.python.org/issue15194 Currently, for the OS X installer builds, we build a number of third-party libs that are either missing from OS X (like lzma) or are too out-of-date on the oldest systems we support. It would be useful to generalize the third-party lib support and move it out of the installer build process so that all builds could take advantage of the libs if needed. libffi could be added to those. Of course, that wouldn't help for Windows builds. -- Ned Deily, nad at acm.org From guido at python.org Thu Apr 18 18:17:02 2013 From: guido at python.org (Guido van Rossum) Date: Thu, 18 Apr 2013 09:17:02 -0700 Subject: [Python-Dev] A decade as a core dev In-Reply-To: References: Message-ID: Congrats Brett! I'd say you have learned a lot more than programming during your time here!! On Thu, Apr 18, 2013 at 8:02 AM, Brett Cannon wrote: > Today marks my 10 year anniversary as a core developer on Python. I > wrote a blog post to mark the occasion > (http://sayspy.blogspot.ca/2013/04/a-decade-of-commits.html), but I > wanted to personally thank python-dev for the past decade (and > whatever comes in the future). All of you taught me how to really > program and for that I will be eternally grateful. And the friendships > I have built through this list are priceless. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) From brett at python.org Thu Apr 18 19:01:33 2013 From: brett at python.org (Brett Cannon) Date: Thu, 18 Apr 2013 13:01:33 -0400 Subject: [Python-Dev] A decade as a core dev In-Reply-To: References: Message-ID: On Thu, Apr 18, 2013 at 12:17 PM, Guido van Rossum wrote: > Congrats Brett! I'd say you have learned a lot more than programming > during your time here!! Oh yes, such as how to put on a flame-retardant suit, the colours of the rainbow based on the bikesheds, and how I will never be able to pronounce your name properly. =) Seriously though, yes, I've learned tons about all sorts of things during my time here. -Brett > > On Thu, Apr 18, 2013 at 8:02 AM, Brett Cannon wrote: >> Today marks my 10 year anniversary as a core developer on Python. I >> wrote a blog post to mark the occasion >> (http://sayspy.blogspot.ca/2013/04/a-decade-of-commits.html), but I >> wanted to personally thank python-dev for the past decade (and >> whatever comes in the future). All of you taught me how to really >> program and for that I will be eternally grateful. And the friendships >> I have built through this list are priceless. >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > > > > -- > --Guido van Rossum (python.org/~guido) From andrew.svetlov at gmail.com Thu Apr 18 19:27:19 2013 From: andrew.svetlov at gmail.com (Andrew Svetlov) Date: Thu, 18 Apr 2013 20:27:19 +0300 Subject: [Python-Dev] A decade as a core dev In-Reply-To: References: Message-ID: My congratulations! On Thu, Apr 18, 2013 at 8:01 PM, Brett Cannon wrote: > On Thu, Apr 18, 2013 at 12:17 PM, Guido van Rossum wrote: >> Congrats Brett! I'd say you have learned a lot more than programming >> during your time here!! > > Oh yes, such as how to put on a flame-retardant suit, the colours of > the rainbow based on the bikesheds, and how I will never be able to > pronounce your name properly. =) > > Seriously though, yes, I've learned tons about all sorts of things > during my time here. > > -Brett > >> >> On Thu, Apr 18, 2013 at 8:02 AM, Brett Cannon wrote: >>> Today marks my 10 year anniversary as a core developer on Python. I >>> wrote a blog post to mark the occasion >>> (http://sayspy.blogspot.ca/2013/04/a-decade-of-commits.html), but I >>> wanted to personally thank python-dev for the past decade (and >>> whatever comes in the future). All of you taught me how to really >>> program and for that I will be eternally grateful. And the friendships >>> I have built through this list are priceless. >>> _______________________________________________ >>> Python-Dev mailing list >>> Python-Dev at python.org >>> http://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org >> >> >> >> -- >> --Guido van Rossum (python.org/~guido) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov From solipsis at pitrou.net Thu Apr 18 19:34:46 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 18 Apr 2013 19:34:46 +0200 Subject: [Python-Dev] A decade as a core dev References: Message-ID: <20130418193446.1ed036e9@fsol> On Thu, 18 Apr 2013 11:02:37 -0400 Brett Cannon wrote: > Today marks my 10 year anniversary as a core developer on Python. I > wrote a blog post to mark the occasion > (http://sayspy.blogspot.ca/2013/04/a-decade-of-commits.html), but I > wanted to personally thank python-dev for the past decade (and > whatever comes in the future). All of you taught me how to really > program and for that I will be eternally grateful. And the friendships > I have built through this list are priceless. Normally you should break a buildbot as a celebration :) cheers Antoine. From benjamin at python.org Thu Apr 18 19:38:33 2013 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 18 Apr 2013 13:38:33 -0400 Subject: [Python-Dev] A decade as a core dev In-Reply-To: <20130418193446.1ed036e9@fsol> References: <20130418193446.1ed036e9@fsol> Message-ID: 2013/4/18 Antoine Pitrou : > On Thu, 18 Apr 2013 11:02:37 -0400 > Brett Cannon wrote: >> Today marks my 10 year anniversary as a core developer on Python. I >> wrote a blog post to mark the occasion >> (http://sayspy.blogspot.ca/2013/04/a-decade-of-commits.html), but I >> wanted to personally thank python-dev for the past decade (and >> whatever comes in the future). All of you taught me how to really >> program and for that I will be eternally grateful. And the friendships >> I have built through this list are priceless. > > Normally you should break a buildbot as a celebration :) He might lose all his friends then. -- Regards, Benjamin From rovitotv at gmail.com Thu Apr 18 20:21:12 2013 From: rovitotv at gmail.com (Todd V Rovito) Date: Thu, 18 Apr 2013 14:21:12 -0400 Subject: [Python-Dev] A decade as a core dev In-Reply-To: References: Message-ID: <73D4E9A8-1E14-4715-9A01-7C0B1339E973@gmail.com> On Apr 18, 2013, at 11:02 AM, Brett Cannon wrote: > Today marks my 10 year anniversary as a core developer on Python. I > wrote a blog post to mark the occasion > (http://sayspy.blogspot.ca/2013/04/a-decade-of-commits.html), but I > wanted to personally thank python-dev for the past decade (and > whatever comes in the future). Wow 10 years that is incredible!! Your blog post was awesome. You are an inspiration to me as I just started contributing this past October. I can honestly see how you have spent 10 years of "spare hours" night time engineering. I find contributing a bit addictive myself. Here is what happens to me at night, on my right shoulder a little Guido appears out of thin air and says "go to bed" but then on my left shoulder another Guido appears and says "all you have to do is test this last patch". Next thing you know it is 2 am and I have to get up at 5 am for work so I convince myself to stay up all night. Indeed contributing to Python is very rewarding. From brett at python.org Thu Apr 18 20:27:41 2013 From: brett at python.org (Brett Cannon) Date: Thu, 18 Apr 2013 14:27:41 -0400 Subject: [Python-Dev] A decade as a core dev In-Reply-To: <73D4E9A8-1E14-4715-9A01-7C0B1339E973@gmail.com> References: <73D4E9A8-1E14-4715-9A01-7C0B1339E973@gmail.com> Message-ID: On Thu, Apr 18, 2013 at 2:21 PM, Todd V Rovito wrote: > On Apr 18, 2013, at 11:02 AM, Brett Cannon wrote: > >> Today marks my 10 year anniversary as a core developer on Python. I >> wrote a blog post to mark the occasion >> (http://sayspy.blogspot.ca/2013/04/a-decade-of-commits.html), but I >> wanted to personally thank python-dev for the past decade (and >> whatever comes in the future). > Wow 10 years that is incredible!! Your blog post was awesome. You are an inspiration to me as I just started contributing this past October. Thanks for the kind words. > I can honestly see how you have spent 10 years of "spare hours" night time engineering. I find contributing a bit addictive myself. Here is what happens to me at night, on my right shoulder a little Guido appears out of thin air and says "go to bed" but then on my left shoulder another Guido appears and says "all you have to do is test this last patch". Next thing you know it is 2 am and I have to get up at 5 am for work so I convince myself to stay up all night. Indeed contributing to Python is very rewarding. I suspect the real Guido would say "go to bed" since there is always tomorrow. =) From fijall at gmail.com Thu Apr 18 20:41:51 2013 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 18 Apr 2013 20:41:51 +0200 Subject: [Python-Dev] libffi inclusion in python In-Reply-To: References: Message-ID: On Thu, Apr 18, 2013 at 2:47 PM, Benjamin Peterson wrote: > 2013/4/18 Maciej Fijalkowski : >> Hi >> >> libffi has bugs sometimes (like this >> http://bugs.python.org/issue17580). Now this is a thing that upstream >> fixes really quickly, but tracking down issues on bugs.python.org is >> annoying (they never get commited as quickly as the upstream). is >> there a good reason why cpython has it's own copy of libffi? I >> understand historical reasons, but PyPy gets along relying on the >> system library, so maybe we can kill the inclusion. > > IIRC, it had (has?) some custom windows patches, which no one knows > whether they're relevant or not. > > I personally would love to see all libraries that we have copies of in > the repo killed. Upstream seems to be incredibly responsive. Why not merge them there? Is it just "we can't patch distutils" crap? If we *really* need to provide libraries for systems that don't have them, how about simply having a separate download for those systems instead of using it on systems that *do* have them and in fact *do* have them in a better version. Note that the recent patches (one for 32bit, one for ARM) are easily security holes. I'm not 100% sure if segfault caused by stack misalignment is an exploitable thing, but you can definitely cause a segfault on any ctypes code, even if perfectly safe. Cheers, fijal From barry at python.org Thu Apr 18 20:57:40 2013 From: barry at python.org (Barry Warsaw) Date: Thu, 18 Apr 2013 14:57:40 -0400 Subject: [Python-Dev] A decade as a core dev In-Reply-To: <20130418193446.1ed036e9@fsol> References: <20130418193446.1ed036e9@fsol> Message-ID: <20130418145740.26da7011@anarchist> On Apr 18, 2013, at 07:34 PM, Antoine Pitrou wrote: >Normally you should break a buildbot as a celebration :) Or do a release and go on vacation. -Barry From theller at ctypes.org Thu Apr 18 21:22:10 2013 From: theller at ctypes.org (Thomas Heller) Date: Thu, 18 Apr 2013 21:22:10 +0200 Subject: [Python-Dev] libffi inclusion in python In-Reply-To: References: Message-ID: >>> libffi has bugs sometimes (like this >>> http://bugs.python.org/issue17580). Now this is a thing that upstream >>> fixes really quickly, but tracking down issues on bugs.python.org is >>> annoying (they never get commited as quickly as the upstream). is >>> there a good reason why cpython has it's own copy of libffi? I >>> understand historical reasons, but PyPy gets along relying on the >>> system library, so maybe we can kill the inclusion. >> >> IIRC, it had (has?) some custom windows patches, which no one knows >> whether they're relevant or not. The only windows patch that is most certainly not in upstream is the ability to check and correct the stack pointer after a windows function call depending on the calling convention (stdcall or cdecl). Since this is windows 32-bit only (on windows 64-bit these calling conventions are aliases for the same thing), it would probably be good to remove the distinction between them for function calls. > Upstream seems to be incredibly responsive. Why not merge them there? At the time when ctypes was implemented, this was different. They didn't even do libffi releases, IIRC. Thomas From fijall at gmail.com Thu Apr 18 21:24:58 2013 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 18 Apr 2013 21:24:58 +0200 Subject: [Python-Dev] libffi inclusion in python In-Reply-To: References: Message-ID: On Thu, Apr 18, 2013 at 9:22 PM, Thomas Heller wrote: >>>> libffi has bugs sometimes (like this >>>> http://bugs.python.org/issue17580). Now this is a thing that upstream >>>> fixes really quickly, but tracking down issues on bugs.python.org is >>>> annoying (they never get commited as quickly as the upstream). is >>>> there a good reason why cpython has it's own copy of libffi? I >>>> understand historical reasons, but PyPy gets along relying on the >>>> system library, so maybe we can kill the inclusion. >>> >>> >>> IIRC, it had (has?) some custom windows patches, which no one knows >>> whether they're relevant or not. > > > The only windows patch that is most certainly not in upstream is the > ability to check and correct the stack pointer after a windows function > call depending on the calling convention (stdcall or cdecl). > > Since this is windows 32-bit only (on windows 64-bit these calling > conventions are aliases for the same thing), it would probably be good > to remove the distinction between them for function calls. > > > >> Upstream seems to be incredibly responsive. Why not merge them there? > > > At the time when ctypes was implemented, this was different. They > didn't even do libffi releases, IIRC. > > Thomas Cool. Note that I don't ask "why the hell was it included", I assume there was the right choice at a time, I ask "can we remove it now?" which seems to be mostly yes. Cheers, fijal From ronaldoussoren at mac.com Thu Apr 18 23:17:38 2013 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu, 18 Apr 2013 23:17:38 +0200 Subject: [Python-Dev] libffi inclusion in python In-Reply-To: References: Message-ID: <55BA0357-2222-46AE-BDBD-A58F07DA760B@mac.com> On 18 apr. 2013, at 18:09, Ned Deily wrote: > In article > , > Benjamin Peterson wrote: >> 2013/4/18 Maciej Fijalkowski : >>> libffi has bugs sometimes (like this >>> http://bugs.python.org/issue17580). Now this is a thing that upstream >>> fixes really quickly, but tracking down issues on bugs.python.org is >>> annoying (they never get commited as quickly as the upstream). is >>> there a good reason why cpython has it's own copy of libffi? I >>> understand historical reasons, but PyPy gets along relying on the >>> system library, so maybe we can kill the inclusion. >> >> IIRC, it had (has?) some custom windows patches, which no one knows >> whether they're relevant or not. > > The cpython copy also has custom OS X patches. I've never looked at > them so I don't have a feel for how much work would be involved in > migrating to current upstream. If it's just a matter of supporting > universal builds, it could be done with some Makefile hacking to do a > lipo dance. Ronald, any additional thoughts? It is probably just a matter of supporting universal builds, but I haven't checked the state of upstream libffi in the last couple of years. The libffi_osx tree is a fork from upstream that started around the time of OSX 10.4, and possibly earlier. As Thomas mentioned the upstream maintainer weren't very responsive in the past, at the time I hacked on libffi (IIRC for Darwin/i386 support) it wasn't even clear how the maintainers could be reached. Stripping libffi from python's source tree would be fine by me, but would require testing with upstream libffi. AFAIK system libffi on osx wouldn't be goog enough, it doesn't work properly with clang. Ronald > > http://bugs.python.org/issue15194 > > Currently, for the OS X installer builds, we build a number of > third-party libs that are either missing from OS X (like lzma) or are > too out-of-date on the oldest systems we support. It would be useful to > generalize the third-party lib support and move it out of the installer > build process so that all builds could take advantage of the libs if > needed. libffi could be added to those. Of course, that wouldn't help > for Windows builds. > > -- > Ned Deily, > nad at acm.org > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com From dirkjan at ochtman.nl Fri Apr 19 10:13:09 2013 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Fri, 19 Apr 2013 10:13:09 +0200 Subject: [Python-Dev] libffi inclusion in python In-Reply-To: <55BA0357-2222-46AE-BDBD-A58F07DA760B@mac.com> References: <55BA0357-2222-46AE-BDBD-A58F07DA760B@mac.com> Message-ID: On Thu, Apr 18, 2013 at 11:17 PM, Ronald Oussoren wrote: > Stripping libffi from python's source tree would be fine by me, but would require testing with upstream libffi. AFAIK system libffi on osx wouldn't be goog enough, it doesn't work properly with clang. If you mean http://bugs.python.org/issue17136, I think that has been fixed in libffi upstream? Cheers, Dirkjan From ronaldoussoren at mac.com Fri Apr 19 10:18:48 2013 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Fri, 19 Apr 2013 10:18:48 +0200 Subject: [Python-Dev] libffi inclusion in python In-Reply-To: References: <55BA0357-2222-46AE-BDBD-A58F07DA760B@mac.com> Message-ID: <0939F657-5374-40CA-B91C-42B079094AD7@mac.com> On 19 Apr, 2013, at 10:13, Dirkjan Ochtman wrote: > On Thu, Apr 18, 2013 at 11:17 PM, Ronald Oussoren > wrote: >> Stripping libffi from python's source tree would be fine by me, but would require testing with upstream libffi. AFAIK system libffi on osx wouldn't be goog enough, it doesn't work properly with clang. > > If you mean http://bugs.python.org/issue17136, I think that has been > fixed in libffi upstream? Appearently, but not in /usr/lib/libffi.dylib (at least nog when I checked it last time, which was at least one minor release of 10.8 ago). Which means that the binary installers will still have to use a bundled copy of libffi, but as Ned wrote that's not really a problem because the script that builds those installers already includes some other libraries that are not available on OSX or are too old. That said, I haven't tested yet if upstream libffi actually works although others seem to have. Ronald From asolano at icai.es Fri Apr 19 11:04:04 2013 From: asolano at icai.es (=?UTF-8?Q?Alfredo_Solano_Mart=C3=ADnez?=) Date: Fri, 19 Apr 2013 11:04:04 +0200 Subject: [Python-Dev] A decade as a core dev In-Reply-To: <20130418145740.26da7011@anarchist> References: <20130418193446.1ed036e9@fsol> <20130418145740.26da7011@anarchist> Message-ID: As an average python user, thank you for your commitment, and keep up the good work! Alfredo On Thu, Apr 18, 2013 at 8:57 PM, Barry Warsaw wrote: > On Apr 18, 2013, at 07:34 PM, Antoine Pitrou wrote: > >>Normally you should break a buildbot as a celebration :) > > Or do a release and go on vacation. > > -Barry > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/asolano%40icai.es From status at bugs.python.org Fri Apr 19 18:07:31 2013 From: status at bugs.python.org (Python tracker) Date: Fri, 19 Apr 2013 18:07:31 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20130419160731.3479556A30@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2013-04-12 - 2013-04-19) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3928 (+20) closed 25642 (+73) total 29570 (+93) Open issues with patches: 1745 Issues opened (66) ================== #13355: random.triangular error when low = high=mode http://bugs.python.org/issue13355 reopened by rhettinger #14398: bz2.BZ2DEcompressor.decompress fail on large files http://bugs.python.org/issue14398 reopened by nadeem.vawda #17705: Fill Character cannot be \0 http://bugs.python.org/issue17705 opened by Julian #17708: sys.flags.hash_randomization doesn't return correct value http://bugs.python.org/issue17708 opened by msmhrt #17709: http://docs.python.org/2.7/objects.inv doesn't support :func:` http://bugs.python.org/issue17709 opened by ras #17711: Persistent id in pickle with protocol version 0 http://bugs.python.org/issue17711 opened by serhiy.storchaka #17712: test_gdb failures http://bugs.python.org/issue17712 opened by pitrou #17714: str.encode('base64') add trailing new line character. It is no http://bugs.python.org/issue17714 opened by alex.75 #17716: From ... import fails when parent package failed but child mod http://bugs.python.org/issue17716 opened by Pascal.Chambon #17717: Set up nasm from external.bat http://bugs.python.org/issue17717 opened by jpe #17718: boolop constant checking for if/while http://bugs.python.org/issue17718 opened by Stephen.Tu #17720: pickle.py's load_appends should call append() on objects other http://bugs.python.org/issue17720 opened by alexandre.vassalotti #17721: Help button on preference window doesn't work http://bugs.python.org/issue17721 opened by Guilherme.Sim??es #17723: Use FileRead and FileWrite in fileio.c on Windows http://bugs.python.org/issue17723 opened by jpe #17725: English mistake in Extending and Embedding Python doc page. http://bugs.python.org/issue17725 opened by Kyle.Simpson #17727: document that some distributions change site.py defaults http://bugs.python.org/issue17727 opened by georg.brandl #17729: advocacy howto improvements http://bugs.python.org/issue17729 opened by georg.brandl #17730: code.interact() doesn't support no banner http://bugs.python.org/issue17730 opened by Drekin #17732: distutils.cfg Can Break venv http://bugs.python.org/issue17732 opened by nicksloan #17733: Add tests to test__header_value_parser for RFC 2231 parsing co http://bugs.python.org/issue17733 opened by Ankur.Ankan #17734: Failure when running test_builtin after test_genexps http://bugs.python.org/issue17734 opened by ncoghlan #17735: inspect.findsource throws IndexError http://bugs.python.org/issue17735 opened by Kyle.Simpson #17736: Misleading method comment in _elementtree.c : get_attrib_from_ http://bugs.python.org/issue17736 opened by Sjlver #17737: test_gdb fails on armv7hl http://bugs.python.org/issue17737 opened by bkabrda #17742: Add _PyBytesWriter API http://bugs.python.org/issue17742 opened by haypo #17743: Use extended syntax of `set` command in activate.bat/deactivat http://bugs.python.org/issue17743 opened by piotr.dobrogost #17744: Unset VIRTUAL_ENV environment variable in deactivate.bat http://bugs.python.org/issue17744 opened by piotr.dobrogost #17745: "packaging" no longer planned to be included http://bugs.python.org/issue17745 opened by tshepang #17746: test_shutil.TestWhich.test_non_matching_mode fails when runnin http://bugs.python.org/issue17746 opened by jibel #17747: Deprecate pickle fast mode http://bugs.python.org/issue17747 opened by alexandre.vassalotti #17750: allow the testsuite to run in the installed location http://bugs.python.org/issue17750 opened by doko #17751: ctypes/test/test_macholib.py fails when run from the installed http://bugs.python.org/issue17751 opened by doko #17752: many distutils tests fail when run from the installed location http://bugs.python.org/issue17752 opened by doko #17753: test_zipfile: requires write access to test and email.test http://bugs.python.org/issue17753 opened by doko #17754: test_ctypes assumes LANG=C LC_ALL=C http://bugs.python.org/issue17754 opened by doko #17755: test_builtin assumes LANG=C http://bugs.python.org/issue17755 opened by doko #17756: test_syntax_error fails when run in the installed location http://bugs.python.org/issue17756 opened by doko #17758: test_site fails when the user does not have a home directory http://bugs.python.org/issue17758 opened by doko #17759: test_urllibnet.test_bad_address() fails on Ubuntu 13.04 http://bugs.python.org/issue17759 opened by barry #17761: platform._parse_release_file doesn't close the /etc/lsb-releas http://bugs.python.org/issue17761 opened by doko #17762: platform.linux_distribution() should honor /etc/os-release http://bugs.python.org/issue17762 opened by doko #17763: test_pydoc fails with the installed testsuite http://bugs.python.org/issue17763 opened by doko #17764: Support http.server passing bind address via commend line argu http://bugs.python.org/issue17764 opened by malte.swart #17765: weakref.ref ignores a 'callback' keyword argument http://bugs.python.org/issue17765 opened by mark.dickinson #17767: Fix test discovery for test_locale.py http://bugs.python.org/issue17767 opened by zach.ware #17768: _decimal: allow NUL fill character http://bugs.python.org/issue17768 opened by skrah #17769: python-config --ldflags gives broken output when statically li http://bugs.python.org/issue17769 opened by Max.Cantor #17772: test_gdb doesn't detect a gdb built with python3.3 (or higher) http://bugs.python.org/issue17772 opened by doko #17773: test_pydoc fails with the installed testsuite (2.7) http://bugs.python.org/issue17773 opened by doko #17774: unable to disable -r in run_tests.py http://bugs.python.org/issue17774 opened by doko #17776: IDLE Internationalization http://bugs.python.org/issue17776 opened by mariedam #17777: Unrecognized string literal escape sequences give SyntaxErrors http://bugs.python.org/issue17777 opened by reynir #17778: Fix test discovery for test_multiprocessing.py http://bugs.python.org/issue17778 opened by zach.ware #17780: the test suite should use a TEMPDIR in the build directory, no http://bugs.python.org/issue17780 opened by doko #17781: optimize compilation options http://bugs.python.org/issue17781 opened by pitrou #17783: run the testsuite in batched mode http://bugs.python.org/issue17783 opened by doko #17784: the test suite should honor an http_proxy for running the test http://bugs.python.org/issue17784 opened by doko #17785: Use faster URL shortener for perf.py http://bugs.python.org/issue17785 opened by alexandre.vassalotti #17786: Crash in test_ctypes.test_callbacks() on AMD64 NetBSD 5.1.2 [S http://bugs.python.org/issue17786 opened by haypo #17787: Optimize pickling function dispatch in hot loops. http://bugs.python.org/issue17787 opened by alexandre.vassalotti #17791: PC/pyconfig.h defines PREFIX macro http://bugs.python.org/issue17791 opened by christian.heimes #17792: Unhelpful UnboundLocalError due to del'ing of exception target http://bugs.python.org/issue17792 opened by barry #17794: Priority Queue http://bugs.python.org/issue17794 opened by Claymore #17795: backwards-incompatible change in SysLogHandler with unix domai http://bugs.python.org/issue17795 opened by Mike.Lundy #17796: No mimetype guessed for tar.xz url http://bugs.python.org/issue17796 opened by lcid-fire #17797: Visual C++ 11.0 reports fileno(stdin) == 0 for non-console pro http://bugs.python.org/issue17797 opened by mloskot Most recent 15 issues with no replies (15) ========================================== #17795: backwards-incompatible change in SysLogHandler with unix domai http://bugs.python.org/issue17795 #17791: PC/pyconfig.h defines PREFIX macro http://bugs.python.org/issue17791 #17786: Crash in test_ctypes.test_callbacks() on AMD64 NetBSD 5.1.2 [S http://bugs.python.org/issue17786 #17785: Use faster URL shortener for perf.py http://bugs.python.org/issue17785 #17784: the test suite should honor an http_proxy for running the test http://bugs.python.org/issue17784 #17783: run the testsuite in batched mode http://bugs.python.org/issue17783 #17780: the test suite should use a TEMPDIR in the build directory, no http://bugs.python.org/issue17780 #17774: unable to disable -r in run_tests.py http://bugs.python.org/issue17774 #17773: test_pydoc fails with the installed testsuite (2.7) http://bugs.python.org/issue17773 #17769: python-config --ldflags gives broken output when statically li http://bugs.python.org/issue17769 #17767: Fix test discovery for test_locale.py http://bugs.python.org/issue17767 #17765: weakref.ref ignores a 'callback' keyword argument http://bugs.python.org/issue17765 #17764: Support http.server passing bind address via commend line argu http://bugs.python.org/issue17764 #17762: platform.linux_distribution() should honor /etc/os-release http://bugs.python.org/issue17762 #17756: test_syntax_error fails when run in the installed location http://bugs.python.org/issue17756 Most recent 15 issues waiting for review (15) ============================================= #17795: backwards-incompatible change in SysLogHandler with unix domai http://bugs.python.org/issue17795 #17792: Unhelpful UnboundLocalError due to del'ing of exception target http://bugs.python.org/issue17792 #17787: Optimize pickling function dispatch in hot loops. http://bugs.python.org/issue17787 #17785: Use faster URL shortener for perf.py http://bugs.python.org/issue17785 #17783: run the testsuite in batched mode http://bugs.python.org/issue17783 #17781: optimize compilation options http://bugs.python.org/issue17781 #17778: Fix test discovery for test_multiprocessing.py http://bugs.python.org/issue17778 #17776: IDLE Internationalization http://bugs.python.org/issue17776 #17768: _decimal: allow NUL fill character http://bugs.python.org/issue17768 #17767: Fix test discovery for test_locale.py http://bugs.python.org/issue17767 #17764: Support http.server passing bind address via commend line argu http://bugs.python.org/issue17764 #17761: platform._parse_release_file doesn't close the /etc/lsb-releas http://bugs.python.org/issue17761 #17747: Deprecate pickle fast mode http://bugs.python.org/issue17747 #17742: Add _PyBytesWriter API http://bugs.python.org/issue17742 #17736: Misleading method comment in _elementtree.c : get_attrib_from_ http://bugs.python.org/issue17736 Top 10 most discussed issues (10) ================================= #17618: base85 encoding http://bugs.python.org/issue17618 18 msgs #17732: distutils.cfg Can Break venv http://bugs.python.org/issue17732 15 msgs #16694: Add pure Python operator module http://bugs.python.org/issue16694 14 msgs #16624: subprocess.check_output should allow specifying stdin as a str http://bugs.python.org/issue16624 13 msgs #5004: socket.getfqdn() doesn't cope properly with purely DNS-based s http://bugs.python.org/issue5004 9 msgs #17792: Unhelpful UnboundLocalError due to del'ing of exception target http://bugs.python.org/issue17792 9 msgs #13355: random.triangular error when low = high=mode http://bugs.python.org/issue13355 8 msgs #15301: os.chown: OverflowError: Python int too large to convert to C http://bugs.python.org/issue15301 8 msgs #17353: Plistlib outputs empty data tags when deeply nested http://bugs.python.org/issue17353 8 msgs #17673: add `copy_from` argument to temporaryfile http://bugs.python.org/issue17673 8 msgs Issues closed (74) ================== #2118: smtplib.SMTP() raises socket.error rather than SMTPConnectErro http://bugs.python.org/issue2118 closed by r.david.murray #4140: urllib2: request with digest auth through proxy fail http://bugs.python.org/issue4140 closed by orsenthil #6386: importing yields unexpected results when initial script is a s http://bugs.python.org/issue6386 closed by brett.cannon #6783: Add a builtin method to 'int' for base/radix conversion http://bugs.python.org/issue6783 closed by r.david.murray #7100: test_xmlrpc: global name 'stop_serving' is not defined http://bugs.python.org/issue7100 closed by pitrou #11182: remove unused undocumented pydoc.Scanner class http://bugs.python.org/issue11182 closed by ezio.melotti #12768: docstrings for the threading module http://bugs.python.org/issue12768 closed by eli.bendersky #13050: RLock support the context manager protocol but this is not doc http://bugs.python.org/issue13050 closed by georg.brandl #13510: Clarify that readlines() is not needed to iterate over a file http://bugs.python.org/issue13510 closed by ezio.melotti #13638: PyErr_SetFromErrnoWithFilenameObject is undocumented http://bugs.python.org/issue13638 closed by python-dev #13775: Access Denied message on symlink creation misleading for an ex http://bugs.python.org/issue13775 closed by serhiy.storchaka #14462: In re's named group the name cannot contain unicode characters http://bugs.python.org/issue14462 closed by python-dev #14735: Version 3.2.3 IDLE CTRL-Z plus Carriage Return to end does not http://bugs.python.org/issue14735 closed by roger.serwy #15480: Drop TYPE_INT64 from marshal in Python 3.4 http://bugs.python.org/issue15480 closed by pitrou #16061: performance regression in string replace for 3.3 http://bugs.python.org/issue16061 closed by serhiy.storchaka #16163: Wrong name in Lib/pkgutil.py:iter_importers http://bugs.python.org/issue16163 closed by ncoghlan #16447: SEGFAULT when setting type.__name__ http://bugs.python.org/issue16447 closed by mark.dickinson #16550: pickletools.py treats 32bit lengths as signed, but pickle.py a http://bugs.python.org/issue16550 closed by serhiy.storchaka #16551: Cleanup the pure Python pickle implementation http://bugs.python.org/issue16551 closed by python-dev #16658: Missing "return" in HTTPConnection.send() http://bugs.python.org/issue16658 closed by asvetlov #16804: python3 -S -m site fails http://bugs.python.org/issue16804 closed by meador.inge #16812: os.symlink can return wrong FileExistsError/WindowsError infor http://bugs.python.org/issue16812 closed by serhiy.storchaka #17012: Differences between /usr/bin/which and shutil.which() http://bugs.python.org/issue17012 closed by barry #17016: _sre: avoid relying on pointer overflow http://bugs.python.org/issue17016 closed by serhiy.storchaka #17135: imp doc should direct to importlib http://bugs.python.org/issue17135 closed by r.david.murray #17244: py_compile.compile() fails to raise exceptions when writing of http://bugs.python.org/issue17244 closed by brett.cannon #17320: os.path.abspath in window7, return error http://bugs.python.org/issue17320 closed by xiaowei.py #17341: Poor error message when compiling invalid regex http://bugs.python.org/issue17341 closed by r.david.murray #17491: Consolidate traceback.format_tb and traceback.print_tb http://bugs.python.org/issue17491 closed by benjamin.peterson #17536: update browser list with additional browser names http://bugs.python.org/issue17536 closed by doko #17555: ForkAwareThreadLock leak after fork http://bugs.python.org/issue17555 closed by sbt #17571: broken links on Lib/datetime.py docstring http://bugs.python.org/issue17571 closed by ezio.melotti #17573: add ElementTree XML processing benchmark to benchmark suite http://bugs.python.org/issue17573 closed by pitrou #17575: HTTPConnection.send http://bugs.python.org/issue17575 closed by dspublic at freemail.hu #17643: Expose weakref callback for introspection purposes. http://bugs.python.org/issue17643 closed by mark.dickinson #17647: subprocess.communicate() should preserve colored output on Win http://bugs.python.org/issue17647 closed by terry.reedy #17653: fix typo in socketserver.rst http://bugs.python.org/issue17653 closed by ezio.melotti #17661: documentation of '%r' links to the wrong repr http://bugs.python.org/issue17661 closed by python-dev #17676: spwd uses -1 for empty attributes http://bugs.python.org/issue17676 closed by terry.reedy #17686: Doc using/unix broken link (http://linuxmafia.com/) http://bugs.python.org/issue17686 closed by ezio.melotti #17694: Enhance _PyUnicodeWriter API to control minimum buffer length http://bugs.python.org/issue17694 closed by python-dev #17702: os.environ converts key type from string to bytes in KeyError http://bugs.python.org/issue17702 closed by python-dev #17703: Trashcan mechanism segfault during interpreter finalization in http://bugs.python.org/issue17703 closed by pitrou #17706: Segfault in PyErr_SetObject http://bugs.python.org/issue17706 closed by pitrou #17707: Multiprocessing queue get method does not block for short time http://bugs.python.org/issue17707 closed by giampaolo.rodola #17710: SystemError in cPickle for incorrect input http://bugs.python.org/issue17710 closed by pitrou #17713: test_logging fails in test_compute_rollover_weekly_attime http://bugs.python.org/issue17713 closed by python-dev #17715: Raising an exception in __trunc__ gives a segmentation fault. http://bugs.python.org/issue17715 closed by mark.dickinson #17719: IDLE help text refers to incorrect Python version http://bugs.python.org/issue17719 closed by r.david.murray #17722: 'round' function doesn't honour a descriptor __round__ http://bugs.python.org/issue17722 closed by python-dev #17724: urllib -- add_handler method refactoring for clarity http://bugs.python.org/issue17724 closed by Max.Mautner #17726: faq/design: improve clarity http://bugs.python.org/issue17726 closed by python-dev #17728: format() default precisions undocumented http://bugs.python.org/issue17728 closed by eric.smith #17731: test_iter_importers intermittent failure in test_pkgutil http://bugs.python.org/issue17731 closed by python-dev #17738: Unnecessary "if" in SHA1_copy http://bugs.python.org/issue17738 closed by python-dev #17739: ssl.SSLSocket.getpeercert does not return client certificate http://bugs.python.org/issue17739 closed by pitrou #17740: :func:`socket` in socket.rst links to socket module, not socke http://bugs.python.org/issue17740 closed by ezio.melotti #17741: event-driven XML parser http://bugs.python.org/issue17741 closed by pitrou #17748: Condition.wait timeout can't be set to more than 50 ms http://bugs.python.org/issue17748 closed by pitrou #17749: root logging functions break logger configuration http://bugs.python.org/issue17749 closed by vinay.sajip #17757: test_executable_without_cwd fails when run in the installed lo http://bugs.python.org/issue17757 closed by ned.deily #17760: No i18n of IDLE's interface in french http://bugs.python.org/issue17760 closed by ezio.melotti #17766: Fix test discovery for test_iterlen.py http://bugs.python.org/issue17766 closed by ezio.melotti #17770: MSI installer default behaviour inconsistent http://bugs.python.org/issue17770 closed by Harry.Johnston #17771: Missing period in concurrent execution doc in standard library http://bugs.python.org/issue17771 closed by ezio.melotti #17775: Error with Hello, World in 3.3.1 http://bugs.python.org/issue17775 closed by walkah21 #17779: Fix test discovery for test_osx_env.py http://bugs.python.org/issue17779 closed by ezio.melotti #17782: Fix test_signal failure on x32 http://bugs.python.org/issue17782 closed by pitrou #17788: Add a with expression, for use in comprehensions http://bugs.python.org/issue17788 closed by eric.smith #17789: Fix test discovery for test_random.py http://bugs.python.org/issue17789 closed by ezio.melotti #17790: Fix test discovery for test_set.py http://bugs.python.org/issue17790 closed by ezio.melotti #17793: spam http://bugs.python.org/issue17793 closed by ezio.melotti #1626300: 'Installing Python Modules' does not work for Windows http://bugs.python.org/issue1626300 closed by eric.araujo #992389: attribute error due to circular import http://bugs.python.org/issue992389 closed by ncoghlan From ncoghlan at gmail.com Fri Apr 19 18:36:59 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 20 Apr 2013 02:36:59 +1000 Subject: [Python-Dev] [Python-checkins] cpython: #11182: remove the unused and undocumented pydoc.Scanner class. Patch by In-Reply-To: <3ZsHFq0rvfzSWf@mail.python.org> References: <3ZsHFq0rvfzSWf@mail.python.org> Message-ID: On Fri, Apr 19, 2013 at 9:53 AM, ezio.melotti wrote: > http://hg.python.org/cpython/rev/465cb5ce5a7e > changeset: 83443:465cb5ce5a7e > user: Ezio Melotti > date: Fri Apr 19 02:53:12 2013 +0300 > summary: > #11182: remove the unused and undocumented pydoc.Scanner class. Patch by Martin Morrison. I was going to comment on the lack of a leading underscore, but then I saw Terry's comment about pydoc.__all__ on the tracker issue. Carry on, nothing to see here :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From barry at python.org Sat Apr 20 20:10:32 2013 From: barry at python.org (Barry Warsaw) Date: Sat, 20 Apr 2013 14:10:32 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> Message-ID: <20130420141032.46acd456@anarchist> On Apr 13, 2013, at 08:37 AM, Tim Delaney wrote: >Just using definition order as the stable iteration order would do the >trick - no need for any comparisons at all. Subclasses (e.g. IntEnum) can >then override it. I think this isn't possible if we want to keep backward compatibility with earlier Pythons, which I want to do. OTOH, we have another natural sorting order for base Enums sitting right in front of us: the attribute name. These have to be unique and ordered, so why not use this for both the __repr__() and the base Enum __iter__()? IntEnum can override __iter__() to iterate over item values, which also must be ordered. I just made this change to flufl.enum and it seems to work well. >>> from flufl.enum import Enum >>> A = Enum('A', 'a b c') >>> A >>> for item in A: print(item) ... A.a A.b A.c >>> B = Enum('B', 'c b a') >>> B >>> for item in B: print(item) ... B.a B.b B.c >>> from flufl.enum import IntEnum >>> C = IntEnum('C', 'c b a') >>> C >>> for item in C: print(item) ... C.c C.b C.a -Barry From rdmurray at bitdance.com Sat Apr 20 20:26:30 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Sat, 20 Apr 2013 14:26:30 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130420141032.46acd456@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> Message-ID: <20130420182630.C36222500B3@webabinitio.net> On Sat, 20 Apr 2013 14:10:32 -0400, Barry Warsaw wrote: > On Apr 13, 2013, at 08:37 AM, Tim Delaney wrote: > > >Just using definition order as the stable iteration order would do the > >trick - no need for any comparisons at all. Subclasses (e.g. IntEnum) can > >then override it. > > I think this isn't possible if we want to keep backward compatibility with > earlier Pythons, which I want to do. OTOH, we have another natural sorting > order for base Enums sitting right in front of us: the attribute name. These > have to be unique and ordered, so why not use this for both the __repr__() and > the base Enum __iter__()? IntEnum can override __iter__() to iterate over > item values, which also must be ordered. > > I just made this change to flufl.enum and it seems to work well. > > >>> from flufl.enum import Enum > >>> A = Enum('A', 'a b c') > >>> A > > >>> for item in A: print(item) > ... > A.a > A.b > A.c > >>> B = Enum('B', 'c b a') > >>> B > > >>> for item in B: print(item) > ... > B.a > B.b > B.c > >>> from flufl.enum import IntEnum > >>> C = IntEnum('C', 'c b a') > >>> C > > >>> for item in C: print(item) > ... > C.c > C.b > C.a I think definition order would be much better, but if we can't have that, this is probably better than value order for non-int. --David From barry at python.org Sat Apr 20 21:18:08 2013 From: barry at python.org (Barry Warsaw) Date: Sat, 20 Apr 2013 15:18:08 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: Message-ID: <20130420151808.62376061@anarchist> On Apr 13, 2013, at 11:31 AM, Serhiy Storchaka wrote: >On 12.04.13 15:55, Eli Bendersky wrote: >> The enumeration value names are available through the class members:: >> >> >>> for member in Colors.__members__: >> ... print(member) >> red >> green >> blue > >This is unnecessary because enumerations are iterable. Colors.__members__ is >equal to [v.name for v in Colors] and the latter looks more preferable, >because it does not use the magic method. __members__ was really a holdover from earlier versions. I've removed this, but I've added an __dir__(). >> The str and repr of the enumeration class also provides useful information:: >> >> >>> print(Colors) >> >> >>> print(repr(Colors)) >> > >Does the enumeration's repr() use str() or repr() for the enumeration values? No, enumeration values have different reprs and strs. >And same question for the enumeration's str(). Enumerations share a repr and str (well, technically, Enums don't define a __str__()). >> To programmatically access enumeration values, use ``getattr``:: >> >> >>> getattr(Colors, 'red') >> > >How to get the enumeration value by its value? Use getitem syntax: >>> from flufl.enum import Enum >>> A = Enum('A', 'a b c') >>> A[2] >> Ordered comparisons between enumeration values are *not* supported. Enums >> are not integers (but see `IntEnum`_ below):: > >It's unexpected if values of the enumeration values have the natural >order. And values of the enumeration values *should be* comparable >("Iteration is defined as the sorted order of the item values"). This is one reason why Enums are not comparable except by equality. While I think it's not a good idea to mix the types of Enum item values, it *is* possible, and I don't think we should add strict checks to enforce this for the base Enum. Thus we cannot guarantee that <, >, <=, or >= will not throw a TypeError. IntEnums do define these because they can be guaranteed to succeed, since their enumeration item values are guaranteed to be integers. >There is some ambiguity in the term "enumeration values". On the one hand, >it's the singleton instances of the enumeration class (Colors.red, >Colors.gree, Colors.blue), and on the other hand it is their values (1, 2, >3). I've just made sure that the flufl.enum using.rst document is consistent here. The terms I'm using are "enumeration item" to define things like Colors.red and "enumeration item value" (or sometimes just "enumeration value") to define the value of the enumeration item, e.g. 2, and it's available on the .value attribute of the item. "Enumeration item name" is essentially the attribute name, and is available on the .name attribute of the item. >> But if the value *is* important, enumerations can have arbitrary values. > >Should enumeration values be hashable? > >At least they should be comparable ("Iteration is defined as the sorted order >of the item values"). Given my previous responses, these questions should be already answered. >> ``IntEnum`` values behave like integers in other ways you'd expect:: >> >> >>> int(Shape.circle) >> 1 >> >>> ['a', 'b', 'c'][Shape.circle] >> 'b' >> >>> [i for i in range(Shape.square)] >> [0, 1] > >What is ``isinstance(Shape.circle, int)``? Does PyLong_Check() return true >for ``IntEnum`` values? True. Yes, because IntEnumValues inherit from int. >Why the enumeration starts from 1? It is not consistent with namedtuple, in >which indices are zero-based, and I believe that in most practical cases the >enumeration integer values are zero-based. There are several reasons: * It's been that way since day 1 of the package * Zero is special in a way; it's the only false integer value * This is very easy to override * If you're using the auto-numbering convenience API, then you don't care about the values anyway >The Python standard library has many places where named integer constants >used as bitmasks (i.e. os.O_CREAT | os.O_WRONLY | os.O_TRUNC, select.POLLIN | >select.POLLPRI, re.IGNORECASE | re.ASCII). The proposed PEP is not applicable >to these cases. Whether it is planned expansion of Enum or additional EnumSet >class to aid in these cases? IntEnums work fine for these cases, but it's true that the result of a logical operation is an int and not a subclass with a nice repr. Contributions are welcome. -Barry From guido at python.org Sat Apr 20 21:26:52 2013 From: guido at python.org (Guido van Rossum) Date: Sat, 20 Apr 2013 12:26:52 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130420182630.C36222500B3@webabinitio.net> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420182630.C36222500B3@webabinitio.net> Message-ID: Can we separate the iteration order and the comparison order? For iteration order, I think by definition order or by attribute name are both great, and better than by value. But for comparing values using <, ==, >, I strongly feel we should defer to the underlying values -- if those cannot be compared, then the enums can't either, but the iteration order is still defined. On Sat, Apr 20, 2013 at 11:26 AM, R. David Murray wrote: > On Sat, 20 Apr 2013 14:10:32 -0400, Barry Warsaw wrote: >> On Apr 13, 2013, at 08:37 AM, Tim Delaney wrote: >> >> >Just using definition order as the stable iteration order would do the >> >trick - no need for any comparisons at all. Subclasses (e.g. IntEnum) can >> >then override it. >> >> I think this isn't possible if we want to keep backward compatibility with >> earlier Pythons, which I want to do. OTOH, we have another natural sorting >> order for base Enums sitting right in front of us: the attribute name. These >> have to be unique and ordered, so why not use this for both the __repr__() and >> the base Enum __iter__()? IntEnum can override __iter__() to iterate over >> item values, which also must be ordered. >> >> I just made this change to flufl.enum and it seems to work well. >> >> >>> from flufl.enum import Enum >> >>> A = Enum('A', 'a b c') >> >>> A >> >> >>> for item in A: print(item) >> ... >> A.a >> A.b >> A.c >> >>> B = Enum('B', 'c b a') >> >>> B >> >> >>> for item in B: print(item) >> ... >> B.a >> B.b >> B.c >> >>> from flufl.enum import IntEnum >> >>> C = IntEnum('C', 'c b a') >> >>> C >> >> >>> for item in C: print(item) >> ... >> C.c >> C.b >> C.a > > I think definition order would be much better, but if we can't have that, > this is probably better than value order for non-int. > > --David > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) From barry at python.org Sat Apr 20 21:31:46 2013 From: barry at python.org (Barry Warsaw) Date: Sat, 20 Apr 2013 15:31:46 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: Message-ID: <20130420153146.0079fccf@anarchist> On Apr 13, 2013, at 08:33 AM, Guido van Rossum wrote: >(And yes, I am now +1 on documenting this mechanism.) Here's what I've added to the flufl.enum documentation: Customization protocol ====================== You can define your own enumeration value types by using the ``__value_factory__`` protocol. This is how the ``IntEnum`` type is defined. As an example, let's say you want to define a new type of enumeration where the values were subclasses of ``str``. First, define your enumeration value subclass. >>> from flufl.enum import EnumValue >>> class StrEnumValue(str, EnumValue): ... def __new__(cls, enum, value, attr): ... return super(StrEnumValue, cls).__new__(cls, value) And then define your enumeration class. You must set the class attribute ``__value_factory__`` to the class of the values you want to create. >>> class StrEnum(Enum): ... __value_factory__ = StrEnumValue Now, when you define your enumerations, the values will be ``str`` subclasses. :: >>> class Noises(StrEnum): ... dog = 'bark' ... cat = 'meow' ... cow = 'moo' >>> isinstance(Noises.cow, str) True -Barry From barry at python.org Sat Apr 20 21:42:59 2013 From: barry at python.org (Barry Warsaw) Date: Sat, 20 Apr 2013 15:42:59 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5168C83F.4050103@pearwood.info> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> <20130412184449.6d1e9806@anarchist> <5168A30E.9000202@g.nevcal.com> <5168C83F.4050103@pearwood.info> Message-ID: <20130420154259.0ed44394@anarchist> On Apr 13, 2013, at 12:51 PM, Steven D'Aprano wrote: >I think that's too strong a restriction. I would expect to be able to do this: > >class Insect(Enum): > wsap = 1 # Oops, needed for backward compatibility, do not remove. > wasp = 1 # Preferred. Use this in new code. > bee = 2 > ant = 3 > > >Or at the very least: > >class Insect(Enum): > wasp = wsap = 1 > bee = 2 > ant = 3 > >What's the justification for this restriction? I have looked in the PEP, and >didn't see one. If you allowed this, there would be no way to look up an enumeration item by value. This is necessary for e.g. storing the value in a database. If you know that the "insect" column is an INTEGER that represents an enumeration item of Insect, then you can just store the int value in the column. To reconstitute the actual enumeration item when you read the column back from the database, you need to be able to look up the item by value. Currently, you do this: >>> my_insect = Insect[database_value] but if the values are not unique, you have no way to reliably do it. I don't much like APIs which return sequences (either always or "on demand") or rely on definition order or some other arbitrary discrimination because I don't think any of those are practical in the real world. I also recall that duplication was a specific anti-feature in the previous python-ideas discussion. It was thought that this would allow for typos in the set of enumeration items to creep in. I don't see how you can reconcile these issues to allow for duplicate values. -Barry From timothy.c.delaney at gmail.com Sun Apr 21 00:34:39 2013 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Sun, 21 Apr 2013 08:34:39 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130420141032.46acd456@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> Message-ID: On 21 April 2013 04:10, Barry Warsaw wrote: > On Apr 13, 2013, at 08:37 AM, Tim Delaney wrote: > > >Just using definition order as the stable iteration order would do the > >trick - no need for any comparisons at all. Subclasses (e.g. IntEnum) can > >then override it. > > I think this isn't possible if we want to keep backward compatibility with > earlier Pythons, which I want to do. Do you want it compatible with Python 2.x? In that case I don't see a way to do it - getting definition order relies on __prepare__ returning an ordered dict, and __prepare__ of course is only available in 3.x. Tim Delaney -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdmurray at bitdance.com Sun Apr 21 01:10:49 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Sat, 20 Apr 2013 19:10:49 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> Message-ID: <20130420231049.E041E2500B3@webabinitio.net> On Sun, 21 Apr 2013 08:34:39 +1000, Tim Delaney wrote: > On 21 April 2013 04:10, Barry Warsaw wrote: > > > On Apr 13, 2013, at 08:37 AM, Tim Delaney wrote: > > > > >Just using definition order as the stable iteration order would do the > > >trick - no need for any comparisons at all. Subclasses (e.g. IntEnum) can > > >then override it. > > > > I think this isn't possible if we want to keep backward compatibility with > > earlier Pythons, which I want to do. > > Do you want it compatible with Python 2.x? In that case I don't see a way > to do it - getting definition order relies on __prepare__ returning an > ordered dict, and __prepare__ of course is only available in 3.x. It seems strange to limit a new Python3 feature to the Python2 feature set. Just saying :) --David From steve at pearwood.info Sun Apr 21 03:29:54 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 21 Apr 2013 11:29:54 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130420154259.0ed44394@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> <20130412184449.6d1e9806@anarchist> <5168A30E.9000202@g.nevcal.com> <5168C83F.4050103@pearwood.info> <20130420154259.0ed44394@anarchist> Message-ID: <51734112.5020701@pearwood.info> On 21/04/13 05:42, Barry Warsaw wrote: > On Apr 13, 2013, at 12:51 PM, Steven D'Aprano wrote: > >> I think that's too strong a restriction. I would expect to be able to do this: >> >> class Insect(Enum): >> wsap = 1 # Oops, needed for backward compatibility, do not remove. >> wasp = 1 # Preferred. Use this in new code. >> bee = 2 >> ant = 3 >> >> >> Or at the very least: >> >> class Insect(Enum): >> wasp = wsap = 1 >> bee = 2 >> ant = 3 >> >> What's the justification for this restriction? I have looked in the PEP, and >> didn't see one. > > If you allowed this, there would be no way to look up an enumeration item by > value. This is necessary for e.g. storing the value in a database. If you > know that the "insect" column is an INTEGER that represents an enumeration > item of Insect, then you can just store the int value in the column. To > reconstitute the actual enumeration item when you read the column back from > the database, you need to be able to look up the item by value. I agree that's a good example of a situation where the user might want unique values. But I don't agree that this is the responsibility of the Enum type itself. Enums are a mapping from name to value, just like dicts: d = {'wasp': 1, 'bee': 2, 'ant': 3} There are use-cases where we might want dicts to be 1:1 too, but we don't force that restriction on all dicts. Even if I want to reconstruct the key from the value, doesn't mean that everybody who uses dicts must be prohibited from using duplicate values. We don't even offer a guaranteed 1:1 mapping type in the standard library. Actual real world enums can and do frequently contain duplicate values, I've previously given examples of such. The ability to have two enums with the same value is in my opinion not just a Nice To Have but is a Must Have. With the ability to have duplicate values, enums are the One Obvious Way to do it. Without it, the decision process becomes more complex: does my application now, or will it ever, need duplicate values? If there's even a tiny chance it might need them in the future, I cannot risk getting stuck with an enum type that prohibits that. I would argue that it is the responsibility of enums to start with the least restrictions as is reasonable, and leave additional restrictions up to subclasses, rather than the other way around. (I'll just quietly mention the Liskov Substitution Principle here...) If some people need enums to have unique values, then enforcing that should be their responsibility, and not forced on all users whether they need that restriction or not. If there is enough demand for that, then perhaps the enum module could provide a standard mechanism for enforcing unique values, via a flag, or a subclass, say. I like the idea of a mixin: class Insect(UniqueValues, Enum): wasp = 1 bee = 2 ant = 3 But it should be off by default. > Currently, you do this: > > >>> my_insect = Insect[database_value] > > but if the values are not unique, you have no way to reliably do it. I don't > much like APIs which return sequences (either always or "on demand") or rely > on definition order or some other arbitrary discrimination because I don't > think any of those are practical in the real world. Neither do I. I would be perfectly happy for enums to raise ValueError in that case, and leave it up to the caller to either prevent duplicate values from occurring, or to deal with the exception in whichever way makes sense for their application. > I also recall that duplication was a specific anti-feature in the previous > python-ideas discussion. It was thought that this would allow for typos in > the set of enumeration items to creep in. Typos occur whether enums allow duplicate values or not. How you deal with such typos depends on whether you are forced to keep it forever, or can define a second enum with the same value and deprecate the typo. -- Steven From rurpy at yahoo.com Sun Apr 21 06:55:25 2013 From: rurpy at yahoo.com (Rurpy) Date: Sat, 20 Apr 2013 21:55:25 -0700 (PDT) Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library Message-ID: <1366520125.74620.YahooMailClassic@web125603.mail.ne1.yahoo.com> On 04/20/2013 01:42 PM, Barry Warsaw wrote:> On Apr 13, 2013, at 12:51 PM, Steven D'Aprano wrote: >[...] >>What's the justification for this [unique values] restriction? I have >>looked in the PEP, and didn't see one. > > If you allowed this, there would be no way to look up an enumeration item by > value. This is necessary for e.g. storing the value in a database. If you > know that the "insect" column is an INTEGER that represents an enumeration > item of Insect, then you can just store the int value in the column. To > reconstitute the actual enumeration item when you read the column back from > the database, you need to be able to look up the item by value. >[...] Composite keys have been part of relational databases from their inception. If you want to store an enumeration value in a database when non-unique values are possible, you can do so simply by storing the name, value pair; i.e. use two columns instead of one. Of course this does not preclude storing just the value when you know they will be unique. But it is not true that unique values are *required* for storing enumeration values in a database. From ncoghlan at gmail.com Sun Apr 21 07:25:05 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 21 Apr 2013 15:25:05 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130420231049.E041E2500B3@webabinitio.net> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> Message-ID: On Sun, Apr 21, 2013 at 9:10 AM, R. David Murray wrote: > On Sun, 21 Apr 2013 08:34:39 +1000, Tim Delaney wrote: >> On 21 April 2013 04:10, Barry Warsaw wrote: >> >> > On Apr 13, 2013, at 08:37 AM, Tim Delaney wrote: >> > >> > >Just using definition order as the stable iteration order would do the >> > >trick - no need for any comparisons at all. Subclasses (e.g. IntEnum) can >> > >then override it. >> > >> > I think this isn't possible if we want to keep backward compatibility with >> > earlier Pythons, which I want to do. >> >> Do you want it compatible with Python 2.x? In that case I don't see a way >> to do it - getting definition order relies on __prepare__ returning an >> ordered dict, and __prepare__ of course is only available in 3.x. > > It seems strange to limit a new Python3 feature to the Python2 feature > set. Just saying :) Agreed. I think the stdlib enum library should use __prepare__ and iterate in definition order (since 2.x compatibility isn't of any concern), while flufl.enum can use "sorted by name" as the iteration order. An "order_by_name" keyword argument to __prepare__ in the stdlib version could then allow the user to opt in to the flufl.enum behaviour, while still using definition order by default. As in: class DefinitionOrder(enum.Enum): first = 1 second = 2 third = 3 list(DefinitionOrder) -> [DefinitionOrder.first, DefinitionOrder.second, DefinitionOrder.third] And: class NameOrder(enum.Enum, order_by_name=True): a = 1 c = 2 b = 3 list(NameOrder) -> [NameOrder.a, NameOrder.b, NameOrder.c] flufl.enum could also offer the "order_by_name" flag on 3.x, but set it to "True" by default. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Sun Apr 21 07:33:52 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 21 Apr 2013 15:33:52 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <51734112.5020701@pearwood.info> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> <20130412184449.6d1e9806@anarchist> <5168A30E.9000202@g.nevcal.com> <5168C83F.4050103@pearwood.info> <20130420154259.0ed44394@anarchist> <51734112.5020701@pearwood.info> Message-ID: On Sun, Apr 21, 2013 at 11:29 AM, Steven D'Aprano wrote: > I would argue that it is the responsibility of enums to start with the least > restrictions as is reasonable, and leave additional restrictions up to > subclasses, rather than the other way around. (I'll just quietly mention the > Liskov Substitution Principle here...) If some people need enums to have > unique > values, then enforcing that should be their responsibility, and not forced > on > all users whether they need that restriction or not. > > If there is enough demand for that, then perhaps the enum module could > provide a > standard mechanism for enforcing unique values, via a flag, or a subclass, > say. The PEP is fine, as it already allows duplicate names without encouraging them: class Insect(Enum): wasp = 1 # Preferred. Use this in new code. bee = 2 ant = 3 # Backwards compatibility aliases Insect.wsap = Insect.wasp If you have a lot of such aliases: aliases = { "wasp": ["wsap"], ... } for attr, names in aliases.items(): for name in names: setattr(Insect, name, getattr(Insect, attr)) A more concise syntax for handling duplicates may prove desirable at some point in the future, but this is a case where encouraging correctness by default is a good idea. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From rurpy at yahoo.com Sun Apr 21 07:44:31 2013 From: rurpy at yahoo.com (Rurpy) Date: Sat, 20 Apr 2013 22:44:31 -0700 (PDT) Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <1366520125.74620.YahooMailClassic@web125603.mail.ne1.yahoo.com> Message-ID: <1366523071.14957.YahooMailClassic@web125604.mail.ne1.yahoo.com> On 04/20/2013 10:55 PM, Rurpy wrote: >[...] > But it is not true that unique values are *required* for > storing enumeration values in a database. I should have added that allowing mixed types for values (e.g. as discussed in http://mail.python.org/pipermail/python-dev/2013-April/125322.html) is far more problematic for database storage than non-unique values are. Nearly all databases (Sqlite being an exception) don't allow different types in a column. (Not sure if mixed types is still an open issue or not...) From greg.ewing at canterbury.ac.nz Sun Apr 21 13:02:13 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 21 Apr 2013 23:02:13 +1200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130420154259.0ed44394@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> <20130412184449.6d1e9806@anarchist> <5168A30E.9000202@g.nevcal.com> <5168C83F.4050103@pearwood.info> <20130420154259.0ed44394@anarchist> Message-ID: <5173C735.3050407@canterbury.ac.nz> Barry Warsaw wrote: > On Apr 13, 2013, at 12:51 PM, Steven D'Aprano wrote: >>class Insect(Enum): >> wasp = wsap = 1 >> bee = 2 >> ant = 3 >> >>What's the justification for this restriction? I have looked in the PEP, and >>didn't see one. > > If you allowed this, there would be no way to look up an enumeration item by > value. This is necessary for e.g. storing the value in a database. Hm. What you really want there isn't two enum objects with the same value, but two names bound to the same enum object. Then looking it up by value would not be a problem. -- Greg From timothy.c.delaney at gmail.com Sun Apr 21 14:18:38 2013 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Sun, 21 Apr 2013 22:18:38 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5173C735.3050407@canterbury.ac.nz> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> <20130412184449.6d1e9806@anarchist> <5168A30E.9000202@g.nevcal.com> <5168C83F.4050103@pearwood.info> <20130420154259.0ed44394@anarchist> <5173C735.3050407@canterbury.ac.nz> Message-ID: On 21 April 2013 21:02, Greg Ewing wrote: > Barry Warsaw wrote: > >> On Apr 13, 2013, at 12:51 PM, Steven D'Aprano wrote: >> > > class Insect(Enum): >>> wasp = wsap = 1 >>> bee = 2 >>> ant = 3 >>> >>> What's the justification for this restriction? I have looked in the PEP, >>> and >>> didn't see one. >>> >> >> If you allowed this, there would be no way to look up an enumeration item >> by >> value. This is necessary for e.g. storing the value in a database. >> > > Hm. What you really want there isn't two enum objects with > the same value, but two names bound to the same enum object. > Then looking it up by value would not be a problem. If there were some way to identify the canonical name a lookup by value would be unambiguous. If we have iteration in definition order, I'd say the first defined name for a value should be the canonical name, and any other name for the value should be considered an alias. That would preclude the syntax above, but the following might be workable: class Insect(Enum): wasp = 1 bee = 2 ant = 3 # aliases wsap = wasp waps = 1 In the above, looking up by the value 1 would always return Insect.wasp. Tim Delaney -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sun Apr 21 16:06:11 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 22 Apr 2013 00:06:11 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> <20130412184449.6d1e9806@anarchist> <5168A30E.9000202@g.nevcal.com> <5168C83F.4050103@pearwood.info> <20130420154259.0ed44394@anarchist> <51734112.5020701@pearwood.info> Message-ID: <5173F253.5040201@pearwood.info> On 21/04/13 15:33, Nick Coghlan wrote: > The PEP is fine, as it already allows duplicate names without encouraging them: > > class Insect(Enum): > wasp = 1 # Preferred. Use this in new code. > bee = 2 > ant = 3 > # Backwards compatibility aliases > Insect.wsap = Insect.wasp Hmmm, I must have missed this. That satisfies my use-cases. Objection withdrawn. -- Steven From barry at python.org Sun Apr 21 23:47:45 2013 From: barry at python.org (Barry Warsaw) Date: Sun, 21 Apr 2013 17:47:45 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130420231049.E041E2500B3@webabinitio.net> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> Message-ID: <20130421174745.269913e5@anarchist> On Apr 20, 2013, at 07:10 PM, R. David Murray wrote: >It seems strange to limit a new Python3 feature to the Python2 feature >set. Just saying :) For a critical feature sure, but I don't put __repr__ or enum item iteration order in that category. There's no need for gratuitous incompatibility either, and attribute name order is just fine. -Barry From guido at python.org Sun Apr 21 23:50:17 2013 From: guido at python.org (Guido van Rossum) Date: Sun, 21 Apr 2013 14:50:17 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130421174745.269913e5@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> Message-ID: Fine with me. On Sun, Apr 21, 2013 at 2:47 PM, Barry Warsaw wrote: > On Apr 20, 2013, at 07:10 PM, R. David Murray wrote: > >>It seems strange to limit a new Python3 feature to the Python2 feature >>set. Just saying :) > > For a critical feature sure, but I don't put __repr__ or enum item iteration > order in that category. There's no need for gratuitous incompatibility > either, and attribute name order is just fine. > > -Barry > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) From barry at python.org Sun Apr 21 23:51:13 2013 From: barry at python.org (Barry Warsaw) Date: Sun, 21 Apr 2013 17:51:13 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> Message-ID: <20130421175113.6f2b79f1@anarchist> On Apr 21, 2013, at 03:25 PM, Nick Coghlan wrote: >Agreed. I think the stdlib enum library should use __prepare__ and >iterate in definition order (since 2.x compatibility isn't of any >concern), while flufl.enum can use "sorted by name" as the iteration >order. > >An "order_by_name" keyword argument to __prepare__ in the stdlib >version could then allow the user to opt in to the flufl.enum >behaviour, while still using definition order by default. Seriously, why all the extra complexity for something you'll never care about? You don't care about the exact contents of __repr__ as long as it's predictable. You don't really care about item iteration order either as long as it's defined. Sorting by attribute name fulfills both use cases *and* it's simple. Zen 3 FTW. -Barry From barry at python.org Sun Apr 21 23:52:32 2013 From: barry at python.org (Barry Warsaw) Date: Sun, 21 Apr 2013 17:52:32 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5173F253.5040201@pearwood.info> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412211702.D393D250BD2@webabinitio.net> <20130412184449.6d1e9806@anarchist> <5168A30E.9000202@g.nevcal.com> <5168C83F.4050103@pearwood.info> <20130420154259.0ed44394@anarchist> <51734112.5020701@pearwood.info> <5173F253.5040201@pearwood.info> Message-ID: <20130421175232.383b5e37@anarchist> On Apr 22, 2013, at 12:06 AM, Steven D'Aprano wrote: >On 21/04/13 15:33, Nick Coghlan wrote: > >> The PEP is fine, as it already allows duplicate names without encouraging >> them: >> >> class Insect(Enum): >> wasp = 1 # Preferred. Use this in new code. >> bee = 2 >> ant = 3 >> # Backwards compatibility aliases >> Insect.wsap = Insect.wasp > >Hmmm, I must have missed this. > >That satisfies my use-cases. Objection withdrawn. Yay! -Barry From ncoghlan at gmail.com Mon Apr 22 01:02:00 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 22 Apr 2013 09:02:00 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130421174745.269913e5@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> Message-ID: On 22 Apr 2013 07:50, "Barry Warsaw" wrote: > > On Apr 20, 2013, at 07:10 PM, R. David Murray wrote: > > >It seems strange to limit a new Python3 feature to the Python2 feature > >set. Just saying :) > > For a critical feature sure, but I don't put __repr__ or enum item iteration > order in that category. There's no need for gratuitous incompatibility > either, and attribute name order is just fine. Iteration order matters a lot if you don't want people complaining about enums being broken: class Days(enum.Enum): Monday = 1 Tuesday = 2 Wednesday = 3 Thursday = 4 Friday = 5 Saturday = 6 Sunday = 7 Cheers, Nick. > > -Barry > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From timothy.c.delaney at gmail.com Mon Apr 22 01:31:46 2013 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Mon, 22 Apr 2013 09:31:46 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> Message-ID: On 22 April 2013 09:02, Nick Coghlan wrote: > > On 22 Apr 2013 07:50, "Barry Warsaw" wrote: > > > > On Apr 20, 2013, at 07:10 PM, R. David Murray wrote: > > > > >It seems strange to limit a new Python3 feature to the Python2 feature > > >set. Just saying :) > > > > For a critical feature sure, but I don't put __repr__ or enum item > iteration > > order in that category. There's no need for gratuitous incompatibility > > either, and attribute name order is just fine. > > Iteration order matters a lot if you don't want people complaining about > enums being broken: > > class Days(enum.Enum): > Monday = 1 > Tuesday = 2 > Wednesday = 3 > Thursday = 4 > Friday = 5 > Saturday = 6 > Sunday = 7 > I'm fine with iteration order being by sorted name by default, so long as it's easily overrideable by enum subclasses or metaclasses e.g. an IntEnum should probably iterate in value order. For definition order, a 3.x-only metaclass could be provided: class Days(enum.Enum, metaclass=enum.DefinitionOrder): Monday = 1 Tuesday = 2 Wednesday = 3 Thursday = 4 Friday = 5 Saturday = 6 Sunday = 7 Tim Delaney -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Mon Apr 22 02:28:16 2013 From: barry at python.org (Barry Warsaw) Date: Sun, 21 Apr 2013 20:28:16 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> Message-ID: <20130421202816.3e22fc8f@anarchist> On Apr 22, 2013, at 09:02 AM, Nick Coghlan wrote: >Iteration order matters a lot if you don't want people complaining about >enums being broken: > > class Days(enum.Enum): > Monday = 1 > Tuesday = 2 > Wednesday = 3 > Thursday = 4 > Friday = 5 > Saturday = 6 > Sunday = 7 Sorry, that's still not a complete use case. I don't see why you'd depend on iteration order over Days for any particular functionality. Besides, if you did, I think it would be better to derive Days from IntEnum and then iteration order is guaranteed over the values. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Mon Apr 22 02:31:12 2013 From: barry at python.org (Barry Warsaw) Date: Sun, 21 Apr 2013 20:31:12 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> Message-ID: <20130421203112.018dd94a@anarchist> On Apr 22, 2013, at 09:31 AM, Tim Delaney wrote: >I'm fine with iteration order being by sorted name by default, so long as >it's easily overrideable by enum subclasses or metaclasses e.g. an IntEnum >should probably iterate in value order. It does. :) >For definition order, a 3.x-only metaclass could be provided: > >class Days(enum.Enum, metaclass=enum.DefinitionOrder): > Monday = 1 > Tuesday = 2 > Wednesday = 3 > Thursday = 4 > Friday = 5 > Saturday = 6 > Sunday = 7 Yep, that's how it works. From flufl.enum: class IntEnumMetaclass(EnumMetaclass): # Define an iteration over the integer values instead of the attribute # names. def __iter__(cls): for key in sorted(cls._enums): yield getattr(cls, cls._enums[key]) IntEnum = IntEnumMetaclass(str('IntEnum'), (Enum,), { '__doc__': 'A specialized enumeration with values that are also integers.', '__value_factory__': IntEnumValue, }) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From ncoghlan at gmail.com Mon Apr 22 02:42:04 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 22 Apr 2013 10:42:04 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130421202816.3e22fc8f@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> <20130421202816.3e22fc8f@anarchist> Message-ID: On Mon, Apr 22, 2013 at 10:28 AM, Barry Warsaw wrote: > On Apr 22, 2013, at 09:02 AM, Nick Coghlan wrote: > >>Iteration order matters a lot if you don't want people complaining about >>enums being broken: >> >> class Days(enum.Enum): >> Monday = 1 >> Tuesday = 2 >> Wednesday = 3 >> Thursday = 4 >> Friday = 5 >> Saturday = 6 >> Sunday = 7 > > Sorry, that's still not a complete use case. I don't see why you'd depend on > iteration order over Days for any particular functionality. You mean other than printing the days of the week in order without needing to worry about the specific values assigned to them? Using sort-by-name also introduces other weirdness, such as subclasses potentially inserting their values in the middle of inherited names, rather than appending to the end as one might reasonably expect. While using sort-by-name is better than not providing a consistent ordering at all, using definition order is substantially less surprising than sorting by key name, and PEP 3115 and collections.OrderedDict makes that easy to support in Python 3.x. The fact that this will make for a behavioural difference between the standard library and flufl.enum does *not* count as an argument for making the behaviour of the standard library version less intuitive (if that was a valid argument, the 3.3+ ipaddress module would look a *lot* more like it's ipaddr inspiration). > Besides, if you > did, I think it would be better to derive Days from IntEnum and then iteration > order is guaranteed over the values. But only at the cost of breaking the other guarantees provided by a standard enumeration. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From timothy.c.delaney at gmail.com Mon Apr 22 02:55:05 2013 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Mon, 22 Apr 2013 10:55:05 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130421203112.018dd94a@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> <20130421203112.018dd94a@anarchist> Message-ID: On 22 April 2013 10:31, Barry Warsaw wrote: > On Apr 22, 2013, at 09:31 AM, Tim Delaney wrote: > > >I'm fine with iteration order being by sorted name by default, so long as > >it's easily overrideable by enum subclasses or metaclasses e.g. an IntEnum > >should probably iterate in value order. > > It does. :) I knew it *did*, but wasn't sure if with the current discussion it was going to continue to do so. > >For definition order, a 3.x-only metaclass could be provided: > > > >class Days(enum.Enum, metaclass=enum.DefinitionOrder): > > Monday = 1 > > Tuesday = 2 > > Wednesday = 3 > > Thursday = 4 > > Friday = 5 > > Saturday = 6 > > Sunday = 7 > > Yep, that's how it works. From flufl.enum: > > class IntEnumMetaclass(EnumMetaclass): > # Define an iteration over the integer values instead of the attribute > # names. > def __iter__(cls): > for key in sorted(cls._enums): > yield getattr(cls, cls._enums[key]) > Would it be worthwhile storing a sorted version of the enum keys here? Or do you think the current space vs speed tradeoff is better? I need to grab the current flufl.enum code and see if I can easily extend it to do some more esoteric things that my enum implementation supports (*not* bare names, but maybe the name = ... syntax, which of course requires the definition order metaclass). I'm in the middle of a release cycle, so my time is somewhat limited right now :( Tim Delaney -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdmurray at bitdance.com Mon Apr 22 03:12:05 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Sun, 21 Apr 2013 21:12:05 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130421202816.3e22fc8f@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> <20130421202816.3e22fc8f@anarchist> Message-ID: <20130422011206.235C7250066@webabinitio.net> On Sun, 21 Apr 2013 20:28:16 -0400, Barry Warsaw wrote: > On Apr 22, 2013, at 09:02 AM, Nick Coghlan wrote: > > >Iteration order matters a lot if you don't want people complaining about > >enums being broken: > > > > class Days(enum.Enum): > > Monday = 1 > > Tuesday = 2 > > Wednesday = 3 > > Thursday = 4 > > Friday = 5 > > Saturday = 6 > > Sunday = 7 > > Sorry, that's still not a complete use case. I don't see why you'd depend on > iteration order over Days for any particular functionality. Besides, if you > did, I think it would be better to derive Days from IntEnum and then iteration > order is guaranteed over the values. I didn't read Nick's message as being about a use case. The key word in the sentence above was "complaining" :) Regardless of the specific values involved, it is pretty much guaranteed that if anything other than definition order is used we *will* get bug reports/enhancement requests to fix it, on a regular basis. We can choose to live with that, but we should admit that it will will happen :) --David From guido at python.org Mon Apr 22 04:58:53 2013 From: guido at python.org (Guido van Rossum) Date: Sun, 21 Apr 2013 19:58:53 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> <20130421203112.018dd94a@anarchist> Message-ID: On Sun, Apr 21, 2013 at 5:55 PM, Tim Delaney wrote: > On 22 April 2013 10:31, Barry Warsaw wrote: >> >> On Apr 22, 2013, at 09:31 AM, Tim Delaney wrote: >> >> >I'm fine with iteration order being by sorted name by default, so long as >> >it's easily overrideable by enum subclasses or metaclasses e.g. an >> > IntEnum >> >should probably iterate in value order. >> >> It does. :) > > I knew it *did*, but wasn't sure if with the current discussion it was going > to continue to do so. In any case I think it would be odd if IntEnum used a different policy than Enum. That would be a disturbing difference in behavior between two otherwise rather similar classes. > Regardless of the specific values involved, it is pretty much guaranteed > that if anything other than definition order is used we *will* get bug > reports/enhancement requests to fix it, on a regular basis. We can choose > to live with that, but we should admit that it will will happen :) I'm convinced. I also think that 2/3 compatibility is not as important as getting it right for the foreseeable future. -- --Guido van Rossum (python.org/~guido) From ram at rachum.com Mon Apr 22 13:29:23 2013 From: ram at rachum.com (Ram Rachum) Date: Mon, 22 Apr 2013 14:29:23 +0300 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? Message-ID: Hi everyone, Take a look at this question: http://stackoverflow.com/questions/16122435/python-3-how-do-i-use-bytes-to-bytes-and-string-to-string-encodings/16122472?noredirect=1#comment23034787_16122472 Is there really no way to use base64 that's as short as: b'whatever'.encode('base64') Because doing this: import codecs codecs.decode(b"whatever", "base64_codec") Or this: import base64 encoded = base64.b64encode(b'whatever') Is cumbersome! Why can't I do something like b'whatever'.encode('base64')? Or maybe using a different method than `encode`? Thanks, Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From victor.stinner at gmail.com Mon Apr 22 13:38:47 2013 From: victor.stinner at gmail.com (Victor Stinner) Date: Mon, 22 Apr 2013 13:38:47 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: Message-ID: Hi, Your question is discussed since 4 years in the following issue: http://bugs.python.org/issue7475 The last proposition is to add transform() and untransform() methods to bytes and str types. But nobody implemented the idea. If I remember correctly, the missing point is how to define which types are supported by a codec (ex: only bytes for bz2 codec, bytes and str for rot13). Victor 2013/4/22 Ram Rachum : > Hi everyone, > > Take a look at this question: > > http://stackoverflow.com/questions/16122435/python-3-how-do-i-use-bytes-to-bytes-and-string-to-string-encodings/16122472?noredirect=1#comment23034787_16122472 > > Is there really no way to use base64 that's as short as: > > b'whatever'.encode('base64') > > Because doing this: > > import codecs > codecs.decode(b"whatever", "base64_codec") > > Or this: > > import base64 > encoded = base64.b64encode(b'whatever') > > Is cumbersome! > > Why can't I do something like b'whatever'.encode('base64')? Or maybe using a > different method than `encode`? > > > Thanks, > Ram. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com > From ironfroggy at gmail.com Mon Apr 22 13:39:25 2013 From: ironfroggy at gmail.com (Calvin Spealman) Date: Mon, 22 Apr 2013 07:39:25 -0400 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: Message-ID: if two lines is cumbersome, you're in for a cumbersome life a programmer. On Apr 22, 2013 7:31 AM, "Ram Rachum" wrote: > Hi everyone, > > Take a look at this question: > > > http://stackoverflow.com/questions/16122435/python-3-how-do-i-use-bytes-to-bytes-and-string-to-string-encodings/16122472?noredirect=1#comment23034787_16122472 > > Is there really no way to use base64 that's as short as: > > b'whatever'.encode('base64') > > Because doing this: > > import codecs > codecs.decode(b"whatever", "base64_codec") > > Or this: > > import base64 > encoded = base64.b64encode(b'whatever') > > Is cumbersome! > > Why can't I do something like b'whatever'.encode('base64')? Or maybe using > a different method than `encode`? > > > Thanks, > Ram. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Mon Apr 22 14:04:40 2013 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 22 Apr 2013 13:04:40 +0100 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: Message-ID: On 22 April 2013 12:39, Calvin Spealman wrote: > if two lines is cumbersome, you're in for a cumbersome life a programmer. > One of which is essentially Python's equivalent of a declaration... Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeanpierreda at gmail.com Mon Apr 22 15:50:14 2013 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Mon, 22 Apr 2013 09:50:14 -0400 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: Message-ID: On Mon, Apr 22, 2013 at 7:39 AM, Calvin Spealman wrote: > if two lines is cumbersome, you're in for a cumbersome life a programmer. Other encodings are either missing completely from the stdlib, or have corrupted behavior. For example, string_escape is gone, and unicode_escape doesn't make any sense anymore -- python code is text, not bytes, so why does 'abc'.encode('unicode_escape') return bytes? I don't think this change was thought through completely before it was implemented. I agree base64 is a bad place to pick at the encode/decode changes, though. :( -- Devin From phd at phdru.name Mon Apr 22 16:25:50 2013 From: phd at phdru.name (Oleg Broytman) Date: Mon, 22 Apr 2013 18:25:50 +0400 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: Message-ID: <20130422142550.GA19204@iskra.aviel.ru> On Mon, Apr 22, 2013 at 09:50:14AM -0400, Devin Jeanpierre wrote: > unicode_escape doesn't make any sense anymore -- python code is text, > not bytes, so why does 'abc'.encode('unicode_escape') return bytes? AFAIU the situation is simple: unicode.encode(encoding) returns bytes, bytes.decode(encoding) returns unicode, and neither unicode.decode() nor bytes.encode() exist. Transformations like base64 and bz2 are nor encoding/decoding -- they are bytes/bytes or unicode/unicode transformations. Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From stephen at xemacs.org Mon Apr 22 17:01:38 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 23 Apr 2013 00:01:38 +0900 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: Message-ID: <87obd6zk19.fsf@uwakimon.sk.tsukuba.ac.jp> Devin Jeanpierre writes: > why does 'abc'.encode('unicode_escape') return bytes? Duck-typing: encode always turns unicode into bytes. From rdmurray at bitdance.com Mon Apr 22 17:31:02 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 22 Apr 2013 11:31:02 -0400 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: Message-ID: <20130422153102.BB66F250BCA@webabinitio.net> On Mon, 22 Apr 2013 09:50:14 -0400, Devin Jeanpierre wrote: > On Mon, Apr 22, 2013 at 7:39 AM, Calvin Spealman wrote: > > if two lines is cumbersome, you're in for a cumbersome life a programmer. > > Other encodings are either missing completely from the stdlib, or have > corrupted behavior. For example, string_escape is gone, and > unicode_escape doesn't make any sense anymore -- python code is text, > not bytes, so why does 'abc'.encode('unicode_escape') return bytes? I > don't think this change was thought through completely before it was > implemented. We use unicode_escape (actually raw_unicode_escape) in the email package, and there we are converting between string and bytes. It is used as an encoder when we are supposed to have ASCII input but have other stuff, and need ASCII output and don't want to lose information. So yes, that encoder does still make sense. It would also be useful as a transform function, but as someone has pointed out there's an issue for that. --David From storchaka at gmail.com Mon Apr 22 19:13:49 2013 From: storchaka at gmail.com (Serhiy Storchaka) Date: Mon, 22 Apr 2013 20:13:49 +0300 Subject: [Python-Dev] cpython: Simplify the code of get_attrib_from_keywords somewhat. In-Reply-To: <3ZvSPN4jzLz7LlX@mail.python.org> References: <3ZvSPN4jzLz7LlX@mail.python.org> Message-ID: On 22.04.13 15:52, eli.bendersky wrote: > http://hg.python.org/cpython/rev/c9674421d78e > changeset: 83494:c9674421d78e > user: Eli Bendersky > date: Mon Apr 22 05:52:16 2013 -0700 > summary: > Simplify the code of get_attrib_from_keywords somewhat. > - PyDict_DelItem(kwds, attrib_str); > + PyDict_DelItemString(kwds, ATTRIB_KEY); PyDict_GetItemString() and PyDict_DelItemString() internally create a Python string. I.e. new code creates one additional string if attrib was found in kwds. > - if (attrib) > - PyDict_Update(attrib, kwds); > + assert(attrib); attrib can be NULL in case of memory allocation error. From eliben at gmail.com Mon Apr 22 20:03:01 2013 From: eliben at gmail.com (Eli Bendersky) Date: Mon, 22 Apr 2013 11:03:01 -0700 Subject: [Python-Dev] cpython: Simplify the code of get_attrib_from_keywords somewhat. In-Reply-To: References: <3ZvSPN4jzLz7LlX@mail.python.org> Message-ID: On Mon, Apr 22, 2013 at 10:13 AM, Serhiy Storchaka wrote: > On 22.04.13 15:52, eli.bendersky wrote: > >> http://hg.python.org/cpython/**rev/c9674421d78e >> changeset: 83494:c9674421d78e >> user: Eli Bendersky >> date: Mon Apr 22 05:52:16 2013 -0700 >> summary: >> Simplify the code of get_attrib_from_keywords somewhat. >> > > - PyDict_DelItem(kwds, attrib_str); >> + PyDict_DelItemString(kwds, ATTRIB_KEY); >> > > PyDict_GetItemString() and PyDict_DelItemString() internally create a > Python string. I.e. new code creates one additional string if attrib was > found in kwds. > > > - if (attrib) >> - PyDict_Update(attrib, kwds); >> + assert(attrib); >> > > attrib can be NULL in case of memory allocation error. > Thanks, I'll review this Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Tue Apr 23 01:16:20 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 23 Apr 2013 11:16:20 +1200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: Message-ID: <5175C4C4.503@canterbury.ac.nz> Victor Stinner wrote: > The last proposition is to add transform() and untransform() methods > to bytes and str types. ... If I remember > correctly, the missing point is how to define which types are > supported by a codec Also, for any given codec, which direction is "transform" and which is "untransform"? Also also, what's so special about base64 et al that they deserve an ultra-special way of invoking them, instead of having to import a class or function like you do for *every* *other* piece of library functionality? -- Greg From rdmurray at bitdance.com Tue Apr 23 03:07:49 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 22 Apr 2013 21:07:49 -0400 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <5175C4C4.503@canterbury.ac.nz> References: <5175C4C4.503@canterbury.ac.nz> Message-ID: <20130423010750.0C88E250BCA@webabinitio.net> On Tue, 23 Apr 2013 11:16:20 +1200, Greg Ewing wrote: > Victor Stinner wrote: > > The last proposition is to add transform() and untransform() methods > > to bytes and str types. ... If I remember > > correctly, the missing point is how to define which types are > > supported by a codec > > Also, for any given codec, which direction is "transform" > and which is "untransform"? You transform *into* the encoding, and untransform *out* of the encoding. Do you have an example where that would be ambiguous? > Also also, what's so special about base64 et al that they > deserve an ultra-special way of invoking them, instead of > having to import a class or function like you do for > *every* *other* piece of library functionality? You can ask the same question about all the other codecs. (And that question has indeed been asked in the past.) (One answer is that they used to work in Python2...but the longer we go without restoring the functionality to Python3, the weaker that particular argument becomes.) --David From guido at python.org Tue Apr 23 03:22:51 2013 From: guido at python.org (Guido van Rossum) Date: Mon, 22 Apr 2013 18:22:51 -0700 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <20130423010750.0C88E250BCA@webabinitio.net> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> Message-ID: --Guido van Rossum (sent from Android phone) On Apr 22, 2013 6:09 PM, "R. David Murray" wrote: > > On Tue, 23 Apr 2013 11:16:20 +1200, Greg Ewing < greg.ewing at canterbury.ac.nz> wrote: > > Victor Stinner wrote: > > > The last proposition is to add transform() and untransform() methods > > > to bytes and str types. ... If I remember > > > correctly, the missing point is how to define which types are > > > supported by a codec > > > > Also, for any given codec, which direction is "transform" > > and which is "untransform"? > > You transform *into* the encoding, and untransform *out* of the encoding. > Do you have an example where that would be ambiguous? > > > Also also, what's so special about base64 et al that they > > deserve an ultra-special way of invoking them, instead of > > having to import a class or function like you do for > > *every* *other* piece of library functionality? > > You can ask the same question about all the other codecs. (And that > question has indeed been asked in the past.) Except for rot13. :-) > (One answer is that they used to work in Python2...but the longer we go > without restoring the functionality to Python3, the weaker that particular > argument becomes.) > > --David > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Apr 23 04:04:45 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 23 Apr 2013 12:04:45 +1000 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <5175C4C4.503@canterbury.ac.nz> References: <5175C4C4.503@canterbury.ac.nz> Message-ID: <5175EC3D.4090007@pearwood.info> On 23/04/13 09:16, Greg Ewing wrote: > Victor Stinner wrote: >> The last proposition is to add transform() and untransform() methods >> to bytes and str types. ... If I remember >> correctly, the missing point is how to define which types are >> supported by a codec > > Also, for any given codec, which direction is "transform" > and which is "untransform"? > > Also also, what's so special about base64 et al that they > deserve an ultra-special way of invoking them, instead of > having to import a class or function like you do for > *every* *other* piece of library functionality? As others have pointed out in the past, repeatedly, the codec system is completely general and can transform bytes->bytes and text->text just as easily as bytes<->text. Or indeed any bijection, as the docs for 2.7 point out. The question isn't "What's so special about base64?" The questions should be: - What's so special about exotic legacy transformations like ISO-8859-10 and MacRoman that they deserve a string method for invoking them? - Why have common transformations like base64, which worked in 2.x, been relegated to second-class status in 3.x? - If it is no burden to have to import a module and call an external function for some transformations, why have encode and decode methods at all? If you haven't read this, you should: http://lucumr.pocoo.org/2012/8/11/codec-confusion/ -- Steven From donald at stufft.io Tue Apr 23 04:30:20 2013 From: donald at stufft.io (Donald Stufft) Date: Mon, 22 Apr 2013 22:30:20 -0400 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <5175EC3D.4090007@pearwood.info> References: <5175C4C4.503@canterbury.ac.nz> <5175EC3D.4090007@pearwood.info> Message-ID: On Apr 22, 2013, at 10:04 PM, Steven D'Aprano wrote: > On 23/04/13 09:16, Greg Ewing wrote: >> Victor Stinner wrote: >>> The last proposition is to add transform() and untransform() methods >>> to bytes and str types. ... If I remember >>> correctly, the missing point is how to define which types are >>> supported by a codec >> >> Also, for any given codec, which direction is "transform" >> and which is "untransform"? >> >> Also also, what's so special about base64 et al that they >> deserve an ultra-special way of invoking them, instead of >> having to import a class or function like you do for >> *every* *other* piece of library functionality? > > > As others have pointed out in the past, repeatedly, the codec system is completely general and can transform bytes->bytes and text->text just as easily as bytes<->text. Or indeed any bijection, as the docs for 2.7 point out. The question isn't "What's so special about base64?" The questions should be: > > - What's so special about exotic legacy transformations like ISO-8859-10 and MacRoman that they deserve a string method for invoking them? > > - Why have common transformations like base64, which worked in 2.x, been relegated to second-class status in 3.x? > > - If it is no burden to have to import a module and call an external function for some transformations, why have encode and decode methods at all? > > > If you haven't read this, you should: > > http://lucumr.pocoo.org/2012/8/11/codec-confusion/ I may be dull, but it wasn't until I started using Python 3 that it really clicked in my head what encode/decode did exactly. In Python2 I just sort of sprinkled one or the other when there was errors until the pain stopped. I mostly attribute this to str.decode and bytes.encode not existing. > > > > > -- > Steven > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/donald%40stufft.io ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 841 bytes Desc: Message signed with OpenPGP using GPGMail URL: From guido at python.org Tue Apr 23 05:16:58 2013 From: guido at python.org (Guido van Rossum) Date: Mon, 22 Apr 2013 20:16:58 -0700 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <5175EC3D.4090007@pearwood.info> References: <5175C4C4.503@canterbury.ac.nz> <5175EC3D.4090007@pearwood.info> Message-ID: On Mon, Apr 22, 2013 at 7:04 PM, Steven D'Aprano wrote: > As others have pointed out in the past, repeatedly, the codec system is > completely general and can transform bytes->bytes and text->text just as > easily as bytes<->text. Or indeed any bijection, as the docs for 2.7 point > out. The question isn't "What's so special about base64?" The questions > should be: > > - What's so special about exotic legacy transformations like ISO-8859-10 and > MacRoman that they deserve a string method for invoking them? > > - Why have common transformations like base64, which worked in 2.x, been > relegated to second-class status in 3.x? > > - If it is no burden to have to import a module and call an external > function for some transformations, why have encode and decode methods at > all? There are good answers to all of these, and your rhetoric is not appreciated. The special status is for the translation between bytes and Unicode characters (code points). There are many contexts where a byte stream is labeled (either separately or in-line) as being encoded using some specific encoding. -- --Guido van Rossum (python.org/~guido) From regebro at gmail.com Tue Apr 23 06:48:04 2013 From: regebro at gmail.com (Lennart Regebro) Date: Tue, 23 Apr 2013 06:48:04 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <5175EC3D.4090007@pearwood.info> References: <5175C4C4.503@canterbury.ac.nz> <5175EC3D.4090007@pearwood.info> Message-ID: On Tue, Apr 23, 2013 at 4:04 AM, Steven D'Aprano wrote: > As others have pointed out in the past, repeatedly, the codec system is > completely general and can transform bytes->bytes and text->text just as > easily as bytes<->text. Yes, but the encode()/decode() methods are not, and the fact that you now know what goes in and what comes out means that people get much fewer Decode/EncodeErrors. Which is a good thing. //Lennart From fabiosantosart at gmail.com Tue Apr 23 07:53:04 2013 From: fabiosantosart at gmail.com (=?ISO-8859-1?Q?F=E1bio_Santos?=) Date: Tue, 23 Apr 2013 06:53:04 +0100 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: <5175C4C4.503@canterbury.ac.nz> <5175EC3D.4090007@pearwood.info> Message-ID: Using decode() and encode() would break that predictability. But someone suggested the use of transform() and untransform() instead. That would clarify that the transformation is bytes > bytes and Unicode string > Unicode string. On 23 Apr 2013 05:50, "Lennart Regebro" wrote: > On Tue, Apr 23, 2013 at 4:04 AM, Steven D'Aprano > wrote: > > As others have pointed out in the past, repeatedly, the codec system is > > completely general and can transform bytes->bytes and text->text just as > > easily as bytes<->text. > > Yes, but the encode()/decode() methods are not, and the fact that you > now know what goes in and what comes out means that people get much > fewer Decode/EncodeErrors. Which is a good thing. > > //Lennart > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fabiosantosart%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Tue Apr 23 08:13:53 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 23 Apr 2013 18:13:53 +1200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <5175EC3D.4090007@pearwood.info> References: <5175C4C4.503@canterbury.ac.nz> <5175EC3D.4090007@pearwood.info> Message-ID: <517626A1.8040501@canterbury.ac.nz> Steven D'Aprano wrote: > - If it is no burden to have to import a module and call an external > function for some transformations, why have encode and decode methods at > all? Now that all text strings are unicode, the unicode codecs are in a sense special, in that you can't do any string I/O at all without using them at some stage. So arguably it makes sense to have a very easy way of invoking them. I suspect that without this, the idea of all strings being unicode would have been even harder to sell than it was. -- Greg From emmanuel.bacry at polytechnique.fr Tue Apr 23 13:18:55 2013 From: emmanuel.bacry at polytechnique.fr (Emmanuel Bacry) Date: Tue, 23 Apr 2013 13:18:55 +0200 Subject: [Python-Dev] distutils, win32, multiple C extensions Message-ID: <2AADBD5C-4CE8-4A34-9B07-4F5C583C07A0@polytechnique.fr> Hello, I am a researcher in Applied Math. For my work I write a lot of code and right now I am moving to python. I am writing some C extensions (using swig). I wrote a first extension (a python module) which I'll call E1. I wrote a setup.py which works fine on all platforms. On Windows 7 it creates a library which is a .pyd file Now I want to write a second C extension E2 that is calling some functions of E1. The same kind of setup.py works fine on all platforms but Windows where it says (during the link) that the function sof E1 that are called by E2 are unreferenced. I am clearly not a Windows pro ... I am using mingw32 on Windows. Surfing the web, I understood that shared libraries work differently on windows than on other platforms. And that I should 1- export the symbols (i.e., the functions) of E1 when I build E1. In the distutils.Extension this seems to be done using argument export_symbols=[list of symbols to export] Now I have one problem : each time I specify in this list the name of a function in E1 it says that it doesn't know this symbol What am I doing wrong here, is there a special syntax ? 2- Now in the setup.py of E2 how do I declare that I want to use the E1 shared lib.? Is the information on the exported symbol of E1 in the .pyd file or in the .def file ? I tried to specify the .pyd library in the arguments of the distutils.Extension .... but I have the feeling that's not the right thing to do (I have the feeling the .pyd is not the right file to point to) Not mentioning the problem specifying the path of the library with runtime_library_dirs ... which does not seem to work at all. Is there a simple way to do what I want to do ? I did not find anything on the web explaining clearly what to do ... I am really really stuck I would appreciate any kind of help Emmanuel From stephen at xemacs.org Tue Apr 23 15:29:33 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 23 Apr 2013 22:29:33 +0900 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <20130423010750.0C88E250BCA@webabinitio.net> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> Message-ID: <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> R. David Murray writes: > You transform *into* the encoding, and untransform *out* of the > encoding. Do you have an example where that would be ambiguous? In the bytes-to-bytes case, any pair of character encodings (eg, UTF-8 and ISO-8859-15) would do. Or how about in text, ReST to HTML? BASE64 itself is ambiguous. By RFC specification, BASE64 is a *textual* representation of arbitrary binary data. (Cf. URIs.) The natural interpretation of .encode('base64') in that context would be as a bytes-to-text encoder. However, this has several problems. In practice, we invariably use an ASCII octet stream to carry BASE64- encoded data. So web developers would almost certainly expect a bytes-to-bytes encoder. Such a bytes-to-bytes encoder can't be duck-typed. Double-encoding bugs wouldn't be detected until the stream arrives at the user. And the RFC-based signature of .encode('base64') as bytes-to-text is precisely opposite to that of .encode('utf-8') (text-to-bytes). It is certainly true that there are many unambiguous cases. In the case of a true text processing facility (eg, Emacs buffers or Python 3 str) where there is an unambiguous text type with a constant and opaque internal representation, it makes a lot of sense to treat the text type as special/central, and use the terminology "encode [from text]" and "decode [to text]". It's easy to remember, which one is special is obvious, and the difference in input and output types means that mistaken use of the API will be detected by duck-typing. However, in the case of bytes-bytes or text-text transformations, it's not the presence of unambiguous cases that should drive API design IMO. It's the presence of the ambiguous cases that we should cater to. I don't see easy solutions to this issue. Steve From curt at hagenlocher.org Tue Apr 23 15:30:48 2013 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Tue, 23 Apr 2013 06:30:48 -0700 Subject: [Python-Dev] distutils, win32, multiple C extensions In-Reply-To: <2AADBD5C-4CE8-4A34-9B07-4F5C583C07A0@polytechnique.fr> References: <2AADBD5C-4CE8-4A34-9B07-4F5C583C07A0@polytechnique.fr> Message-ID: Python-List (http://mail.python.org/mailman/listinfo/python-list) is the better place for this kind of question; Python-Dev is for the development of Python itself, not for development using Python. When you built E1, it should have also built a ".lib" file in addition to the ".pyd". It's the .lib that needs to be referenced when building E2. If it did not, you may find this link helpful: http://www.mingw.org/wiki/CreateImportLibraries On Tue, Apr 23, 2013 at 4:18 AM, Emmanuel Bacry < emmanuel.bacry at polytechnique.fr> wrote: > Hello, > > I am a researcher in Applied Math. > For my work I write a lot of code and right now I am moving to python. > > I am writing some C extensions (using swig). > I wrote a first extension (a python module) which I'll call E1. > I wrote a setup.py which works fine on all platforms. > On Windows 7 it creates a library which is a .pyd file > > Now I want to write a second C extension E2 that is calling some functions > of E1. > The same kind of setup.py works fine on all platforms but Windows where it > says (during the link) that the function sof E1 that are called by E2 are > unreferenced. > I am clearly not a Windows pro ... I am using mingw32 on Windows. > Surfing the web, I understood that shared libraries work differently on > windows than on other platforms. And that I should > > 1- export the symbols (i.e., the functions) of E1 when I build E1. In the > distutils.Extension this seems to be done using > argument export_symbols=[list of symbols to export] > Now I have one problem : each time I specify in this list the name of a > function in E1 it says that it doesn't know this symbol > What am I doing wrong here, is there a special syntax ? > > 2- Now in the setup.py of E2 how do I declare that I want to use the E1 > shared lib.? Is the information on the exported symbol of E1 in the .pyd > file or in the .def file ? > I tried to specify the .pyd library in the arguments of the > distutils.Extension .... but I have the feeling that's not the right thing > to do (I have the feeling the .pyd is not the right file to point to) > Not mentioning the problem specifying the path of the library with > runtime_library_dirs ... which does not seem to work at all. > > Is there a simple way to do what I want to do ? > I did not find anything on the web explaining clearly what to do ... > I am really really stuck > > I would appreciate any kind of help > > > Emmanuel > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/curt%40hagenlocher.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Tue Apr 23 15:42:40 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 23 Apr 2013 15:42:40 +0200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library References: Message-ID: <20130423154240.243dc295@pitrou.net> Hello, Le Fri, 12 Apr 2013 05:55:00 -0700, Eli Bendersky a ?crit : > > We're happy to present the revised PEP 435, collecting valuable > feedback from python-ideas discussions as well as in-person > discussions and decisions made during the latest PyCon language > summit. We believe the proposal is now better than the original one, > providing both a wider set of features and more convenient ways to > use those features. I'm having a problem with the proposed implementation. I haven't found any mention of it, so apologies if this has already been discussed: > > Link to the PEP: http://www.python.org/dev/peps/pep-0435/ [it's also > pasted fully below for convenience]. > > Reference implementation is available as the recently released > flufl.enum version 4.0 - you can get it either from PyPi or > https://launchpad.net/flufl.enum. flufl.enum 4.0 was developed in > parallel with revising PEP 435. > > Comments welcome, > > Barry and Eli > > ---------------------------------- > > PEP: 435 > Title: Adding an Enum type to the Python standard library > Version: $Revision$ > Last-Modified: $Date$ > Author: Barry Warsaw , > Eli Bendersky > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 2013-02-23 > Python-Version: 3.4 > Post-History: 2013-02-23 > > > Abstract > ======== > > This PEP proposes adding an enumeration type to the Python standard > library. Specifically, it proposes moving the existing ``flufl.enum`` > package by Barry > Warsaw into the standard library. Much of this PEP is based on the > "using" [1]_ document from the documentation of ``flufl.enum``. > > An enumeration is a set of symbolic names bound to unique, constant > values. Within an enumeration, the values can be compared by > identity, and the enumeration itself can be iterated over. > > > Decision > ======== > > TODO: update decision here once pronouncement is made. > > > Status of discussions > ===================== > > The idea of adding an enum type to Python is not new - PEP 354 [2]_ > is a previous attempt that was rejected in 2005. Recently a new set > of discussions > was initiated [3]_ on the ``python-ideas`` mailing list. Many new > ideas were > proposed in several threads; after a lengthy discussion Guido proposed > adding > ``flufl.enum`` to the standard library [4]_. During the PyCon 2013 > language summit the issue was discussed further. It became clear > that many developers > want to see an enum that subclasses ``int``, which can allow us to > replace many integer constants in the standard library by enums with > friendly string representations, without ceding backwards > compatibility. An additional discussion among several interested > core developers led to the proposal of having ``IntEnum`` as a > special case of ``Enum``. > > The key dividing issue between ``Enum`` and ``IntEnum`` is whether > comparing to integers is semantically meaningful. For most uses of > enumerations, it's a **feature** to reject comparison to integers; > enums that compare to integers > lead, through transitivity, to comparisons between enums of unrelated > types, which isn't desirable in most cases. For some uses, however, > greater interoperatiliby with integers is desired. For instance, this > is the case for > replacing existing standard library constants (such as > ``socket.AF_INET``) with enumerations. > > This PEP is an attempt to formalize this decision as well as discuss a > number > of variations that were discussed and can be considered for inclusion. > > > Motivation > ========== > > *[Based partly on the Motivation stated in PEP 354]* > > The properties of an enumeration are useful for defining an immutable, > related > set of constant values that have a defined sequence but no inherent > semantic meaning. Classic examples are days of the week (Sunday > through Saturday) and > school assessment grades ('A' through 'D', and 'F'). Other examples > include error status values and states within a defined process. > > It is possible to simply define a sequence of values of some other > basic type, > such as ``int`` or ``str``, to represent discrete arbitrary values. > However, > an enumeration ensures that such values are distinct from any others > including, > importantly, values within other enumerations, and that operations > without meaning ("Wednesday times two") are not defined for these > values. It also provides a convenient printable representation of > enum values without requiring > tedious repetition while defining them (i.e. no ``GREEN = 'green'``). > > > Module and type name > ==================== > > We propose to add a module named ``enum`` to the standard library. > The main type exposed by this module is ``Enum``. Hence, to import > the ``Enum`` type user code will run:: > > >>> from enum import Enum > > > Proposed semantics for the new enumeration type > =============================================== > > Creating an Enum > ---------------- > > Enumerations are created using the class syntax, which makes them > easy to read > and write. An alternative creation method is described in > `Convenience API`_. > To define an enumeration, derive from the ``Enum`` class and add > attributes with assignment to their integer values:: > > >>> from enum import Enum > >>> class Colors(Enum): > ... red = 1 > ... green = 2 > ... blue = 3 > > Enumeration values have nice, human readable string representations:: > > >>> print(Colors.red) > Colors.red > > ...while their repr has more information:: > > >>> print(repr(Colors.red)) > > > The enumeration value names are available through the class members:: > > >>> for member in Colors.__members__: > ... print(member) > red > green > blue > > Let's say you wanted to encode an enumeration value in a database. > You might > want to get the enumeration class object from an enumeration value:: > > >>> cls = Colors.red.enum > >>> print(cls.__name__) > Colors > > Enums also have a property that contains just their item name:: > > >>> print(Colors.red.name) > red > >>> print(Colors.green.name) > green > >>> print(Colors.blue.name) > blue > > The str and repr of the enumeration class also provides useful > information:: > > >>> print(Colors) > > >>> print(repr(Colors)) > > > The ``Enum`` class supports iteration. Iteration is defined as the > sorted order of the item values:: > > >>> class FiveColors(Enum): > ... pink = 4 > ... cyan = 5 > ... green = 2 > ... blue = 3 > ... red = 1 > >>> [v.name for v in FiveColors] > ['red', 'green', 'blue', 'pink', 'cyan'] > > Enumeration values are hashable, so they can be used in dictionaries > and sets:: > > >>> apples = {} > >>> apples[Colors.red] = 'red delicious' > >>> apples[Colors.green] = 'granny smith' > >>> apples > {: 'granny smith', Colors.red [value=1]>: 'red delicious'} > > To programmatically access enumeration values, use ``getattr``:: > > >>> getattr(Colors, 'red') > > > Comparisons > ----------- > > Enumeration values are compared by identity:: > > >>> Colors.red is Colors.red > True > >>> Colors.blue is Colors.blue > True > >>> Colors.red is not Colors.blue > True > >>> Colors.blue is Colors.red > False > > Ordered comparisons between enumeration values are *not* supported. > Enums are > not integers (but see `IntEnum`_ below):: > > >>> Colors.red < Colors.blue > Traceback (most recent call last): > ... > NotImplementedError > >>> Colors.red <= Colors.blue > Traceback (most recent call last): > ... > NotImplementedError > >>> Colors.blue > Colors.green > Traceback (most recent call last): > ... > NotImplementedError > >>> Colors.blue >= Colors.green > Traceback (most recent call last): > ... > NotImplementedError > > Equality comparisons are defined though:: > > >>> Colors.blue == Colors.blue > True > >>> Colors.green != Colors.blue > True > > Comparisons against non-enumeration values will always compare not > equal:: > > >>> Colors.green == 2 > False > >>> Colors.blue == 3 > False > >>> Colors.green != 3 > True > >>> Colors.green == 'green' > False > > > Extending enumerations by subclassing > ------------------------------------- > > You can extend previously defined Enums by subclassing:: > > >>> class MoreColors(Colors): > ... pink = 4 > ... cyan = 5 > > When extended in this way, the base enumeration's values are > identical to the > same named values in the derived class:: > > >>> Colors.red is MoreColors.red > True > >>> Colors.blue is MoreColors.blue > True > > However, these are not doing comparisons against the integer > equivalent values, because if you define an enumeration with similar > item names and integer values, they will not be identical:: > > >>> class OtherColors(Enum): > ... red = 1 > ... blue = 2 > ... yellow = 3 > >>> Colors.red is OtherColors.red > False > >>> Colors.blue is not OtherColors.blue > True > > These enumeration values are not equal, nor do they and hence may > exist in the same set, or as distinct keys in the same dictionary:: > > >>> Colors.red == OtherColors.red > False > >>> len(set((Colors.red, OtherColors.red))) > 2 > > You may not define two enumeration values with the same integer > value:: > > >>> class Bad(Enum): > ... cartman = 1 > ... stan = 2 > ... kyle = 3 > ... kenny = 3 # Oops! > ... butters = 4 > Traceback (most recent call last): > ... > ValueError: Conflicting enums with value '3': 'kenny' and 'kyle' > > You also may not duplicate values in derived enumerations:: > > >>> class BadColors(Colors): > ... yellow = 4 > ... chartreuse = 2 # Oops! > Traceback (most recent call last): > ... > ValueError: Conflicting enums with value '2': 'green' and > 'chartreuse' > > > Enumeration values > ------------------ > > The examples above use integers for enumeration values. Using > integers is short and handy (and provided by default by the > `Convenience API`_), but not strictly enforced. In the vast majority > of use-cases, one doesn't care what the actual value of an > enumeration is. But if the value *is* important, enumerations can > have arbitrary values. The following example uses strings:: > > >>> class SpecialId(Enum): > ... selector = '$IM($N)' > ... adaptor = '~$IM' > ... > >>> SpecialId.selector > > >>> SpecialId.selector.value > '$IM($N)' > >>> a = SpecialId.adaptor > >>> a == '~$IM' > False > >>> a == SpecialId.adaptor > True > >>> print(a) > SpecialId.adaptor > >>> print(a.value) > ~$IM > > Here ``Enum`` is used to provide readable (and syntactically valid!) > names for > some special values, as well as group them together. > > While ``Enum`` supports this flexibility, one should only use it in > very special cases. Code will be most readable when actual values of > enumerations aren't important and enumerations are just used for their > naming and comparison properties. > > > IntEnum > ------- > > A variation of ``Enum`` is proposed where the enumeration values also > subclasses ``int`` - ``IntEnum``. These values can be compared to > integers; by extension, enumerations of different types can also be > compared to each other:: > > >>> from enum import IntEnum > >>> class Shape(IntEnum): > ... circle = 1 > ... square = 2 > ... > >>> class Request(IntEnum): > ... post = 1 > ... get = 2 > ... > >>> Shape == 1 > False > >>> Shape.circle == 1 > True > >>> Shape.circle == Request.post > True > > However they still can't be compared to ``Enum``:: > > >>> class Shape(IntEnum): > ... circle = 1 > ... square = 2 > ... > >>> class Colors(Enum): > ... red = 1 > ... green = 2 > ... > >>> Shape.circle == Colors.red > False > > ``IntEnum`` values behave like integers in other ways you'd expect:: > > >>> int(Shape.circle) > 1 > >>> ['a', 'b', 'c'][Shape.circle] > 'b' > >>> [i for i in range(Shape.square)] > [0, 1] > > For the vast majority of code, ``Enum`` is strongly recommended. > Since ``IntEnum`` breaks some semantic promises of an enumeration (by > being comparable to integers, and thus by transitivity to other > unrelated enumerations), it should be used only in special cases where > there's no other choice; for example, when integer constants are > replaced with enumerations and backwards compatibility is required > with code that still expects integers. > > > Pickling > -------- > > Enumerations created with the class syntax can also be pickled and > unpickled:: > > >>> from enum.tests.fruit import Fruit > >>> from pickle import dumps, loads > >>> Fruit.tomato is loads(dumps(Fruit.tomato)) > True > > > Convenience API > --------------- > > The ``Enum`` class is callable, providing the following convenience > API:: > > >>> Animals = Enum('Animals', 'ant bee cat dog') > >>> Animals > > >>> Animals.ant > > >>> Animals.ant.value > 1 > > The semantics of this API resemble ``namedtuple``. The first argument > of the call to ``Enum`` is the name of the enumeration. The second > argument is a source of enumeration value names. It can be a > whitespace-separated string > of names, a sequence of names or a sequence of 2-tuples with key/value > pairs. > The last option enables assigning arbitrary values to enumerations; > the others > auto-assign increasing integers starting with 1. A new class derived > from ``Enum`` is returned. In other words, the above assignment to > ``Animals`` is > equivalent to:: > > >>> class Animals(Enum): > ... ant = 1 > ... bee = 2 > ... cat = 3 > ... dog = 4 > > Examples of alternative name/value specifications:: > > >>> Enum('Animals', ['ant', 'bee', 'cat', 'dog']) > > >>> Enum('Animals', (('ant', 'one'), ('bee', 'two'), ('cat', > >>> 'three'), > ('dog', 'four'))) > > > The second argument can also be a dictionary mapping names to values:: > > >>> levels = dict(debug=10, info=20, warning=30, severe=40) > >>> Enum('Levels', levels) > > > > Proposed variations > =================== > > Some variations were proposed during the discussions in the mailing > list. Here's some of the more popular ones. > > > Not having to specify values for enums > -------------------------------------- > > Michael Foord proposed (and Tim Delaney provided a proof-of-concept > implementation) to use metaclass magic that makes this possible:: > > class Color(Enum): > red, green, blue > > The values get actually assigned only when first looked up. > > Pros: cleaner syntax that requires less typing for a very common task > (just listing enumeration names without caring about the values). > > Cons: involves much magic in the implementation, which makes even the > definition of such enums baffling when first seen. Besides, explicit > is better than implicit. > > > Using special names or forms to auto-assign enum values > ------------------------------------------------------- > > A different approach to avoid specifying enum values is to use a > special name > or form to auto assign them. For example:: > > class Color(Enum): > red = None # auto-assigned to 0 > green = None # auto-assigned to 1 > blue = None # auto-assigned to 2 > > More flexibly:: > > class Color(Enum): > red = 7 > green = None # auto-assigned to 8 > blue = 19 > purple = None # auto-assigned to 20 > > Some variations on this theme: > > #. A special name ``auto`` imported from the enum package. > #. Georg Brandl proposed ellipsis (``...``) instead of ``None`` to > achieve the > same effect. > > Pros: no need to manually enter values. Makes it easier to change the > enum and > extend it, especially for large enumerations. > > Cons: actually longer to type in many simple cases. The argument of > explicit > vs. implicit applies here as well. > > > Use-cases in the standard library > ================================= > > The Python standard library has many places where the usage of enums > would be > beneficial to replace other idioms currently used to represent them. > Such usages can be divided to two categories: user-code facing > constants, and internal constants. > > User-code facing constants like ``os.SEEK_*``, ``socket`` module > constants, decimal rounding modes and HTML error codes could require > backwards compatibility since user code may expect integers. > ``IntEnum`` as described above provides the required semantics; being > a subclass of ``int``, it does not > affect user code that expects integers, while on the other hand > allowing printable representations for enumeration values:: > > >>> import socket > >>> family = socket.AF_INET > >>> family == 2 > True > >>> print(family) > SocketFamily.AF_INET > > Internal constants are not seen by user code but are employed > internally by stdlib modules. These can be implemented with > ``Enum``. Some examples uncovered by a very partial skim through the > stdlib: ``binhex``, ``imaplib``, > ``http/client``, ``urllib/robotparser``, ``idlelib``, > ``concurrent.futures``, > ``turtledemo``. > > In addition, looking at the code of the Twisted library, there are > many use cases for replacing internal state constants with enums. > The same can be said > about a lot of networking code (especially implementation of > protocols) and can be seen in test protocols written with the Tulip > library as well. > > > Differences from PEP 354 > ======================== > > Unlike PEP 354, enumeration values are not defined as a sequence of > strings, but as attributes of a class. This design was chosen > because it was felt that > class syntax is more readable. > > Unlike PEP 354, enumeration values require an explicit integer > value. This difference recognizes that enumerations often represent > real-world values, or > must interoperate with external real-world systems. For example, to > store an > enumeration in a database, it is better to convert it to an integer > on the way > in and back to an enumeration on the way out. Providing an integer > value also > provides an explicit ordering. However, there is no automatic > conversion to and from the integer values, because explicit is better > than implicit. > > Unlike PEP 354, this implementation does use a metaclass to define the > enumeration's syntax, and allows for extended base-enumerations so > that the common values in derived classes are identical (a singleton > model). While PEP > 354 dismisses this approach for its complexity, in practice any > perceived complexity, though minimal, is hidden from users of the > enumeration. > > Unlike PEP 354, enumeration values should only be tested by identity > comparison. This is to emphasize the fact that enumeration values are > singletons, much like ``None``. > > > Acknowledgments > =============== > > This PEP describes the ``flufl.enum`` package by Barry Warsaw. > ``flufl.enum`` > is based on an example by Jeremy Hylton. It has been modified and > extended by Barry Warsaw for use in the GNU Mailman [5]_ project. > Ben Finney is the author of the earlier enumeration PEP 354. > > > References > ========== > > .. [1] http://pythonhosted.org/flufl.enum/docs/using.html > .. [2] http://www.python.org/dev/peps/pep-0354/ > .. [3] > http://mail.python.org/pipermail/python-ideas/2013-January/019003.html > .. [4] > http://mail.python.org/pipermail/python-ideas/2013-February/019373.html > .. [5] http://www.list.org > > > Copyright > ========= > > This document has been placed in the public domain. > > > Todo > ==== > > * Mark PEP 354 "superseded by" this one, if accepted > > .. > Local Variables: > mode: indented-text > indent-tabs-mode: nil > sentence-end-double-space: t > fill-column: 70 > coding: utf-8 > End: > From solipsis at pitrou.net Tue Apr 23 15:44:58 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 23 Apr 2013 15:44:58 +0200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library References: Message-ID: <20130423154458.3bba1e57@pitrou.net> Hello, (sorry for the previous message attempt - my mouse pointer hit the send button before I was finished with it) Le Fri, 12 Apr 2013 05:55:00 -0700, Eli Bendersky a ?crit : > > We're happy to present the revised PEP 435, collecting valuable > feedback from python-ideas discussions as well as in-person > discussions and decisions made during the latest PyCon language > summit. We believe the proposal is now better than the original one, > providing both a wider set of features and more convenient ways to > use those features. I'm having a problem with the proposed implementation. I haven't found any mention of it, so apologies if this has already been discussed: >>> from flufl.enum import * >>> class C(Enum): ... a = 1 ... b = 2 ... >>> C.a.__class__ >>> isinstance(C.a, C) False >>> isinstance(C(1), C) False It would really be better if instances were actual instances of the class, IMO. Regards Antoine. From rdmurray at bitdance.com Tue Apr 23 16:16:01 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 23 Apr 2013 10:16:01 -0400 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130423141602.47FAC250BCA@webabinitio.net> On Tue, 23 Apr 2013 22:29:33 +0900, "Stephen J. Turnbull" wrote: > R. David Murray writes: > > > You transform *into* the encoding, and untransform *out* of the > > encoding. Do you have an example where that would be ambiguous? > > In the bytes-to-bytes case, any pair of character encodings (eg, UTF-8 > and ISO-8859-15) would do. Or how about in text, ReST to HTML? If I write: bytestring.transform('ISO-8859-15') that would indeed be ambiguous, but only because I haven't named the source encoding of the bytestring. So the above is obviously nonsense, and the easiest "fix" is to have the things that are currently bytes-to-text or text-to-bytes character set transformations *only* work with encode/decode, and not transform/untransform. > BASE64 itself is ambiguous. By RFC specification, BASE64 is a > *textual* representation of arbitrary binary data. (Cf. URIs.) The > natural interpretation of .encode('base64') in that context would be > as a bytes-to-text encoder. However, this has several problems. In > practice, we invariably use an ASCII octet stream to carry BASE64- > encoded data. So web developers would almost certainly expect a > bytes-to-bytes encoder. Such a bytes-to-bytes encoder can't be > duck-typed. Double-encoding bugs wouldn't be detected until the > stream arrives at the user. And the RFC-based signature of > .encode('base64') as bytes-to-text is precisely opposite to that of > .encode('utf-8') (text-to-bytes). I believe that after much discussion we have settled on these transformations (in their respective modules) accepting either bytes or strings as input for decoding, only bytes as input for encoding, and *always* producing bytes as output. (Note that the base64 docs need some clarification about this.) Given this, the possible valid transformations would be: bytestring.transform('base64') bytesstring.untransform('base64') string.untransform('base64') and all would produce a byte string. That byte string would be in base64 for the first one, and a decoded binary string for the second two. Given our existing API, I don't think we want string.encode('base64') to work (taking an ascii-only unicode string and returning bytes), and we've already agreed that adding a 'decode' method to string is not going to happen. We could, however, and quite possibly should, disallow string.untransform('base64') even though the underly module supports it. Thus we would only have bytes-to-bytes transformations for 'base64' and its siblings, and you would write the unicode-ascii-to-bytes transformation as: string.encode('ascii').untransform('base64') which has some pedagogical value :). If you do transform('base64') on a bytestring already encoded as base64 you get a double encoding, yes. I don't see that it is our responsibility to try to protect you from this mistake. The module functions certainly don't. Given that, is there anything ambiguous about the proposed API? (Note: if you would like to argue that, eg, base64.b64encode or binascii.b2a_base64 should return a string, it is too late for that argument for backward compatibility reasons.) > It is certainly true that there are many unambiguous cases. In the > case of a true text processing facility (eg, Emacs buffers or Python 3 > str) where there is an unambiguous text type with a constant and > opaque internal representation, it makes a lot of sense to treat the > text type as special/central, and use the terminology "encode [from > text]" and "decode [to text]". It's easy to remember, which one is > special is obvious, and the difference in input and output types means > that mistaken use of the API will be detected by duck-typing. > > However, in the case of bytes-bytes or text-text transformations, it's > not the presence of unambiguous cases that should drive API design > IMO. It's the presence of the ambiguous cases that we should cater > to. I don't see easy solutions to this issue. When I asked about ambiguous cases, I was asking for cases where the meaning of "transform('somecodec')" was ambiguous. Sure, it is possible to feed the wrong input into that transformation, but I consider that a programming error, not an ambiguity in the API. After all, you have exactly the same problem if you use the module functions directly, which is currently the only option. --David From rdmurray at bitdance.com Tue Apr 23 16:24:18 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 23 Apr 2013 10:24:18 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130423154458.3bba1e57@pitrou.net> References: <20130423154458.3bba1e57@pitrou.net> Message-ID: <20130423142418.652DE250BCA@webabinitio.net> On Tue, 23 Apr 2013 15:44:58 +0200, Antoine Pitrou wrote: > Le Fri, 12 Apr 2013 05:55:00 -0700, > Eli Bendersky a ??crit : > > > > We're happy to present the revised PEP 435, collecting valuable > > feedback from python-ideas discussions as well as in-person > > discussions and decisions made during the latest PyCon language > > summit. We believe the proposal is now better than the original one, > > providing both a wider set of features and more convenient ways to > > use those features. > > I'm having a problem with the proposed implementation. I haven't found > any mention of it, so apologies if this has already been discussed: > > >>> from flufl.enum import * > >>> class C(Enum): > ... a = 1 > ... b = 2 > ... > >>> C.a.__class__ > > >>> isinstance(C.a, C) > False > >>> isinstance(C(1), C) > False > > It would really be better if instances were actual instances of the > class, IMO. The first False looks correct to me, I would not expect an enum value to be an instance of the class that holds it as an attribute. The second certainly looks odd, but what does it even mean to have an instance of an Enum class? --David From ncoghlan at gmail.com Tue Apr 23 16:22:40 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 24 Apr 2013 00:22:40 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130423154458.3bba1e57@pitrou.net> References: <20130423154458.3bba1e57@pitrou.net> Message-ID: On Tue, Apr 23, 2013 at 11:44 PM, Antoine Pitrou wrote: > I'm having a problem with the proposed implementation. I haven't found > any mention of it, so apologies if this has already been discussed: > >>>> from flufl.enum import * >>>> class C(Enum): > ... a = 1 > ... b = 2 > ... >>>> C.a.__class__ > >>>> isinstance(C.a, C) > False >>>> isinstance(C(1), C) > False > > > It would really be better if instances were actual instances of the > class, IMO. Looking at the source (https://bazaar.launchpad.net/~barry/flufl.enum/trunk/view/head:/flufl/enum/_enum.py), I'm not seeing any fundamental technical issues with merging the Enum and EnumValue class definitions, and then using "cls" where the metaclass code currently uses "cls.__value_factory__" (even for the backwards compatible variant, the 2v3 metaclass issue isn't a problem, you can just define a _BaseEnum class with the right metaclass using the 2 & 3 compatible notation and inherit from that in a normal class definition) However, there's one non-technical aspect of such a merger which does concern me, which is the fact that you would lose the distinct docstrings for the class and the values: >>> class C(flufl.enum.Enum): ... "My enum" ... a = 1 ... >>> print(C.__doc__) My enum >>> print(type(C.a).__doc__) Class to represent an enumeration value. EnumValue('Color', 'red', 12) prints as 'Color.red' and can be converted to the integer 12. So I'm not sure the PEP has made the wrong choice here, but I agree the point is at least worth mentioning. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Tue Apr 23 16:31:04 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 24 Apr 2013 00:31:04 +1000 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <20130423141602.47FAC250BCA@webabinitio.net> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <20130423141602.47FAC250BCA@webabinitio.net> Message-ID: On Wed, Apr 24, 2013 at 12:16 AM, R. David Murray wrote: > On Tue, 23 Apr 2013 22:29:33 +0900, "Stephen J. Turnbull" wrote: >> R. David Murray writes: >> >> > You transform *into* the encoding, and untransform *out* of the >> > encoding. Do you have an example where that would be ambiguous? >> >> In the bytes-to-bytes case, any pair of character encodings (eg, UTF-8 >> and ISO-8859-15) would do. Or how about in text, ReST to HTML? > > If I write: > > bytestring.transform('ISO-8859-15') > > that would indeed be ambiguous, but only because I haven't named the > source encoding of the bytestring. So the above is obviously > nonsense, and the easiest "fix" is to have the things that are currently > bytes-to-text or text-to-bytes character set transformations *only* > work with encode/decode, and not transform/untransform. And that's where it all falls down - to make that work, you need to engineer a complex system into the codecs module to say "this codec can be used with that API, but not with this one". I designed such a beast in http://bugs.python.org/issue7475 and I now think it's a *bad idea*. By contrast, the convenience function approach dispenses with all that, and simply says: 1. If you just want to deal with text encodings, use str.encode (which always produces bytes), along with bytes.decode and bytearray.decode (which always produce str) 2. If you want to use arbitrary codecs without any additional type constraints, do "from codecs import encode, decode" I think there's value in hiding the arbitrary codec support behind an import barrier (as they definitely have potential to be an attractive nuisance that makes it harder to grasp the nature of Unicode and text encodings, particularly for those coming from Python 2.x), but I'm not hugely opposed to providing them as builtins either. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From solipsis at pitrou.net Tue Apr 23 16:33:02 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 23 Apr 2013 16:33:02 +0200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> Message-ID: <20130423163302.195e4587@pitrou.net> Le Tue, 23 Apr 2013 10:24:18 -0400, "R. David Murray" a ?crit : > > > > I'm having a problem with the proposed implementation. I haven't > > found any mention of it, so apologies if this has already been > > discussed: > > > > >>> from flufl.enum import * > > >>> class C(Enum): > > ... a = 1 > > ... b = 2 > > ... > > >>> C.a.__class__ > > > > >>> isinstance(C.a, C) > > False > > >>> isinstance(C(1), C) > > False > > > > It would really be better if instances were actual instances of the > > class, IMO. > > The first False looks correct to me, I would not expect an enum value > to be an instance of the class that holds it as an attribute. The > second certainly looks odd, but what does it even mean to have an > instance of an Enum class? Perhaps I should have added some context: >>> class C(Enum): ... a = 1 ... b = 2 ... >>> C(1) >>> C[1] It is simply the same as a __getattr__ call. That said, I don't see why it wouldn't make sense for an enum value to be an instance of that class. It can be useful to write `isinstance(value, MyEnumClass)`. Also, any debug facility which has a preference for writing out class names would produce a better output than the generic "EnumValue". Regards Antoine. From rdmurray at bitdance.com Tue Apr 23 16:53:54 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 23 Apr 2013 10:53:54 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130423163302.195e4587@pitrou.net> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130423163302.195e4587@pitrou.net> Message-ID: <20130423145355.2F527250BCA@webabinitio.net> On Tue, 23 Apr 2013 16:33:02 +0200, Antoine Pitrou wrote: > Le Tue, 23 Apr 2013 10:24:18 -0400, > "R. David Murray" a ??crit : > > > > > > I'm having a problem with the proposed implementation. I haven't > > > found any mention of it, so apologies if this has already been > > > discussed: > > > > > > >>> from flufl.enum import * > > > >>> class C(Enum): > > > ... a = 1 > > > ... b = 2 > > > ... > > > >>> C.a.__class__ > > > > > > >>> isinstance(C.a, C) > > > False > > > >>> isinstance(C(1), C) > > > False > > > > > > It would really be better if instances were actual instances of the > > > class, IMO. > > > > The first False looks correct to me, I would not expect an enum value > > to be an instance of the class that holds it as an attribute. The > > second certainly looks odd, but what does it even mean to have an > > instance of an Enum class? > > Perhaps I should have added some context: > > >>> class C(Enum): > ... a = 1 > ... b = 2 > ... > >>> C(1) > > >>> C[1] > > > It is simply the same as a __getattr__ call. > > That said, I don't see why it wouldn't make sense for an enum value to > be an instance of that class. It can be useful to write > `isinstance(value, MyEnumClass)`. Also, any debug facility which has a > preference for writing out class names would produce a better output > than the generic "EnumValue". Ah. I'd be looking for a bug every time I saw isinstance(value, myEnumClass). A better class name for values and an API for getting that class from the EnumClass would be nice, though. (So that you could write "isinstance(value, MyEnumClass.ValueClass)", say.) --David From solipsis at pitrou.net Tue Apr 23 16:57:23 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 23 Apr 2013 16:57:23 +0200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library References: <20130423154458.3bba1e57@pitrou.net> Message-ID: <20130423165723.78e286a5@pitrou.net> Le Wed, 24 Apr 2013 00:22:40 +1000, Nick Coghlan a ?crit : > > Looking at the source > (https://bazaar.launchpad.net/~barry/flufl.enum/trunk/view/head:/flufl/enum/_enum.py), > I'm not seeing any fundamental technical issues with merging the Enum > and EnumValue class definitions, and then using "cls" where the > metaclass code currently uses "cls.__value_factory__" (even for the > backwards compatible variant, the 2v3 metaclass issue isn't a problem, > you can just define a _BaseEnum class with the right metaclass using > the 2 & 3 compatible notation and inherit from that in a normal class > definition) > > However, there's one non-technical aspect of such a merger which does > concern me, which is the fact that you would lose the distinct > docstrings for the class and the values: You can work it around by making __doc__ a descriptor that behaves differently when called on a class or an instance. There is a slight metaclass complication because __doc__ is not writable on a class (but I suppose enums are already using a metaclass, so it's not much of an issue): class docwrapper: def __init__(self, class_doc, instance_doc_func): self.class_doc = class_doc self.instance_doc_func = instance_doc_func def __get__(self, instance, owner): if instance is None: return self.class_doc else: return self.instance_doc_func(instance) def instancedocwrapper(instance_doc_func): class metaclass(type): def __new__(meta, name, bases, dct): dct['__doc__'] = docwrapper(dct['__doc__'], instance_doc_func) return type.__new__(meta, name, bases, dct) return metaclass class D(metaclass=instancedocwrapper( lambda self: "My instance:{}".format(self.x))): """My beautiful, documented class.""" def __init__(self, x): self.x = x class E(D): """My magnificent subclass.""" print("class doc:", D.__doc__) print("subclass doc:", E.__doc__) print("instance doc:", E(5).__doc__) Note that the builtin help() function always displays the class's __doc__, even when called on an instance which has its own __doc__. Regards Antoine. From barry at python.org Tue Apr 23 17:07:25 2013 From: barry at python.org (Barry Warsaw) Date: Tue, 23 Apr 2013 11:07:25 -0400 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: <5175C4C4.503@canterbury.ac.nz> <5175EC3D.4090007@pearwood.info> Message-ID: <20130423110725.223f5df9@limelight.wooz.org> On Apr 22, 2013, at 10:30 PM, Donald Stufft wrote: >I may be dull, but it wasn't until I started using Python 3 that it really >clicked in my head what encode/decode did exactly. In Python2 I just sort of >sprinkled one or the other when there was errors until the pain stopped. I >mostly attribute this to str.decode and bytes.encode not existing. This is a key observation. It's also now much easier to *explain* what's going on and recommend correct code in Python 3, so overall it's a win. That's not to downplay the inconvenience of not being able to easily do bytes->bytes or str->str transformations as easily as was possible in Python 2. I've not thought about it much, but placing those types of transformations on a different set of functions (methods or builtins) seems like the right direction. IOW, don't mess with encode/decode. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From guido at python.org Tue Apr 23 17:11:06 2013 From: guido at python.org (Guido van Rossum) Date: Tue, 23 Apr 2013 08:11:06 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130423165723.78e286a5@pitrou.net> References: <20130423154458.3bba1e57@pitrou.net> <20130423165723.78e286a5@pitrou.net> Message-ID: I gotta say, I'm with Antoine here. It's pretty natural (also coming from other languages) to assume that the class used to define the enums is also the type of the enum values. Certainly this is how it works in Java and C++, and I would say it's the same in Pascal and probably most other languages. This doesn't seem to matter much today, but various people are thinking about adding optional type annotations to Python code (use cases include hints for liters and better suggestions for IDEs). What would be more natural than being able to write class Color(Enum): RED = 1 WHITE = 2 BLUE = 3 ORANGE = 4 class Bikeshed: def paint(self, hue: Color, opacity: int): ... It would be awkward if the 'how' parameter had to be annotated with Color.ValueClass, or if the type checkers had to learn to imply .ValueClass if an attribute was labeled with an Enum type. Enums just aren't special enough. I find the docstring issue secondary. --Guido On Tue, Apr 23, 2013 at 7:57 AM, Antoine Pitrou wrote: > Le Wed, 24 Apr 2013 00:22:40 +1000, > Nick Coghlan a ?crit : >> >> Looking at the source >> (https://bazaar.launchpad.net/~barry/flufl.enum/trunk/view/head:/flufl/enum/_enum.py), >> I'm not seeing any fundamental technical issues with merging the Enum >> and EnumValue class definitions, and then using "cls" where the >> metaclass code currently uses "cls.__value_factory__" (even for the >> backwards compatible variant, the 2v3 metaclass issue isn't a problem, >> you can just define a _BaseEnum class with the right metaclass using >> the 2 & 3 compatible notation and inherit from that in a normal class >> definition) >> >> However, there's one non-technical aspect of such a merger which does >> concern me, which is the fact that you would lose the distinct >> docstrings for the class and the values: > > You can work it around by making __doc__ a descriptor that behaves > differently when called on a class or an instance. There is a slight > metaclass complication because __doc__ is not writable on a class (but > I suppose enums are already using a metaclass, so it's not much of an > issue): > > > class docwrapper: > def __init__(self, class_doc, instance_doc_func): > self.class_doc = class_doc > self.instance_doc_func = instance_doc_func > > def __get__(self, instance, owner): > if instance is None: > return self.class_doc > else: > return self.instance_doc_func(instance) > > def instancedocwrapper(instance_doc_func): > class metaclass(type): > def __new__(meta, name, bases, dct): > dct['__doc__'] = docwrapper(dct['__doc__'], > instance_doc_func) > return type.__new__(meta, name, bases, dct) > return metaclass > > class D(metaclass=instancedocwrapper( > lambda self: "My instance:{}".format(self.x))): > """My beautiful, documented class.""" > def __init__(self, x): > self.x = x > > class E(D): > """My magnificent subclass.""" > > print("class doc:", D.__doc__) > print("subclass doc:", E.__doc__) > print("instance doc:", E(5).__doc__) > > > Note that the builtin help() function always displays the class's > __doc__, even when called on an instance which has its own __doc__. > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) From barry at python.org Tue Apr 23 17:15:23 2013 From: barry at python.org (Barry Warsaw) Date: Tue, 23 Apr 2013 11:15:23 -0400 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> Message-ID: <20130423111523.31af429f@limelight.wooz.org> On Apr 22, 2013, at 06:22 PM, Guido van Rossum wrote: >> You can ask the same question about all the other codecs. (And that >> question has indeed been asked in the past.) > >Except for rot13. :-) The fact that you can do this instead *is* a bit odd. ;) from codecs import getencoder encoder = getencoder('rot-13') r13 = encoder('hello world')[0] -Barry From emmanuel.bacry at polytechnique.fr Tue Apr 23 12:32:41 2013 From: emmanuel.bacry at polytechnique.fr (ebacry) Date: Tue, 23 Apr 2013 03:32:41 -0700 (PDT) Subject: [Python-Dev] distutils win32 multiple extensions Message-ID: <1366713161031-5015137.post@n6.nabble.com> Hello,I am writing some C extensions (using swig).A first extension (a python module) which I'll call E1.I wrote a setup.py which works fine on all platforms.On Windows 7 it creates a library which is a .pyd fileNow I want to write a second C extension E2 that is calling some functions of E1.The same kind of setup.py works fine on all platforms but Windows where it says (during the link) that the function sof E1 that are called by E2 are unreferenced.I am clearly not a Windows pro ... I am using mingw32 on Windows.Surfing the web, I understood that shared libraries work differently on windows than on other platforms. And that I should1- export the symbols (i.e., the functions) of E1 when I build E1. In the distutils.Extension this seems to be done using argument export_symbols=[list of symbols to export]Now I have one problem : each time I specify in this list the name of a function in E1 it says that it doesn't know this symbolWhat am I doing wrong here, is there a special syntax ?2- Now in the setup.py of E2 how do I declare that I want to use the E1 shared lib.? Is the information on the exported symbol of E1 in the .pyd file or in the .def file ?I tried to specify the .pyd library in the arguments of the distutils.Extension .... but I have the feeling that's not the right thing to do (I have the feeling the .pyd is not the right file to point to)Not mentioning the problem specifying the path of the library with runtime_library_dirs ... which does not seem to work at all.Is there a simple way to do what I want to do ?I did not find anything on the web explaining clearly what to do ...I am really really stuckI would appreciate any kind of help Emmanuel -- View this message in context: http://python.6.x6.nabble.com/distutils-win32-multiple-extensions-tp5015137.html Sent from the Python - python-dev mailing list archive at Nabble.com. -------------- next part -------------- An HTML attachment was scrubbed... URL: From emmanuel.bacry at polytechnique.fr Tue Apr 23 12:35:43 2013 From: emmanuel.bacry at polytechnique.fr (ebacry) Date: Tue, 23 Apr 2013 03:35:43 -0700 (PDT) Subject: [Python-Dev] distutils win32 multiple extensions Message-ID: <1366713343803-5015138.post@n6.nabble.com> Hello, I am writing some C extensions (using swig). A first extension (a python module) which I'll call E1. I wrote a setup.py which works fine on all platforms. On Windows 7 it creates a library which is a .pyd file Now I want to write a second C extension E2 that is calling some functions of E1. The same kind of setup.py works fine on all platforms but Windows where it says (during the link) that the function sof E1 that are called by E2 are unreferenced. I am clearly not a Windows pro ... I am using mingw32 on Windows. Surfing the web, I understood that shared libraries work differently on windows than on other platforms. And that I should 1- export the symbols (i.e., the functions) of E1 when I build E1. In the distutils.Extension this seems to be done using argument export_symbols=[list of symbols to export] Now I have one problem : each time I specify in this list the name of a function in E1 it says that it doesn't know this symbol What am I doing wrong here, is there a special syntax ? 2- Now in the setup.py of E2 how do I declare that I want to use the E1 shared lib.? Is the information on the exported symbol of E1 in the .pyd file or in the .def file ? I tried to specify the .pyd library in the arguments of the distutils.Extension .... but I have the feeling that's not the right thing to do (I have the feeling the .pyd is not the right file to point to) Not mentioning the problem specifying the path of the library with runtime_library_dirs ... which does not seem to work at all. Is there a simple way to do what I want to do ? I did not find anything on the web explaining clearly what to do ... I am really really stuck I would appreciate any kind of help Emmanuel -- View this message in context: http://python.6.x6.nabble.com/distutils-win32-multiple-extensions-tp5015138.html Sent from the Python - python-dev mailing list archive at Nabble.com. From mal at egenix.com Tue Apr 23 17:22:35 2013 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 23 Apr 2013 17:22:35 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <20130423111523.31af429f@limelight.wooz.org> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <20130423111523.31af429f@limelight.wooz.org> Message-ID: <5176A73B.1050809@egenix.com> On 23.04.2013 17:15, Barry Warsaw wrote: > On Apr 22, 2013, at 06:22 PM, Guido van Rossum wrote: > >>> You can ask the same question about all the other codecs. (And that >>> question has indeed been asked in the past.) >> >> Except for rot13. :-) > > The fact that you can do this instead *is* a bit odd. ;) > > from codecs import getencoder > encoder = getencoder('rot-13') > r13 = encoder('hello world')[0] Just as reminder: we have the general purpose encode()/decode() functions in the codecs module: import codecs r13 = codecs.encode('hello world', 'rot-13') These interface directly to the codec interfaces, without enforcing type restrictions. The codec defines the supported input and output types. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 23 2013) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2013-04-17: Released eGenix mx Base 3.2.6 ... http://egenix.com/go43 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From rdmurray at bitdance.com Tue Apr 23 17:31:06 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 23 Apr 2013 11:31:06 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423165723.78e286a5@pitrou.net> Message-ID: <20130423153106.4B25225003E@webabinitio.net> On Tue, 23 Apr 2013 08:11:06 -0700, Guido van Rossum wrote: > I gotta say, I'm with Antoine here. It's pretty natural (also coming > from other languages) to assume that the class used to define the > enums is also the type of the enum values. Certainly this is how it > works in Java and C++, and I would say it's the same in Pascal and > probably most other languages. Well, I guess I can wrap my head around it :) An Enum is an odd duck anyway, which I suppose is one of the things that makes it worth adding. --David From guido at python.org Tue Apr 23 17:44:21 2013 From: guido at python.org (Guido van Rossum) Date: Tue, 23 Apr 2013 08:44:21 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130423153106.4B25225003E@webabinitio.net> References: <20130423154458.3bba1e57@pitrou.net> <20130423165723.78e286a5@pitrou.net> <20130423153106.4B25225003E@webabinitio.net> Message-ID: On Tue, Apr 23, 2013 at 8:31 AM, R. David Murray wrote: > On Tue, 23 Apr 2013 08:11:06 -0700, Guido van Rossum wrote: >> I gotta say, I'm with Antoine here. It's pretty natural (also coming >> from other languages) to assume that the class used to define the >> enums is also the type of the enum values. Certainly this is how it >> works in Java and C++, and I would say it's the same in Pascal and >> probably most other languages. > > Well, I guess I can wrap my head around it :) An Enum is an odd duck > anyway, which I suppose is one of the things that makes it worth adding. Sorry, you're being to literary/poetic. I can't tell what you meant by this response. -- --Guido van Rossum (python.org/~guido) From guido at python.org Tue Apr 23 17:47:34 2013 From: guido at python.org (Guido van Rossum) Date: Tue, 23 Apr 2013 08:47:34 -0700 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <5176A73B.1050809@egenix.com> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <20130423111523.31af429f@limelight.wooz.org> <5176A73B.1050809@egenix.com> Message-ID: On Tue, Apr 23, 2013 at 8:22 AM, M.-A. Lemburg wrote: > Just as reminder: we have the general purpose > encode()/decode() functions in the codecs module: > > import codecs > r13 = codecs.encode('hello world', 'rot-13') > > These interface directly to the codec interfaces, without > enforcing type restrictions. The codec defines the supported > input and output types. As an implementation mechanism I see nothing wrong with this. I hope the codecs module lets you introspect the input and output types of a codec given by name? -- --Guido van Rossum (python.org/~guido) From scott+python-dev at scottdial.com Tue Apr 23 17:46:50 2013 From: scott+python-dev at scottdial.com (Scott Dial) Date: Tue, 23 Apr 2013 11:46:50 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130423145355.2F527250BCA@webabinitio.net> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130423163302.195e4587@pitrou.net> <20130423145355.2F527250BCA@webabinitio.net> Message-ID: <5176ACEA.7060903@scottdial.com> On 4/23/2013 10:53 AM, R. David Murray wrote: > Ah. I'd be looking for a bug every time I saw isinstance(value, > myEnumClass). A better class name for values and an API for > getting that class from the EnumClass would be nice, though. > (So that you could write "isinstance(value, MyEnumClass.ValueClass)", > say.) Reading what you have wrote, it seems that the issue is whether you consider an instance of Enum a "thing" or a "class of things". If you think of it as a "thing", then "C" is a object that has attributes for other "things" that are not like "C". However, if you think of "C" as a "class of things", then "C" having attributes that are instances of it's type is completely natural. Fundamentally, the question is whether an instance of Enum is a new type or an instance. And for me, it's a new type and I expect enum values to be instance of that type. -Scott -- Scott Dial scott at scottdial.com From rdmurray at bitdance.com Tue Apr 23 17:56:16 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 23 Apr 2013 11:56:16 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423165723.78e286a5@pitrou.net> <20130423153106.4B25225003E@webabinitio.net> Message-ID: <20130423155616.7B61625003E@webabinitio.net> On Tue, 23 Apr 2013 08:44:21 -0700, Guido van Rossum wrote: > On Tue, Apr 23, 2013 at 8:31 AM, R. David Murray wrote: > > On Tue, 23 Apr 2013 08:11:06 -0700, Guido van Rossum wrote: > >> I gotta say, I'm with Antoine here. It's pretty natural (also coming > >> from other languages) to assume that the class used to define the > >> enums is also the type of the enum values. Certainly this is how it > >> works in Java and C++, and I would say it's the same in Pascal and > >> probably most other languages. > > > > Well, I guess I can wrap my head around it :) An Enum is an odd duck > > anyway, which I suppose is one of the things that makes it worth adding. > > Sorry, you're being to literary/poetic. I can't tell what you meant by > this response. I was alluding to the fact that in Python we usually work with instances of classes (not always, I know, but still...), but with Enum we are really using the class as a first level object. Given that, breaking my (questionable?) intuition about isinstance should not be unexpected. And by that last phrase I meant to refer to the fact that getting this right is obviously non-trivial, which is one of the things that makes it worth adding as a Python feature. --David From guido at python.org Tue Apr 23 17:58:06 2013 From: guido at python.org (Guido van Rossum) Date: Tue, 23 Apr 2013 08:58:06 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5176ACEA.7060903@scottdial.com> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130423163302.195e4587@pitrou.net> <20130423145355.2F527250BCA@webabinitio.net> <5176ACEA.7060903@scottdial.com> Message-ID: You seem to be mixing up classes and metaclasses. On Tue, Apr 23, 2013 at 8:46 AM, Scott Dial wrote: > On 4/23/2013 10:53 AM, R. David Murray wrote: >> Ah. I'd be looking for a bug every time I saw isinstance(value, >> myEnumClass). A better class name for values and an API for >> getting that class from the EnumClass would be nice, though. >> (So that you could write "isinstance(value, MyEnumClass.ValueClass)", >> say.) > > Reading what you have wrote, it seems that the issue is whether you > consider an instance of Enum a "thing" or a "class of things". If you > think of it as a "thing", then "C" is a object that has attributes for > other "things" that are not like "C". However, if you think of "C" as a > "class of things", then "C" having attributes that are instances of it's > type is completely natural. > > Fundamentally, the question is whether an instance of Enum is a new type > or an instance. And for me, it's a new type and I expect enum values to > be instance of that type. > > -Scott > > -- > Scott Dial > scott at scottdial.com > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) From mal at egenix.com Tue Apr 23 18:04:37 2013 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 23 Apr 2013 18:04:37 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <20130423111523.31af429f@limelight.wooz.org> <5176A73B.1050809@egenix.com> Message-ID: <5176B115.8050101@egenix.com> On 23.04.2013 17:47, Guido van Rossum wrote: > On Tue, Apr 23, 2013 at 8:22 AM, M.-A. Lemburg wrote: >> Just as reminder: we have the general purpose >> encode()/decode() functions in the codecs module: >> >> import codecs >> r13 = codecs.encode('hello world', 'rot-13') >> >> These interface directly to the codec interfaces, without >> enforcing type restrictions. The codec defines the supported >> input and output types. > > As an implementation mechanism I see nothing wrong with this. I hope > the codecs module lets you introspect the input and output types of a > codec given by name? At the moment there is no standard interface to access supported input and output types... but then: regular Python functions or methods also don't provide such functionality, so no surprise there ;-) It's mostly a matter of specifying the supported type combinations in the codec documentation. BTW: What would be a use case where you'd want to programmatically access such information before calling the codec ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 23 2013) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2013-04-17: Released eGenix mx Base 3.2.6 ... http://egenix.com/go43 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From stephen at xemacs.org Tue Apr 23 18:49:39 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 24 Apr 2013 01:49:39 +0900 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <20130423141602.47FAC250BCA@webabinitio.net> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <20130423141602.47FAC250BCA@webabinitio.net> Message-ID: <87ehe1yyxo.fsf@uwakimon.sk.tsukuba.ac.jp> R. David Murray writes: > On Tue, 23 Apr 2013 22:29:33 +0900, "Stephen J. Turnbull" wrote: > > R. David Murray writes: > > > > > You transform *into* the encoding, and untransform *out* of the > > > encoding. Do you have an example where that would be ambiguous? > > > > In the bytes-to-bytes case, any pair of character encodings (eg, UTF-8 > > and ISO-8859-15) would do. Or how about in text, ReST to HTML? > > If I write: > > bytestring.transform('ISO-8859-15') > > that would indeed be ambiguous, but only because I haven't named the > source encoding of the bytestring. So the above is obviously > nonsense, and the easiest "fix" is to have the things that are currently > bytes-to-text or text-to-bytes character set transformations *only* > work with encode/decode, and not transform/untransform. I think you're completely missing my point here. The problem is that in the cases I mention, what is encoded data and what is decoded data can only be decided by asking the user. > I believe that after much discussion we have settled on these > transformations (in their respective modules) accepting either bytes > or strings as input for decoding, only bytes as input for encoding, > and *always* producing bytes as output. Which, of course, is quite broken from the point of view of the RFC! Of course, the RFC be damned[1], for the purposes of the Python stdlib, the specific codecs used for Content-Transfer-Encoding have a clear intuitive directionality, and their encoding methods should turn bytes into bytes (and str or bytes into bytes on decoding). Nevertheless, it's not TOOWTDI, it's a careful compromise. > Given this, the possible valid transformations would be: > > bytestring.transform('base64') > bytesstring.untransform('base64') > string.untransform('base64') Which is an obnoxious API, since (1) you've now made it impossible to use "transform" for bytestring.transform(from='utf-8', to='iso-8859-1') bytestring.transform(from='ulaw', to='mp3') textstring.transform(from='rest', to='html') without confusion, and (2) the whole world is going to wonder why you don't use .encode and .decode instead of .transform and .untransform. The idea in the examples is that we could generalize the codec registry to look up codecs by pairs of media-types. I'm not sure this makes sense ... much of the codec API presumes a stream, especially the incremental methods. But many MIME media types are streams only because they're serializations, incremental en/decoding is nonsense. So I suppose I would want to write bytestring.transform(from='octet-stream', to='BASE64') for this hypothetical API. (I suspect that in practice the 'application/octet-stream' media type would be spelled 'bytes', of course.) This kind of API could be used to improve the security of composition of transforms. In the case of BASE64, it would make sense to match anything at all as the other type (as long as it's represented in Python by a bytes object). So it would be possible to do object = bytestring.transform(from='BASE64', to='PNG') giving object a media_type attribute such that object.decode('iso-8859-1') would fail. (This would require changes to the charset codecs, to pay heed to the media_type attribute, so it's not immediately feasible.) > and all would produce a byte string. That byte string would be in > base64 for the first one, and a decoded binary string for the second two. > > Given our existing API, I don't think we want > > string.encode('base64') > > to work (taking an ascii-only unicode string and returning bytes), No, we don't, but for reasons that have little to do with "ASCII-only". The problem with supporting that idiom is that *people can't read strs* [in the Python 3 internal representation] -- they can only read a str that has been encoded implicitly into the PYTHONIOENCODING or explicitly to an explicitly requested encoding. So the usage above is clearly ambiguous. Even if it is ASCII-only, in theory the user could want EBCDIC. > If you do transform('base64') on a bytestring already encoded as > base64 you get a double encoding, yes. I don't see that it is our > responsibility to try to protect you from this mistake. The module > functions certainly don't. > > Given that, is there anything ambiguous about the proposed API? Not for BASE64. But what's so special about BASE64 that it deserves a new method name for the same old idiom, using a word that's an obvious candidate for naming a more general idiom? > (Note: if you would like to argue that, eg, base64.b64encode or > binascii.b2a_base64 should return a string, it is too late for that > argument for backward compatibility reasons.) Even if it weren't too late, the byte-shoveling lobby is way too strong; that's not a winnable agument. > When I asked about ambiguous cases, I was asking for cases where > the meaning of "transform('somecodec')" was ambiguous. If "transform('somecodec')" isn't ambiguous, you really really want to spell it "encode" instead of "transform" IMO. Even though I don't see how to do that without generating more confusion than it's worth at the moment, I still harbor the hope that somebody will come up with a way to do it so everything still fits together. Footnotes: [1] I am *not* one to damn RFCs lightly! From guido at python.org Tue Apr 23 19:24:53 2013 From: guido at python.org (Guido van Rossum) Date: Tue, 23 Apr 2013 10:24:53 -0700 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <5176B115.8050101@egenix.com> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <20130423111523.31af429f@limelight.wooz.org> <5176A73B.1050809@egenix.com> <5176B115.8050101@egenix.com> Message-ID: On Tue, Apr 23, 2013 at 9:04 AM, M.-A. Lemburg wrote: > On 23.04.2013 17:47, Guido van Rossum wrote: >> On Tue, Apr 23, 2013 at 8:22 AM, M.-A. Lemburg wrote: >>> Just as reminder: we have the general purpose >>> encode()/decode() functions in the codecs module: >>> >>> import codecs >>> r13 = codecs.encode('hello world', 'rot-13') >>> >>> These interface directly to the codec interfaces, without >>> enforcing type restrictions. The codec defines the supported >>> input and output types. >> >> As an implementation mechanism I see nothing wrong with this. I hope >> the codecs module lets you introspect the input and output types of a >> codec given by name? > > At the moment there is no standard interface to access supported > input and output types... but then: regular Python functions or > methods also don't provide such functionality, so no surprise > there ;-) Not quite the same though. Each function has its own unique behavior. But codecs support a standard interface, *except* that the input and output types sometimes vary. > It's mostly a matter of specifying the supported type > combinations in the codec documentation. > > BTW: What would be a use case where you'd want to > programmatically access such information before calling > the codec ? As you know, in Python 3, most code working with bytes doesn't also work with strings, and vice versa (except for a few cases where we've gone out of our way to write polymorphic code -- but users rarely do so, and any time you use a string or bytes literal you basically limit yourself to that type). Suppose I write a command-line utility that reads a file, runs it through a codec, and writes the result to another file. Suppose the name of the codec is a command-line argument (as well as the filenames). I need to know whether to open the files in text or binary mode based on the name of the codec. -- --Guido van Rossum (python.org/~guido) From rdmurray at bitdance.com Tue Apr 23 20:05:20 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 23 Apr 2013 14:05:20 -0400 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <87ehe1yyxo.fsf@uwakimon.sk.tsukuba.ac.jp> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <20130423141602.47FAC250BCA@webabinitio.net> <87ehe1yyxo.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130423180520.B621D250BCA@webabinitio.net> On Wed, 24 Apr 2013 01:49:39 +0900, "Stephen J. Turnbull" wrote: > R. David Murray writes: > > On Tue, 23 Apr 2013 22:29:33 +0900, "Stephen J. Turnbull" wrote: > > > R. David Murray writes: > > > > > > > You transform *into* the encoding, and untransform *out* of the > > > > encoding. Do you have an example where that would be ambiguous? > > > > > > In the bytes-to-bytes case, any pair of character encodings (eg, UTF-8 > > > and ISO-8859-15) would do. Or how about in text, ReST to HTML? > > > > If I write: > > > > bytestring.transform('ISO-8859-15') > > > > that would indeed be ambiguous, but only because I haven't named the > > source encoding of the bytestring. So the above is obviously > > nonsense, and the easiest "fix" is to have the things that are currently > > bytes-to-text or text-to-bytes character set transformations *only* > > work with encode/decode, and not transform/untransform. > > I think you're completely missing my point here. The problem is that > in the cases I mention, what is encoded data and what is decoded data > can only be decided by asking the user. I think I understood that. I don't understand why that's a problem. (But see below.) > > Given this, the possible valid transformations would be: > > > > bytestring.transform('base64') > > bytesstring.untransform('base64') > > string.untransform('base64') > > Which is an obnoxious API, since (1) you've now made it impossible to > use "transform" for > > bytestring.transform(from='utf-8', to='iso-8859-1') > bytestring.transform(from='ulaw', to='mp3') > textstring.transform(from='rest', to='html') > > without confusion, and (2) the whole world is going to wonder why you > don't use .encode and .decode instead of .transform and .untransform. I've been trying to explain what I thought the transform/untransform proposal was: a minimalist extension of the encode/decode semantic (under a different name) so that functionality that was lost from Python2 encode/decode could be restored to Python3 in a reasonably understandable way. This would be a *limited* convenience function, just as encode/decode are limited convenience functions with respect to the full power of the codecs module. I myself don't have any real investment in the proposal, or I would have long since tried to push the tracker issue forward. People (at least you and Nick, and maybe Guido) seem to be more interested in a more general/powerful mechanism. I'm fine with that :) --David From tjreedy at udel.edu Tue Apr 23 20:07:39 2013 From: tjreedy at udel.edu (Terry Jan Reedy) Date: Tue, 23 Apr 2013 14:07:39 -0400 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <87ehe1yyxo.fsf@uwakimon.sk.tsukuba.ac.jp> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <20130423141602.47FAC250BCA@webabinitio.net> <87ehe1yyxo.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On 4/23/2013 12:49 PM, Stephen J. Turnbull wrote: > Which is an obnoxious API, since (1) you've now made it impossible to > use "transform" for > > bytestring.transform(from='utf-8', to='iso-8859-1') > bytestring.transform(from='ulaw', to='mp3') > textstring.transform(from='rest', to='html') > > without confusion, and (2) the whole world is going to wonder why you > don't use .encode and .decode instead of .transform and .untransform. I think the unambiguous solution is to get rid of the notion of 'untransform' (which only means 'transform in the other direction'), since it requires and presumes an asymmetry that is not always present. It it precisely the lack of asymmetry in examples like the above that makes the transform/untransform pair ambiguous as to which is which. .transform should be explicit and always take two args, no implicit defaults, the 'from form' and the 'to' form. They can labelled by position in the natural order (from, to) or by keyword, as in your examples. For text, the plain undifferentiated form which one might think of as default could be called 'text' and that for bytes 'bytes' (as you suggest) or 'ascii' as appropriate. str.transform would always be unicode to unicode and bytes.transform always bytes to bytes. From pjenvey at underboss.org Tue Apr 23 22:20:48 2013 From: pjenvey at underboss.org (Philip Jenvey) Date: Tue, 23 Apr 2013 13:20:48 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423165723.78e286a5@pitrou.net> Message-ID: <7BF5B3DE-E84C-4FD5-8F12-8967A5A37590@underboss.org> On Apr 23, 2013, at 8:11 AM, Guido van Rossum wrote: > I gotta say, I'm with Antoine here. It's pretty natural (also coming > from other languages) to assume that the class used to define the > enums is also the type of the enum values. Certainly this is how it > works in Java and C++, and I would say it's the same in Pascal and > probably most other languages. Furthermore, you could define methods/fields on enum values, like Java's enums. E.g.: Errno.EBADF.strerror() -> 'Bad file descriptor' Easily adapt to additional fields: class Protocol(Enum): HOPOPT = 0, "IPv6 Hop-by-Hop Option" ICMP = 1, "Internet Control Message" IGMP = 2, "Internet Group Management" @property def value(self): return self._value[0] @property def description(self): return self._value[1] -- Philip Jenvey From ncoghlan at gmail.com Tue Apr 23 23:37:29 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 24 Apr 2013 07:37:29 +1000 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <5176A73B.1050809@egenix.com> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <20130423111523.31af429f@limelight.wooz.org> <5176A73B.1050809@egenix.com> Message-ID: On 24 Apr 2013 01:25, "M.-A. Lemburg" wrote: > > On 23.04.2013 17:15, Barry Warsaw wrote: > > On Apr 22, 2013, at 06:22 PM, Guido van Rossum wrote: > > > >>> You can ask the same question about all the other codecs. (And that > >>> question has indeed been asked in the past.) > >> > >> Except for rot13. :-) > > > > The fact that you can do this instead *is* a bit odd. ;) > > > > from codecs import getencoder > > encoder = getencoder('rot-13') > > r13 = encoder('hello world')[0] > > Just as reminder: we have the general purpose > encode()/decode() functions in the codecs module: > > import codecs > r13 = codecs.encode('hello world', 'rot-13') > > These interface directly to the codec interfaces, without > enforcing type restrictions. The codec defines the supported > input and output types. If we already have those, why aren't they documented? If they exist, they should be the first thing in the codecs module docs and the porting guide should list them as the replacement for the method versions when using encodings that aren't directly related to the text model, or when the input buffer for decoding isn't a bytes or bytearray object. Regards, Nick. > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Apr 23 2013) > >>> Python Projects, Consulting and Support ... http://www.egenix.com/ > >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ > >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > 2013-04-17: Released eGenix mx Base 3.2.6 ... http://egenix.com/go43 > > ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From scott+python-dev at scottdial.com Wed Apr 24 00:48:31 2013 From: scott+python-dev at scottdial.com (Scott Dial) Date: Tue, 23 Apr 2013 18:48:31 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130423163302.195e4587@pitrou.net> <20130423145355.2F527250BCA@webabinitio.net> <5176ACEA.7060903@scottdial.com> Message-ID: <51770FBF.2080006@scottdial.com> On 4/23/2013 11:58 AM, Guido van Rossum wrote: > You seem to be mixing up classes and metaclasses. I was trying to explain it in more plain terms, but maybe I made it even more confusing.. Anyways, I agree with you that isinstance(Color.RED, Color) should be True. -- Scott Dial scott at scottdial.com From greg.ewing at canterbury.ac.nz Wed Apr 24 01:19:44 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 24 Apr 2013 11:19:44 +1200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <51771710.6050107@canterbury.ac.nz> Stephen J. Turnbull wrote: > By RFC specification, BASE64 is a > *textual* representation of arbitrary binary data. (Cf. URIs.) The > natural interpretation of .encode('base64') in that context would be > as a bytes-to-text encoder. However, ... In > practice, we invariably use an ASCII octet stream to carry BASE64- > encoded data. As an aside, if we'd had the flexible string representation sooner, this needn't have been such a big problem. With it, the base64 encoder could return a unicode string with 8-bit representation, which could then be turned into an ascii byte string with negligible overhead. Web developers might grumble about the need for an extra call, but they can no longer claim it would kill the performance of their web server. -- Greg From greg.ewing at canterbury.ac.nz Wed Apr 24 01:37:16 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 24 Apr 2013 11:37:16 +1200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130423142418.652DE250BCA@webabinitio.net> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> Message-ID: <51771B2C.3070003@canterbury.ac.nz> R. David Murray wrote: > The first False looks correct to me, I would not expect an enum value to > be an instance of the class that holds it as an attribute. The second > certainly looks odd, but what does it even mean to have an instance of > an Enum class? This attitude baffles me. In most other languages having a notion of an enum, when you define an enum, you're defining a type. The name of the enum is the name of the type, and its values are instances of that type. Why should our enums be any different? -- Greg From rdmurray at bitdance.com Wed Apr 24 04:12:22 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 23 Apr 2013 22:12:22 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <51771B2C.3070003@canterbury.ac.nz> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <51771B2C.3070003@canterbury.ac.nz> Message-ID: <20130424021222.C08FE250BCA@webabinitio.net> On Wed, 24 Apr 2013 11:37:16 +1200, Greg Ewing wrote: > R. David Murray wrote: > > The first False looks correct to me, I would not expect an enum value to > > be an instance of the class that holds it as an attribute. The second > > certainly looks odd, but what does it even mean to have an instance of > > an Enum class? > > This attitude baffles me. In most other languages having a > notion of an enum, when you define an enum, you're defining > a type. The name of the enum is the name of the type, and > its values are instances of that type. > > Why should our enums be any different? Obviously they don't need to be, since people have discussed how to implement this. But I am primarily a Python programmer, so my intuition is based on my Python experience, not on what other languages do. Given: MyEnum(Enum): a = 1 b = 2 I understood that 'a' and 'b' are class attributes, where the int value had been transformed into instances of a (separate) value class rather than being ints. The fact that there was a specific value class had been discussed. If 'a' is now an instance of MyEnum, then I would expect that: MyEnum.a.b would be valid (b being an attribute of the MyEnum class which should therefore be accessible from an instance of that class). That seems a bit odd, and based on my Python-only mindset, I saw no particular reason why an enum value *should* be instance of the enum class, since it would lead to that oddity. (Well, I'm not sure I was concious of that *particular* oddity, but in general it seemed like an odd thing to have class attributes of a class be instances of that class when the set of class attributes was the most important thing about that class...). It seemed more natural for the values to be instances of a separate value class. Now, given that I was viewing the Enum as being a collection of attributes whose values were instances of a different class, what would it mean to create an instance of the Enum class itself? You'd have an instance with access to those class attributes...but the getitem wouldn't work, because that's on the metaclass. You'd really want the Enum class to be a singleton...the important thing was that it was an instance of the metaclass, its instances would be pretty useless. I don't have any *problem* with enum values being instances of the class. If you make the attribute values instances of the enum class, then yes instances of enum class have a meaning. And then having attributes of the class be instances of the class makes perfect sense in hindsight. It's just not where *my* Python-only intuition, or my understanding of the discussion, led me. I feel like I'm revealing my ignorance or stupidity here, but what the heck, that's what was going through my brain and I might as well own up to it :). --David From stephen at xemacs.org Wed Apr 24 04:23:42 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 24 Apr 2013 11:23:42 +0900 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <20130423180520.B621D250BCA@webabinitio.net> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <20130423141602.47FAC250BCA@webabinitio.net> <87ehe1yyxo.fsf@uwakimon.sk.tsukuba.ac.jp> <20130423180520.B621D250BCA@webabinitio.net> Message-ID: <87a9oozmxd.fsf@uwakimon.sk.tsukuba.ac.jp> R. David Murray writes: > > I think you're completely missing my point here. The problem is that > > in the cases I mention, what is encoded data and what is decoded data > > can only be decided by asking the user. > > I think I understood that. I don't understand why that's a > problem. It's a problem because in that case it's hard for users to remember the directionality of the codec based only on a single name; the API needs to indicate what is being transformed to what else. > I've been trying to explain what I thought the transform/untransform > proposal was: a minimalist extension of the encode/decode semantic > (under a different name) so that functionality that was lost from > Python2 encode/decode could be restored to Python3 in a reasonably > understandable way. I think that the intention of the proposal is reasonably understandable, and reasonable. I just don't think the API proposed is understandable, and therefore it's not reasonable. > People (at least you and Nick, and maybe Guido) seem to be more > interested in a more general/powerful mechanism. I'm fine with > that :) I can't speak to the opinions of people who actually know about language design. For myself, I'm sympathetic to the proposal of a specific API limited to cases where the directionality is clear as a generality. I just don't think the "transform" proposal helps much, partly because the actual applications are few, and partly because "transform" is more ambiguous (to be unambiguous in English, you need both the direct object ("from media type") and the indirect object ("to media type") specified. It is quite possible to say "transform encoded text to raw text" or similar. At least for me, "encode transformed text to raw text" raises a WTFAssertion. I know that I've experienced worlds of pain in the character coding sphere from Emacs APIs and UIs that don't indicate directionality clearly. This is very delicate; GNU Emacs had an ugly bug that regressed multiple times over more than a decade merely because they exposed the internal representation of text to Lisp. XEmacs has never experienced that bug (to be precise, the presence of that kind of bug resulted in an immediate assertion, so it was eliminated early in development). Surprisingly to me, the fact that XEmacs uses the internal representation of *text* to also represent "byte streams" (with bytes of variable width!) has never caused me confusion. It does cause others confusion, though, so although the XEmacs model of text is easier to work with than Emacs's, I tend to think Python 3's (which never confounds text with bytes) is better. I suspect that delicacy extends to non-character transformations, so I am pretty demanding about proposals in this area. Specifically I insist on EIBTI and TOOWTDI. From stephen at xemacs.org Wed Apr 24 04:40:11 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 24 Apr 2013 11:40:11 +0900 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <51771710.6050107@canterbury.ac.nz> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <51771710.6050107@canterbury.ac.nz> Message-ID: <878v48zm5w.fsf@uwakimon.sk.tsukuba.ac.jp> Greg Ewing writes: > Web developers might grumble about the need for an extra call, > but they can no longer claim it would kill the performance of > their web server. Of course they can. There never was any performance measurement that supported that claim in the first place. I don't see how PEP 393 makes a difference to them. The real problem for them is that conceptually they think ASCII in byte form *is* text, and they want to do text processing on it. They'll use any flimsy excuse to avoid a transform to str, because it's just unbearably ugly given their givens. I have sympathy for their position, I just (even today) think it's the wrong thing for Python. However, I've long since been overruled, and I have no evidence to justify saying "I told you so". From stephen at xemacs.org Wed Apr 24 04:45:43 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 24 Apr 2013 11:45:43 +0900 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <20130423141602.47FAC250BCA@webabinitio.net> <87ehe1yyxo.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <877gjszlwo.fsf@uwakimon.sk.tsukuba.ac.jp> Terry Jan Reedy writes: > .transform should be explicit and always take two args, no implicit > defaults, the 'from form' and the 'to' form. They can labelled by > position in the natural order (from, to) Not natural to escaped-from-C programmers, though. I hesitate to say "make it keywords-only", but using keywords should be *strongly* encouraged. > str.transform would always be unicode to unicode and bytes.transform > always bytes to bytes. Which leaves the salient cases (MIME content transfer encodings) out in the cold, although I guess string.encode('ascii').transform(from='base64', to='bytes') isn't too horrible. From guido at python.org Wed Apr 24 05:11:35 2013 From: guido at python.org (Guido van Rossum) Date: Tue, 23 Apr 2013 20:11:35 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130424021222.C08FE250BCA@webabinitio.net> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <51771B2C.3070003@canterbury.ac.nz> <20130424021222.C08FE250BCA@webabinitio.net> Message-ID: On Tue, Apr 23, 2013 at 7:12 PM, R. David Murray wrote: > On Wed, 24 Apr 2013 11:37:16 +1200, Greg Ewing wrote: >> R. David Murray wrote: >> > The first False looks correct to me, I would not expect an enum value to >> > be an instance of the class that holds it as an attribute. The second >> > certainly looks odd, but what does it even mean to have an instance of >> > an Enum class? >> >> This attitude baffles me. In most other languages having a >> notion of an enum, when you define an enum, you're defining >> a type. The name of the enum is the name of the type, and >> its values are instances of that type. >> >> Why should our enums be any different? > > Obviously they don't need to be, since people have discussed how to > implement this. > > But I am primarily a Python programmer, so my intuition is based on my > Python experience, not on what other languages do. > > Given: > > MyEnum(Enum): > a = 1 > b = 2 > > I understood that 'a' and 'b' are class attributes, where the int value > had been transformed into instances of a (separate) value class rather > than being ints. The fact that there was a specific value class > had been discussed. > > If 'a' is now an instance of MyEnum, then I would expect that: > > MyEnum.a.b > > would be valid (b being an attribute of the MyEnum class which should > therefore be accessible from an instance of that class). That seems > a bit odd, and based on my Python-only mindset, I saw no particular > reason why an enum value *should* be instance of the enum class, since > it would lead to that oddity. (Well, I'm not sure I was concious of > that *particular* oddity, but in general it seemed like an odd thing to > have class attributes of a class be instances of that class when the set > of class attributes was the most important thing about that class...). > It seemed more natural for the values to be instances of a separate > value class. > > Now, given that I was viewing the Enum as being a collection of attributes > whose values were instances of a different class, what would it mean > to create an instance of the Enum class itself? You'd have an instance > with access to those class attributes...but the getitem wouldn't work, > because that's on the metaclass. You'd really want the Enum class to > be a singleton...the important thing was that it was an instance of the > metaclass, its instances would be pretty useless. > > I don't have any *problem* with enum values being instances of the class. > If you make the attribute values instances of the enum class, then > yes instances of enum class have a meaning. And then having attributes > of the class be instances of the class makes perfect sense in > hindsight. > > It's just not where *my* Python-only intuition, or my understanding of > the discussion, led me. > > I feel like I'm revealing my ignorance or stupidity here, but what the > heck, that's what was going through my brain and I might as well own up > to it :). I'm not too worried. Clearly the Enum base class is magic, because, contrary to the behavior of regular class definitions, MyEnum.a is *not* just the integer 1 (even if it compares == to 1). At that point we may as well accept that the entire construct is magic, and the only consequences we can reason about are those that are documented as consequences of using this particular kind of magic. I'm not sure that we can implement a hack to prevent MyEnum.a.b from being an alias for MyEnum.b, but even if we cannot prevent that, I don't think we need to worry about it. (However, I suspect we can override __getattr__ to prevent this if it is considered too unsavory.) -- --Guido van Rossum (python.org/~guido) From ronaldoussoren at mac.com Wed Apr 24 08:14:13 2013 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 24 Apr 2013 08:14:13 +0200 Subject: [Python-Dev] [Python-checkins] cpython (3.3): backported rev 79713 from 3.4, test_recursion_limit skipped for -O0 In-Reply-To: <3ZwLYM24xjz7Lrl@mail.python.org> References: <3ZwLYM24xjz7Lrl@mail.python.org> Message-ID: <7A37BC47-1391-4ACA-8756-241844EEF2E7@mac.com> On 24 Apr, 2013, at 1:32, "lukasz.langa" wrote: > http://hg.python.org/cpython/rev/9755036c81d0 > changeset: 83510:9755036c81d0 > branch: 3.3 > parent: 83508:44d764238f0d > user: ?ukasz Langa > date: Wed Apr 24 01:29:26 2013 +0200 > summary: > backported rev 79713 from 3.4, test_recursion_limit skipped for -O0 > > files: > Lib/test/test_threading.py | 3 ++- > 1 files changed, 2 insertions(+), 1 deletions(-) > > > diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py > --- a/Lib/test/test_threading.py > +++ b/Lib/test/test_threading.py > @@ -754,7 +754,8 @@ > lock = threading.Lock() > self.assertRaises(RuntimeError, lock.release) > > - @unittest.skipUnless(sys.platform == 'darwin', 'test macosx problem') > + @unittest.skipUnless(sys.platform == 'darwin' and test.support.python_is_optimized(), > + 'test macosx problem') Wouldn't it be better to just fix the issue? thread_pthread already sets an explicit stack size on OSX, but that value is appearently too small. Ronald From ronaldoussoren at mac.com Wed Apr 24 08:44:38 2013 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 24 Apr 2013 08:44:38 +0200 Subject: [Python-Dev] [Python-checkins] cpython (3.3): backported rev 79713 from 3.4, test_recursion_limit skipped for -O0 In-Reply-To: <7A37BC47-1391-4ACA-8756-241844EEF2E7@mac.com> References: <3ZwLYM24xjz7Lrl@mail.python.org> <7A37BC47-1391-4ACA-8756-241844EEF2E7@mac.com> Message-ID: On 24 Apr, 2013, at 8:14, Ronald Oussoren wrote: > > On 24 Apr, 2013, at 1:32, "lukasz.langa" wrote: > >> http://hg.python.org/cpython/rev/9755036c81d0 >> changeset: 83510:9755036c81d0 >> branch: 3.3 >> parent: 83508:44d764238f0d >> user: ?ukasz Langa >> date: Wed Apr 24 01:29:26 2013 +0200 >> summary: >> backported rev 79713 from 3.4, test_recursion_limit skipped for -O0 >> >> files: >> Lib/test/test_threading.py | 3 ++- >> 1 files changed, 2 insertions(+), 1 deletions(-) >> >> >> diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py >> --- a/Lib/test/test_threading.py >> +++ b/Lib/test/test_threading.py >> @@ -754,7 +754,8 @@ >> lock = threading.Lock() >> self.assertRaises(RuntimeError, lock.release) >> >> - @unittest.skipUnless(sys.platform == 'darwin', 'test macosx problem') >> + @unittest.skipUnless(sys.platform == 'darwin' and test.support.python_is_optimized(), >> + 'test macosx problem') > > Wouldn't it be better to just fix the issue? thread_pthread already sets an explicit stack size on OSX, but that value is appearently too small. In particular, this patch appears to fix the crash that's the reason for disabling the test: diff --git a/Python/thread_pthread.h b/Python/thread_pthread.h --- a/Python/thread_pthread.h +++ b/Python/thread_pthread.h @@ -28,7 +28,7 @@ */ #if defined(__APPLE__) && defined(THREAD_STACK_SIZE) && THREAD_STACK_SIZE == 0 #undef THREAD_STACK_SIZE -#define THREAD_STACK_SIZE 0x500000 +#define THREAD_STACK_SIZE 0x550000 #endif #if defined(__FreeBSD__) && defined(THREAD_STACK_SIZE) && THREAD_STACK_SIZE == 0 #undef THREAD_STACK_SIZE Without this patch test_recursion_limit fails due to a crash, with the patch the test passes (debug build, x86_64, OSX 10.8.3). Ronald From mal at egenix.com Wed Apr 24 09:51:31 2013 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 24 Apr 2013 09:51:31 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <20130423111523.31af429f@limelight.wooz.org> <5176A73B.1050809@egenix.com> Message-ID: <51778F03.8090009@egenix.com> On 23.04.2013 23:37, Nick Coghlan wrote: > On 24 Apr 2013 01:25, "M.-A. Lemburg" wrote: >> >> On 23.04.2013 17:15, Barry Warsaw wrote: >>> On Apr 22, 2013, at 06:22 PM, Guido van Rossum wrote: >>> >>>>> You can ask the same question about all the other codecs. (And that >>>>> question has indeed been asked in the past.) >>>> >>>> Except for rot13. :-) >>> >>> The fact that you can do this instead *is* a bit odd. ;) >>> >>> from codecs import getencoder >>> encoder = getencoder('rot-13') >>> r13 = encoder('hello world')[0] >> >> Just as reminder: we have the general purpose >> encode()/decode() functions in the codecs module: >> >> import codecs >> r13 = codecs.encode('hello world', 'rot-13') >> >> These interface directly to the codec interfaces, without >> enforcing type restrictions. The codec defines the supported >> input and output types. > > If we already have those, why aren't they documented? Good question. I added them in 2004 and probably just forgot to add the documentation: http://hg.python.org/cpython-fullhistory/rev/8ea2cb1ec598 I guess the doc-strings could be used as basis for the documentation. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 24 2013) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2013-04-17: Released eGenix mx Base 3.2.6 ... http://egenix.com/go43 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mal at egenix.com Wed Apr 24 10:22:42 2013 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 24 Apr 2013 10:22:42 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <20130423111523.31af429f@limelight.wooz.org> <5176A73B.1050809@egenix.com> <5176B115.8050101@egenix.com> Message-ID: <51779652.6030504@egenix.com> On 23.04.2013 19:24, Guido van Rossum wrote: > On Tue, Apr 23, 2013 at 9:04 AM, M.-A. Lemburg wrote: >> On 23.04.2013 17:47, Guido van Rossum wrote: >>> On Tue, Apr 23, 2013 at 8:22 AM, M.-A. Lemburg wrote: >>>> Just as reminder: we have the general purpose >>>> encode()/decode() functions in the codecs module: >>>> >>>> import codecs >>>> r13 = codecs.encode('hello world', 'rot-13') >>>> >>>> These interface directly to the codec interfaces, without >>>> enforcing type restrictions. The codec defines the supported >>>> input and output types. >>> >>> As an implementation mechanism I see nothing wrong with this. I hope >>> the codecs module lets you introspect the input and output types of a >>> codec given by name? >> >> At the moment there is no standard interface to access supported >> input and output types... but then: regular Python functions or >> methods also don't provide such functionality, so no surprise >> there ;-) > > Not quite the same though. Each function has its own unique behavior. > But codecs support a standard interface, *except* that the input and > output types sometimes vary. The codec system itself >> It's mostly a matter of specifying the supported type >> combinations in the codec documentation. >> >> BTW: What would be a use case where you'd want to >> programmatically access such information before calling >> the codec ? > > As you know, in Python 3, most code working with bytes doesn't also > work with strings, and vice versa (except for a few cases where we've > gone out of our way to write polymorphic code -- but users rarely do > so, and any time you use a string or bytes literal you basically limit > yourself to that type). > > Suppose I write a command-line utility that reads a file, runs it > through a codec, and writes the result to another file. Suppose the > name of the codec is a command-line argument (as well as the > filenames). I need to know whether to open the files in text or binary > mode based on the name of the codec. Ok, so you need to know which codecs your tool can support and which of those need text input and which bytes input. I've been thinking about this some more: I think that type information alone is not flexible enough to cover such use cases. In your use case you'd want to only permit use of a certain set of codecs, not simply all of them, since some might not implement what you actually want to achieve with the tool, e.g. a user might have installed a codec set that adds support for reading and writing image data, but your intended use was to only support text data. So what we need is a way to allow the codecs to say e.g. "I work on text", "I support encoding bytes and text", "I encode to bytes", "I'm reversible", "I transform input data", "I support bytes and text, and will create same type output", "I work on image data", "I work on X509 certificates", "I work on XML data", etc. In other words, we need a form of tagging system, with a set of standard tags that each codec can publish and which also allows non-standard tags (which can then at some point be made standard, if there's agreement on them). Given a codec name you could then ask the codec registry for the codec tags and verify that the chosen codec handles text data, needs bytes or text encoding input and creates bytes as encoding output. If the registry returns codec tags that don't include the "I work on text" tag, the tool could then raise an error. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 24 2013) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2013-04-17: Released eGenix mx Base 3.2.6 ... http://egenix.com/go43 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From v+python at g.nevcal.com Wed Apr 24 11:18:06 2013 From: v+python at g.nevcal.com (Glenn Linderman) Date: Wed, 24 Apr 2013 02:18:06 -0700 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <51779652.6030504@egenix.com> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <20130423111523.31af429f@limelight.wooz.org> <5176A73B.1050809@egenix.com> <5176B115.8050101@egenix.com> <51779652.6030504@egenix.com> Message-ID: <5177A34E.4020903@g.nevcal.com> On 4/24/2013 1:22 AM, M.-A. Lemburg wrote: > On 23.04.2013 19:24, Guido van Rossum wrote: >> On Tue, Apr 23, 2013 at 9:04 AM, M.-A. Lemburg wrote: >>> On 23.04.2013 17:47, Guido van Rossum wrote: >>>> On Tue, Apr 23, 2013 at 8:22 AM, M.-A. Lemburg wrote: >>>>> Just as reminder: we have the general purpose >>>>> encode()/decode() functions in the codecs module: >>>>> >>>>> import codecs >>>>> r13 = codecs.encode('hello world', 'rot-13') >>>>> >>>>> These interface directly to the codec interfaces, without >>>>> enforcing type restrictions. The codec defines the supported >>>>> input and output types. >>>> As an implementation mechanism I see nothing wrong with this. I hope >>>> the codecs module lets you introspect the input and output types of a >>>> codec given by name? >>> At the moment there is no standard interface to access supported >>> input and output types... but then: regular Python functions or >>> methods also don't provide such functionality, so no surprise >>> there ;-) >> Not quite the same though. Each function has its own unique behavior. >> But codecs support a standard interface, *except* that the input and >> output types sometimes vary. > The codec system itself > >>> It's mostly a matter of specifying the supported type >>> combinations in the codec documentation. >>> >>> BTW: What would be a use case where you'd want to >>> programmatically access such information before calling >>> the codec ? >> As you know, in Python 3, most code working with bytes doesn't also >> work with strings, and vice versa (except for a few cases where we've >> gone out of our way to write polymorphic code -- but users rarely do >> so, and any time you use a string or bytes literal you basically limit >> yourself to that type). >> >> Suppose I write a command-line utility that reads a file, runs it >> through a codec, and writes the result to another file. Suppose the >> name of the codec is a command-line argument (as well as the >> filenames). I need to know whether to open the files in text or binary >> mode based on the name of the codec. > Ok, so you need to know which codecs your tool can support and > which of those need text input and which bytes input. > > I've been thinking about this some more: I think that type > information alone is not flexible enough to cover such > use cases. Maybe MIME type and encoding would be sufficient type information, but probably not str vs. bytes. > In your use case you'd want to only permit use of a certain > set of codecs, not simply all of them, since some might > not implement what you actually want to achieve with the tool, > e.g. a user might have installed a codec set that adds > support for reading and writing image data, but your > intended use was to only support text data. MIME type supports this sort of concept, with the two-level hierarchy of naming the type... text/xml text/plain image/jpeg > So what we need is a way to allow the codecs to say e.g. > "I work on text", "I support encoding bytes and text", > "I encode to bytes", "I'm reversible", "I transform > input data", "I support bytes and text, and will create > same type output", "I work on image data", "I work on > X509 certificates", "I work on XML data", etc. Guess what I think you are re-inventing here.... Nope, guess again.... Yep, MIME types _plus_ encodings. > In other words, we need a form of tagging system, with a > set of standard tags that each codec can publish and > which also allows non-standard tags (which can then at > some point be made standard, if there's agreement on them). Hmm. Sounds just like the registry for, um, you guessed it: MIME types. > Given a codec name you could then ask the codec registry for > the codec tags and verify that the chosen codec handles > text data, needs bytes or text encoding input and > creates bytes as encoding output. If the registry returns > codec tags that don't include the "I work on text" tag, > the tool could then raise an error. For just doing text encoding transformations, text/plain would work as a MIME type, and the encodings of interest for the encodings. Seems like "str" always means "Unicode" but the MIME type can vary; "bytes" might mean encoded text, and the MIME type can also vary. For non-textual transformations, "encoding" might mean Base 64, BinHex, or other such representations... but those can also be applied to text, so it might be a 3rd dimension, or it might just be a list of encodings rather than a single encoding. Compression could be another dimension, or perhaps another encoding. But really, then, a transformation needs to be a list of steps; a codec can sign up to perform one or more of the steps, a sequence of codecs would have to be found, capable of performing a subsequence of the steps, and then run in the appropriate order. This all sounds so general, that probably the Python compiler could be implemented as a codec :) Or any compiler. Probably a web server could be implemented as a codec too :) Well, maybe not, codecs have limited error handling and reporting abilities. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tseaver at palladion.com Wed Apr 24 19:01:10 2013 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 24 Apr 2013 13:01:10 -0400 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/23/2013 09:29 AM, Stephen J. Turnbull wrote: > By RFC specification, BASE64 is a *textual* representation of > arbitrary binary data. It isn't "text" in the sense Py3k means: it is a representation for transmission on-the-wire for protocols which requre 7-bit-safe data. Nobody working with base64-encoded data is going to expect to do "normal" string processing on that data: the closest thing to that is splitting it into 72-byte chunks for transmission via e-mail. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlF4D9YACgkQ+gerLs4ltQ5nUACfWm4YEMarjUb7fEEpP+aMtaQr a7kAn1Pc8ufUwJzKHD0DgSxQ4H/uqf82 =CzTZ -----END PGP SIGNATURE----- From allyourcode at gmail.com Wed Apr 24 20:35:49 2013 From: allyourcode at gmail.com (Daniel Wong) Date: Wed, 24 Apr 2013 11:35:49 -0700 Subject: [Python-Dev] I cannot create bug reports Message-ID: Glorious members of python-dev, I'd like to submit a patch, but I cannot create a bug report. As of this morning (US West Coast), when I go to http://bugs.python.org/issue?@template=item I get no form fields. I went there last night, and I was able to get a form. I kept that tab open over night, and tried to submit this morning. When I did that, I got permission denied errors. It seems that something weird has happened to my account, or bug tracker itself changed in my sleep. Anyone have any idea what's going on here? Daniel -------------- next part -------------- An HTML attachment was scrubbed... URL: From graffatcolmingov at gmail.com Wed Apr 24 20:45:26 2013 From: graffatcolmingov at gmail.com (Ian Cordasco) Date: Wed, 24 Apr 2013 14:45:26 -0400 Subject: [Python-Dev] I cannot create bug reports In-Reply-To: References: Message-ID: The first thing that comes to mind is that your session expired and you need to log-in again. After logging in myself I see the form in all of it's glory. On Wed, Apr 24, 2013 at 2:35 PM, Daniel Wong wrote: > Glorious members of python-dev, > > I'd like to submit a patch, but I cannot create a bug report. As of this > morning (US West Coast), when I go to > http://bugs.python.org/issue?@template=item I get no form fields. > > I went there last night, and I was able to get a form. I kept that tab open > over night, and tried to submit this morning. When I did that, I got > permission denied errors. It seems that something weird has happened to my > account, or bug tracker itself changed in my sleep. > > Anyone have any idea what's going on here? > > Daniel > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/graffatcolmingov%40gmail.com > From allyourcode at gmail.com Wed Apr 24 20:55:19 2013 From: allyourcode at gmail.com (Daniel Wong) Date: Wed, 24 Apr 2013 11:55:19 -0700 Subject: [Python-Dev] I cannot create bug reports In-Reply-To: References: Message-ID: Thank you. That was the problem. I feel kind of stupid now. In my defense, the error message could have been more helpful, and requesting the bug creation form could have thrown up a login error instead of showing up blank. File another bug? On Wed, Apr 24, 2013 at 11:45 AM, Ian Cordasco wrote: > The first thing that comes to mind is that your session expired and > you need to log-in again. After logging in myself I see the form in > all of it's glory. > > On Wed, Apr 24, 2013 at 2:35 PM, Daniel Wong > wrote: > > Glorious members of python-dev, > > > > I'd like to submit a patch, but I cannot create a bug report. As of this > > morning (US West Coast), when I go to > > http://bugs.python.org/issue?@template=item I get no form fields. > > > > I went there last night, and I was able to get a form. I kept that tab > open > > over night, and tried to submit this morning. When I did that, I got > > permission denied errors. It seems that something weird has happened to > my > > account, or bug tracker itself changed in my sleep. > > > > Anyone have any idea what's going on here? > > > > Daniel > > > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > > > http://mail.python.org/mailman/options/python-dev/graffatcolmingov%40gmail.com > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian at python.org Wed Apr 24 21:50:00 2013 From: brian at python.org (Brian Curtin) Date: Wed, 24 Apr 2013 14:50:00 -0500 Subject: [Python-Dev] I cannot create bug reports In-Reply-To: References: Message-ID: On Wed, Apr 24, 2013 at 1:55 PM, Daniel Wong wrote: > Thank you. That was the problem. > > I feel kind of stupid now. In my defense, the error message could have been > more helpful, and requesting the bug creation form could have thrown up a > login error instead of showing up blank. File another bug? Bugs about the bug tracker go to http://psf.upfronthosting.co.za/roundup/meta/ From ether.joe at gmail.com Thu Apr 25 00:32:43 2013 From: ether.joe at gmail.com (Sean Felipe Wolfe) Date: Wed, 24 Apr 2013 15:32:43 -0700 Subject: [Python-Dev] A decade as a core dev In-Reply-To: References: Message-ID: On Thu, Apr 18, 2013 at 8:02 AM, Brett Cannon wrote: > Today marks my 10 year anniversary as a core developer on Python. I > wrote a blog post to mark the occasion > (http://sayspy.blogspot.ca/2013/04/a-decade-of-commits.html), but I > wanted to personally thank python-dev for the past decade (and > whatever comes in the future). All of you taught me how to really > program and for that I will be eternally grateful. And the friendships > I have built through this list are priceless. Congratulations Brett :) I am just getting started on my contribuatory journey and this is good positive reinforcement. Saludos!! From ether.joe at gmail.com Thu Apr 25 01:17:27 2013 From: ether.joe at gmail.com (Sean Felipe Wolfe) Date: Wed, 24 Apr 2013 16:17:27 -0700 Subject: [Python-Dev] slow hg clone of python repo? Message-ID: Hey everybody, I'm trying to download the python sources with hg and it's taking a while ... 7+ minutes so far and all I've got is .../cpython and .../cypython/.hg . Any ideas as to why there's a delay? I'm following the dev guide with this command: hg clone http://hg.python.org/cpython I'm on Linux Mint 14, using the supplied hg version 2.2.2 . My internet connection seems speedy enough. TIA! Sean -- A musician must make music, an artist must paint, a poet must write, if he is to be ultimately at peace with himself. - Abraham Maslow From guido at python.org Thu Apr 25 01:24:15 2013 From: guido at python.org (Guido van Rossum) Date: Wed, 24 Apr 2013 16:24:15 -0700 Subject: [Python-Dev] slow hg clone of python repo? In-Reply-To: References: Message-ID: It's a big repo. Patience. On Wed, Apr 24, 2013 at 4:17 PM, Sean Felipe Wolfe wrote: > Hey everybody, I'm trying to download the python sources with hg and > it's taking a while ... 7+ minutes so far and all I've got is > .../cpython and .../cypython/.hg . Any ideas as to why there's a > delay? > > I'm following the dev guide with this command: > hg clone http://hg.python.org/cpython > > I'm on Linux Mint 14, using the supplied hg version 2.2.2 . My > internet connection seems speedy enough. > > TIA! > Sean > > -- > A musician must make music, an artist must paint, a poet must write, > if he is to be ultimately at peace with himself. > - Abraham Maslow > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) From ether.joe at gmail.com Thu Apr 25 01:37:41 2013 From: ether.joe at gmail.com (Sean Felipe Wolfe) Date: Wed, 24 Apr 2013 16:37:41 -0700 Subject: [Python-Dev] slow hg clone of python repo? In-Reply-To: References: Message-ID: On Wed, Apr 24, 2013 at 4:24 PM, Guido van Rossum wrote: > It's a big repo. Patience. > > On Wed, Apr 24, 2013 at 4:17 PM, Sean Felipe Wolfe wrote: >> Hey everybody, I'm trying to download the python sources with hg and >> it's taking a while ... 7+ minutes so far and all I've got is >> .../cpython and .../cypython/.hg . Any ideas as to why there's a >> delay? >> >> I'm following the dev guide with this command: >> hg clone http://hg.python.org/cpython >> >> I'm on Linux Mint 14, using the supplied hg version 2.2.2 . My >> internet connection seems speedy enough. >> >> TIA! >> Sean Thanks :) It actually completed quickly after I sent the email. :P From greg.ewing at canterbury.ac.nz Thu Apr 25 02:33:07 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 25 Apr 2013 12:33:07 +1200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130424021222.C08FE250BCA@webabinitio.net> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <51771B2C.3070003@canterbury.ac.nz> <20130424021222.C08FE250BCA@webabinitio.net> Message-ID: <517879C3.2050609@canterbury.ac.nz> R. David Murray wrote: > If 'a' is now an instance of MyEnum, then I would expect that: > > MyEnum.a.b > > would be valid That is indeed a quirk, but it's not unprecedented. Exactly the same thing happens in Java. This compiles and runs: enum Foo { a, b } public class Main { public static void main(String[] args) { System.out.printf("%s\n", Foo.a.b); } } There probably isn't much use for that behaviour, but on the other hand, it's probably not worth going out of our way to prevent it. -- Greg From eliben at gmail.com Thu Apr 25 03:14:18 2013 From: eliben at gmail.com (Eli Bendersky) Date: Wed, 24 Apr 2013 18:14:18 -0700 Subject: [Python-Dev] slow hg clone of python repo? In-Reply-To: References: Message-ID: On Wed, Apr 24, 2013 at 4:37 PM, Sean Felipe Wolfe wrote: > On Wed, Apr 24, 2013 at 4:24 PM, Guido van Rossum > wrote: > > It's a big repo. Patience. > > > > On Wed, Apr 24, 2013 at 4:17 PM, Sean Felipe Wolfe > wrote: > >> Hey everybody, I'm trying to download the python sources with hg and > >> it's taking a while ... 7+ minutes so far and all I've got is > >> .../cpython and .../cypython/.hg . Any ideas as to why there's a > >> delay? > >> > >> I'm following the dev guide with this command: > >> hg clone http://hg.python.org/cpython > >> > >> I'm on Linux Mint 14, using the supplied hg version 2.2.2 . My > >> internet connection seems speedy enough. > >> > Sean, 7 minutes doesn't sound bad. Keep in mind that with Hg, the whole repository is being cloned to your computer - all active (and inactive) branches, all history, etc. The up-side is that after this initial clone, subsequent pulls are pretty quick and all other operations are local and super fast (log, blame, etc.) Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Thu Apr 25 03:54:06 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 25 Apr 2013 10:54:06 +0900 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> Tres Seaver writes: > On 04/23/2013 09:29 AM, Stephen J. Turnbull wrote: > > By RFC specification, BASE64 is a *textual* representation of > > arbitrary binary data. > > It isn't "text" in the sense Py3k means: RFC 4648 repeatedly refers to *characters*, without specifying an encoding for them. In fact, if you copy accurately, you can write BASE64 on a napkin and that napkin will accurate transmit the data (assuming it doesn't run into sleet or gloom of night). What else is that but "text in the sense of Py3k"? My point is not that Python's base64 codec *should* be bytes-to-str and back. My point is that, both in the formal spec and in historical evolution, that is a plausible interpretation of ".encode('base64')" which happens to be the reverse of the normal codec convention, where ".encode(codec)" is a *string* method, and ".decode(codec)" is a *bytes* method. This is not harder to learn for people (for BASE64 encoding or for coded character sets), because in each case there's a natural sense of direction for *en*coding vs. *de*coding. But it does break duck- typing, as does the web developer bytes-to-bytes usage of BASE64. What I'm groping toward is an idea of a "variable method", so that we could use .encode and .decode where they are TOOWTDI for people even though a purely formal interpretation of duck-typing would say "but why is that blue whale quacking, waddling, and flying?" In other words (although I have no idea how best to implement it), I would like "somestring.encode('base64')" to fail with "I don't know how to do that" (an attribute lookup error?), the same way that "somebytes.encode('utf-8')" does in Python 3 today. From rdmurray at bitdance.com Thu Apr 25 03:56:56 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 24 Apr 2013 21:56:56 -0400 Subject: [Python-Dev] slow hg clone of python repo? In-Reply-To: References: Message-ID: <20130425015656.BF4D8250BCA@webabinitio.net> On Wed, 24 Apr 2013 18:14:18 -0700, Eli Bendersky wrote: > On Wed, Apr 24, 2013 at 4:37 PM, Sean Felipe Wolfe wrote: > > > On Wed, Apr 24, 2013 at 4:24 PM, Guido van Rossum > > wrote: > > > It's a big repo. Patience. > > > > > > On Wed, Apr 24, 2013 at 4:17 PM, Sean Felipe Wolfe > > wrote: > > >> Hey everybody, I'm trying to download the python sources with hg and > > >> it's taking a while ... 7+ minutes so far and all I've got is > > >> .../cpython and .../cypython/.hg . Any ideas as to why there's a > > >> delay? > > >> > > >> I'm following the dev guide with this command: > > >> hg clone http://hg.python.org/cpython > > >> > > >> I'm on Linux Mint 14, using the supplied hg version 2.2.2 . My > > >> internet connection seems speedy enough. > > >> > > > > Sean, 7 minutes doesn't sound bad. Keep in mind that with Hg, the whole > repository is being cloned to your computer - all active (and inactive) > branches, all history, etc. The up-side is that after this initial clone, > subsequent pulls are pretty quick and all other operations are local and > super fast (log, blame, etc.) To further clarify what Eli said, "the whole repo" gets put into that .hg directory *first*, and only at the end is a working directory checkout done. So all you will see is cpython/.hg until the very last moment when it will start telling about the checkout being done. --David From regebro at gmail.com Thu Apr 25 04:19:36 2013 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 25 Apr 2013 04:19:36 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Thu, Apr 25, 2013 at 3:54 AM, Stephen J. Turnbull wrote: > RFC 4648 repeatedly refers to *characters*, without specifying an > encoding for them. In fact, if you copy accurately, you can write > BASE64 on a napkin and that napkin will accurate transmit the data > (assuming it doesn't run into sleet or gloom of night). Or Mrs Cake. > What else is that but "text in the sense of Py3k"? Text in the sense of Py3k is Unicode. That a 8-bit character stream (or in this case 6-bit) fits in the 31 bit character space of Unicode doesn't make it Unicode, and hence not text. (Napkins of course have even higher bit density than 31 bits per character, unless you write very small). From the viewpoint of Py3k, bytes data is not text. This is a very useful way to deal with Unicode. See also http://regebro.wordpress.com/2011/03/23/unconfusing-unicode-what-is-unicode/ > My point is not that Python's base64 codec *should* be bytes-to-str > and back. Base64 does not convert between a Unicode character stream and an 8-bite byte stream. It converts between a 8-bit byte-stream and an 8-bit byte stream. It therefore should be bytes to bytes. To fit Unicode text into Base64 you have to first use an encoding on that Unicode text to convert it to bytes. > What I'm groping toward is an idea of a "variable method", so that we > could use .encode and .decode where they are TOOWTDI for people even > though a purely formal interpretation of duck-typing would say "but > why is that blue whale quacking, waddling, and flying?" In other > words (although I have no idea how best to implement it), I would like > "somestring.encode('base64')" to fail with "I don't know how to do > that" (an attribute lookup error?), the same way that > "somebytes.encode('utf-8')" does in Python 3 today. There's only two options there. Either you get a "LookupError: unknown encoding: base64", which is what you get now, or you get an UnicodeEncodingError if the text is not ASCII. We don't want the latter, because it means that code that looks fine for the developer breaks in real life because the developer was American and didn't think of this, but his client happens to have an accent in the name. Base64 is an encoding that transforms between 8-bit streams. Let it be that. Don't try to shoehorn it into a completely different kind of encoding. //Lennart From solipsis at pitrou.net Thu Apr 25 07:43:54 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 25 Apr 2013 07:43:54 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20130425074354.66e522d2@fsol> On Thu, 25 Apr 2013 04:19:36 +0200 Lennart Regebro wrote: > On Thu, Apr 25, 2013 at 3:54 AM, Stephen J. Turnbull wrote: > > RFC 4648 repeatedly refers to *characters*, without specifying an > > encoding for them. [...] > > Base64 is an encoding that transforms between 8-bit streams. No, it isn't. What Stephen wrote above. > Either you get a "LookupError: unknown > encoding: base64", which is what you get now, or you get an > UnicodeEncodingError if the text is not ASCII. We don't want the > latter, because it means that code that looks fine for the developer > breaks in real life because the developer was American That's bogus. By the same argument, we should suppress any encoding which isn't able to represent all possible unicode strings. That's almost all encodings provided by Python (including utf-8, if you consider lone surrogates). I'm sorry for Americans, but they *still* must know about character encodings, and be ready to handle UnicodeErrors, when using Python 3 for encoding/decoding bytestrings. There's no way around it. Regards Antoine. From solipsis at pitrou.net Thu Apr 25 07:45:50 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 25 Apr 2013 07:45:50 +0200 Subject: [Python-Dev] slow hg clone of python repo? References: Message-ID: <20130425074550.332185fa@fsol> On Wed, 24 Apr 2013 16:24:15 -0700 Guido van Rossum wrote: > It's a big repo. Patience. We are actually having bandwidth issues with the current OSU/OSL hosting of python.org machines, which is affecting not only hg.python.org but also pypi.python.org, for at least some users. I believe Noah and friends/colleagues are investigating :-) Regards Antoine. From regebro at gmail.com Thu Apr 25 08:38:12 2013 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 25 Apr 2013 08:38:12 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <20130425074354.66e522d2@fsol> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> Message-ID: On Thu, Apr 25, 2013 at 7:43 AM, Antoine Pitrou wrote: > On Thu, 25 Apr 2013 04:19:36 +0200 > Lennart Regebro wrote: >> On Thu, Apr 25, 2013 at 3:54 AM, Stephen J. Turnbull wrote: >> > RFC 4648 repeatedly refers to *characters*, without specifying an >> > encoding for them. > [...] >> >> Base64 is an encoding that transforms between 8-bit streams. > > No, it isn't. What Stephen wrote above. Yes it is. Base64 takes 8-bit bytes and transforms them into another 8-bit stream that can be safely transmitted over various channels that would mangle an unencoded 8-bit stream, such as email etc. http://en.wikipedia.org/wiki/Base64 >> Either you get a "LookupError: unknown >> encoding: base64", which is what you get now, or you get an >> UnicodeEncodingError if the text is not ASCII. We don't want the >> latter, because it means that code that looks fine for the developer >> breaks in real life because the developer was American > > That's bogus. No, that's real life. > By the same argument, we should suppress any > encoding which isn't able to represent all possible unicode strings. No, if you explicitly use such an encoding it is because you need to because you are transferring data to a system that needs the encoding in question. Unicode errors are unavoidable at that point, not an unexpected surprise because a conversion happened implicitly that you didn't know about. //Lennart From g.brandl at gmx.net Thu Apr 25 08:49:23 2013 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 25 Apr 2013 08:49:23 +0200 Subject: [Python-Dev] A decade as a core dev In-Reply-To: References: Message-ID: Am 18.04.2013 17:02, schrieb Brett Cannon: > Today marks my 10 year anniversary as a core developer on Python. I > wrote a blog post to mark the occasion > (http://sayspy.blogspot.ca/2013/04/a-decade-of-commits.html), but I > wanted to personally thank python-dev for the past decade (and > whatever comes in the future). All of you taught me how to really > program and for that I will be eternally grateful. And the friendships > I have built through this list are priceless. Hah, I only have 2 years to go. Time flies like an unladen swallow... Congrats :) Georg From stephen at xemacs.org Thu Apr 25 08:57:32 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 25 Apr 2013 15:57:32 +0900 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87sj2fxfkz.fsf@uwakimon.sk.tsukuba.ac.jp> Lennart Regebro writes: > Base64 is an encoding that transforms between 8-bit streams. Let it be > that. Don't try to shoehorn it into a completely different kind of > encoding. By "completely different kind of encoding" do you mean "codec"? I think that would be an unfortunate result. These operations on streams are theoretically nicely composable. It would be nice if practice reflected that by having a uniform API for all of these operations (charset translation, encoded text to internal, content transfer encoding, compression ...). I think it would be useful, too, though I can't prove that. Anyway, this discussion belongs on python-ideas at this point. Or would, if I had an idea about implementation. I'll take it there when I do have something to say about implementation. From regebro at gmail.com Thu Apr 25 09:03:08 2013 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 25 Apr 2013 09:03:08 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <87sj2fxfkz.fsf@uwakimon.sk.tsukuba.ac.jp> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <87sj2fxfkz.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Thu, Apr 25, 2013 at 8:57 AM, Stephen J. Turnbull wrote: > I think that would be an unfortunate result. These operations on > streams are theoretically nicely composable. It would be nice if > practice reflected that by having a uniform API for all of these > operations (charset translation, encoded text to internal, content > transfer encoding, compression ...). I think it would be useful, too, > though I can't prove that. But the translation to and from Unicode to some 8-bit encoding is different from the others. It makes sense that they have a different API. If you have a Unicode string you can go: Unicode text -> UTF8 -> ZIP -> BASE64. Or you can go Unicode text -> UTF8 -> BASE64 -> ZIP Although admittedly that makes much less sense. :-) But you can not go: Unicode text -> BASE64 -> ZIP -> UTF8 The str/bytes encoding/decoding is not like the others. //Lennart From ncoghlan at gmail.com Thu Apr 25 09:49:33 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 25 Apr 2013 17:49:33 +1000 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <87sj2fxfkz.fsf@uwakimon.sk.tsukuba.ac.jp> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <87sj2fxfkz.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Thu, Apr 25, 2013 at 4:57 PM, Stephen J. Turnbull wrote: > Lennart Regebro writes: > > > Base64 is an encoding that transforms between 8-bit streams. Let it be > > that. Don't try to shoehorn it into a completely different kind of > > encoding. > > By "completely different kind of encoding" do you mean "codec"? > > I think that would be an unfortunate result. These operations on > streams are theoretically nicely composable. It would be nice if > practice reflected that by having a uniform API for all of these > operations (charset translation, encoded text to internal, content > transfer encoding, compression ...). I think it would be useful, too, > though I can't prove that. > > Anyway, this discussion belongs on python-ideas at this point. Or > would, if I had an idea about implementation. I'll take it there when > I do have something to say about implementation. Bringing the mailing list thread up to date with the state of the relevant tracker issues: I created http://bugs.python.org/issue17827 to cover adding the missing documentation for "codecs.encode" and "codecs.decode" as the officially supported solutions for easy use of the codec infrastructure *without* the additional text model specific input and output type restrictions imposed by the str.encode, bytes.decode and bytearray.decode methods. I created http://bugs.python.org/issue17828 to cover emitting more meaningful exceptions when a codec throws TypeError or ValueError, as well as when the additional type checking fails for str.encode, bytes.decode and bytearray.decode. I created http://bugs.python.org/issue17839 to cover the fact that part of the problem here is that the base64 module currently only accepts bytes and bytearray as inputs, rather than anything that supports the PEP 3118 buffer interface. http://bugs.python.org/issue7475 (linked earlier in the thread) is now strictly about restoring the shorthand aliases for "base64_codec", "bz2_codec" et al that were removed in http://bugs.python.org/issue10807. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From solipsis at pitrou.net Thu Apr 25 11:25:36 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 25 Apr 2013 11:25:36 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> Message-ID: <20130425112536.07f0b782@pitrou.net> Le Thu, 25 Apr 2013 08:38:12 +0200, Lennart Regebro a ?crit : > On Thu, Apr 25, 2013 at 7:43 AM, Antoine Pitrou > wrote: > > On Thu, 25 Apr 2013 04:19:36 +0200 > > Lennart Regebro wrote: > >> On Thu, Apr 25, 2013 at 3:54 AM, Stephen J. Turnbull > >> wrote: > >> > RFC 4648 repeatedly refers to *characters*, without specifying an > >> > encoding for them. > > [...] > >> > >> Base64 is an encoding that transforms between 8-bit streams. > > > > No, it isn't. What Stephen wrote above. > > Yes it is. Base64 takes 8-bit bytes and transforms them into another > 8-bit stream that can be safely transmitted over various channels that > would mangle an unencoded 8-bit stream, such as email etc. > > http://en.wikipedia.org/wiki/Base64 I don't see anything in that Wikipedia page that validates your opinion. The Wikipedia page does talk about *text* and *characters* for the result of base64 encoding. Besides, I would consider a RFC more authoritative than a Wikipedia definition. > > By the same argument, we should suppress any > > encoding which isn't able to represent all possible unicode strings. > > No, if you explicitly use such an encoding it is because you need to > because you are transferring data to a system that needs the encoding > in question. Unicode errors are unavoidable at that point, not an > unexpected surprise because a conversion happened implicitly that you > didn't know about. I don't know what "implicit conversion" you are talking about. There's no "implicit conversion" in a scheme where the result of base64 encoding is a text string. Regards Antoine. From solipsis at pitrou.net Thu Apr 25 11:42:57 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 25 Apr 2013 11:42:57 +0200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library References: Message-ID: <20130425114257.24548e24@pitrou.net> Le Fri, 12 Apr 2013 05:55:00 -0700, Eli Bendersky a ?crit : > > To programmatically access enumeration values, use ``getattr``:: > > >>> getattr(Colors, 'red') > The PEP should mention how to get an enum from its raw value: >>> Colors[1] or: >>> Colors(1) It would perhaps be nice to have a .get() method that return None if the raw value is unknown: >>> Colors(42) ... ValueError: 42 >>> Colors.get(42) >>> Regards Anroine. From regebro at gmail.com Thu Apr 25 12:05:01 2013 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 25 Apr 2013 12:05:01 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <20130425112536.07f0b782@pitrou.net> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> <20130425112536.07f0b782@pitrou.net> Message-ID: On Thu, Apr 25, 2013 at 11:25 AM, Antoine Pitrou wrote: > Le Thu, 25 Apr 2013 08:38:12 +0200, >> Yes it is. Base64 takes 8-bit bytes and transforms them into another >> 8-bit stream that can be safely transmitted over various channels that >> would mangle an unencoded 8-bit stream, such as email etc. >> >> http://en.wikipedia.org/wiki/Base64 > > I don't see anything in that Wikipedia page that validates your opinion. OK, quote me the exact page text from the Wikipedia article or RFC that explains how you map the 31-bit character space of Unicode to Base64. > The Wikipedia page does talk about *text* and *characters* for > the result of base64 encoding. So are saying that you want the Python implementation of base64 encoding to take 8-bit binary data in bytes format and return a Unicode string containing the Base64 encoded data? I think that would surprise most people, and be of significantly less use than a base64 encoding that returns bytes. Python 3 still views text as Unicode only. Everything else is not text, but binary data. This makes sense, is consistent and makes things easier to handle. This is the whole point of making the str into Unicode in Python 3. >> No, if you explicitly use such an encoding it is because you need to >> because you are transferring data to a system that needs the encoding >> in question. Unicode errors are unavoidable at that point, not an >> unexpected surprise because a conversion happened implicitly that you >> didn't know about. > > I don't know what "implicit conversion" you are talking about. There's > no "implicit conversion" in a scheme where the result of base64 > encoding is a text string. I'm sorry, I thought you were arguing for a base64 encoding taking Unicode strings and returning 8-bit bytes. That position I can understand, although I disagree with it. The position that a base64 encoding should take 8-bit bytes and return Unicode strings is incomprehensible to me. I have no idea why you would want that, how you would use it, how you would implement that API in a reasonable way, nor how you would explain why it is like that. I can't think of any usecase where you would want base64 encoded data unless you intend to transmit it over an 8-bit channel, so why it should return a Unicode string instead of 8-bit bytes is completely beyond my comprehension. Sorry. //Lennart From catch-all at masklinn.net Thu Apr 25 12:46:43 2013 From: catch-all at masklinn.net (Xavier Morel) Date: Thu, 25 Apr 2013 12:46:43 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <20130425112536.07f0b782@pitrou.net> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> <20130425112536.07f0b782@pitrou.net> Message-ID: <48B33979-7D64-4239-A100-EA5E9BCB5B1F@masklinn.net> On 2013-04-25, at 11:25 , Antoine Pitrou wrote: > > Besides, I would consider a RFC more authoritative than a > Wikipedia definition. > Base encoding of data is used in many situations to store or transfer > data in environments that, perhaps for legacy reasons, are restricted > to US-ASCII [1] data. so the output is US-ASCII data, a byte stream. Stephen is correct that you could decide you don't care about those semantics, and implement base64 encoding as a bytes -> str decoding then requiring a re-encoding (to ascii) before wire transmission. The clarity of the interface (or lack thereof) would probably make users want to send a strongly worded letter to whoever implemented it though, I don't think `data.decode('base64').encode('ascii')` would fit the "obviousness" or "readability" expectations of most users. From eliben at gmail.com Thu Apr 25 14:21:50 2013 From: eliben at gmail.com (Eli Bendersky) Date: Thu, 25 Apr 2013 05:21:50 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130425114257.24548e24@pitrou.net> References: <20130425114257.24548e24@pitrou.net> Message-ID: On Thu, Apr 25, 2013 at 2:42 AM, Antoine Pitrou wrote: > Le Fri, 12 Apr 2013 05:55:00 -0700, > Eli Bendersky a ?crit : > > > > To programmatically access enumeration values, use ``getattr``:: > > > > >>> getattr(Colors, 'red') > > > > The PEP should mention how to get an enum from its raw value: > > >>> Colors[1] > > > Yes, this is mentioned in the beginning of the same paragraph. > or: > > >>> Colors(1) > > > This syntax was considered initially but then rejected because it's confusing, and there already exists a way to lookup by value (Colors[1]). Eli > It would perhaps be nice to have a .get() method that return None if the > raw value is unknown: > > >>> Colors(42) > ... > ValueError: 42 > >>> Colors.get(42) > >>> > > Regards > > Anroine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/eliben%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Thu Apr 25 14:29:04 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 25 Apr 2013 14:29:04 +0200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library References: <20130425114257.24548e24@pitrou.net> Message-ID: <20130425142904.4a287fab@pitrou.net> Le Thu, 25 Apr 2013 05:21:50 -0700, Eli Bendersky a ?crit : > > > > or: > > > > >>> Colors(1) > > > > > > > This syntax was considered initially but then rejected because it's > confusing, and there already exists a way to lookup by value > (Colors[1]). Well, it works in latest flufl.enum. Is there a difference between the PEP and the implementation. Regards Antoine. From solipsis at pitrou.net Thu Apr 25 14:38:31 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 25 Apr 2013 14:38:31 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> <20130425112536.07f0b782@pitrou.net> <48B33979-7D64-4239-A100-EA5E9BCB5B1F@masklinn.net> Message-ID: <20130425143831.371667fe@pitrou.net> Le Thu, 25 Apr 2013 12:46:43 +0200, Xavier Morel a ?crit : > On 2013-04-25, at 11:25 , Antoine Pitrou wrote: > > > > Besides, I would consider a RFC more authoritative than a > > Wikipedia definition. > > > Base encoding of data is used in many situations to store or > > transfer data in environments that, perhaps for legacy reasons, are > > restricted to US-ASCII [1] data. > > so the output is US-ASCII data, a byte stream. Well, depending on the context, US-ASCII can be a character set or a character encoding. If some specification is talking about text and characters, then it is something that can reasonably be a str in Python land. Similarly, we have chosen to make filesystem paths str by default in Python 3, even though many Unix-heads would claim that filesystem paths are "bytes only". The reason is that while they are technically bytes (under Unix), they are functionally text. Now, if the base64-encoded data is your entire payload, this clearly amounts to nitpicking. But when you want to *embed* that data in some larger chunk of text (e.g. a JSON object), then it makes a lot of sense to consider the base64-encoded data a piece of *text*, not bytes. Regards Antoine. From solipsis at pitrou.net Thu Apr 25 14:57:45 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 25 Apr 2013 14:57:45 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> <20130425112536.07f0b782@pitrou.net> Message-ID: <20130425145745.16f87748@pitrou.net> Le Thu, 25 Apr 2013 12:05:01 +0200, Lennart Regebro a ?crit : > > The Wikipedia page does talk about *text* and *characters* for > > the result of base64 encoding. > > So are saying that you want the Python implementation of base64 > encoding to take 8-bit binary data in bytes format and return a > Unicode string containing the Base64 encoded data? I'm not wanting anything here, since that would clearly break backwards compatibility. But I think binascii should have gone that way in Python 3, indeed. binascii.b2a_hex(), for example, would be much more practical if it returned str, rather than bytes. > Python 3 still views text as Unicode only. Python 3 doesn't *view* text as unicode, it *represents* it as unicode. That is, unicode is the character set that Python 3 is able to represent in the canonical text type, str. If you ever encounter a hypothetical text that uses characters outside of Unicode (obviously it will be encoded using a non-unicode encoding :-)), then you can't represent it as a str. And base64 is clearly representable as unicode, since it's representable using the ASCII character set (which is a subset of the unicode character set). > I can't think of > any usecase where you would want base64 encoded data unless you intend > to transmit it over an 8-bit channel, I can think of many usecases where I want to *embed* base64-encoded data in a larger text *before* encoding that text and transmitting it over a 8-bit channel. (GPG signatures, binary data embedded in JSON objects, etc.) Regards Antoine. From eliben at gmail.com Thu Apr 25 15:03:05 2013 From: eliben at gmail.com (Eli Bendersky) Date: Thu, 25 Apr 2013 06:03:05 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130425142904.4a287fab@pitrou.net> References: <20130425114257.24548e24@pitrou.net> <20130425142904.4a287fab@pitrou.net> Message-ID: On Thu, Apr 25, 2013 at 5:29 AM, Antoine Pitrou wrote: > Le Thu, 25 Apr 2013 05:21:50 -0700, > Eli Bendersky a ?crit : > > > > > > > or: > > > > > > >>> Colors(1) > > > > > > > > > > > This syntax was considered initially but then rejected because it's > > confusing, and there already exists a way to lookup by value > > (Colors[1]). > > Well, it works in latest flufl.enum. Is there a difference between the > PEP and the implementation. > It produces a deprecation warning in flufl.enum because flufl.enum specifically supported this earlier. It should not be supported in the stdlib implementation. The __call__ syntax has been repurposed for the convenience API: >>> Animals = Enum('Animals', 'ant bee cat dog') >>> Animals >>> Animals.ant >>> Animals.ant.value 1 The aforementioned deprecated syntax refers to __call__ with a single arguments (the convenience API by definition requires more than one). Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From regebro at gmail.com Thu Apr 25 15:34:45 2013 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 25 Apr 2013 15:34:45 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <20130425145745.16f87748@pitrou.net> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> <20130425112536.07f0b782@pitrou.net> <20130425145745.16f87748@pitrou.net> Message-ID: On Thu, Apr 25, 2013 at 2:57 PM, Antoine Pitrou wrote: > I can think of many usecases where I want to *embed* base64-encoded > data in a larger text *before* encoding that text and transmitting > it over a 8-bit channel. That still doesn't mean that this should be the default behavior. Just because you *can* represent base64 as Unicode text doesn't mean that it should be. > (GPG signatures, binary data embedded in JSON objects, etc.) Is the GPG signature calculated on the *Unicode* data? How is that done? Isn't it done on the encoded message? As I understand it a GPG signature is done on any sort of document. Either me or you have completely misunderstood how GPG works, I think. :-) In the case of JSON objects, they are intended for data exchange, and hence in the end need to be byte strings. So if you have a byte string you want to base64 encode before transmitting it with json, you would just end up transforming it to a unicode string and then back. That doesn't seem useful. One use case where you clearly *do* want the base64 encoded data to be unicode strings is because you want to embed it in a text discussing base64 strings, for a blog or a book or something. That doesn't seem to be a very common usecase. For the most part you base64 encode things because it's going to be transmitted, and hence the natural result of a base64 encoding should be data that is ready to be transmitted, hence byte strings, and not Unicode strings. > Python 3 doesn't *view* text as unicode, it *represents* it as unicode. I don't agree that there is a significant difference between those wordings in this context. The end result is the same: Things intended to be handled/seen as textual should be unicode strings, things intended for data exchange should be byte strings. Something that is base64 encoded is primarily intended for data exchange. A base64 encoding should therefore return byte strings, especially since most API's that perform this transmission will take byte strings as input. If you want to include this in textual data, for whatever reason, like printing it in a book, then the conversion is trivial, but that is clearly the less common use case, and should therefore not be the default behavior. //Lennart From tseaver at palladion.com Thu Apr 25 15:55:26 2013 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 25 Apr 2013 09:55:26 -0400 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <20130425074354.66e522d2@fsol> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/25/2013 01:43 AM, Antoine Pitrou wrote: > On Thu, 25 Apr 2013 04:19:36 +0200 Lennart Regebro > wrote: >> On Thu, Apr 25, 2013 at 3:54 AM, Stephen J. Turnbull >> wrote: >>> RFC 4648 repeatedly refers to *characters*, without specifying an >>> encoding for them. > [...] >> >> Base64 is an encoding that transforms between 8-bit streams. > > No, it isn't. What Stephen wrote above. Stephen was incorrect: the base64 standard is about encoding a binary stream (8-bit bites) onto another binary stream (6-bit bytes), but one which can be safely transmitted over a 7-bit-only medium. Text in Py3ks sense is irrelevant. >> Either you get a "LookupError: unknown encoding: base64", which is >> what you get now, or you get an UnicodeEncodingError if the text is >> not ASCII. We don't want the latter, because it means that code that >> looks fine for the developer breaks in real life because the >> developer was American > > That's bogus. By the same argument, we should suppress any encoding > which isn't able to represent all possible unicode strings. That's > almost all encodings provided by Python (including utf-8, if you > consider lone surrogates). > > I'm sorry for Americans, but they *still* must know about character > encodings, and be ready to handle UnicodeErrors, when using Python 3 > for encoding/decoding bytestrings. There's no way around it. WHat does that snark have to do with this discussion? base64 has no more to do with character set encodings than it does the moon. It would be a "transform" (bytes -> bytes), not an "encoding". Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlF5Nc4ACgkQ+gerLs4ltQ7f9ACgx19dzyLXCDzkLkWITSU+7WyD XEMAn38mZgK8F1/FGWJc+ANOJz2tfHI/ =qpSL -----END PGP SIGNATURE----- From barry at python.org Thu Apr 25 16:07:30 2013 From: barry at python.org (Barry Warsaw) Date: Thu, 25 Apr 2013 10:07:30 -0400 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> <20130425112536.07f0b782@pitrou.net> <20130425145745.16f87748@pitrou.net> Message-ID: <20130425100730.5b1924f6@anarchist> On Apr 25, 2013, at 03:34 PM, Lennart Regebro wrote: >In the case of JSON objects, they are intended for data exchange, and >hence in the end need to be byte strings. Except that they're not. http://bugs.python.org/issue10976 -Barry From python at mrabarnett.plus.com Thu Apr 25 16:22:00 2013 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 25 Apr 2013 15:22:00 +0100 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> <20130425112536.07f0b782@pitrou.net> <20130425145745.16f87748@pitrou.net> Message-ID: <51793C08.4020703@mrabarnett.plus.com> On 25/04/2013 14:34, Lennart Regebro wrote: > On Thu, Apr 25, 2013 at 2:57 PM, Antoine Pitrou wrote: >> I can think of many usecases where I want to *embed* base64-encoded >> data in a larger text *before* encoding that text and transmitting >> it over a 8-bit channel. > > That still doesn't mean that this should be the default behavior. Just > because you *can* represent base64 as Unicode text doesn't mean that > it should be. > >> (GPG signatures, binary data embedded in JSON objects, etc.) > > Is the GPG signature calculated on the *Unicode* data? How is that > done? Isn't it done on the encoded message? As I understand it a GPG > signature is done on any sort of document. Either me or you have > completely misunderstood how GPG works, I think. :-) > > In the case of JSON objects, they are intended for data exchange, and > hence in the end need to be byte strings. So if you have a byte string > you want to base64 encode before transmitting it with json, you would > just end up transforming it to a unicode string and then back. That > doesn't seem useful. > The JSON specification says that it's text. Its string literals can contain Unicode codepoints. It needs to be encoded to bytes for transmission and storage, but JSON itself is not a bytestring format. > One use case where you clearly *do* want the base64 encoded data to be > unicode strings is because you want to embed it in a text discussing > base64 strings, for a blog or a book or something. That doesn't seem > to be a very common usecase. > > For the most part you base64 encode things because it's going to be > transmitted, and hence the natural result of a base64 encoding should > be data that is ready to be transmitted, hence byte strings, and not > Unicode strings. > >> Python 3 doesn't *view* text as unicode, it *represents* it as unicode. > > I don't agree that there is a significant difference between those > wordings in this context. The end result is the same: Things intended > to be handled/seen as textual should be unicode strings, things > intended for data exchange should be byte strings. Something that is > base64 encoded is primarily intended for data exchange. A base64 > encoding should therefore return byte strings, especially since most > API's that perform this transmission will take byte strings as input. > If you want to include this in textual data, for whatever reason, like > printing it in a book, then the conversion is trivial, but that is > clearly the less common use case, and should therefore not be the > default behavior. > base64 is a way of encoding binary data as text. The problem is that traditionally text has been encoded with one byte per character, except in those locales where there were too many characters in the character set for that to be possible. In Python 3 we're trying to stop mixing binary data (bytestrings) with text (Unicode strings). From dholth at gmail.com Thu Apr 25 16:25:21 2013 From: dholth at gmail.com (Daniel Holth) Date: Thu, 25 Apr 2013 10:25:21 -0400 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <20130425100730.5b1924f6@anarchist> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> <20130425112536.07f0b782@pitrou.net> <20130425145745.16f87748@pitrou.net> <20130425100730.5b1924f6@anarchist> Message-ID: On Thu, Apr 25, 2013 at 10:07 AM, Barry Warsaw wrote: > On Apr 25, 2013, at 03:34 PM, Lennart Regebro wrote: > >>In the case of JSON objects, they are intended for data exchange, and >>hence in the end need to be byte strings. > > Except that they're not. > > http://bugs.python.org/issue10976 > > -Barry > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/dholth%40gmail.com What am I doing wrong in this JSON crypto signature verification snippet that features many conversions between binary and text? recipients = jwsjs["recipients"] encoded_payload = binary(jwsjs["payload"]) headers = [] for recipient in recipients: h = binary(recipient["header"]) s = binary(recipient["signature"]) header = json.loads(native(urlsafe_b64decode(h))) vk = urlsafe_b64decode(binary(header["jwk"]["vk"])) secured_input = b".".join((h, encoded_payload)) sig = urlsafe_b64decode(s) sig_msg = sig+secured_input verified_input = native(ed25519ll.crypto_sign_open(sig_msg, vk)) verified_header, verified_payload = verified_input.split('.') verified_header = binary(verified_header) decoded_header = native(urlsafe_b64decode(verified_header)) headers.append(json.loads(decoded_header)) verified_payload = binary(verified_payload) # only return header, payload that have passed through the crypto library. payload = json.loads(native(urlsafe_b64decode(verified_payload))) return headers, payload From regebro at gmail.com Thu Apr 25 16:39:01 2013 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 25 Apr 2013 16:39:01 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <51793C08.4020703@mrabarnett.plus.com> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> <20130425112536.07f0b782@pitrou.net> <20130425145745.16f87748@pitrou.net> <51793C08.4020703@mrabarnett.plus.com> Message-ID: On Thu, Apr 25, 2013 at 4:22 PM, MRAB wrote: > The JSON specification says that it's text. Its string literals can > contain Unicode codepoints. It needs to be encoded to bytes for > transmission and storage, but JSON itself is not a bytestring format. OK, fair enough. > base64 is a way of encoding binary data as text. It's a way of encoding binary data using ASCII. There is a subtle but important difference. > In Python 3 we're trying to stop mixing binary data (bytestrings) with > text (Unicode strings). Yup. And that's why a byte64 encoding shouldn't return Unicode strings. //Lennart From solipsis at pitrou.net Thu Apr 25 17:15:45 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 25 Apr 2013 17:15:45 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> Message-ID: <20130425171545.029dc012@pitrou.net> Le Thu, 25 Apr 2013 09:55:26 -0400, Tres Seaver a ?crit : > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 04/25/2013 01:43 AM, Antoine Pitrou wrote: > > On Thu, 25 Apr 2013 04:19:36 +0200 Lennart Regebro > > wrote: > >> On Thu, Apr 25, 2013 at 3:54 AM, Stephen J. Turnbull > >> wrote: > >>> RFC 4648 repeatedly refers to *characters*, without specifying an > >>> encoding for them. > > [...] > >> > >> Base64 is an encoding that transforms between 8-bit streams. > > > > No, it isn't. What Stephen wrote above. > > Stephen was incorrect: the base64 standard is about encoding a binary > stream (8-bit bites) onto another binary stream (6-bit bytes), but one > which can be safely transmitted over a 7-bit-only medium. So where does the RFC talk about "6-bit bytes" at all? Or did you just invent it? From solipsis at pitrou.net Thu Apr 25 17:27:54 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 25 Apr 2013 17:27:54 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> <20130425112536.07f0b782@pitrou.net> <20130425145745.16f87748@pitrou.net> Message-ID: <20130425172754.58ba6ba4@pitrou.net> Le Thu, 25 Apr 2013 15:34:45 +0200, Lennart Regebro a ?crit : > > I don't agree that there is a significant difference between those > wordings in this context. The end result is the same: Things intended > to be handled/seen as textual should be unicode strings, things > intended for data exchange should be byte strings. I don't think this distinction is meaningful at all. In the end, everything is a byte string on a classical computer (including unicode strings displayed on your monitor, obviously). If you think the technicalities of an operation should never be hidden or abstracted away, then you're better off with C than Python ;-) Regards Antoine. From ethan at stoneleaf.us Thu Apr 25 17:46:22 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 08:46:22 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130425114257.24548e24@pitrou.net> <20130425142904.4a287fab@pitrou.net> Message-ID: <51794FCE.10708@stoneleaf.us> On 04/25/2013 06:03 AM, Eli Bendersky wrote: > > The __call__ syntax has been repurposed for the convenience API: > > --> Animals = Enum('Animals', 'ant bee cat dog') > --> Animals > > --> Animals.ant > > --> Animals.ant.value > 1 > > The aforementioned deprecated syntax refers to __call__ with a single arguments (the convenience API by definition > requires more than one). I don't understand why having Enum() be the convenience function rules out `Animals(1)` from returning `Animals.ant`. -- ~Ethan~ From guido at python.org Thu Apr 25 18:32:32 2013 From: guido at python.org (Guido van Rossum) Date: Thu, 25 Apr 2013 09:32:32 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130425114257.24548e24@pitrou.net> <20130425142904.4a287fab@pitrou.net> Message-ID: FWIW, If we're going ahead with the plan to make Colors.red an instance of Colors, maybe the class names used in examples should be singular, i.e. isinstance(Color.red, Color)? -- --Guido van Rossum (python.org/~guido) From eliben at gmail.com Thu Apr 25 18:34:09 2013 From: eliben at gmail.com (Eli Bendersky) Date: Thu, 25 Apr 2013 09:34:09 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <51794FCE.10708@stoneleaf.us> References: <20130425114257.24548e24@pitrou.net> <20130425142904.4a287fab@pitrou.net> <51794FCE.10708@stoneleaf.us> Message-ID: On Thu, Apr 25, 2013 at 8:46 AM, Ethan Furman wrote: > On 04/25/2013 06:03 AM, Eli Bendersky wrote: > >> >> The __call__ syntax has been repurposed for the convenience API: >> >> --> Animals = Enum('Animals', 'ant bee cat dog') >> --> Animals >> >> >> --> Animals.ant >> >> --> Animals.ant.value >> >> 1 >> >> The aforementioned deprecated syntax refers to __call__ with a single >> arguments (the convenience API by definition >> requires more than one). >> > > I don't understand why having Enum() be the convenience function rules out > `Animals(1)` from returning `Animals.ant`. > Because we already have a way to do that: Animals[1]. Why do you need two slightly different ways to do the same? Moreover, why do you want to make Animals.__call__ behave very differently based only on the number of args? This seems to be un-pythonic in multiple ways. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From eliben at gmail.com Thu Apr 25 18:47:45 2013 From: eliben at gmail.com (Eli Bendersky) Date: Thu, 25 Apr 2013 09:47:45 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <51795C25.6060206@stoneleaf.us> References: <20130425114257.24548e24@pitrou.net> <20130425142904.4a287fab@pitrou.net> <51794FCE.10708@stoneleaf.us> <51795C25.6060206@stoneleaf.us> Message-ID: On Thu, Apr 25, 2013 at 9:39 AM, Ethan Furman wrote: > On 04/25/2013 09:34 AM, Eli Bendersky wrote: > > >> >> >> On Thu, Apr 25, 2013 at 8:46 AM, Ethan Furman > ethan at stoneleaf.us>> wrote: >> >> On 04/25/2013 06:03 AM, Eli Bendersky wrote: >> >> >> The __call__ syntax has been repurposed for the convenience API: >> >> --> Animals = Enum('Animals', 'ant bee cat dog') >> --> Animals >> >> >> --> Animals.ant >> >> --> Animals.ant.value >> >> 1 >> >> The aforementioned deprecated syntax refers to __call__ with a >> single arguments (the convenience API by definition >> requires more than one). >> >> >> I don't understand why having Enum() be the convenience function >> rules out `Animals(1)` from returning `Animals.ant`. >> >> >> Because we already have a way to do that: Animals[1]. Why do you need two >> slightly different ways to do the same? >> Moreover, why do you want to make Animals.__call__ behave very >> differently based only on the number of args? This seems >> to be un-pythonic in multiple ways. >> > > I think we're talking past each other (or I'm not awake yet ;). > > Animals is a class. Giving Animals a parameter (such as 1 or 'ant') > should return the instance that matches. This is how classes work. > > I don't understand your assertion that there is another way to call > Animals... do you mean something like: > > --> MoreAnimals = Animals('MoreAnimals', 'bird worm insect') > Yes, this works in the current implementation. But I'm realizing that the recent proposals of making isinstance(Color.red, Color) True will turn things around anyway so this discussion may be moot. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Thu Apr 25 18:53:53 2013 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 25 Apr 2013 17:53:53 +0100 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <51793C08.4020703@mrabarnett.plus.com> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> <20130425112536.07f0b782@pitrou.net> <20130425145745.16f87748@pitrou.net> <51793C08.4020703@mrabarnett.plus.com> Message-ID: <51795FA1.2070004@mrabarnett.plus.com> On 25/04/2013 15:22, MRAB wrote: > On 25/04/2013 14:34, Lennart Regebro wrote: >> On Thu, Apr 25, 2013 at 2:57 PM, Antoine Pitrou wrote: >>> I can think of many usecases where I want to *embed* base64-encoded >>> data in a larger text *before* encoding that text and transmitting >>> it over a 8-bit channel. >> >> That still doesn't mean that this should be the default behavior. Just >> because you *can* represent base64 as Unicode text doesn't mean that >> it should be. >> [snip] >> One use case where you clearly *do* want the base64 encoded data to be >> unicode strings is because you want to embed it in a text discussing >> base64 strings, for a blog or a book or something. That doesn't seem >> to be a very common usecase. >> >> For the most part you base64 encode things because it's going to be >> transmitted, and hence the natural result of a base64 encoding should >> be data that is ready to be transmitted, hence byte strings, and not >> Unicode strings. >> >>> Python 3 doesn't *view* text as unicode, it *represents* it as unicode. >> >> I don't agree that there is a significant difference between those >> wordings in this context. The end result is the same: Things intended >> to be handled/seen as textual should be unicode strings, things >> intended for data exchange should be byte strings. Something that is >> base64 encoded is primarily intended for data exchange. A base64 >> encoding should therefore return byte strings, especially since most >> API's that perform this transmission will take byte strings as input. >> If you want to include this in textual data, for whatever reason, like >> printing it in a book, then the conversion is trivial, but that is >> clearly the less common use case, and should therefore not be the >> default behavior. >> > base64 is a way of encoding binary data as text. The problem is that > traditionally text has been encoded with one byte per character, except > in those locales where there were too many characters in the character > set for that to be possible. > > In Python 3 we're trying to stop mixing binary data (bytestrings) with > text (Unicode strings). > RFC 4648 says """Base encoding of data is used in many situations to store or transfer data in environments that, perhaps for legacy reasons, are restricted to US-ASCII [1] data.""". To me, "US-ASCII" is an encoding, so it appears to be talking about encoding binary data (bytestrings) to ASCII-encoded text (bytestrings). From ethan at stoneleaf.us Thu Apr 25 18:39:01 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 09:39:01 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130425114257.24548e24@pitrou.net> <20130425142904.4a287fab@pitrou.net> <51794FCE.10708@stoneleaf.us> Message-ID: <51795C25.6060206@stoneleaf.us> On 04/25/2013 09:34 AM, Eli Bendersky wrote: > > > > On Thu, Apr 25, 2013 at 8:46 AM, Ethan Furman > wrote: > > On 04/25/2013 06:03 AM, Eli Bendersky wrote: > > > The __call__ syntax has been repurposed for the convenience API: > > --> Animals = Enum('Animals', 'ant bee cat dog') > --> Animals > > > --> Animals.ant > > --> Animals.ant.value > > 1 > > The aforementioned deprecated syntax refers to __call__ with a single arguments (the convenience API by definition > requires more than one). > > > I don't understand why having Enum() be the convenience function rules out `Animals(1)` from returning `Animals.ant`. > > > Because we already have a way to do that: Animals[1]. Why do you need two slightly different ways to do the same? > Moreover, why do you want to make Animals.__call__ behave very differently based only on the number of args? This seems > to be un-pythonic in multiple ways. I think we're talking past each other (or I'm not awake yet ;). Animals is a class. Giving Animals a parameter (such as 1 or 'ant') should return the instance that matches. This is how classes work. I don't understand your assertion that there is another way to call Animals... do you mean something like: --> MoreAnimals = Animals('MoreAnimals', 'bird worm insect') ? -- ~Ethan~ From regebro at gmail.com Thu Apr 25 19:04:01 2013 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 25 Apr 2013 19:04:01 +0200 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <20130425172754.58ba6ba4@pitrou.net> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> <20130425112536.07f0b782@pitrou.net> <20130425145745.16f87748@pitrou.net> <20130425172754.58ba6ba4@pitrou.net> Message-ID: On Thu, Apr 25, 2013 at 5:27 PM, Antoine Pitrou wrote: > Le Thu, 25 Apr 2013 15:34:45 +0200, > Lennart Regebro a ?crit : >> >> I don't agree that there is a significant difference between those >> wordings in this context. The end result is the same: Things intended >> to be handled/seen as textual should be unicode strings, things >> intended for data exchange should be byte strings. > > I don't think this distinction is meaningful at all. OK, then I think we have found the core of the problem, and the end of the discussion (from my side, that is). > In the end, > everything is a byte string on a classical computer (including unicode > strings displayed on your monitor, obviously). Yes of course. Especially since my monitor is an output device. ;-) > If you think the technicalities of an operation should never be hidden > or abstracted away, then you're better off with C than Python ;-) The whole point is that Python *does* abstract it away. It abstract the internals of Unicode strings in such a way that they are no longer, conceptually, 8-bit data. This *is* a distinction Python does, and it is a useful distinction. I do not see any reason to remove it. http://regebro.wordpress.com/2011/03/23/unconfusing-unicode-what-is-unicode/ //Lennart From stephen at xemacs.org Thu Apr 25 19:31:25 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 26 Apr 2013 02:31:25 +0900 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> <20130425112536.07f0b782@pitrou.net> <20130425145745.16f87748@pitrou.net> <51793C08.4020703@mrabarnett.plus.com> Message-ID: <87ppxiy0sy.fsf@uwakimon.sk.tsukuba.ac.jp> Lennart Regebro writes: > On Thu, Apr 25, 2013 at 4:22 PM, MRAB wrote: > > The JSON specification says that it's text. Its string literals can > > contain Unicode codepoints. It needs to be encoded to bytes for > > transmission and storage, but JSON itself is not a bytestring format. > > OK, fair enough. > > > base64 is a way of encoding binary data as text. > > It's a way of encoding binary data using ASCII. There is a subtle but > important difference. Yes, there is a difference, but I think you're wrong. RFC 4648 explicitly states that Base-n encodings are intended for "human handling" and even makes reference to character glyphs (the rationale for excluding confusable digits from the Base32 alphabet). That's text. Even if it is a rather restricted subset of text, those restrictions are much stronger than merely to ASCII, and they are based on aspects of text that go well beyond merely an encoding with a small code unit. > > In Python 3 we're trying to stop mixing binary data (bytestrings) with > > text (Unicode strings). > > Yup. And that's why a byte64 encoding shouldn't return Unicode strings. That's inaccurate. Antoine has presented several examples of why *some* base64 encoders might return Unicode strings, precisely because their output will be embedded in Unicode streams. Debugging the MIME composition functions in the email module is another. An accurate statement is that these use cases are relatively unusual. The common use case is feeding a binary stream directly into a wire protocol. Supporting that use case demands a base64 encoder with a bytes-to-bytes signature in the stdlib, for both convenience and to some extent efficiency. I don't really care if the stdlib supports the specialized use cases with a separate base64 encoder (Antoine suggested the binascii module), or if it leaves that up to the user (it's just an occasional use of ".decode('ascii')", after all). From ijmorlan at uwaterloo.ca Thu Apr 25 19:29:32 2013 From: ijmorlan at uwaterloo.ca (Isaac Morland) Date: Thu, 25 Apr 2013 13:29:32 -0400 (EDT) Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> <20130425112536.07f0b782@pitrou.net> <20130425145745.16f87748@pitrou.net> <51793C08.4020703@mrabarnett.plus.com> Message-ID: On Thu, 25 Apr 2013, Lennart Regebro wrote: > On Thu, Apr 25, 2013 at 4:22 PM, MRAB wrote: >> The JSON specification says that it's text. Its string literals can >> contain Unicode codepoints. It needs to be encoded to bytes for >> transmission and storage, but JSON itself is not a bytestring format. > > OK, fair enough. > >> base64 is a way of encoding binary data as text. > > It's a way of encoding binary data using ASCII. There is a subtle but > important difference. It is a way of encoding arrays of 8-bit bytes as arrays of characters that are part of the printable, non-whitespace subset of the ASCII repertoire. Since the ASCII repertoire is now simply the first 128 code points in the Unicode repertoire, it is equally correct to say that base64 is a way of encoding binary data as Unicode text. >> In Python 3 we're trying to stop mixing binary data (bytestrings) with >> text (Unicode strings). > > Yup. And that's why a byte64 encoding shouldn't return Unicode strings. That is exactly why it should return Unicode strings. What bytes should get sent if base64 is used to send a byte array over an EBCDIC link? [*] Having said that, there may be other reasons for base64 encoding to return bytes - I can conceive of arguments involving efficiency, or practicality, or the most common use cases. So I can't say for sure what base64 encoding actually ought to return in Python. But the purist stance should be that base64 encoding should return text, i.e. a string, i.e. unicode. [*] I apologize to anybody who just ate. Isaac Morland CSCF Web Guru DC 2554C, x36650 WWW Software Specialist From barry at python.org Thu Apr 25 20:44:10 2013 From: barry at python.org (Barry Warsaw) Date: Thu, 25 Apr 2013 14:44:10 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130423154458.3bba1e57@pitrou.net> References: <20130423154458.3bba1e57@pitrou.net> Message-ID: <20130425144410.096a65c4@anarchist> On Apr 23, 2013, at 03:44 PM, Antoine Pitrou wrote: >I'm having a problem with the proposed implementation. I haven't found >any mention of it, so apologies if this has already been discussed: > >>>> from flufl.enum import * >>>> class C(Enum): >... a = 1 >... b = 2 >... >>>> C.a.__class__ > >>>> isinstance(C.a, C) >False >>>> isinstance(C(1), C) >False > >It would really be better if instances were actual instances of the >class, IMO. Ignore the single argument call syntax for Enums please. As Eli pointed out, you have getitem syntax for this and the single argument call syntax is deprecated. It will be removed in a future version of flufl.enum and need not appear in stdlib enum. TOOWTDI. C.a and C[1] return the same object, and it seems perfectly natural to me that this object is *not* an instance of the enum class. In fact, it seems completely weird to me that C.a would be an instance of the enum class. It seems very rare that a class has attributes that are instances of that class. It's not even easy to do with traditional syntax. class Foo: a = Foo() b = Foo() c = Foo() Huh? -Barry From stephen at xemacs.org Thu Apr 25 20:44:58 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 26 Apr 2013 03:44:58 +0900 Subject: [Python-Dev] Why can't I encode/decode base64 without importing a module? In-Reply-To: <51795FA1.2070004@mrabarnett.plus.com> References: <5175C4C4.503@canterbury.ac.nz> <20130423010750.0C88E250BCA@webabinitio.net> <87k3ntz876.fsf@uwakimon.sk.tsukuba.ac.jp> <87wqrrxtmp.fsf@uwakimon.sk.tsukuba.ac.jp> <20130425074354.66e522d2@fsol> <20130425112536.07f0b782@pitrou.net> <20130425145745.16f87748@pitrou.net> <51793C08.4020703@mrabarnett.plus.com> <51795FA1.2070004@mrabarnett.plus.com> Message-ID: <87obd2xxed.fsf@uwakimon.sk.tsukuba.ac.jp> MRAB writes: > RFC 4648 says """Base encoding of data is used in many situations to > store or transfer data in environments that, perhaps for legacy reasons, > are restricted to US-ASCII [1] data.""". > > To me, "US-ASCII" is an encoding, so it appears to be talking about > encoding binary data (bytestrings) to ASCII-encoded text (bytestrings). I think that's a misreading, inconsistent with the rest of the RFC. The references to US-ASCII are not clearly normative, as the value- character mappings are given in tables, and are self-contained. (The one you quote is clearly informative, since it describes a use-case.) The term "subset of US-ASCII" suggests repertoire, not encoding, as does the use of "alphabet" to refer to these subsets. *Every* (other?) normative statement is very careful to say that input of a Base-n encoder is "octets" (with two uses of "bytes" in the definition of Base32), and the output is "characters". There are no exceptions, and there are *no* references to encoding of characters or the corresponding character codes (except the possible implicit reference via "US-ASCII"). I can make no sense of those facts if the intent of the RFC is to restrict the output of a Base-n encoder to characters encoded in (8-bit) US-ASCII. Why not just say so, and use "octets" and their ASCII codes throughout, with the corresponding characters used as informative commentary? I think it much more likely that "subset of the character repertoire of US-ASCII" was intended, but abbreviated to "subset of US-ASCII". This kind of abbreviation is very common in informal discussion of coded character sets. I admit it's a little surprising that the author would be so incautious in his use of "US-ASCII", but if he really meant US-ASCII- the-encoding, I find the style of the rest of the RFC astonishing! From barry at python.org Thu Apr 25 20:48:09 2013 From: barry at python.org (Barry Warsaw) Date: Thu, 25 Apr 2013 14:48:09 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <7BF5B3DE-E84C-4FD5-8F12-8967A5A37590@underboss.org> References: <20130423154458.3bba1e57@pitrou.net> <20130423165723.78e286a5@pitrou.net> <7BF5B3DE-E84C-4FD5-8F12-8967A5A37590@underboss.org> Message-ID: <20130425144809.2c998bdf@anarchist> On Apr 23, 2013, at 01:20 PM, Philip Jenvey wrote: >Furthermore, you could define methods/fields on enum values, like Java's >enums. There's no reason why you couldn't do this now, even with the class separation. I've documented the customization protocol, so just add these methods to the class you set in __value_factory__. No need to conflate the classes, which I think just adds unnecessary complexity and confusion. -Barry From barry at python.org Thu Apr 25 20:50:12 2013 From: barry at python.org (Barry Warsaw) Date: Thu, 25 Apr 2013 14:50:12 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130423142418.652DE250BCA@webabinitio.net> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> Message-ID: <20130425145012.6f58b91b@anarchist> On Apr 23, 2013, at 10:24 AM, R. David Murray wrote: >> >>> isinstance(C.a, C) >> False >> >>> isinstance(C(1), C) >> False >> >> It would really be better if instances were actual instances of the >> class, IMO. > >The first False looks correct to me, I would not expect an enum value to be >an instance of the class that holds it as an attribute. Agreed, completely. >The second certainly looks odd, but what does it even mean to have an >instance of an Enum class? It only looks odd because it's using failed, duplicate, deprecated syntax. Does this look similarly odd? >>> isinstance(C[1], C) False given that ``C[1] is C.a``? -Barry From tseaver at palladion.com Thu Apr 25 21:02:07 2013 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 25 Apr 2013 15:02:07 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <51795C25.6060206@stoneleaf.us> References: <20130425114257.24548e24@pitrou.net> <20130425142904.4a287fab@pitrou.net> <51794FCE.10708@stoneleaf.us> <51795C25.6060206@stoneleaf.us> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/25/2013 12:39 PM, Ethan Furman wrote: > Animals is a class. Giving Animals a parameter (such as 1 or 'ant') > should return the instance that matches. Animals is *not* a class -- it just uses the class syntax as a convenient way to set up the names used to construct the new type. (This subtlety is why the metaclass hook is reputed to make peoples' brains explode). > This is how classes work. Not really. Normal classes, when called, give you a new instance: they don't look up existing instances. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlF5fa4ACgkQ+gerLs4ltQ7FSwCgzhcoXonMO/7W+xYMpM4EvtTj nPIAnAkHtWxFMaU3dqfFUclNQkUcJ2FZ =C0/7 -----END PGP SIGNATURE----- From barry at python.org Thu Apr 25 21:15:12 2013 From: barry at python.org (Barry Warsaw) Date: Thu, 25 Apr 2013 15:15:12 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130423163302.195e4587@pitrou.net> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130423163302.195e4587@pitrou.net> Message-ID: <20130425151512.44cb87ab@anarchist> On Apr 23, 2013, at 04:33 PM, Antoine Pitrou wrote: >That said, I don't see why it wouldn't make sense for an enum value to be an >instance of that class. It can be useful to write `isinstance(value, >MyEnumClass)`. Also, any debug facility which has a preference for writing >out class names would produce a better output than the generic "EnumValue". These semantics seem very weird to me, but at least we have a principled way to lie about it in Python 3. We could add this to the metaclass: def __instancecheck__(cls, instance): return instance.enum is cls or cls in instance.enum.__bases__ Thus: >>> X = Enum('X', 'a b c') >>> Y = Enum('Y', 'z y x') >>> class Z(Y): ... d = 4 ... e = 5 ... >>> isinstance(Z.d, Y) True >>> isinstance(Z.d, Z) True >>> isinstance(Z.d, X) False >>> isinstance(Y.z, Y) True >>> isinstance(Y.z, Z) False -Barry From barry at python.org Thu Apr 25 21:20:10 2013 From: barry at python.org (Barry Warsaw) Date: Thu, 25 Apr 2013 15:20:10 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130423145355.2F527250BCA@webabinitio.net> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130423163302.195e4587@pitrou.net> <20130423145355.2F527250BCA@webabinitio.net> Message-ID: <20130425152010.250b97ab@anarchist> On Apr 23, 2013, at 10:53 AM, R. David Murray wrote: >Ah. I'd be looking for a bug every time I saw isinstance(value, >myEnumClass). A better class name for values and an API for getting that >class from the EnumClass would be nice, though. (So that you could write >"isinstance(value, MyEnumClass.ValueClass)", say.) I think if we did this, the attribute should be the same name as the one used to customize the value factory. Okay, this is horrible, but you could use isinstance(value, MyEnumClass.__value_factory__) The only thing stopping you from doing this right now is that when __value_factory__ is not given, a default is used which is not available on that attribute. That's easily corrected though. 1) really, to be consistent with the documentation, this should be __item_factory__ since the attributes of the enum class are called "items", while items have an underlying value (e.g. A.b.value). 2) A better name than either __item_factory__ or __value_factory__ is welcome, though I guess that will spawn another bikeshedding, soul-destroying centi-thread. ;) 3) I'd like to make the implementation names consistent with the documentation in the next version of flufl.enum. -Barry From barry at python.org Thu Apr 25 21:30:19 2013 From: barry at python.org (Barry Warsaw) Date: Thu, 25 Apr 2013 15:30:19 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130425114257.24548e24@pitrou.net> References: <20130425114257.24548e24@pitrou.net> Message-ID: <20130425153019.459be82b@anarchist> On Apr 25, 2013, at 11:42 AM, Antoine Pitrou wrote: >The PEP should mention how to get an enum from its raw value: > > >>> Colors[1] > > >or: > > >>> Colors(1) > No, this is a failed experiment. It's deprecated in flufl.enum and shouldn't make it into stdlib enum. getitem syntax is TOOWTDI. >It would perhaps be nice to have a .get() method that return None if the >raw value is unknown: > > >>> Colors(42) > ... > ValueError: 42 > >>> Colors.get(42) > >>> class Methods(Enum): set = 1 get = 2 delete = 3 What is "Methods.get"? Catching the ValueError ``Methods[5]`` would raise doesn't seem so bad. -Barry From barry at python.org Thu Apr 25 21:32:01 2013 From: barry at python.org (Barry Warsaw) Date: Thu, 25 Apr 2013 15:32:01 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130425114257.24548e24@pitrou.net> <20130425142904.4a287fab@pitrou.net> <51794FCE.10708@stoneleaf.us> <51795C25.6060206@stoneleaf.us> Message-ID: <20130425153201.765bca94@anarchist> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On Apr 25, 2013, at 03:02 PM, Tres Seaver wrote: >On 04/25/2013 12:39 PM, Ethan Furman wrote: >> Animals is a class. Giving Animals a parameter (such as 1 or 'ant') >> should return the instance that matches. > >Animals is *not* a class -- it just uses the class syntax as a convenient >way to set up the names used to construct the new type. (This subtlety >is why the metaclass hook is reputed to make peoples' brains explode). Thanks for the great explanation! :) - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQIcBAEBCAAGBQJReYSxAAoJEBJutWOnSwa/1a8QALXlY5GkrDTgjTNc+i3VFbpW nQ+5iYSUKk7E7A5HTE/GO1thG/3lVVLQpDvuTP+ETqHiVgZ/oKI0WFpfC7C6h85W U7qclwjfZ0kng529FscwNcICgPilHng/gOR4msatje59R9Sw54mDkIYDHllhYzdZ R1NeWzfBmKL9aZJ3xGaIKHEB6Cwwk6ZL0F7S87Voe9ipd6ihRPbJK4y574XPe/4i m8jIrEVHI/9KCiDEdJrMXX7u+61jLEmWW9UnMuDJkNItxg/oiVwknSd2munDh1Ti yf7dZPkfx4Dcarv3DH5K/D/wo8OaOiqnvrM0qTImQObCWk8RswnGT3/SJmBNsfmN gndc+UTcVDYsVHtr57SEc79Y6vEAgdOlkrOChgzEoXUh9DS2slYFGuVMajncRJTI RY14k2KzPy1FdPysADk3KW6BIZRvZHMvszNFcEQiMwrDv98zFPfCqsm1KME7eBSg 4fjD0e5f8V+yLCyeUMUZxB6KhFyPdDi53w9X2mOQx9TPJqOJPNS7kzgGcMxagLMI BEw+3L3c5B5FZSd9JIQeIm3r5Cfee9Dvfgcfd5Y3QsHzaGCSSermRXs0cqRS51Bl 4LT39RDm6E+rSguR8PSOabmhwUrfhr2KMzkfZXtR8RwStPH4Tii8zpiSgY88k9JD XUZvZxjFCwWl3syPcMNR =Eglp -----END PGP SIGNATURE----- From ethan at stoneleaf.us Thu Apr 25 22:14:19 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 13:14:19 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130425114257.24548e24@pitrou.net> <20130425142904.4a287fab@pitrou.net> <51794FCE.10708@stoneleaf.us> <51795C25.6060206@stoneleaf.us> Message-ID: <51798E9B.3030802@stoneleaf.us> On 04/25/2013 12:02 PM, Tres Seaver wrote: > On 04/25/2013 12:39 PM, Ethan Furman wrote: >> Animals is a class. Giving Animals a parameter (such as 1 or 'ant') >> should return the instance that matches. > > Animals is *not* a class -- it just uses the class syntax as a convenient > way to set up the names used to construct the new type. (This subtlety > is why the metaclass hook is reputed to make peoples' brains explode). So Animals is a type? Like int? --> int('5') # 5 --> Animal(1) # I think it should be Animal.ant ;) >> This is how classes work. > > Not really. Normal classes, when called, give you a new instance: they > don't look up existing instances. You mean like bool? -- ~Ethan~ From ethan at stoneleaf.us Thu Apr 25 22:18:47 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 13:18:47 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130425144410.096a65c4@anarchist> References: <20130423154458.3bba1e57@pitrou.net> <20130425144410.096a65c4@anarchist> Message-ID: <51798FA7.3000205@stoneleaf.us> On 04/25/2013 11:44 AM, Barry Warsaw wrote: > On Apr 23, 2013, at 03:44 PM, Antoine Pitrou wrote: > >> I'm having a problem with the proposed implementation. I haven't found >> any mention of it, so apologies if this has already been discussed: >> >>>>> from flufl.enum import * >>>>> class C(Enum): >> ... a = 1 >> ... b = 2 >> ... >>>>> C.a.__class__ >> >>>>> isinstance(C.a, C) >> False >>>>> isinstance(C(1), C) >> False >> >> It would really be better if instances were actual instances of the >> class, IMO. > > Ignore the single argument call syntax for Enums please. As Eli pointed out, > you have getitem syntax for this and the single argument call syntax is > deprecated. It will be removed in a future version of flufl.enum and need not > appear in stdlib enum. TOOWTDI. For me, the getitem syntax on a class seems odd and the call syntax is TOOWTDI. > C.a and C[1] return the same object, and it seems perfectly natural to me that > this object is *not* an instance of the enum class. In fact, it seems > completely weird to me that C.a would be an instance of the enum class. It > seems very rare that a class has attributes that are instances of that class. > It's not even easy to do with traditional syntax. > > class Foo: > a = Foo() > b = Foo() > c = Foo() Obviously you need a metaclass in there. ;) -- ~Ethan~ From barry at python.org Thu Apr 25 23:17:24 2013 From: barry at python.org (Barry Warsaw) Date: Thu, 25 Apr 2013 17:17:24 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <51798FA7.3000205@stoneleaf.us> References: <20130423154458.3bba1e57@pitrou.net> <20130425144410.096a65c4@anarchist> <51798FA7.3000205@stoneleaf.us> Message-ID: <20130425171724.2105d079@anarchist> On Apr 25, 2013, at 01:18 PM, Ethan Furman wrote: >> Ignore the single argument call syntax for Enums please. As Eli pointed >> out, you have getitem syntax for this and the single argument call syntax >> is deprecated. It will be removed in a future version of flufl.enum and >> need not appear in stdlib enum. TOOWTDI. > >For me, the getitem syntax on a class seems odd and the call syntax is >TOOWTDI. Not if you think of it as a lookup operation instead of an instantiation operation. It really is the former because neither syntax creates new enum item objects, it just returns an already existing one. -Barry From eliben at gmail.com Thu Apr 25 23:25:27 2013 From: eliben at gmail.com (Eli Bendersky) Date: Thu, 25 Apr 2013 14:25:27 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130425171724.2105d079@anarchist> References: <20130423154458.3bba1e57@pitrou.net> <20130425144410.096a65c4@anarchist> <51798FA7.3000205@stoneleaf.us> <20130425171724.2105d079@anarchist> Message-ID: On Thu, Apr 25, 2013 at 2:17 PM, Barry Warsaw wrote: > On Apr 25, 2013, at 01:18 PM, Ethan Furman wrote: > > >> Ignore the single argument call syntax for Enums please. As Eli pointed > >> out, you have getitem syntax for this and the single argument call > syntax > >> is deprecated. It will be removed in a future version of flufl.enum and > >> need not appear in stdlib enum. TOOWTDI. > > > >For me, the getitem syntax on a class seems odd and the call syntax is > >TOOWTDI. > > Not if you think of it as a lookup operation instead of an instantiation > operation. It really is the former because neither syntax creates new enum > item objects, it just returns an already existing one. > I think it's important to stress what this syntax is actually going to be used for. No one (I hope) is actually going to write Animals(1) or Animals[1]. They will write Animals.ant - this is what enums are for in the first place! The way I see it, this syntax is for enabling *programmatic access* - if you pull the value from a DB and want to convert it to an actual enum value, etc. So do we really need to have two syntaxes for this? The call syntax already has other uses, and it's weird because: Enum(....) -> Creates new enums Animals(....) --> accesses values ?! This is contradictory Animals[...] to serve as a by-value lookup makes sense, though. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Thu Apr 25 23:39:45 2013 From: barry at python.org (Barry Warsaw) Date: Thu, 25 Apr 2013 17:39:45 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130425144410.096a65c4@anarchist> <51798FA7.3000205@stoneleaf.us> <20130425171724.2105d079@anarchist> Message-ID: <20130425173945.3a4aea35@anarchist> On Apr 25, 2013, at 02:25 PM, Eli Bendersky wrote: >I think it's important to stress what this syntax is actually going to be >used for. No one (I hope) is actually going to write Animals(1) or >Animals[1]. They will write Animals.ant - this is what enums are for in the >first place! The way I see it, this syntax is for enabling *programmatic >access* - if you pull the value from a DB and want to convert it to an >actual enum value, etc. So do we really need to have two syntaxes for this? Excellent point, and no, we don't :). >The call syntax already has other uses, and it's weird because: > >Enum(....) -> Creates new enums >Animals(....) --> accesses values ?! This is contradictory > >Animals[...] to serve as a by-value lookup makes sense, though. I think so too. :) Note that I discovered that the same two-value call syntax on Enum can be used on the derived classes. It's exactly the same as using subclassing syntax to extend an existing enum. E.g. >>> class A(Enum): ... a = 1 ... b = 2 ... c = 3 ... >>> class B(A): ... d = 4 ... e = 5 ... >>> B.a is A.a True >>> X = Enum('X', 'a b c') >>> Y = X('Y', (('d', 4), ('e', 5))) >>> Y.a is X.a True That's a nice symmetry. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From ethan at stoneleaf.us Thu Apr 25 23:30:11 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 14:30:11 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130425171724.2105d079@anarchist> References: <20130423154458.3bba1e57@pitrou.net> <20130425144410.096a65c4@anarchist> <51798FA7.3000205@stoneleaf.us> <20130425171724.2105d079@anarchist> Message-ID: <5179A063.2020809@stoneleaf.us> On 04/25/2013 02:17 PM, Barry Warsaw wrote: > On Apr 25, 2013, at 01:18 PM, Ethan Furman wrote: > >>> Ignore the single argument call syntax for Enums please. As Eli pointed >>> out, you have getitem syntax for this and the single argument call syntax >>> is deprecated. It will be removed in a future version of flufl.enum and >>> need not appear in stdlib enum. TOOWTDI. >> >> For me, the getitem syntax on a class seems odd and the call syntax is >> TOOWTDI. > > Not if you think of it as a lookup operation instead of an instantiation > operation. It really is the former because neither syntax creates new enum > item objects, it just returns an already existing one. True, but I don't. ;) I think the closest comparable thing in Python is the boolean class; while True and False are not attributes of bool, they are the only two instances, and invoking bool is going to return one of the existing bool instances (which is to say, True or False). It's basically a conversion from whatever to bool. --> bool('something') # returns True --> bool(None) # returns False Similarly, an Enum converts a string or a number to it's comparable enumerator (right word, Stephen?) --> class Animal(Enum): ... ant = 1 ... bee = 2 ... fly = 3 --> Animal(2) # should return Animal.bee --> Animal('ant') # should return Animal.ant It seems to me that storing the instances on the class as attributes is mostly for convenience; we could just as easily not, and change the repr of enumerators to ' ant [int=1]'. -- ~Ethan~ From guido at python.org Thu Apr 25 23:54:54 2013 From: guido at python.org (Guido van Rossum) Date: Thu, 25 Apr 2013 14:54:54 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130425145012.6f58b91b@anarchist> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> Message-ID: On Thu, Apr 25, 2013 at 11:50 AM, Barry Warsaw wrote: > On Apr 23, 2013, at 10:24 AM, R. David Murray wrote: > >>> >>> isinstance(C.a, C) >>> False >>> >>> isinstance(C(1), C) >>> False >>> >>> It would really be better if instances were actual instances of the >>> class, IMO. >> >>The first False looks correct to me, I would not expect an enum value to be >>an instance of the class that holds it as an attribute. > > Agreed, completely. > >>The second certainly looks odd, but what does it even mean to have an >>instance of an Enum class? > > It only looks odd because it's using failed, duplicate, deprecated syntax. > Does this look similarly odd? > >>>> isinstance(C[1], C) > False > > given that ``C[1] is C.a``? I don't know what's going on, but it feels like we had this same discussion a week ago, and I still disagree. Disregarding, the C[i] notation, I feel quite strongly that in the following example: class Color(Enum): red = 1 white = 2 blue = 3 orange = 4 the values Color.red etc. should be instances of Color. This is how things work in all other languages that I am aware of that let you define enums. -- --Guido van Rossum (python.org/~guido) From barry at python.org Fri Apr 26 00:02:18 2013 From: barry at python.org (Barry Warsaw) Date: Thu, 25 Apr 2013 18:02:18 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> Message-ID: <20130425180218.69ed8004@anarchist> On Apr 25, 2013, at 02:54 PM, Guido van Rossum wrote: >I don't know what's going on, Mostly that this is my first opportunity to chime in on the subject. >but it feels like we had this same discussion a week ago, and I still >disagree. Disregarding, the C[i] notation, I feel quite strongly that in the >following example: > >class Color(Enum): > red = 1 > white = 2 > blue = 3 > orange = 4 > >the values Color.red etc. should be instances of Color. This is how >things work in all other languages that I am aware of that let you >define enums. Is it enough that isinstance(Color.red, Color) returns True, or do you really-and-truly want them to be instances of Color? I still think it's weird, but I could accept the former if you're flexible on the latter, which in some sense is just an implementation detail anyway. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Fri Apr 26 00:07:19 2013 From: barry at python.org (Barry Warsaw) Date: Thu, 25 Apr 2013 18:07:19 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179A063.2020809@stoneleaf.us> References: <20130423154458.3bba1e57@pitrou.net> <20130425144410.096a65c4@anarchist> <51798FA7.3000205@stoneleaf.us> <20130425171724.2105d079@anarchist> <5179A063.2020809@stoneleaf.us> Message-ID: <20130425180719.75035ee9@anarchist> On Apr 25, 2013, at 02:30 PM, Ethan Furman wrote: >--> class Animal(Enum): >... ant = 1 >... bee = 2 >... fly = 3 > >--> Animal(2) # should return Animal.bee > >--> Animal('ant') # should return Animal.ant > >It seems to me that storing the instances on the class as attributes is >mostly for convenience; we could just as easily not, and change the repr of >enumerators to ' ant [int=1]'. So I guess you would still expect these to hold true: >>> Animal.ant is Animal(2) True >>> Animal.ant is Animal('ant') True -Barry From barry at python.org Fri Apr 26 00:11:01 2013 From: barry at python.org (Barry Warsaw) Date: Thu, 25 Apr 2013 18:11:01 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130422011206.235C7250066@webabinitio.net> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> <20130421202816.3e22fc8f@anarchist> <20130422011206.235C7250066@webabinitio.net> Message-ID: <20130425181101.4e822cbf@anarchist> On Apr 21, 2013, at 09:12 PM, R. David Murray wrote: >Regardless of the specific values involved, it is pretty much guaranteed >that if anything other than definition order is used we *will* get bug >reports/enhancement requests to fix it, on a regular basis. We can choose >to live with that, but we should admit that it will will happen :) What's the definition order here? >>> Methods = Enum('Methods', {'set': 8, 'get': 15, 'delete': 16}) -Barry From barry at python.org Fri Apr 26 00:12:01 2013 From: barry at python.org (Barry Warsaw) Date: Thu, 25 Apr 2013 18:12:01 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> <20130421203112.018dd94a@anarchist> Message-ID: <20130425181201.75484f0f@anarchist> On Apr 22, 2013, at 10:55 AM, Tim Delaney wrote: >Would it be worthwhile storing a sorted version of the enum keys here? Or >do you think the current space vs speed tradeoff is better? It's an implementation detail that doesn't bother me too much either way. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From guido at python.org Fri Apr 26 00:19:48 2013 From: guido at python.org (Guido van Rossum) Date: Thu, 25 Apr 2013 15:19:48 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130425180218.69ed8004@anarchist> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> Message-ID: On Thu, Apr 25, 2013 at 3:02 PM, Barry Warsaw wrote: > On Apr 25, 2013, at 02:54 PM, Guido van Rossum wrote: > >>I don't know what's going on, > > Mostly that this is my first opportunity to chime in on the subject. > >>but it feels like we had this same discussion a week ago, and I still >>disagree. Disregarding, the C[i] notation, I feel quite strongly that in the >>following example: >> >>class Color(Enum): >> red = 1 >> white = 2 >> blue = 3 >> orange = 4 >> >>the values Color.red etc. should be instances of Color. This is how >>things work in all other languages that I am aware of that let you >>define enums. > > Is it enough that isinstance(Color.red, Color) returns True, or do you > really-and-truly want them to be instances of Color? > > I still think it's weird, but I could accept the former if you're flexible on > the latter, which in some sense is just an implementation detail anyway. Clearly this is a trick question. :-) I was told when this was brought up previously (a week ago?) that it would be simple to make it truly the same class. I suppose you were going to propose to use isinstance() overloading, but I honestly think that Color.red.__class__ should be the same object as Color. -- --Guido van Rossum (python.org/~guido) From barry at python.org Fri Apr 26 00:23:10 2013 From: barry at python.org (Barry Warsaw) Date: Thu, 25 Apr 2013 18:23:10 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> <20130421202816.3e22fc8f@anarchist> Message-ID: <20130425182310.1dc455ba@anarchist> On Apr 22, 2013, at 10:42 AM, Nick Coghlan wrote: >On Mon, Apr 22, 2013 at 10:28 AM, Barry Warsaw wrote: >> On Apr 22, 2013, at 09:02 AM, Nick Coghlan wrote: >> >>>Iteration order matters a lot if you don't want people complaining about >>>enums being broken: >>> >>> class Days(enum.Enum): >>> Monday = 1 >>> Tuesday = 2 >>> Wednesday = 3 >>> Thursday = 4 >>> Friday = 5 >>> Saturday = 6 >>> Sunday = 7 >> >> Sorry, that's still not a complete use case. I don't see why you'd depend >> on iteration order over Days for any particular functionality. > >You mean other than printing the days of the week in order without >needing to worry about the specific values assigned to them? My point is, "days of the week" has a natural ordering, so why wouldn't you use IntEnum for that? Problem solved. There's no natural ordering for things like colors or animals, so the values don't matter. I claim that neither does the repr or iteration order except that the former should be *predictable* and it would be nice to define the latter, but that's not actually necessary. Undefined iteration order would be just as fine for Enum. >Using sort-by-name also introduces other weirdness, such as subclasses >potentially inserting their values in the middle of inherited names, >rather than appending to the end as one might reasonably expect. I don't see how iteration order could affect how you'd write the derived class syntax. >While using sort-by-name is better than not providing a consistent >ordering at all, using definition order is substantially less >surprising than sorting by key name, and PEP 3115 and >collections.OrderedDict makes that easy to support in Python 3.x. > >The fact that this will make for a behavioural difference between the >standard library and flufl.enum does *not* count as an argument for >making the behaviour of the standard library version less intuitive >(if that was a valid argument, the 3.3+ ipaddress module would look a >*lot* more like it's ipaddr inspiration). Maybe. If they care about iteration order at all, then they will have to special case their code for Python's < 3.3, which means they'll probably have to explicitly sort it anyway for the foreseeable future. I guess it'll be moot in 10 years though. ;) They probably don't care about iteration order, which I think will be the common case (heck, iteration over the enum will be pretty rare *anyway*). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From ethan at stoneleaf.us Thu Apr 25 23:37:29 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 14:37:29 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130425144410.096a65c4@anarchist> <51798FA7.3000205@stoneleaf.us> <20130425171724.2105d079@anarchist> Message-ID: <5179A219.20109@stoneleaf.us> On 04/25/2013 02:25 PM, Eli Bendersky wrote: > On Thu, Apr 25, 2013 at 2:17 PM, Barry Warsaw > wrote: >> On Apr 25, 2013, at 01:18 PM, Ethan Furman wrote: > >>> For me, the getitem syntax on a class seems odd and the call syntax is >>> TOOWTDI. >> >> Not if you think of it as a lookup operation instead of an instantiation >> operation. It really is the former because neither syntax creates new enum >> item objects, it just returns an already existing one. > > I think it's important to stress what this syntax is actually going to be used for. No one (I hope) is actually going to > write Animals(1) or Animals[1]. They will write Animals.ant - this is what enums are for in the first place! The way I > see it, this syntax is for enabling *programmatic access* - if you pull the value from a DB and want to convert it to an > actual enum value, etc. So do we really need to have two syntaxes for this? > > The call syntax already has other uses, and it's weird because: > > Enum(....) -> Creates new enums > Animals(....) --> accesses values ?! This is contradictory > > Animals[...] to serve as a by-value lookup makes sense, though. How about consistency? If I'm converting several types of items from a database I'd like to do something like: result = [] for field in row: type = get_type(field) # returns int, bool, str, or an Enum type result.append(type(field)) What you're suggesting means complicating the logic: result = [] for field in row: type = get_type(field) # returns int, bool, str, or an Enum type if isinstance(type, Enum): result.append(type[field]) else: result.append(type(field)) We just got NoneType fixed to actually return None instead of raising an error for this same type of scenario, why should we muddy it up again? -- ~Ethan~ From barry at python.org Fri Apr 26 00:29:34 2013 From: barry at python.org (Barry Warsaw) Date: Thu, 25 Apr 2013 18:29:34 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> Message-ID: <20130425182934.3eca2633@anarchist> On Apr 25, 2013, at 03:19 PM, Guido van Rossum wrote: >Clearly this is a trick question. :-) A bit, yes. :) >I was told when this was brought up previously (a week ago?) that it >would be simple to make it truly the same class. It didn't sound simple to me, but I haven't seen any actual code yet. >I suppose you were going to propose to use isinstance() overloading, >but I honestly think that Color.red.__class__ should be the same >object as Color. Yes, a custom __instancecheck__() is two lines of code. I just can't get over the weirdness of a class having attributes which are actual instances of itself. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From guido at python.org Fri Apr 26 00:37:16 2013 From: guido at python.org (Guido van Rossum) Date: Thu, 25 Apr 2013 15:37:16 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130425182934.3eca2633@anarchist> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> Message-ID: On Thu, Apr 25, 2013 at 3:29 PM, Barry Warsaw wrote: > On Apr 25, 2013, at 03:19 PM, Guido van Rossum wrote: >>I suppose you were going to propose to use isinstance() overloading, >>but I honestly think that Color.red.__class__ should be the same >>object as Color. > > Yes, a custom __instancecheck__() is two lines of code. > > I just can't get over the weirdness of a class having attributes which are > actual instances of itself. TBH I had a hard time getting over the fact that even though the class said "a = 1", C.a is not the integer 1. But I did get over it. Hopefully you can get over *this* weirdness. -- --Guido van Rossum (python.org/~guido) From rdmurray at bitdance.com Fri Apr 26 00:46:23 2013 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 25 Apr 2013 18:46:23 -0400 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179A219.20109@stoneleaf.us> References: <20130423154458.3bba1e57@pitrou.net> <20130425144410.096a65c4@anarchist> <51798FA7.3000205@stoneleaf.us> <20130425171724.2105d079@anarchist> <5179A219.20109@stoneleaf.us> Message-ID: <20130425224623.727AE250BD0@webabinitio.net> On Thu, 25 Apr 2013 14:37:29 -0700, Ethan Furman wrote: > On 04/25/2013 02:25 PM, Eli Bendersky wrote: > > On Thu, Apr 25, 2013 at 2:17 PM, Barry Warsaw > wrote: > >> On Apr 25, 2013, at 01:18 PM, Ethan Furman wrote: > > > >>> For me, the getitem syntax on a class seems odd and the call syntax is > >>> TOOWTDI. > >> > >> Not if you think of it as a lookup operation instead of an instantiation > >> operation. It really is the former because neither syntax creates new enum > >> item objects, it just returns an already existing one. > > > > I think it's important to stress what this syntax is actually going to be used for. No one (I hope) is actually going to > > write Animals(1) or Animals[1]. They will write Animals.ant - this is what enums are for in the first place! The way I > > see it, this syntax is for enabling *programmatic access* - if you pull the value from a DB and want to convert it to an > > actual enum value, etc. So do we really need to have two syntaxes for this? > > > > The call syntax already has other uses, and it's weird because: > > > > Enum(....) -> Creates new enums > > Animals(....) --> accesses values ?! This is contradictory > > > > Animals[...] to serve as a by-value lookup makes sense, though. > > How about consistency? > > If I'm converting several types of items from a database I'd like to do something like: > > result = [] > for field in row: > type = get_type(field) # returns int, bool, str, or an Enum type > result.append(type(field)) > > > What you're suggesting means complicating the logic: > > result = [] > for field in row: > type = get_type(field) # returns int, bool, str, or an Enum type > if isinstance(type, Enum): > result.append(type[field]) > else: > result.append(type(field)) > > We just got NoneType fixed to actually return None instead of raising an error for this same type of scenario, why > should we muddy it up again? I haven't cared much about this particular bikeshed, but I find this a somewhat compelling argument. I'm working on a system that depends on exactly this standard behavior of (most?) built in types in Python: if you pass an instance or something that can be converted to an instance to the type constructor, you get back an instance. If Enums break that paradigm(*), someone would have to write a custom class that provided that behavior in order to use Enums with my system. I wouldn't say that was a show stopper, especially since my system may never go anywhere :), but it certainly is an exemplar of the issue Eli is talking about. --David (*) Hmm. NoneType(None) is still an error. From ethan at stoneleaf.us Fri Apr 26 00:23:55 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 15:23:55 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130425180719.75035ee9@anarchist> References: <20130423154458.3bba1e57@pitrou.net> <20130425144410.096a65c4@anarchist> <51798FA7.3000205@stoneleaf.us> <20130425171724.2105d079@anarchist> <5179A063.2020809@stoneleaf.us> <20130425180719.75035ee9@anarchist> Message-ID: <5179ACFB.7000602@stoneleaf.us> On 04/25/2013 03:07 PM, Barry Warsaw wrote: > On Apr 25, 2013, at 02:30 PM, Ethan Furman wrote: > >> --> class Animal(Enum): >> ... ant = 1 >> ... bee = 2 >> ... fly = 3 >> >> --> Animal(2) # should return Animal.bee >> >> --> Animal('ant') # should return Animal.ant >> >> It seems to me that storing the instances on the class as attributes is >> mostly for convenience; we could just as easily not, and change the repr of >> enumerators to ' ant [int=1]'. > > So I guess you would still expect these to hold true: > > >>> Animal.ant is Animal(2) > True > >>> Animal.ant is Animal('ant') > True Well, except for ant being 1, yeah. ;) -- ~Ethan~ From ethan at stoneleaf.us Fri Apr 26 00:26:43 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 15:26:43 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130425181101.4e822cbf@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> <20130421202816.3e22fc8f@anarchist> <20130422011206.235C7250066@webabinitio.net> <20130425181101.4e822cbf@anarchist> Message-ID: <5179ADA3.6080706@stoneleaf.us> On 04/25/2013 03:11 PM, Barry Warsaw wrote: > On Apr 21, 2013, at 09:12 PM, R. David Murray wrote: > >> Regardless of the specific values involved, it is pretty much guaranteed >> that if anything other than definition order is used we *will* get bug >> reports/enhancement requests to fix it, on a regular basis. We can choose >> to live with that, but we should admit that it will will happen :) > > What's the definition order here? > > >>> Methods = Enum('Methods', {'set': 8, 'get': 15, 'delete': 16}) Whatever random order {}.keys() returns -- and python programmers should be expecting that (or at least smack themselves for forgetting it ;) -- ~Ethan~ From ethan at stoneleaf.us Fri Apr 26 01:03:47 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 16:03:47 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130425224623.727AE250BD0@webabinitio.net> References: <20130423154458.3bba1e57@pitrou.net> <20130425144410.096a65c4@anarchist> <51798FA7.3000205@stoneleaf.us> <20130425171724.2105d079@anarchist> <5179A219.20109@stoneleaf.us> <20130425224623.727AE250BD0@webabinitio.net> Message-ID: <5179B653.3070409@stoneleaf.us> On 04/25/2013 03:46 PM, R. David Murray wrote: > On Thu, 25 Apr 2013 14:37:29 -0700, Ethan Furman wrote: >> On 04/25/2013 02:25 PM, Eli Bendersky wrote: >>> On Thu, Apr 25, 2013 at 2:17 PM, Barry Warsaw > wrote: >>>> On Apr 25, 2013, at 01:18 PM, Ethan Furman wrote: >>> >>>>> For me, the getitem syntax on a class seems odd and the call syntax is >>>>> TOOWTDI. >>>> >>>> Not if you think of it as a lookup operation instead of an instantiation >>>> operation. It really is the former because neither syntax creates new enum >>>> item objects, it just returns an already existing one. >>> >>> I think it's important to stress what this syntax is actually going to be used for. No one (I hope) is actually going to >>> write Animals(1) or Animals[1]. They will write Animals.ant - this is what enums are for in the first place! The way I >>> see it, this syntax is for enabling *programmatic access* - if you pull the value from a DB and want to convert it to an >>> actual enum value, etc. So do we really need to have two syntaxes for this? >>> >>> The call syntax already has other uses, and it's weird because: >>> >>> Enum(....) -> Creates new enums >>> Animals(....) --> accesses values ?! This is contradictory >>> >>> Animals[...] to serve as a by-value lookup makes sense, though. >> >> How about consistency? >> >> If I'm converting several types of items from a database I'd like to do something like: >> >> result = [] >> for field in row: >> type = get_type(field) # returns int, bool, str, or an Enum type >> result.append(type(field)) >> >> >> What you're suggesting means complicating the logic: >> >> result = [] >> for field in row: >> type = get_type(field) # returns int, bool, str, or an Enum type >> if isinstance(type, Enum): >> result.append(type[field]) >> else: >> result.append(type(field)) >> >> We just got NoneType fixed to actually return None instead of raising an error for this same type of scenario, why >> should we muddy it up again? > > I haven't cared much about this particular bikeshed, but I find this a > somewhat compelling argument. I'm working on a system that depends on > exactly this standard behavior of (most?) built in types in Python: if > you pass an instance or something that can be converted to an instance > to the type constructor, you get back an instance. If Enums break that > paradigm(*), someone would have to write a custom class that provided > that behavior in order to use Enums with my system. I wouldn't say that > was a show stopper, especially since my system may never go anywhere :), > but it certainly is an exemplar of the issue Eli is talking about. Um, did you mean "of the issue Ethan is talking about"? 'Cause Eli is against it. > (*) Hmm. NoneType(None) is still an error. Hmm, so it is. When I pushed for the change just having NoneType() work was sufficient. -- ~Ethan~ From ethan at stoneleaf.us Fri Apr 26 00:36:53 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 15:36:53 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130425182310.1dc455ba@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> <20130421202816.3e22fc8f@anarchist> <20130425182310.1dc455ba@anarchist> Message-ID: <5179B005.2030601@stoneleaf.us> On 04/25/2013 03:23 PM, Barry Warsaw wrote: > On Apr 22, 2013, at 10:42 AM, Nick Coghlan wrote: > >> On Mon, Apr 22, 2013 at 10:28 AM, Barry Warsaw wrote: >>> On Apr 22, 2013, at 09:02 AM, Nick Coghlan wrote: >>> >>>> Iteration order matters a lot if you don't want people complaining about >>>> enums being broken: >>>> >>>> class Days(enum.Enum): >>>> Monday = 1 >>>> Tuesday = 2 >>>> Wednesday = 3 >>>> Thursday = 4 >>>> Friday = 5 >>>> Saturday = 6 >>>> Sunday = 7 >>> >>> Sorry, that's still not a complete use case. I don't see why you'd depend >>> on iteration order over Days for any particular functionality. >> >> You mean other than printing the days of the week in order without >> needing to worry about the specific values assigned to them? > > My point is, "days of the week" has a natural ordering, so why wouldn't you > use IntEnum for that? Problem solved. Because by using an IntEnum we lose the Enum type protection, which is one of the reasons to want an Enum type to begin with. > There's no natural ordering for things like colors or animals, so the values > don't matter. I claim that neither does the repr or iteration order except > that the former should be *predictable* and it would be nice to define the > latter, but that's not actually necessary. Undefined iteration order would be > just as fine for Enum. People like things sorted (or am I alone here?) There are three obvious natural orderings: 1 - value 2 - definition order 3 - name And that's my order of preference for them. >> Using sort-by-name also introduces other weirdness, such as subclasses >> potentially inserting their values in the middle of inherited names, >> rather than appending to the end as one might reasonably expect. > > I don't see how iteration order could affect how you'd write the derived class > syntax. It probably wouldn't, but if I had: --> class Color(Enum): ... red = 1 ... blue = 2 ... green = 3 --> class MoreColor(Color): ... cyan = 4 ... magenta = 5 ... yellow = 6 I would be very unhappy with: --> list(MoreColor) [ MoreColor.blue, MoreColor.cyan, MoreColor.green, MoreColor.magenta, MoreColor.red, MoreColor.yellow, ] because 1) it's not the order I defined it in; and 2) it's not in value order. -- ~Ethan~ From v+python at g.nevcal.com Fri Apr 26 01:26:54 2013 From: v+python at g.nevcal.com (Glenn Linderman) Date: Thu, 25 Apr 2013 16:26:54 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> Message-ID: <5179BBBE.4010309@g.nevcal.com> On 4/25/2013 3:37 PM, Guido van Rossum wrote: > On Thu, Apr 25, 2013 at 3:29 PM, Barry Warsaw wrote: >> >On Apr 25, 2013, at 03:19 PM, Guido van Rossum wrote: >>> >>I suppose you were going to propose to use isinstance() overloading, >>> >>but I honestly think that Color.red.__class__ should be the same >>> >>object as Color. >> > >> >Yes, a custom __instancecheck__() is two lines of code. >> > >> >I just can't get over the weirdness of a class having attributes which are >> >actual instances of itself. > TBH I had a hard time getting over the fact that even though the class > said "a = 1", C.a is not the integer 1. But I did get over it. > Hopefully you can get over*this* weirdness. :) The minute "metaclass" is involved, it is no longer a class, but a whatchamacallit. What blows people's minds regarding metaclasses is that they aren't necessarily what they look like. In this case, we are defining enumerations using a syntax that sort of looks like class attribute syntax, which are neither classes, nor types, but rather a collection of cooperative objects with shared methods in a shared subsidiary namespace. Perhaps instead of metaclass altering the behavior of classes, it might have been less confusing to simply define a new keyword "whatchamacallit" (or "metaclass") that have behaviors defined by the various methods defined in connection with it... while the existing techniques exist and would require a deprecation cycle to eliminate, it may not be too late to define alternate keywords that make it clearer that using such keywords, is making user-defined objects of some sort, class-like or -unlike, type-like or -unlike, object-like or -unlike, cooperative or not, etc., all by user definition. Any behavioral resemblance to classes would be only by user specification of same (perhaps such a specification of "same as class" should be easy). My question is, once an enumeration is defined, is there a way, short of element-by-element assignment, to import the individual enumeration instances into the current namespace, so that I can say "red" instead of "Color.red" ? I understand the benefits of avoiding name collisions when there are lots of enumerations, and lots of opportunities for name collections between, say, RGBColor and CYMKColor... but there are lots of uses for enumerations where the subsidiary namespace is just aggravating noise. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Fri Apr 26 01:09:41 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 16:09:41 -0700 Subject: [Python-Dev] NoneType(None) raises exception Message-ID: <5179B7B5.2030901@stoneleaf.us> We just fixed NoneType() to return None instead of raising an exception. Another use-case for calling NoneType is working with ORMs: result = [] for field in row: type = get_type(field) # returns int, bool, str, NoneType, ... result.append(type(field)) if field is None, the resulting call is NoneType(None), and since NoneType doesn't take any parameters we get an exception. Is it worth filing a bug to have NoneType accept one optional argument, which defaults to None, and must be None, else raise an exception? Or should it be: class NoneType: def __new__(cls, *args, **kws): return None Which is basically what my 2.x none() helper function does... -- ~Ethan~ From benjamin at python.org Fri Apr 26 01:35:10 2013 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 25 Apr 2013 19:35:10 -0400 Subject: [Python-Dev] NoneType(None) raises exception In-Reply-To: <5179B7B5.2030901@stoneleaf.us> References: <5179B7B5.2030901@stoneleaf.us> Message-ID: 2013/4/25 Ethan Furman : > We just fixed NoneType() to return None instead of raising an exception. > > Another use-case for calling NoneType is working with ORMs: > > result = [] > for field in row: > type = get_type(field) # returns int, bool, str, NoneType, ... > result.append(type(field)) > > > if field is None, the resulting call is NoneType(None), and since NoneType > doesn't take any parameters we get an exception. > > Is it worth filing a bug to have NoneType accept one optional argument, > which defaults to None, and must be None, else raise an exception? IMO, that has no interesting semantic meaning and defining your own none function is a perfectly acceptable way of dealing with your problem. -- Regards, Benjamin From python at mrabarnett.plus.com Fri Apr 26 01:56:46 2013 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 26 Apr 2013 00:56:46 +0100 Subject: [Python-Dev] NoneType(None) raises exception In-Reply-To: <5179B7B5.2030901@stoneleaf.us> References: <5179B7B5.2030901@stoneleaf.us> Message-ID: <5179C2BE.1040704@mrabarnett.plus.com> On 26/04/2013 00:09, Ethan Furman wrote: > We just fixed NoneType() to return None instead of raising an exception. > > Another use-case for calling NoneType is working with ORMs: > > result = [] > for field in row: > type = get_type(field) # returns int, bool, str, NoneType, ... > result.append(type(field)) > > > if field is None, the resulting call is NoneType(None), and since NoneType doesn't take any parameters we get an exception. > > Is it worth filing a bug to have NoneType accept one optional argument, which defaults to None, and must be None, else > raise an exception? > > Or should it be: > > class NoneType: > def __new__(cls, *args, **kws): > return None > > Which is basically what my 2.x none() helper function does... > On the one hand, NoneType(None) seems a strange thing to do. On the other hand: type(value)(value) == value would return True for the built-in types (will certain exceptions, such as when value is float("NaN")). Let's ask the Zen: Special cases aren't special enough to break the rules. Although practicality beats purity. From v+python at g.nevcal.com Fri Apr 26 01:59:52 2013 From: v+python at g.nevcal.com (Glenn Linderman) Date: Thu, 25 Apr 2013 16:59:52 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130425182310.1dc455ba@anarchist> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> <20130421202816.3e22fc8f@anarchist> <20130425182310.1dc455ba@anarchist> Message-ID: <5179C378.60906@g.nevcal.com> On 4/25/2013 3:23 PM, Barry Warsaw wrote: > My point is, "days of the week" has a natural ordering, so why wouldn't you > use IntEnum for that? Problem solved. While the ordering is natural, some implementations start from 0, some start from 1, and on the naming side, some start from Sunday, and some start from Monday. So there are lots of opportunities for the numbers vary, and having easy conversions to int which expose those numbers and allow comparisons to integers, is adding problems, not solving them. On 4/25/2013 3:36 PM, Ethan Furman wrote: > People like things sorted (or am I alone here?) There are three > obvious natural orderings: > > 1 - value > > 2 - definition order > > 3 - name > > And that's my order of preference for them. So, being blissfully unaware of the implementation issues, not having yet read the implementation, I suggest that the preferred iteration order should be a characteristic of the Enum when defined, and/or that there should be a method to obtain a list in any of the natural orderings (specifying value ordering may add a restriction that the values be orderable, if done at definition time; if done only on retrieval, attempting to access the list by value would raise an exception). Another possible ordering would be "random", and for enumerations with values that are tuples or lists or hashes, ordering by some specified element would be conceivable. On the other hand, except for "definition" order, all the other possible orderings could be derived externally to the enumeration. So if enumerations preserve/provide "definition" order, all the others could be implemented externally or as subtypes. "Definition" order, then is the basic ordering for enumerations because it cannot be externally derived, other than by reading the source code. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Fri Apr 26 02:27:59 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 26 Apr 2013 10:27:59 +1000 Subject: [Python-Dev] NoneType(None) raises exception In-Reply-To: <5179C2BE.1040704@mrabarnett.plus.com> References: <5179B7B5.2030901@stoneleaf.us> <5179C2BE.1040704@mrabarnett.plus.com> Message-ID: <5179CA0F.7020605@pearwood.info> On 26/04/13 09:56, MRAB wrote: > On the one hand, NoneType(None) seems a strange thing to do. Only when you write it out like that as constants. It's no more, or less, strange than str('spam') or int(1) or list([]). Why would you do that? But as soon as you think of it in general terms: some_type(some_instance) that's a pretty normal thing to do. And if it just so happened that some_instance were an instance of some_type, it would be surprising if the call failed. (I initially wrote "astonishing", but then I realised that some types take more than one argument, e.g. FunctionType. So it's merely surprising.) > On the other hand: > > type(value)(value) == value > > would return True for the built-in types (will certain exceptions, such > as when value is float("NaN")). Not an exception, that works fine in 3.3: >>> value = float('nan') >>> type(value)(value) nan > Let's ask the Zen: > > Special cases aren't special enough to break the rules. > > Although practicality beats purity. I cannot think of any use-case where I would actively want NoneType(None) to fail. That would be like having bool(True) raise an exception. On the other hand, NoneType(x) for any other x ought to fail. -- Steven From ethan at stoneleaf.us Fri Apr 26 01:53:31 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 16:53:31 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179BBBE.4010309@g.nevcal.com> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179BBBE.4010309@g.nevcal.com> Message-ID: <5179C1FB.6010805@stoneleaf.us> On 04/25/2013 04:26 PM, Glenn Linderman wrote: > My question is, once an enumeration is defined, is there a way, short of element-by-element assignment, to import the > individual enumeration instances into the current namespace, so that I can say? "red" instead of "Color.red" ? I > understand the benefits of avoiding name collisions when there are lots of enumerations, and lots of opportunities for > name collections between, say, RGBColor and CYMKColor... but there are lots of uses for enumerations where the > subsidiary namespace is just aggravating noise. You mean something like: --> class Color(Enum): ... RED = 1 ... GREEN = 2 ... BLUE = 3 --> Color.register() # puts Color in sys.modules --> from Color import * # doesn't work in a function, though :( --> BLUE Color.BLUE Yeah, that would be nice. ;) A bit dangerous, though -- what if another module does the same thing, but its Color is different? Better would be: --> Color.export(globals()) # put the enumerators in globals --> RED Color.RED -- ~Ethan~ From greg.ewing at canterbury.ac.nz Fri Apr 26 02:54:39 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 26 Apr 2013 12:54:39 +1200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> Message-ID: <5179D04F.1030401@canterbury.ac.nz> > On Thu, Apr 25, 2013 at 3:29 PM, Barry Warsaw wrote: >> I just can't get over the weirdness of a class having attributes which are >> actual instances of itself. I wonder how many programmers will even notice that this characteristic exists. Exactly the same weirdness occurs in Java, but I had never thought about it until this discussion came up, and I wondered "Hmmm... seems like this ought to happen in Java too", tried it out, and found that it did. -- Greg From python at mrabarnett.plus.com Fri Apr 26 03:03:24 2013 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 26 Apr 2013 02:03:24 +0100 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179C378.60906@g.nevcal.com> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> <20130421202816.3e22fc8f@anarchist> <20130425182310.1dc455ba@anarchist> <5179C378.60906@g.nevcal.com> Message-ID: <5179D25C.6040305@mrabarnett.plus.com> On 26/04/2013 00:59, Glenn Linderman wrote: > On 4/25/2013 3:23 PM, Barry Warsaw wrote: >> My point is, "days of the week" has a natural ordering, so why wouldn't you >> use IntEnum for that? Problem solved. > > While the ordering is natural, some implementations start from 0, some > start from 1, and on the naming side, some start from Sunday, and some > start from Monday. So there are lots of opportunities for the numbers > vary, and having easy conversions to int which expose those numbers and > allow comparisons to integers, is adding problems, not solving them. > [snip] But there _is_ an ordering problem, in that the days wrap around. From greg.ewing at canterbury.ac.nz Fri Apr 26 03:10:07 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 26 Apr 2013 13:10:07 +1200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179C1FB.6010805@stoneleaf.us> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179BBBE.4010309@g.nevcal.com> <5179C1FB.6010805@stoneleaf.us> Message-ID: <5179D3EF.5030106@canterbury.ac.nz> On 26/04/13 11:53, Ethan Furman wrote: > > --> from Color import * # doesn't work in a function, though :( > > Yeah, that would be nice. ;) A bit dangerous, though -- what if another module > does the same thing, but its Color is different? It needs to put foo.Color in sys.modules, where foo is the name of the defining module. Then you do from foo.Color import * The drawback being that you need to write the name of the module into the import statement. It's disappointing that the import syntax doesn't have a way of saying "this module". -- Greg From python at mrabarnett.plus.com Fri Apr 26 03:12:56 2013 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 26 Apr 2013 02:12:56 +0100 Subject: [Python-Dev] NoneType(None) raises exception In-Reply-To: <5179CA0F.7020605@pearwood.info> References: <5179B7B5.2030901@stoneleaf.us> <5179C2BE.1040704@mrabarnett.plus.com> <5179CA0F.7020605@pearwood.info> Message-ID: <5179D498.6030804@mrabarnett.plus.com> On 26/04/2013 01:27, Steven D'Aprano wrote: > On 26/04/13 09:56, MRAB wrote: > >> On the one hand, NoneType(None) seems a strange thing to do. > > Only when you write it out like that as constants. It's no more, > or less, strange than str('spam') or int(1) or list([]). Why > would you do that? > None is a singleton, but instances of str, int, list, etc aren't. Why can it take an argument when there's only ever one of them? That's why it seems strange to me. > But as soon as you think of it in general terms: > > some_type(some_instance) > > that's a pretty normal thing to do. And if it just so happened > that some_instance were an instance of some_type, it would be > surprising if the call failed. > > (I initially wrote "astonishing", but then I realised that some > types take more than one argument, e.g. FunctionType. So it's > merely surprising.) > >> On the other hand: >> >> type(value)(value) == value >> >> would return True for the built-in types (will certain exceptions, such >> as when value is float("NaN")). > > Not an exception, that works fine in 3.3: > >>>> value = float('nan') >>>> type(value)(value) > nan > But: >>> value = float('NaN') >>> type(value)(value) == value False That's what I mean by it being an "exception". >> Let's ask the Zen: >> >> Special cases aren't special enough to break the rules. >> >> Although practicality beats purity. > > I cannot think of any use-case where I would actively want > NoneType(None) to fail. That would be like having bool(True) > raise an exception. > > On the other hand, NoneType(x) for any other x ought to fail. > OK, so practicality (or pragmatism) wins. From greg.ewing at canterbury.ac.nz Fri Apr 26 03:23:19 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 26 Apr 2013 13:23:19 +1200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179D25C.6040305@mrabarnett.plus.com> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> <20130421202816.3e22fc8f@anarchist> <20130425182310.1dc455ba@anarchist> <5179C378.60906@g.nevcal.com> <5179D25C.6040305@mrabarnett.plus.com> Message-ID: <5179D707.1020402@canterbury.ac.nz> On 26/04/13 13:03, MRAB wrote: > But there _is_ an ordering problem, in that the days wrap around. Do we want a CircularEnum, then? Ordering would be defined only up to the starting value, which you would be required to specify when doing anything where it mattered. class Day(CircularEnum): sunday = 0 monday = 1 ... saturday = 6 list(Day.startingat(Day.tuesday)) --> [Day.tuesday, Day,wednesday, Day.thursday, Day.friday, Day.saturday, Day.sunday, Day.monday] Modular arithmetic would apply, so Day.saturday + 3 --> Day.tuesday That would be the replacement for Day(3), which would be disallowed. -- Greg From ethan at stoneleaf.us Fri Apr 26 02:43:21 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 17:43:21 -0700 Subject: [Python-Dev] NoneType(None) raises exception In-Reply-To: <5179CA0F.7020605@pearwood.info> References: <5179B7B5.2030901@stoneleaf.us> <5179C2BE.1040704@mrabarnett.plus.com> <5179CA0F.7020605@pearwood.info> Message-ID: <5179CDA9.6030905@stoneleaf.us> On 04/25/2013 05:27 PM, Steven D'Aprano wrote: > On 26/04/13 09:56, MRAB wrote: >> On the other hand: >> >> type(value)(value) == value >> >> would return True for the built-in types (will certain exceptions, such >> as when value is float("NaN")). > > Not an exception, that works fine in 3.3: > >>>> value = float('nan') >>>> type(value)(value) > nan You missed the == part: >>> type(value)(value) == value False >> Let's ask the Zen: >> >> Special cases aren't special enough to break the rules. >> >> Although practicality beats purity. > > > I cannot think of any use-case where I would actively want > NoneType(None) to fail. That would be like having bool(True) > raise an exception. > > On the other hand, NoneType(x) for any other x ought to fail. Or, since the purpose of NoneType is to return None, just return None no matter what! Kind'a like 0 * anything == 0. -- ~Ethan~ From ethan at stoneleaf.us Fri Apr 26 03:15:32 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 18:15:32 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179D25C.6040305@mrabarnett.plus.com> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> <20130421202816.3e22fc8f@anarchist> <20130425182310.1dc455ba@anarchist> <5179C378.60906@g.nevcal.com> <5179D25C.6040305@mrabarnett.plus.com> Message-ID: <5179D534.6050506@stoneleaf.us> On 04/25/2013 06:03 PM, MRAB wrote: > On 26/04/2013 00:59, Glenn Linderman wrote: >> On 4/25/2013 3:23 PM, Barry Warsaw wrote: >>> My point is, "days of the week" has a natural ordering, so why wouldn't you >>> use IntEnum for that? Problem solved. >> >> While the ordering is natural, some implementations start from 0, some >> start from 1, and on the naming side, some start from Sunday, and some >> start from Monday. So there are lots of opportunities for the numbers >> vary, and having easy conversions to int which expose those numbers and >> allow comparisons to integers, is adding problems, not solving them. >> > [snip] > But there _is_ an ordering problem, in that the days wrap around. Lots of counting systems wrap: seconds, minutes, hours, days of week, days of month, days of year, millimeters, inches, etc., etc., and so forth. We still apply ordering to them, and talk about 15 being less than 42 . -- ~Ethan~ From ethan at stoneleaf.us Fri Apr 26 03:28:17 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 18:28:17 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179D707.1020402@canterbury.ac.nz> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> <20130421202816.3e22fc8f@anarchist> <20130425182310.1dc455ba@anarchist> <5179C378.60906@g.nevcal.com> <5179D25C.6040305@mrabarnett.plus.com> <5179D707.1020402@canterbury.ac.nz> Message-ID: <5179D831.9060903@stoneleaf.us> On 04/25/2013 06:23 PM, Greg Ewing wrote: > On 26/04/13 13:03, MRAB wrote: >> But there _is_ an ordering problem, in that the days wrap around. > > Do we want a CircularEnum, then? > > Ordering would be defined only up to the starting > value, which you would be required to specify when > doing anything where it mattered. > > class Day(CircularEnum): > > sunday = 0 > monday = 1 > ... > saturday = 6 > > list(Day.startingat(Day.tuesday)) --> > [Day.tuesday, Day,wednesday, Day.thursday, > Day.friday, Day.saturday, Day.sunday, > Day.monday] > > Modular arithmetic would apply, so > > Day.saturday + 3 --> Day.tuesday > > That would be the replacement for Day(3), > which would be disallowed. Interesting idea, but why does Day(3) have to be disallowed to make it work? -- ~Ethan~ From barry at python.org Fri Apr 26 04:05:27 2013 From: barry at python.org (Barry Warsaw) Date: Thu, 25 Apr 2013 22:05:27 -0400 Subject: [Python-Dev] NoneType(None) raises exception In-Reply-To: <5179B7B5.2030901@stoneleaf.us> References: <5179B7B5.2030901@stoneleaf.us> Message-ID: <20130425220527.6533e3ee@anarchist> On Apr 25, 2013, at 04:09 PM, Ethan Furman wrote: >We just fixed NoneType() to return None instead of raising an exception. > >Another use-case for calling NoneType is working with ORMs: > >result = [] >for field in row: > type = get_type(field) # returns int, bool, str, NoneType, ... > result.append(type(field)) I know this use case came up w.r.t. the enum discussion, but I'm not sure it's relevant in a practical sense. Most of the Python ORMs I've worked with have an extension facility to allow custom types/classes to be stored and retrieved into database columns, and these converters always have intimate knowledge of the types they're dealing with, both on the Python side and on the db side. E.g. datetimes to take a pretty common standard Python data type. So while a generic callable API is nice, I'm not sure you'll ever be able to get full coverage over common Python types. -Barry From v+python at g.nevcal.com Fri Apr 26 04:09:14 2013 From: v+python at g.nevcal.com (Glenn Linderman) Date: Thu, 25 Apr 2013 19:09:14 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179C1FB.6010805@stoneleaf.us> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179BBBE.4010309@g.nevcal.com> <5179C1FB.6010805@stoneleaf.us> Message-ID: <5179E1CA.3000301@g.nevcal.com> On 4/25/2013 4:53 PM, Ethan Furman wrote: > On 04/25/2013 04:26 PM, Glenn Linderman wrote: >> My question is, once an enumeration is defined, is there a way, short >> of element-by-element assignment, to import the >> individual enumeration instances into the current namespace, so that >> I can say? "red" instead of "Color.red" ? I >> understand the benefits of avoiding name collisions when there are >> lots of enumerations, and lots of opportunities for >> name collections between, say, RGBColor and CYMKColor... but there >> are lots of uses for enumerations where the >> subsidiary namespace is just aggravating noise. > > You mean something like: > > --> class Color(Enum): > ... RED = 1 > ... GREEN = 2 > ... BLUE = 3 > > --> Color.register() # puts Color in sys.modules > > --> from Color import * # doesn't work in a function, though :( > > --> BLUE > Color.BLUE Something like that, but that works in a function too :) > Yeah, that would be nice. ;) A bit dangerous, though -- what if > another module does the same thing, but its Color is different? > > Better would be: > > --> Color.export(globals()) # put the enumerators in globals > > --> RED > Color.RED Globals? locals should be possible too. Or even something like: with Color: BLUE RED Although the extra indentation could also be annoying. One wouldn't want the module defining Color to automatically 'export' the colors: but rather a way to request an 'export' them into a particular scope. That way the proliferation of names into scopes is chosen by the programmer. import module_containing_color module_containing_color.Color.export_enumerations( globals ) or import module_containing_color module_containing_color.Color.export_enumerations( locals ) Or maybe locals is implicit, and in the file scope of a module, locals are globals anyway, so doing module_containing_color.Color.export_enumerations() would make the enumerations available to all definitions in the file, but inside a class or def doing the same thing would make the names direct members of the class or locals in the function. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Apr 26 04:13:12 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 26 Apr 2013 12:13:12 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130425182934.3eca2633@anarchist> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> Message-ID: On Fri, Apr 26, 2013 at 8:29 AM, Barry Warsaw wrote: > On Apr 25, 2013, at 03:19 PM, Guido van Rossum wrote: > >>Clearly this is a trick question. :-) > > A bit, yes. :) > >>I was told when this was brought up previously (a week ago?) that it >>would be simple to make it truly the same class. > > It didn't sound simple to me, but I haven't seen any actual code yet. I'm the one who said I didn't see any obvious barriers to the merger, but I've realised there is one, and it's similar to one namedtuple struggles with: how to handle method definitions. With the current flufl.enum design, if you want to give instances of a particular enum additional behaviour, you can use a custom item type and add the methods there. Simple and relatively obvious. With a merged design, it becomes *really* hard to give the instances custom behaviour, because the metaclass will somehow have to differentiate between namespace entries that are intended to be callables, and those which are intended to be instances of the enum. This is not an easy problem to solve. So, while I was initially in the "merge them" camp, I'm back to thinking the core architecture of flufl.enum is correct, but there may be some API adjustment to do, such as: 1. Lose the __getitem__ on the metaclass, and replace that with __call__ 2. Ensure that isinstance(MyEnum.item, MyEnum) returns True (even though it isn't really) 3. Inspired by namedtuple, move the current Enum constructor functionality to an Enum._make() API (implemented either as a class method in Enum or as an ordinary method on the metaclass) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From jeanpierreda at gmail.com Fri Apr 26 04:25:02 2013 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Thu, 25 Apr 2013 22:25:02 -0400 Subject: [Python-Dev] NoneType(None) raises exception In-Reply-To: <5179D498.6030804@mrabarnett.plus.com> References: <5179B7B5.2030901@stoneleaf.us> <5179C2BE.1040704@mrabarnett.plus.com> <5179CA0F.7020605@pearwood.info> <5179D498.6030804@mrabarnett.plus.com> Message-ID: On Thu, Apr 25, 2013 at 9:12 PM, MRAB wrote: >> Only when you write it out like that as constants. It's no more, >> or less, strange than str('spam') or int(1) or list([]). Why >> would you do that? >> > None is a singleton, but instances of str, int, list, etc aren't. Why > can it take an argument when there's only ever one of them? > > That's why it seems strange to me. How about bool? False and True are singletons much like None is, and bool(False) == False; bool(True) == True. Sure the distinction is that all of those are useful as conversion functions, whereas NoneType would never be used that way. -- Devin From guido at python.org Fri Apr 26 04:38:22 2013 From: guido at python.org (Guido van Rossum) Date: Thu, 25 Apr 2013 19:38:22 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> Message-ID: On Thu, Apr 25, 2013 at 7:13 PM, Nick Coghlan wrote: > I'm the one who said I didn't see any obvious barriers to the merger, > but I've realised there is one, and it's similar to one namedtuple > struggles with: how to handle method definitions. > > With the current flufl.enum design, if you want to give instances of a > particular enum additional behaviour, you can use a custom item type > and add the methods there. Simple and relatively obvious. > > With a merged design, it becomes *really* hard to give the instances > custom behaviour, because the metaclass will somehow have to > differentiate between namespace entries that are intended to be > callables, and those which are intended to be instances of the enum. > This is not an easy problem to solve. It's really sad that this technical problem exists (and I realize what it is) -- because the obvious syntax (whose equivalent works in Java, BTW) *seems* so natural: class Color: red = 1 white = 2 blue = 3 orange = 4 def wave(self, n=1): for _ in range(n): print('Waving', self) a = Color.red a.wave() Color.orange.wave(3) > So, while I was initially in the "merge them" camp, I'm back to > thinking the core architecture of flufl.enum is correct, but there may > be some API adjustment to do, such as: > > 1. Lose the __getitem__ on the metaclass, and replace that with __call__ > 2. Ensure that isinstance(MyEnum.item, MyEnum) returns True (even > though it isn't really) If the above syntax won't work, that isinstance() outcome isn't really important. :-( Can't we do some kind of callable check? There may be some weird decorators that won't work, but they aren't likely to be useful in this context. > 3. Inspired by namedtuple, move the current Enum constructor > functionality to an Enum._make() API (implemented either as a class > method in Enum or as an ordinary method on the metaclass) -- --Guido van Rossum (python.org/~guido) From ethan at stoneleaf.us Fri Apr 26 04:25:06 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 19:25:06 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179E1CA.3000301@g.nevcal.com> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179BBBE.4010309@g.nevcal.com> <5179C1FB.6010805@stoneleaf.us> <5179E1CA.3000301@g.nevcal.com> Message-ID: <5179E582.5080102@stoneleaf.us> On 04/25/2013 07:09 PM, Glenn Linderman wrote: > On 4/25/2013 4:53 PM, Ethan Furman wrote: >> On 04/25/2013 04:26 PM, Glenn Linderman wrote: >>> My question is, once an enumeration is defined, is there a way, short of element-by-element assignment, to import the >>> individual enumeration instances into the current namespace, so that I can say?? "red" instead of "Color.red" ? I >>> understand the benefits of avoiding name collisions when there are lots of enumerations, and lots of opportunities for >>> name collections between, say, RGBColor and CYMKColor... but there are lots of uses for enumerations where the >>> subsidiary namespace is just aggravating noise. >> >> You mean something like: >> >> --> class Color(Enum): >> ... RED = 1 >> ... GREEN = 2 >> ... BLUE = 3 >> >> --> Color.register() # puts Color in sys.modules >> >> --> from Color import * # doesn't work in a function, though :( >> >> --> BLUE >> Color.BLUE > > Something like that, but that works in a function too :) Not in Py3 it doesn't: Python 3.2.3 (default, Oct 19 2012, 19:53:16) [GCC 4.7.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. --> def test(): ... from sys import * ... print('huh') ... File "", line 1 SyntaxError: import * only allowed at module level >> Yeah, that would be nice. ;) A bit dangerous, though -- what if another module does the same thing, but its Color >> is different? >> >> Better would be: >> >> --> Color.export(globals()) # put the enumerators in globals >> >> --> RED >> Color.RED > > Globals? locals should be possible too. At least in Cpython, updating locals() does not work in functions. > Or even something like: > > with Color: > BLUE > RED > > Although the extra indentation could also be annoying. > > One wouldn't want the module defining Color to automatically 'export' the colors: but rather a way to request an > 'export' them into a particular scope. That way the proliferation of names into scopes is chosen by the programmer. > > import module_containing_color > module_containing_color.Color.export_enumerations( globals ) > > or > > import module_containing_color > module_containing_color.Color.export_enumerations( locals ) > > Or maybe locals is implicit, and in the file scope of a module, locals are globals anyway, so doing > > module_containing_color.Color.export_enumerations() locals() can't be implicit, at least not without deep black magic of inspecting frames in the call stack -- which is hardly portable. -- ~Ethan~ From ncoghlan at gmail.com Fri Apr 26 04:49:40 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 26 Apr 2013 12:49:40 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> Message-ID: On Fri, Apr 26, 2013 at 12:38 PM, Guido van Rossum wrote: > If the above syntax won't work, that isinstance() outcome isn't really > important. :-( > > Can't we do some kind of callable check? There may be some weird > decorators that won't work, but they aren't likely to be useful in > this context. Yeah, it may not be as tricky as I feared: adding "not callable(attr_val)" to the conditions for deciding whether or not to convert a class attribute to an instance of the enum would likely suffice to address the method problem. You couldn't create an enum of callables, but that would be a seriously weird thing to do anyway.... Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From greg.ewing at canterbury.ac.nz Fri Apr 26 05:04:39 2013 From: greg.ewing at canterbury.ac.nz (Greg) Date: Fri, 26 Apr 2013 15:04:39 +1200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179D534.6050506@stoneleaf.us> References: <20130412110254.0c4c7a44@anarchist> <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> <20130421202816.3e22fc8f@anarchist> <20130425182310.1dc455ba@anarchist> <5179C378.60906@g.nevcal.com> <5179D25C.6040305@mrabarnett.plus.com> <5179D534.6050506@stoneleaf.us> Message-ID: <5179EEC7.9010601@canterbury.ac.nz> On 26/04/2013 1:15 p.m., Ethan Furman wrote: > Lots of counting systems wrap: seconds, minutes, hours, days of week, > days of month, days of year, millimeters, inches, etc., etc. But we don't disagree on which is the first minute of an hour. > We still apply ordering to them, and talk about 15 > being less than 42 . When we do that, we're using cardinal numbers (how many), not ordinal numbers (what order). -- Greg From v+python at g.nevcal.com Fri Apr 26 05:06:03 2013 From: v+python at g.nevcal.com (Glenn Linderman) Date: Thu, 25 Apr 2013 20:06:03 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179E582.5080102@stoneleaf.us> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179BBBE.4010309@g.nevcal.com> <5179C1FB.6010805@stoneleaf.us> <5179E1CA.3000301@g.nevcal.com> <5179E582.5080102@stoneleaf.us> Message-ID: <5179EF1B.8060001@g.nevcal.com> On 4/25/2013 7:25 PM, Ethan Furman wrote: > On 04/25/2013 07:09 PM, Glenn Linderman wrote: >> On 4/25/2013 4:53 PM, Ethan Furman wrote: >>> On 04/25/2013 04:26 PM, Glenn Linderman wrote: >>>> My question is, once an enumeration is defined, is there a way, >>>> short of element-by-element assignment, to import the >>>> individual enumeration instances into the current namespace, so >>>> that I can say?? "red" instead of "Color.red" ? I >>>> understand the benefits of avoiding name collisions when there are >>>> lots of enumerations, and lots of opportunities for >>>> name collections between, say, RGBColor and CYMKColor... but there >>>> are lots of uses for enumerations where the >>>> subsidiary namespace is just aggravating noise. >>> >>> You mean something like: >>> >>> --> class Color(Enum): >>> ... RED = 1 >>> ... GREEN = 2 >>> ... BLUE = 3 >>> >>> --> Color.register() # puts Color in sys.modules >>> >>> --> from Color import * # doesn't work in a function, though :( >>> >>> --> BLUE >>> Color.BLUE >> >> Something like that, but that works in a function too :) > > Not in Py3 it doesn't: Parse error. "Something like that, but something like that that works in a function too :)" is what I meant. I understand that the feature you demonstrated doesn't work in Py3... that's why we need "something like that" rather than "that" :) > > Python 3.2.3 (default, Oct 19 2012, 19:53:16) > [GCC 4.7.2] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > --> def test(): > ... from sys import * > ... print('huh') > ... > File "", line 1 > SyntaxError: import * only allowed at module level > > >>> Yeah, that would be nice. ;) A bit dangerous, though -- what if >>> another module does the same thing, but its Color >>> is different? >>> >>> Better would be: >>> >>> --> Color.export(globals()) # put the enumerators in globals >>> >>> --> RED >>> Color.RED >> >> Globals? locals should be possible too. > > At least in Cpython, updating locals() does not work in functions. > >> Or even something like: >> >> with Color: >> BLUE >> RED >> >> Although the extra indentation could also be annoying. >> >> One wouldn't want the module defining Color to automatically 'export' >> the colors: but rather a way to request an >> 'export' them into a particular scope. That way the proliferation of >> names into scopes is chosen by the programmer. >> >> import module_containing_color >> module_containing_color.Color.export_enumerations( globals ) >> >> or >> >> import module_containing_color >> module_containing_color.Color.export_enumerations( locals ) >> >> Or maybe locals is implicit, and in the file scope of a module, >> locals are globals anyway, so doing >> >> module_containing_color.Color.export_enumerations() > > locals() can't be implicit, at least not without deep black magic of > inspecting frames in the call stack -- which is hardly portable. So what I'm hearing is that enumerations need to be a language feature, rather than a module: Can't combine Enum and EnumItem Can't import into locals The compiler could do those things, though. -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Fri Apr 26 05:08:13 2013 From: greg.ewing at canterbury.ac.nz (Greg) Date: Fri, 26 Apr 2013 15:08:13 +1200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> Message-ID: <5179EF9D.1080309@canterbury.ac.nz> On 26/04/2013 2:38 p.m., Guido van Rossum wrote: > Can't we do some kind of callable check? There may be some weird > decorators that won't work, but they aren't likely to be useful in > this context. Another possible solution: class Color: red = 1 white = 2 blue = 3 orange = 4 class __methods__: def wave(self, n=1): for _ in range(n): print('Waving', self) and have the metaclass pull the functions out of the __methods__ sub-object. -- Greg From v+python at g.nevcal.com Fri Apr 26 05:12:56 2013 From: v+python at g.nevcal.com (Glenn Linderman) Date: Thu, 25 Apr 2013 20:12:56 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> Message-ID: <5179F0B8.8010601@g.nevcal.com> On 4/25/2013 7:49 PM, Nick Coghlan wrote: > On Fri, Apr 26, 2013 at 12:38 PM, Guido van Rossum wrote: >> If the above syntax won't work, that isinstance() outcome isn't really >> important. :-( >> >> Can't we do some kind of callable check? There may be some weird >> decorators that won't work, but they aren't likely to be useful in >> this context. > Yeah, it may not be as tricky as I feared: adding "not > callable(attr_val)" to the conditions for deciding whether or not to > convert a class attribute to an instance of the enum would likely > suffice to address the method problem. > > You couldn't create an enum of callables, but that would be a > seriously weird thing to do anyway.... But aren't all classes callable? But you are referring to the initial values of the items, but still, why should those be restricted from being any general object? Not being _functions_ is probably OK, but I'm not sure how strong the distinction is between functions and classes, regarding being callable... So objects are only callable if the class contains a def __call__, but still, that seems quite restrictive. -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Fri Apr 26 05:14:51 2013 From: greg.ewing at canterbury.ac.nz (Greg) Date: Fri, 26 Apr 2013 15:14:51 +1200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179D831.9060903@stoneleaf.us> References: <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> <20130421202816.3e22fc8f@anarchist> <20130425182310.1dc455ba@anarchist> <5179C378.60906@g.nevcal.com> <5179D25C.6040305@mrabarnett.plus.com> <5179D707.1020402@canterbury.ac.nz> <5179D831.9060903@stoneleaf.us> Message-ID: <5179F12B.7050902@canterbury.ac.nz> On 26/04/2013 1:28 p.m., Ethan Furman wrote: > Interesting idea, but why does Day(3) have to be disallowed to make it > work? Because it's ambiguous. Which day of the week is number 3? It depends on where you start. I should perhaps point out that the numbers assigned to the values initially are just to establish the relative ordering. They wouldn't be directly accessible once the values are created. To get an integer value corresponding to a Day value, you would have to do arithmetic: Day.wednesday - Day.sunday --> 3 -- Greg From greg.ewing at canterbury.ac.nz Fri Apr 26 05:19:32 2013 From: greg.ewing at canterbury.ac.nz (Greg) Date: Fri, 26 Apr 2013 15:19:32 +1200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179EF1B.8060001@g.nevcal.com> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179BBBE.4010309@g.nevcal.com> <5179C1FB.6010805@stoneleaf.us> <5179E1CA.3000301@g.nevcal.com> <5179E582.5080102@stoneleaf.us> <5179EF1B.8060001@g.nevcal.com> Message-ID: <5179F244.3010306@canterbury.ac.nz> On 26/04/2013 3:06 p.m., Glenn Linderman wrote: > So what I'm hearing is that enumerations need to be a language feature, > rather than a module: > > Can't combine Enum and EnumItem > Can't import into locals > > The compiler could do those things, though. Maybe we've found a use case for the recently advertised macro system? ("One, two, five..." runs for cover...) -- Greg From greg.ewing at canterbury.ac.nz Fri Apr 26 05:22:53 2013 From: greg.ewing at canterbury.ac.nz (Greg) Date: Fri, 26 Apr 2013 15:22:53 +1200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179F0B8.8010601@g.nevcal.com> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> Message-ID: <5179F30D.3050508@canterbury.ac.nz> On 26/04/2013 3:12 p.m., Glenn Linderman wrote: > On 4/25/2013 7:49 PM, Nick Coghlan wrote: >> You couldn't create an enum of callables, but that would be a >> seriously weird thing to do anyway.... > > But aren't all classes callable? An enum of classes would be seriously weird as well, I think. -- Greg From v+python at g.nevcal.com Fri Apr 26 05:39:24 2013 From: v+python at g.nevcal.com (Glenn Linderman) Date: Thu, 25 Apr 2013 20:39:24 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179F30D.3050508@canterbury.ac.nz> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> Message-ID: <5179F6EC.5040301@g.nevcal.com> On 4/25/2013 8:22 PM, Greg wrote: > On 26/04/2013 3:12 p.m., Glenn Linderman wrote: >> On 4/25/2013 7:49 PM, Nick Coghlan wrote: > >>> You couldn't create an enum of callables, but that would be a >>> seriously weird thing to do anyway.... >> >> But aren't all classes callable? > > An enum of classes would be seriously weird as well, I think. > Perhaps so, but an enumeration of objects whose class defines __call__ would not be so weird. Does your __methods__ cure this problem? It looked interesting to me, but I'm not familiar enough with metaclass to be sure of anything about it :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Fri Apr 26 06:19:33 2013 From: guido at python.org (Guido van Rossum) Date: Thu, 25 Apr 2013 21:19:33 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179F6EC.5040301@g.nevcal.com> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> Message-ID: On Thu, Apr 25, 2013 at 8:39 PM, Glenn Linderman wrote: > an enumeration of objects whose class defines __call__ would > not be so weird. Seriously? You'd complexificate the basic usage in order to cater for such an esoteric use case? The *only* use cases that matter at all for enum values are ints and strings, and even the latter could be considered a luxury when compared to other languages' enums. -- --Guido van Rossum (python.org/~guido) From steve at pearwood.info Fri Apr 26 07:01:06 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 26 Apr 2013 15:01:06 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179F30D.3050508@canterbury.ac.nz> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> Message-ID: <517A0A12.20309@pearwood.info> On 26/04/13 13:22, Greg wrote: > On 26/04/2013 3:12 p.m., Glenn Linderman wrote: >> On 4/25/2013 7:49 PM, Nick Coghlan wrote: > >>> You couldn't create an enum of callables, but that would be a >>> seriously weird thing to do anyway.... >> >> But aren't all classes callable? > > An enum of classes would be seriously weird as well, I think. I don't think iscallable will work, since that descriptors like staticmethod and classmethod aren't callable. Nor are properties. I think a solution may be an explicit decorator that tells the metaclass not to skip the object into an enum value: class Insect(enum.Enum): ant = 1 bee = 2 @enum.skip @classmethod def spam(cls, args): pass assert isinstance(Insect.spam, classmethod) One side effect of this is that now you can (ab)use the decorator to have regular data attributes. Whether that counts as a good thing or a bad thing, I leave up to others to decide... -- Steven From ncoghlan at gmail.com Fri Apr 26 07:30:06 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 26 Apr 2013 15:30:06 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517A0A12.20309@pearwood.info> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <517A0A12.20309@pearwood.info> Message-ID: On Fri, Apr 26, 2013 at 3:01 PM, Steven D'Aprano wrote: > On 26/04/13 13:22, Greg wrote: >> >> On 26/04/2013 3:12 p.m., Glenn Linderman wrote: >>> >>> On 4/25/2013 7:49 PM, Nick Coghlan wrote: >> >> >>>> You couldn't create an enum of callables, but that would be a >>>> seriously weird thing to do anyway.... >>> >>> >>> But aren't all classes callable? >> >> >> An enum of classes would be seriously weird as well, I think. > > I don't think iscallable will work, since that descriptors like > staticmethod and classmethod aren't callable. Nor are properties. My point was there *are* broad categories that we can reasonably say "you can't use these as values in an enumeration". Callables are one, descriptors are probably another. Alternatively, we can flip it around and require that each enum definition nominate an expected value type (defaulting to int) and only convert class attributes that are instances of that type to instances of the enum class. Either can be made to work, so it's really Guido's call as to which he would prefer. As Barry noted, the next step is actually for someone to create a proof of concept that shows the merge is possible in practice, not just theory. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ethan at stoneleaf.us Fri Apr 26 07:07:37 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 22:07:37 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> Message-ID: <517A0B99.9070400@stoneleaf.us> On 04/25/2013 07:13 PM, Nick Coghlan wrote: > On Fri, Apr 26, 2013 at 8:29 AM, Barry Warsaw wrote: >> On Apr 25, 2013, at 03:19 PM, Guido van Rossum wrote: >> >>> Clearly this is a trick question. :-) >> >> A bit, yes. :) >> >>> I was told when this was brought up previously (a week ago?) that it >>> would be simple to make it truly the same class. >> >> It didn't sound simple to me, but I haven't seen any actual code yet. > > I'm the one who said I didn't see any obvious barriers to the merger, > but I've realised there is one, and it's similar to one namedtuple > struggles with: how to handle method definitions. > > [snip] > > With a merged design, it becomes *really* hard to give the instances > custom behaviour, because the metaclass will somehow have to > differentiate between namespace entries that are intended to be > callables, and those which are intended to be instances of the enum. > This is not an easy problem to solve. I'm probably going to regret asking this, but what's difficult with the following? 8<----------------------------------------------------------------------------------------- class EnumDict(dict): """ automatically assigns the next _int for an enum """ def __init__(yo): super().__init__() yo._allow_duplicates = False yo._value = 1 yo._base = 1 yo._enums = [] yo._type = None # object means enum, anything else means all must be of that type def __setitem__(yo, key, new_value): """ main purpose is to support auto-numbering of members """ existing = yo.get(key) if type(existing) is attrs: # attrs is a helper class raise TypeError('Attempted to reuse key: %s' % key) if not key[:2] == key[-2:] == '__': old_value = None if isinstance(new_value, attrs): old_value = new_value if new_value.integer is None: new_value = yo._value else: new_value = new_value.integer if not isinstance(new_value, int): raise TypeError( "an enum integer must be an instance of type , not %s" % type(new_value) ) if not callable(new_value): # this if-else is probably not final if (isinstance(new_value, int) and old_value is not None or yo._type in (object, )): yo._check_duplicate_integer(new_value, key) yo._value = new_value yo._inc_integer() yo._enums.append(key) new_value = attrs(integer=new_value) elif yo._type is None: # random bunch of named constants yo._check_duplicate_integer(new_value, key) value = yo._value yo._inc_integer() yo._enums.append(key) new_value = attrs(value=new_value, integer=value) elif isinstance(new_value, yo._type): # single type of named constants if isinstance(yo._type, int): value = new_value else: value = yo._value yo._check_duplicate_integer(value, key) yo._inc_integer() yo._enums.append(key) new_value = attrs(value=new_value, integer=value) if old_value is not None: new_value, old_value.integer = old_value, new_value.integer dict.__setitem__(yo, key, new_value) def _inc_integer(yo): if yo._base == 1: yo._value += 1 else: if yo._value == 0: value = 1 else: value = floor(log(yo._value, 2)) value = 2 ** value value <<= 1 yo._value = value def _check_duplicate_integer(yo, new_value, key): for name, value in yo.items(): if not isinstance(value, attrs): continue if value.integer == new_value and name != key and not yo._allow_duplicates: raise TypeError('duplicate value for %s: %d' % (key, value)) 8<------------------------------------------------------------------------------------------- Basically, if the assigned value is not `attrs` or a literal of the same type as the enum, the metaclass ignores it. -- ~Ethan~ From ethan at stoneleaf.us Fri Apr 26 07:21:15 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 22:21:15 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <5179F12B.7050902@canterbury.ac.nz> References: <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> <20130421202816.3e22fc8f@anarchist> <20130425182310.1dc455ba@anarchist> <5179C378.60906@g.nevcal.com> <5179D25C.6040305@mrabarnett.plus.com> <5179D707.1020402@canterbury.ac.nz> <5179D831.9060903@stoneleaf.us> <5179F12B.7050902@canterbury.ac.nz> Message-ID: <517A0ECB.9080601@stoneleaf.us> On 04/25/2013 08:14 PM, Greg wrote: > On 26/04/2013 1:28 p.m., Ethan Furman wrote: >> Interesting idea, but why does Day(3) have to be disallowed to make it >> work? > > Because it's ambiguous. Which day of the week is number 3? It > depends on where you start. Ah. > I should perhaps point out that the numbers assigned to the > values initially are just to establish the relative ordering. > They wouldn't be directly accessible once the values are > created. To get an integer value corresponding to a Day value, > you would have to do arithmetic: > > Day.wednesday - Day.sunday --> 3 That would make it very hard to do data storage and retrieval. -- ~Ethan~ From ethan at stoneleaf.us Fri Apr 26 07:17:23 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 22:17:23 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517A0A12.20309@pearwood.info> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <517A0A12.20309@pearwood.info> Message-ID: <517A0DE3.9080101@stoneleaf.us> On 04/25/2013 10:01 PM, Steven D'Aprano wrote: > On 26/04/13 13:22, Greg wrote: >> On 26/04/2013 3:12 p.m., Glenn Linderman wrote: >>> On 4/25/2013 7:49 PM, Nick Coghlan wrote: >> >>>> You couldn't create an enum of callables, but that would be a >>>> seriously weird thing to do anyway.... >>> >>> But aren't all classes callable? >> >> An enum of classes would be seriously weird as well, I think. > > > I don't think iscallable will work, since that descriptors like > staticmethod and classmethod aren't callable. Nor are properties. > > > I think a solution may be an explicit decorator that tells the > metaclass not to skip the object into an enum value: Another option is to check if the item is a descriptor (class, static, property, or other); yet another option is to check if the item is the type of the enum class (int for IntEnum, str for StrEnum, etc.). The code I posted earlier checks for callable and type -- checking for descriptor also would not be much more effort. -- ~Ethan~ From duda.piotr at gmail.com Fri Apr 26 07:59:03 2013 From: duda.piotr at gmail.com (Piotr Duda) Date: Fri, 26 Apr 2013 07:59:03 +0200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> Message-ID: 2013/4/26 Nick Coghlan > > On Fri, Apr 26, 2013 at 8:29 AM, Barry Warsaw wrote: > > On Apr 25, 2013, at 03:19 PM, Guido van Rossum wrote: > > > >>Clearly this is a trick question. :-) > > > > A bit, yes. :) > > > >>I was told when this was brought up previously (a week ago?) that it > >>would be simple to make it truly the same class. > > > > It didn't sound simple to me, but I haven't seen any actual code yet. > > I'm the one who said I didn't see any obvious barriers to the merger, > but I've realised there is one, and it's similar to one namedtuple > struggles with: how to handle method definitions. There is at least one more problem, enum inheritance, given: class Colors(Enum): red = 1 green = 2 blue = 3 class MoreColors(Color): cyan = 4 magenta = 5 yellow = 6 what type is MoreColors.red? -- ???????? ?????? From ethan at stoneleaf.us Fri Apr 26 08:36:12 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 25 Apr 2013 23:36:12 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <517A0A12.20309@pearwood.info> Message-ID: <517A205C.6020603@stoneleaf.us> On 04/25/2013 10:30 PM, Nick Coghlan wrote: > > Alternatively, we can flip it around and require that each enum > definition nominate an expected value type (defaulting to int) and > only convert class attributes that are instances of that type to > instances of the enum class. I think this option makes the most sense. -- ~Ethan~ From solipsis at pitrou.net Fri Apr 26 09:01:07 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 26 Apr 2013 09:01:07 +0200 Subject: [Python-Dev] NoneType(None) raises exception References: <5179B7B5.2030901@stoneleaf.us> Message-ID: <20130426090107.14ebcc7d@fsol> On Thu, 25 Apr 2013 16:09:41 -0700 Ethan Furman wrote: > We just fixed NoneType() to return None instead of raising an exception. > > Another use-case for calling NoneType is working with ORMs: > > result = [] > for field in row: > type = get_type(field) # returns int, bool, str, NoneType, ... > result.append(type(field)) I don't understand what the use case is. If you already have a value of None, why do you call NoneType on it again? Perhaps you should write: if not isinstance(type, field): field = type(field) Rehatds Antoine. From regebro at gmail.com Fri Apr 26 09:19:12 2013 From: regebro at gmail.com (Lennart Regebro) Date: Fri, 26 Apr 2013 09:19:12 +0200 Subject: [Python-Dev] NoneType(None) raises exception In-Reply-To: <5179B7B5.2030901@stoneleaf.us> References: <5179B7B5.2030901@stoneleaf.us> Message-ID: On Fri, Apr 26, 2013 at 1:09 AM, Ethan Furman wrote: > We just fixed NoneType() to return None instead of raising an exception. > > Another use-case for calling NoneType is working with ORMs: > > result = [] > for field in row: > type = get_type(field) # returns int, bool, str, NoneType, ... > result.append(type(field)) Having it return NoneType there seems a strange thing to do, as have a None field makes no sense. Why would you have a column that can only contain the value None? What is returning NoneType at that point supposed to signify? If it's supposed to signify that the value is NULL, I think the above code is a very strange way of handling that. get_type(field) should reasonably return the type of the column, not the type of the value you got, as that would be just type(field) and doing type(field)(field) is a bit pointless. :-) //Lennart From greg.ewing at canterbury.ac.nz Fri Apr 26 09:34:55 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 26 Apr 2013 19:34:55 +1200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517A0A12.20309@pearwood.info> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <517A0A12.20309@pearwood.info> Message-ID: <517A2E1F.90001@canterbury.ac.nz> Steven D'Aprano wrote: > I don't think iscallable will work, since that descriptors like > staticmethod and classmethod aren't callable. Nor are properties. Hmmm, maybe we should look for a __get__ method as well? Enums of descriptors would seem to fall into the seriously-weird category as well. Or if, as Guido says, the only sensible things to use as enum values are ints and strings, just leave anything alone that isn't one of those. -- Greg From greg.ewing at canterbury.ac.nz Fri Apr 26 10:00:15 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 26 Apr 2013 20:00:15 +1200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> Message-ID: <517A340F.5090309@canterbury.ac.nz> Piotr Duda wrote: > There is at least one more problem, enum inheritance, given: > > class Colors(Enum): > red = 1 > green = 2 > blue = 3 > > class MoreColors(Color): > cyan = 4 > magenta = 5 > yellow = 6 > > what type is MoreColors.red? Given the implementation we're considering, it would probably be Colors. However, there's a worse problem with defining enum inheritance that way. The subtype relation for extensible enums works the opposite way to that of classes. To see this, imagine a function expecting something of type Colors. It knows what to do with red, green and blue, but not anything else. So you *can't* pass it something of type MoreColors, because not all values of type MoreColors are of type Colors. On the other hand, you *can* pass a value of type Colors to something expecting MoreColors, because every value of Colors is also in MoreColors. Moreover, suppose we have another type: class YetMoreColors(Colors): orange = 4 purple = 5 pink = 6 Now suppose a function expecting Colors gets an enum with the integer value 4. How should it be interpreted? Is it cyan or orange? What about if you write it to a database column and read it back? These considerations suggest to me that subclassing enums should be disallowed, or at least not officially supported. -- Greg From storchaka at gmail.com Fri Apr 26 11:16:34 2013 From: storchaka at gmail.com (Serhiy Storchaka) Date: Fri, 26 Apr 2013 12:16:34 +0300 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517A340F.5090309@canterbury.ac.nz> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <517A340F.5090309@canterbury.ac.nz> Message-ID: 26.04.13 11:00, Greg Ewing ???????(??): > However, there's a worse problem with defining enum > inheritance that way. The subtype relation for extensible > enums works the opposite way to that of classes. > > To see this, imagine a function expecting something > of type Colors. It knows what to do with red, green and > blue, but not anything else. So you *can't* pass it > something of type MoreColors, because not all values > of type MoreColors are of type Colors. This is why enums are not subclassable in other languages (i.e. Java). From steve at pearwood.info Fri Apr 26 14:32:53 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 26 Apr 2013 22:32:53 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517A340F.5090309@canterbury.ac.nz> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <517A340F.5090309@canterbury.ac.nz> Message-ID: <517A73F5.3080505@pearwood.info> On 26/04/13 18:00, Greg Ewing wrote: > However, there's a worse problem with defining enum > inheritance that way. The subtype relation for extensible > enums works the opposite way to that of classes. > > To see this, imagine a function expecting something > of type Colors. It knows what to do with red, green and > blue, but not anything else. So you *can't* pass it > something of type MoreColors, because not all values > of type MoreColors are of type Colors. > > On the other hand, you *can* pass a value of type Colors > to something expecting MoreColors, because every value of > Colors is also in MoreColors. > > Moreover, suppose we have another type: > > class YetMoreColors(Colors): > orange = 4 > purple = 5 > pink = 6 > > Now suppose a function expecting Colors gets an enum > with the integer value 4. How should it be interpreted? > Is it cyan or orange? What about if you write it to a > database column and read it back? There are many places where Python demands an actual int, not a subclass. See the recent thread "Semantics of __int__, index". There's no reason why a function that expects a Color *must* accept subclasses as well. If it can, great. If it can't, document it and move on. It's not Color's responsibility to know everything about every subclass. Invert your thinking: the subclasses are in charge, not Color. Color can't be expected to give a value to 4. Only the subclass that defines it can. This is only a problem if you believe that subclassing == taxonomy hierarchy. It isn't. http://pyvideo.org/video/879/the-art-of-subclassing -- Steven From python at mrabarnett.plus.com Fri Apr 26 17:41:18 2013 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 26 Apr 2013 16:41:18 +0100 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517A0ECB.9080601@stoneleaf.us> References: <20130412153302.402eca6a@anarchist> <20130412205216.7216B250BD2@webabinitio.net> <20130412183241.18d6749f@anarchist> <20130420141032.46acd456@anarchist> <20130420231049.E041E2500B3@webabinitio.net> <20130421174745.269913e5@anarchist> <20130421202816.3e22fc8f@anarchist> <20130425182310.1dc455ba@anarchist> <5179C378.60906@g.nevcal.com> <5179D25C.6040305@mrabarnett.plus.com> <5179D707.1020402@canterbury.ac.nz> <5179D831.9060903@stoneleaf.us> <5179F12B.7050902@canterbury.ac.nz> <517A0ECB.9080601@stoneleaf.us> Message-ID: <517AA01E.2010206@mrabarnett.plus.com> On 26/04/2013 06:21, Ethan Furman wrote: > On 04/25/2013 08:14 PM, Greg wrote: >> On 26/04/2013 1:28 p.m., Ethan Furman wrote: >>> Interesting idea, but why does Day(3) have to be disallowed to make it >>> work? >> >> Because it's ambiguous. Which day of the week is number 3? It >> depends on where you start. > > Ah. > > >> I should perhaps point out that the numbers assigned to the >> values initially are just to establish the relative ordering. >> They wouldn't be directly accessible once the values are >> created. To get an integer value corresponding to a Day value, >> you would have to do arithmetic: >> >> Day.wednesday - Day.sunday --> 3 > > That would make it very hard to do data storage and retrieval. > I think that if the definition of Day says wednesday = 3 then Day(3) should return Day.wednesday, but if it's a circular enum then <, etc, should be disallowed. From larry at hastings.org Fri Apr 26 17:50:16 2013 From: larry at hastings.org (Larry Hastings) Date: Fri, 26 Apr 2013 08:50:16 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517A2E1F.90001@canterbury.ac.nz> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <517A0A12.20309@pearwood.info> <517A2E1F.90001@canterbury.ac.nz> Message-ID: <517AA238.1060803@hastings.org> On 04/26/2013 12:34 AM, Greg Ewing wrote: > Or if, as Guido says, the only sensible things to use > as enum values are ints and strings, just leave anything > alone that isn't one of those. The standard Java documentation on enums: http://docs.oracle.com/javase/tutorial/java/javaOO/enum.html has an example enum of a "Planet", a small record type containing mass and radius--each of which are floats. I don't know whether or not it constitutes good programming, but I'd be crestfallen if Java enums were more expressive than Python enums ;-) FWIW I'm +0.5 on "the enum metaclass ignores callables and descriptors". This seems reasonably Pythonic, much more so than "ignore everything except ints and strings". And as long as we're special-casing it I think we should opt for flexibility. Certainly I see nothing wrong with enums of float, complex, Decimal, and Fraction, so I don't see a good place to draw the line with a whitelist. //arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From status at bugs.python.org Fri Apr 26 18:07:31 2013 From: status at bugs.python.org (Python tracker) Date: Fri, 26 Apr 2013 18:07:31 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20130426160731.8390456A38@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2013-04-19 - 2013-04-26) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3949 (+21) closed 25674 (+32) total 29623 (+53) Open issues with patches: 1760 Issues opened (43) ================== #16316: Support xz compression in mimetypes module http://bugs.python.org/issue16316 reopened by eric.araujo #17192: libffi-3.0.13 import http://bugs.python.org/issue17192 reopened by r.david.murray #17798: IDLE: can not edit new file names when using -e http://bugs.python.org/issue17798 opened by roger.serwy #17799: settrace docs are wrong about "c_call" events http://bugs.python.org/issue17799 opened by nedbat #17800: Add gc.needs_finalizing() to check if an object needs finalisi http://bugs.python.org/issue17800 opened by ncoghlan #17802: html.HTMLParser raises UnboundLocalError: http://bugs.python.org/issue17802 opened by bmispelon #17803: Calling Tkinter.Tk() with a baseName keyword argument throws U http://bugs.python.org/issue17803 opened by y-fujii #17804: streaming struct unpacking http://bugs.python.org/issue17804 opened by pitrou #17805: No such class: multiprocessing.pool.AsyncResult http://bugs.python.org/issue17805 opened by tkf #17806: Add keyword args support to str/bytes.expandtabs() http://bugs.python.org/issue17806 opened by ezio.melotti #17807: Generator cleanup without tp_del http://bugs.python.org/issue17807 opened by pitrou #17808: No code example for Event object in threading module http://bugs.python.org/issue17808 opened by amysyk #17809: FAIL: test_expanduser when $HOME ends with / http://bugs.python.org/issue17809 opened by koobs #17810: Implement PEP 3154 (pickle protocol 4) http://bugs.python.org/issue17810 opened by alexandre.vassalotti #17811: Improve os.readv() and os.writev() documentation and docstring http://bugs.python.org/issue17811 opened by Nikratio #17812: Quadratic complexity in b32encode http://bugs.python.org/issue17812 opened by serhiy.storchaka #17814: Popen.stdin/stdout/stderr documentation should mention object http://bugs.python.org/issue17814 opened by Nikratio #17816: Weak*Dictionary KeyErrors during callbacks http://bugs.python.org/issue17816 opened by Nils.Bruin #17818: aifc.Aifc_read/Aifc_write.getparams can return a namedtuple http://bugs.python.org/issue17818 opened by Claudiu.Popa #17819: removes need for CONFIG_SITE external configuration http://bugs.python.org/issue17819 opened by cavallo71 #17822: Save on Close windows (IDLE) http://bugs.python.org/issue17822 opened by Guilherme.Sim??es #17823: 2to3 fixers for missing codecs http://bugs.python.org/issue17823 opened by serhiy.storchaka #17824: pty.spawn handles errors improperly http://bugs.python.org/issue17824 opened by niemeyer #17825: Indentation.offset and SyntaxError.offset mismatch http://bugs.python.org/issue17825 opened by flox #17826: Setting a side_effect on mock from create_autospec doesn't wor http://bugs.python.org/issue17826 opened by michael.foord #17827: Document codecs.encode and codecs.decode http://bugs.python.org/issue17827 opened by ncoghlan #17828: More informative error handling when encoding and decoding http://bugs.python.org/issue17828 opened by ncoghlan #17829: csv.Sniffer.snif doesn't set up the dialect properly for a csv http://bugs.python.org/issue17829 opened by GhislainHivon #17831: urllib.URLopener.open breaks ActiveDirectory user http://bugs.python.org/issue17831 opened by Nataly.Glazyrina #17833: test_gdb broken PPC64 Linux http://bugs.python.org/issue17833 opened by David.Edelsohn #17834: Add Heap (and DynamicHeap) classes to heapq module http://bugs.python.org/issue17834 opened by allyourcode #17837: Support for building on ppc64p7 http://bugs.python.org/issue17837 opened by bkabrda #17838: Can't assign a different value for sys.stdin in IDLE http://bugs.python.org/issue17838 opened by Guilherme.Sim??es #17839: base64 module should use memoryview http://bugs.python.org/issue17839 opened by ncoghlan #17840: base64_codec uses assert for runtime validity checks http://bugs.python.org/issue17840 opened by ncoghlan #17841: Remove missing aliases from codecs documentation http://bugs.python.org/issue17841 opened by ncoghlan #17842: Add base64 module tests for a bytearray argument http://bugs.python.org/issue17842 opened by serhiy.storchaka #17843: Lib/test/testbz2_bigmem.bz2 trigger virus warnings http://bugs.python.org/issue17843 opened by christian.heimes #17844: Add link to alternatives for bytes-to-bytes codecs http://bugs.python.org/issue17844 opened by serhiy.storchaka #17845: Clarify successful build message http://bugs.python.org/issue17845 opened by brett.cannon #17846: Building Python on Windows - Supplementary info http://bugs.python.org/issue17846 opened by michael.kearney #17848: issue about compile with clang and build a shared lib http://bugs.python.org/issue17848 opened by matrixsystem #17849: Missing size argument in readline() method for httplib's class http://bugs.python.org/issue17849 opened by stamparm Most recent 15 issues with no replies (15) ========================================== #17848: issue about compile with clang and build a shared lib http://bugs.python.org/issue17848 #17846: Building Python on Windows - Supplementary info http://bugs.python.org/issue17846 #17844: Add link to alternatives for bytes-to-bytes codecs http://bugs.python.org/issue17844 #17841: Remove missing aliases from codecs documentation http://bugs.python.org/issue17841 #17840: base64_codec uses assert for runtime validity checks http://bugs.python.org/issue17840 #17829: csv.Sniffer.snif doesn't set up the dialect properly for a csv http://bugs.python.org/issue17829 #17824: pty.spawn handles errors improperly http://bugs.python.org/issue17824 #17819: removes need for CONFIG_SITE external configuration http://bugs.python.org/issue17819 #17818: aifc.Aifc_read/Aifc_write.getparams can return a namedtuple http://bugs.python.org/issue17818 #17809: FAIL: test_expanduser when $HOME ends with / http://bugs.python.org/issue17809 #17806: Add keyword args support to str/bytes.expandtabs() http://bugs.python.org/issue17806 #17805: No such class: multiprocessing.pool.AsyncResult http://bugs.python.org/issue17805 #17799: settrace docs are wrong about "c_call" events http://bugs.python.org/issue17799 #17798: IDLE: can not edit new file names when using -e http://bugs.python.org/issue17798 #17791: PC/pyconfig.h defines PREFIX macro http://bugs.python.org/issue17791 Most recent 15 issues waiting for review (15) ============================================= #17844: Add link to alternatives for bytes-to-bytes codecs http://bugs.python.org/issue17844 #17842: Add base64 module tests for a bytearray argument http://bugs.python.org/issue17842 #17839: base64 module should use memoryview http://bugs.python.org/issue17839 #17838: Can't assign a different value for sys.stdin in IDLE http://bugs.python.org/issue17838 #17837: Support for building on ppc64p7 http://bugs.python.org/issue17837 #17834: Add Heap (and DynamicHeap) classes to heapq module http://bugs.python.org/issue17834 #17833: test_gdb broken PPC64 Linux http://bugs.python.org/issue17833 #17826: Setting a side_effect on mock from create_autospec doesn't wor http://bugs.python.org/issue17826 #17822: Save on Close windows (IDLE) http://bugs.python.org/issue17822 #17819: removes need for CONFIG_SITE external configuration http://bugs.python.org/issue17819 #17818: aifc.Aifc_read/Aifc_write.getparams can return a namedtuple http://bugs.python.org/issue17818 #17812: Quadratic complexity in b32encode http://bugs.python.org/issue17812 #17810: Implement PEP 3154 (pickle protocol 4) http://bugs.python.org/issue17810 #17808: No code example for Event object in threading module http://bugs.python.org/issue17808 #17807: Generator cleanup without tp_del http://bugs.python.org/issue17807 Top 10 most discussed issues (10) ================================= #7475: codecs missing: base64 bz2 hex zlib hex_codec ... http://bugs.python.org/issue7475 24 msgs #17468: Generator memory leak http://bugs.python.org/issue17468 17 msgs #17845: Clarify successful build message http://bugs.python.org/issue17845 16 msgs #17804: streaming struct unpacking http://bugs.python.org/issue17804 15 msgs #15392: Create a unittest framework for IDLE http://bugs.python.org/issue15392 9 msgs #17646: traceback.py has a lot of code duplication http://bugs.python.org/issue17646 8 msgs #17822: Save on Close windows (IDLE) http://bugs.python.org/issue17822 8 msgs #17833: test_gdb broken PPC64 Linux http://bugs.python.org/issue17833 8 msgs #17742: Add _PyBytesWriter API http://bugs.python.org/issue17742 7 msgs #17800: Add gc.needs_finalizing() to check if an object needs finalisi http://bugs.python.org/issue17800 7 msgs Issues closed (31) ================== #9607: Test file 'test_keyword.py' submission for use with keyword.py http://bugs.python.org/issue9607 closed by r.david.murray #11714: threading.Semaphore does not use try...finally http://bugs.python.org/issue11714 closed by serhiy.storchaka #12825: Missing and incorrect link to a command line option. http://bugs.python.org/issue12825 closed by eric.araujo #15575: Tutorial is unclear on multiple imports of a module. http://bugs.python.org/issue15575 closed by r.david.murray #15642: Integrate pickle protocol version 4 GSoC work by Stefan Mihail http://bugs.python.org/issue15642 closed by alexandre.vassalotti #16347: configure.ac patch http://bugs.python.org/issue16347 closed by cavallo71 #16624: subprocess.check_output should allow specifying stdin as a str http://bugs.python.org/issue16624 closed by serhiy.storchaka #16694: Add pure Python operator module http://bugs.python.org/issue16694 closed by pitrou #16942: urllib still doesn't support persistent connections http://bugs.python.org/issue16942 closed by orsenthil #17065: Fix sporadic buildbot failures for test_winreg http://bugs.python.org/issue17065 closed by r.david.murray #17353: Plistlib outputs empty data tags when deeply nested http://bugs.python.org/issue17353 closed by ronaldoussoren #17409: resource.setrlimit doesn't respect -1 http://bugs.python.org/issue17409 closed by r.david.murray #17413: format_exception() breaks on exception tuples from trace funct http://bugs.python.org/issue17413 closed by r.david.murray #17720: pickle.py's load_appends should call append() on objects other http://bugs.python.org/issue17720 closed by alexandre.vassalotti #17729: advocacy howto improvements http://bugs.python.org/issue17729 closed by ezio.melotti #17736: Misleading method comment in _elementtree.c : get_attrib_from_ http://bugs.python.org/issue17736 closed by eli.bendersky #17785: Use faster URL shortener for perf.py http://bugs.python.org/issue17785 closed by alexandre.vassalotti #17795: backwards-incompatible change in SysLogHandler with unix domai http://bugs.python.org/issue17795 closed by python-dev #17796: No mimetype guessed for tar.xz url http://bugs.python.org/issue17796 closed by eric.araujo #17801: Tools/scripts/gprof2html.py: `#! /usr/bin/env python32.3` http://bugs.python.org/issue17801 closed by ned.deily #17813: lzma and bz2 decompress methods lack max_size parameter http://bugs.python.org/issue17813 closed by ezio.melotti #17815: itertools.combinations example is overly complicated http://bugs.python.org/issue17815 closed by Theodoros.Ikonomou #17817: ??Bug Python 2.7.4 and version 3.3.1? http://bugs.python.org/issue17817 closed by roger.serwy #17820: Nothing about editors in "Key Resources" http://bugs.python.org/issue17820 closed by ned.deily #17821: Different division results with / and // operators with large http://bugs.python.org/issue17821 closed by sbt #17830: keyword.py main does not preserve line endings when rewriting http://bugs.python.org/issue17830 closed by r.david.murray #17832: pythonrun.c:_print_total_refs missing prototype http://bugs.python.org/issue17832 closed by pitrou #17835: test_io broken on PPC64 Linux http://bugs.python.org/issue17835 closed by pitrou #17836: multiprocessing exceptions with useful traceback http://bugs.python.org/issue17836 closed by sbt #17847: Glossary lacks permalinks http://bugs.python.org/issue17847 closed by pitrou #17850: unicode_escape encoding fails for '\\Upsilon' http://bugs.python.org/issue17850 closed by ezio.melotti From storchaka at gmail.com Fri Apr 26 18:27:18 2013 From: storchaka at gmail.com (Serhiy Storchaka) Date: Fri, 26 Apr 2013 19:27:18 +0300 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517AA238.1060803@hastings.org> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <517A0A12.20309@pearwood.info> <517A2E1F.90001@canterbury.ac.nz> <517AA238.1060803@hastings.org> Message-ID: 26.04.13 18:50, Larry Hastings ???????(??): > On 04/26/2013 12:34 AM, Greg Ewing wrote: >> Or if, as Guido says, the only sensible things to use >> as enum values are ints and strings, just leave anything >> alone that isn't one of those. > > The standard Java documentation on enums: > > http://docs.oracle.com/javase/tutorial/java/javaOO/enum.html > > has an example enum of a "Planet", a small record type containing mass > and radius--each of which are floats. I don't know whether or not it > constitutes good programming, but I'd be crestfallen if Java enums were > more expressive than Python enums ;-) This example requires more than features discussed here. It requires an enum constructor. class Planet(Enum): MERCURY = Planet(3.303e+23, 2.4397e6) VENUS = Planet(4.869e+24, 6.0518e6) EARTH = Planet(5.976e+24, 6.37814e6) MARS = Planet(6.421e+23, 3.3972e6) JUPITER = Planet(1.9e+27, 7.1492e7) SATURN = Planet(5.688e+26, 6.0268e7) URANUS = Planet(8.686e+25, 2.5559e7) NEPTUNE = Planet(1.024e+26, 2.4746e7) def __init__(self, mass, radius): self.mass = mass # in kilograms self.radius = radius # in meters @property def surfaceGravity(self): # universal gravitational constant (m3 kg-1 s-2) G = 6.67300E-11 return G * self.mass / (self.radius * self.radius) def surfaceWeight(self, otherMass): return otherMass * self.surfaceGravity This can't work because the name Planet in the class definition is not defined. From ethan at stoneleaf.us Fri Apr 26 18:25:56 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 26 Apr 2013 09:25:56 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517AA238.1060803@hastings.org> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <517A0A12.20309@pearwood.info> <517A2E1F.90001@canterbury.ac.nz> <517AA238.1060803@hastings.org> Message-ID: <517AAA94.5040603@stoneleaf.us> On 04/26/2013 08:50 AM, Larry Hastings wrote: > FWIW I'm +0.5 on "the enum metaclass ignores callables and > descriptors". This seems reasonably Pythonic, much more so than "ignore > everything except ints and strings". And as long as we're > special-casing it I think we should opt for flexibility. Certainly I see > nothing wrong with enums of float, complex, Decimal, and Fraction, so I > don't see a good place to draw the line with a whitelist. A whitelist, if we go that route, should be on a per Enum basis -- so something like: class Planets(str, Enum): PLUTO = 'it is too a planet!' MERCURY = 'definitely a planet' SATURN = 'darn big planet' solar_mass = 82738273 # or some really big number light_year = 1827499 # another really big number (of feet!) def horrorscope(self, message): return ('%s will be in retrograde... ' 'don't communicate today.' % self) and only PLUTO, MERCURY, and SATURN would be converted to enums. -- ~Ethan~ From larry at hastings.org Fri Apr 26 19:30:40 2013 From: larry at hastings.org (Larry Hastings) Date: Fri, 26 Apr 2013 10:30:40 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <517A0A12.20309@pearwood.info> <517A2E1F.90001@canterbury.ac.nz> <517AA238.1060803@hastings.org> Message-ID: <517AB9C0.6070608@hastings.org> On 04/26/2013 09:27 AM, Serhiy Storchaka wrote: > 26.04.13 18:50, Larry Hastings ???????(??): >> The standard Java documentation on enums: >> >> http://docs.oracle.com/javase/tutorial/java/javaOO/enum.html > > This example requires more than features discussed here. It requires > an enum constructor. > > class Planet(Enum): > MERCURY = Planet(3.303e+23, 2.4397e6) > [...] > This can't work because the name Planet in the class definition is not > defined. It can't work because you inserted the word "Planet" there. If you omit the word "Planet", this would work fine with something like the metaclass instantiate-all-data-members behavior in flufl.enum 4. Here is the hack, demonstrated in Python 3: class Metaclass(type): def __new__(cls, name, bases, namespace): result = type.__new__(cls, name, bases, dict(namespace)) for name, value in namespace.items(): if not (callable(value) or name.startswith("__")): value = result(name, value) setattr(result, name, value) return result class Planet(metaclass=Metaclass): MERCURY = (3.303e+23, 2.4397e6) def __init__(self, name, value): self.mass, self.radius = value def surfaceGravity(self): return 6.67300E-11 * self.mass / (self.radius ** 2) def surfaceWeight(self, otherMass): return otherMass * self.surfaceGravity() print("If you weigh 175 pounds, on Mercury you'd weigh", Planet.MERCURY.surfaceWeight(175)) //arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Fri Apr 26 19:33:36 2013 From: v+python at g.nevcal.com (Glenn Linderman) Date: Fri, 26 Apr 2013 10:33:36 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> Message-ID: <517ABA70.8020909@g.nevcal.com> On 4/25/2013 9:19 PM, Guido van Rossum wrote: > On Thu, Apr 25, 2013 at 8:39 PM, Glenn Linderman wrote: >> an enumeration of objects whose class defines __call__ would >> not be so weird. > Seriously? You'd complexificate the basic usage in order to cater for > such an esoteric use case? The *only* use cases that matter at all for > enum values are ints and strings, and even the latter could be > considered a luxury when compared to other languages' enums. No, I'd look for a solution/implementation that doesn't divide objects into "plain" and "esoteric" cases. Py3 now treats everything as objects. So an enumeration should be able to deal with any object as a value. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Fri Apr 26 19:36:08 2013 From: guido at python.org (Guido van Rossum) Date: Fri, 26 Apr 2013 10:36:08 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517ABA70.8020909@g.nevcal.com> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> Message-ID: On Fri, Apr 26, 2013 at 10:33 AM, Glenn Linderman wrote: > On 4/25/2013 9:19 PM, Guido van Rossum wrote: > > On Thu, Apr 25, 2013 at 8:39 PM, Glenn Linderman > wrote: > > an enumeration of objects whose class defines __call__ would > not be so weird. > > Seriously? You'd complexificate the basic usage in order to cater for > such an esoteric use case? The *only* use cases that matter at all for > enum values are ints and strings, and even the latter could be > considered a luxury when compared to other languages' enums. > > > No, I'd look for a solution/implementation that doesn't divide objects into > "plain" and "esoteric" cases. Py3 now treats everything as objects. So an > enumeration should be able to deal with any object as a value. I think you've lost track of the Zen of Python. -- --Guido van Rossum (python.org/~guido) From ethan at stoneleaf.us Fri Apr 26 18:54:07 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 26 Apr 2013 09:54:07 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <517A0A12.20309@pearwood.info> <517A2E1F.90001@canterbury.ac.nz> <517AA238.1060803@hastings.org> Message-ID: <517AB12F.3040308@stoneleaf.us> On 04/26/2013 09:27 AM, Serhiy Storchaka wrote: > 26.04.13 18:50, Larry Hastings ???????(??): >> On 04/26/2013 12:34 AM, Greg Ewing wrote: >>> Or if, as Guido says, the only sensible things to use >>> as enum values are ints and strings, just leave anything >>> alone that isn't one of those. >> >> The standard Java documentation on enums: >> >> http://docs.oracle.com/javase/tutorial/java/javaOO/enum.html >> >> has an example enum of a "Planet", a small record type containing mass >> and radius--each of which are floats. I don't know whether or not it >> constitutes good programming, but I'd be crestfallen if Java enums were >> more expressive than Python enums ;-) > > This example requires more than features discussed here. It requires an > enum constructor. > > This can't work because the name Planet in the class definition is not > defined. The metaclass can define it easily: - have generic helper class (I call mine `attrs`) - __prepare__ creates an instance of the custom dict - __prepare__ then inserts the helper class into the custom dict with the same name as the (to be created) custom Enum type - return the custom dict to Python class is processed using custom dict - __new__ gets the custom dict back from Python - __new__ replaces all instances of the helper class with actual Enum instances (which it creates on the spot) - any other housekeeping necessary - __new__ returns the new Enum type, complete with all Enum instances Here's an example run: 8<----------planets.py------------------------------------------------ from aenum import Enum class Planet(Enum): MERCURY = Planet(3.303e+23, 2.4397e6) VENUS = Planet(4.869e+24, 6.0518e6) EARTH = Planet(5.976e+24, 6.37814e6) MARS = Planet(6.421e+23, 3.3972e6) JUPITER = Planet(1.9e+27, 7.1492e7) SATURN = Planet(5.688e+26, 6.0268e7) URANUS = Planet(8.686e+25, 2.5559e7) NEPTUNE = Planet(1.024e+26, 2.4746e7) def __init__(self, mass, radius): self.mass = mass # in kilograms self.radius = radius # in meters @property def surfaceGravity(self): # universal gravitational constant (m3 kg-1 s-2) G = 6.67300E-11 return G * self.mass / (self.radius * self.radius) def surfaceWeight(self, otherMass): return otherMass * self.surfaceGravity print(int(Planet.VENUS)) print(repr(Planet.VENUS)) print(Planet.VENUS.surfaceGravity) 8<----------planets.py------------------------------------------------ 8<----------actual run------------------------------------------------ 2 Planet('VENUS', 4.869e+24, 6051800.0, integer=2) 8.871391908774457 8<----------actual run------------------------------------------------ -- ~Ethan~ From eliben at gmail.com Fri Apr 26 20:17:57 2013 From: eliben at gmail.com (Eli Bendersky) Date: Fri, 26 Apr 2013 11:17:57 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> Message-ID: On Fri, Apr 26, 2013 at 10:36 AM, Guido van Rossum wrote: > On Fri, Apr 26, 2013 at 10:33 AM, Glenn Linderman > wrote: > > On 4/25/2013 9:19 PM, Guido van Rossum wrote: > > > > On Thu, Apr 25, 2013 at 8:39 PM, Glenn Linderman > > wrote: > > > > an enumeration of objects whose class defines __call__ would > > not be so weird. > > > > Seriously? You'd complexificate the basic usage in order to cater for > > such an esoteric use case? The *only* use cases that matter at all for > > enum values are ints and strings, and even the latter could be > > considered a luxury when compared to other languages' enums. > > > > > > No, I'd look for a solution/implementation that doesn't divide objects > into > > "plain" and "esoteric" cases. Py3 now treats everything as objects. So an > > enumeration should be able to deal with any object as a value. > > I think you've lost track of the Zen of Python. > I feel that this thread has lost track of it long ago. Some time back in the Enum discussions (some 350 messages ago or so), there was a proposal to have this: class Color(Enum): RED, BLUE, GREEN By doing some crazy-cool shenanigans. Although the syntax is great, it was rejected on the basis of being too magic. The recent proposals of folding Enum and EnumValue into one, having class members be instances of the class they're members of while supporting a bunch of other Enum requirements also go off the rails in terms of complexity and magic. In contrast, personally I feel the current proposal in PEP 435 has an appeal from the POV of simplicity. It really is a very nice separation of concerns between enum values and Enum as a container of such values. It even allows significant customization (IntEnum, etc) which is pretty simple to grok. It would be a shame to lose these for the sake of making Python a bit more like Java. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From storchaka at gmail.com Fri Apr 26 22:59:01 2013 From: storchaka at gmail.com (Serhiy Storchaka) Date: Fri, 26 Apr 2013 23:59:01 +0300 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517A340F.5090309@canterbury.ac.nz> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <517A340F.5090309@canterbury.ac.nz> Message-ID: 26.04.13 11:00, Greg Ewing ???????(??): > However, there's a worse problem with defining enum > inheritance that way. The subtype relation for extensible > enums works the opposite way to that of classes. > > To see this, imagine a function expecting something > of type Colors. It knows what to do with red, green and > blue, but not anything else. So you *can't* pass it > something of type MoreColors, because not all values > of type MoreColors are of type Colors. > > On the other hand, you *can* pass a value of type Colors > to something expecting MoreColors, because every value of > Colors is also in MoreColors. I propose do not use an inheritance for extending enums, but use an import. class Colors(Enum): red = 1 green = 2 blue = 3 class MoreColors(Enum): from Colors import * cyan = 4 magenta = 5 yellow = 6 An inheritance we can use to limit a type of values. class Colors(int, Enum): # only int values red = 1 green = 2 blue = 3 Colors.viridity = green From storchaka at gmail.com Fri Apr 26 23:11:11 2013 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sat, 27 Apr 2013 00:11:11 +0300 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> Message-ID: 26.04.13 05:13, Nick Coghlan ???????(??): > With a merged design, it becomes *really* hard to give the instances > custom behaviour, because the metaclass will somehow have to > differentiate between namespace entries that are intended to be > callables, and those which are intended to be instances of the enum. > This is not an easy problem to solve. What if use mixins? Shouldn't it work without magic? class ColorMethods: def wave(self, n=1): for _ in range(n): print('Waving', self) class Color(ColorMethods, Enum): red = 1 white = 2 blue = 3 orange = 4 From storchaka at gmail.com Fri Apr 26 23:36:13 2013 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sat, 27 Apr 2013 00:36:13 +0300 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <20130420151808.62376061@anarchist> References: <20130420151808.62376061@anarchist> Message-ID: Thank you for your answers, Barry. Eli already answered me most of my questions. 20.04.13 22:18, Barry Warsaw ???????(??): > On Apr 13, 2013, at 11:31 AM, Serhiy Storchaka wrote: >>> The str and repr of the enumeration class also provides useful information:: >>> >>> >>> print(Colors) >>> >>> >>> print(repr(Colors)) >>> >> >> Does the enumeration's repr() use str() or repr() for the enumeration values? > > No, enumeration values have different reprs and strs. Yes, values can have different reprs and strs (but ints haven't). What of them uses repr of an enumeration item? I.e. what is str(Noises): '' or ''? class Noises(Enum) dog = 'bark' flufl.enum uses str(), but is it intentional? If yes, than it should be specified in the PEP. >>> But if the value *is* important, enumerations can have arbitrary values. >> >> Should enumeration values be hashable? >> >> At least they should be comparable ("Iteration is defined as the sorted order >> of the item values"). > > Given my previous responses, these questions should be already answered. Eli and you have missed my first question. Should enumeration values be hashable? If yes (flufl.enum requires hashability), then this should be specified in the PEP. If no, then how you implements __getitem__? You can use binary search (but values can be noncomparable) or linear search which is not efficient. From guido at python.org Fri Apr 26 23:41:23 2013 From: guido at python.org (Guido van Rossum) Date: Fri, 26 Apr 2013 14:41:23 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> Message-ID: On Fri, Apr 26, 2013 at 11:17 AM, Eli Bendersky wrote: > On Fri, Apr 26, 2013 at 10:36 AM, Guido van Rossum wrote: >> > On 4/25/2013 9:19 PM, Guido van Rossum wrote: >> I think you've lost track of the Zen of Python. > > I feel that this thread has lost track of it long ago. Perhaps. The thread certainly has lost focus -- there are separate subthreads about the ordering of items when iterating over the enum class and about the unification of the Enum and EnumValue classes, and perhaps others. > Some time back in the > Enum discussions (some 350 messages ago or so), there was a proposal to have > this: > > class Color(Enum): > RED, BLUE, GREEN > > By doing some crazy-cool shenanigans. Although the syntax is great, it was > rejected on the basis of being too magic. FWIW, I didn't think the syntax was great. "Great" syntax would have been something like enum Color: RED, BLUE, GREEN but introducing a new keyword isn't justifiable. > The recent proposals of folding Enum and EnumValue into one, having class > members be instances of the class they're members of while supporting a > bunch of other Enum requirements also go off the rails in terms of > complexity and magic. Yet the goal of the proposal is conceptual simplification: one class instead of two. Having class members that are also class instances is a non-issue. The complexity of the proposal is an implementation issue: we'd like to be able to define methods for the enum values, and the simplest way (for the user) to define methods for the enum values would be to allow def statements, possibly decorated, in the class. But now the implementation has to draw a somewhat murky line between which definitions in the class should be interpreted as enum value definitions, and which should be interpreted as method definitions. If we had access to the syntax used for the definition, this would be simple: assignments define items, def statements define methods. But at run time we only see the final object resulting from the definition, which may not even be callable in the case of certain decorators. I am still optimistic that we can come up with a rule that works well enough in practice (and the Zen rule to which I was referring was, of course, "practicality beats purity"). > In contrast, personally I feel the current proposal in PEP 435 has an appeal > from the POV of simplicity. It really is a very nice separation of concerns > between enum values and Enum as a container of such values. It even allows > significant customization (IntEnum, etc) which is pretty simple to grok. It > would be a shame to lose these for the sake of making Python a bit more like > Java. But it's not so much the "like Java" that matters to me. It's the realization that for the user who wants to define an enum type with some extra functionality, having a single class and putting the methods and the items in the same class is the simplest way to do it. The Java reference is just to point out that we're not exactly breaking new ground here. -- --Guido van Rossum (python.org/~guido) From solipsis at pitrou.net Sat Apr 27 00:27:33 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 27 Apr 2013 00:27:33 +0200 Subject: [Python-Dev] Generator finalization and reference cycles Message-ID: <20130427002733.4191af9d@fsol> Hello, I have proposed a new generator finalization scheme in http://bugs.python.org/issue17807: it removes the tp_del from generator objects, instead electing to cleanup the generator (equivalent of calling close()) in the frame's tp_clean function. This way, generators with a try / finally block can be finalized even when they are part of reference cycles. This has stemmed from http://bugs.python.org/issue17468, which mentions a memory leak in Django due to uncollectable generators. Feedback welcome. Regards Antoine. From eliben at gmail.com Sat Apr 27 01:01:08 2013 From: eliben at gmail.com (Eli Bendersky) Date: Fri, 26 Apr 2013 16:01:08 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> Message-ID: On Fri, Apr 26, 2013 at 2:41 PM, Guido van Rossum wrote: > On Fri, Apr 26, 2013 at 11:17 AM, Eli Bendersky wrote: > > On Fri, Apr 26, 2013 at 10:36 AM, Guido van Rossum > wrote: > >> > On 4/25/2013 9:19 PM, Guido van Rossum wrote: > >> I think you've lost track of the Zen of Python. > > > > I feel that this thread has lost track of it long ago. > > Perhaps. The thread certainly has lost focus -- there are separate > subthreads about the ordering of items when iterating over the enum > class and about the unification of the Enum and EnumValue classes, and > perhaps others. > I agree, and thanks for the thoughtful reply to my half-rant ;-) More comments below. > > > Some time back in the > > Enum discussions (some 350 messages ago or so), there was a proposal to > have > > this: > > > > class Color(Enum): > > RED, BLUE, GREEN > > > > By doing some crazy-cool shenanigans. Although the syntax is great, it > was > > rejected on the basis of being too magic. > > FWIW, I didn't think the syntax was great. "Great" syntax would have > been something like > > enum Color: > RED, BLUE, GREEN > > but introducing a new keyword isn't justifiable. > > > The recent proposals of folding Enum and EnumValue into one, having class > > members be instances of the class they're members of while supporting a > > bunch of other Enum requirements also go off the rails in terms of > > complexity and magic. > > Yet the goal of the proposal is conceptual simplification: one class > instead of two. > > But users are not exposed to two classes. And for us, implementors, keeping things separate is IMHO simplification. There's a conceptual difference between a value of an enumeration and a collection of such values. This difference helps, I think, make IntEnum possible relatively easily - we can actually have an enumerated value that is-a integer and can be used instead of integers in legacy code. This requirement will heap additional complexity on the alternative where the enum and enum value are unified into the same class. > Having class members that are also class instances is a non-issue. > > The complexity of the proposal is an implementation issue: we'd like > to be able to define methods for the enum values, and the simplest way > (for the user) to define methods for the enum values would be to allow > def statements, possibly decorated, in the class. But now the > implementation has to draw a somewhat murky line between which > definitions in the class should be interpreted as enum value > definitions, and which should be interpreted as method definitions. If > we had access to the syntax used for the definition, this would be > simple: assignments define items, def statements define methods. But > at run time we only see the final object resulting from the > definition, which may not even be callable in the case of certain > decorators. I am still optimistic that we can come up with a rule that > works well enough in practice (and the Zen rule to which I was > referring was, of course, "practicality beats purity"). > > > In contrast, personally I feel the current proposal in PEP 435 has an > appeal > > from the POV of simplicity. It really is a very nice separation of > concerns > > between enum values and Enum as a container of such values. It even > allows > > significant customization (IntEnum, etc) which is pretty simple to grok. > It > > would be a shame to lose these for the sake of making Python a bit more > like > > Java. > > But it's not so much the "like Java" that matters to me. It's the > realization that for the user who wants to define an enum type with > some extra functionality, having a single class and putting the > methods and the items in the same class is the simplest way to do it. > The Java reference is just to point out that we're not exactly > breaking new ground here. > My reference to Java was w.r.t. the static-typing-minded requirement that isinstance(Color.red, Color) == True. I don't think that 99% of use-cases will care about the type of Color.red, and those that do for obscure reasons can accept an obscure class (EnumValue-something) as long as they have something distinct to dispatch upon. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Sat Apr 27 03:22:53 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 27 Apr 2013 13:22:53 +1200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> Message-ID: <517B286D.2020601@canterbury.ac.nz> Guido van Rossum wrote: > If we had access to the syntax used for the definition, this would be > simple: assignments define items, def statements define methods. But > at run time we only see the final object resulting from the > definition, Another way we could tell the difference is if the def statement used a different protocol for making bindings than assignment. Suppose a def statement used in a class body called __defitem__, if it exists, instead of __setitem__. Then the metaclass would be able to do different things for defs and assignments. -- Greg From greg.ewing at canterbury.ac.nz Sat Apr 27 03:37:23 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 27 Apr 2013 13:37:23 +1200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> Message-ID: <517B2BD3.8060000@canterbury.ac.nz> Eli Bendersky wrote: > There's a conceptual > difference between a value of an enumeration and a collection of such > values. Not if you think of an enum as a type and a type as defining a set of values. From that point of view, the enum itself is already a collection of values, and introducing another object is creating an artificial distinction. -- Greg From v+python at g.nevcal.com Sat Apr 27 04:29:44 2013 From: v+python at g.nevcal.com (Glenn Linderman) Date: Fri, 26 Apr 2013 19:29:44 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517B286D.2020601@canterbury.ac.nz> References: <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517B286D.2020601@canterbury.ac.nz> Message-ID: <517B3818.4010405@g.nevcal.com> On 4/26/2013 6:22 PM, Greg Ewing wrote: > Guido van Rossum wrote: >> If we had access to the syntax used for the definition, this would be >> simple: assignments define items, def statements define methods. But >> at run time we only see the final object resulting from the >> definition, > > Another way we could tell the difference is if the def > statement used a different protocol for making bindings > than assignment. > > Suppose a def statement used in a class body called > __defitem__, if it exists, instead of __setitem__. Then > the metaclass would be able to do different things for > defs and assignments. Well, some assignments could be for non-enumeration items, once you start allowing EnumItem in the list. Some method of grouping enumeration items, or grouping non-enumeration items would solve the problem. class Color( Enum ): Enum.__enumerationItems__( red=1, green=2, blue=3, ) # other methods and assignments -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Sat Apr 27 04:09:15 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 26 Apr 2013 19:09:15 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517B2BD3.8060000@canterbury.ac.nz> References: <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517B2BD3.8060000@canterbury.ac.nz> Message-ID: <517B334B.90309@stoneleaf.us> On 04/26/2013 06:37 PM, Greg Ewing wrote: > Eli Bendersky wrote: >> There's a conceptual difference between a value of an enumeration and a collection of such values. > > Not if you think of an enum as a type and a type as > defining a set of values. From that point of view, the > enum itself is already a collection of values, and > introducing another object is creating an artificial > distinction. I agree (FWIW ;). It seems to me that the closest existing Python data type is bool. bool is a type and has exactly two members, which are static/singleton/only created once. Enum is a metatype which we use to create a type with a fixed number of members which are static/singleton/only created once. The salient differences: with Enum we name the type and the members with Enum the members are also attributes of the type As a concrete example, consider: class WeekDay(Enum): SUNDAY = 1 MONDAY = 2 TUESDAY = 3 WEDNESDAY = 4 THURSDAY = 5 FRIDAY = 6 SATURDAY = 7 If we follow bool's example, then like True and False are of type(bool), TUESDAY should be of type(WeekDay). -- ~Ethan~ From ethan at stoneleaf.us Sat Apr 27 04:59:23 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 26 Apr 2013 19:59:23 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> Message-ID: <517B3F0B.5030709@stoneleaf.us> On 04/26/2013 11:17 AM, Eli Bendersky wrote: > I feel that this thread has lost track of it long ago. Some time back in the Enum discussions (some 350 messages ago or > so), there was a proposal to have this: > > class Color(Enum): > RED, BLUE, GREEN > > By doing some crazy-cool shenanigans. Although the syntax is great, it was rejected on the basis of being too magic. Although explicit reasons were not mentioned (and perhaps not even consciously recognized -- *sigh* someday I hope to be that good), there are very good ones beyond "being too magic" -- way too easy to introduce bugs: name lookup success looks exactly like name lookup failure, but the consequences were drastically different: class Color(Enum): BLACK RED GREEN BLUE class MoreColor(Enum): BLACK CYAN MAGENTA YELLOW BLACK in MoreColor is a bug that cannot easily be detected as it is a successful name lookup; the consequence is that CYAN, MAGENTA, and YELLOW are now off by one. Not being Dutch, I had to smack into that one before I gave up on the idea. -- ~Ethan~ From Nikolaus at rath.org Sat Apr 27 04:39:46 2013 From: Nikolaus at rath.org (Nikolaus Rath) Date: Fri, 26 Apr 2013 19:39:46 -0700 Subject: [Python-Dev] Destructors and Closing of File Objects References: <87a9p41gr6.fsf@vostro.rath.org> <877gk35aqx.fsf@vostro.rath.org> Message-ID: <87y5c4r91p.fsf@vostro.rath.org> Guido van Rossum writes: > On Monday, April 15, 2013, Nikolaus Rath wrote: >> Brian Curtin > writes: >> > On Fri, Apr 12, 2013 at 12:04 AM, Nikolaus Rath > >> wrote: >> >> [ Note: I already asked this on >> >> http://stackoverflow.com/questions/15917502 but didn't get any >> >> satisfactory answers] >> > >> > Sorry, but that's not a reason to repost your question to this list. >> > If you have to ask somewhere else, it would be python-list, aka, >> > comp.lang.python. >> >> I figured it belonged here because the question is really about the >> internal implementation of file objects, which to me didn't seem like a >> question about using Python. But I'll give it a few days and send >> another mail there if still haven't found the answer by then. > > You got your answer 16 hours ago on S.O. I guess you are referring to http://stackoverflow.com/a/15968516/293003 from Armin Ringo? ,---- | On Windows, NamedTemporaryFile uses a Windows-specific extension | (os.O_TEMPORARY) to ensure that the file is deleted when it is closed. | This probably also works if the process is killed in any way. However | there is no obvious equivalent on POSIX, most likely because on POSIX | you can simply delete files that are still in use; it only deletes the | name, and the file's content is only removed after it is closed (in any | way). But indeed assuming that we want the file name to persist until | the file is closed, like with NamedTemporaryFile, then we need "magic". | | We cannot use the same magic as for flushing buffered files. What occurs | there is that the C library handles it (in Python 2): the files are FILE | objects in C, and the C guarantees that they are flushed on normal | program exit (but not if the process is killed). In the case of Python | 3, there is custom C code to achieve the same effect. But it's specific | to this use case, not anything directly reusable. [...] `---- It's indeed very informative, but it doesn't fully address the question because of the _pyio module which certainly can't use any custom C code. Does that mean that when I'm using x = _pyio.BufferedWriter(), I could loose data in the write buffer when the interpreter exits without me calling x.close(), but when using x = io.BufferedWriter(), the buffer is guaranteed to get flushed? (Note: this isn't a complaint, just curiosity about the Python internals). Best, -Nikolaus -- ?Time flies like an arrow, fruit flies like a Banana.? PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C From Nikolaus at rath.org Sat Apr 27 04:10:53 2013 From: Nikolaus at rath.org (Nikolaus Rath) Date: Fri, 26 Apr 2013 19:10:53 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <517A0A12.20309@pearwood.info> Message-ID: <871u9wsoya.fsf@vostro.rath.org> Steven D'Aprano writes: > On 26/04/13 13:22, Greg wrote: >> On 26/04/2013 3:12 p.m., Glenn Linderman wrote: >>> On 4/25/2013 7:49 PM, Nick Coghlan wrote: >> >>>> You couldn't create an enum of callables, but that would be a >>>> seriously weird thing to do anyway.... >>> >>> But aren't all classes callable? >> >> An enum of classes would be seriously weird as well, I think. > > > I don't think iscallable will work, since that descriptors like > staticmethod and classmethod aren't callable. Nor are properties. > > > I think a solution may be an explicit decorator that tells the > metaclass not to skip the object into an enum value: > > > class Insect(enum.Enum): > ant = 1 > bee = 2 > > @enum.skip > @classmethod > def spam(cls, args): > pass In this case, wouldn't it be nicer to "decorate" those attributes that are meant to be enum values? I think having to use the class keyword to define something that really doesn't behave like an ordinary class is pretty confusing, and the following syntax would be a lot easier to understand at first sight: class Insect(enum.Enum): ant = enum.EnumValue(1) bee = enum.EnumValue(2) @classmethod def spam(cls, args): pass def ham(self, args): pass Obviously, EnumValue() would automatically assign a suitable number. Best, -Nikolaus -- ?Time flies like an arrow, fruit flies like a Banana.? PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C From guido at python.org Sat Apr 27 05:20:04 2013 From: guido at python.org (Guido van Rossum) Date: Fri, 26 Apr 2013 20:20:04 -0700 Subject: [Python-Dev] Destructors and Closing of File Objects In-Reply-To: <87y5c4r91p.fsf@vostro.rath.org> References: <87a9p41gr6.fsf@vostro.rath.org> <877gk35aqx.fsf@vostro.rath.org> <87y5c4r91p.fsf@vostro.rath.org> Message-ID: There are no guarantees in life. On the other hand: Don't worry, be happy. On Fri, Apr 26, 2013 at 7:39 PM, Nikolaus Rath wrote: > Guido van Rossum writes: >> On Monday, April 15, 2013, Nikolaus Rath wrote: >>> Brian Curtin > writes: >>> > On Fri, Apr 12, 2013 at 12:04 AM, Nikolaus Rath > >>> wrote: >>> >> [ Note: I already asked this on >>> >> http://stackoverflow.com/questions/15917502 but didn't get any >>> >> satisfactory answers] >>> > >>> > Sorry, but that's not a reason to repost your question to this list. >>> > If you have to ask somewhere else, it would be python-list, aka, >>> > comp.lang.python. >>> >>> I figured it belonged here because the question is really about the >>> internal implementation of file objects, which to me didn't seem like a >>> question about using Python. But I'll give it a few days and send >>> another mail there if still haven't found the answer by then. >> >> You got your answer 16 hours ago on S.O. > > I guess you are referring to http://stackoverflow.com/a/15968516/293003 > from Armin Ringo? > > ,---- > | On Windows, NamedTemporaryFile uses a Windows-specific extension > | (os.O_TEMPORARY) to ensure that the file is deleted when it is closed. > | This probably also works if the process is killed in any way. However > | there is no obvious equivalent on POSIX, most likely because on POSIX > | you can simply delete files that are still in use; it only deletes the > | name, and the file's content is only removed after it is closed (in any > | way). But indeed assuming that we want the file name to persist until > | the file is closed, like with NamedTemporaryFile, then we need "magic". > | > | We cannot use the same magic as for flushing buffered files. What occurs > | there is that the C library handles it (in Python 2): the files are FILE > | objects in C, and the C guarantees that they are flushed on normal > | program exit (but not if the process is killed). In the case of Python > | 3, there is custom C code to achieve the same effect. But it's specific > | to this use case, not anything directly reusable. > [...] > `---- > > It's indeed very informative, but it doesn't fully address the question > because of the _pyio module which certainly can't use any custom C code. > Does that mean that when I'm using x = _pyio.BufferedWriter(), I could loose > data in the write buffer when the interpreter exits without me calling > x.close(), but when using x = io.BufferedWriter(), the buffer is > guaranteed to get flushed? > > (Note: this isn't a complaint, just curiosity about the Python > internals). > > > Best, > > -Nikolaus > > -- > ?Time flies like an arrow, fruit flies like a Banana.? > > PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) From ethan at stoneleaf.us Sat Apr 27 04:51:45 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 26 Apr 2013 19:51:45 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517B3818.4010405@g.nevcal.com> References: <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517B286D.2020601@canterbury.ac.nz> <517B3818.4010405@g.nevcal.com> Message-ID: <517B3D41.6060109@stoneleaf.us> On 04/26/2013 07:29 PM, Glenn Linderman wrote: > On 4/26/2013 6:22 PM, Greg Ewing wrote: >> Guido van Rossum wrote: >>> If we had access to the syntax used for the definition, this would be >>> simple: assignments define items, def statements define methods. But >>> at run time we only see the final object resulting from the >>> definition, >> >> Another way we could tell the difference is if the def >> statement used a different protocol for making bindings >> than assignment. >> >> Suppose a def statement used in a class body called >> __defitem__, if it exists, instead of __setitem__. Then >> the metaclass would be able to do different things for >> defs and assignments. > > Well, some assignments could be for non-enumeration items, once you start allowing EnumItem in the list. Some method > of grouping enumeration items, or grouping non-enumeration items would solve the problem. > > class Color( Enum ): > Enum.__enumerationItems__( > red=1, > green=2, > blue=3, > ) > # other methods and assignments Or, if we go with the metaclass magic of re-using the class/type name (and who doesn't love metaclass magic??): class Color(Enum): red = Color(1) green = Color(2) blue = Color 3) look_ma_not_an_enum = 4 -- ~Ethan~ From victor.stinner at gmail.com Sat Apr 27 10:03:13 2013 From: victor.stinner at gmail.com (Victor Stinner) Date: Sat, 27 Apr 2013 10:03:13 +0200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <871u9wsoya.fsf@vostro.rath.org> References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <517A0A12.20309@pearwood.info> <871u9wsoya.fsf@vostro.rath.org> Message-ID: Why not defining new methods/changing the behaviour using a different metaclass? Victor Le 27 avr. 2013 05:12, "Nikolaus Rath" a ?crit : > Steven D'Aprano writes: > > On 26/04/13 13:22, Greg wrote: > >> On 26/04/2013 3:12 p.m., Glenn Linderman wrote: > >>> On 4/25/2013 7:49 PM, Nick Coghlan wrote: > >> > >>>> You couldn't create an enum of callables, but that would be a > >>>> seriously weird thing to do anyway.... > >>> > >>> But aren't all classes callable? > >> > >> An enum of classes would be seriously weird as well, I think. > > > > > > I don't think iscallable will work, since that descriptors like > > staticmethod and classmethod aren't callable. Nor are properties. > > > > > > I think a solution may be an explicit decorator that tells the > > metaclass not to skip the object into an enum value: > > > > > > class Insect(enum.Enum): > > ant = 1 > > bee = 2 > > > > @enum.skip > > @classmethod > > def spam(cls, args): > > pass > > > In this case, wouldn't it be nicer to "decorate" those attributes that > are meant to be enum values? I think having to use the class keyword to > define something that really doesn't behave like an ordinary class is > pretty confusing, and the following syntax would be a lot easier to > understand at first sight: > > class Insect(enum.Enum): > ant = enum.EnumValue(1) > bee = enum.EnumValue(2) > > @classmethod > def spam(cls, args): > pass > > def ham(self, args): > pass > > > Obviously, EnumValue() would automatically assign a suitable number. > > > Best, > > -Nikolaus > > -- > ?Time flies like an arrow, fruit flies like a Banana.? > > PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sat Apr 27 15:17:36 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 27 Apr 2013 23:17:36 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> Message-ID: On Sat, Apr 27, 2013 at 7:41 AM, Guido van Rossum wrote: > On Fri, Apr 26, 2013 at 11:17 AM, Eli Bendersky wrote: >> In contrast, personally I feel the current proposal in PEP 435 has an appeal >> from the POV of simplicity. It really is a very nice separation of concerns >> between enum values and Enum as a container of such values. It even allows >> significant customization (IntEnum, etc) which is pretty simple to grok. It >> would be a shame to lose these for the sake of making Python a bit more like >> Java. > > But it's not so much the "like Java" that matters to me. It's the > realization that for the user who wants to define an enum type with > some extra functionality, having a single class and putting the > methods and the items in the same class is the simplest way to do it. > The Java reference is just to point out that we're not exactly > breaking new ground here. A common idiom in some other use cases (like ORMs) is to allow an inner class to customise behaviour beyond what the basic class syntax allows. It seems like that may be a good fit here, as a couple of simple changes should greatly simplify the process of tweaking the behaviour of the enum items, without adding more complexity to the implementation: 1. Change the name of "__value_factory__" to something more concise like "__enumitem__". 2. Make EnumItem and IntEnumItem public classes (Barry has already noted that the current naming the associated classes in flufl.enum and PEP 435 isn't quite right, since the intended terminology is enum for the class, enum item for the labelled values, and value for the raw unlabelled objects, but the class names are currently EnumValue and IntEnumValue). Then, if you want to add custom behaviour to your enum, you would be able to do use a nested class relatively cleanly: class MyEnum(Enum): itemA = 1 itemB = 2 class __enumitem__(EnumItem): def __init__(self, enum, value, name): if not name.startswith("item"): raise ValueError("Item name {} doesn't start with 'item'".format(name)) super().__init__(enum, value, name) @property def letter(self): return self.name[4:] class MyExtendedEnum(MyEnum): # The custom __enumitem__ is inherited along with the original attributes itemC = 3 itemD = 4 Furthermore, rather than tweaking isinstance checks, it may make more sense to support containment testing for enums, such that you can write things like: assert MyEnum.itemA in MyEnum assert 1 not in MyEnum Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From zuo at chopin.edu.pl Sat Apr 27 17:58:01 2013 From: zuo at chopin.edu.pl (=?utf-8?B?SmFuIEthbGlzemV3c2tp?=) Date: Sat, 27 Apr 2013 17:58:01 +0200 Subject: [Python-Dev] =?utf-8?q?Odp=3A__PEP_435_--_Adding_an_Enum_type_to_?= =?utf-8?q?the_Python_standard_library?= Message-ID: <20130427155804.14D2418089B@filifionka.chopin.edu.pl> Guido van Rossum wrote: > we'd like to be able to define methods for the enum values, and the simplest way (for the user) to define methods for the enum values would be to allow def statements, possibly decorated, in the class. But now the implementation has to draw a somewhat murky line between which definitions in the class should be interpreted as enum value definitions, and which should be interpreted as method definitions. If we had access to the syntax used for the definition, this would be simple: assignments define items, def statements define methods. But at run time we only see the final object resulting from the definition, which may not even be callable in the case of certain decorators. I am still optimistic that we can come up with a rule that works well enough in practice (and the Zen rule to which I was referring was, of course, "practicality beats purity"). Maybe only names that do *not* start with underscore should be treated as enum value names; and those starting with underscore could be used e.g. to define methods etc.? Python has a long tradition of treating names differently depending of that feature. *j -- Sent from phone... -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Sat Apr 27 18:01:54 2013 From: guido at python.org (Guido van Rossum) Date: Sat, 27 Apr 2013 09:01:54 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423154458.3bba1e57@pitrou.net> <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> Message-ID: I've always found the nested class solution confusing when I had to use it in Django. It is a technical solution to a technical problem, but the resulting code is not very readable unless you have seen it a lot; it is a new, exceptional pattern. And as for using 'in' instead of 'isinstance' to check whether a value belongs to a given enum class, that, too, is a deviation from normal practice, which will require special cases in frameworks provide e.g. type checking. (As I mentioned before, any framework using argument annotations to indicate the expected type(s) for arguments would have to special-case the checks for enums.) Please let's try harder to come up with a way to separate enum value definitions from method definitions that works as expected in most common cases. I am willing to concede that it's hard to support things like class variables; maybe __private variables can be used for these; or the code can just use globals instead of class variables to hold per-class state. And __init__/__new__ probably shouldn't be overridden. But instance methods, class methods, static methods, properties, and other decorated methods should all work, as should special methods like __add__ or __getitem__. Hm... A lightbulb just went off. Objects representing both undecorated and decorated methods have a __get__() method, courtesy of the descriptor protocol. Maybe checking for that will work? It feels Pythonic to me: it uses a corner of the language that most people don't even know exists (*), but it produces the desired effect in almost all cases that matter, the pattern is simple to describe and easy to use without thinking about it, and for experts the rules are completely clear, uncomplicated, and free of heuristics, so it is possible to reason about corner cases. (*) Proof: even I didn't think of it until just now. :-) On Sat, Apr 27, 2013 at 6:17 AM, Nick Coghlan wrote: > On Sat, Apr 27, 2013 at 7:41 AM, Guido van Rossum wrote: >> On Fri, Apr 26, 2013 at 11:17 AM, Eli Bendersky wrote: >>> In contrast, personally I feel the current proposal in PEP 435 has an appeal >>> from the POV of simplicity. It really is a very nice separation of concerns >>> between enum values and Enum as a container of such values. It even allows >>> significant customization (IntEnum, etc) which is pretty simple to grok. It >>> would be a shame to lose these for the sake of making Python a bit more like >>> Java. >> >> But it's not so much the "like Java" that matters to me. It's the >> realization that for the user who wants to define an enum type with >> some extra functionality, having a single class and putting the >> methods and the items in the same class is the simplest way to do it. >> The Java reference is just to point out that we're not exactly >> breaking new ground here. > > A common idiom in some other use cases (like ORMs) is to allow an > inner class to customise behaviour beyond what the basic class syntax > allows. It seems like that may be a good fit here, as a couple of > simple changes should greatly simplify the process of tweaking the > behaviour of the enum items, without adding more complexity to the > implementation: > > 1. Change the name of "__value_factory__" to something more concise > like "__enumitem__". > 2. Make EnumItem and IntEnumItem public classes > > (Barry has already noted that the current naming the associated > classes in flufl.enum and PEP 435 isn't quite right, since the > intended terminology is enum for the class, enum item for the labelled > values, and value for the raw unlabelled objects, but the class names > are currently EnumValue and IntEnumValue). > > Then, if you want to add custom behaviour to your enum, you would be > able to do use a nested class relatively cleanly: > > class MyEnum(Enum): > itemA = 1 > itemB = 2 > > class __enumitem__(EnumItem): > def __init__(self, enum, value, name): > if not name.startswith("item"): > raise ValueError("Item name {} doesn't start with > 'item'".format(name)) > super().__init__(enum, value, name) > @property > def letter(self): > return self.name[4:] > > class MyExtendedEnum(MyEnum): > # The custom __enumitem__ is inherited along with the original > attributes > itemC = 3 > itemD = 4 > > Furthermore, rather than tweaking isinstance checks, it may make more > sense to support containment testing for enums, such that you can > write things like: > > assert MyEnum.itemA in MyEnum > assert 1 not in MyEnum > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -- --Guido van Rossum (python.org/~guido) From larry at hastings.org Sat Apr 27 18:11:14 2013 From: larry at hastings.org (Larry Hastings) Date: Sat, 27 Apr 2013 09:11:14 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> Message-ID: <517BF8A2.50008@hastings.org> On 04/26/2013 02:41 PM, Guido van Rossum wrote: > I am still optimistic that we can come up with a rule that > works well enough in practice (and the Zen rule to which I was > referring was, of course, "practicality beats purity"). The rule I liked best is "ignore callables, descriptors, and anything with leading & trailing double underscores". Personally I'd modify that to simply "anything with two leading underscores" so you can have private variables. It seems Pythonic to me in that classes already treat all those things special. And if you want enums of any of those things you can instantiate & insert them by hand after the class definition. Does that fail in an important way? //arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sat Apr 27 18:41:32 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 28 Apr 2013 02:41:32 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517BF8A2.50008@hastings.org> References: <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517BF8A2.50008@hastings.org> Message-ID: On Sun, Apr 28, 2013 at 2:11 AM, Larry Hastings wrote: > On 04/26/2013 02:41 PM, Guido van Rossum wrote: > > I am still optimistic that we can come up with a rule that > works well enough in practice (and the Zen rule to which I was > referring was, of course, "practicality beats purity"). > > > The rule I liked best is "ignore callables, descriptors, and anything with > leading & trailing double underscores". Personally I'd modify that to > simply "anything with two leading underscores" so you can have private > variables. It seems Pythonic to me in that classes already treat all those > things special. And if you want enums of any of those things you can > instantiate & insert them by hand after the class definition. > > Does that fail in an important way? Nope, those cover it. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ethan at stoneleaf.us Sat Apr 27 19:04:10 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 27 Apr 2013 10:04:10 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> Message-ID: <517C050A.7070305@stoneleaf.us> On 04/27/2013 09:01 AM, Guido van Rossum wrote: > > Hm... A lightbulb just went off. Objects representing both undecorated > and decorated methods have a __get__() method, courtesy of the > descriptor protocol. Maybe checking for that will work? It feels > Pythonic to me: it uses a corner of the language that most people > don't even know exists (*), but it produces the desired effect in > almost all cases that matter, the pattern is simple to describe and > easy to use without thinking about it, and for experts the rules are > completely clear, uncomplicated, and free of heuristics, so it is > possible to reason about corner cases. While this will certainly work, it means you can't have class variables that happen to be the same type as the enum -- so no int in an IntEnum, for example. The solution I like best is the helper class (called, originally enough, enum), and only those items get transformed: class Planet(IntEnum): MERCURY = enum(1) VENUS = enum(2) EARTH = enum(3) rough_pi = 3 # not transformed -- ~Ethan~ From rz1991 at foxmail.com Sat Apr 27 19:23:38 2013 From: rz1991 at foxmail.com (=?gb18030?B?yO7vow==?=) Date: Sun, 28 Apr 2013 01:23:38 +0800 Subject: [Python-Dev] Questions about Codon Alignment Proposal Message-ID: Hi Eric and Peter, I'm preparing the proposal for the codon alignment project. Two things I may want to hear your advice. 1) In the biopython wiki page, you mentioned "model selection" in the Approach & Goals. I'm not sure if there are any advantages to use codon alignment for model selection. Could you give me some references? Another thing is that model selection involves estimation of tree topology as well as branch lengthes and parameters across many substitution models. Will it be too computationally intensive for a python implementation? 2) You also mentioned the "validation (testing for frame shift)". Is there a test for frame shift? Or I can simply detect it by comparing amino acid sequences and nucleotide sequences. Best, Zheng Ruan -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Sat Apr 27 19:30:50 2013 From: guido at python.org (Guido van Rossum) Date: Sat, 27 Apr 2013 10:30:50 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130423142418.652DE250BCA@webabinitio.net> <20130425145012.6f58b91b@anarchist> <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517BF8A2.50008@hastings.org> Message-ID: On Sat, Apr 27, 2013 at 9:41 AM, Nick Coghlan wrote: > On Sun, Apr 28, 2013 at 2:11 AM, Larry Hastings wrote: >> On 04/26/2013 02:41 PM, Guido van Rossum wrote: >> >> I am still optimistic that we can come up with a rule that >> works well enough in practice (and the Zen rule to which I was >> referring was, of course, "practicality beats purity"). >> >> >> The rule I liked best is "ignore callables, descriptors, and anything with >> leading & trailing double underscores". Personally I'd modify that to >> simply "anything with two leading underscores" so you can have private >> variables. It seems Pythonic to me in that classes already treat all those >> things special. And if you want enums of any of those things you can >> instantiate & insert them by hand after the class definition. >> >> Does that fail in an important way? > > Nope, those cover it. Great, sounds like a plan. The exception for callables may not even be needed -- the callables we care about (and some non-callables, like properties) are all descriptors. Or do we care about nested class definitions? (The reason I'm not keen on a general exemption for callables is that some 3rd party objects you wouldn't necessarily expect to be callable actually are.) I agree on a general exemption for __dunder__ names. The problem with exempting __private is that by the time the metaclass sees them, they've already been mangled to _classname__private. And I could just about imagine a use case for having a private value in an enum. -- --Guido van Rossum (python.org/~guido) From guido at python.org Sat Apr 27 19:36:40 2013 From: guido at python.org (Guido van Rossum) Date: Sat, 27 Apr 2013 10:36:40 -0700 Subject: [Python-Dev] Questions about Codon Alignment Proposal In-Reply-To: References: Message-ID: Sounds like this was accidentally CC'ed to python-dev. On Sat, Apr 27, 2013 at 10:23 AM, ?? wrote: > Hi Eric and Peter, > > I'm preparing the proposal for the codon alignment project. Two things I may > want to hear your advice. > > 1) In the biopython wiki page, you mentioned "model selection" in the > Approach & Goals. I'm not sure if there are any advantages to use codon > alignment for model selection. Could you give me some references? Another > thing is that model selection involves estimation of tree topology as well > as branch lengthes and parameters across many substitution models. Will it > be too computationally intensive for a python implementation? > > 2) You also mentioned the "validation (testing for frame shift)". Is there a > test for frame shift? Or I can simply detect it by comparing amino acid > sequences and nucleotide sequences. > > Best, > Zheng Ruan > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) From guido at python.org Sat Apr 27 19:35:43 2013 From: guido at python.org (Guido van Rossum) Date: Sat, 27 Apr 2013 10:35:43 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517C050A.7070305@stoneleaf.us> References: <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517C050A.7070305@stoneleaf.us> Message-ID: On Sat, Apr 27, 2013 at 10:04 AM, Ethan Furman wrote: > While this will certainly work, it means you can't have class variables that > happen to be the same type as the enum -- so no int in an IntEnum, for > example. > > The solution I like best is the helper class (called, originally enough, > enum), and only those items get transformed: > > class Planet(IntEnum): > MERCURY = enum(1) > VENUS = enum(2) > EARTH = enum(3) > rough_pi = 3 # not transformed If this means that the most plain vanilla enum definition still has to use the enum(i) notation, I'm against it. Why do you want rough_pi to be a class variable anyway? The whole point of an enum is that it's *not* a kitchen sink class. An enum for the planets will need other support code that doesn't live in the enum class -- it shouldn't be considered a general scope for miscellanea. (TBH, I think that using classes to scope variables is mostly misguided anyway -- the standard mechanism for scoping is the module.) -- --Guido van Rossum (python.org/~guido) From ethan at stoneleaf.us Sat Apr 27 20:27:39 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 27 Apr 2013 11:27:39 -0700 Subject: [Python-Dev] class name spaces inside an outer function Message-ID: <517C189B.1000504@stoneleaf.us> I filed bug http://bugs.python.org/issue17853 last night. If somebody could point me in the right direction (mainly which files to look in), I'd be happy to attempt a patch. -- ~Ethan~ From steve at pearwood.info Sat Apr 27 20:45:32 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 28 Apr 2013 04:45:32 +1000 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517B3D41.6060109@stoneleaf.us> References: <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517B286D.2020601@canterbury.ac.nz> <517B3818.4010405@g.nevcal.com> <517B3D41.6060109@stoneleaf.us> Message-ID: <517C1CCC.9090203@pearwood.info> On 27/04/13 12:51, Ethan Furman wrote: > On 04/26/2013 07:29 PM, Glenn Linderman wrote: [...] >> class Color( Enum ): >> Enum.__enumerationItems__( >> red=1, >> green=2, >> blue=3, >> ) >> # other methods and assignments > > Or, if we go with the metaclass magic of re-using the class/type name (and who doesn't love metaclass magic??): > > class Color(Enum): > red = Color(1) > green = Color(2) > blue = Color 3) > look_ma_not_an_enum = 4 and from a later email: > The solution I like best is the helper class (called, originally enough, enum), and only those items get transformed: > > class Planet(IntEnum): > MERCURY = enum(1) > VENUS = enum(2) > EARTH = enum(3) > rough_pi = 3 # not transformed I'm sorry, but all these suggestions are getting the API completely backwards by making the common case harder than the rare case. We're creating an Enum, right? So the *common case* is to populate it with enum values. 99% of the time, enumerated values will be all that we want from an enum. So that's the case that needs to be simple, not the rare case where you have a non enum value in an enum class. The common case (enum values in an Enum class) should be easy, and the rare cases (ordinary class-like attributes) possible. Explicit is better than implicit: if you want something to *not* be processed by the Enum metaclass, you have to explicitly mark it as special. Dunders excepted, because they *are* special enough to break the rules. Since dunders are reserved for Python, I'm happy with a rule that says that dunders cannot be set as enum values (at least not via the metaclass). Otherwise, everything inside an Enum class is treated as an enum value unless explicitly flagged as not. Here's a dirty hack that demonstrates what I'm talking about. class EnumValue: # Mock EnumValue class. def __new__(cls, name, obj): print("making enum {!s} from {!r}".format(name, obj)) return obj class MetaEnum(type): def __new__(meta, name, bases, namespace): cls = super().__new__(meta, name, bases, {}) for name, value in namespace.items(): if meta.isspecial(value): value = value.original elif not meta.isdunder(name): value = EnumValue(name, value) setattr(cls, name, value) return cls @staticmethod def isdunder(name): return name.startswith('__') and name.endswith('__') @staticmethod def isspecial(obj): return isinstance(obj, skip) class skip: def __init__(self, obj): self.original = obj class Example(metaclass=MetaEnum): red = 1 blue = 2 green = lambda: 'good lord, even functions can be enums!' def __init__(self, count=3): self.count = count food = skip('spam') @skip def spam(self): return self.count * self.food -- Steven From ethan at stoneleaf.us Sat Apr 27 20:24:40 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 27 Apr 2013 11:24:40 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517C050A.7070305@stoneleaf.us> Message-ID: <517C17E8.60607@stoneleaf.us> On 04/27/2013 10:35 AM, Guido van Rossum wrote: > On Sat, Apr 27, 2013 at 10:04 AM, Ethan Furman wrote: >> While this will certainly work, it means you can't have class variables that >> happen to be the same type as the enum -- so no int in an IntEnum, for >> example. >> >> The solution I like best is the helper class (called, originally enough, >> enum), and only those items get transformed: >> >> class Planet(IntEnum): >> MERCURY = enum(1) >> VENUS = enum(2) >> EARTH = enum(3) >> rough_pi = 3 # not transformed > > If this means that the most plain vanilla enum definition still has to > use the enum(i) notation, I'm against it. Why do you want rough_pi to > be a class variable anyway? The whole point of an enum is that it's > *not* a kitchen sink class. An enum for the planets will need other > support code that doesn't live in the enum class -- it shouldn't be > considered a general scope for miscellanea. (TBH, I think that using > classes to scope variables is mostly misguided anyway -- the standard > mechanism for scoping is the module.) The two primary use cases I see are the (1) quick give me some names to values so I can fiddle and experiment, and the (2) production code with nice docs and the whole shebang. For (1) I would just use the _make (or whatever it's called) to give me something; for (2) using 'enum()' so that a docstring can also be added (even encouraged ;) seems like a Good Thing. And no, I have no idea what rough_pi is doing there, besides being an example on an int that doesn't get transformed. ;) -- ~Ethan~ From p.j.a.cock at googlemail.com Sat Apr 27 22:04:02 2013 From: p.j.a.cock at googlemail.com (Peter Cock) Date: Sat, 27 Apr 2013 21:04:02 +0100 Subject: [Python-Dev] Questions about Codon Alignment Proposal In-Reply-To: References: Message-ID: Hi Guido, Yes indeed, wrong list - I've forwarded this to the intended destination, biopython-dev, and we'll continue the discussion there: http://lists.open-bio.org/pipermail/biopython-dev/2013-April/010560.html (It is nice to see Python getting used in all sorts of Google Summer of Code projects though - and I'm sure we're all keen to try and welcome and encourage new students to the different open source communities.) Thanks, Peter On Sat, Apr 27, 2013 at 6:36 PM, Guido van Rossum wrote: > Sounds like this was accidentally CC'ed to python-dev. > > On Sat, Apr 27, 2013 at 10:23 AM, ?? wrote: >> Hi Eric and Peter, >> >> I'm preparing the proposal for the codon alignment project. Two things I may >> want to hear your advice. >> >> 1) In the biopython wiki page, you mentioned "model selection" in the >> Approach & Goals. I'm not sure if there are any advantages to use codon >> alignment for model selection. Could you give me some references? Another >> thing is that model selection involves estimation of tree topology as well >> as branch lengthes and parameters across many substitution models. Will it >> be too computationally intensive for a python implementation? >> >> 2) You also mentioned the "validation (testing for frame shift)". Is there a >> test for frame shift? Or I can simply detect it by comparing amino acid >> sequences and nucleotide sequences. >> >> Best, >> Zheng Ruan >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/guido%40python.org >> > > > > -- > --Guido van Rossum (python.org/~guido) From ethan at stoneleaf.us Sat Apr 27 22:01:01 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 27 Apr 2013 13:01:01 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517C2B45.9010806@stoneleaf.us> References: <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517B286D.2020601@canterbury.ac.nz> <517B3818.4010405@g.nevcal.com> <517B3D41.6060109@stoneleaf.us> <517C1CCC.9090203@pearwood.info> <517C2B45.9010806@stoneleaf.us> Message-ID: <517C2E7D.4040407@stoneleaf.us> On 04/27/2013 12:47 PM, Ethan Furman wrote: > On 04/27/2013 11:45 AM, Steven D'Aprano wrote: >> On 27/04/13 12:51, Ethan Furman wrote: >>> On 04/26/2013 07:29 PM, Glenn Linderman wrote: >> [...] >>>> class Color( Enum ): >>>> Enum.__enumerationItems__( >>>> red=1, >>>> green=2, >>>> blue=3, >>>> ) >>>> # other methods and assignments >>> >>> Or, if we go with the metaclass magic of re-using the class/type name (and who doesn't love metaclass magic??): >>> >>> class Color(Enum): >>> red = Color(1) >>> green = Color(2) >>> blue = Color 3) >>> look_ma_not_an_enum = 4 >> >> >> and from a later email: >> >>> The solution I like best is the helper class (called, originally enough, enum), and only those items get transformed: >>> >>> class Planet(IntEnum): >>> MERCURY = enum(1) >>> VENUS = enum(2) >>> EARTH = enum(3) >>> rough_pi = 3 # not transformed >> >> >> >> I'm sorry, but all these suggestions are getting the API completely backwards by making the common case harder than the >> rare case. > > I should have made a better example. In production code, doc strings can be priceless, so encouraging them seems like a > good idea: > > class Planet(IntEnum): > MERCURY = enum(1, doc='closest planet to the sun (?)') > VENUS = enum(2, doc='the one with two names') > EARTH = enum(3, doc='home sweet home') > random_value = 42 > > Of course, I think it would be even better if the name were 'Planet' instead of 'enum' as that would emphasize the fact > that we are actually creating items of the enumeration inside the enumeration. Kind of a shorthand for: > > class Planet(IntEnum): > def __init__(...): > ... > def blahblah(...): > ... > Planet.MERCURY = Planet(...) > Planet.VENUS = Planet(...) > Planet.EARTH = Planet(...) > > which is the way I've done it for other classes in a similar situation. > > >> We're creating an Enum, right? So the *common case* is to populate it with enum values. 99% of the time, enumerated >> values will be all that we want from an enum. So that's the case that needs to be simple, not the rare case where you >> have a non enum value in an enum class. >> >> The common case (enum values in an Enum class) should be easy, and the rare cases (ordinary class-like attributes) >> possible. >> >> Explicit is better than implicit: if you want something to *not* be processed by the Enum metaclass, you have to >> explicitly mark it as special. Dunders excepted, because they *are* special enough to break the rules. Since dunders are >> reserved for Python, I'm happy with a rule that says that dunders cannot be set as enum values (at least not via the >> metaclass). Otherwise, everything inside an Enum class is treated as an enum value unless explicitly flagged as not. > > While I agree that the common case should be simple, I also disagree that everything (especially functions) should > easily be an enumerated value; the nice thing about being explicit as to which are the values (using 'enum' for example) > is that it can also be used to capture functions in the rare case where that's what is desired. Just a quick followup: It seems to me that the *most* common case will be a simple name mapping, in which case one can do: Planet = Enum._make('Planet', 'MERCURY VENUS EARTH') and be done with it. -- ~Ethan~ From ethan at stoneleaf.us Sat Apr 27 21:47:17 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 27 Apr 2013 12:47:17 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517C1CCC.9090203@pearwood.info> References: <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517B286D.2020601@canterbury.ac.nz> <517B3818.4010405@g.nevcal.com> <517B3D41.6060109@stoneleaf.us> <517C1CCC.9090203@pearwood.info> Message-ID: <517C2B45.9010806@stoneleaf.us> On 04/27/2013 11:45 AM, Steven D'Aprano wrote: > On 27/04/13 12:51, Ethan Furman wrote: >> On 04/26/2013 07:29 PM, Glenn Linderman wrote: > [...] >>> class Color( Enum ): >>> Enum.__enumerationItems__( >>> red=1, >>> green=2, >>> blue=3, >>> ) >>> # other methods and assignments >> >> Or, if we go with the metaclass magic of re-using the class/type name (and who doesn't love metaclass magic??): >> >> class Color(Enum): >> red = Color(1) >> green = Color(2) >> blue = Color 3) >> look_ma_not_an_enum = 4 > > > and from a later email: > >> The solution I like best is the helper class (called, originally enough, enum), and only those items get transformed: >> >> class Planet(IntEnum): >> MERCURY = enum(1) >> VENUS = enum(2) >> EARTH = enum(3) >> rough_pi = 3 # not transformed > > > > I'm sorry, but all these suggestions are getting the API completely backwards by making the common case harder than the > rare case. I should have made a better example. In production code, doc strings can be priceless, so encouraging them seems like a good idea: class Planet(IntEnum): MERCURY = enum(1, doc='closest planet to the sun (?)') VENUS = enum(2, doc='the one with two names') EARTH = enum(3, doc='home sweet home') random_value = 42 Of course, I think it would be even better if the name were 'Planet' instead of 'enum' as that would emphasize the fact that we are actually creating items of the enumeration inside the enumeration. Kind of a shorthand for: class Planet(IntEnum): def __init__(...): ... def blahblah(...): ... Planet.MERCURY = Planet(...) Planet.VENUS = Planet(...) Planet.EARTH = Planet(...) which is the way I've done it for other classes in a similar situation. > We're creating an Enum, right? So the *common case* is to populate it with enum values. 99% of the time, enumerated > values will be all that we want from an enum. So that's the case that needs to be simple, not the rare case where you > have a non enum value in an enum class. > > The common case (enum values in an Enum class) should be easy, and the rare cases (ordinary class-like attributes) > possible. > > Explicit is better than implicit: if you want something to *not* be processed by the Enum metaclass, you have to > explicitly mark it as special. Dunders excepted, because they *are* special enough to break the rules. Since dunders are > reserved for Python, I'm happy with a rule that says that dunders cannot be set as enum values (at least not via the > metaclass). Otherwise, everything inside an Enum class is treated as an enum value unless explicitly flagged as not. While I agree that the common case should be simple, I also disagree that everything (especially functions) should easily be an enumerated value; the nice thing about being explicit as to which are the values (using 'enum' for example) is that it can also be used to capture functions in the rare case where that's what is desired. -- ~Ethan~ From guido at python.org Sat Apr 27 23:57:11 2013 From: guido at python.org (Guido van Rossum) Date: Sat, 27 Apr 2013 14:57:11 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517C2E7D.4040407@stoneleaf.us> References: <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517B286D.2020601@canterbury.ac.nz> <517B3818.4010405@g.nevcal.com> <517B3D41.6060109@stoneleaf.us> <517C1CCC.9090203@pearwood.info> <517C2B45.9010806@stoneleaf.us> <517C2E7D.4040407@stoneleaf.us> Message-ID: On Sat, Apr 27, 2013 at 1:01 PM, Ethan Furman wrote: > It seems to me that the *most* common case will be a simple name mapping, in > which case one can do: > > Planet = Enum._make('Planet', 'MERCURY VENUS EARTH') > > and be done with it. That looks horrible. -- --Guido van Rossum (python.org/~guido) From ethan at stoneleaf.us Sun Apr 28 00:17:25 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 27 Apr 2013 15:17:25 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517B286D.2020601@canterbury.ac.nz> <517B3818.4010405@g.nevcal.com> <517B3D41.6060109@stoneleaf.us> <517C1CCC.9090203@pearwood.info> <517C2B45.9010806@stoneleaf.us> <517C2E7D.4040407@stoneleaf.us> Message-ID: <517C4E75.8090206@stoneleaf.us> On 04/27/2013 02:57 PM, Guido van Rossum wrote: > On Sat, Apr 27, 2013 at 1:01 PM, Ethan Furman wrote: >> It seems to me that the *most* common case will be a simple name mapping, in >> which case one can do: >> >> Planet = Enum._make('Planet', 'MERCURY VENUS EARTH') >> >> and be done with it. > > That looks horrible. I believe that's one of the current methods in flufl.enum. -- ~Ethan~ From pjenvey at underboss.org Sun Apr 28 01:07:45 2013 From: pjenvey at underboss.org (Philip Jenvey) Date: Sat, 27 Apr 2013 16:07:45 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517B286D.2020601@canterbury.ac.nz> <517B3818.4010405@g.nevcal.com> <517B3D41.6060109@stoneleaf.us> <517C1CCC.9090203@pearwood.info> <517C2B45.9010806@stoneleaf.us> <517C2E7D.4040407@stoneleaf.us> Message-ID: On Apr 27, 2013, at 2:57 PM, Guido van Rossum wrote: > On Sat, Apr 27, 2013 at 1:01 PM, Ethan Furman wrote: >> It seems to me that the *most* common case will be a simple name mapping, in >> which case one can do: >> >> Planet = Enum._make('Planet', 'MERCURY VENUS EARTH') >> >> and be done with it. > > That looks horrible. Call me crazy, but might I suggest: class Planet(Enum, values='MERCURY VENUS EARTH'): """Planets of the Solar System""" I've always wanted something similar for namedtuples, such as: class Point(NamedTuple, field_names='x y z'): """Planet location with Sun as etc""" Especially when the common idiom for specifying namedtuples w/ docstrings looks similar but leaves a lot to be desired w/ the required duplication of the name: class Point(namedtuple('Point', 'x y z')): """Planet location with Sun as etc""" (Raymond's even endorsed the former): http://bugs.python.org/msg111722 -- Philip Jenvey From ethan at stoneleaf.us Sun Apr 28 01:17:46 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 27 Apr 2013 16:17:46 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517B286D.2020601@canterbury.ac.nz> <517B3818.4010405@g.nevcal.com> <517B3D41.6060109@stoneleaf.us> <517C1CCC.9090203@pearwood.info> <517C2B45.9010806@stoneleaf.us> <517C2E7D.4040407@stoneleaf.us> Message-ID: <517C5C9A.7020406@stoneleaf.us> On 04/27/2013 04:07 PM, Philip Jenvey wrote: > > Call me crazy, but might I suggest: > > class Planet(Enum, values='MERCURY VENUS EARTH'): > """Planets of the Solar System""" Okay, you're crazy! ;) I must be too, 'cause I really like that suggestion. Works easily, simple metaclass (or simple addition to current metaclass). Very nice. -- ~Ethan~ From ethan at stoneleaf.us Sun Apr 28 02:10:24 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 27 Apr 2013 17:10:24 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517C5C9A.7020406@stoneleaf.us> References: <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517B286D.2020601@canterbury.ac.nz> <517B3818.4010405@g.nevcal.com> <517B3D41.6060109@stoneleaf.us> <517C1CCC.9090203@pearwood.info> <517C2B45.9010806@stoneleaf.us> <517C2E7D.4040407@stoneleaf.us> <517C5C9A.7020406@stoneleaf.us> Message-ID: <517C68F0.1010201@stoneleaf.us> On 04/27/2013 04:17 PM, Ethan Furman wrote: > On 04/27/2013 04:07 PM, Philip Jenvey wrote: >> >> class Planet(Enum, values='MERCURY VENUS EARTH'): >> """Planets of the Solar System""" > > I must be too, 'cause I really like that suggestion. Works easily, simple metaclass (or simple addition to current > metaclass). > > Very nice. Having said that, what does it look like for a longer enum? class Planet( Enum, names=''' MERCURY VENUS EARTH MARS SATURN JUPITER URANUS PLUTO ''', ): '''Planets of the Solar System''' Not sure I like that. Ah well. -- ~Ethan~ From Nikolaus at rath.org Sun Apr 28 02:38:23 2013 From: Nikolaus at rath.org (Nikolaus Rath) Date: Sat, 27 Apr 2013 17:38:23 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library References: <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517B286D.2020601@canterbury.ac.nz> <517B3818.4010405@g.nevcal.com> <517B3D41.6060109@stoneleaf.us> <517C1CCC.9090203@pearwood.info> Message-ID: <87bo8zjxq8.fsf@vostro.rath.org> Steven D'Aprano writes: > I'm sorry, but all these suggestions are getting the API completely > backwards by making the common case harder than the rare case. > > We're creating an Enum, right? So the *common case* is to populate it > with enum values. 99% of the time, enumerated values will be all that > we want from an enum. So that's the case that needs to be simple, not > the rare case where you have a non enum value in an enum class. > > The common case (enum values in an Enum class) should be easy, and the > rare cases (ordinary class-like attributes) possible. > > Explicit is better than implicit: if you want something to *not* be > processed by the Enum metaclass, you have to explicitly mark it as > special. Dunders excepted, because they *are* special enough to break > the rules. Since dunders are reserved for Python, I'm happy with a > rule that says that dunders cannot be set as enum values (at least not > via the metaclass). Otherwise, everything inside an Enum class is > treated as an enum value unless explicitly flagged as not. > > Here's a dirty hack that demonstrates what I'm talking about. [...] > class Example(metaclass=MetaEnum): > red = 1 > blue = 2 > green = lambda: 'good lord, even functions can be enums!' > def __init__(self, count=3): > self.count = count > food = skip('spam') > @skip > def spam(self): > return self.count * self.food However, without knowing that the MetaEnum metaclass will do some magic here, there's no way to know that there's anything special about red, blue and green. So I think there's actually a lot of implicit stuff happening here. In contrast, class Example(metaclass=MetaEnum): red = EnumValue(1) blue = EnumValue(2) green = EnumValue(lambda: 'good lord, even functions can be enums!') def __init__(self, count=3): self.count = count def spam(self): return self.count * self.food Makes it very clear that red, blue will not be attributes of type int, even if one has never heard of Enums or metaclasses before. I don't think this syntax is making the common case hard. By the same logic, you'd need to introduce C-style i++ postincrement because having just "i += x" makes the common case with x=1 "hard" as well. Best, -Nikolaus -- ?Time flies like an arrow, fruit flies like a Banana.? PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C From pje at telecommunity.com Sun Apr 28 03:07:49 2013 From: pje at telecommunity.com (PJ Eby) Date: Sat, 27 Apr 2013 21:07:49 -0400 Subject: [Python-Dev] class name spaces inside an outer function In-Reply-To: <517C189B.1000504@stoneleaf.us> References: <517C189B.1000504@stoneleaf.us> Message-ID: On Sat, Apr 27, 2013 at 2:27 PM, Ethan Furman wrote: > I filed bug http://bugs.python.org/issue17853 last night. > > If somebody could point me in the right direction (mainly which files to > look in), I'd be happy to attempt a patch. Wow. I had no idea Python actually did this (override class-local references with ; I'd have expected your code to work. I was even more surprised to find that the same thing happens all the way back to Python 2.3. Guess I'm not nearly the wizard of abusing scope rules that I thought I was. ;-) About the only workaround I can see is to put "Season = Season" at the top of a class that uses this inside a function definition, or else to define a special name 'enum' instead and hope that nobody ever tries to define an enumeration inside a function with a local variable named 'enum'. ;-) From guido at python.org Sun Apr 28 03:09:02 2013 From: guido at python.org (Guido van Rossum) Date: Sat, 27 Apr 2013 18:09:02 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517C68F0.1010201@stoneleaf.us> References: <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517B286D.2020601@canterbury.ac.nz> <517B3818.4010405@g.nevcal.com> <517B3D41.6060109@stoneleaf.us> <517C1CCC.9090203@pearwood.info> <517C2B45.9010806@stoneleaf.us> <517C2E7D.4040407@stoneleaf.us> <517C5C9A.7020406@stoneleaf.us> <517C68F0.1010201@stoneleaf.us> Message-ID: On Sat, Apr 27, 2013 at 5:10 PM, Ethan Furman wrote: > class Planet( > Enum, > names=''' > MERCURY > VENUS > EARTH > MARS > SATURN > JUPITER > URANUS > PLUTO > ''', > ): > '''Planets of the Solar System''' > > Not sure I like that. Ah well. The problem with this and similar proposals is that it puts things inside string quotes that belong outside them. -- --Guido van Rossum (python.org/~guido) From Nikolaus at rath.org Sun Apr 28 02:42:03 2013 From: Nikolaus at rath.org (Nikolaus Rath) Date: Sat, 27 Apr 2013 17:42:03 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library References: <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517C050A.7070305@stoneleaf.us> Message-ID: <878v43jxk4.fsf@vostro.rath.org> Guido van Rossum writes: > On Sat, Apr 27, 2013 at 10:04 AM, Ethan Furman wrote: >> While this will certainly work, it means you can't have class variables that >> happen to be the same type as the enum -- so no int in an IntEnum, for >> example. >> >> The solution I like best is the helper class (called, originally enough, >> enum), and only those items get transformed: >> >> class Planet(IntEnum): >> MERCURY = enum(1) >> VENUS = enum(2) >> EARTH = enum(3) >> rough_pi = 3 # not transformed > > If this means that the most plain vanilla enum definition still has to > use the enum(i) notation, I'm against it. I think this is actually a big advantage. It makes it obvious that something special is going on without having to know that IntEnum uses a special metaclass. Best, -Nikolaus -- ?Time flies like an arrow, fruit flies like a Banana.? PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C From ncoghlan at gmail.com Sun Apr 28 03:26:28 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 28 Apr 2013 11:26:28 +1000 Subject: [Python-Dev] class name spaces inside an outer function In-Reply-To: <517C189B.1000504@stoneleaf.us> References: <517C189B.1000504@stoneleaf.us> Message-ID: On 28 Apr 2013 04:30, "Ethan Furman" wrote: > > I filed bug http://bugs.python.org/issue17853 last night. > > If somebody could point me in the right direction (mainly which files to look in), I'd be happy to attempt a patch. Hmm, interesting challenge. A key part of the problem is that the 3.x compiler assumes there's no way to inject names it doesn't know about into code inside a function - we missed the fact that you could still do it with a nested class and a metaclass __prepare__ method. I suspect resolving it sensibly will require a new opcode that tries a local-only load and then falls back to loading from a cell rather than from a global/builtins lookup. Cheers, Nick. > > -- > ~Ethan~ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Sun Apr 28 03:38:08 2013 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 27 Apr 2013 21:38:08 -0400 Subject: [Python-Dev] class name spaces inside an outer function In-Reply-To: References: <517C189B.1000504@stoneleaf.us> Message-ID: 2013/4/27 Nick Coghlan : > > On 28 Apr 2013 04:30, "Ethan Furman" wrote: >> >> I filed bug http://bugs.python.org/issue17853 last night. >> >> If somebody could point me in the right direction (mainly which files to >> look in), I'd be happy to attempt a patch. > > Hmm, interesting challenge. A key part of the problem is that the 3.x > compiler assumes there's no way to inject names it doesn't know about into > code inside a function - we missed the fact that you could still do it with > a nested class and a metaclass __prepare__ method. That's not the problem. You can't inject names dynamically into a function scope with variables in a class scope, since nothing closes over them. > > I suspect resolving it sensibly will require a new opcode that tries a > local-only load and then falls back to loading from a cell rather than from > a global/builtins lookup. Yes. -- Regards, Benjamin From greg.ewing at canterbury.ac.nz Sun Apr 28 04:01:32 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 28 Apr 2013 14:01:32 +1200 Subject: [Python-Dev] class name spaces inside an outer function In-Reply-To: References: <517C189B.1000504@stoneleaf.us> Message-ID: <517C82FC.5040401@canterbury.ac.nz> PJ Eby wrote: > On Sat, Apr 27, 2013 at 2:27 PM, Ethan Furman wrote: > >>I filed bug http://bugs.python.org/issue17853 last night. > About the only workaround I can see is to put "Season = Season" at the > top of a class that uses this inside a function definition, This whole business can be avoided by doing things differently in the first place. Instead of initialising the enum items by calling the class, just assign a tuple of args to the name and have the metaclass make the constructor call. class Planet(Enum): MERCURY = (3.303e+23, 2.4397e6) VENUS = (4.869e+24, 6.0518e6) EARTH = (5.976e+24, 6.37814e6) MARS = (6.421e+23, 3.3972e6) JUPITER = (1.9e+27, 7.1492e7) SATURN = (5.688e+26, 6.0268e7) URANUS = (8.686e+25, 2.5559e7) NEPTUNE = (1.024e+26, 2.4746e7) def __init__(self, mass, radius): self.mass = mass self.radius = radius I think that's better anyway, since it avoids aggravated violation of DRY by repeating the class name umpteen times. -- Greg From greg.ewing at canterbury.ac.nz Sun Apr 28 04:12:22 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 28 Apr 2013 14:12:22 +1200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: References: <20130425180218.69ed8004@anarchist> <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> Message-ID: <517C8586.2000804@canterbury.ac.nz> Guido van Rossum wrote: > And __init__/__new__ probably shouldn't be > overridden. Why shouldn't __init__ be overridden? It's the obvious way to support Java-style enum-items-with-attributes. -- Greg From ethan at stoneleaf.us Sun Apr 28 04:22:48 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 27 Apr 2013 19:22:48 -0700 Subject: [Python-Dev] class name spaces inside an outer function In-Reply-To: <517C82FC.5040401@canterbury.ac.nz> References: <517C189B.1000504@stoneleaf.us> <517C82FC.5040401@canterbury.ac.nz> Message-ID: <517C87F8.7010403@stoneleaf.us> On 04/27/2013 07:01 PM, Greg Ewing wrote: > PJ Eby wrote: >> On Sat, Apr 27, 2013 at 2:27 PM, Ethan Furman wrote: >> >>> I filed bug http://bugs.python.org/issue17853 last night. > >> About the only workaround I can see is to put "Season = Season" at the >> top of a class that uses this inside a function definition, > > This whole business can be avoided by doing things differently > in the first place. Instead of initialising the enum items by > calling the class, just assign a tuple of args to the name > and have the metaclass make the constructor call. > > class Planet(Enum): > MERCURY = (3.303e+23, 2.4397e6) > VENUS = (4.869e+24, 6.0518e6) > EARTH = (5.976e+24, 6.37814e6) > MARS = (6.421e+23, 3.3972e6) > JUPITER = (1.9e+27, 7.1492e7) > SATURN = (5.688e+26, 6.0268e7) > URANUS = (8.686e+25, 2.5559e7) > NEPTUNE = (1.024e+26, 2.4746e7) > > def __init__(self, mass, radius): > self.mass = mass > self.radius = radius > > I think that's better anyway, since it avoids aggravated > violation of DRY by repeating the class name umpteen times. You certainly have a point about DRY, and generally I agree with you, but given the nature of Enums I can see a little extra RY being useful. Regardless of the outcome for Enums, I can see another metaclass doing the same kind of thing and having it work just fine until an unsuspecting soul tries to reuse an inserted name further down the function and suddenly the whole thing blows up on him. Now, I'll grant you it's not like a seg fault, but it would be nice if Python followed its own lookup rules. -- ~Ethan~ From ethan at stoneleaf.us Sun Apr 28 04:29:55 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 27 Apr 2013 19:29:55 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517C8586.2000804@canterbury.ac.nz> References: <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517C8586.2000804@canterbury.ac.nz> Message-ID: <517C89A3.4090002@stoneleaf.us> On 04/27/2013 07:12 PM, Greg Ewing wrote: > Guido van Rossum wrote: >> And __init__/__new__ probably shouldn't be >> overridden. > > Why shouldn't __init__ be overridden? It's the obvious > way to support Java-style enum-items-with-attributes. Overriding __init__ is a PITA because __init__ is also called when you do Planet(3) # get EARTH and __init__ was expecting a gravitational constant and radius (or something like that). A couple ways around that: 1) have the metaclass store the args somewhere special (e.g. _args), have __init__ look like `def __init__(self, value=None)`, and have the body treat _args as if it were *args 2) have a `_init` that the metaclass calls with the args instead of __init__ -- ~Ethan~ From brett at python.org Sun Apr 28 05:23:16 2013 From: brett at python.org (Brett Cannon) Date: Sat, 27 Apr 2013 23:23:16 -0400 Subject: [Python-Dev] [Python-checkins] cpython (3.3): Issue #17357: Use more stern wording for In-Reply-To: <3ZyvRl2W06z7LjT@mail.python.org> References: <3ZyvRl2W06z7LjT@mail.python.org> Message-ID: This actually should have been for issue #17330, but I had the wrong bug open when I copy-and-pasted the number. On Sat, Apr 27, 2013 at 11:21 PM, brett.cannon wrote: > http://hg.python.org/cpython/rev/75e32a0bfd74 > changeset: 83525:75e32a0bfd74 > branch: 3.3 > parent: 83517:4b4ed1e11fd0 > user: Brett Cannon > date: Sat Apr 27 23:20:32 2013 -0400 > summary: > Issue #17357: Use more stern wording for > importlib.invalidate_caches(). > > files: > Doc/library/importlib.rst | 10 +++++----- > 1 files changed, 5 insertions(+), 5 deletions(-) > > > diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst > --- a/Doc/library/importlib.rst > +++ b/Doc/library/importlib.rst > @@ -103,9 +103,9 @@ > > Invalidate the internal caches of finders stored at > :data:`sys.meta_path`. If a finder implements ``invalidate_caches()`` then it > - will be called to perform the invalidation. This function may be needed if > - some modules are installed while your program is running and you expect the > - program to notice the changes. > + will be called to perform the invalidation. This function should be called > + if any modules are created/installed while your program is running to > + guarantee all finders will notice the new module's existence. > > .. versionadded:: 3.3 > > @@ -182,7 +182,7 @@ > > .. versionadded:: 3.3 > > - .. method:: find_loader(fullname): > + .. method:: find_loader(fullname) > > An abstract method for finding a :term:`loader` for the specified > module. Returns a 2-tuple of ``(loader, portion)`` where ``portion`` > @@ -194,7 +194,7 @@ > the empty list then no loader or location for a namespace package were > found (i.e. failure to find anything for the module). > > - .. method:: find_module(fullname): > + .. method:: find_module(fullname) > > A concrete implementation of :meth:`Finder.find_module` which is > equivalent to ``self.find_loader(fullname)[0]``. > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > From ncoghlan at gmail.com Sun Apr 28 05:23:54 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 28 Apr 2013 13:23:54 +1000 Subject: [Python-Dev] class name spaces inside an outer function In-Reply-To: References: <517C189B.1000504@stoneleaf.us> Message-ID: On Sun, Apr 28, 2013 at 11:38 AM, Benjamin Peterson wrote: > 2013/4/27 Nick Coghlan : >> >> On 28 Apr 2013 04:30, "Ethan Furman" wrote: >>> >>> I filed bug http://bugs.python.org/issue17853 last night. >>> >>> If somebody could point me in the right direction (mainly which files to >>> look in), I'd be happy to attempt a patch. >> >> Hmm, interesting challenge. A key part of the problem is that the 3.x >> compiler assumes there's no way to inject names it doesn't know about into >> code inside a function - we missed the fact that you could still do it with >> a nested class and a metaclass __prepare__ method. > > That's not the problem. You can't inject names dynamically into a > function scope with variables in a class scope, since nothing closes > over them. Yeah, what I wrote didn't quite capture what I meant: - in Python 2.x, using LOAD_DEREF when a nested class body references a lexically scoped name is correct - in Python 3.x, it is no longer correct, because __prepare__ may inject additional names that the compiler doesn't know about Previously, the compiler new just as much about the nested class namespaces as it did about the function locals. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From benjamin at python.org Sun Apr 28 05:28:36 2013 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 27 Apr 2013 23:28:36 -0400 Subject: [Python-Dev] class name spaces inside an outer function In-Reply-To: References: <517C189B.1000504@stoneleaf.us> Message-ID: 2013/4/27 Nick Coghlan : > On Sun, Apr 28, 2013 at 11:38 AM, Benjamin Peterson wrote: >> 2013/4/27 Nick Coghlan : >>> >>> On 28 Apr 2013 04:30, "Ethan Furman" wrote: >>>> >>>> I filed bug http://bugs.python.org/issue17853 last night. >>>> >>>> If somebody could point me in the right direction (mainly which files to >>>> look in), I'd be happy to attempt a patch. >>> >>> Hmm, interesting challenge. A key part of the problem is that the 3.x >>> compiler assumes there's no way to inject names it doesn't know about into >>> code inside a function - we missed the fact that you could still do it with >>> a nested class and a metaclass __prepare__ method. >> >> That's not the problem. You can't inject names dynamically into a >> function scope with variables in a class scope, since nothing closes >> over them. > > Yeah, what I wrote didn't quite capture what I meant: > > - in Python 2.x, using LOAD_DEREF when a nested class body references > a lexically scoped name is correct > - in Python 3.x, it is no longer correct, because __prepare__ may > inject additional names that the compiler doesn't know about You could still get the same "bug" in Python 2 by messing with locals() in a class within a function. -- Regards, Benjamin From greg.ewing at canterbury.ac.nz Sun Apr 28 05:59:25 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 28 Apr 2013 15:59:25 +1200 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517C89A3.4090002@stoneleaf.us> References: <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517C8586.2000804@canterbury.ac.nz> <517C89A3.4090002@stoneleaf.us> Message-ID: <517C9E9D.7030508@canterbury.ac.nz> Ethan Furman wrote: > Overriding __init__ is a PITA because __init__ is also called when you do > > Planet(3) # get EARTH > > and __init__ was expecting a gravitational constant and radius (or > something like that). > > A couple ways around that: > > 1) have the metaclass store the args somewhere special > > 2) have a `_init` that the metaclass calls with the args instead of > __init__ I don't much like either of those. It would be much nicer if one could just write an ordinary __init__ method and have it work as expected. It's possible to make it work, I think. The __call__ method of the metaclass is going to have to do something special anyway, so that Planet(3) can look up and return an existing instance instead of making a new one. And if it doesn't make a new instance, it's not going to call the __init__ method. -- Greg From guido at python.org Sun Apr 28 06:20:07 2013 From: guido at python.org (Guido van Rossum) Date: Sat, 27 Apr 2013 21:20:07 -0700 Subject: [Python-Dev] class name spaces inside an outer function In-Reply-To: <517C82FC.5040401@canterbury.ac.nz> References: <517C189B.1000504@stoneleaf.us> <517C82FC.5040401@canterbury.ac.nz> Message-ID: On Saturday, April 27, 2013, Greg Ewing wrote: > > This whole business can be avoided by doing things differently > in the first place. Instead of initialising the enum items by > calling the class, just assign a tuple of args to the name > and have the metaclass make the constructor call. > > class Planet(Enum): > MERCURY = (3.303e+23, 2.4397e6) > VENUS = (4.869e+24, 6.0518e6) > EARTH = (5.976e+24, 6.37814e6) > MARS = (6.421e+23, 3.3972e6) > JUPITER = (1.9e+27, 7.1492e7) > SATURN = (5.688e+26, 6.0268e7) > URANUS = (8.686e+25, 2.5559e7) > NEPTUNE = (1.024e+26, 2.4746e7) > > def __init__(self, mass, radius): > self.mass = mass > self.radius = radius > > I think that's better anyway, since it avoids aggravated > violation of DRY by repeating the class name umpteen times. > If you want something like this, doyou really have to inherit from Enum? -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Sun Apr 28 08:10:42 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 27 Apr 2013 23:10:42 -0700 Subject: [Python-Dev] class name spaces inside an outer function In-Reply-To: References: <517C189B.1000504@stoneleaf.us> <517C82FC.5040401@canterbury.ac.nz> Message-ID: <517CBD62.1060007@stoneleaf.us> On 04/27/2013 09:20 PM, Guido van Rossum wrote: > On Saturday, April 27, 2013, Greg Ewing wrote: > > This whole business can be avoided by doing things differently > in the first place. Instead of initialising the enum items by > calling the class, just assign a tuple of args to the name > and have the metaclass make the constructor call. > > class Planet(Enum): > MERCURY = (3.303e+23, 2.4397e6) > VENUS = (4.869e+24, 6.0518e6) > EARTH = (5.976e+24, 6.37814e6) > MARS = (6.421e+23, 3.3972e6) > JUPITER = (1.9e+27, 7.1492e7) > SATURN = (5.688e+26, 6.0268e7) > URANUS = (8.686e+25, 2.5559e7) > NEPTUNE = (1.024e+26, 2.4746e7) > > def __init__(self, mass, radius): > self.mass = mass > self.radius = radius > > I think that's better anyway, since it avoids aggravated > violation of DRY by repeating the class name umpteen times. > > If you want something like this, doyou really have to inherit from Enum? If I'm saying what you already know I apologize now, but this thread is about what happens when: class InsertsName(type): @classmethod def __prepare__(metacls, cls, bases): classdict = {'new_name': lambda: 'haha!'} return classdict def test(): new_name = 'Jose' # if here will result in str not callable error class SomeClass(metaclass=InsertsName): surprise = new_name() new_name = 'Clara' # if here will result in NameError: free variable... However, if that class definition is either top level, or if the function itself does not define nor use the 'new_name', there is no problem. Enum was being used in the example because that's what I was toying with when I found the problem. -- ~Ethan~ From ethan at stoneleaf.us Sun Apr 28 08:37:24 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 27 Apr 2013 23:37:24 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517C9E9D.7030508@canterbury.ac.nz> References: <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517C8586.2000804@canterbury.ac.nz> <517C89A3.4090002@stoneleaf.us> <517C9E9D.7030508@canterbury.ac.nz> Message-ID: <517CC3A4.4020203@stoneleaf.us> On 04/27/2013 08:59 PM, Greg Ewing wrote: > Ethan Furman wrote: >> Overriding __init__ is a PITA because __init__ is also called when you do >> >> Planet(3) # get EARTH >> >> and __init__ was expecting a gravitational constant and radius (or something like that). >> >> A couple ways around that: >> >> 1) have the metaclass store the args somewhere special >> >> 2) have a `_init` that the metaclass calls with the args instead of __init__ > > I don't much like either of those. It would be much nicer if > one could just write an ordinary __init__ method and have it > work as expected. Agreed, on both counts. > It's possible to make it work, I think. The __call__ method > of the metaclass is going to have to do something special > anyway, so that Planet(3) can look up and return an existing > instance instead of making a new one. And if it doesn't make > a new instance, it's not going to call the __init__ method. So far I've had that logic in __new__ (which, of course, has no control on whether __init__ is called); I'll check out __call__ as soon as I can. Thanks for the tip! -- ~Ethan~ From ethan at stoneleaf.us Sun Apr 28 17:48:13 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 28 Apr 2013 08:48:13 -0700 Subject: [Python-Dev] PEP 435 -- Adding an Enum type to the Python standard library In-Reply-To: <517C9E9D.7030508@canterbury.ac.nz> References: <20130425182934.3eca2633@anarchist> <5179F0B8.8010601@g.nevcal.com> <5179F30D.3050508@canterbury.ac.nz> <5179F6EC.5040301@g.nevcal.com> <517ABA70.8020909@g.nevcal.com> <517C8586.2000804@canterbury.ac.nz> <517C89A3.4090002@stoneleaf.us> <517C9E9D.7030508@canterbury.ac.nz> Message-ID: <517D44BD.3090507@stoneleaf.us> On 04/27/2013 08:59 PM, Greg Ewing wrote: > > It's possible to make it work, I think. The __call__ method > of the metaclass is going to have to do something special > anyway, so that Planet(3) can look up and return an existing > instance instead of making a new one. And if it doesn't make > a new instance, it's not going to call the __init__ method. It works beautifully! It's not even complicated because the metaclass __new__ uses object.__new__ to create the instances, so EnumType.__call__ is /only/ called in cases like Planet(3), or Planet('EARTH'). Sweet! -- ~Ethan~ From ethan at stoneleaf.us Sun Apr 28 18:26:41 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 28 Apr 2013 09:26:41 -0700 Subject: [Python-Dev] class name spaces inside an outer function In-Reply-To: References: <517C189B.1000504@stoneleaf.us> <517C82FC.5040401@canterbury.ac.nz> Message-ID: <517D4DC1.4010808@stoneleaf.us> On 04/27/2013 09:20 PM, Guido van Rossum wrote: > On Saturday, April 27, 2013, Greg Ewing wrote: > >> class Planet(Enum): >> MERCURY = (3.303e+23, 2.4397e6) >> VENUS = (4.869e+24, 6.0518e6) >> EARTH = (5.976e+24, 6.37814e6) >> MARS = (6.421e+23, 3.3972e6) >> JUPITER = (1.9e+27, 7.1492e7) >> SATURN = (5.688e+26, 6.0268e7) >> URANUS = (8.686e+25, 2.5559e7) >> NEPTUNE = (1.024e+26, 2.4746e7) >> >> def __init__(self, mass, radius): >> self.mass = mass >> self.radius = radius > > If you want something like this, do you really have to inherit from Enum? To answer your question: Somewhere in the previous threads about enums a couple people had use-cases for an enum with extra attributes. So, while you don't /have/ to enherit from Enum, if Enum provides the basics of what you need, and you can extend it with the extra functionality that you need, why shouldn't you? (Not a rhetorical question -- I'm happy to learn something I don't know.) -- ~Ethan~ From g.brandl at gmx.net Sun Apr 28 21:13:44 2013 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 28 Apr 2013 21:13:44 +0200 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? Message-ID: Hi all, I've just read a few dozen enum-related emails, and there are so many more. I would like to form an opinion about the proposal(s), but I feel I don't know what the actual issues are anymore. In the past, somebody usually presented a summary of the issues so far, and that was a good point for late comers to get up to speed and weigh in. (It can be here or in the PEP.) It is also a good point to focus the discussion (which seems to have wandered quite far from sensible Pythonic design in places). Thanks in advance, Georg From ethan at stoneleaf.us Sun Apr 28 21:32:20 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 28 Apr 2013 12:32:20 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: Message-ID: <517D7944.4050107@stoneleaf.us> Example enumeration: class Seasons(Enum): SPRING = 1 SUMMER = 2 AUTUMN = 3 WINTER = 4 days_in_year = 365 @property def avg_temp(self): return (75, 92, 66, 33)[int(self)+1] # enums are 1-based Definite Issues: - should enum items be of the type of the Enum class? (i.e. type(SPRING) is Seasons) - should an enum item be selectable via __call__ instead of __getitem__ (i.e. Seasons(3) is AUTUMN) - should days_in_year be enumerated? - should avg_temp be enumerated? - for the above two, how should they be included/excluded? From guido at python.org Sun Apr 28 22:02:11 2013 From: guido at python.org (Guido van Rossum) Date: Sun, 28 Apr 2013 13:02:11 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <517D7944.4050107@stoneleaf.us> References: <517D7944.4050107@stoneleaf.us> Message-ID: My opinions added On Sun, Apr 28, 2013 at 12:32 PM, Ethan Furman wrote: > Example enumeration: > > class Seasons(Enum): > SPRING = 1 > SUMMER = 2 > AUTUMN = 3 > WINTER = 4 > > days_in_year = 365 > > @property > def avg_temp(self): > return (75, 92, 66, 33)[int(self)+1] # enums are 1-based > > > Definite Issues: > > - should enum items be of the type of the Enum class? (i.e. type(SPRING) > is Seasons) IMO Yes. > - should an enum item be selectable via __call__ instead of __getitem__ > (i.e. Seasons(3) is AUTUMN) No opinion. > - should days_in_year be enumerated? Yes. (If you don't want it to be, and it's not a method/descriptor, move it out of the class.) > - should avg_temp be enumerated? IMO No. > - for the above two, how should they be included/excluded? IMO Everything should be enumerated except (a) things with a __get__() method (i.e. descriptors) (b) __dunder__ names Also, I believe there's still an issue on the order in which items are returned by iter(Seasons), but I don't know which way this is heading. -- --Guido van Rossum (python.org/~guido) From ethan at stoneleaf.us Sun Apr 28 22:36:30 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 28 Apr 2013 13:36:30 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: <517D7944.4050107@stoneleaf.us> Message-ID: <517D884E.9000007@stoneleaf.us> On 04/28/2013 01:02 PM, Guido van Rossum wrote: > > My opinions added Mine also now added. >> Example enumeration: >> >> class Seasons(Enum): >> SPRING = 1 >> SUMMER = 2 >> AUTUMN = 3 >> WINTER = 4 >> >> days_in_year = 365 >> >> @property >> def avg_temp(self): >> return (75, 92, 66, 33)[int(self)+1] # enums are 1-based >> >> >> Definite Issues: >> >> - should enum items be of the type of the Enum class? (i.e. type(SPRING) >> is Seasons) > > IMO Yes. I agree. >> - should an enum item be selectable via __call__ instead of __getitem__ >> (i.e. Seasons(3) is AUTUMN) > > No opinion. I think the callable syntax should be supported for database integration and consistency with every (?) other type in Python. No opinion about the __getitem__ portion. >> - should days_in_year be enumerated? > > Yes. (If you don't want it to be, and it's not a method/descriptor, > move it out of the class.) Making it a property to have it in the class certainly works for me. >> - should avg_temp be enumerated? > > IMO No. I agree. >> - for the above two, how should they be included/excluded? > > IMO Everything should be enumerated except > (a) things with a __get__() method (i.e. descriptors) > (b) __dunder__ names This also works for me. > Also, I believe there's still an issue on the order in which items are > returned by iter(Seasons), but I don't know which way this is heading. As somebody pointed out earlier, the only order which cannot be reconstructed after the fact is definition order (value order can be, lexical order can be, etc.). So my vote is to have the default iteration order be the original definition order, as any other desired order can be added to the class. -- ~Ethan~ From greg.ewing at canterbury.ac.nz Sun Apr 28 23:16:09 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 29 Apr 2013 09:16:09 +1200 Subject: [Python-Dev] class name spaces inside an outer function In-Reply-To: References: <517C189B.1000504@stoneleaf.us> <517C82FC.5040401@canterbury.ac.nz> Message-ID: <517D9199.9090009@canterbury.ac.nz> Guido van Rossum wrote: > On Saturday, April 27, 2013, Greg Ewing wrote: > > class Planet(Enum): > MERCURY = (3.303e+23, 2.4397e6) > VENUS = (4.869e+24, 6.0518e6) > EARTH = (5.976e+24, 6.37814e6) > > def __init__(self, mass, radius): > self.mass = mass > self.radius = radius > > If you want something like this, doyou really have to inherit from Enum? I suppose not, but the same could be said for cases where you want to add methods to enums, etc. -- Greg From solipsis at pitrou.net Sun Apr 28 23:29:32 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 28 Apr 2013 23:29:32 +0200 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? References: <517D7944.4050107@stoneleaf.us> Message-ID: <20130428232932.4651f405@fsol> On Sun, 28 Apr 2013 13:02:11 -0700 Guido van Rossum wrote: > > > - for the above two, how should they be included/excluded? > > IMO Everything should be enumerated except > (a) things with a __get__() method (i.e. descriptors) > (b) __dunder__ names I think it would be nice to define regular methods on enums. Regards Antoine. From ncoghlan at gmail.com Mon Apr 29 00:28:34 2013 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 29 Apr 2013 08:28:34 +1000 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <20130428232932.4651f405@fsol> References: <517D7944.4050107@stoneleaf.us> <20130428232932.4651f405@fsol> Message-ID: On 29 Apr 2013 07:32, "Antoine Pitrou" wrote: > > On Sun, 28 Apr 2013 13:02:11 -0700 > Guido van Rossum wrote: > > > > > - for the above two, how should they be included/excluded? > > > > IMO Everything should be enumerated except > > (a) things with a __get__() method (i.e. descriptors) > > (b) __dunder__ names > > I think it would be nice to define regular methods on enums. Functions are descriptors, so this rule already covers ordinary methods. The slight concern I have with making the duck typed exclusion only descriptors (rather than descriptors and callables) is that it means things like functools.partial objects will be treated as enum values rather than as static methods. OTOH, explicitly wrapping such callables in staticmethod should still work, so the simpler rule is probably better. Cheers, Nick. > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Mon Apr 29 00:35:26 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 29 Apr 2013 00:35:26 +0200 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: <517D7944.4050107@stoneleaf.us> <20130428232932.4651f405@fsol> Message-ID: <20130429003526.3609f5af@fsol> On Mon, 29 Apr 2013 08:28:34 +1000 Nick Coghlan wrote: > On 29 Apr 2013 07:32, "Antoine Pitrou" wrote: > > > > On Sun, 28 Apr 2013 13:02:11 -0700 > > Guido van Rossum wrote: > > > > > > > - for the above two, how should they be included/excluded? > > > > > > IMO Everything should be enumerated except > > > (a) things with a __get__() method (i.e. descriptors) > > > (b) __dunder__ names > > > > I think it would be nice to define regular methods on enums. > > Functions are descriptors, so this rule already covers ordinary methods. Oh, I had missed that. Regards Antoine. From ethan at stoneleaf.us Sun Apr 28 23:46:23 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 28 Apr 2013 14:46:23 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <20130428232932.4651f405@fsol> References: <517D7944.4050107@stoneleaf.us> <20130428232932.4651f405@fsol> Message-ID: <517D98AF.6040800@stoneleaf.us> On 04/28/2013 02:29 PM, Antoine Pitrou wrote: > On Sun, 28 Apr 2013 13:02:11 -0700 > Guido van Rossum wrote: >> >>> - for the above two, how should they be included/excluded? >> >> IMO Everything should be enumerated except >> (a) things with a __get__() method (i.e. descriptors) >> (b) __dunder__ names > > I think it would be nice to define regular methods on enums. No worries -- regular methods have a __get__. -- ~Ethan~ From steve at pearwood.info Mon Apr 29 01:37:44 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 29 Apr 2013 09:37:44 +1000 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: <517D7944.4050107@stoneleaf.us> Message-ID: <517DB2C8.9090002@pearwood.info> On 29/04/13 06:02, Guido van Rossum wrote: > My opinions added > > On Sun, Apr 28, 2013 at 12:32 PM, Ethan Furman wrote: >> Definite Issues: >> >> - should enum items be of the type of the Enum class? (i.e. type(SPRING) >> is Seasons) > > IMO Yes. +1 >> - should an enum item be selectable via __call__ instead of __getitem__ >> (i.e. Seasons(3) is AUTUMN) > > No opinion. Does anyone know why this is even an issue? Is this pure bike-shedding over the API, or are there technical reasons for choosing one over the other? >> - should days_in_year be enumerated? > > Yes. (If you don't want it to be, and it's not a method/descriptor, > move it out of the class.) +1, but see below. >> - should avg_temp be enumerated? > > IMO No. > >> - for the above two, how should they be included/excluded? > > IMO Everything should be enumerated except > (a) things with a __get__() method (i.e. descriptors) > (b) __dunder__ names +1 on excluding dunder names. +0 on excluding things with __get__. I have also suggested that that the enum package provide a decorator which can be used to explicitly flag values to *not* be turned into enum values. See here: http://mail.python.org/pipermail/python-dev/2013-April/125641.html Even if the Enum class doesn't support this feature, I ask that it be written in such a way that a subclass could add it (i.e. please expose a public method for deciding what to exclude). -- Steven From pje at telecommunity.com Mon Apr 29 02:05:59 2013 From: pje at telecommunity.com (PJ Eby) Date: Sun, 28 Apr 2013 20:05:59 -0400 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <517DB2C8.9090002@pearwood.info> References: <517D7944.4050107@stoneleaf.us> <517DB2C8.9090002@pearwood.info> Message-ID: On Sun, Apr 28, 2013 at 7:37 PM, Steven D'Aprano wrote: > I have also suggested that that the enum package provide a decorator > which can be used to explicitly flag values to *not* be turned into > enum values. See here: > > http://mail.python.org/pipermail/python-dev/2013-April/125641.html In that example, 'food = property(lambda:"skip")' would work in a pinch. (Granted, it wouldn't be a *class* attribute, but you can make a class attribute by assiging it after class creation is completed.) And if you want to make your enum instances callable, ISTM the right (or at least the One Obvious) way to do it is to add a __call__ method to the class. > Even if the Enum class doesn't support this feature, I ask that it be > written in such a way that a subclass could add it (i.e. please expose > a public method for deciding what to exclude). Since you can exclude anything by it having a __get__ method, or include it by making it *not* have a __get__ method, I'm not sure what use case you're actually looking for. From v+python at g.nevcal.com Mon Apr 29 02:07:32 2013 From: v+python at g.nevcal.com (Glenn Linderman) Date: Sun, 28 Apr 2013 17:07:32 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <517DB2C8.9090002@pearwood.info> References: <517D7944.4050107@stoneleaf.us> <517DB2C8.9090002@pearwood.info> Message-ID: <517DB9C4.8010204@g.nevcal.com> On 4/28/2013 4:37 PM, Steven D'Aprano wrote: > I have also suggested that that the enum package provide a decorator > which can be used to explicitly flag values to *not* be turned into > enum values. See here: > > http://mail.python.org/pipermail/python-dev/2013-April/125641.html > > Even if the Enum class doesn't support this feature, I ask that it be > written in such a way that a subclass could add it (i.e. please expose > a public method for deciding what to exclude). That last comment is very interesting to me... I think it would be good if the enum implementation doesn't prevent the creation of a subclass of integer enumerations which can be extended so that when arithmetic is done on them, that new enumeration values and names can be added to the collection (think flag fields). The add ones might have names that do not follow the rules for identifiers. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Mon Apr 29 02:29:35 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 28 Apr 2013 17:29:35 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <517DB2C8.9090002@pearwood.info> References: <517D7944.4050107@stoneleaf.us> <517DB2C8.9090002@pearwood.info> Message-ID: <517DBEEF.6060904@stoneleaf.us> On 04/28/2013 04:37 PM, Steven D'Aprano wrote: >> On Sun, Apr 28, 2013 at 12:32 PM, Ethan Furman wrote: >> >>> - should an enum item be selectable via __call__ instead of __getitem__ >>> (i.e. Seasons(3) is AUTUMN) > > Does anyone know why this is even an issue? Is this pure bike-shedding over the API, or are there > technical reasons for choosing one over the other? This is an issue because currently every other type* in Python creates (or selects ;) its instances via the call syntax: - bool(1) # True - int('11') # 11 - str(var) # whatever var had in it, now as a str But one of the latest changes to flufl.enum was to take out the call syntax, and have only getitem syntax - Season('AUTUMN') # raises an exception - Season['AUTUMN'] # Season.AUTUMN Not only is this inconsistent with the rest of Python*, but it's going to be a PITA for data storage/retrieval: datastore = dbf.Table('storage.dbf', 'event_name C(50); date D; season SEASON') def retrieve_record(...): result = [] for field_type, field_data in record: result.append(field_type(field_data)) vs. def retrieve_record(...): result = [] for field_type, field_data in record: if isinstance(field_type, Enum): result.append(field_type[field_data]) else: result.append(field_type(field_data)) Not being able to use call syntax on the Enum class unnecessarily complicates other code. -- ~Ethan~ * Please correct me if I'm wrong. From steve at pearwood.info Mon Apr 29 03:52:51 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 29 Apr 2013 11:52:51 +1000 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <517DBEEF.6060904@stoneleaf.us> References: <517D7944.4050107@stoneleaf.us> <517DB2C8.9090002@pearwood.info> <517DBEEF.6060904@stoneleaf.us> Message-ID: <517DD273.3020008@pearwood.info> On 29/04/13 10:29, Ethan Furman wrote: > On 04/28/2013 04:37 PM, Steven D'Aprano wrote: >>> On Sun, Apr 28, 2013 at 12:32 PM, Ethan Furman wrote: >>> >>>> - should an enum item be selectable via __call__ instead of __getitem__ >>>> (i.e. Seasons(3) is AUTUMN) >> >> Does anyone know why this is even an issue? Is this pure bike-shedding over the API, or are there >> technical reasons for choosing one over the other? > > This is an issue because currently every other type* in Python creates (or selects ;) its instances via the call syntax: > > - bool(1) # True > - int('11') # 11 > - str(var) # whatever var had in it, now as a str I think that's a red herring, because you're comparing the use of the object constructor with look-up by name. Seasons[3] should not be considered as constructing an instance from argument 3, but a reverse lookup from raw value to enum value. Hence the use of __getitem__ rather than __call__. > But one of the latest changes to flufl.enum was to take out the call syntax, and have only getitem syntax > > - Season('AUTUMN') # raises an exception > - Season['AUTUMN'] # Season.AUTUMN I'm not sure whether flufl.enums support creating additional instances after the event, but if it did, I would expect that I could say Season('WET') to get a new instance. I am indifferent to whether or not Season('AUTUMN') should return the existing AUTUMN enum value. I think that I lean very slightly to these rules: - Season(x) constructs new instances. If x is already a Season enum, it returns x; otherwise if x is a value already used for a Season enum, it raises. - Season[x] looks up existing instances, and raises if x is not already a Season value. but only very slightly. > Not only is this inconsistent with the rest of Python*, but it's going to be a PITA for data storage/retrieval: > > datastore = dbf.Table('storage.dbf', 'event_name C(50); date D; season SEASON') > > def retrieve_record(...): > result = [] > for field_type, field_data in record: > result.append(field_type(field_data)) Instead of having field_type be Season, couldn't you make it Season.__getitem__ ? -- Steven From stephen at xemacs.org Mon Apr 29 04:10:04 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 29 Apr 2013 11:10:04 +0900 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <517DB2C8.9090002@pearwood.info> References: <517D7944.4050107@stoneleaf.us> <517DB2C8.9090002@pearwood.info> Message-ID: <87a9oiw0hv.fsf@uwakimon.sk.tsukuba.ac.jp> Steven D'Aprano writes: > >> - should an enum item be selectable via __call__ instead of __getitem__ > >> (i.e. Seasons(3) is AUTUMN) > > > > No opinion. > > Does anyone know why this is even an issue? Is this pure > bike-shedding over the API, or are there technical reasons for > choosing one over the other? Ethan thinks that "Seasons(3)" is a typecast, not an access into a mapping (which would be better expressed by "Seasons[3]"). Ie, the inverse of "int(AUTUMN)". This is consistent with the "AUTUMN is-a Seasons" position that Ethan and Guido take. It's inconsistent with the "AUTUMN is-a Seasons_VALUE" implementation of Flufl.Enum. @Ethan: I have real trouble sympathizing with your point of view because you consistently pluralize your Enum names. AUTUMN *is not* a SeasonZZ, it is an element of the *collection* Seasons. OTOH, AUTUMN *is* a Season (look Ma, no ZZ!) I wonder if you might not get more sympathy from Guido if you named your Enums with the singular form. Note that for some reason I don't have the same problem if Barry names an Enum "Season" (no ZZ!) I don't know why, maybe because the semantics of type is to define a collection (which in English is invariably denoted by the plural of the type name), so there's implicitly a plural there. But I'm not confident in that psychoanalysis. From ethan at stoneleaf.us Mon Apr 29 04:57:38 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 28 Apr 2013 19:57:38 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <87a9oiw0hv.fsf@uwakimon.sk.tsukuba.ac.jp> References: <517D7944.4050107@stoneleaf.us> <517DB2C8.9090002@pearwood.info> <87a9oiw0hv.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <517DE1A2.2050006@stoneleaf.us> On 04/28/2013 07:10 PM, Stephen J. Turnbull wrote: > > @Ethan: I have real trouble sympathizing with your point of view > because you consistently pluralize your Enum names. AUTUMN *is not* a > SeasonZZ, it is an element of the *collection* Seasons. OTOH, AUTUMN > *is* a Season (look Ma, no ZZ!) I would hope that you would pay more attention to my arguments and rationale than to poorly chosen names. The way English does things is not the only way things are done. As it happens, I agree with you, and changed (and will change) all mention of Seasons to Season, and WeekDays to WeekDay, etc., etc., in my responses. -- ~Ethan~ From ethan at stoneleaf.us Mon Apr 29 04:46:59 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 28 Apr 2013 19:46:59 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <517DD273.3020008@pearwood.info> References: <517D7944.4050107@stoneleaf.us> <517DB2C8.9090002@pearwood.info> <517DBEEF.6060904@stoneleaf.us> <517DD273.3020008@pearwood.info> Message-ID: <517DDF23.3020101@stoneleaf.us> On 04/28/2013 06:52 PM, Steven D'Aprano wrote: > On 29/04/13 10:29, Ethan Furman wrote: >> On 04/28/2013 04:37 PM, Steven D'Aprano wrote: >>>> On Sun, Apr 28, 2013 at 12:32 PM, Ethan Furman wrote: >>>> >>>>> - should an enum item be selectable via __call__ instead of __getitem__ >>>>> (i.e. Season(3) is AUTUMN) >>> >>> Does anyone know why this is even an issue? Is this pure bike-shedding over the API, or are there >>> technical reasons for choosing one over the other? >> >> This is an issue because currently every other type* in Python creates (or selects ;) its instances via the call syntax: >> >> - bool(1) # True >> - int('11') # 11 >> - str(var) # whatever var had in it, now as a str > > I think that's a red herring, because you're comparing the use of the object constructor with look-up by name. int, float, and bool all have object constructors that take the given string and return a matching instance; int /may/ return a pre-existing instance, bool /will/ return a pre-existing instance. As I've stated before, 'bool's are the closest existing data type to enums, in that bools use the object constructor to convert the incoming parameter to an existing bool instance, of which there will only ever be two. bool(9) is bool('maybe') is bool(True) is True and similarly, Enum behavior /should be/ (in my opinion ;) Season.AUTUMN is Season('AUTUMN') is Season(3) Like it or not, when you write `class Season(Enum)` a new *type* is created called Season, and Season(x) should either return a Season instance that matches x or raise. Having a match and raising anyway just doesn't seem to me to be the Python Way. >> But one of the latest changes to flufl.enum was to take out the call syntax, and have only getitem syntax >> >> - Season('AUTUMN') # raises an exception >> - Season['AUTUMN'] # Season.AUTUMN > > I'm not sure whether flufl.enums support creating additional instances after the event, but if it did, I would expect > that I could say Season('WET') to get a new instance. I am indifferent to whether or not Season('AUTUMN') should return > the existing AUTUMN enum value. Adding more enumerators after the fact should not be supported; there is subclassing for that. Not worth actively preventing, but not encouraged. -- ~Ethan~ From guido at python.org Mon Apr 29 06:09:42 2013 From: guido at python.org (Guido van Rossum) Date: Sun, 28 Apr 2013 21:09:42 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: <517D7944.4050107@stoneleaf.us> <20130428232932.4651f405@fsol> Message-ID: On Sun, Apr 28, 2013 at 3:28 PM, Nick Coghlan wrote: > Functions are descriptors, so this rule already covers ordinary methods. The > slight concern I have with making the duck typed exclusion only descriptors > (rather than descriptors and callables) is that it means things like > functools.partial objects will be treated as enum values rather than as > static methods. OTOH, explicitly wrapping such callables in staticmethod > should still work, so the simpler rule is probably better. I think the practice of using callables (other than possibly decorated functions) as global variables is confusing at best, even in a non-enum class, since few people will be able to predict whether they will get the instance passed or not. So I think the rule looking only for descriptors is superior. There has been a whole lot more discussion in this thread. I am now asking everyone to stop bikeshedding, even if you think *your* point isn't bikeshedding -- the cacophony makes it impossible to hear anyone. There are a few more issues on which I'd like to pronounce. 1. The order in which iter(Color) should produce the items. The argument has been made that definition order is the only order that is not easily reconstructed, and I agree. So let's use definition order, even if that means it can't be implemented in Python 2. 2. Whether Color(x) should be allowed, and what it should mean. Taking the example of bool, it should return an existing enum if the argument is either a Color or one of the values used for Color items (choosing one somehow if there are multiple with the same value -- it doesn't matter which one), and raise an exception if the argument is neither. E.g.: class Color(Enum): red = 1 white = 2 blue = 3 x = Color.red assert Color(x) is Color.red assert Color(1) is Color.red Color(42) # raises (2a. We could also allow Color('red') is Color.red, but that could be confusing, and we can already do that with getattr(Color, 'red'), and bool('False') doesn't return False anyway, so let's not do that.) (2b. Let's not hypergeneralize and try to make bool and enums completely alike. I see no advantage to adding bool.True and bool.False, nor in making enums "final" classes, or giving a meaning to iter(bool). Type bool is its own thing, but that doesn't mean enums can't emulate aspects of it.) Together with my pronouncements earlier in this thread I am hoping that Eli and Barry will update the PEP and implementation (let's give them a week) and everyone else will quiet down. I realize we're deviating further from flufl.enum than our initial hope, but I think the outcome is better. -- --Guido van Rossum (python.org/~guido) From stephen at xemacs.org Mon Apr 29 06:42:37 2013 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 29 Apr 2013 13:42:37 +0900 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <517DE1A2.2050006@stoneleaf.us> References: <517D7944.4050107@stoneleaf.us> <517DB2C8.9090002@pearwood.info> <87a9oiw0hv.fsf@uwakimon.sk.tsukuba.ac.jp> <517DE1A2.2050006@stoneleaf.us> Message-ID: <8738uavtfm.fsf@uwakimon.sk.tsukuba.ac.jp> Ethan Furman writes: > I would hope that you would pay more attention to my arguments and > rationale than to poorly chosen names. I do. Nevertheless, it requires conscious effort. It's quite appropriate for you to ask that of me, but ... do you think you're doing Python any good to ask more effort of Guido? (Assuming he needs it, of course -- I'm just conjecturing that the form of your examples may make the whole thing less persuasive for him. But of course he has advantages, being Dutch and all.... :-) > The way English does things is not the only way things are done. I am nearly-native fluent in Japanese[1] -- I know better than most people just how differently things *can* be done. That doesn't help much with the effort gradient in English, though. Footnotes: [1] Even my 15-year-old daughter admits that, so it must be true. From ethan at stoneleaf.us Mon Apr 29 06:02:15 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 28 Apr 2013 21:02:15 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: <517D7944.4050107@stoneleaf.us> <517DB2C8.9090002@pearwood.info> <517DBEEF.6060904@stoneleaf.us> <517DD273.3020008@pearwood.info> <517DDF23.3020101@stoneleaf.us> Message-ID: <517DF0C7.7080905@stoneleaf.us> [re-directing back to python-dev] On 04/28/2013 08:42 PM, Davis Silverman wrote: > as a not super experienced python developer, when i see Season('AUTUMN') it looks like im creating an a Season object. I > understand your resoning, that it acts like a boolean singleton, however, i feel it would confuse many, and isn't worth > it. From what i see, its meant to be a lookup sort of thing, but it doesnt really feel like one. > > Or am i completely wrong about something? As far as you are concerned, you are creating a Season object. That it happens to be a pre-existing Season object is an implementation detail, and the whole thing would work just as well if it did indeed create a brand new object (you couldn't use `is` then, but in general `is` shouldn't be used anyway and I see no reason why `is` should be encouraged with Enums; use `==`). Two examples: - the first few integers (up to 256 now, I think) are pre-created by the interpreter; when you do `int('7')` you are not getting a brand-new, never before used, integer 7 object, you're getting a cached integer 7 object. - all booleans (yup, both of them ;) are pre-created; when you ask for a True or a False, you are not getting a brand new one. Since `is` is discouraged, both of those cases could go the other way (always creating a brand new object) and properly written programs would continue to run just fine -- slower, but fine. Enums are the same: they could return brand new instances every time, and programs using `==` to compare will keep on working. That they don't is an implementation detail. The real guarantee with enums is that once you have one created you'll only ever see the original values; so class Last(Enum): X = 1 Y = 2 Z = 3 will only have three possible elements, X, Y, and Z, and X will always have the value of 1, Y will always have the vale of 2, and Z will always have the value of 3. If you try and get `Last("W")` you'll trigger an exception, just like you do with `int("house")`. -- ~Ethan~ From v+python at g.nevcal.com Mon Apr 29 06:49:10 2013 From: v+python at g.nevcal.com (Glenn Linderman) Date: Sun, 28 Apr 2013 21:49:10 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: <517D7944.4050107@stoneleaf.us> <20130428232932.4651f405@fsol> Message-ID: <517DFBC6.1030203@g.nevcal.com> On 4/28/2013 9:09 PM, Guido van Rossum wrote: > (2a. We could also allow Color('red') is Color.red, but that could be > confusing, and we can already do that with getattr(Color, 'red'), and > bool('False') doesn't return False anyway, so let's not do that.) Glad you made this pronouncement in this way, because otherwise there would be ambiguity in the case: class Color(Enum): red = 'white' white = 'blue' blue = 'red' although that would no doubt drive the programmers batty anyway... but there may be instances where code is generated that could produce something ambiguous, even though this example is atrocious. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Apr 29 06:54:14 2013 From: guido at python.org (Guido van Rossum) Date: Sun, 28 Apr 2013 21:54:14 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <517DF0C7.7080905@stoneleaf.us> References: <517D7944.4050107@stoneleaf.us> <517DB2C8.9090002@pearwood.info> <517DBEEF.6060904@stoneleaf.us> <517DD273.3020008@pearwood.info> <517DDF23.3020101@stoneleaf.us> <517DF0C7.7080905@stoneleaf.us> Message-ID: On Sunday, April 28, 2013, Ethan Furman wrote: > > Enums are the same: they could return brand new instances every time, and > programs using `==` to compare will keep on working. That they don't is an > implementation detail. > Whoa. In this case the identity property is not justban implementation issue, it is part of the spec. Same for bool. (But == works too.) --Guido -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.brandl at gmx.net Mon Apr 29 07:03:27 2013 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 29 Apr 2013 07:03:27 +0200 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <517D884E.9000007@stoneleaf.us> References: <517D7944.4050107@stoneleaf.us> <517D884E.9000007@stoneleaf.us> Message-ID: Am 28.04.2013 22:36, schrieb Ethan Furman: >>> Example enumeration: >>> >>> class Seasons(Enum): SPRING = 1 SUMMER = 2 AUTUMN = 3 WINTER = 4 >>> >>> days_in_year = 365 >>> >>> @property def avg_temp(self): return (75, 92, 66, 33)[int(self)+1] # >>> enums are 1-based >>> >>> >>> Definite Issues: >>> >>> - should enum items be of the type of the Enum class? (i.e. type(SPRING) >>> is Seasons) >> >> IMO Yes. > > I agree. > >>> - should an enum item be selectable via __call__ instead of __getitem__ >>> (i.e. Seasons(3) is AUTUMN) >> >> No opinion. > > I think the callable syntax should be supported for database integration and > consistency with every (?) other type in Python. No opinion about the > __getitem__ portion. > > >>> - should days_in_year be enumerated? >> >> Yes. (If you don't want it to be, and it's not a method/descriptor, move it >> out of the class.) > > Making it a property to have it in the class certainly works for me. > > >>> - should avg_temp be enumerated? >> >> IMO No. > > I agree. > > >>> - for the above two, how should they be included/excluded? >> >> IMO Everything should be enumerated except (a) things with a __get__() >> method (i.e. descriptors) (b) __dunder__ names > > This also works for me. > > >> Also, I believe there's still an issue on the order in which items are >> returned by iter(Seasons), but I don't know which way this is heading. > > As somebody pointed out earlier, the only order which cannot be reconstructed > after the fact is definition order (value order can be, lexical order can be, > etc.). So my vote is to have the default iteration order be the original > definition order, as any other desired order can be added to the class. Thanks for the summary, that was very helpful. I find myself agreeing with every one of your opinions. Good job :) Georg From steve at pearwood.info Mon Apr 29 07:36:23 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 29 Apr 2013 15:36:23 +1000 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <517DF0C7.7080905@stoneleaf.us> References: <517D7944.4050107@stoneleaf.us> <517DB2C8.9090002@pearwood.info> <517DBEEF.6060904@stoneleaf.us> <517DD273.3020008@pearwood.info> <517DDF23.3020101@stoneleaf.us> <517DF0C7.7080905@stoneleaf.us> Message-ID: <20130429053623.GA3804@ando> On Sun, Apr 28, 2013 at 09:02:15PM -0700, Ethan Furman wrote: > Two examples: > > - the first few integers (up to 256 now, I think) are pre-created by the > interpreter; when you do `int('7')` you are not getting a brand-new, never > before used, integer 7 object, you're getting a cached integer 7 object. > > - all booleans (yup, both of them ;) are pre-created; when you ask for a > True or a False, you are not getting a brand new one. > > Since `is` is discouraged, both of those cases could go the other way > (always creating a brand new object) and properly written programs would > continue to run just fine -- slower, but fine. > > Enums are the same: they could return brand new instances every time, and > programs using `==` to compare will keep on working. That they don't is an > implementation detail. That's not how I understand it. I expected that the correct way to use enums is with identity checks: if arg is Season.SUMMER: handle_summer() At least, that's how I'm used to dealing with sentinels or pseudo-enums created with object(), and I just expected it to carry over to actual enums. Should I reset my thinking and use == with flufl.enums? -- Steven From greg.ewing at canterbury.ac.nz Mon Apr 29 08:40:16 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 29 Apr 2013 18:40:16 +1200 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <517DD273.3020008@pearwood.info> References: <517D7944.4050107@stoneleaf.us> <517DB2C8.9090002@pearwood.info> <517DBEEF.6060904@stoneleaf.us> <517DD273.3020008@pearwood.info> Message-ID: <517E15D0.6000407@canterbury.ac.nz> Steven D'Aprano wrote: > On 29/04/13 10:29, Ethan Furman wrote: > >> - bool(1) # True >> - int('11') # 11 >> - str(var) # whatever var had in it, now as a str > > I think that's a red herring, because you're comparing the use of the > object constructor with look-up by name. How does what bool() is doing differ from a lookup? It's not constructing a new instance. Neither is int() in the cases where the argument is in the range of values that it caches. More generally, the built-in types can be thought of as coercion functions -- they take an argument and return some related value from the type's repertoire. Whether they do that by constructing a new object or not is an implementation detail. -- Greg From cs at zip.com.au Mon Apr 29 08:36:15 2013 From: cs at zip.com.au (Cameron Simpson) Date: Mon, 29 Apr 2013 16:36:15 +1000 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <517DDF23.3020101@stoneleaf.us> References: <517DDF23.3020101@stoneleaf.us> Message-ID: <20130429063615.GA31496@cskk.homeip.net> On 28Apr2013 19:46, Ethan Furman wrote: | int, float, and bool all have object constructors that take the | given string and return a matching instance; int /may/ return a | pre-existing instance, bool /will/ return a pre-existing instance. I think Guido's already pointed out this: >>> bool('False') True | As I've stated before, 'bool's are the closest existing data type to | enums, in that bools use the object constructor to convert the | incoming parameter to an existing bool instance, of which there will | only ever be two. | | bool(9) is bool('maybe') is bool(True) is True | | and similarly, Enum behavior /should be/ (in my opinion ;) | | Season.AUTUMN is Season('AUTUMN') is Season(3) I think that would be _efficient_, but as an outside user I wouldn't necessarily expect it unless I'd ready the spec very closely and the spec was explicit about it. Coming from (in the fairly distant past) a C background, I naively think of enums as nicely named ordinals of some kind and expect to compare them with ==, not "is". If you want to guarantee "is", be my guest though. I don't know how that might make things go with some hypothetical subclass with many many and extensible enum values. | Like it or not, when you write `class Season(Enum)` a new *type* is | created called Season, and Season(x) should either return a Season | instance that matches x or raise. Having a match and raising anyway | just doesn't seem to me to be the Python Way. I'm +1 on all of this. [...] | >I'm not sure whether flufl.enums support creating additional instances after the event, but if it did, I would expect | >that I could say Season('WET') to get a new instance. I am indifferent to whether or not Season('AUTUMN') should return | >the existing AUTUMN enum value. | | Adding more enumerators after the fact should not be supported; | there is subclassing for that. Not worth actively preventing, but | not encouraged. I'd go a bit further here: I'd take this final sentence as being -0 on preventing adding more enumerals(?), whereas I'm a solid -1 on preventing it. By all means not actively support it, but very against doing things that make it hard for a subclass to support it. Just 2c, -- Would you remember a one-line .sig? - Paul Thompson, thompson at apple.com From solipsis at pitrou.net Mon Apr 29 08:54:16 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 29 Apr 2013 08:54:16 +0200 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? References: <517D7944.4050107@stoneleaf.us> <517DB2C8.9090002@pearwood.info> <517DBEEF.6060904@stoneleaf.us> Message-ID: <20130429085416.79cf2fe6@fsol> On Sun, 28 Apr 2013 17:29:35 -0700 Ethan Furman wrote: > > Not only is this inconsistent with the rest of Python*, but it's going to be a PITA for data storage/retrieval: > > datastore = dbf.Table('storage.dbf', 'event_name C(50); date D; season SEASON') > > def retrieve_record(...): > result = [] > for field_type, field_data in record: > result.append(field_type(field_data)) I've never seen any kind of "data retrieval" which works like that. Would you care to explain us the context? Regards Antoine. From ethan at stoneleaf.us Mon Apr 29 08:50:16 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 28 Apr 2013 23:50:16 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: <517D7944.4050107@stoneleaf.us> <517DB2C8.9090002@pearwood.info> <517DBEEF.6060904@stoneleaf.us> <517DD273.3020008@pearwood.info> <517DDF23.3020101@stoneleaf.us> <517DF0C7.7080905@stoneleaf.us> Message-ID: <517E1828.1080802@stoneleaf.us> On 04/28/2013 09:54 PM, Guido van Rossum wrote: > On Sunday, April 28, 2013, Ethan Furman wrote: > >> Enums are the same: they could return brand new instances every time, and programs using `==` to compare will keep >> on working. That they don't is an implementation detail. > > Whoa. In this case the identity property is not just an implementation issue, it is part of the spec. Same for bool. > (But == works too.) I realize that, and I have no problems with the singleton (multiton?) approach; however, part of the spec is that a subclass shares the enum items, and if the enum items are actual instances of the type that will no longer be correct. In other words, currently: class Color(Enum): red = 1 green = 2 blue = 3 class MoreColor(Color): cyan = 4 magenta = 5 yellow = 6 black = 7 MoreColor.red is Color.red # True But as soon as: type(Color.red) is Color # True type(MoreColor.red) is MoreColor # True then: Color.red is MoreColor.red # must be False, no? If that last statement can still be True, I'd love it if someone showed me how. -- ~Ethan~ P.S. Apologies for the additional post. From ethan at stoneleaf.us Mon Apr 29 09:13:53 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 29 Apr 2013 00:13:53 -0700 Subject: [Python-Dev] Enums and data retrieval Message-ID: <517E1DB1.4020200@stoneleaf.us> [starting new thread to not pollute the summary thread] On 04/28/2013 11:54 PM, Antoine Pitrou wrote:> On Sun, 28 Apr 2013 17:29:35 -0700 > Ethan Furman wrote: >> >> Not only is this inconsistent with the rest of Python*, but it's going to be a PITA for data storage/retrieval: >> >> datastore = dbf.Table('storage.dbf', 'event_name C(50); date D; season SEASON') >> >> def retrieve_record(...): >> result = [] >> for field_type, field_data in record: >> result.append(field_type(field_data)) > > I've never seen any kind of "data retrieval" which works like that. > Would you care to explain us the context? The more specific context would be my dbf package, which works with dBase III, Clipper, and Foxpro tables. When the fields of a record are requested they are transformed into Python data types, with code that looks pretty much like that retrieve_record snippet (w/o all the error checks, etc.). And no, it doesn't support enumerations (yet). A more general context would be anywhere that you need to convert the integer offset of an enum item back into the enum item itself; you may have gotten the integer offset from a postgres database, or an RPC call, and it seems to me the natural way get the enum item from that is with `EnumClass(offset)`. -- ~Ethan~ From greg.ewing at canterbury.ac.nz Mon Apr 29 09:42:04 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 29 Apr 2013 19:42:04 +1200 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <20130429063615.GA31496@cskk.homeip.net> References: <517DDF23.3020101@stoneleaf.us> <20130429063615.GA31496@cskk.homeip.net> Message-ID: <517E244C.1080207@canterbury.ac.nz> Cameron Simpson wrote: > I'd go a bit further here: I'd take this final sentence as being > -0 on preventing adding more enumerals(?), whereas I'm a solid -1 > on preventing it. By all means not actively support it, but very > against doing things that make it hard for a subclass to support > it. I had another thought about subclassing. I don't think it would be a good idea to totally forbid subclassing classes derived from Enum, because you may want to define an Enum subclass for the purpose of creating a new *kind* of enum. For example, class FancyEnum(Enum): def fancy_str(self): return str(self) + " with bells on" If subclassing a subclass of Enum were prevented somehow, you would't be able to create any actual enums based on FancyEnum. -- Greg From greg.ewing at canterbury.ac.nz Mon Apr 29 09:42:02 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 29 Apr 2013 19:42:02 +1200 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: <517D7944.4050107@stoneleaf.us> <20130428232932.4651f405@fsol> Message-ID: <517E244A.2000500@canterbury.ac.nz> Guido van Rossum wrote: > (2a. We could also allow Color('red') is Color.red, but that could be > confusing, and we can already do that with getattr(Color, 'red'), That doesn't quite give you the same thing. Presumably Color('__str__') would be expected to raise a ValueError, for example. -- Greg From steve at pearwood.info Mon Apr 29 10:30:13 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 29 Apr 2013 18:30:13 +1000 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <517E1828.1080802@stoneleaf.us> References: <517D7944.4050107@stoneleaf.us> <517DB2C8.9090002@pearwood.info> <517DBEEF.6060904@stoneleaf.us> <517DD273.3020008@pearwood.info> <517DDF23.3020101@stoneleaf.us> <517DF0C7.7080905@stoneleaf.us> <517E1828.1080802@stoneleaf.us> Message-ID: <20130429083012.GB3804@ando> On Sun, Apr 28, 2013 at 11:50:16PM -0700, Ethan Furman wrote: > In other words, currently: > > class Color(Enum): > red = 1 > green = 2 > blue = 3 > > class MoreColor(Color): > cyan = 4 > magenta = 5 > yellow = 6 > black = 7 > > MoreColor.red is Color.red # True Correct. > But as soon as: > > type(Color.red) is Color # True > type(MoreColor.red) is MoreColor # True I don't believe this is correct. As I understand it, the proposal is the weaker guarantee: isinstance(Color.red, Color) # True, possibly using __instancecheck__ Since red is not defined by MoreColor, there's no reason to expect that it will be a MoreColor instance. As far as I understand it, there is no guarantee that isinstance(MoreColor.red, MoreColor) will be true, even if that is possible using __instancecheck__. -- Steven From solipsis at pitrou.net Mon Apr 29 10:55:54 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 29 Apr 2013 10:55:54 +0200 Subject: [Python-Dev] Enums and data retrieval References: <517E1DB1.4020200@stoneleaf.us> Message-ID: <20130429105554.4df50403@pitrou.net> Le Mon, 29 Apr 2013 00:13:53 -0700, Ethan Furman a ?crit : > [starting new thread to not pollute the summary thread] > > On 04/28/2013 11:54 PM, Antoine Pitrou wrote:> On Sun, 28 Apr 2013 > 17:29:35 -0700 > > Ethan Furman wrote: > >> > >> Not only is this inconsistent with the rest of Python*, but it's > >> going to be a PITA for data storage/retrieval: > >> > >> datastore = dbf.Table('storage.dbf', 'event_name C(50); date > >> D; season SEASON') > >> > >> def retrieve_record(...): > >> result = [] > >> for field_type, field_data in record: > >> result.append(field_type(field_data)) > > > > I've never seen any kind of "data retrieval" which works like that. > > Would you care to explain us the context? > > The more specific context would be my dbf package, which works with > dBase III, Clipper, and Foxpro tables. When the fields of a record > are requested they are transformed into Python data types, with code > that looks pretty much like that retrieve_record snippet (w/o all the > error checks, etc.). And no, it doesn't support enumerations (yet). Hmm, ok. So the context is the database adapter itself, right? I wouldn't be shocked for a database adapter to have specific code to handle various datatypes. My point was that this kind of code generally doesn't leak into application code. That said, I agree that the general constructor syntax should be allowed on Enum instances. The inconsistency looks a bit gratuitous. Regards Antoine. From bkabrda at redhat.com Mon Apr 29 11:27:36 2013 From: bkabrda at redhat.com (Bohuslav Kabrda) Date: Mon, 29 Apr 2013 05:27:36 -0400 (EDT) Subject: [Python-Dev] Purpose of files in $(DESTDIR)$(LIBPL) In-Reply-To: <935786590.3353924.1367227261200.JavaMail.root@redhat.com> Message-ID: <1100748677.3355198.1367227656892.JavaMail.root@redhat.com> Hi, I'd like to ask about the purpose of files in $(DESTDIR)$(LIBPL) [1] - what is the reason to keep them/what are they useful for? I'm currently "taking over" Python packaging in Fedora and I'd like to know if these have some meaning for a distro-packaged Python (Dave Malcolm is not sure about them ;)). Thanks, Slavek. -- Regards, Bohuslav "Slavek" Kabrda. [1] http://hg.python.org/cpython/file/3.3/Makefile.pre.in#l1206 From eliben at gmail.com Mon Apr 29 14:24:59 2013 From: eliben at gmail.com (Eli Bendersky) Date: Mon, 29 Apr 2013 05:24:59 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <517D7944.4050107@stoneleaf.us> References: <517D7944.4050107@stoneleaf.us> Message-ID: On Sun, Apr 28, 2013 at 12:32 PM, Ethan Furman wrote: > Example enumeration: > > class Seasons(Enum): > SPRING = 1 > SUMMER = 2 > AUTUMN = 3 > WINTER = 4 > > days_in_year = 365 > > @property > def avg_temp(self): > return (75, 92, 66, 33)[int(self)+1] # enums are 1-based > > > Definite Issues: > > - should enum items be of the type of the Enum class? (i.e. type(SPRING) > is Seasons) > > - should an enum item be selectable via __call__ instead of __getitem__ > (i.e. Seasons(3) is AUTUMN) > > - should days_in_year be enumerated? > > - should avg_temp be enumerated? > > - for the above two, how should they be included/excluded? > Thanks for the summary. One issue I don't see addressed here is int-compatibility. Am I correct to assume that nothing changes w.r.t. that, and that an IntEnum subclass of Enum will be provided which is isinstance(integer)? Does that become straightforward by nature of enum values being the type of their enumerations? Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From eliben at gmail.com Mon Apr 29 14:41:32 2013 From: eliben at gmail.com (Eli Bendersky) Date: Mon, 29 Apr 2013 05:41:32 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: <517D7944.4050107@stoneleaf.us> <20130428232932.4651f405@fsol> Message-ID: On Sun, Apr 28, 2013 at 9:09 PM, Guido van Rossum wrote: > On Sun, Apr 28, 2013 at 3:28 PM, Nick Coghlan wrote: > > Functions are descriptors, so this rule already covers ordinary methods. > The > > slight concern I have with making the duck typed exclusion only > descriptors > > (rather than descriptors and callables) is that it means things like > > functools.partial objects will be treated as enum values rather than as > > static methods. OTOH, explicitly wrapping such callables in staticmethod > > should still work, so the simpler rule is probably better. > > I think the practice of using callables (other than possibly decorated > functions) as global variables is confusing at best, even in a > non-enum class, since few people will be able to predict whether they > will get the instance passed or not. So I think the rule looking only > for descriptors is superior. > > There has been a whole lot more discussion in this thread. I am now > asking everyone to stop bikeshedding, even if you think *your* point > isn't bikeshedding -- the cacophony makes it impossible to hear > anyone. > > There are a few more issues on which I'd like to pronounce. > > 1. The order in which iter(Color) should produce the items. The > argument has been made that definition order is the only order that is > not easily reconstructed, and I agree. So let's use definition order, > even if that means it can't be implemented in Python 2. > > 2. Whether Color(x) should be allowed, and what it should mean. Taking > the example of bool, it should return an existing enum if the argument > is either a Color or one of the values used for Color items (choosing > one somehow if there are multiple with the same value -- it doesn't > matter which one), and raise an exception if the argument is neither. > E.g.: > > class Color(Enum): > red = 1 > white = 2 > blue = 3 > > x = Color.red > assert Color(x) is Color.red > assert Color(1) is Color.red > Color(42) # raises > > (2a. We could also allow Color('red') is Color.red, but that could be > confusing, and we can already do that with getattr(Color, 'red'), and > bool('False') doesn't return False anyway, so let's not do that.) > > (2b. Let's not hypergeneralize and try to make bool and enums > completely alike. I see no advantage to adding bool.True and > bool.False, nor in making enums "final" classes, or giving a meaning > to iter(bool). Type bool is its own thing, but that doesn't mean enums > can't emulate aspects of it.) > > Together with my pronouncements earlier in this thread I am hoping > that Eli and Barry will update the PEP and implementation (let's give > them a week) and everyone else will quiet down. I realize we're > deviating further from flufl.enum than our initial hope, but I think > the outcome is better. > Sounds like a plan. Thanks for trying to converge things, Guido. Personally I don't have strong opinions about the changes inflicted by recent threads, but I may stumble into something as I dive deeper. I was away for a couple of days and a lot has changed - I'll need some time to catch up, but updating the PEP by the end of the week seems reasonable. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From eliben at gmail.com Mon Apr 29 15:51:59 2013 From: eliben at gmail.com (Eli Bendersky) Date: Mon, 29 Apr 2013 06:51:59 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: <517D7944.4050107@stoneleaf.us> <20130428232932.4651f405@fsol> Message-ID: On Sun, Apr 28, 2013 at 9:09 PM, Guido van Rossum wrote: > On Sun, Apr 28, 2013 at 3:28 PM, Nick Coghlan wrote: > > Functions are descriptors, so this rule already covers ordinary methods. > The > > slight concern I have with making the duck typed exclusion only > descriptors > > (rather than descriptors and callables) is that it means things like > > functools.partial objects will be treated as enum values rather than as > > static methods. OTOH, explicitly wrapping such callables in staticmethod > > should still work, so the simpler rule is probably better. > > I think the practice of using callables (other than possibly decorated > functions) as global variables is confusing at best, even in a > non-enum class, since few people will be able to predict whether they > will get the instance passed or not. So I think the rule looking only > for descriptors is superior. > > There has been a whole lot more discussion in this thread. I am now > asking everyone to stop bikeshedding, even if you think *your* point > isn't bikeshedding -- the cacophony makes it impossible to hear > anyone. > > There are a few more issues on which I'd like to pronounce. > > 1. The order in which iter(Color) should produce the items. The > argument has been made that definition order is the only order that is > not easily reconstructed, and I agree. So let's use definition order, > even if that means it can't be implemented in Python 2. > > 2. Whether Color(x) should be allowed, and what it should mean. Taking > the example of bool, it should return an existing enum if the argument > is either a Color or one of the values used for Color items (choosing > one somehow if there are multiple with the same value -- it doesn't > matter which one), and raise an exception if the argument is neither. > I don't feel strongly about allowing ()-lookup in addition to []-lookup, but in this paragraph the issue of multiple definitions has sneaked in :-) flufl.enum disallows this: class Color(Enum): red = 1 blue = 2 green = 1 # oops! Has it been decided that this is now allowed? If this is indeed the case, then Color(1) is a problem. The options are: A. Return either Color.red or Color.green B. Throwing an error Do we have a decision on this? Personally I think the latter is better; the former is error prone and doesn't seem to be useful too often. Eli [trying to tie loose ends for updating the PEP]. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Mon Apr 29 15:32:58 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 29 Apr 2013 06:32:58 -0700 Subject: [Python-Dev] Enumeration items: `type(EnumClass.item) is EnumClass` ? Message-ID: <517E768A.9090602@stoneleaf.us> [creating new thread] On 04/29/2013 01:30 AM, Steven D'Aprano wrote: > On Sun, Apr 28, 2013 at 11:50:16PM -0700, Ethan Furman wrote: > >> In other words, currently: >> >> class Color(Enum): >> red = 1 >> green = 2 >> blue = 3 >> >> class MoreColor(Color): >> cyan = 4 >> magenta = 5 >> yellow = 6 >> black = 7 >> >> MoreColor.red is Color.red # True > > Correct. > > >> But as soon as: >> >> type(Color.red) is Color # True >> type(MoreColor.red) is MoreColor # True > > I don't believe this is correct. As I understand it, the proposal is the > weaker guarantee: > > isinstance(Color.red, Color) # True, possibly using __instancecheck__ Words from Guido: On 04/23/2013 08:11 AM, Guido van Rossum wrote: > I gotta say, I'm with Antoine here. It's pretty natural (also coming > from other languages) to assume that the class used to define the > enums is also the type of the enum values. Certainly this is how it > works in Java and C++, and I would say it's the same in Pascal and > probably most other languages. > On 04/25/2013 02:54 PM, Guido van Rossum wrote: > I don't know what's going on, but it feels like we had this same > discussion a week ago, and I still disagree. Disregarding, the C[i] > notation, I feel quite strongly that in the following example: > > class Color(Enum): > red = 1 > white = 2 > blue = 3 > orange = 4 > > the values Color.red etc. should be instances of Color. This is how > things work in all other languages that I am aware of that let you > define enums. On 04/25/2013 03:19 PM, Guido van Rossum wrote: > I suppose you were going to propose to use isinstance() overloading, > but I honestly think that Color.red.__class__ should be the same > object as Color. On 04/25/2013 03:37 PM, Guido van Rossum wrote: > TBH I had a hard time getting over the fact that even though the class > said "a = 1", C.a is not the integer 1. But I did get over it. > Hopefully you can get over *this* weirdness. [and from the summary thread] On 04/28/2013 01:02 PM, Guido van Rossum wrote: > On Sun, Apr 28, 2013 at 12:32 PM, Ethan Furman wrote: >> >> - should enum items be of the type of the Enum class? (i.e. type(SPRING) >> is Seasons) > > IMO Yes. From arigo at tunes.org Mon Apr 29 16:42:38 2013 From: arigo at tunes.org (Armin Rigo) Date: Mon, 29 Apr 2013 16:42:38 +0200 Subject: [Python-Dev] Destructors and Closing of File Objects In-Reply-To: <87y5c4r91p.fsf@vostro.rath.org> References: <87a9p41gr6.fsf@vostro.rath.org> <877gk35aqx.fsf@vostro.rath.org> <87y5c4r91p.fsf@vostro.rath.org> Message-ID: Hi Nikolaus, On Sat, Apr 27, 2013 at 4:39 AM, Nikolaus Rath wrote: > It's indeed very informative, but it doesn't fully address the question > because of the _pyio module which certainly can't use any custom C code. > Does that mean that when I'm using x = _pyio.BufferedWriter(), I could loose > data in the write buffer when the interpreter exits without me calling > x.close(), but when using x = io.BufferedWriter(), the buffer is > guaranteed to get flushed? I actually described the behavior of CPython 2 while not realizing that CPython 3 silently dropped this guarantee. (I also never realized that Jython/IronPython don't have the same guarantee; they could, if they implement 'atexit', like we did in PyPy. That's however more acceptable if the platform itself doesn't offer the guarantee.) Anyway, it's a guarantee that the C offers, so personally I find it reasonable to expect CPython files to offer it too; not offering it is kind of saying that there is a feature of C that is actually present at a higher level than the exact same feature in Python, which looks backward to me. Additionally, this might be introducing subtle bugs in programs when porting them to Python 3. However I realize that the two arguments presented above might not be accepted as relevant. (http://bugs.python.org/issue17852) A bient?t, Armin. From asampson at cs.washington.edu Mon Apr 29 17:04:29 2013 From: asampson at cs.washington.edu (Adrian Sampson) Date: Mon, 29 Apr 2013 08:04:29 -0700 Subject: [Python-Dev] Enumeration items: `type(EnumClass.item) is EnumClass` ? Message-ID: <6EF0F6A3-5180-4FCE-9753-FD2EF2151742@cs.washington.edu> On Apr 29, 2013, at 7:32 AM, Ethan Furman wrote: > On 04/28/2013 01:02 PM, Guido van Rossum wrote: >> On Sun, Apr 28, 2013 at 12:32 PM, Ethan Furman wrote: >> - should enum items be of the type of the Enum class? (i.e. type(SPRING) >> is Seasons) > > IMO Yes. This decision seems natural to me, so I wrote an enumeration library some time ago that uses a simple metaclass to achieve "type(Season.spring) is Season": https://github.com/sampsyo/beets/blob/master/beets/util/enumeration.py The module has other warts but perhaps it can be helpful anyway. Cheers, Adrian From guido at python.org Mon Apr 29 17:39:41 2013 From: guido at python.org (Guido van Rossum) Date: Mon, 29 Apr 2013 08:39:41 -0700 Subject: [Python-Dev] Enumeration items: `type(EnumClass.item) is EnumClass` ? In-Reply-To: <517E768A.9090602@stoneleaf.us> References: <517E768A.9090602@stoneleaf.us> Message-ID: Indeed, the "type(Color.red) is Color" claim was meant for the situation where red is defined directly in Color, and I used type() instead of isinstance() because Barry was proposing to overload isinstance() to make this true without equating the classes. But for the subclass case, I want MoreColor.red is Color.red and isinstance(MoreColor.red, Color), but not isinstance(Color.red, MoreColor). If you can't live with that, don't subclass enums. On Mon, Apr 29, 2013 at 6:32 AM, Ethan Furman wrote: > [creating new thread] > > On 04/29/2013 01:30 AM, Steven D'Aprano wrote: >> >> On Sun, Apr 28, 2013 at 11:50:16PM -0700, Ethan Furman wrote: >> >>> In other words, currently: >>> >>> class Color(Enum): >>> red = 1 >>> green = 2 >>> blue = 3 >>> >>> class MoreColor(Color): >>> cyan = 4 >>> magenta = 5 >>> yellow = 6 >>> black = 7 >>> >>> MoreColor.red is Color.red # True >> >> >> Correct. >> >> >>> But as soon as: >>> >>> type(Color.red) is Color # True >>> type(MoreColor.red) is MoreColor # True >> >> >> I don't believe this is correct. As I understand it, the proposal is the >> weaker guarantee: >> >> isinstance(Color.red, Color) # True, possibly using __instancecheck__ > > > > Words from Guido: > > On 04/23/2013 08:11 AM, Guido van Rossum wrote: >> >> I gotta say, I'm with Antoine here. It's pretty natural (also coming >> from other languages) to assume that the class used to define the >> enums is also the type of the enum values. Certainly this is how it >> works in Java and C++, and I would say it's the same in Pascal and >> probably most other languages. >> > > On 04/25/2013 02:54 PM, Guido van Rossum wrote: >> >> I don't know what's going on, but it feels like we had this same >> discussion a week ago, and I still disagree. Disregarding, the C[i] >> notation, I feel quite strongly that in the following example: >> >> class Color(Enum): >> red = 1 >> white = 2 >> blue = 3 >> orange = 4 >> >> the values Color.red etc. should be instances of Color. This is how >> things work in all other languages that I am aware of that let you >> define enums. > > > On 04/25/2013 03:19 PM, Guido van Rossum wrote: >> >> I suppose you were going to propose to use isinstance() overloading, >> but I honestly think that Color.red.__class__ should be the same >> object as Color. > > > On 04/25/2013 03:37 PM, Guido van Rossum wrote: >> >> TBH I had a hard time getting over the fact that even though the class >> said "a = 1", C.a is not the integer 1. But I did get over it. >> Hopefully you can get over *this* weirdness. > > > [and from the summary thread] > > On 04/28/2013 01:02 PM, Guido van Rossum wrote: >> >> On Sun, Apr 28, 2013 at 12:32 PM, Ethan Furman wrote: >>> >>> >>> - should enum items be of the type of the Enum class? (i.e. >>> type(SPRING) >>> is Seasons) >> >> >> IMO Yes. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) From solipsis at pitrou.net Mon Apr 29 18:02:23 2013 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 29 Apr 2013 18:02:23 +0200 Subject: [Python-Dev] Destructors and Closing of File Objects References: <87a9p41gr6.fsf@vostro.rath.org> <877gk35aqx.fsf@vostro.rath.org> <87y5c4r91p.fsf@vostro.rath.org> Message-ID: <20130429180223.2a4a3028@pitrou.net> Le Mon, 29 Apr 2013 16:42:38 +0200, Armin Rigo a ?crit : > Hi Nikolaus, > > On Sat, Apr 27, 2013 at 4:39 AM, Nikolaus Rath > wrote: > > It's indeed very informative, but it doesn't fully address the > > question because of the _pyio module which certainly can't use any > > custom C code. Does that mean that when I'm using x = > > _pyio.BufferedWriter(), I could loose data in the write buffer when > > the interpreter exits without me calling x.close(), but when using > > x = io.BufferedWriter(), the buffer is guaranteed to get flushed? > > I actually described the behavior of CPython 2 while not realizing > that CPython 3 silently dropped this guarantee. It is dropped in the case of reference cycles, since there's no general way to decide in which order the tp_clear calls have to be done. Thus in the following layered situation: a TextIOWrapper on top of a BufferedWriter on top of a FileIO, if BufferedWriter.tp_clear is called first, it will flush and then close itself, closing the FileIO at the same time, and when TextIOWrapper.tp_clear will be called it will be too late to flush its own buffer. (I have to investigate a bit to confirm it is what happens) I will try to think of a scheme to make flushing more reliable, but nothing springs to my mind right now. Note that the issue of how reference cycles involving globals are collected at interpreter shutdown is an orthogonal one, as pointed out in http://bugs.python.org/issue17852. Regards Antoine. From larry at hastings.org Mon Apr 29 18:30:35 2013 From: larry at hastings.org (Larry Hastings) Date: Mon, 29 Apr 2013 09:30:35 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: <517D7944.4050107@stoneleaf.us> <20130428232932.4651f405@fsol> Message-ID: <517EA02B.2000409@hastings.org> On 04/29/2013 06:51 AM, Eli Bendersky wrote: > flufl.enum disallows this: > > class Color(Enum): > red = 1 > blue = 2 > green = 1 # oops! > > Has it been decided that this is now allowed? If this is indeed the > case, then Color(1) is a problem. The options are: > > A. Return either Color.red or Color.green > B. Throwing an error > > Do we have a decision on this? Personally I think the latter is > better; the former is error prone and doesn't seem to be useful too often. > > Eli [trying to tie loose ends for updating the PEP]. Would you disallow this too? class Color(Enum): red = 1 blue = 2 Color.green = Color.red IMO the more Pythonic behavior would be to allow multiple names for the same value, and your example should produce the same result as my example. I'm not convinced enums are special enough to break the rules. See also "consenting adults", //arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Apr 29 18:35:19 2013 From: guido at python.org (Guido van Rossum) Date: Mon, 29 Apr 2013 09:35:19 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: <517D7944.4050107@stoneleaf.us> Message-ID: On Mon, Apr 29, 2013 at 5:24 AM, Eli Bendersky wrote: > Thanks for the summary. One issue I don't see addressed here is > int-compatibility. Am I correct to assume that nothing changes w.r.t. that, > and that an IntEnum subclass of Enum will be provided which is > isinstance(integer)? Does that become straightforward by nature of enum > values being the type of their enumerations? Correct, we'll still need an IntEnum subclass of Enum. I have no idea how this works out in the implementation, sorry. -- --Guido van Rossum (python.org/~guido) From ethan at stoneleaf.us Mon Apr 29 18:12:57 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 29 Apr 2013 09:12:57 -0700 Subject: [Python-Dev] Enumeration items: `type(EnumClass.item) is EnumClass` ? In-Reply-To: References: <517E768A.9090602@stoneleaf.us> Message-ID: <517E9C09.3010806@stoneleaf.us> On 04/29/2013 08:39 AM, Guido van Rossum wrote: > Indeed, the "type(Color.red) is Color" claim was meant for the > situation where red is defined directly in Color, and I used type() > instead of isinstance() because Barry was proposing to overload > isinstance() to make this true without equating the classes. But for > the subclass case, I want MoreColor.red is Color.red and > isinstance(MoreColor.red, Color), but not isinstance(Color.red, > MoreColor). If you can't live with that, don't subclass enums. So if I understand: --> class Color(Enum): ... red = 1 ... green = 2 ... blue = 3 --> class MoreColor(Color): ... cyan = 4 ... magenta = 5 ... yellow = 6 --> type(MoreColor.red) is Color --> type(MoreColor.red) is not MoreColor In other words, while `red` is accessible in MoreColor, it's actually a Color instance? -- ~Ethan~ From guido at python.org Mon Apr 29 18:42:00 2013 From: guido at python.org (Guido van Rossum) Date: Mon, 29 Apr 2013 09:42:00 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: <517D7944.4050107@stoneleaf.us> <20130428232932.4651f405@fsol> Message-ID: On Mon, Apr 29, 2013 at 6:51 AM, Eli Bendersky wrote: > I don't feel strongly about allowing ()-lookup in addition to []-lookup, but > in this paragraph the issue of multiple definitions has sneaked in :-) > flufl.enum disallows this: > > class Color(Enum): > red = 1 > blue = 2 > green = 1 # oops! > > Has it been decided that this is now allowed? I don't recall if it was decided. I think it should be possible to create aliases like this. The main thing I care about is that Color.green == Color.red. I wouldn't mind if Color.green ends up as just a different name for the Color.red object. The most common use case is probably providing backwards compatibility with an old version of the enum when renaming something -- e.g. one could write class Color(Enum): ... turquoise = 42 aqua = turquoise # I can't tell them apart Here the metaclass would see two different names with the same value (42). > If this is indeed the case, then Color(1) is a problem. The options are: > > A. Return either Color.red or Color.green > B. Throwing an error > > Do we have a decision on this? Personally I think the latter is better; the > former is error prone and doesn't seem to be useful too often. I think it should be A, and the choice should be the first one in definition order. -- --Guido van Rossum (python.org/~guido) From guido at python.org Mon Apr 29 19:01:44 2013 From: guido at python.org (Guido van Rossum) Date: Mon, 29 Apr 2013 10:01:44 -0700 Subject: [Python-Dev] Enumeration items: `type(EnumClass.item) is EnumClass` ? In-Reply-To: <517E9C09.3010806@stoneleaf.us> References: <517E768A.9090602@stoneleaf.us> <517E9C09.3010806@stoneleaf.us> Message-ID: On Mon, Apr 29, 2013 at 9:12 AM, Ethan Furman wrote: > On 04/29/2013 08:39 AM, Guido van Rossum wrote: >> >> Indeed, the "type(Color.red) is Color" claim was meant for the >> situation where red is defined directly in Color, and I used type() >> instead of isinstance() because Barry was proposing to overload >> isinstance() to make this true without equating the classes. But for >> the subclass case, I want MoreColor.red is Color.red and >> isinstance(MoreColor.red, Color), but not isinstance(Color.red, >> MoreColor). If you can't live with that, don't subclass enums. > > > So if I understand: > > --> class Color(Enum): > ... red = 1 > ... green = 2 > ... blue = 3 > > --> class MoreColor(Color): > ... cyan = 4 > ... magenta = 5 > ... yellow = 6 > > --> type(MoreColor.red) is Color > > --> type(MoreColor.red) is not MoreColor > > In other words, while `red` is accessible in MoreColor, it's actually a > Color instance? Oh dear, this is actually a mess. I don't want MoreColor.red and Color.red to be distinct objects, but then the isinstance() checks will become confusing. If we don't override isinstance(), we'll get not isinstance(Color.red, MoreColor) isinstance(MoreColor.yellow, Color) This would be pretty backwards. I Ggoogled "enum subclassing" and found this StackOverflow article explaining why you can't subclass enums in Java: http://stackoverflow.com/questions/4604978/subclassing-an-enum which refers to this more elaborate answer: http://stackoverflow.com/questions/3427947/enumeration-inheritence-in-java/3428050#3428050 I think this conclusively shows that it's better to disallow subclassing enums. (It also brings enum in line with bool, which is also a "final" class.) Adding Eli to the thread explicitly, because this needs to be explained in the PEP. -- --Guido van Rossum (python.org/~guido) From ethan at stoneleaf.us Mon Apr 29 18:47:43 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 29 Apr 2013 09:47:43 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <517EA02B.2000409@hastings.org> References: <517D7944.4050107@stoneleaf.us> <20130428232932.4651f405@fsol> <517EA02B.2000409@hastings.org> Message-ID: <517EA42F.7040501@stoneleaf.us> On 04/29/2013 09:30 AM, Larry Hastings wrote: > > On 04/29/2013 06:51 AM, Eli Bendersky wrote: >> flufl.enum disallows this: >> >> class Color(Enum): >> red = 1 >> blue = 2 >> green = 1 # oops! >> >> Has it been decided that this is now allowed? If this is indeed the case, then Color(1) is a problem. The options are: At this point I think the best course is to not allow duplicates directly in the enum definition, but allow them after the fact: --> class Color(Enum): ... red = 1 ... green = 2 ... blue = 3 --> Color.grene = Color.green # stupid legacy typo This gives us both some protection against accidental duplicates, while still allowing them when necessary; also, no confusion about which is the canonical name. Guido? -- ~Ethan~ From steve at pearwood.info Mon Apr 29 19:35:22 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 30 Apr 2013 03:35:22 +1000 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: <517D7944.4050107@stoneleaf.us> <20130428232932.4651f405@fsol> Message-ID: <517EAF5A.8000606@pearwood.info> On 30/04/13 02:42, Guido van Rossum wrote: > On Mon, Apr 29, 2013 at 6:51 AM, Eli Bendersky wrote: >> I don't feel strongly about allowing ()-lookup in addition to []-lookup, but >> in this paragraph the issue of multiple definitions has sneaked in :-) >> flufl.enum disallows this: >> >> class Color(Enum): >> red = 1 >> blue = 2 >> green = 1 # oops! >> >> Has it been decided that this is now allowed? > > I don't recall if it was decided. I think it should be possible to > create aliases like this. The main thing I care about is that > Color.green == Color.red. I believe that Barry had decided that it should be prohibited. I objected, and Nick pointed out that although declaring two enums with the same value inside the class is prohibited, aliases are supported by adding them from the outside: class Color(Enum): red = 1 blue = 2 Color.green = Color.red which satisfies me. -- Steven From guido at python.org Mon Apr 29 19:38:46 2013 From: guido at python.org (Guido van Rossum) Date: Mon, 29 Apr 2013 10:38:46 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <517EA42F.7040501@stoneleaf.us> References: <517D7944.4050107@stoneleaf.us> <20130428232932.4651f405@fsol> <517EA02B.2000409@hastings.org> <517EA42F.7040501@stoneleaf.us> Message-ID: On Mon, Apr 29, 2013 at 9:47 AM, Ethan Furman wrote: > At this point I think the best course is to not allow duplicates directly in > the enum definition, but allow them after the fact: > > --> class Color(Enum): > ... red = 1 > ... green = 2 > ... blue = 3 > > --> Color.grene = Color.green # stupid legacy typo > > This gives us both some protection against accidental duplicates, while > still allowing them when necessary; also, no confusion about which is the > canonical name. No. Poking a class has a code smell. It should be possible to define the aliases inside the enum class definition. The canonical one is the first one in definition order. -- --Guido van Rossum (python.org/~guido) From larry at hastings.org Mon Apr 29 19:58:45 2013 From: larry at hastings.org (Larry Hastings) Date: Mon, 29 Apr 2013 10:58:45 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: <517EAF5A.8000606@pearwood.info> References: <517D7944.4050107@stoneleaf.us> <20130428232932.4651f405@fsol> <517EAF5A.8000606@pearwood.info> Message-ID: <517EB4D5.9030508@hastings.org> On 04/29/2013 10:35 AM, Steven D'Aprano wrote: > On 30/04/13 02:42, Guido van Rossum wrote: >> On Mon, Apr 29, 2013 at 6:51 AM, Eli Bendersky wrote: >>> I don't feel strongly about allowing ()-lookup in addition to >>> []-lookup, but >>> in this paragraph the issue of multiple definitions has sneaked in :-) >>> flufl.enum disallows this: >>> >>> class Color(Enum): >>> red = 1 >>> blue = 2 >>> green = 1 # oops! >>> >>> Has it been decided that this is now allowed? >> >> I don't recall if it was decided. I think it should be possible to >> create aliases like this. The main thing I care about is that >> Color.green == Color.red. > > I believe that Barry had decided that it should be prohibited. I > objected, and Nick pointed out that although declaring two enums with > the same value inside the class is prohibited, aliases are supported > by adding them from the outside: > > class Color(Enum): > red = 1 > blue = 2 > > Color.green = Color.red > > which satisfies me. Assuming that Color(1) always returns the same object, then we could also write this: class Color(Enum): red = 1 blue = 2 Color.green = Color(1) Which should be identical to class Color(Enum): red = 1 blue = 2 green = 1 To declare that my first example is okay but the second is not strikes me as awfully special. And I do mean awful. //arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Mon Apr 29 20:14:45 2013 From: v+python at g.nevcal.com (Glenn Linderman) Date: Mon, 29 Apr 2013 11:14:45 -0700 Subject: [Python-Dev] Enumeration items: `type(EnumClass.item) is EnumClass` ? In-Reply-To: References: <517E768A.9090602@stoneleaf.us> <517E9C09.3010806@stoneleaf.us> Message-ID: <517EB895.1060602@g.nevcal.com> On 4/29/2013 10:01 AM, Guido van Rossum wrote: > On Mon, Apr 29, 2013 at 9:12 AM, Ethan Furman wrote: >> On 04/29/2013 08:39 AM, Guido van Rossum wrote: >>> Indeed, the "type(Color.red) is Color" claim was meant for the >>> situation where red is defined directly in Color, and I used type() >>> instead of isinstance() because Barry was proposing to overload >>> isinstance() to make this true without equating the classes. But for >>> the subclass case, I want MoreColor.red is Color.red and >>> isinstance(MoreColor.red, Color), but not isinstance(Color.red, >>> MoreColor). If you can't live with that, don't subclass enums. >> >> So if I understand: >> >> --> class Color(Enum): >> ... red = 1 >> ... green = 2 >> ... blue = 3 >> >> --> class MoreColor(Color): >> ... cyan = 4 >> ... magenta = 5 >> ... yellow = 6 >> >> --> type(MoreColor.red) is Color >> >> --> type(MoreColor.red) is not MoreColor >> >> In other words, while `red` is accessible in MoreColor, it's actually a >> Color instance? > Oh dear, this is actually a mess. I don't want MoreColor.red and > Color.red to be distinct objects, but then the isinstance() checks > will become confusing. If we don't override isinstance(), we'll get > > not isinstance(Color.red, MoreColor) > isinstance(MoreColor.yellow, Color) > > This would be pretty backwards. > > I Ggoogled "enum subclassing" and found this StackOverflow article > explaining why you can't subclass enums in Java: > http://stackoverflow.com/questions/4604978/subclassing-an-enum which > refers to this more elaborate answer: > http://stackoverflow.com/questions/3427947/enumeration-inheritence-in-java/3428050#3428050 > > I think this conclusively shows that it's better to disallow > subclassing enums. (It also brings enum in line with bool, which is > also a "final" class.) > > Adding Eli to the thread explicitly, because this needs to be > explained in the PEP. The only thing that needs to be disallowed is defining additional named elements, for Liskov Substitution to not be violated. So in an inheritance tree derived from Enum, it would be a requirement that all members of the enumeration be enumerated in one class. It would be extremely nice if: 1) Enum could be subclassed to provide different, sharable, types of behaviors, then further subclassed to provide a number of distinct sets of values with those behaviors. 2) Enum could be subclassed to provide one set of values, and then further subclassed to provide a number a distinct sets of behaviors for those sets of values. (I see a lot more benefit to #1, than #2, if a choice must be made.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Mon Apr 29 20:24:59 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 30 Apr 2013 04:24:59 +1000 Subject: [Python-Dev] Enumeration items: `type(EnumClass.item) is EnumClass` ? In-Reply-To: References: <517E768A.9090602@stoneleaf.us> <517E9C09.3010806@stoneleaf.us> Message-ID: <517EBAFB.3060109@pearwood.info> On 30/04/13 03:01, Guido van Rossum wrote: > Oh dear, this is actually a mess. I don't want MoreColor.red and > Color.red to be distinct objects, but then the isinstance() checks > will become confusing. If we don't override isinstance(), we'll get > > not isinstance(Color.red, MoreColor) > isinstance(MoreColor.yellow, Color) > > This would be pretty backwards. Why is that backwards? MoreColor is a subclass of Color, so instances of MoreColor are instances of Color, but instances of Color are not instances of MoreColor. That's normal behaviour for subclasses. (All cats are mammals, but not all mammals are cats.) The only "funny" thing about this is that enumeration values of a Enum are only instances of that Enum if they are declared inside that Enum, and as far as I'm concerned that's not especially funny at all. So: Color declares red, so Color.red is a Color. MoreColor does not declare red, it merely inherits it from Color, so MoreColor.red is Color.red which is not a MoreColor. This leads to a simple rule: Enum values are either instances of the enum they are defined in, or instances of the parent class they are inherited from. If the user doesn't like that, they're free to not subclass Enums :-) > I Ggoogled "enum subclassing" and found this StackOverflow article > explaining why you can't subclass enums in Java: > http://stackoverflow.com/questions/4604978/subclassing-an-enum which > refers to this more elaborate answer: > http://stackoverflow.com/questions/3427947/enumeration-inheritence-in-java/3428050#3428050 The first link says: "And generally, a proper subclass is a specialization of its superclass." and gives the Liskov Substitution Principle as the reason for prohibiting subclassing. LSP is a very useful principle, but it is not the only model for subclassing. That's why it's called a *principle* and not the Liskov Substitution *Law*. (Yes, I stole that from Raymond Hettinger's talk on subclassing at PyCon.) I think that the ability to extend enums with new ones, without duplicating code, is more important for enums than satisfying LSP. The common use-cases for enums do not lend itself to subtyping in the Liskov sense. E.g. if I have a function that expects a Color red/blue/green enum, and I subclass it to give MoreColor yellow, I can't expect the function to suddenly recognise yellow. So I'm not going to use subclassing in this case, because it can't work. But I will use subclassing to reuse values: if I have a function that expects red/blue/green/yellow, and red/blue/green are already defined in Color, I'll want to reuse them rather than duplicate them. Hence I will subclass Color specifically to extend it, not to specialise it. The second link quotes: "For the most part, extensibility of enums turns out to be a bad idea. It is confusing that elements of an extension type are instances of the base type and not vice versa." Confusing for who, and why? You're going to confuse *somebody* no matter what you do. Either: - confuse people who try to subclass enums, and can't; or - confuse people who try to subclass enums, and can, but then get confused by the result. -- Steven From guido at python.org Mon Apr 29 20:29:22 2013 From: guido at python.org (Guido van Rossum) Date: Mon, 29 Apr 2013 11:29:22 -0700 Subject: [Python-Dev] Enumeration items: `type(EnumClass.item) is EnumClass` ? In-Reply-To: <517EBAFB.3060109@pearwood.info> References: <517E768A.9090602@stoneleaf.us> <517E9C09.3010806@stoneleaf.us> <517EBAFB.3060109@pearwood.info> Message-ID: You are too verbose. I have already said what I needed to say. On Mon, Apr 29, 2013 at 11:24 AM, Steven D'Aprano wrote: > On 30/04/13 03:01, Guido van Rossum wrote: > >> Oh dear, this is actually a mess. I don't want MoreColor.red and >> Color.red to be distinct objects, but then the isinstance() checks >> will become confusing. If we don't override isinstance(), we'll get >> >> not isinstance(Color.red, MoreColor) >> isinstance(MoreColor.yellow, Color) >> >> This would be pretty backwards. > > > Why is that backwards? MoreColor is a subclass of Color, so instances of > MoreColor are instances of Color, but instances of Color are not instances > of MoreColor. That's normal behaviour for subclasses. (All cats are mammals, > but not all mammals are cats.) > > The only "funny" thing about this is that enumeration values of a Enum are > only instances of that Enum if they are declared inside that Enum, and as > far as I'm concerned that's not especially funny at all. So: > > Color declares red, so Color.red is a Color. > > MoreColor does not declare red, it merely inherits it from Color, so > MoreColor.red is Color.red which is not a MoreColor. > > > This leads to a simple rule: > > Enum values are either instances of the enum they are defined in, or > instances of the parent class they are inherited from. > > If the user doesn't like that, they're free to not subclass Enums :-) > > > > >> I Ggoogled "enum subclassing" and found this StackOverflow article >> explaining why you can't subclass enums in Java: >> http://stackoverflow.com/questions/4604978/subclassing-an-enum which >> refers to this more elaborate answer: >> >> http://stackoverflow.com/questions/3427947/enumeration-inheritence-in-java/3428050#3428050 > > > The first link says: > > "And generally, a proper subclass is a specialization of its superclass." > > and gives the Liskov Substitution Principle as the reason for prohibiting > subclassing. LSP is a very useful principle, but it is not the only model > for subclassing. That's why it's called a *principle* and not the Liskov > Substitution *Law*. > > (Yes, I stole that from Raymond Hettinger's talk on subclassing at PyCon.) > > I think that the ability to extend enums with new ones, without duplicating > code, is more important for enums than satisfying LSP. The common use-cases > for enums do not lend itself to subtyping in the Liskov sense. > > E.g. if I have a function that expects a Color red/blue/green enum, and I > subclass it to give MoreColor yellow, I can't expect the function to > suddenly recognise yellow. So I'm not going to use subclassing in this case, > because it can't work. > > But I will use subclassing to reuse values: if I have a function that > expects red/blue/green/yellow, and red/blue/green are already defined in > Color, I'll want to reuse them rather than duplicate them. Hence I will > subclass Color specifically to extend it, not to specialise it. > > > The second link quotes: > > "For the most part, extensibility of enums turns out to be a bad idea. It is > confusing that elements of an extension type are instances of the base type > and not vice versa." > > Confusing for who, and why? > > You're going to confuse *somebody* no matter what you do. Either: > > - confuse people who try to subclass enums, and can't; or > > - confuse people who try to subclass enums, and can, but then get confused > by the result. > > > > > > -- > Steven > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) From larry at hastings.org Mon Apr 29 20:34:59 2013 From: larry at hastings.org (Larry Hastings) Date: Mon, 29 Apr 2013 11:34:59 -0700 Subject: [Python-Dev] Enumeration items: `type(EnumClass.item) is EnumClass` ? In-Reply-To: References: <517E768A.9090602@stoneleaf.us> <517E9C09.3010806@stoneleaf.us> Message-ID: <517EBD53.4000708@hastings.org> On 04/29/2013 10:01 AM, Guido van Rossum wrote: > On Mon, Apr 29, 2013 at 9:12 AM, Ethan Furman wrote: >> On 04/29/2013 08:39 AM, Guido van Rossum wrote: >>> Indeed, the "type(Color.red) is Color" claim was meant for the >>> situation where red is defined directly in Color, and I used type() >>> instead of isinstance() because Barry was proposing to overload >>> isinstance() to make this true without equating the classes. But for >>> the subclass case, I want MoreColor.red is Color.red and >>> isinstance(MoreColor.red, Color), but not isinstance(Color.red, >>> MoreColor). If you can't live with that, don't subclass enums. >> >> So if I understand: >> >> --> class Color(Enum): >> ... red = 1 >> ... green = 2 >> ... blue = 3 >> >> --> class MoreColor(Color): >> ... cyan = 4 >> ... magenta = 5 >> ... yellow = 6 >> >> --> type(MoreColor.red) is Color >> >> --> type(MoreColor.red) is not MoreColor >> >> In other words, while `red` is accessible in MoreColor, it's actually a >> Color instance? > Oh dear, this is actually a mess. I don't want MoreColor.red and > Color.red to be distinct objects, but then the isinstance() checks > will become confusing. If we don't override isinstance(), we'll get > > not isinstance(Color.red, MoreColor) > isinstance(MoreColor.yellow, Color) > > This would be pretty backwards. What's the problem with overriding the isinstance checks? You mention it but seem to assume it's a bad idea. That seems to me like it'd adequately solve that problem with an acceptable level of magic. //arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Mon Apr 29 21:02:50 2013 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 29 Apr 2013 20:02:50 +0100 Subject: [Python-Dev] Enumeration items: `type(EnumClass.item) is EnumClass` ? In-Reply-To: <517EBAFB.3060109@pearwood.info> References: <517E768A.9090602@stoneleaf.us> <517E9C09.3010806@stoneleaf.us> <517EBAFB.3060109@pearwood.info> Message-ID: <517EC3DA.9090105@mrabarnett.plus.com> On 29/04/2013 19:24, Steven D'Aprano wrote: > On 30/04/13 03:01, Guido van Rossum wrote: > >> Oh dear, this is actually a mess. I don't want MoreColor.red and >> Color.red to be distinct objects, but then the isinstance() checks >> will become confusing. If we don't override isinstance(), we'll >> get >> >> not isinstance(Color.red, MoreColor) isinstance(MoreColor.yellow, >> Color) >> >> This would be pretty backwards. > > Why is that backwards? MoreColor is a subclass of Color, so instances > of MoreColor are instances of Color, but instances of Color are not > instances of MoreColor. That's normal behaviour for subclasses. (All > cats are mammals, but not all mammals are cats.) > Let's say that Color is red, green, or blue. Let's also say that MoreColor is superset of Color, in other words, any of Color, plus cyan, magenta, or yellow. Red is a Color and a MoreColor (member of Color and MoreColor). Yellow is a MoreColor, but not a Color (member of MoreColor but not Color). That's the opposite of the rules of inheritance. > The only "funny" thing about this is that enumeration values of a > Enum are only instances of that Enum if they are declared inside that > Enum, and as far as I'm concerned that's not especially funny at all. > So: > > Color declares red, so Color.red is a Color. > > MoreColor does not declare red, it merely inherits it from Color, so > MoreColor.red is Color.red which is not a MoreColor. > > > This leads to a simple rule: > > Enum values are either instances of the enum they are defined in, or > instances of the parent class they are inherited from. > > If the user doesn't like that, they're free to not subclass Enums > :-) > > > >> I Ggoogled "enum subclassing" and found this StackOverflow article >> explaining why you can't subclass enums in Java: >> http://stackoverflow.com/questions/4604978/subclassing-an-enum >> which refers to this more elaborate answer: >> http://stackoverflow.com/questions/3427947/enumeration-inheritence-in-java/3428050#3428050 > >> > The first link says: > > "And generally, a proper subclass is a specialization of its > superclass." > > and gives the Liskov Substitution Principle as the reason for > prohibiting subclassing. LSP is a very useful principle, but it is > not the only model for subclassing. That's why it's called a > *principle* and not the Liskov Substitution *Law*. > > (Yes, I stole that from Raymond Hettinger's talk on subclassing at > PyCon.) > > I think that the ability to extend enums with new ones, without > duplicating code, is more important for enums than satisfying LSP. > The common use-cases for enums do not lend itself to subtyping in the > Liskov sense. > > E.g. if I have a function that expects a Color red/blue/green enum, > and I subclass it to give MoreColor yellow, I can't expect the > function to suddenly recognise yellow. So I'm not going to use > subclassing in this case, because it can't work. > > But I will use subclassing to reuse values: if I have a function that > expects red/blue/green/yellow, and red/blue/green are already defined > in Color, I'll want to reuse them rather than duplicate them. Hence I > will subclass Color specifically to extend it, not to specialise it. > On the other hand, a function that expects MoreColor should also accept Color, which is a subset. > > The second link quotes: > > "For the most part, extensibility of enums turns out to be a bad > idea. It is confusing that elements of an extension type are > instances of the base type and not vice versa." > > Confusing for who, and why? > > You're going to confuse *somebody* no matter what you do. Either: > > - confuse people who try to subclass enums, and can't; or > > - confuse people who try to subclass enums, and can, but then get > confused by the result. > From guido at python.org Mon Apr 29 21:04:35 2013 From: guido at python.org (Guido van Rossum) Date: Mon, 29 Apr 2013 12:04:35 -0700 Subject: [Python-Dev] Enumeration items: `type(EnumClass.item) is EnumClass` ? In-Reply-To: <517EBD53.4000708@hastings.org> References: <517E768A.9090602@stoneleaf.us> <517E9C09.3010806@stoneleaf.us> <517EBD53.4000708@hastings.org> Message-ID: On Mon, Apr 29, 2013 at 11:34 AM, Larry Hastings wrote: > What's the problem with overriding the isinstance checks? You mention it > but seem to assume it's a bad idea. That seems to me like it'd adequately > solve that problem with an acceptable level of magic. Depending on whether you are using isinstance() to check if an enum value belongs in the set of acceptable enums, or to check it it makes sense to call a certain method, you need isinstance() to behave differently. In particular, the default isinstance() would correctly state that MoreColor.red is not a MoreColor instance, so any methods defined only on MoreColor should not be called. OTOH it gives the wrong answer if you are checking that Color.red is an acceptable MoreColor value. But if you override isinstance() to give the right answer for the latter (so isinstance(Color.red, MoreColor) is True), then you could incorrectly conclude that it is safe to call a MoreColor method on Color.red. Please do read the StackOverflow links I gave: http://stackoverflow.com/questions/4604978/subclassing-an-enum http://stackoverflow.com/questions/3427947/enumeration-inheritence-in-java/3428050#3428050 -- --Guido van Rossum (python.org/~guido) From storchaka at gmail.com Mon Apr 29 21:15:38 2013 From: storchaka at gmail.com (Serhiy Storchaka) Date: Mon, 29 Apr 2013 22:15:38 +0300 Subject: [Python-Dev] Enumeration items: `type(EnumClass.item) is EnumClass` ? In-Reply-To: <517EB895.1060602@g.nevcal.com> References: <517E768A.9090602@stoneleaf.us> <517E9C09.3010806@stoneleaf.us> <517EB895.1060602@g.nevcal.com> Message-ID: 29.04.13 21:14, Glenn Linderman ???????(??): > 1) Enum could be subclassed to provide different, sharable, types of > behaviors, then further subclassed to provide a number of distinct sets > of values with those behaviors. You can use a multiclass inheritance for this. > 2) Enum could be subclassed to provide one set of values, and then > further subclassed to provide a number a distinct sets of behaviors for > those sets of values. How is it possible? You haven't any instance of subclass. From steve at pearwood.info Mon Apr 29 21:20:38 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 30 Apr 2013 05:20:38 +1000 Subject: [Python-Dev] Enumeration items: `type(EnumClass.item) is EnumClass` ? In-Reply-To: References: <517E768A.9090602@stoneleaf.us> <517E9C09.3010806@stoneleaf.us> <517EBAFB.3060109@pearwood.info> Message-ID: <517EC806.3090806@pearwood.info> On 30/04/13 04:29, Guido van Rossum wrote: > You are too verbose. I have already said what I needed to say. Sorry about that, sometimes I do go on and on. Let me be more terse. When it comes to enums, I believe that violating Liskov is a feature, not a bug. Also, I understand that both Scala and Kotlin allow subclassing enums in exactly the way we're talking about. -- Steven From oscar.j.benjamin at gmail.com Mon Apr 29 21:52:47 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 29 Apr 2013 20:52:47 +0100 Subject: [Python-Dev] Enumeration items: `type(EnumClass.item) is EnumClass` ? In-Reply-To: References: <517E768A.9090602@stoneleaf.us> <517E9C09.3010806@stoneleaf.us> <517EBD53.4000708@hastings.org> Message-ID: On 29 April 2013 20:04, Guido van Rossum wrote: > On Mon, Apr 29, 2013 at 11:34 AM, Larry Hastings wrote: >> What's the problem with overriding the isinstance checks? You mention it >> but seem to assume it's a bad idea. That seems to me like it'd adequately >> solve that problem with an acceptable level of magic. > > Depending on whether you are using isinstance() to check if an enum > value belongs in the set of acceptable enums, or to check it it makes > sense to call a certain method, you need isinstance() to behave > differently. Would it not be better to avoid using isinstance() for this? I would have expected membership testing to use "Red in Colors" rather than "isinstance(Red, Color)" like in this stackoverflow question about an enum class: http://stackoverflow.com/questions/10445819/overriding-contains-method-for-a-class Oscar From steve at pearwood.info Mon Apr 29 22:00:57 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 30 Apr 2013 06:00:57 +1000 Subject: [Python-Dev] Enumeration items: `type(EnumClass.item) is EnumClass` ? In-Reply-To: <517EC3DA.9090105@mrabarnett.plus.com> References: <517E768A.9090602@stoneleaf.us> <517E9C09.3010806@stoneleaf.us> <517EBAFB.3060109@pearwood.info> <517EC3DA.9090105@mrabarnett.plus.com> Message-ID: <517ED179.2080100@pearwood.info> On 30/04/13 05:02, MRAB wrote: >> Why is that backwards? MoreColor is a subclass of Color, so instances >> of MoreColor are instances of Color, but instances of Color are not >> instances of MoreColor. That's normal behaviour for subclasses. (All >> cats are mammals, but not all mammals are cats.) >> > Let's say that Color is red, green, or blue. > > Let's also say that MoreColor is superset of Color, in other words, any > of Color, plus cyan, magenta, or yellow. > > Red is a Color and a MoreColor (member of Color and MoreColor). > > Yellow is a MoreColor, but not a Color (member of MoreColor but not > Color). > > That's the opposite of the rules of inheritance. Membership != inheritance, you are conflating two independent concepts. I would expect that for membership testing, you should say "value in MoreColor", not isinstance(value, MoreColor). flufl.enum has been in use for Mailman for many years, and I would like to hear Barry's opinion on this. -- Steven From guido at python.org Mon Apr 29 22:00:41 2013 From: guido at python.org (Guido van Rossum) Date: Mon, 29 Apr 2013 13:00:41 -0700 Subject: [Python-Dev] Enumeration items: `type(EnumClass.item) is EnumClass` ? In-Reply-To: References: <517E768A.9090602@stoneleaf.us> <517E9C09.3010806@stoneleaf.us> <517EBD53.4000708@hastings.org> Message-ID: On Mon, Apr 29, 2013 at 12:52 PM, Oscar Benjamin wrote: > Would it not be better to avoid using isinstance() for this? In a recent thread I explained why using isinstance() is important. It's how we check for every other type, and it would be awlward for type-checking frameworks to have to special-case enums. (When we're not using duck typing.) -- --Guido van Rossum (python.org/~guido) From tjreedy at udel.edu Mon Apr 29 23:45:30 2013 From: tjreedy at udel.edu (Terry Jan Reedy) Date: Mon, 29 Apr 2013 17:45:30 -0400 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: <517D7944.4050107@stoneleaf.us> Message-ID: On 4/29/2013 8:24 AM, Eli Bendersky wrote: > Thanks for the summary. One issue I don't see addressed here is > int-compatibility. Am I correct to assume that nothing changes w.r.t. > that, and that an IntEnum subclass of Enum will be provided which is > isinstance(integer)? Does that become straightforward by nature of enum > values being the type of their enumerations? My only concern is that whatever you do does not break transitivity of ==. Aside from that, I will be happy with whatever you end up with and use it as appropriate. From nad at acm.org Mon Apr 29 23:50:55 2013 From: nad at acm.org (Ned Deily) Date: Mon, 29 Apr 2013 14:50:55 -0700 Subject: [Python-Dev] Purpose of files in $(DESTDIR)$(LIBPL) References: <935786590.3353924.1367227261200.JavaMail.root@redhat.com> <1100748677.3355198.1367227656892.JavaMail.root@redhat.com> Message-ID: In article <1100748677.3355198.1367227656892.JavaMail.root at redhat.com>, Bohuslav Kabrda wrote: > I'd like to ask about the purpose of files in $(DESTDIR)$(LIBPL) [1] - what > is the reason to keep them/what are they useful for? > I'm currently "taking over" Python packaging in Fedora and I'd like to know > if these have some meaning for a distro-packaged Python (Dave Malcolm is not > sure about them ;)). As is noted a bit further up in Makefile.pre.in: 1178 # Install the library and miscellaneous stuff needed for extending/embedding 1179 # This goes into $(exec_prefix) 1180 LIBPL= $(LIBDEST)/config-$(LDVERSION) As I understand it, LIBPL is the directory that contains the development files needed for embedding Python in C, things like the static and shared libpythox.x and the Makefile itself. They are intended to be referenced through the pythonx.y-config command. For example, on Debian, LIBPL is /usr/lib/python2.7/config. $ python2.7-config --ldflags -L/usr/lib/python2.7/config -lpthread -ldl -lutil -lm -lpython2.7 -Xlinker -export-dynamic The usage is documented here: http://docs.python.org/dev/extending/embedding.html#compiling-and-linking -under-unix-like-systems -- Ned Deily, nad at acm.org From ethan at stoneleaf.us Mon Apr 29 23:59:25 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 29 Apr 2013 14:59:25 -0700 Subject: [Python-Dev] Enumeration item arguments? Message-ID: <517EED3D.70705@stoneleaf.us> In the Planet example we saw the possibility of specifying arguments to enum item __init__: class Planet(Enum): MERCURY = (3.303e+23, 2.4397e6) VENUS = (4.869e+24, 6.0518e6) EARTH = (5.976e+24, 6.37814e6) MARS = (6.421e+23, 3.3972e6) JUPITER = (1.9e+27, 7.1492e7) SATURN = (5.688e+26, 6.0268e7) URANUS = (8.686e+25, 2.5559e7) NEPTUNE = (1.024e+26, 2.4746e7) def __init__(self, mass, radius): self.mass = mass # in kilograms self.radius = radius # in meters Do we want to support this? -- ~Ethan~ From storchaka at gmail.com Tue Apr 30 00:05:31 2013 From: storchaka at gmail.com (Serhiy Storchaka) Date: Tue, 30 Apr 2013 01:05:31 +0300 Subject: [Python-Dev] Enumeration item arguments? In-Reply-To: <517EED3D.70705@stoneleaf.us> References: <517EED3D.70705@stoneleaf.us> Message-ID: 30.04.13 00:59, Ethan Furman ???????(??): > In the Planet example we saw the possibility of specifying arguments to > enum item __init__: > > class Planet(Enum): > MERCURY = (3.303e+23, 2.4397e6) > VENUS = (4.869e+24, 6.0518e6) > EARTH = (5.976e+24, 6.37814e6) > MARS = (6.421e+23, 3.3972e6) > JUPITER = (1.9e+27, 7.1492e7) > SATURN = (5.688e+26, 6.0268e7) > URANUS = (8.686e+25, 2.5559e7) > NEPTUNE = (1.024e+26, 2.4746e7) > > def __init__(self, mass, radius): > self.mass = mass # in kilograms > self.radius = radius # in meters It should have different signature as Larry proposed: def __init__(self, value): self.mass, self.radius = *value From guido at python.org Tue Apr 30 00:13:13 2013 From: guido at python.org (Guido van Rossum) Date: Mon, 29 Apr 2013 15:13:13 -0700 Subject: [Python-Dev] Enumeration item arguments? In-Reply-To: <517EED3D.70705@stoneleaf.us> References: <517EED3D.70705@stoneleaf.us> Message-ID: Only if it is easy to implement. This example doesn't look like a good fit for enums, but if the enum implementation can easily support this (e.g. if there's nothing special about __init__) I don't want to forcibly rule it out. I don't want to have to bend over backwards to support it, however, if it causes problems for the implementation. On Mon, Apr 29, 2013 at 2:59 PM, Ethan Furman wrote: > In the Planet example we saw the possibility of specifying arguments to enum > item __init__: > > class Planet(Enum): > MERCURY = (3.303e+23, 2.4397e6) > VENUS = (4.869e+24, 6.0518e6) > EARTH = (5.976e+24, 6.37814e6) > MARS = (6.421e+23, 3.3972e6) > JUPITER = (1.9e+27, 7.1492e7) > SATURN = (5.688e+26, 6.0268e7) > URANUS = (8.686e+25, 2.5559e7) > NEPTUNE = (1.024e+26, 2.4746e7) > > def __init__(self, mass, radius): > self.mass = mass # in kilograms > self.radius = radius # in meters > > Do we want to support this? > > -- > ~Ethan~ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) From ethan at stoneleaf.us Mon Apr 29 23:53:58 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 29 Apr 2013 14:53:58 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: <517D7944.4050107@stoneleaf.us> Message-ID: <517EEBF6.7050204@stoneleaf.us> On 04/29/2013 02:45 PM, Terry Jan Reedy wrote: > On 4/29/2013 8:24 AM, Eli Bendersky wrote: > >> Thanks for the summary. One issue I don't see addressed here is >> int-compatibility. Am I correct to assume that nothing changes w.r.t. >> that, and that an IntEnum subclass of Enum will be provided which is >> isinstance(integer)? Does that become straightforward by nature of enum >> values being the type of their enumerations? > > My only concern is that whatever you do does not break transitivity of ==. Aside from that, I will be happy with > whatever you end up with and use it as appropriate. It will not be broken. -- ~Ethan~ From guido at python.org Tue Apr 30 00:18:59 2013 From: guido at python.org (Guido van Rossum) Date: Mon, 29 Apr 2013 15:18:59 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: <517D7944.4050107@stoneleaf.us> Message-ID: On Mon, Apr 29, 2013 at 2:45 PM, Terry Jan Reedy wrote: > On 4/29/2013 8:24 AM, Eli Bendersky wrote: > >> Thanks for the summary. One issue I don't see addressed here is >> int-compatibility. Am I correct to assume that nothing changes w.r.t. >> that, and that an IntEnum subclass of Enum will be provided which is >> isinstance(integer)? Does that become straightforward by nature of enum >> values being the type of their enumerations? > > > My only concern is that whatever you do does not break transitivity of ==. > Aside from that, I will be happy with whatever you end up with and use it as > appropriate. This is a trick question though, isn't it? Example: class Color(Enum): red = 1 white = 2 blue = 3 class State(Enum): idle = 0 busy = 1 We would have State.busy == 1 and Color.red == 1, so now State.busy == Color.red. -- --Guido van Rossum (python.org/~guido) From eliben at gmail.com Tue Apr 30 00:23:00 2013 From: eliben at gmail.com (Eli Bendersky) Date: Mon, 29 Apr 2013 15:23:00 -0700 Subject: [Python-Dev] enum discussion: can someone please summarize open issues? In-Reply-To: References: <517D7944.4050107@stoneleaf.us> Message-ID: On Mon, Apr 29, 2013 at 2:45 PM, Terry Jan Reedy wrote: > On 4/29/2013 8:24 AM, Eli Bendersky wrote: > > Thanks for the summary. One issue I don't see addressed here is >> int-compatibility. Am I correct to assume that nothing changes w.r.t. >> that, and that an IntEnum subclass of Enum will be provided which is >> isinstance(integer)? Does that become straightforward by nature of enum >> values being the type of their enumerations? >> > > My only concern is that whatever you do does not break transitivity of ==. > Aside from that, I will be happy with whatever you end up with and use it > as appropriate. > The transitivity of == is not broken by either proposal. In PEP 435, the split to Enum and IntEnum is made in part for this reason. Enum values don't compare equal even if they have the same underlying values. IntEnum values do compare equal to ints, and hence to each other. So: class Color(Enum): red = 1 blue = 2 class Animal(Enum): dog = 2 sheep = 7 Color.blue != Animal.dog, and neither compares to 2, of course. However, had both enums been IntEnum, we'd have: Color.blue == Animal.dog == 2 This is a werdness users of IntEnum have to live with, and that's why it's explicitly discouraged for 99% of the use-cases. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From eliben at gmail.com Tue Apr 30 00:25:48 2013 From: eliben at gmail.com (Eli Bendersky) Date: Mon, 29 Apr 2013 15:25:48 -0700 Subject: [Python-Dev] Enumeration item arguments? In-Reply-To: <517EED3D.70705@stoneleaf.us> References: <517EED3D.70705@stoneleaf.us> Message-ID: On Mon, Apr 29, 2013 at 2:59 PM, Ethan Furman wrote: > In the Planet example we saw the possibility of specifying arguments to > enum item __init__: > > class Planet(Enum): > MERCURY = (3.303e+23, 2.4397e6) > VENUS = (4.869e+24, 6.0518e6) > EARTH = (5.976e+24, 6.37814e6) > MARS = (6.421e+23, 3.3972e6) > JUPITER = (1.9e+27, 7.1492e7) > SATURN = (5.688e+26, 6.0268e7) > URANUS = (8.686e+25, 2.5559e7) > NEPTUNE = (1.024e+26, 2.4746e7) > > def __init__(self, mass, radius): > self.mass = mass # in kilograms > self.radius = radius # in meters > > Do we want to support this? > I'm -1, and this is yet another bad sign of conflating enums with classes. If planets want to have attributes and behaviors, let them be normal classes. If they want a PlanetId *enum member*, that's OK, but there's no need to intermix the two. Besides, did we not agree that the only acceptable *members* for enums are going to be descriptors? In the above, mass & radius are not descriptors. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Tue Apr 30 00:33:25 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 29 Apr 2013 15:33:25 -0700 Subject: [Python-Dev] Enumeration item arguments? In-Reply-To: References: <517EED3D.70705@stoneleaf.us> Message-ID: <517EF535.1030001@stoneleaf.us> On 04/29/2013 03:25 PM, Eli Bendersky wrote: > On Mon, Apr 29, 2013 at 2:59 PM, Ethan Furman wrote: >> >> In the Planet example we saw the possibility of specifying arguments to enum item __init__: >> >> class Planet(Enum): >> MERCURY = (3.303e+23, 2.4397e6) >> VENUS = (4.869e+24, 6.0518e6) >> EARTH = (5.976e+24, 6.37814e6) >> MARS = (6.421e+23, 3.3972e6) >> JUPITER = (1.9e+27, 7.1492e7) >> SATURN = (5.688e+26, 6.0268e7) >> URANUS = (8.686e+25, 2.5559e7) >> NEPTUNE = (1.024e+26, 2.4746e7) >> >> def __init__(self, mass, radius): >> self.mass = mass # in kilograms >> self.radius = radius # in meters >> >> Do we want to support this? > > I'm -1, and this is yet another bad sign of conflating enums with classes. If planets want to have attributes and > behaviors, let them be normal classes. If they want a PlanetId *enum member*, that's OK, but there's no need to intermix > the two. > > Besides, did we not agree that the only acceptable *members* for enums are going to be descriptors? In the above, mass & > radius are not descriptors. Good point. We can leave that out, then. -- ~Ethan~ From ethan at stoneleaf.us Tue Apr 30 00:18:50 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 29 Apr 2013 15:18:50 -0700 Subject: [Python-Dev] Enumeration item arguments? In-Reply-To: References: <517EED3D.70705@stoneleaf.us> Message-ID: <517EF1CA.4050105@stoneleaf.us> On 04/29/2013 03:13 PM, Guido van Rossum wrote: > Only if it is easy to implement. This example doesn't look like a good > fit for enums, but if the enum implementation can easily support this > (e.g. if there's nothing special about __init__) I don't want to > forcibly rule it out. I don't want to have to bend over backwards to > support it, however, if it causes problems for the implementation. Beautiful answer. :) -- ~Ethan~ From ethan at stoneleaf.us Tue Apr 30 00:50:22 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 29 Apr 2013 15:50:22 -0700 Subject: [Python-Dev] Enumeration items: mixed types? Message-ID: <517EF92E.5050906@stoneleaf.us> This just doesn't make sense to me: --> class Stuff(Enum): ... blue = 1 ... china = 'really big country' ... random = (8273.199, 517) --> Stuff.blue.name == 'blue' --> Stuff.blue.value == 1 --> Stuff.china.name == 'china' --> Stuff.china.value == ??? --> Stuff.random.name == 'random' --> Stuff.china.value == ??? In order to make this work at all, we have to support auto-numbering, and I didn't think we were going to do that in the class syntax? -- ~Ethan~ P.S. Apologies for all the questions, I'm just hoping to get the last details hammered out so we can stop discussing Enums. ;) From larry at hastings.org Tue Apr 30 01:12:14 2013 From: larry at hastings.org (Larry Hastings) Date: Mon, 29 Apr 2013 16:12:14 -0700 Subject: [Python-Dev] Enumeration item arguments? In-Reply-To: <517EF535.1030001@stoneleaf.us> References: <517EED3D.70705@stoneleaf.us> <517EF535.1030001@stoneleaf.us> Message-ID: <517EFE4E.5050100@hastings.org> On 04/29/2013 03:33 PM, Ethan Furman wrote: > On 04/29/2013 03:25 PM, Eli Bendersky wrote: >> On Mon, Apr 29, 2013 at 2:59 PM, Ethan Furman wrote: >>> >>> In the Planet example we saw the possibility of specifying arguments >>> to enum item __init__: >>> >>> class Planet(Enum): >>> MERCURY = (3.303e+23, 2.4397e6) >>> VENUS = (4.869e+24, 6.0518e6) >>> EARTH = (5.976e+24, 6.37814e6) >>> MARS = (6.421e+23, 3.3972e6) >>> JUPITER = (1.9e+27, 7.1492e7) >>> SATURN = (5.688e+26, 6.0268e7) >>> URANUS = (8.686e+25, 2.5559e7) >>> NEPTUNE = (1.024e+26, 2.4746e7) >>> >>> def __init__(self, mass, radius): >>> self.mass = mass # in kilograms >>> self.radius = radius # in meters >>> >>> Do we want to support this? >> >> Besides, did we not agree that the only acceptable *members* for >> enums are going to be descriptors? In the above, mass & >> radius are not descriptors. > Good point. We can leave that out, then. self.__mass = mass And surely we were discussing members of the class, not members of instances. //arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Tue Apr 30 01:22:49 2013 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 30 Apr 2013 00:22:49 +0100 Subject: [Python-Dev] Enumeration items: `type(EnumClass.item) is EnumClass` ? In-Reply-To: <517ED179.2080100@pearwood.info> References: <517E768A.9090602@stoneleaf.us> <517E9C09.3010806@stoneleaf.us> <517EBAFB.3060109@pearwood.info> <517EC3DA.9090105@mrabarnett.plus.com> <517ED179.2080100@pearwood.info> Message-ID: <517F00C9.10609@mrabarnett.plus.com> On 29/04/2013 21:00, Steven D'Aprano wrote: > On 30/04/13 05:02, MRAB wrote: > >>> Why is that backwards? MoreColor is a subclass of Color, so >>> instances of MoreColor are instances of Color, but instances of >>> Color are not instances of MoreColor. That's normal behaviour for >>> subclasses. (All cats are mammals, but not all mammals are >>> cats.) >>> >> Let's say that Color is red, green, or blue. >> >> Let's also say that MoreColor is superset of Color, in other words, >> any of Color, plus cyan, magenta, or yellow. >> >> Red is a Color and a MoreColor (member of Color and MoreColor). >> >> Yellow is a MoreColor, but not a Color (member of MoreColor but >> not Color). >> >> That's the opposite of the rules of inheritance. > > Membership != inheritance, you are conflating two independent > concepts. I would expect that for membership testing, you should say > "value in MoreColor", not isinstance(value, MoreColor). > I do know the difference, but it's not helpful that you're creating the enum with a "class" statement! > flufl.enum has been in use for Mailman for many years, and I would > like to hear Barry's opinion on this. > From ethan at stoneleaf.us Tue Apr 30 00:40:46 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 29 Apr 2013 15:40:46 -0700 Subject: [Python-Dev] Enumeration item arguments? In-Reply-To: References: <517EED3D.70705@stoneleaf.us> Message-ID: <517EF6EE.7040000@stoneleaf.us> On 04/29/2013 03:25 PM, Eli Bendersky wrote: > > Besides, did we not agree that the only acceptable *members* for enums are going to be descriptors? In the above, mass & > radius are not descriptors. Actually, it's the other way -- descriptors are excluded from being enum items, along with dunders. -- ~Ethan~ From ethan at stoneleaf.us Tue Apr 30 01:16:48 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 29 Apr 2013 16:16:48 -0700 Subject: [Python-Dev] Enumeration items: mixed types? In-Reply-To: <517EF92E.5050906@stoneleaf.us> References: <517EF92E.5050906@stoneleaf.us> Message-ID: <517EFF60.6000902@stoneleaf.us> On 04/29/2013 03:50 PM, Ethan Furman wrote: > This just doesn't make sense to me: > > --> class Stuff(Enum): > ... blue = 1 > ... china = 'really big country' > ... random = (8273.199, 517) > > --> Stuff.blue.name == 'blue' > --> Stuff.blue.value == 1 > > --> Stuff.china.name == 'china' > --> Stuff.china.value == ??? > > --> Stuff.random.name == 'random' > --> Stuff.china.value == ??? > > In order to make this work at all, we have to support auto-numbering, and I didn't think we were going to do that in the > class syntax? I suppose the other option is to have `.value` be whatever was assigned (1, 'really big country', and (8273.199, 517) ), and the fact that `int(Stuff.china) ` blows up and doesn't store easily in a database is the programmers issue... -- ~Ethan~ From "ja...py" at farowl.co.uk Mon Apr 29 23:58:48 2013 From: "ja...py" at farowl.co.uk (Jeff Allen) Date: Mon, 29 Apr 2013 22:58:48 +0100 Subject: [Python-Dev] Destructors and Closing of File Objects In-Reply-To: References: <87a9p41gr6.fsf@vostro.rath.org> <877gk35aqx.fsf@vostro.rath.org> <87y5c4r91p.fsf@vostro.rath.org> Message-ID: <517EED18.5000707@farowl.co.uk> On 29/04/2013 15:42, Armin Rigo wrote: > On Sat, Apr 27, 2013 at 4:39 AM, Nikolaus Rath wrote: >> It's indeed very informative, but it doesn't fully address the question >> because of the _pyio module which certainly can't use any custom C code. >> Does that mean that when I'm using x = _pyio.BufferedWriter(), I could loose >> data in the write buffer when the interpreter exits without me calling >> x.close(), but when using x = io.BufferedWriter(), the buffer is >> guaranteed to get flushed? > I actually described the behavior of CPython 2 while not realizing > that CPython 3 silently dropped this guarantee. (I also never > realized that Jython/IronPython don't have the same guarantee; they > could, if they implement 'atexit', like we did in PyPy. On 29/04/2013 17:02, Antoine Pitrou wrote: > It is dropped in the case of reference cycles, since there's no general > way to decide in which order the tp_clear calls have to be done. > Thus in the following layered situation: a TextIOWrapper on top of a > BufferedWriter on top of a FileIO, if BufferedWriter.tp_clear is called > first, it will flush and then close itself, closing the FileIO at the > same time, and when TextIOWrapper.tp_clear will be called it will be > too late to flush its own buffer. > > (I have to investigate a bit to confirm it is what happens) > > I will try to think of a scheme to make flushing more reliable, but > nothing springs to my mind right now. > In Jython, objects are not "cleared" immediately they become unreachable and if the JVM does not collect them before it shuts down, no programmed finalization may be called. To get round this, files in need of closing are hooked to a list that is worked off as the JVM shuts down, the equivalent of atexit (I assume). It has the unfortunate effect that forgotten files may live even longer, making it even more desirable that the user remember to close them. (The io tests themselves are not good at this!) But at least the close comes eventually. After discussion on jython-dev, I recently changed this mechanism (aiming at v2.7) so that every layer e.g. TextIOWrapper, BufferedWriter, FileIO is separately hooked to the list, and these are closed in reverse order of creation. Since close invokes flush when it matters, this will nearly always mean data is flushed down the stack before the path to disk gets severed, and always if you used open() to create the stack. I couldn't think of a perfect solution that didn't mean change to the API. This idea, and some tidying up I did in the io tests, might be of use in CPython. Jeff From greg.ewing at canterbury.ac.nz Tue Apr 30 02:32:28 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 30 Apr 2013 12:32:28 +1200 Subject: [Python-Dev] Enumeration item arguments? In-Reply-To: References: <517EED3D.70705@stoneleaf.us> Message-ID: <517F111C.1020700@canterbury.ac.nz> Eli Bendersky wrote: > Besides, did we not agree that the only acceptable *members* for enums > are going to be descriptors? No, that only applies to names assigned in the class body. Here, mass and radius are being set a different way, so there is no restriction on them. -- Greg From greg.ewing at canterbury.ac.nz Tue Apr 30 02:38:14 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 30 Apr 2013 12:38:14 +1200 Subject: [Python-Dev] Enumeration items: mixed types? In-Reply-To: <517EFF60.6000902@stoneleaf.us> References: <517EF92E.5050906@stoneleaf.us> <517EFF60.6000902@stoneleaf.us> Message-ID: <517F1276.8070201@canterbury.ac.nz> Ethan Furman wrote: > I suppose the other option is to have `.value` be whatever was assigned > (1, 'really big country', and (8273.199, 517) ), I thought that was the intention all along, and that we'd given up on the idea of auto-assigning integer values (because it would require either new syntax or extremely dark magic). -- Greg From Nikolaus at rath.org Tue Apr 30 03:00:30 2013 From: Nikolaus at rath.org (Nikolaus Rath) Date: Mon, 29 Apr 2013 18:00:30 -0700 Subject: [Python-Dev] Destructors and Closing of File Objects References: <87a9p41gr6.fsf@vostro.rath.org> <877gk35aqx.fsf@vostro.rath.org> <87y5c4r91p.fsf@vostro.rath.org> Message-ID: <87sj28al3l.fsf@vostro.rath.org> Armin Rigo writes: > Hi Nikolaus, > > On Sat, Apr 27, 2013 at 4:39 AM, Nikolaus Rath wrote: >> It's indeed very informative, but it doesn't fully address the question >> because of the _pyio module which certainly can't use any custom C code. >> Does that mean that when I'm using x = _pyio.BufferedWriter(), I could loose >> data in the write buffer when the interpreter exits without me calling >> x.close(), but when using x = io.BufferedWriter(), the buffer is >> guaranteed to get flushed? > > I actually described the behavior of CPython 2 while not realizing > that CPython 3 silently dropped this guarantee. (I also never > realized that Jython/IronPython don't have the same guarantee; they > could, if they implement 'atexit', like we did in PyPy. That's > however more acceptable if the platform itself doesn't offer the > guarantee.) > > Anyway, it's a guarantee that the C offers, so personally I find it > reasonable to expect CPython files to offer it too; not offering it is > kind of saying that there is a feature of C that is actually present > at a higher level than the exact same feature in Python, which looks > backward to me. > > Additionally, this might be introducing subtle bugs in programs when > porting them to Python 3. > > However I realize that the two arguments presented above might not be > accepted as relevant. (http://bugs.python.org/issue17852) Thanks for the research! I tried writing a test for this myself, but failed to get the cyclic reference in place properly for it to happen. Seems I'm not the only one who was unaware and/or surprised by he change in behavior. Best, -Nikolaus -- ?Time flies like an arrow, fruit flies like a Banana.? PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C From marco.hemmelrath at googlemail.com Tue Apr 30 03:23:06 2013 From: marco.hemmelrath at googlemail.com (Marco Hemmelrath) Date: Tue, 30 Apr 2013 03:23:06 +0200 Subject: [Python-Dev] enum instances Message-ID: <517F1CFA.3070905@googlemail.com> First of all, hi, I'm new to this list. Following the enum discussions on this list I am kind of confused about how enums and their respective instances, i.e. the values, should behave in "normal" context. I apologize beforehand for the mass of "questions" because the following contains really many discussed things but they all kinda depend on one another, and for terminology which I might've missed. Considering we have[1]: class Color(enum): red = 1 white = 2 other = "undefined" class State(enum): idle = 0 busy = 1 idling = idle ideling = 0 together with the premises: 1. type(State.busy) == State 2. type(State) == enum 3. isinstance(State.idle, State) 4. State.idle is State.idle which should mostly be agreed on (if I didn't misinterpret). How would an enum instance (e.g. State.busy) behave in normal Python expressions? Should these instances just wrap their values and provide some simple overhead useful for enums? I'll just note down a few examples of how I think it could work and express a few thoughts on them: 1. State.busy == 1 2. State.busy == Color.red 3. int(State.Busy) is 1 4. isinstance(State.busy, int) 5. Color.other.startswith("und") 6. State.busy is not 1 7. State.busy is not Color.red 8. State.idle in State 9. 0 in State # True, False or raise? 10. State.idling is State.idle 11. State.idle == State.idling 12. State.idle is not State.idling 1. & 2. Considering that enum components contain a value this value should be accessable and comperable. If it wasn't then there would be no real need to assign a value to it because everything must be compared using the is operator anyway. In this case an entirely new syntax could be implemented but since I think this is not what we want I'll skip this. Futhermore, if enum instances should compare equal to their values but unequal to each other this creates weird circumstances like: Color.red == 1 == State.busy -but- Color.red != State.busy == 1 3. Similar to 1. an enum instance's value /should/ be accessable as an expression by itself when needed besides simple operations like compatisons. This might be really tricky because the object in question is still of type State and some existing uses might break even though I can't think of any. What would certainly be a problem in this case though is that repr() would not reveal the value but the actual enum instance. This could be problematic when the enum instance's value is not a standard type or when repr() and str() differ and repr() is what's needed. 4. Combines 1. and 2. in class relation. It could be argued that the actual value should be an instance of its type subclassed by the enum class (State) which is in turn a subclass of enum. Going further on this it could also allow various things like class EnumWithInt(enum, int) which would only allow integers as its enum identifiers. This goes slightly into the IntEnum direction but with more flexibility. 5. Related to 3. in that this also allows attribute access to the enum instance's value. Considering that the enum instance should already be an instance of its value type as well this kinda speaks for itself. 6. & 7. Obviously, these can't be true even though they compare equal (see 1. and 2.). 8. Check if an enum instance is a member of State. Since in this case the same can also be achieved by using isinstance or by comparing the instance's type, this is mostly interesting for subclassing enums because, but I won't that cover here because it would probably be too much. 9. Analog to 8. this could either return just work, simply return False because this object is obviously not a member of State or raise an exception because of a type mismatch (not an enum instance). Also interesting for subclassing enums. 10. See premise 4. on this. 11. & 12. Even though these compare equal and even if their value object is actually the same they themseves should /not/ be the same object. Someone needs to find a use case for this though because as of now I can only think of this being Pythonic. So, the most critical part of this would probably be 3. to 7. regarding the value types. I'd love to read some thoughts and comments on that. Regards, Marco [1]: http://mail.python.org/pipermail/python-dev/2013-April/125738.html From pjenvey at underboss.org Tue Apr 30 04:30:24 2013 From: pjenvey at underboss.org (Philip Jenvey) Date: Mon, 29 Apr 2013 19:30:24 -0700 Subject: [Python-Dev] Enumeration item arguments? In-Reply-To: References: <517EED3D.70705@stoneleaf.us> Message-ID: <26EA6795-72AB-4FF2-9939-5937B7A4B4D7@underboss.org> On Apr 29, 2013, at 3:25 PM, Eli Bendersky wrote: > On Mon, Apr 29, 2013 at 2:59 PM, Ethan Furman wrote: > In the Planet example we saw the possibility of specifying arguments to enum item __init__: > > class Planet(Enum): > MERCURY = (3.303e+23, 2.4397e6) > VENUS = (4.869e+24, 6.0518e6) > EARTH = (5.976e+24, 6.37814e6) > MARS = (6.421e+23, 3.3972e6) > JUPITER = (1.9e+27, 7.1492e7) > SATURN = (5.688e+26, 6.0268e7) > URANUS = (8.686e+25, 2.5559e7) > NEPTUNE = (1.024e+26, 2.4746e7) > > def __init__(self, mass, radius): > self.mass = mass # in kilograms > self.radius = radius # in meters > > Do we want to support this? > > I'm -1, and this is yet another bad sign of conflating enums with classes. If planets want to have attributes and behaviors, let them be normal classes. If they want a PlanetId *enum member*, that's OK, but there's no need to intermix the two. You may be right about the Planet example, but I wouldn't go that far. Additional metadata on Enum instances can be a handy feature in some cases, like documentation, e.g.: class Protocol(Enum): HOPOPT = 0, "IPv6 Hop-by-Hop Option" ICMP = 1, "Internet Control Message" IGMP = 2, "Internet Group Management" @property def value(self): return self._value[0] @property def description(self): return self._value[1] -- Philip Jenvey From ethan at stoneleaf.us Tue Apr 30 04:17:08 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 29 Apr 2013 19:17:08 -0700 Subject: [Python-Dev] enum instances In-Reply-To: <517F1CFA.3070905@googlemail.com> References: <517F1CFA.3070905@googlemail.com> Message-ID: <517F29A4.9020109@stoneleaf.us> On 04/29/2013 06:23 PM, Marco Hemmelrath wrote: > First of all, hi, I'm new to this list. > > Following the enum discussions on this list I am kind of confused about how enums and their respective instances, i.e. > the values, should behave in "normal" context. > I apologize beforehand for the mass of "questions" because the following contains really many discussed things but they > all kinda depend on one another, and for terminology which I might've missed. > > Considering we have[1]: > > class Color(enum): > red = 1 > white = 2 > other = "undefined" > > class State(enum): > idle = 0 > busy = 1 > idling = idle > ideling = 0 > > together with the premises: > > 1. type(State.busy) == State > 2. type(State) == enum > 3. isinstance(State.idle, State) > 4. State.idle is State.idle > > which should mostly be agreed on (if I didn't misinterpret). > > > How would an enum instance (e.g. State.busy) behave in normal Python expressions? Should these instances just wrap their > values and provide some simple overhead useful for enums? > > I'll just note down a few examples of how I think it could work and express a few thoughts on them: > > 1. State.busy == 1 > 2. State.busy == Color.red > 3. int(State.Busy) is 1 > 4. isinstance(State.busy, int) > 5. Color.other.startswith("und") > 6. State.busy is not 1 > 7. State.busy is not Color.red > 8. State.idle in State > 9. 0 in State # True, False or raise? > 10. State.idling is State.idle > 11. State.idle == State.idling > 12. State.idle is not State.idling 1 & 2 are False (would be true if using `.value`) 3 is True 4 & 5 are False (again, need `.value`) 6 is not the correct way to use `is` (should be `==`) 7, 8, 9, 10, 11 are True 12 is False From Nikolaus at rath.org Tue Apr 30 04:42:35 2013 From: Nikolaus at rath.org (Nikolaus Rath) Date: Mon, 29 Apr 2013 19:42:35 -0700 Subject: [Python-Dev] enum instances References: <517F1CFA.3070905@googlemail.com> Message-ID: <87ppxcagdg.fsf@vostro.rath.org> Marco Hemmelrath writes: > class State(enum): > idle = 0 > busy = 1 > idling = idle > ideling = 0 > > together with the premises: > > 1. type(State.busy) == State > 2. type(State) == enum State is a class, it just inherits from enum. Thus: type(State) == type(enum) == type(EnumMetaclass) issubclass(State, enum) == True HTH, -Nikolaus -- ?Time flies like an arrow, fruit flies like a Banana.? PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C From larry at hastings.org Tue Apr 30 05:45:52 2013 From: larry at hastings.org (Larry Hastings) Date: Mon, 29 Apr 2013 20:45:52 -0700 Subject: [Python-Dev] enum instances In-Reply-To: <87ppxcagdg.fsf@vostro.rath.org> References: <517F1CFA.3070905@googlemail.com> <87ppxcagdg.fsf@vostro.rath.org> Message-ID: <517F3E70.2010108@hastings.org> On 04/29/2013 07:42 PM, Nikolaus Rath wrote: > State is a class, it just inherits from enum. Thus: > > type(State) == type(enum) == type(EnumMetaclass) > issubclass(State, enum) == True > > > HTH, > > -Nikolaus If you'd tried it, you'd have found that that isn't true. enum has a metaclass, EnumMetaclass. Thus type(enum) == EnumMetaClass. That didn't help, //arry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Tue Apr 30 05:45:29 2013 From: barry at python.org (Barry Warsaw) Date: Mon, 29 Apr 2013 20:45:29 -0700 Subject: [Python-Dev] Enumeration items: mixed types? In-Reply-To: <517EFF60.6000902@stoneleaf.us> References: <517EF92E.5050906@stoneleaf.us> <517EFF60.6000902@stoneleaf.us> Message-ID: <20130429204529.4e9c6fa1@anarchist> On Apr 29, 2013, at 04:16 PM, Ethan Furman wrote: >I suppose the other option is to have `.value` be whatever was assigned (1, >'really big country', and (8273.199, 517) ), and the fact that >`int(Stuff.china) ` blows up and doesn't store easily in a database is the >programmers issue... Correct. flufl.enum.IntEnums will blow up earlier, when the class is defined assigning the attributes to non-int values. -Barry From barry at python.org Tue Apr 30 05:47:01 2013 From: barry at python.org (Barry Warsaw) Date: Mon, 29 Apr 2013 20:47:01 -0700 Subject: [Python-Dev] Enumeration item arguments? In-Reply-To: References: <517EED3D.70705@stoneleaf.us> Message-ID: <20130429204701.29dd234d@anarchist> On Apr 29, 2013, at 03:25 PM, Eli Bendersky wrote: >I'm -1, and this is yet another bad sign of conflating enums with classes. >If planets want to have attributes and behaviors, let them be normal >classes. If they want a PlanetId *enum member*, that's OK, but there's no >need to intermix the two. I agree with this. -Barry From eliben at gmail.com Tue Apr 30 06:47:56 2013 From: eliben at gmail.com (Eli Bendersky) Date: Mon, 29 Apr 2013 21:47:56 -0700 Subject: [Python-Dev] Enumeration item arguments? In-Reply-To: <26EA6795-72AB-4FF2-9939-5937B7A4B4D7@underboss.org> References: <517EED3D.70705@stoneleaf.us> <26EA6795-72AB-4FF2-9939-5937B7A4B4D7@underboss.org> Message-ID: On Mon, Apr 29, 2013 at 7:30 PM, Philip Jenvey wrote: > > On Apr 29, 2013, at 3:25 PM, Eli Bendersky wrote: > > > On Mon, Apr 29, 2013 at 2:59 PM, Ethan Furman > wrote: > > In the Planet example we saw the possibility of specifying arguments to > enum item __init__: > > > > class Planet(Enum): > > MERCURY = (3.303e+23, 2.4397e6) > > VENUS = (4.869e+24, 6.0518e6) > > EARTH = (5.976e+24, 6.37814e6) > > MARS = (6.421e+23, 3.3972e6) > > JUPITER = (1.9e+27, 7.1492e7) > > SATURN = (5.688e+26, 6.0268e7) > > URANUS = (8.686e+25, 2.5559e7) > > NEPTUNE = (1.024e+26, 2.4746e7) > > > > def __init__(self, mass, radius): > > self.mass = mass # in kilograms > > self.radius = radius # in meters > > > > Do we want to support this? > > > > I'm -1, and this is yet another bad sign of conflating enums with > classes. If planets want to have attributes and behaviors, let them be > normal classes. If they want a PlanetId *enum member*, that's OK, but > there's no need to intermix the two. > > You may be right about the Planet example, but I wouldn't go that far. > Additional metadata on Enum instances can be a handy feature in some cases, > like documentation, e.g.: > > class Protocol(Enum): > > HOPOPT = 0, "IPv6 Hop-by-Hop Option" > ICMP = 1, "Internet Control Message" > IGMP = 2, "Internet Group Management" > > @property > def value(self): > return self._value[0] > > @property > def description(self): > return self._value[1] > Note that I don't object to adding certain behaviors to enums. I'm against taking it too far. One example of taking it too far is special implementation provisions to make enum be more like other classes, complicating the implementation just for this cause. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Apr 30 08:15:34 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 30 Apr 2013 16:15:34 +1000 Subject: [Python-Dev] Enumeration items: mixed types? In-Reply-To: <517EF92E.5050906@stoneleaf.us> References: <517EF92E.5050906@stoneleaf.us> Message-ID: <20130430061534.GB4530@ando> On Mon, Apr 29, 2013 at 03:50:22PM -0700, Ethan Furman wrote: > This just doesn't make sense to me: > > --> class Stuff(Enum): > ... blue = 1 > ... china = 'really big country' > ... random = (8273.199, 517) > > --> Stuff.blue.name == 'blue' > --> Stuff.blue.value == 1 > > --> Stuff.china.name == 'china' > --> Stuff.china.value == ??? Quoting the PEP: "In the vast majority of use-cases, one doesn't care what the actual value of an enumeration is. But if the value is important, enumerations can have arbitrary values." http://www.python.org/dev/peps/pep-0435/#id21 So in the above, Stuff.china.value == 'really big country' and Stuff.random.value == (8273.199, 517). Mixing enumeration values like this would be unusual, but it will work. It's no different from having a list [1, 'really big country', (8273.199, 517)]. Lists can deal with it, but if you pass a list of arbitrary types to something that expects a list of ints, it will complain. -- Steven From g.brandl at gmx.net Tue Apr 30 09:06:26 2013 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 30 Apr 2013 09:06:26 +0200 Subject: [Python-Dev] Regression fix releases coming Message-ID: Hi all, since there turned out to be a "critical mass" of regressions in 2.7.4/ 3.2.4/3.3.1, Benjamin and I would like to release a releasing fixing these next weekend. Since we would like to cherry-pick only regression fixes into these releases, please make sure any regression issues you know of are set to "open" (even if fixed in the main repo) and status "release blocker" for the appropriate versions. If they aren't fixed yet, it would also be nice to have a patch ready by Saturday :) Thanks, Georg From arigo at tunes.org Tue Apr 30 10:49:19 2013 From: arigo at tunes.org (Armin Rigo) Date: Tue, 30 Apr 2013 10:49:19 +0200 Subject: [Python-Dev] Destructors and Closing of File Objects In-Reply-To: <517EED18.5000707@farowl.co.uk> References: <87a9p41gr6.fsf@vostro.rath.org> <877gk35aqx.fsf@vostro.rath.org> <87y5c4r91p.fsf@vostro.rath.org> <517EED18.5000707@farowl.co.uk> Message-ID: Hi Jeff, On Mon, Apr 29, 2013 at 11:58 PM, Jeff Allen <"ja...py"@farowl.co.uk> wrote: > In Jython, (...) Thanks Jeff for pointing this out. Jython thus uses a custom mechanism similar to PyPy's, which is also similar to atexit's. It should not be too hard to implement it in CPython 3 as well, if this ends up classified as a bug. This is what my bug report was about (sorry if I failed to be clear enough about it). Nikolaus: the bug report contains a failing test, is that what you're looking for? A bient?t, Armin. From p.f.moore at gmail.com Tue Apr 30 11:15:58 2013 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 30 Apr 2013 10:15:58 +0100 Subject: [Python-Dev] Getting a list of registered codecs Message-ID: Before I raise a bug for this, can someone confirm if I've simply missed something? I don't see any way, either in the docs or in the helpstrings from the codecs, of listing the codecs that have been registered. FWIW, I picked this up when I was looking at writing a simple encoding converter, and I wanted to add a flag to list what conversions were supported. Paul. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Apr 30 11:36:31 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 30 Apr 2013 19:36:31 +1000 Subject: [Python-Dev] Getting a list of registered codecs In-Reply-To: References: Message-ID: <20130430093628.GD4530@ando> On Tue, Apr 30, 2013 at 10:15:58AM +0100, Paul Moore wrote: > Before I raise a bug for this, can someone confirm if I've simply missed > something? I don't see any way, either in the docs or in the helpstrings > from the codecs, of listing the codecs that have been registered. > > FWIW, I picked this up when I was looking at writing a simple encoding > converter, and I wanted to add a flag to list what conversions were > supported. This may be of help: http://hg.python.org/releasing/3.3.1/file/tip/Tools/unicode/listcodecs.py -- Steven From mal at egenix.com Tue Apr 30 11:42:49 2013 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 30 Apr 2013 11:42:49 +0200 Subject: [Python-Dev] Getting a list of registered codecs In-Reply-To: References: Message-ID: <517F9219.4030103@egenix.com> On 30.04.2013 11:15, Paul Moore wrote: > Before I raise a bug for this, can someone confirm if I've simply missed > something? I don't see any way, either in the docs or in the helpstrings > from the codecs, of listing the codecs that have been registered. > > FWIW, I picked this up when I was looking at writing a simple encoding > converter, and I wanted to add a flag to list what conversions were > supported. > Paul. It would be possible to get a list of registered codec search functions, but there's no API to ask the search functions for a list of supported codecs. If you're just looking for a list of codecs supported by the stdlib encodings module, you can use the helper that Steven mentioned, or you can scan the encoding aliases dictionary for codecs (but this will not necessarily return all available codecs). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 30 2013) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2013-04-30: Released eGenix PyRun 1.2.0 ... http://egenix.com/go44 2013-04-17: Released eGenix mx Base 3.2.6 ... http://egenix.com/go43 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mal at egenix.com Tue Apr 30 12:18:27 2013 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 30 Apr 2013 12:18:27 +0200 Subject: [Python-Dev] Getting a list of registered codecs In-Reply-To: References: <517F9219.4030103@egenix.com> Message-ID: <517F9A73.7080704@egenix.com> On 30.04.2013 11:52, Paul Moore wrote: > On 30 April 2013 10:42, M.-A. Lemburg wrote: > >> It would be possible to get a list of registered codec search functions, >> but there's no API to ask the search functions for a list of supported >> codecs. >> > > OK, so there's no way to determine in advance what values of enc will work > in bytestr.decode(enc) or str.encode(enc)? > > Is there a reason why not? Nothing in particular, except maybe that it can be expensive to generate such a list (e.g. you'd have to verify that the codec modules import correctly and provide the needed getregentry() API). > As I say, a tool that offers to re-encode a file > could reasonably be expected to list the encodings it supported (if only to > help the user work out which way to spell utf-16le or utf16le or utf16-le > or utf-16-le or... :-)) > > I've raised http://bugs.python.org/issue17878 for this. Further discussion > may be more appropriate there than on python-dev. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 30 2013) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2013-04-30: Released eGenix PyRun 1.2.0 ... http://egenix.com/go44 2013-04-17: Released eGenix mx Base 3.2.6 ... http://egenix.com/go43 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From p.f.moore at gmail.com Tue Apr 30 11:52:49 2013 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 30 Apr 2013 10:52:49 +0100 Subject: [Python-Dev] Getting a list of registered codecs In-Reply-To: <517F9219.4030103@egenix.com> References: <517F9219.4030103@egenix.com> Message-ID: On 30 April 2013 10:42, M.-A. Lemburg wrote: > It would be possible to get a list of registered codec search functions, > but there's no API to ask the search functions for a list of supported > codecs. > OK, so there's no way to determine in advance what values of enc will work in bytestr.decode(enc) or str.encode(enc)? Is there a reason why not? As I say, a tool that offers to re-encode a file could reasonably be expected to list the encodings it supported (if only to help the user work out which way to spell utf-16le or utf16le or utf16-le or utf-16-le or... :-)) I've raised http://bugs.python.org/issue17878 for this. Further discussion may be more appropriate there than on python-dev. Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Tue Apr 30 15:57:13 2013 From: guido at python.org (Guido van Rossum) Date: Tue, 30 Apr 2013 06:57:13 -0700 Subject: [Python-Dev] Regression fix releases coming In-Reply-To: References: Message-ID: Can we do something about the problem that virus checkers complain about the xml bomb test file? E.g. Generate it dynamically in a script? On Tuesday, April 30, 2013, Georg Brandl wrote: > Hi all, > > since there turned out to be a "critical mass" of regressions in 2.7.4/ > 3.2.4/3.3.1, Benjamin and I would like to release a releasing fixing these > next weekend. > > Since we would like to cherry-pick only regression fixes into these > releases, > please make sure any regression issues you know of are set to "open" (even > if > fixed in the main repo) and status "release blocker" for the appropriate > versions. > > If they aren't fixed yet, it would also be nice to have a patch ready by > Saturday :) > > Thanks, > Georg > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Tue Apr 30 16:07:17 2013 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 30 Apr 2013 10:07:17 -0400 Subject: [Python-Dev] Regression fix releases coming In-Reply-To: References: Message-ID: 2013/4/30 Guido van Rossum : > Can we do something about the problem that virus checkers complain about the > xml bomb test file? E.g. Generate it dynamically in a script? That's been dealt with. See http://bugs.python.org/issue17843 -- Regards, Benjamin From ethan at stoneleaf.us Tue Apr 30 17:58:02 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 30 Apr 2013 08:58:02 -0700 Subject: [Python-Dev] Enumeration items: mixed types? In-Reply-To: <517F1276.8070201@canterbury.ac.nz> References: <517EF92E.5050906@stoneleaf.us> <517EFF60.6000902@stoneleaf.us> <517F1276.8070201@canterbury.ac.nz> Message-ID: <517FEA0A.4090605@stoneleaf.us> On 04/29/2013 05:38 PM, Greg Ewing wrote: > Ethan Furman wrote: >> I suppose the other option is to have `.value` be whatever was assigned (1, 'really big country', and (8273.199, 517) ), > > I thought that was the intention all along, and that we'd > given up on the idea of auto-assigning integer values > (because it would require either new syntax or extremely > dark magic). Not that dark, actually -- just a little dim. ;) I just had it stuck in my head that every enum item would have an integer associated with it, and possibly have another value as well. So, for example: --> class Constants(float, Enum): ... e = 2.81843 # am I close? ... pi = 3.141596 ... tau = 2 * pi I cannot do --> Constants(2) and get pi. I have to do --> Constants(3.141596) and given the nature of floating point I can see that failing at some, er, point. Likewise, if I have normal, but jumbled, Enum: --> class Jumble(Enum): ... eggs = 1 ... method = 'scramble' ... cost = 2.5 then to get method back I have to use --> Jumble('scramble') It just seems odd to me. Oh, and it would seem just as odd using __getitem__. ;) -- ~Ethan~ From eliben at gmail.com Tue Apr 30 18:27:54 2013 From: eliben at gmail.com (Eli Bendersky) Date: Tue, 30 Apr 2013 09:27:54 -0700 Subject: [Python-Dev] Enumeration items: mixed types? In-Reply-To: <517F1276.8070201@canterbury.ac.nz> References: <517EF92E.5050906@stoneleaf.us> <517EFF60.6000902@stoneleaf.us> <517F1276.8070201@canterbury.ac.nz> Message-ID: On Mon, Apr 29, 2013 at 5:38 PM, Greg Ewing wrote: > Ethan Furman wrote: > >> I suppose the other option is to have `.value` be whatever was assigned >> (1, 'really big country', and (8273.199, 517) ), >> > > I thought that was the intention all along, and that we'd > given up on the idea of auto-assigning integer values > (because it would require either new syntax or extremely > dark magic). > Yes, Guido rejected the auto-numbering syntax a while back. The only case in which auto-numbering occurs (per PEP 435) is the "convenience syntax": Animal = Enum('Animal', 'fox dog cat') Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Tue Apr 30 22:12:05 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 30 Apr 2013 13:12:05 -0700 Subject: [Python-Dev] PEP-435 reference implementation Message-ID: <51802595.2040305@stoneleaf.us> Greetings, Eli asked me to put the reference implementation here for review. It is available at https://bitbucket.org/stoneleaf/aenum in ref435.py and test_ref435.py -- ~Ethan~ From nadeem.vawda at gmail.com Tue Apr 30 23:06:22 2013 From: nadeem.vawda at gmail.com (Nadeem Vawda) Date: Tue, 30 Apr 2013 23:06:22 +0200 Subject: [Python-Dev] Regression fix releases coming In-Reply-To: References: Message-ID: On Tue, Apr 30, 2013 at 4:07 PM, Benjamin Peterson wrote: > 2013/4/30 Guido van Rossum : > > Can we do something about the problem that virus checkers complain about > the > > xml bomb test file? E.g. Generate it dynamically in a script? > > That's been dealt with. See http://bugs.python.org/issue17843 I don't think that's the same issue. The link you gave is about *BZ2* test data triggering antivirus warnings, not XML. Is there another issue for a similar problem with an XML test file? (Searching bugs.python.org for "xml antivirus" doesn't turn up any results...) - Nadeem -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Tue Apr 30 23:07:16 2013 From: guido at python.org (Guido van Rossum) Date: Tue, 30 Apr 2013 14:07:16 -0700 Subject: [Python-Dev] Regression fix releases coming In-Reply-To: References: Message-ID: Sorry, that was my confusion. The virus warnings were about the bz2 test data. XML bombs have not yet been addressed in any revision AFAIK. On Tue, Apr 30, 2013 at 2:06 PM, Nadeem Vawda wrote: > On Tue, Apr 30, 2013 at 4:07 PM, Benjamin Peterson > wrote: >> >> 2013/4/30 Guido van Rossum : >> > Can we do something about the problem that virus checkers complain about >> > the >> > xml bomb test file? E.g. Generate it dynamically in a script? >> >> That's been dealt with. See http://bugs.python.org/issue17843 > > > I don't think that's the same issue. The link you gave is about *BZ2* test > data triggering antivirus warnings, not XML. Is there another issue for a > similar problem with an XML test file? (Searching bugs.python.org for "xml > antivirus" doesn't turn up any results...) > > - Nadeem -- --Guido van Rossum (python.org/~guido) From eliben at gmail.com Tue Apr 30 23:07:34 2013 From: eliben at gmail.com (Eli Bendersky) Date: Tue, 30 Apr 2013 14:07:34 -0700 Subject: [Python-Dev] PEP-435 reference implementation In-Reply-To: <51802595.2040305@stoneleaf.us> References: <51802595.2040305@stoneleaf.us> Message-ID: On Tue, Apr 30, 2013 at 1:12 PM, Ethan Furman wrote: > Greetings, > > Eli asked me to put the reference implementation here for review. > > It is available at https://bitbucket.org/stoneleaf/aenum in ref435.py and > test_ref435.py > Thanks, Ethan. All - note that strictly speaking this implements PEP 435 with the modifications that were decided in recent discussions and pronouncement by Guido. If you're not up to date with the discussion, look for Ethan's summary and Guido's pronouncement in the archives. I'll work on updating PEP 435 to reflect these decisions by the end of this week. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Tue Apr 30 22:54:44 2013 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 30 Apr 2013 13:54:44 -0700 Subject: [Python-Dev] PEP-435 reference implementation In-Reply-To: <51802595.2040305@stoneleaf.us> References: <51802595.2040305@stoneleaf.us> Message-ID: <51802F94.5020605@stoneleaf.us> On 04/30/2013 01:12 PM, Ethan Furman wrote: > > It is available at https://bitbucket.org/stoneleaf/aenum in ref435.py and test_ref435.py Oh, as written in requires 3.3+. If you want to play around with it and are stuck on an earlier version, remove the `from None` on line 68. -- ~Ethan~