From erno at iki.fi Tue Apr 3 08:47:36 2007 From: erno at iki.fi (Erno Kuusela) Date: Tue, 3 Apr 2007 09:47:36 +0300 Subject: [Python-3000] IPv6 address tuple In-Reply-To: <20070323220556.8C8A21C047@coconut.itojun.org> References: <20070323093628.BC6A61C04E@coconut.itojun.org> <20070323220556.8C8A21C047@coconut.itojun.org> Message-ID: <20070403064736.GC24629@erno.iki.fi> hello, On Sat, 24 Mar 2007, Jun-ichiro itojun Hagino wrote: | > i understand your concern. current 4-tuple is direct mapping from | > struct sockaddr_in6 (which has those two additional members). | | note however, you must rewrite DNS resolving from gethostby* to | get*info anyways, so i'm not too sure if the number of tuple members in | sockaddr matters that much. This is true when using explicit DNS resolving, but Python socket API allows you to pass in a string in place of the address in the tuple, and then resolves it to an address behind your back. This is probably the most common way of using DNS in Python. It seems that when you have an AF_INET6 socket, it looks for an AAAA record, and when you have an AF_INET socket, it goes for A. Unfortunately this bumps the address family-agnostic get*info API to second place in the convenience-of-use competition (getaddrinfo example at ). -- erno From itojun at itojun.org Tue Apr 3 08:54:57 2007 From: itojun at itojun.org (Jun-ichiro itojun Hagino) Date: Tue, 3 Apr 2007 15:54:57 +0900 (JST) Subject: [Python-3000] IPv6 address tuple In-Reply-To: Your message of "Tue, 3 Apr 2007 09:47:36 +0300" <20070403064736.GC24629@erno.iki.fi> References: <20070403064736.GC24629@erno.iki.fi> Message-ID: <20070403065457.13D8A1C047@coconut.itojun.org> > | > i understand your concern. current 4-tuple is direct mapping from > | > struct sockaddr_in6 (which has those two additional members). > | > | note however, you must rewrite DNS resolving from gethostby* to > | get*info anyways, so i'm not too sure if the number of tuple members in > | sockaddr matters that much. > > This is true when using explicit DNS resolving, but Python socket API > allows you to pass in a string in place of the address in the tuple, > and then resolves it to an address behind your back. This > is probably the most common way of using DNS in Python. > > It seems that when you have an AF_INET6 socket, it looks for > an AAAA record, and when you have an AF_INET socket, it goes for A. hmm. how would Python behave when there are multiple AAAA (or A) records? just pick the first one? > Unfortunately this bumps the address family-agnostic get*info API > to second place in the convenience-of-use competition > (getaddrinfo example at ). with mixed IPv4/v6 environment (this would be the network environment for today and coming decade(s)), a program should try to connect both A and AAAA records, since connectivity situation varies by configuration such as network configuration, kernel compilation options and so forth. for this reason it would be better (or i might say we have to) for all libraries (like HTTP, FTP, and stuff) as well as user applications to use get*info API, instead of just building a tuple and then use AF_INET/6 socket. itojun From aahz at pythoncraft.com Tue Apr 3 21:08:48 2007 From: aahz at pythoncraft.com (Aahz) Date: Tue, 3 Apr 2007 12:08:48 -0700 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: References: <45F7D7FC.5080908@gmail.com> Message-ID: <20070403190847.GA24460@panix.com> [Taking the first day of Pesach as vacation gives me time to respond to some old threads] On Wed, Mar 14, 2007, Patrick Maupin wrote: > On 3/14/07, Nick Coghlan wrote: >> >> I detest the name 'flag_format', though - the function doesn't format a >> flag! > > I don't like it that much myself, either. I considered > "extended_format" and a few others, but wasn't happy with any of them. > Name suggestions are certainly welcome. Why not just string.format()? There's precedence for module-level functions to have a different API from a method. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Need a book? Use your library! From aahz at pythoncraft.com Tue Apr 3 21:15:35 2007 From: aahz at pythoncraft.com (Aahz) Date: Tue, 3 Apr 2007 12:15:35 -0700 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: References: Message-ID: <20070403191535.GB24460@panix.com> On Mon, Mar 12, 2007, Patrick Maupin wrote: > > Feature: Alternate syntaxes for escape to markup. > > The second method is the well-understood ${} syntax. The $ is easy to > find in a sea of { characters, and the only special handling required > is that every $ must be doubled. -0 > The third method is something I came up with a couple of years ago > when I was generating a lot of C code from templates. It would make > any non-Python programmer blanch, because it relies on significant > whitespace, but it made for very readable technical templates. WIth > this method "{foo}" escapes to markup, but when there is whitespace > after the leading "{", e.g. "{ foo}", the brace is not an escape to > markup. If the whitespace is a space, it is removed from the output, > but if it is '\r', '\n', or '\t', then it is left in the output. The > result is that, for example, most braces in most C texts do not need > to have spaces inserted after them, because they immediately precede a > newline. -1 > Feature: Automatic search of locals() and globals() for name lookups > if no parameters are given. -1 -- make it a flag for string.format() or use a different function name. > Feature: Placement of a dummy record on the traceback stack for > underlying errors. +1 > Changed feature: By default, not using all arguments is not an exception -0 for *args, +1 for **kwargs While I understand your reasoning, I still think that mismatched strings and arguments is likely to be a significant problem; instead I favor allowing this feature to be turned off. > Feature: Exception raised if attribute with leading underscore accessed. +1 -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Need a book? Use your library! From oliphant.travis at ieee.org Wed Apr 4 01:03:05 2007 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Tue, 03 Apr 2007 17:03:05 -0600 Subject: [Python-3000] PEP for extended buffer protocol Message-ID: I know everybody is busy, but I would appreciate some kind of feedback from Python developers about the attached extended buffer protocol which I am proposing for Python 3k. I am working on the changes to the struct module needed to accomodate the enhanced syntax. I'd like to know if I these changes will be accepted into Python and if the remaining extended buffer protocol changes will be allowed as well. Thank you very much, -Travis -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: pep_buffer.txt Url: http://mail.python.org/pipermail/python-3000/attachments/20070403/d2d72d76/attachment-0001.txt From guido at python.org Tue Apr 10 19:53:40 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Apr 2007 10:53:40 -0700 Subject: [Python-3000] Reminder: Py3k PEPs due by April Message-ID: As I stated in my PyCon keynote and repeated (though somewhat hidden) in PEP 3000, all PEPs for Py3k are due by April. The end of April is only a few weeks away, so now is the time to start drafting that PEP if you care about Python 3000! This is so that we will be able to concentrate on *implementing* the accepted proposals by June, in time for the planned release 3.0a1. Your PEP doesn't have to be *accepted* by April 30; but it should be in good shape, have seen some discussion on python-3000 (or python-ideas), and have popular support. We can do the final discussion about acceptance of individual PEPs (including some revisions) after the deadline; but no PEP submitted after the deadline has a chance. This applies to my own pet projects as well; in particular, if I want Abstract Base Classes in Py3k, I will have to write up a PEP by April 30. If you want interfaces or generic functions, likewise, you need to have a decent PEP submitted by April 30! I'm currently working on the new I/O library, and am confident that this will be in a usable shape by the end of this week. fter that, I plan to concentrate on the str/unicode unification, which is by far the riskiest project still to be done. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Tue Apr 10 20:28:02 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 10 Apr 2007 12:28:02 -0600 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: References: Message-ID: On 4/10/07, Guido van Rossum wrote: > As I stated in my PyCon keynote and repeated (though somewhat hidden) > in PEP 3000, all PEPs for Py3k are due by April. The end of April is > only a few weeks away, so now is the time to start drafting that PEP > if you care about Python 3000! This is so that we will be able to > concentrate on *implementing* the accepted proposals by June, in time > for the planned release 3.0a1. Does this also include proposals for parts of the Python 3000 stdlib, e.g. changing or replacing a particular module API? STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From python at rcn.com Tue Apr 10 20:42:10 2007 From: python at rcn.com (Raymond Hettinger) Date: Tue, 10 Apr 2007 14:42:10 -0400 (EDT) Subject: [Python-3000] Reminder: Py3k PEPs due by April Message-ID: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> Thanks for the reminder. I expect to write two: * PEP to eliminate __del__ in favor of weakref idioms * PEP on simpler alternative to abstract-base-classes suggesting an occasional method special attribute (so for instance a __getitem__ method can tell you whether it thinks it is a sequence or mapping). Here's that I think might not need a PEP: * Eliminate implicit string concatenation: "abc" "def" in favor of an explicit + operation. That simplifies the grammar just a bit and the compiler already is smart enough to do constant fold this operation at compile time. When there are multi-line concats, I think the parenthesis serves us much better than a trailing \ which is ugly and relies on having no trailing whitespace. Replace: 'hello ' \ 'world ' With: ('hello ' + 'world ') Raymond From guido at python.org Tue Apr 10 20:50:30 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Apr 2007 11:50:30 -0700 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: References: Message-ID: On 4/10/07, Steven Bethard wrote: > On 4/10/07, Guido van Rossum wrote: > > As I stated in my PyCon keynote and repeated (though somewhat hidden) > > in PEP 3000, all PEPs for Py3k are due by April. The end of April is > > only a few weeks away, so now is the time to start drafting that PEP > > if you care about Python 3000! This is so that we will be able to > > concentrate on *implementing* the accepted proposals by June, in time > > for the planned release 3.0a1. > > Does this also include proposals for parts of the Python 3000 stdlib, > e.g. changing or replacing a particular module API? Good question. I think it is wholly unrealistic to expect much movement in the stdlib before July; let's plan to do the stdlib upgrade after 3.0a1 is released, and exempt it from the PEP deadline. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Apr 10 20:53:27 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Apr 2007 11:53:27 -0700 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> Message-ID: On 4/10/07, Raymond Hettinger wrote: > Thanks for the reminder. I expect to write two: > > * PEP to eliminate __del__ in favor of weakref idioms > > * PEP on simpler alternative to abstract-base-classes > suggesting an occasional method special attribute > (so for instance a __getitem__ method can tell you > whether it thinks it is a sequence or mapping). > > Here's that I think might not need a PEP: > > * Eliminate implicit string concatenation: "abc" "def" > in favor of an explicit + operation. That simplifies > the grammar just a bit and the compiler already is > smart enough to do constant fold this operation at > compile time. When there are multi-line concats, I think > the parenthesis serves us much better than a trailing \ > which is ugly and relies on having no trailing whitespace. > Replace: > 'hello ' \ > 'world ' > With: > ('hello ' + > 'world ') That needs a PEP too; there are subtle issues like ('hello %s' 'world' % 42) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Tue Apr 10 21:03:46 2007 From: brett at python.org (Brett Cannon) Date: Tue, 10 Apr 2007 12:03:46 -0700 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: References: Message-ID: On 4/10/07, Guido van Rossum wrote: > > As I stated in my PyCon keynote and repeated (though somewhat hidden) > in PEP 3000, all PEPs for Py3k are due by April. The end of April is > only a few weeks away, so now is the time to start drafting that PEP > if you care about Python 3000! This is so that we will be able to > concentrate on *implementing* the accepted proposals by June, in time > for the planned release 3.0a1. Just so people know what I have in the pipeline, I have two PEPs planned: 1. Add a built-in named __main__ so as to fix the problem of executing a script that exists in a package and uses new relative imports (see http://sayspy.blogspot.com/2007/03/possible-change-to-handle-main-code-in.htmlfor a more detailed explanation). 2. Remove file.__init__, code.__init__, and object.__subclasses__ for security reasons. I am trying to get approval from my supervisor to make public a doc I wrote for a class that explains my security work that I presented on at PyCon to vet the design one more time on python-dev before I push for these removals. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070410/ac79adb7/attachment.htm From brett at python.org Tue Apr 10 21:05:06 2007 From: brett at python.org (Brett Cannon) Date: Tue, 10 Apr 2007 12:05:06 -0700 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> Message-ID: On 4/10/07, Raymond Hettinger wrote: > > Thanks for the reminder. I expect to write two: > > * PEP to eliminate __del__ in favor of weakref idioms Woohoo! And I still expect a named tuple implementation in the collections module. =) -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070410/630e3297/attachment.html From python at rcn.com Tue Apr 10 21:05:18 2007 From: python at rcn.com (Raymond Hettinger) Date: Tue, 10 Apr 2007 15:05:18 -0400 (EDT) Subject: [Python-3000] Fwd: Re: Reminder: Py3k PEPs due by April Message-ID: <20070410150518.BEU90548@ms09.lnh.mail.rcn.net> > there are subtle issues like > > ('hello %s' > 'world' % 42) Thank goodness the % formatting operator is going away. Having the same precendence as the modulo-operation has not been a good thing. Raymond -------------- next part -------------- An embedded message was scrubbed... From: "Guido van Rossum" Subject: Re: [Python-3000] Reminder: Py3k PEPs due by April Date: Tue, 10 Apr 2007 11:53:27 -0700 Size: 3647 Url: http://mail.python.org/pipermail/python-3000/attachments/20070410/a9c63f7b/attachment.eml From pje at telecommunity.com Tue Apr 10 21:23:22 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 10 Apr 2007 15:23:22 -0400 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: References: Message-ID: <5.1.1.6.0.20070410151803.02c2e440@sparrow.telecommunity.com> At 12:03 PM 4/10/2007 -0700, Brett Cannon wrote: >2. Remove file.__init__, code.__init__, and object.__subclasses__ for >security reasons. The __subclasses__ method is useful -- even more so in 3.x than in 2.x, because in 3.x there are no classic classes. I was planning to make use of this in my generic function libraries to allow automatic checking for completeness and ambiguity of a ruleset, given the classes that are currently defined. Currently, this is limited by the inability to access classic classes' subclasses, but in 3.x __subclasses__ is an ideal way to locate active classes. From brett at python.org Tue Apr 10 21:25:29 2007 From: brett at python.org (Brett Cannon) Date: Tue, 10 Apr 2007 12:25:29 -0700 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: <5.1.1.6.0.20070410151803.02c2e440@sparrow.telecommunity.com> References: <5.1.1.6.0.20070410151803.02c2e440@sparrow.telecommunity.com> Message-ID: On 4/10/07, Phillip J. Eby wrote: > > At 12:03 PM 4/10/2007 -0700, Brett Cannon wrote: > >2. Remove file.__init__, code.__init__, and object.__subclasses__ for > >security reasons. > > The __subclasses__ method is useful -- even more so in 3.x than in 2.x, > because in 3.x there are no classic classes. > > I was planning to make use of this in my generic function libraries to > allow automatic checking for completeness and ambiguity of a ruleset, > given > the classes that are currently defined. Currently, this is limited by the > inability to access classic classes' subclasses, but in 3.x __subclasses__ > is an ideal way to locate active classes. > > The problem with it, though, is it does expose *so* much. If I have a class defined that is not exposed within the interpreter but is live in the process, I can still get to it with this method. That's a problem if the class has a security component to it where there is a class attribute that should not be exposed within the interpreter. This can be discussed more if/when I get my security paper out in the public and get a PEP written to actually push for the removals. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070410/be95fe46/attachment.html From steven.bethard at gmail.com Tue Apr 10 21:31:45 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 10 Apr 2007 13:31:45 -0600 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> Message-ID: On 4/10/07, Brett Cannon wrote: > And I still expect a named tuple implementation in the collections module. Fortunately, Raymond borrowed the time machine already:: Python 2.6a0 (trunk:54711M, Apr 6 2007, 23:48:33) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import collections >>> collections.NamedTuple Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From pje at telecommunity.com Tue Apr 10 22:47:44 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 10 Apr 2007 16:47:44 -0400 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: References: <5.1.1.6.0.20070410151803.02c2e440@sparrow.telecommunity.com> <5.1.1.6.0.20070410151803.02c2e440@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070410164341.02c47678@sparrow.telecommunity.com> At 12:25 PM 4/10/2007 -0700, Brett Cannon wrote: >On 4/10/07, Phillip J. Eby ><pje at telecommunity.com> wrote: >>At 12:03 PM 4/10/2007 -0700, Brett Cannon wrote: >> >2. Remove file.__init__, code.__init__, and object.__subclasses__ for >> >security reasons. >> >>The __subclasses__ method is useful -- even more so in 3.x than in 2.x, >>because in 3.x there are no classic classes. >> >>I was planning to make use of this in my generic function libraries to >>allow automatic checking for completeness and ambiguity of a ruleset, given >>the classes that are currently defined. Currently, this is limited by the >>inability to access classic classes' subclasses, but in 3.x __subclasses__ >>is an ideal way to locate active classes. > >The problem with it, though, is it does expose *so* much. From my POV, that's a feature, not a bug. :) > If I have a class defined that is not exposed within the interpreter > but is live in the process, I can still get to it with this > method. That's a problem if the class has a security component to it > where there is a class attribute that should not be exposed within the > interpreter. > >This can be discussed more if/when I get my security paper out in the >public and get a PEP written to actually push for the removals. If you're using a capabilities-based system (or a multiparadigm system like Zope 3's zope.security), it's easy enough to restrict access to special attributes like __subclasses__ (or for that matter, any other sensitive attributes), so I don't see why the feature itself needs to be removed. From brett at python.org Tue Apr 10 23:27:21 2007 From: brett at python.org (Brett Cannon) Date: Tue, 10 Apr 2007 14:27:21 -0700 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> Message-ID: On 4/10/07, Steven Bethard wrote: > > On 4/10/07, Brett Cannon wrote: > > And I still expect a named tuple implementation in the collections > module. > > Fortunately, Raymond borrowed the time machine already:: > > Python 2.6a0 (trunk:54711M, Apr 6 2007, 23:48:33) [MSC v.1310 32 bit > (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import collections > >>> collections.NamedTuple > Ah, now I see the NEWS entry for it. I guess my expectations have been met then. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070410/f91d84bb/attachment.htm From jcarlson at uci.edu Tue Apr 10 23:47:43 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 10 Apr 2007 14:47:43 -0700 Subject: [Python-3000] Fwd: Re: Reminder: Py3k PEPs due by April In-Reply-To: <20070410150518.BEU90548@ms09.lnh.mail.rcn.net> References: <20070410150518.BEU90548@ms09.lnh.mail.rcn.net> Message-ID: <20070410143944.62A5.JCARLSON@uci.edu> Raymond Hettinger wrote: > > there are subtle issues like > > > > ('hello %s' > > 'world' % 42) > > Thank goodness the % formatting operator is going away. Having the > same precendence as the modulo-operation has not been a good thing. Did I miss a PEP somewhere? - Josiah From greg.ewing at canterbury.ac.nz Wed Apr 11 02:24:39 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 11 Apr 2007 12:24:39 +1200 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> Message-ID: <461C2AC7.4090908@canterbury.ac.nz> Raymond Hettinger wrote: > Here's that I think might not need a PEP: > > * Eliminate implicit string concatenation: "abc" "def" > in favor of an explicit + operation. -0.707. I've used languages where the only way of splitting a string over lines is to use concatenation, and it's tedious. > When there are multi-line concats, I think > the parenthesis serves us much better than a trailing \ Which can be done perfectly well as things are: ('hello ' 'world ') So no, I don't think this is uncontroversial enough to get by without a PEP. -- Greg From greg.ewing at canterbury.ac.nz Wed Apr 11 02:38:03 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 11 Apr 2007 12:38:03 +1200 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: <5.1.1.6.0.20070410151803.02c2e440@sparrow.telecommunity.com> References: <5.1.1.6.0.20070410151803.02c2e440@sparrow.telecommunity.com> Message-ID: <461C2DEB.4070907@canterbury.ac.nz> Phillip J. Eby wrote: > The __subclasses__ method is useful -- even more so in 3.x than in 2.x, > because in 3.x there are no classic classes. For security purposes, I think it would be better to adopt a principle that it shouldn't be possible to do anything dangerous merely by instantiating a class (e.g. open() opens files, but file() doesn't). Then __subclasses__ would be harmless. -- Greg From greg.ewing at canterbury.ac.nz Wed Apr 11 02:54:01 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 11 Apr 2007 12:54:01 +1200 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: References: Message-ID: <461C31A9.5090204@canterbury.ac.nz> Guido van Rossum wrote: > all PEPs for Py3k are due by April. Can we nominate one of our 2.x PEPs for promotion to a 3000 PEP? If so, I'd like to nominate PEP 335 - Overloadable Boolean Operators. -- Greg From brett at python.org Wed Apr 11 03:22:56 2007 From: brett at python.org (Brett Cannon) Date: Tue, 10 Apr 2007 18:22:56 -0700 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: <461C2DEB.4070907@canterbury.ac.nz> References: <5.1.1.6.0.20070410151803.02c2e440@sparrow.telecommunity.com> <461C2DEB.4070907@canterbury.ac.nz> Message-ID: On 4/10/07, Greg Ewing wrote: > > Phillip J. Eby wrote: > > > The __subclasses__ method is useful -- even more so in 3.x than in 2.x, > > because in 3.x there are no classic classes. > > For security purposes, I think it would be better to > adopt a principle that it shouldn't be possible to > do anything dangerous merely by instantiating a > class (e.g. open() opens files, but file() doesn't). > Then __subclasses__ would be harmless. True. As long as you don't use class attributes or have dangerous default arguments (and that is the kicker usually) then that policy is doable. And I should clarify that the method does not need to disappear as it can move to an extension module somewhere. I just don't want it exposed right off of object and thus in the built-in namespace. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070410/cfbcd37e/attachment.html From guido at python.org Wed Apr 11 06:50:19 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Apr 2007 21:50:19 -0700 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: <461C31A9.5090204@canterbury.ac.nz> References: <461C31A9.5090204@canterbury.ac.nz> Message-ID: On 4/10/07, Greg Ewing wrote: > Guido van Rossum wrote: > > all PEPs for Py3k are due by April. > > Can we nominate one of our 2.x PEPs for promotion > to a 3000 PEP? > > If so, I'd like to nominate PEP 335 - Overloadable > Boolean Operators. Sure. That looks like something I recall intending to do and never getting to. Or maybe I'm imagining it. Anyway, it sounds like a good idea if it can be done efficiently. I'm kind of worried about the performance though -- won't you have to pay at least for the lookup of __and1__ or __or1__ in the MRO of the first argument? Also, given the plans for backporting significant portions of 3.0 to 2.6, why not propose this for 2.6? I see no incompatibilities that would prevent this. If it is checked in soon enough it should be straightforward to merge it into 3.0. Backporting after first implementing it in 3.0 would be more work. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From walter at livinglogic.de Wed Apr 11 10:10:20 2007 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Wed, 11 Apr 2007 10:10:20 +0200 Subject: [Python-3000] [Python-3000-checkins] r54742 - in python/branches/p3yk/Lib: io.py test/test_io.py In-Reply-To: <20070411010907.226411E400D@bag.python.org> References: <20070411010907.226411E400D@bag.python.org> Message-ID: <461C97EC.8040903@livinglogic.de> guido.van.rossum wrote: > Author: guido.van.rossum > Date: Wed Apr 11 03:09:03 2007 > New Revision: 54742 > > Modified: > python/branches/p3yk/Lib/io.py > python/branches/p3yk/Lib/test/test_io.py > Log: > Checkpoint so I can continue to work on this at a different box. > There is somewhat working (but slow) code supporting seek/tell for text files, > but extensive testing exposes a bug I can't nail down. > > > Modified: python/branches/p3yk/Lib/io.py > [...] > + decoder_state = pickle.dumps(self._decoder, 2) Would it make sense to make the state of the decoder public, e.g. by adding setstate() and getstate() methods? This would give a cleaner API. Should I work on a patch? Servus, Walter From guido at python.org Wed Apr 11 16:47:25 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 11 Apr 2007 07:47:25 -0700 Subject: [Python-3000] [Python-3000-checkins] r54742 - in python/branches/p3yk/Lib: io.py test/test_io.py In-Reply-To: <461C97EC.8040903@livinglogic.de> References: <20070411010907.226411E400D@bag.python.org> <461C97EC.8040903@livinglogic.de> Message-ID: On 4/11/07, Walter D?rwald wrote: > Would it make sense to make the state of the decoder public, e.g. by > adding setstate() and getstate() methods? This would give a cleaner API. I've been thinking of the same thing! I wonder if it would be possible to return the state as a pair (unread, flags) where unread is a (byte) string of unprocessed bytes and flags is some other state, with the constraint that in the initial state the flags must be zero. Then I can optimize the case where flags is returned as zero by subtracting len(unread) from the current position and that'd be the correct seek position. I imagine most decoders have only very few flags they care about. (The worst might be the utf-16 decoder which must have a flag to remember whether it already saw a byte order marker, and another indicating the byte order. Maybe we'll have to special-case that one, so don't worry too much about it.) > Should I work on a patch? That would be great! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From talin at acm.org Wed Apr 11 17:01:45 2007 From: talin at acm.org (Talin) Date: Wed, 11 Apr 2007 08:01:45 -0700 Subject: [Python-3000] Line continuation using ellipsis Message-ID: <461CF859.6050003@acm.org> I'm not sure that I am the first to bring this up, but I might as well get it written down. I've never liked using the backslash operator for line continuation, for two reasons: 1) It requires that it be the last character of a line, meaning that you have to insure that there's no trailing whitespace, which can be difficult with some editors. 2) As a wise sage once said, "it looks like line noise on my monitor". I'm not proposing to remove the feature, however I'd like to see an alternative method for declaring statements that cross a line boundary. I seem to recall at one point someone suggesting the use of ellipsis, which makes a lot of sense to me: sorted_result = partition_lower( input_list, pivot ) ... + pivot ... + partition_upper( input_list, pivot ) Parsing-wise, an ellipsis operator would have the following meaning: Inside a parenthesized or bracketed expression, it would have the same meaning it does now. Otherwise, it has the meaning "consume all whitespace, including newlines". -- Talin From fdrake at acm.org Wed Apr 11 17:30:29 2007 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Wed, 11 Apr 2007 11:30:29 -0400 Subject: [Python-3000] Line continuation using ellipsis In-Reply-To: <461CF859.6050003@acm.org> References: <461CF859.6050003@acm.org> Message-ID: <200704111130.30019.fdrake@acm.org> On Wednesday 11 April 2007 11:01, Talin wrote: > Parsing-wise, an ellipsis operator would have the following meaning: > Inside a parenthesized or bracketed expression, it would have the same > meaning it does now. Otherwise, it has the meaning "consume all > whitespace, including newlines". Really? What about this: ----------------------------------- print "strings are cool", ... "really!" ----------------------------------- (Note that there are two blank lines between the two non-blank lines.) Would this be a single print statement, or a print statement and an expression statement? -Fred -- Fred L. Drake, Jr. From pje at telecommunity.com Wed Apr 11 18:04:58 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 11 Apr 2007 12:04:58 -0400 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: References: <461C31A9.5090204@canterbury.ac.nz> <461C31A9.5090204@canterbury.ac.nz> Message-ID: <5.1.1.6.0.20070411115258.02ac4588@sparrow.telecommunity.com> At 09:50 PM 4/10/2007 -0700, Guido van Rossum wrote: >I'm kind of worried about the performance though -- won't you have to >pay at least for the lookup of __and1__ or __or1__ in the MRO of the >first argument? Not if they're tp_* slots. And we could introduce the feature only for new-style classes, thereby preventing any performance hit for existing classic classes. (Classic classes' tp_and1 et al would simply invoke normal boolean testing.) What I find interesting about this proposal is that it comes very close to allowing abstract interpretation of arbitrary Python expressions. If there were a couple of other operator pairs like 'if1/if2', you'd be able to take a generator expression and transform it to SQL without parsing or bytecode decompiling (the two techniques currently used by today's Python ORM's like DejaVu). Unfortunately, to get full abstract interpretation (ala PyPy, but without looking at any bytecode), there would need to be a way to handle try/except/finally and while loops, and I don't have any ideas about how one would implement those. From g.brandl at gmx.net Wed Apr 11 18:08:36 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 11 Apr 2007 18:08:36 +0200 Subject: [Python-3000] Line continuation using ellipsis In-Reply-To: <461CF859.6050003@acm.org> References: <461CF859.6050003@acm.org> Message-ID: Talin schrieb: > I'm not sure that I am the first to bring this up, but I might as well > get it written down. > > I've never liked using the backslash operator for line continuation, for > two reasons: > > 1) It requires that it be the last character of a line, meaning that you > have to insure that there's no trailing whitespace, which can be > difficult with some editors. Then use parentheses. That should be possible almost everywhere. Also, if we have to have a line-continuation token, the backslash is a good choice because it's present in so many other languages. > 2) As a wise sage once said, "it looks like line noise on my monitor". But it's end-of-line noise ;) > I'm not proposing to remove the feature, however I'd like to see an > alternative method for declaring statements that cross a line boundary. > I seem to recall at one point someone suggesting the use of ellipsis, > which makes a lot of sense to me: > > sorted_result = partition_lower( input_list, pivot ) ... > + pivot ... > + partition_upper( input_list, pivot ) > > Parsing-wise, an ellipsis operator would have the following meaning: > Inside a parenthesized or bracketed expression, it would have the same > meaning it does now. Otherwise, it has the meaning "consume all > whitespace, including newlines". This doesn't work with Py3k as the ellipsis is a general expression element there. You can already write ... and it will be short for Ellipsis cheers, Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From eoghan at qatano.com Wed Apr 11 18:42:08 2007 From: eoghan at qatano.com (Eoghan Murray) Date: Wed, 11 Apr 2007 17:42:08 +0100 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> Message-ID: On 10/04/07, Raymond Hettinger wrote: > > [snip] > > Here's that I think might not need a PEP: > > * Eliminate implicit string concatenation: "abc" "def" > in favor of an explicit + operation. That simplifies > the grammar just a bit and the compiler already is > smart enough to do constant fold this operation at > compile time. When there are multi-line concats, I think > the parenthesis serves us much better than a trailing \ > which is ugly and relies on having no trailing whitespace. > Replace: > 'hello ' \ > 'world ' > With: > ('hello ' + > 'world ') > > > > Raymond > ______________________________________________ I've started a discussion on python-ideas proposing to allow implicit concatenation of string variables as well as string literals. -1 for elimination! Eoghan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070411/764822d5/attachment.html From jason.orendorff at gmail.com Wed Apr 11 19:03:52 2007 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Wed, 11 Apr 2007 13:03:52 -0400 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> Message-ID: On 4/10/07, Raymond Hettinger wrote: > * Eliminate implicit string concatenation: "abc" "def" > in favor of an explicit + operation. That simplifies > the grammar just a bit and the compiler already is > smart enough to do constant fold this operation at > compile time. [...] My gut instinct was "whoa, don't change that". But: I think this behavior comes from C, where it's useful mainly because of the preprocessor. In Python it's not as useful. And C experience is not as universal as it once was. I bet this looks pretty confusing the first time you run into it. Sure, I use it, but if it went away, I would type the plus sign. Not a problem. And it would be one less thing for newcomers to learn, and explicit is better, right? Maybe this little slice of the complexity budget could be better invested elsewhere. -j From pje at telecommunity.com Wed Apr 11 19:17:39 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 11 Apr 2007 13:17:39 -0400 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> Message-ID: <5.1.1.6.0.20070411131550.02b985c8@sparrow.telecommunity.com> At 01:03 PM 4/11/2007 -0400, Jason Orendorff wrote: >On 4/10/07, Raymond Hettinger wrote: > > * Eliminate implicit string concatenation: "abc" "def" > > in favor of an explicit + operation. That simplifies > > the grammar just a bit and the compiler already is > > smart enough to do constant fold this operation at > > compile time. [...] > >My gut instinct was "whoa, don't change that". But: > >I think this behavior comes from C, where it's useful mainly because >of the preprocessor. In Python it's not as useful. And C experience >is not as universal as it once was. I bet this looks pretty confusing >the first time you run into it. > >Sure, I use it, but if it went away, I would type the plus sign. Not >a problem. And it would be one less thing for newcomers to learn, and >explicit is better, right? But there's another Python principle here, I think... complexity of computation should be represented by complexity of syntax. We don't generally like to use properties for expensive computation, or methods for simple field access, for example. Putting in a '+' sign makes the code *feel* like there's more computation going on, even if the computation gets optimized away. From jason.orendorff at gmail.com Wed Apr 11 19:53:22 2007 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Wed, 11 Apr 2007 13:53:22 -0400 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: <5.1.1.6.0.20070411131550.02b985c8@sparrow.telecommunity.com> References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> <5.1.1.6.0.20070411131550.02b985c8@sparrow.telecommunity.com> Message-ID: On 4/11/07, Phillip J. Eby wrote: > But there's another Python principle here, I think... complexity of > computation should be represented by complexity of syntax. We don't > generally like to use properties for expensive computation, or methods for > simple field access, for example. Putting in a '+' sign makes the code > *feel* like there's more computation going on, even if the computation gets > optimized away. Hmm. It sounds like what you're saying is, if we're going to do string constant folding, then it makes sense to have a special case in the grammar to reflect that visually. It's a nice principle, but I wouldn't take it that far. -j From steven.bethard at gmail.com Wed Apr 11 20:00:34 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 11 Apr 2007 12:00:34 -0600 Subject: [Python-3000] Line continuation using ellipsis In-Reply-To: References: <461CF859.6050003@acm.org> Message-ID: On 4/11/07, Georg Brandl wrote: > Talin schrieb: > > I'm not sure that I am the first to bring this up, but I might as well > > get it written down. > > > > I've never liked using the backslash operator for line continuation, for > > two reasons: > > > > 1) It requires that it be the last character of a line, meaning that you > > have to insure that there's no trailing whitespace, which can be > > difficult with some editors. > > Then use parentheses. That should be possible almost everywhere. Yes, please. There's currently no need for the backslash in pretty much anything but triple-quoted strings. Since they're not needed now, I see no reason to introduce a replacement for them. If you're going to propose something, propose removing them. Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From bwinton at latte.ca Wed Apr 11 20:34:15 2007 From: bwinton at latte.ca (Blake Winton) Date: Wed, 11 Apr 2007 14:34:15 -0400 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: <5.1.1.6.0.20070411131550.02b985c8@sparrow.telecommunity.com> References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> <5.1.1.6.0.20070411131550.02b985c8@sparrow.telecommunity.com> Message-ID: <461D2A27.8060200@latte.ca> Phillip J. Eby wrote: >>> * Eliminate implicit string concatenation: "abc" "def" >> Sure, I use it, but if it went away, I would type the plus sign. Not >> a problem. And it would be one less thing for newcomers to learn, and >> explicit is better, right? > But there's another Python principle here, I think... complexity of > computation should be represented by complexity of syntax. We don't > generally like to use properties for expensive computation, or methods for > simple field access, for example. Putting in a '+' sign makes the code > *feel* like there's more computation going on, even if the computation gets > optimized away. Given that the set of things the compiler can optimize grows at a faster rate than the syntax changes to the language, I'm not sure that that's really a principle. Yeah, sure, you don't want to hide, say, a reverse-dns-lookup-with-associated-timeout behind the creation of a socket (to take a horrible example from Java that's just finished biting me), but + isn't really a heavyweight operator, especially if you're thinking of (small-ish) integers, and if the compiler optimizes it into less than one instruction, then great. I suppose I see the + case for strings, in particular, as being more similar to the + case for numbers than calling methods or properties... Later, Blake. From greg.ewing at canterbury.ac.nz Thu Apr 12 00:55:55 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 12 Apr 2007 10:55:55 +1200 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> <5.1.1.6.0.20070411131550.02b985c8@sparrow.telecommunity.com> Message-ID: <461D677B.7000604@canterbury.ac.nz> Jason Orendorff wrote: > Hmm. It sounds like what you're saying is, if we're going to do > string constant folding, then it makes sense to have a special case in > the grammar to reflect that visually. I think it's more a matter of wanting to write things declaratively rather than procedurally. You want to say "print this long string" rather than "join these strings together and print the result". -- Greg From walter at livinglogic.de Thu Apr 12 10:10:13 2007 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu, 12 Apr 2007 10:10:13 +0200 Subject: [Python-3000] [Python-3000-checkins] r54742 - in python/branches/p3yk/Lib: io.py test/test_io.py In-Reply-To: References: <20070411010907.226411E400D@bag.python.org> <461C97EC.8040903@livinglogic.de> Message-ID: <461DE965.3050302@livinglogic.de> Guido van Rossum wrote: > On 4/11/07, Walter D?rwald wrote: >> Would it make sense to make the state of the decoder public, e.g. by >> adding setstate() and getstate() methods? This would give a cleaner API. > > I've been thinking of the same thing! > > I wonder if it would be possible to return the state as a pair > (unread, flags) where unread is a (byte) string of unprocessed bytes > and flags is some other state, with the constraint that in the initial > state the flags must be zero. Then I can optimize the case where flags > is returned as zero by subtracting len(unread) from the current > position and that'd be the correct seek position. I'd say that bytestream.tell() is the correct position. Or should seek() return to the last position where the codec was in a default state without anything buffered? (This can't work for UTF-16, because the codec almost never is in the default state.) > I imagine most > decoders have only very few flags they care about. (The worst might be > the utf-16 decoder which must have a flag to remember whether it > already saw a byte order marker, and another indicating the byte > order. Maybe we'll have to special-case that one, so don't worry too > much about it.) > >> Should I work on a patch? > > That would be great! OK, here's the patch: http://bugs.python.org/1698994 The state returned from getstate() should be treated as an opaque value (e.g. for the buffered incremental codecs it is the buffered string, for the UTF-16 encoder it's the flag indicating whether a BOM has been written etc.). The codecs try to return None, if they are in some kind of default state (e.g. there's nothing buffered). I'm going to add tests next. Servus, Walter From python at rcn.com Thu Apr 12 18:04:40 2007 From: python at rcn.com (Raymond Hettinger) Date: Thu, 12 Apr 2007 12:04:40 -0400 (EDT) Subject: [Python-3000] Marshal Version and Pickle Protocols Message-ID: <20070412120440.BFD12018@ms09.lnh.mail.rcn.net> Can we drop the old protocols? Raymond From guido at python.org Thu Apr 12 18:57:26 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Apr 2007 09:57:26 -0700 Subject: [Python-3000] Marshal Version and Pickle Protocols In-Reply-To: <20070412120440.BFD12018@ms09.lnh.mail.rcn.net> References: <20070412120440.BFD12018@ms09.lnh.mail.rcn.net> Message-ID: On 4/12/07, Raymond Hettinger wrote: > Can we drop the old protocols? I'd like to keep supporting the old pickle for data interchangeability, but default to pickle protocol 2, which has been supported by Python for several versions. I'm not aware of Marshal protocol versions. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fumanchu at amor.org Thu Apr 12 19:25:05 2007 From: fumanchu at amor.org (Robert Brewer) Date: Thu, 12 Apr 2007 10:25:05 -0700 Subject: [Python-3000] Marshal Version and Pickle Protocols In-Reply-To: Message-ID: <435DF58A933BA74397B42CDEB8145A860AF33CE8@ex9.hostedexchange.local> Guido van Rossum wrote: > On 4/12/07, Raymond Hettinger wrote: > > Can we drop the old protocols? > > I'd like to keep supporting the old pickle for data > interchangeability, but default to pickle protocol 2, which has been > supported by Python for several versions. I still use pickle protocol 0, since it does not output \x00 like protocols 1 and 2. Some databases either don't provide a way to escape \x00 in SQL statements, or have nasty bugs when comparing or operating on string values with \x00 in them. Robert Brewer System Architect Amor Ministries fumanchu at amor.org From barry at python.org Thu Apr 12 19:26:20 2007 From: barry at python.org (Barry Warsaw) Date: Thu, 12 Apr 2007 13:26:20 -0400 Subject: [Python-3000] Marshal Version and Pickle Protocols In-Reply-To: References: <20070412120440.BFD12018@ms09.lnh.mail.rcn.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 12, 2007, at 12:57 PM, Guido van Rossum wrote: > On 4/12/07, Raymond Hettinger wrote: >> Can we drop the old protocols? > > I'd like to keep supporting the old pickle for data > interchangeability, but default to pickle protocol 2, which has been > supported by Python for several versions. +1, certainly for reading. I wouldn't care too much if only protocol 2 is supported for writing new pickles, but I'm probably -0 on removing protocol 0 and 1 for writing. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRh5rw3EjvBPtnXfVAQIMuAP+MzSVxI8/23FsK4xb+TqnLqJgF6r6HLXQ NToWp/WyWgIeAe9eP2g/zyItRF+WxNGG26wtY01sa5JH0d9SUYQ1E1XMug8PAyg+ 53apmhd+pJe2MhD5R+KRbwc6i3LloDl/b7eef7HqpFXWdavD2P7OFPakMR+tzGks pi+agCJWjPM= =DJq4 -----END PGP SIGNATURE----- From guido at python.org Thu Apr 12 19:30:04 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Apr 2007 10:30:04 -0700 Subject: [Python-3000] [Python-3000-checkins] r54742 - in python/branches/p3yk/Lib: io.py test/test_io.py In-Reply-To: <461DE965.3050302@livinglogic.de> References: <20070411010907.226411E400D@bag.python.org> <461C97EC.8040903@livinglogic.de> <461DE965.3050302@livinglogic.de> Message-ID: On 4/12/07, Walter D?rwald wrote: > Guido van Rossum wrote: > > On 4/11/07, Walter D?rwald wrote: > >> Would it make sense to make the state of the decoder public, e.g. by > >> adding setstate() and getstate() methods? This would give a cleaner API. > > > > I've been thinking of the same thing! > > > > I wonder if it would be possible to return the state as a pair > > (unread, flags) where unread is a (byte) string of unprocessed bytes > > and flags is some other state, with the constraint that in the initial > > state the flags must be zero. Then I can optimize the case where flags > > is returned as zero by subtracting len(unread) from the current > > position and that'd be the correct seek position. > > I'd say that bytestream.tell() is the correct position. > > Or should seek() return to the last position where the codec was in a > default state without anything buffered? (This can't work for UTF-16, > because the codec almost never is in the default state.) That was my hope, yes (and I realize that UTF-16 is an exception). Consider UTF-8 though. If the chunk we read from the byte stream ended in the middle of a multi-byte character, the codec will have the first part of that character buffered. In general we want to subtract buffered data from the byte stream's position when reporting the position of the text stream. The idea is that if we later seek to the reported position, we should be reading the same character data. This can be accomplished in two ways: by backing up the byte stream to the previous character boundary, and resetting the decoder to neutral; or by positioning the byte stream to where it was originally and setting the state of the decoder to what it was before. However, backing up the byte stream has the advantage that no decoder state needs to be encoded in the position cookie. > > I imagine most > > decoders have only very few flags they care about. (The worst might be > > the utf-16 decoder which must have a flag to remember whether it > > already saw a byte order marker, and another indicating the byte > > order. Maybe we'll have to special-case that one, so don't worry too > > much about it.) > > > >> Should I work on a patch? > > > > That would be great! > > OK, here's the patch: http://bugs.python.org/1698994 > > The state returned from getstate() should be treated as an opaque value > (e.g. for the buffered incremental codecs it is the buffered string, for > the UTF-16 encoder it's the flag indicating whether a BOM has been > written etc.). The codecs try to return None, if they are in some kind > of default state (e.g. there's nothing buffered). I would like to await completion of those unit tests; there seem to be some subtle issues. I wonder if setstate() should call self.reset() first. I'd also like to ask if setstate() could default to "" only if the argument is None, not if it is empty; I'd like to use it to change the buffer to be a bytes object. (And yes, I need to maintain more hacks for that, alas). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Apr 12 19:34:52 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Apr 2007 10:34:52 -0700 Subject: [Python-3000] Marshal Version and Pickle Protocols In-Reply-To: References: <20070412120440.BFD12018@ms09.lnh.mail.rcn.net> Message-ID: On 4/12/07, Barry Warsaw wrote: > On Apr 12, 2007, at 12:57 PM, Guido van Rossum wrote: > > > On 4/12/07, Raymond Hettinger wrote: > >> Can we drop the old protocols? > > > > I'd like to keep supporting the old pickle for data > > interchangeability, but default to pickle protocol 2, which has been > > supported by Python for several versions. > > +1, certainly for reading. Duh. For reading you don't need to specify a protocol, it finds out based on what it reads. The default is only necessary for writing. > I wouldn't care too much if only protocol > 2 is supported for writing new pickles, but I'm probably -0 on > removing protocol 0 and 1 for writing. That is exactly my proposal. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Thu Apr 12 19:38:57 2007 From: barry at python.org (Barry Warsaw) Date: Thu, 12 Apr 2007 13:38:57 -0400 Subject: [Python-3000] Marshal Version and Pickle Protocols In-Reply-To: References: <20070412120440.BFD12018@ms09.lnh.mail.rcn.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 12, 2007, at 1:34 PM, Guido van Rossum wrote: > On 4/12/07, Barry Warsaw wrote: >> On Apr 12, 2007, at 12:57 PM, Guido van Rossum wrote: >> >> > On 4/12/07, Raymond Hettinger wrote: >> >> Can we drop the old protocols? >> > >> > I'd like to keep supporting the old pickle for data >> > interchangeability, but default to pickle protocol 2, which has >> been >> > supported by Python for several versions. >> >> +1, certainly for reading. > > Duh. For reading you don't need to specify a protocol, it finds out > based on what it reads. The default is only necessary for writing. I was actually responding to what I thought Raymond was suggesting, but never mind, because I agree with you! >> I wouldn't care too much if only protocol >> 2 is supported for writing new pickles, but I'm probably -0 on >> removing protocol 0 and 1 for writing. > > That is exactly my proposal. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRh5usnEjvBPtnXfVAQIWTAQApdlQAEORnFF3G6qswpFECC34EH+IsnVA kqr1m/PlpLLxAc7PBKtUMuanxHb/i2XjGwkB2d9DOsya9vTg4KNAyDCIC9rFMkz2 B7db40JfyA63tIekZaiOO4llYIr18VFQHxy0Uuo1aQFKQ3vqyDIuS2k0EZ59c+wq oJzUT2JlqbU= =hHUb -----END PGP SIGNATURE----- From walter at livinglogic.de Thu Apr 12 21:32:08 2007 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu, 12 Apr 2007 21:32:08 +0200 Subject: [Python-3000] [Python-3000-checkins] r54742 - in python/branches/p3yk/Lib: io.py test/test_io.py In-Reply-To: References: <20070411010907.226411E400D@bag.python.org> <461C97EC.8040903@livinglogic.de> <461DE965.3050302@livinglogic.de> Message-ID: <461E8938.80304@livinglogic.de> Guido van Rossum wrote: > On 4/12/07, Walter D?rwald wrote: >> Guido van Rossum wrote: >> > On 4/11/07, Walter D?rwald wrote: >> >> Would it make sense to make the state of the decoder public, e.g. by >> >> adding setstate() and getstate() methods? This would give a cleaner >> API. >> > >> > I've been thinking of the same thing! >> > >> > I wonder if it would be possible to return the state as a pair >> > (unread, flags) where unread is a (byte) string of unprocessed bytes >> > and flags is some other state, with the constraint that in the initial >> > state the flags must be zero. Then I can optimize the case where flags >> > is returned as zero by subtracting len(unread) from the current >> > position and that'd be the correct seek position. >> >> I'd say that bytestream.tell() is the correct position. >> >> Or should seek() return to the last position where the codec was in a >> default state without anything buffered? (This can't work for UTF-16, >> because the codec almost never is in the default state.) > > That was my hope, yes (and I realize that UTF-16 is an exception). We could designate natural endianness as the default state, but that would mean that a codec state can't be transferred to a different machine (or we could declare little (or big) endianness to be the default state). > Consider UTF-8 though. If the chunk we read from the byte stream ended > in the middle of a multi-byte character, the codec will have the first > part of that character buffered. In general we want to subtract > buffered data from the byte stream's position when reporting the > position of the text stream. The idea is that if we later seek to the > reported position, we should be reading the same character data. This > can be accomplished in two ways: by backing up the byte stream to the > previous character boundary, and resetting the decoder to neutral; or > by positioning the byte stream to where it was originally and setting > the state of the decoder to what it was before. However, backing up > the byte stream has the advantage that no decoder state needs to be > encoded in the position cookie. OK, so for decoders getstate() should always return a tuple, with the first entry being the buffered byte string (or bytes object?) and the second being additional state data. Do we need any specification for encoders? >> > I imagine most >> > decoders have only very few flags they care about. (The worst might be >> > the utf-16 decoder which must have a flag to remember whether it >> > already saw a byte order marker, and another indicating the byte >> > order. Maybe we'll have to special-case that one, so don't worry too >> > much about it.) >> > >> >> Should I work on a patch? >> > >> > That would be great! >> >> OK, here's the patch: http://bugs.python.org/1698994 >> >> The state returned from getstate() should be treated as an opaque value >> (e.g. for the buffered incremental codecs it is the buffered string, for >> the UTF-16 encoder it's the flag indicating whether a BOM has been >> written etc.). The codecs try to return None, if they are in some kind >> of default state (e.g. there's nothing buffered). > > I would like to await completion of those unit tests; The second version of the patch includes the unit tests (and fixes the utf-8-sig codec). > there seem to be > some subtle issues. Can you be more concrete? > I wonder if setstate() should call self.reset() > first. Calling reset() and calling setstate() with the initial state should have the same effect. > I'd also like to ask if setstate() could default to "" only if > the argument is None, not if it is empty; I'd like to use it to change > the buffer to be a bytes object. I'd say for Python 3000 it should always be a bytes object. Will this interoperate seamlessly with the C part of the codec machinery? > (And yes, I need to maintain more > hacks for that, alas). I'l try to update the patch tomorrow or over the weekend. Servus, Walter From guido at python.org Thu Apr 12 23:48:21 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Apr 2007 14:48:21 -0700 Subject: [Python-3000] [Python-3000-checkins] r54742 - in python/branches/p3yk/Lib: io.py test/test_io.py In-Reply-To: <461E8938.80304@livinglogic.de> References: <20070411010907.226411E400D@bag.python.org> <461C97EC.8040903@livinglogic.de> <461DE965.3050302@livinglogic.de> <461E8938.80304@livinglogic.de> Message-ID: > >> > I wonder if it would be possible to return the state as a pair > >> > (unread, flags) where unread is a (byte) string of unprocessed bytes > >> > and flags is some other state, with the constraint that in the initial > >> > state the flags must be zero. Then I can optimize the case where flags > >> > is returned as zero by subtracting len(unread) from the current > >> > position and that'd be the correct seek position. > >> > >> I'd say that bytestream.tell() is the correct position. > >> > >> Or should seek() return to the last position where the codec was in a > >> default state without anything buffered? (This can't work for UTF-16, > >> because the codec almost never is in the default state.) > > > > That was my hope, yes (and I realize that UTF-16 is an exception). > > We could designate natural endianness as the default state, but that > would mean that a codec state can't be transferred to a different > machine (or we could declare little (or big) endianness to be the > default state). I think it's okay for file positions involving codec states not to be tranferable between platforms. I think they wouldn't even be guaranteed between subsequent runs of the same program. > > Consider UTF-8 though. If the chunk we read from the byte stream ended > > in the middle of a multi-byte character, the codec will have the first > > part of that character buffered. In general we want to subtract > > buffered data from the byte stream's position when reporting the > > position of the text stream. The idea is that if we later seek to the > > reported position, we should be reading the same character data. This > > can be accomplished in two ways: by backing up the byte stream to the > > previous character boundary, and resetting the decoder to neutral; or > > by positioning the byte stream to where it was originally and setting > > the state of the decoder to what it was before. However, backing up > > the byte stream has the advantage that no decoder state needs to be > > encoded in the position cookie. > > OK, so for decoders getstate() should always return a tuple, with the > first entry being the buffered byte string (or bytes object?) and the > second being additional state data. > > Do we need any specification for encoders? I don't need this for encoders at all -- we don't use incremental encoders, only incremental decoders. > >> The state returned from getstate() should be treated as an opaque value > >> (e.g. for the buffered incremental codecs it is the buffered string, for > >> the UTF-16 encoder it's the flag indicating whether a BOM has been > >> written etc.). The codecs try to return None, if they are in some kind > >> of default state (e.g. there's nothing buffered). > > > > I would like to await completion of those unit tests; > > The second version of the patch includes the unit tests (and fixes the > utf-8-sig codec). > > > there seem to be > > some subtle issues. > > Can you be more concrete? I think I just meant the str/bytes issue I already mentioned. > > I wonder if setstate() should call self.reset() > > first. > > Calling reset() and calling setstate() with the initial state should > have the same effect. OK, I should do that anyway. (I wasn't aware of reset() until I saw your patch. ;-) > > I'd also like to ask if setstate() could default to "" only if > > the argument is None, not if it is empty; I'd like to use it to change > > the buffer to be a bytes object. > > I'd say for Python 3000 it should always be a bytes object. Eventually, yes. But right now we're in a world where sometimes there are bytes and sometimes there are (8-bit) strings -- and I'd like to get as many tests passing with the new IO library without making it the default first. > Will this > interoperate seamlessly with the C part of the codec machinery? It should if it uses the buffer API as it should. When I encounter places where it requires 8-bit strings I'll fix them opportunistically. > > (And yes, I need to maintain more > > hacks for that, alas). > > I'l try to update the patch tomorrow or over the weekend. Thanks! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Fri Apr 13 01:24:59 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 13 Apr 2007 11:24:59 +1200 Subject: [Python-3000] Line continuation using ellipsis In-Reply-To: <461CF859.6050003@acm.org> References: <461CF859.6050003@acm.org> Message-ID: <461EBFCB.6050404@canterbury.ac.nz> Talin wrote: > I've never liked using the backslash operator for line continuation How about regarding a line as a continuation if it's indented more than the one which started the statement? > 1) It requires that it be the last character of a line, That requirement could be removed, couldn't it? There's no other meaning for a backslash followed by nothing except whitespace. -- Greg From greg.ewing at canterbury.ac.nz Fri Apr 13 01:27:44 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 13 Apr 2007 11:27:44 +1200 Subject: [Python-3000] [Python-ideas] Implicit String Concatenation In-Reply-To: References: Message-ID: <461EC070.9060802@canterbury.ac.nz> For Py3k, how about changing the definition of triple quoted strings so that indentation is stripped up to the level of the line where the string began? In other words, apply an implicit dedent() to it in the parser. -- Greg From greg.ewing at canterbury.ac.nz Fri Apr 13 02:05:29 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 13 Apr 2007 12:05:29 +1200 Subject: [Python-3000] [Python-Dev] PEP 3118: Extended buffer protocol (new version) In-Reply-To: References: <461D8B5F.2020208@canterbury.ac.nz> Message-ID: <461EC949.5080900@canterbury.ac.nz> Lisandro Dalcin wrote: > What about keeping flags it provides instead of the > requested ones? It's hard to guess what kind of info the exporter might want to keep. May there could just be a general-purpose void * field in the bufferinfo for the exporter to use however it wants. -- Greg From collinw at gmail.com Fri Apr 13 05:16:26 2007 From: collinw at gmail.com (Collin Winter) Date: Thu, 12 Apr 2007 20:16:26 -0700 Subject: [Python-3000] PEP 344: attaching tracebacks to exceptions + pickle Message-ID: <43aa6ff70704122016r252ecce7v6bcf39f759036d31@mail.gmail.com> In 2.x, exceptions are pickle-able; tracebacks, however, are not. If tracebacks are attached to exceptions in Python 3 (as specified in PEP 344), exceptions will no longer be pickle-able. There are of course options: 1. Discard an exception's __traceback__ attribute when pickling. 2. Include __traceback__ in the pickle, but in a more limited form. That is: the hard parts of exceptions (frames, code objects) aren't fully needed to, say, print out a traceback. The following is all you need: class Code(object): def __init__(self, code): self.co_filename = code.co_filename self.co_name = code.co_name class Frame(object): def __init__(self, frame): self.f_globals = {"__file__": frame.f_globals["__file__"]} self.f_code = Code(frame.f_code) class Traceback(object): def __init__(self, tb): self.tb_frame = Frame(tb.tb_frame) self.tb_lineno = tb.tb_lineno if tb.tb_next is None: self.tb_next = None else: self.tb_next = Traceback(tb.tb_next) When pickling an exception, __traceback__ could be replaced with the limited Traceback implementation above. 3. Reject the proposal to attach tracebacks to exceptions. (This is my least favorite.) Thoughts? Collin Winter From jcarlson at uci.edu Fri Apr 13 07:28:04 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu, 12 Apr 2007 22:28:04 -0700 Subject: [Python-3000] PEP 344: attaching tracebacks to exceptions + pickle In-Reply-To: <43aa6ff70704122016r252ecce7v6bcf39f759036d31@mail.gmail.com> References: <43aa6ff70704122016r252ecce7v6bcf39f759036d31@mail.gmail.com> Message-ID: <20070412222659.62EF.JCARLSON@uci.edu> "Collin Winter" wrote: > In 2.x, exceptions are pickle-able; tracebacks, however, are not. If > tracebacks are attached to exceptions in Python 3 (as specified in PEP > 344), exceptions will no longer be pickle-able. I know that idlelib packages tracebacks for transfer across a socket connection, it may offer more complete pickled traceback information. - Josiah From walter at livinglogic.de Fri Apr 13 13:49:47 2007 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Fri, 13 Apr 2007 13:49:47 +0200 Subject: [Python-3000] [Python-3000-checkins] r54742 - in python/branches/p3yk/Lib: io.py test/test_io.py In-Reply-To: References: <20070411010907.226411E400D@bag.python.org> <461C97EC.8040903@livinglogic.de> <461DE965.3050302@livinglogic.de> <461E8938.80304@livinglogic.de> Message-ID: <461F6E5B.2090908@livinglogic.de> Guido van Rossum wrote: >> >> > I wonder if it would be possible to return the state as a pair >> >> > (unread, flags) where unread is a (byte) string of unprocessed bytes >> >> > and flags is some other state, with the constraint that in the >> initial >> >> > state the flags must be zero. Then I can optimize the case where >> flags >> >> > is returned as zero by subtracting len(unread) from the current >> >> > position and that'd be the correct seek position. >> >> >> >> I'd say that bytestream.tell() is the correct position. >> >> >> >> Or should seek() return to the last position where the codec was in a >> >> default state without anything buffered? (This can't work for UTF-16, >> >> because the codec almost never is in the default state.) >> > >> > That was my hope, yes (and I realize that UTF-16 is an exception). >> >> We could designate natural endianness as the default state, but that >> would mean that a codec state can't be transferred to a different >> machine (or we could declare little (or big) endianness to be the >> default state). > > I think it's okay for file positions involving codec states not to be > tranferable between platforms. I think they wouldn't even be > guaranteed between subsequent runs of the same program. OK, done in the third version of the patch. >> > Consider UTF-8 though. If the chunk we read from the byte stream ended >> > in the middle of a multi-byte character, the codec will have the first >> > part of that character buffered. In general we want to subtract >> > buffered data from the byte stream's position when reporting the >> > position of the text stream. The idea is that if we later seek to the >> > reported position, we should be reading the same character data. This >> > can be accomplished in two ways: by backing up the byte stream to the >> > previous character boundary, and resetting the decoder to neutral; or >> > by positioning the byte stream to where it was originally and setting >> > the state of the decoder to what it was before. However, backing up >> > the byte stream has the advantage that no decoder state needs to be >> > encoded in the position cookie. >> >> OK, so for decoders getstate() should always return a tuple, with the >> first entry being the buffered byte string (or bytes object?) and the >> second being additional state data. >> >> Do we need any specification for encoders? > > I don't need this for encoders at all -- we don't use incremental > encoders, only incremental decoders. True for reading, but what about writing? >> >> The state returned from getstate() should be treated as an opaque >> value >> >> (e.g. for the buffered incremental codecs it is the buffered >> string, for >> >> the UTF-16 encoder it's the flag indicating whether a BOM has been >> >> written etc.). The codecs try to return None, if they are in some kind >> >> of default state (e.g. there's nothing buffered). >> > >> > I would like to await completion of those unit tests; >> >> The second version of the patch includes the unit tests (and fixes the >> utf-8-sig codec). >> >> > there seem to be >> > some subtle issues. >> >> Can you be more concrete? > > I think I just meant the str/bytes issue I already mentioned. Since the new version never sets the buffer to an explicit value except in the constructor this problem should have disappeared. >> > I wonder if setstate() should call self.reset() >> > first. >> >> Calling reset() and calling setstate() with the initial state should >> have the same effect. > > OK, I should do that anyway. (I wasn't aware of reset() until I saw > your patch. ;-) > >> > I'd also like to ask if setstate() could default to "" only if >> > the argument is None, not if it is empty; I'd like to use it to change >> > the buffer to be a bytes object. >> >> I'd say for Python 3000 it should always be a bytes object. > > Eventually, yes. But right now we're in a world where sometimes there > are bytes and sometimes there are (8-bit) strings -- and I'd like to > get as many tests passing with the new IO library without making it > the default first. OK. >> Will this >> interoperate seamlessly with the C part of the codec machinery? > > It should if it uses the buffer API as it should. When I encounter > places where it requires 8-bit strings I'll fix them > opportunistically. > >> > (And yes, I need to maintain more >> > hacks for that, alas). >> >> I'l try to update the patch tomorrow or over the weekend. > > Thanks! Done. I've also added documentation (The description of the constraints on the decoder state sounds quite esoteric ;)). Servus, Walter From paul-python at svensson.org Fri Apr 13 15:24:49 2007 From: paul-python at svensson.org (Paul Svensson) Date: Fri, 13 Apr 2007 09:24:49 -0400 (EDT) Subject: [Python-3000] Line continuation using ellipsis In-Reply-To: <461EBFCB.6050404@canterbury.ac.nz> References: <461CF859.6050003@acm.org> <461EBFCB.6050404@canterbury.ac.nz> Message-ID: <20070413091624.H68198@familjen.svensson.org> Talin wrote: > I've never liked using the backslash operator for line continuation Me neither, but the suggested replacements don't appeal to me either. Is there any reason we can't just remove them ? Is backslash continuation still needed anywhere, or does the grammar now allow for sufficient parenteses anywhere a line might be 'too long' ? /Paul From jimjjewett at gmail.com Fri Apr 13 16:19:33 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 13 Apr 2007 10:19:33 -0400 Subject: [Python-3000] Line continuation using ellipsis In-Reply-To: <20070413091624.H68198@familjen.svensson.org> References: <461CF859.6050003@acm.org> <461EBFCB.6050404@canterbury.ac.nz> <20070413091624.H68198@familjen.svensson.org> Message-ID: On 4/13/07, Paul Svensson wrote: > Talin wrote: > > I've never liked using the backslash operator for line continuation > Me neither, but the suggested replacements don't appeal to me either. > Is there any reason we can't just remove them ? > Is backslash continuation still needed anywhere, >>> assert True, SyntaxError: invalid syntax >>> assert True, \ "ASDF" >>> From steven.bethard at gmail.com Fri Apr 13 17:33:33 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Fri, 13 Apr 2007 09:33:33 -0600 Subject: [Python-3000] Line continuation using ellipsis In-Reply-To: References: <461CF859.6050003@acm.org> <461EBFCB.6050404@canterbury.ac.nz> <20070413091624.H68198@familjen.svensson.org> Message-ID: On 4/13/07, Jim Jewett wrote: > On 4/13/07, Paul Svensson wrote: > > Talin wrote: > > > I've never liked using the backslash operator for line continuation > > > Me neither, but the suggested replacements don't appeal to me either. > > Is there any reason we can't just remove them ? > > Is backslash continuation still needed anywhere, > > > >>> assert True, > SyntaxError: invalid syntax > >>> assert True, \ > "ASDF" > >>> Just like everywhere else, you can use parentheses: >>> assert True, ( ... 'ASDF') STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From ark-mlist at att.net Fri Apr 13 21:07:41 2007 From: ark-mlist at att.net (Andrew Koenig) Date: Fri, 13 Apr 2007 15:07:41 -0400 Subject: [Python-3000] Line continuation using ellipsis In-Reply-To: <461CF859.6050003@acm.org> Message-ID: <002b01c77dff$03121e40$6402a8c0@arkdesktop> > I'm not proposing to remove the feature, however I'd like to see an > alternative method for declaring statements that cross a line boundary. > I seem to recall at one point someone suggesting the use of ellipsis, > which makes a lot of sense to me: > > sorted_result = partition_lower( input_list, pivot ) ... > + pivot ... > + partition_upper( input_list, pivot ) I once used a programming language named EFL that had what I think is a nice convention: If the last token on a line is an operator (i.e. something that cannot syntactically be the last token of a statement), the next line is deemed a continuation. So your example would be able to be written this way: sorted_result = partition_lower( input_list, pivot) + pivot + partition_upper( input_list, pivot ) Interestingly, if I remember correctly, EFL did *not* assume a continuation for unmatched parentheses -- it looked only at the last token on the line. For Python, I guess I'd like it if either unmatched parentheses or ending with an operator were to signal continuation. From oliphant.travis at ieee.org Fri Apr 13 23:59:59 2007 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri, 13 Apr 2007 15:59:59 -0600 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: <461C31A9.5090204@canterbury.ac.nz> References: <461C31A9.5090204@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Guido van Rossum wrote: > >>all PEPs for Py3k are due by April. > > > Can we nominate one of our 2.x PEPs for promotion > to a 3000 PEP? > > If so, I'd like to nominate PEP 335 - Overloadable > Boolean Operators. > That one would sure improve a lot of trouble we currently have with NumPy users not understanding why x[x>3 and x<5] doesn't give them what they think it should. I'm +1 on this proposal for Python 2.6 and Python 3.0 -Travis From rrr at ronadam.com Sat Apr 14 03:17:03 2007 From: rrr at ronadam.com (Ron Adam) Date: Fri, 13 Apr 2007 20:17:03 -0500 Subject: [Python-3000] Line continuation using ellipsis In-Reply-To: <002b01c77dff$03121e40$6402a8c0@arkdesktop> References: <461CF859.6050003@acm.org> <002b01c77dff$03121e40$6402a8c0@arkdesktop> Message-ID: Andrew Koenig wrote: >> I'm not proposing to remove the feature, however I'd like to see an >> alternative method for declaring statements that cross a line boundary. >> I seem to recall at one point someone suggesting the use of ellipsis, >> which makes a lot of sense to me: >> >> sorted_result = partition_lower( input_list, pivot ) ... >> + pivot ... >> + partition_upper( input_list, pivot ) > > I once used a programming language named EFL that had what I think is a nice > convention: If the last token on a line is an operator (i.e. something that > cannot syntactically be the last token of a statement), the next line is > deemed a continuation. So your example would be able to be written this > way: > > sorted_result = partition_lower( input_list, pivot) + > pivot + > partition_upper( input_list, pivot ) > > Interestingly, if I remember correctly, EFL did *not* assume a continuation > for unmatched parentheses -- it looked only at the last token on the line. > > For Python, I guess I'd like it if either unmatched parentheses or ending > with an operator were to signal continuation. I tend to do it the other way around because starting off each continued line with an operator makes for more readable code. It's even nicer with syntax highlighting. Using either a backslash or parentheses is fine for me in those situations. The attractiveness of the ellipsis is it is already used in an interactive console to indicate continued lines, but you still need the slash or parentheses to get that. ;-) The only problem I have with the '\' is I don't like it's behavior in strings. I would prefer that a single back slash in regular strings to always be an escape introducer and cause a syntax error if it's an invalid escape character or it is not at the end of a line, and for it to always be a single back slash in raw strings, except at the end of a physical line, where it can be a continue line indicator as usual. >>> r"\" File "", line 1 r"\" ^ SyntaxError: EOL while scanning single-quoted string >>> "\"" '"' >>> r"\"" '\\"' Shouldn't r"\" result in "\\" ? >>> print '1 \ 2' 1 \ 2 >>> print '1 \\ 2' 1 \ 2 >>> print '1 \\\ 2' 1 \\ 2 >>> print '1 \\\\ 2' 1 \\ 2 >>> """ hello ... world""" ' hello\nworld' >>> """ hello \ ... world""" ' hello world' >>> """hello\ ... world""" 'hello\\ \nworld' I think this does result in silent bugs at times, but more often result in the output not being what was expected. Cheers, Ron From greg.ewing at canterbury.ac.nz Sat Apr 14 04:18:42 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 14 Apr 2007 14:18:42 +1200 Subject: [Python-3000] Line continuation using ellipsis In-Reply-To: References: <461CF859.6050003@acm.org> <461EBFCB.6050404@canterbury.ac.nz> <20070413091624.H68198@familjen.svensson.org> Message-ID: <46203A02.2010600@canterbury.ac.nz> Steven Bethard wrote: > Just like everywhere else, you can use parentheses: > >>>>assert True, ( > ... 'ASDF') Yes, but you have to put the opening paren in a somewhat awkward place to get it to work. Seems to me it would be much more intuitive to use what Haskell calls the "offside rule", i.e. anything indented more than the beginning of a statement is part of that statement. The only drawback might be that an indentation error could lead to something being interpreted in an unintended way. But I suspect it would result in a syntax error in most situations. -- Greg From talin at acm.org Sat Apr 14 19:57:28 2007 From: talin at acm.org (Talin) Date: Sat, 14 Apr 2007 10:57:28 -0700 Subject: [Python-3000] An introduction to ABC's Message-ID: <46211608.7010604@acm.org> Part of the reason why I haven't volunteered to write a PEP for ABC's is because I don't feel that I understand the various proposals and the background discussion well enough. However, it occurs to me that writing the rationale section of a PEP can often be the hardest part, and I think I understand the issues well enough to write that preface. So here's my contribution: --- In the domain of object-oriented programming, the usage patterns for interacting with an object can be divided into two basic categories, which are 'invocation' and 'inspection'. Invocation means interacting with an object by invoking its methods. Usually this is combined with polymorphism, so that invoking a given method may run different code depending on the type of an object. Inspection means the ability for external code (outside of the object's methods) to examine the type or properties of that object, and make decisions on how to treat that object based on that information. Both usage patterns serve the same general end, which is to be able to support the processing of diverse and potentially novel objects in a uniform way, but at the same time allowing processing decisions to be customized for each different type of object. In classical OOP theory, invocation is the preferred usage pattern, and inspection is actively discouraged, being considered a relic of an earlier, procedural programming style. However, in practice this view is simply too dogmatic and inflexible, and leads to a kind of design rigidity that is very much at odds with the dynamic nature of a language like Python. In particular, there is often a need to process objects in a way that wasn't anticipated by the creator of the object class. It is not always the best solution to build in to every object methods that satisfy the needs of every possible user of that object. Moreover, there are many powerful dispatch philosophies that are in direct contrast to the classic OOP requirement of behavior being strictly encapsulated within an object, examples being rule or pattern-match driven logic. On the the other hand, one of the criticisms of inspection by classic OOP theorists is the lack of formalisms and the ad hoc nature of what is being inspected. In a language such as Python, in which almost any aspect of an object can be reflected and directly accessed by external code, there are many different ways to test whether an object conforms to a particular protocol or not. For example, if asking 'is this object a mutable sequence container?', one can look for a base class of 'list', or one can look for a method named '__getitem__'. But note that although these tests may seem obvious, neither of them are correct, as one generates false negatives, and the other false positives. The generally agreed-upon remedy is to standardize the tests, and group them into a formal arrangement. This is most easily done by associating with each class a set of standard testable properties, either via the inheritance mechanism or some other means. Each test carries with it a set of promises: it contains a promise about the general behavior of the class, and a promise as to what other class methods will be available. This PEP proposes a particular strategy for organizing these tests known as Abstract Base Classes, or ABC. ABCs are simply Python classes that are added into an object's inheritance tree to signal certain features of that object to an external inspector. Tests are done using isinstance(), and the presence of a particular ABC means that the test has passed. Like all other things in Python, these promises are in the nature of a gentlemen's agreement - which means that the language does not attempt to enforce that these promises are kept. From martin at v.loewis.de Sun Apr 15 11:46:50 2007 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 15 Apr 2007 11:46:50 +0200 Subject: [Python-3000] Draft PEP: Using UTF-8 as the default source encoding Message-ID: <4621F48A.1090308@v.loewis.de> I'd like to submit the following PEP for Python 3. Regards, Martin PEP: xxx Title: Using UTF-8 as the default source encoding Version: $Revision $ Last-Modified: $Date $ Author: Martin v. L?wis Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 15-Apr-2007 Python-Version: 3.0 Post-History: Specification ============= This PEP proposes to change the default source encoding from ASCII to UTF-8. Support for alternative source encodings [#pep263]_ continues to exist; an explicit encoding declaration takes precedence over the default. A Bit of History ================ In Python 1, the source encoding was unspecified, except that the source encoding had to be a superset of the system's basic execution character set (i.e. an ASCII superset, on most systems). The source encoding was only relevant for the lexis itself (bytes representing letters for keywords, identifiers, punctuation, line breaks, etc). The contents of a string literal was copied literally from the file on source. In Python 2.0, the source encoding changed to Latin-1 as a side effect of introducing Unicode. For Unicode string literals, the characters were still copied literally from the source file, but widened on a character-by-character basis. As Unicode gives a fixed interpretation to code points, this algorithm effectively fixed a source encoding, at least for files containing non-ASCII characters in Unicode literals. PEP 263 identified the problem that you can use only those Unicode characters in a Unicode literal which are also in Latin-1, and introduced a syntax for declaring the source encoding. If no source encoding was given, the default should be ASCII. For compatibility with Python 2.0 and 2.1, files were interpreted as Latin-1 for a transitional period. This transition ended with Python 2.5, which gives an error if non-ASCII characters are encountered and no source encoding is declared. Rationale ========= With PEP 263, using arbitrary non-ASCII characters in a Python file is possible, but tedious. One has to explicitly add an encoding declaration. Even though some editors (like IDLE and Emacs) support the declarations of PEP 263, many editors still do not (and never will); users have to explicitly adjust the encoding which the editor assumes on a file-by-file basis. When the default encoding is changed to UTF-8, adding non-ASCII text to Python files becomes easier and more portable: On some systems, editors will automatically choose UTF-8 when saving text (e.g. on Unix systems where the locale uses UTF-8). On other systems, editors will guess the encoding when reading the file, and UTF-8 is easy to guess. Yet other editors support associating a default encoding with a file extension, allowing users to associate .py with UTF-8. For Python 2, an important reason for using non-UTF-8 encodings was that byte string literals would be in the source encoding at run-time, allowing then to output them to a file or render them to the user as-is. With Python 3, all strings will be Unicode strings, so the original encoding of the source will have no impact at run-time. Implementation ============== The parser needs to be changed to accept bytes > 127 if no source encoding is specified; instead of giving an error, it needs to check that the bytes are well-formed UTF-8 (decoding is not necessary, as the parser converts all source code to UTF-8, anyway). IDLE needs to be changed to use UTF-8 as the default encoding. References ========== .. [#pep263] http://www.python.org/dev/peps/pep-0263/ Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From g.brandl at gmx.net Sun Apr 15 18:53:25 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 15 Apr 2007 18:53:25 +0200 Subject: [Python-3000] Draft PEP: Using UTF-8 as the default source encoding In-Reply-To: <4621F48A.1090308@v.loewis.de> References: <4621F48A.1090308@v.loewis.de> Message-ID: Martin v. L?wis schrieb: > I'd like to submit the following PEP for Python 3. > > Regards, > Martin > > > PEP: xxx > Title: Using UTF-8 as the default source encoding > Version: $Revision $ > Last-Modified: $Date $ > Author: Martin v. L?wis > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 15-Apr-2007 > Python-Version: 3.0 > Post-History: > > > Specification > ============= > > This PEP proposes to change the default source encoding from ASCII to > UTF-8. Support for alternative source encodings [#pep263]_ continues to > exist; an explicit encoding declaration takes precedence over the > default. +1 in general. Do you want to add a sentence or two about the infamous UTF-8 "BOM"? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From martin at v.loewis.de Sun Apr 15 19:20:19 2007 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 15 Apr 2007 19:20:19 +0200 Subject: [Python-3000] Draft PEP: Using UTF-8 as the default source encoding In-Reply-To: References: <4621F48A.1090308@v.loewis.de> Message-ID: <46225ED3.9010809@v.loewis.de> > +1 in general. Do you want to add a sentence or two about the infamous > UTF-8 "BOM"? Can do. Strictly speaking, it's covered by PEP 263 already, which makes presence of the BOM equivalent to an encoding declaration. Regards, Martin From nnorwitz at gmail.com Mon Apr 16 09:29:30 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 16 Apr 2007 00:29:30 -0700 Subject: [Python-3000] removing functions from string module Message-ID: Can we remove all the functions from the string module that are also existing methods on string? Then we can figure out what to do with what's left over? I'm tempted to ask the same thing about the types module. TOOWTDI. n From nevillegrech at hotmail.com Mon Apr 16 13:12:11 2007 From: nevillegrech at hotmail.com (Neville Grech) Date: Mon, 16 Apr 2007 13:12:11 +0200 Subject: [Python-3000] Empty set and empty dictionary Message-ID: This is just some syntax sugar: Since set literals will change to for example {1,2,3} from set([1,2,3]) and set comprehensions will be specified inside {} I feel that {} will be more naturally associated with sets than dicts (or at least as much). What if the empty set literal is changed to {} and an empty dict literal changed to {:}. Performing the conversion automatically wouldn't be so complex and also un-ambiguous. The hardest thing to change would be the mentality then. i.e: {} ::= set([]) {1,2,3} ::= set([1,2,3]) {x for x in y} ::= set(x for x in y) {:} ::= dict() {a:b, h:j} is a dict since it contains colons. I hope this isn't a stupid suggestion (I'm new here). -Neville -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070416/6cf865e9/attachment.html From guido at python.org Mon Apr 16 18:33:45 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 16 Apr 2007 09:33:45 -0700 Subject: [Python-3000] Empty set and empty dictionary In-Reply-To: References: Message-ID: It's not stupid, but it's been brought up before (exactly like you propose) and rejected, on the basis that dicts are still much more common than sets in most code. On 4/16/07, Neville Grech wrote: > > > This is just some syntax sugar: > > Since set literals will change to for example {1,2,3} from set([1,2,3]) and > set comprehensions will be specified inside {} I feel that {} will be more > naturally associated with sets than dicts (or at least as much). > > What if the empty set literal is changed to {} and an empty dict literal > changed to {:}. Performing the conversion automatically wouldn't be so > complex and also un-ambiguous. The hardest thing to change would be the > mentality then. > > i.e: > {} ::= set([]) > {1,2,3} ::= set([1,2,3]) > {x for x in y} ::= set(x for x in y) > {:} ::= dict() > > {a:b, h:j} is a dict since it contains colons. > > I hope this isn't a stupid suggestion (I'm new here). > > -Neville > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/guido%40python.org > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Apr 16 18:34:12 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 16 Apr 2007 09:34:12 -0700 Subject: [Python-3000] removing functions from string module In-Reply-To: References: Message-ID: On 4/16/07, Neal Norwitz wrote: > Can we remove all the functions from the string module that are also > existing methods on string? Then we can figure out what to do with > what's left over? +1 > I'm tempted to ask the same thing about the types module. +1 -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jcarlson at uci.edu Mon Apr 16 18:16:35 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 16 Apr 2007 09:16:35 -0700 Subject: [Python-3000] Empty set and empty dictionary In-Reply-To: References: Message-ID: <20070416090725.6311.JCARLSON@uci.edu> "Neville Grech" wrote: > This is just some syntax sugar: > > Since set literals will change to for example {1,2,3} from set([1,2,3]) > and set comprehensions will be specified inside {} I feel that {} > will be more naturally associated with sets than dicts (or at least > as much). > > What if the empty set literal is changed to {} and an empty dict > literal changed to {:}. Performing the conversion automatically > wouldn't be so complex and also un-ambiguous. The hardest thing to > change would be the mentality then. Because the mentality would be so difficult to change, and because sets are still used less frequently than dicts, it doesn't make sense to alter dictionary syntax (that people have used for over a decade) for sets (which don't yet have a literal syntax in a released Python). Your heart is in the right place, but your logic is jumping the gun by a while. Offer the suggestion after the syntax has been available in a released Python for at least a year, when everyone has a chance to use set syntax. - Josiah From brett at python.org Mon Apr 16 21:00:22 2007 From: brett at python.org (Brett Cannon) Date: Mon, 16 Apr 2007 12:00:22 -0700 Subject: [Python-3000] removing functions from string module In-Reply-To: References: Message-ID: On 4/16/07, Guido van Rossum wrote: > > On 4/16/07, Neal Norwitz wrote: > > Can we remove all the functions from the string module that are also > > existing methods on string? Then we can figure out what to do with > > what's left over? > > +1 > > > I'm tempted to ask the same thing about the types module. > > +1 What Guido said. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070416/e43e5311/attachment.html From dillonco at comcast.net Mon Apr 16 19:08:23 2007 From: dillonco at comcast.net (DillonCo) Date: Mon, 16 Apr 2007 13:08:23 -0400 Subject: [Python-3000] Empty set and empty dictionary In-Reply-To: References: Message-ID: <200704161308.23924.dillonco@comcast.net> On Monday 16 April 2007, Neville Grech wrote: > Since set literals will change to for example {1,2,3} from set([1,2,3]) > and set comprehensions will be specified inside {} I feel that {} will be > more naturally associated with sets than dicts (or at least as much). While the topic of set literals is around, I figure I ought to ask something that's been on my mind: Why not use "<>" for sets? As far as I can tell, they're only currently used in "expr '<' | '>' " contexts, so it'd be easy to distinguish usages. (I think. The "[]" operators are used similarly.) I know the traditional mathematical notation uses "{}" and having it overloaded isn't a big deal; I'm simply curious. From nevillegrech at hotmail.com Mon Apr 16 21:49:17 2007 From: nevillegrech at hotmail.com (Neville Grech) Date: Mon, 16 Apr 2007 21:49:17 +0200 Subject: [Python-3000] Empty set and empty dictionary References: <200704161308.23924.dillonco@comcast.net> Message-ID: That would probably be very difficult to parse because it introduces ambiguities. Take a look at this situation: <8,True, 3>=4, 5 > ----- Original Message ----- From: "DillonCo" To: Sent: Monday, April 16, 2007 7:08 PM Subject: Re: [Python-3000] Empty set and empty dictionary > On Monday 16 April 2007, Neville Grech wrote: >> Since set literals will change to for example {1,2,3} from set([1,2,3]) >> and set comprehensions will be specified inside {} I feel that {} will be >> more naturally associated with sets than dicts (or at least as much). > > While the topic of set literals is around, I figure I ought to ask > something > that's been on my mind: > > Why not use "<>" for sets? As far as I can tell, they're only currently > used > in "expr '<' | '>' " contexts, so it'd be easy to distinguish usages. (I > think. The "[]" operators are used similarly.) > > I know the traditional mathematical notation uses "{}" and having it > overloaded isn't a big deal; I'm simply curious. > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/nevillegrech%40hotmail.com > From greg.ewing at canterbury.ac.nz Tue Apr 17 02:22:29 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 17 Apr 2007 12:22:29 +1200 Subject: [Python-3000] Empty set and empty dictionary In-Reply-To: <200704161308.23924.dillonco@comcast.net> References: <200704161308.23924.dillonco@comcast.net> Message-ID: <46241345.6070605@canterbury.ac.nz> DillonCo wrote: > Why not use "<>" for sets? Some possible reasons: * It would look ugly * There could be visual confusion with comparison operators * There could be parsing difficulties distinguishing nested set bracketing from << and >> operators -- Greg From eoghan at qatano.com Tue Apr 17 02:31:56 2007 From: eoghan at qatano.com (Eoghan Murray) Date: Tue, 17 Apr 2007 01:31:56 +0100 Subject: [Python-3000] Empty set and empty dictionary In-Reply-To: <46241345.6070605@canterbury.ac.nz> References: <200704161308.23924.dillonco@comcast.net> <46241345.6070605@canterbury.ac.nz> Message-ID: I had another idea on this theme.. going by unicode and raw string literals, how about s[1, 2, 3, 4] for a set? Eoghan On 17/04/07, Greg Ewing wrote: > > DillonCo wrote: > > > Why not use "<>" for sets? > > Some possible reasons: > > * It would look ugly > > * There could be visual confusion with comparison operators > > * There could be parsing difficulties distinguishing > nested set bracketing from << and >> operators > > -- > Greg > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/eoghan%40qatano.com > -- http://qatano.com/ http://eoghan.qatano.org/ +353 86 236 1190 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070417/472d19b9/attachment-0001.html From eoghan at qatano.com Tue Apr 17 02:35:49 2007 From: eoghan at qatano.com (Eoghan Murray) Date: Tue, 17 Apr 2007 01:35:49 +0100 Subject: [Python-3000] Empty set and empty dictionary In-Reply-To: References: <200704161308.23924.dillonco@comcast.net> <46241345.6070605@canterbury.ac.nz> Message-ID: s[1] being a list index of the list 's' s[1,] being a set literal with one element, '1' On 17/04/07, Eoghan Murray wrote: > > I had another idea on this theme.. going by unicode and raw string > literals, how about > s[1, 2, 3, 4] > for a set? > > Eoghan > > On 17/04/07, Greg Ewing wrote: > > > > DillonCo wrote: > > > > > Why not use "<>" for sets? > > > > Some possible reasons: > > > > * It would look ugly > > > > * There could be visual confusion with comparison operators > > > > * There could be parsing difficulties distinguishing > > nested set bracketing from << and >> operators > > > > -- > > Greg > > _______________________________________________ > > Python-3000 mailing list > > Python-3000 at python.org > > http://mail.python.org/mailman/listinfo/python-3000 > > Unsubscribe: > > http://mail.python.org/mailman/options/python-3000/eoghan%40qatano.com > > > > > > -- > http://qatano.com/ > http://eoghan.qatano.org/ > +353 86 236 1190 -- http://qatano.com/ http://eoghan.qatano.org/ +353 86 236 1190 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070417/4b9dc874/attachment.htm From greg.ewing at canterbury.ac.nz Tue Apr 17 02:36:01 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 17 Apr 2007 12:36:01 +1200 Subject: [Python-3000] Empty set and empty dictionary In-Reply-To: References: <200704161308.23924.dillonco@comcast.net> <46241345.6070605@canterbury.ac.nz> Message-ID: <46241671.10108@canterbury.ac.nz> Eoghan Murray wrote: > how about > s[1, 2, 3, 4] > for a set? Already meaningful syntax. Try this: d = {} d[1, 2, 3, 4] = 'spam' print d -- Greg From cvrebert at gmail.com Tue Apr 17 03:08:22 2007 From: cvrebert at gmail.com (Chris Rebert) Date: Mon, 16 Apr 2007 18:08:22 -0700 Subject: [Python-3000] Empty set and empty dictionary In-Reply-To: References: <200704161308.23924.dillonco@comcast.net> <46241345.6070605@canterbury.ac.nz> Message-ID: <46241E06.2010907@gmail.com> I think someone has probably proposed this before, but why not use "{,}" as the empty set literal? It's somewhat analogous to the 1-element tuple syntax and fairly similar to the syntax that the original set PEP proposed ("{-}"). From what I understand, {-} was nixed for being too hard to parse because the minus sign could be the start of an expression. I don't think this would apply to the comma as it's basically used as an expression separator; however I'm no expert on Python's grammar and parser, so I can't be sure. Thoughts? - Chris Rebert Eoghan Murray wrote: > I had another idea on this theme.. going by unicode and raw string > literals, > how about > s[1, 2, 3, 4] > for a set? > > Eoghan > > On 17/04/07, Greg Ewing wrote: >> >> DillonCo wrote: >> >> > Why not use "<>" for sets? >> >> Some possible reasons: >> >> * It would look ugly >> >> * There could be visual confusion with comparison operators >> >> * There could be parsing difficulties distinguishing >> nested set bracketing from << and >> operators >> >> -- >> Greg >> _______________________________________________ >> Python-3000 mailing list >> Python-3000 at python.org >> http://mail.python.org/mailman/listinfo/python-3000 >> Unsubscribe: >> http://mail.python.org/mailman/options/python-3000/eoghan%40qatano.com >> > > > > > ------------------------------------------------------------------------ > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/cvrebert%40gmail.com From mike.klaas at gmail.com Tue Apr 17 03:27:02 2007 From: mike.klaas at gmail.com (Mike Klaas) Date: Mon, 16 Apr 2007 18:27:02 -0700 Subject: [Python-3000] Empty set and empty dictionary In-Reply-To: <46241E06.2010907@gmail.com> References: <200704161308.23924.dillonco@comcast.net> <46241345.6070605@canterbury.ac.nz> <46241E06.2010907@gmail.com> Message-ID: <3d2ce8cb0704161827p48acdd5avffe157112999d830@mail.gmail.com> On 4/16/07, Chris Rebert wrote: > I think someone has probably proposed this before, but why not use "{,}" > as the empty set literal? If you're sure, did you look for it first? http://mail.python.org/pipermail/python-3000/2006-April/001400.html -Mike From cvrebert at gmail.com Tue Apr 17 04:03:35 2007 From: cvrebert at gmail.com (Chris Rebert) Date: Mon, 16 Apr 2007 19:03:35 -0700 Subject: [Python-3000] Empty set and empty dictionary In-Reply-To: <3d2ce8cb0704161827p48acdd5avffe157112999d830@mail.gmail.com> References: <200704161308.23924.dillonco@comcast.net> <46241345.6070605@canterbury.ac.nz> <46241E06.2010907@gmail.com> <3d2ce8cb0704161827p48acdd5avffe157112999d830@mail.gmail.com> Message-ID: <46242AF7.6060303@gmail.com> Ah, that must have been it! Never mind then ;-). - Chris Rebert Mike Klaas wrote: > On 4/16/07, Chris Rebert wrote: >> I think someone has probably proposed this before, but why not use "{,}" >> as the empty set literal? > > If you're sure, did you look for it first? > > http://mail.python.org/pipermail/python-3000/2006-April/001400.html > > -Mike From guido at python.org Tue Apr 17 05:24:09 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 16 Apr 2007 20:24:09 -0700 Subject: [Python-3000] Empty set and empty dictionary In-Reply-To: <46242AF7.6060303@gmail.com> References: <200704161308.23924.dillonco@comcast.net> <46241345.6070605@canterbury.ac.nz> <46241E06.2010907@gmail.com> <3d2ce8cb0704161827p48acdd5avffe157112999d830@mail.gmail.com> <46242AF7.6060303@gmail.com> Message-ID: Can we move this thread out of the python-3000? This issue has already been decided and is unlikely to be reopened. On 4/16/07, Chris Rebert wrote: > Ah, that must have been it! Never mind then ;-). > > - Chris Rebert > > Mike Klaas wrote: > > On 4/16/07, Chris Rebert wrote: > >> I think someone has probably proposed this before, but why not use "{,}" > >> as the empty set literal? > > > > If you're sure, did you look for it first? > > > > http://mail.python.org/pipermail/python-3000/2006-April/001400.html > > > > -Mike > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From oliphant.travis at ieee.org Tue Apr 17 06:39:07 2007 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Mon, 16 Apr 2007 22:39:07 -0600 Subject: [Python-3000] Draft PEP: Using UTF-8 as the default source encoding In-Reply-To: <4621F48A.1090308@v.loewis.de> References: <4621F48A.1090308@v.loewis.de> Message-ID: Martin v. L?wis wrote: > I'd like to submit the following PEP for Python 3. > > Regards, > Martin > > > PEP: xxx > Title: Using UTF-8 as the default source encoding > Version: $Revision $ > Last-Modified: $Date $ > Author: Martin v. L?wis > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 15-Apr-2007 > Python-Version: 3.0 > Post-History: > +1 From humberto at digi.com.br Tue Apr 17 07:49:49 2007 From: humberto at digi.com.br (Humberto Diogenes) Date: Tue, 17 Apr 2007 02:49:49 -0300 Subject: [Python-3000] Reminder: Py3k PEPs due by April In-Reply-To: <461C2AC7.4090908@canterbury.ac.nz> References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net> <461C2AC7.4090908@canterbury.ac.nz> Message-ID: <605D1AB0-DFDC-43FD-9283-527AC6149B41@digi.com.br> On 10/04/2007, at 21:24, Greg Ewing wrote: > Raymond Hettinger wrote: > >> Here's that I think might not need a PEP: >> >> * Eliminate implicit string concatenation: "abc" "def" >> in favor of an explicit + operation. > > -0.707. I've used languages where the only way of > splitting a string over lines is to use concatenation, > and it's tedious. > >> When there are multi-line concats, I think >> the parenthesis serves us much better than a trailing \ > > Which can be done perfectly well as things are: > > ('hello ' > 'world ') Yes, but this causes subtle bugs when you forget a comma: what should be the second parameter ends up in the first one. You run the program with no syntax errors and later you'll find that e-mails are being sent to 'me at domain.comHello!' or something like that. As they say: just my 2 cents. -- Humberto Di?genes http://humberto.digi.com.br From humberto at digi.com.br Tue Apr 17 07:35:26 2007 From: humberto at digi.com.br (Humberto Diogenes) Date: Tue, 17 Apr 2007 02:35:26 -0300 Subject: [Python-3000] i18n prefix Message-ID: <662B0B21-F14D-4EFC-BB57-9ACC22E879E6@digi.com.br> Hi, folks!! This is my first post to python-3000. After Guido PEPs-deadline announcement, I decided to discuss an old idea while it still has some chance of being accepted. :) No, I do not want i18n identifiers. What I'd like to propose is a standard syntax for i18n strings. Something like: print i"my string" My points are: * i18n is everywhere (or is it the other way around? :-) * Python has a tradition of incorporating good practices * _ = gettext.gettext; print _("my string") isn't very nice So, do you think that something like i"my string" would be better than _("my string")? Would it add unnecessary complexity to Python? -- Humberto Di?genes http://humberto.digi.com.br From greg.ewing at canterbury.ac.nz Tue Apr 17 08:44:56 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 17 Apr 2007 18:44:56 +1200 Subject: [Python-3000] i18n prefix In-Reply-To: <662B0B21-F14D-4EFC-BB57-9ACC22E879E6@digi.com.br> References: <662B0B21-F14D-4EFC-BB57-9ACC22E879E6@digi.com.br> Message-ID: <46246CE8.7090904@canterbury.ac.nz> Humberto Diogenes wrote: > * _ = gettext.gettext; print _("my string") isn't very nice If you don't like _, you're free to use something else, e.g. from gettext import gettext as i i("This is a an i18m string") which is not very far from an i prefix. -- Greg From talin at acm.org Tue Apr 17 09:18:05 2007 From: talin at acm.org (Talin) Date: Tue, 17 Apr 2007 00:18:05 -0700 Subject: [Python-3000] An introduction to ABC's In-Reply-To: <20070416150746.GE17621@crater.logilab.fr> References: <46211608.7010604@acm.org> <20070416150746.GE17621@crater.logilab.fr> Message-ID: <462474AD.2050807@acm.org> Aur?lien Camp?as wrote: > On Sat, Apr 14, 2007 at 10:57:28AM -0700, Talin wrote: >> Part of the reason why I haven't volunteered to write a PEP for ABC's is because I don't feel that I understand the various proposals and the background discussion well enough. However, it occurs to me that writing the rationale section of a PEP can often be the hardest part, and I think I understand the issues well enough to write that preface. So here's my contribution: >> >> --- >> >> In the domain of object-oriented programming, the usage patterns for interacting with an object can be divided into two basic categories, which are 'invocation' and 'inspection'. > > I'd put 'definition' there. Especially in a language like Python > (dynamic isn't it ?), where additional definitions can be introduced > at run-time, or existing definitions be subtituted to existing ones > (aka 'monkey-patching'). At least where I work, this is an important > category. Java/C++ monkeys can't really understand the importance of > this, tough. > >> Invocation means interacting with an object by invoking its methods. Usually this is combined with polymorphism, so that invoking a given method may run different code depending on the type of an object. >> >> Inspection means the ability for external code (outside of the object's methods) to examine the type or properties of that object, and make decisions on how to treat that object based on that information. >> >> Both usage patterns serve the same general end, which is to be able to support the processing of diverse and potentially novel objects in a uniform way, but at the same time allowing processing decisions to be customized for each different type of object. >> >> In classical OOP theory, invocation is the preferred usage pattern, >> and > > uh, well, but what is classical OOP theory ? What is defined by > SmallTalk, C++ and Java ? Feh ! Since my knowledge of programming is almost entirely self-taught, I don't have much experience as to what are the formalisms that are being taught to young programmers today. However, based on my interactions with programmers who have gone through a formal CS education, I can infer the existence of a "school of thought" of object decomposition and software architecture that follows rules very similar to what I have described. What I lack, however, is bibliographic references for it. >> inspection is actively discouraged, being considered a relic of an earlier, procedural programming style. However, in practice this view is simply too dogmatic and inflexible, and leads to a kind of design rigidity that is very much at odds with the dynamic nature of a language like Python. > > imho it's not a too dogmatic & inflexible view (it is quite correct > that inspection-based dispatch, aka duck-typing (I call it > {L,S,F}uck-typing sometimes, in anger ...), is an evil hack that > should avoided from production software, for obious maintenance > problems) ; but it is very useful in the context of rapid, interactive > developement, when one has to explore the shape of things to come ... I think it's important to understand that 'duck-typing' is not the same as inspection-based dispatch - it is an entirely orthogonal concept. 'Duck-typing' simply means that two classes can be polymorphic without having a common base class or interface - all they have to do is support the same methods. If I want to create a new kind of widget, I don't have to inherit from the base Widget class, all I have to do is make sure that my custom widget class supports all of the proper Widget methods. This is not possible in Java or C++. Duck typing can apply to both invocation ('use this object as a widget') and inspection ('is this object a widget?'). That is why I claim it is orthogonal to the issue of inspection. And I don't think that inspection-based dispatch is an evil hack, but it certainly can be misused. Inspection-based dispatch can be something as simple as "if this is a list, do A, else do B". The evil comes when you use it in a way that lacks discipline - what you might call 'spaghetti inspection'. But it's hard to use proper discipline, because we haven't yet defined what the proper discipline *is* - we haven't set the standard. That is what the ABCs stuff is attempting to establish - to make it possible to use inspection in a way that is 'not evil' because it conforms to a well-understood pattern. >> In particular, there is often a need to process objects in a way that wasn't anticipated by the creator of the object class. It is not always the best solution to build in to every object methods that satisfy >> the > > it is just not possible > >> needs of every possible user of that object. Moreover, there are many powerful dispatch philosophies that are in direct contrast to the classic OOP requirement of behavior being strictly encapsulated >> within > > 'classic OOP requirement' is clearly the dogma there :) > Python developpers should never feel obliged towards it ; especially > since Python owes (almost) nothing to this dogma. > >> an object, examples being rule or pattern-match driven logic. >> >> On the the other hand, one of the criticisms of inspection by classic OOP theorists is the lack of formalisms and the ad hoc nature of what is being inspected. In a language such as Python, in which almost any > > The school of static verification of simple program properties will > never cease to come after us, programmers, to try to sell its highly > unpractical theoretical belief about ways to ensure programs > correctness ; let's not fall in the traps of their evil ways^H^H^H > hidden academic agenda ... > >> aspect of an object can be reflected and directly accessed by external code, there are many different ways to test whether an object conforms to a particular protocol or not. For example, if asking 'is this object a mutable sequence container?', one can look for a base class of 'list', or one can look for a method named '__getitem__'. But note that although these tests may seem obvious, neither of them are correct, as one generates false negatives, and the other false positives. >> >> The generally agreed-upon remedy is to standardize the tests, and group them into a formal arrangement. This is most easily done by associating with each class a set of standard testable properties, either via the inheritance mechanism or some other means. Each test carries with it a set of promises: it contains a promise about the general behavior of the class, and a promise as to what other class methods will be available. >> >> This PEP proposes a particular strategy for organizing these tests known as Abstract Base Classes, or ABC. ABCs are simply Python classes that are added into an object's inheritance tree to signal certain features of that object to an external inspector. Tests are done using isinstance(), and the presence of a particular ABC means that the test has passed. >> >> Like all other things in Python, these promises are in the nature of a gentlemen's agreement - which means that the language does not attempt to enforce that these promises are kept. > > bleh > > What's wrong with associating behaviour to methods ? Currently, the > wrongness lies in the strong coupling between class definition and > method definition. Hence you (also in the name of others) propose an > ABC, for instance, in oder to assess that object conforms to a > protocol. But suppose you work with a class whose definition you don't > control (because it's a piece of Plone/Archetype for instance) and you > want to make it indexable. With ABC, you'd need to basically add a > super class to add behaviour, at runtime, monkey-patching things up > till you make it work, most of the time. This is a terrible thing to > do, even in Python. Or maybe I miss some wonderful well_known recipe > about it ... > > Let's use generic functions instead of ABC's, for god's sake ! > Need to define a common protocol for item access in an object : There's nothing in what I said that prevents the use of generic functions. Part of the reason for defining the ABC's is to give the generic functions something to operate on. If you look at the research material that's been done on generic functions, you'll notice that there's a generalization of generic functions called "predicated-based dispatch". That's simply a fancy way of saying that we do some tests on the calling arguments, and depending on how those tests come out we decide which actual piece of code to run. A subset of predicate-based dispatch is called "type-based dispatch" -- what most people think of as generic functions. Type based dispatch is a specialized form of predicate-based dispatch where all of the tests are type tests, i.e. "if the first argument is of type A, then use this code..." The point is, regardless of whether you are using the full predicate-based dispatch or the more specialized type-based dispatch, the generic function dispatch is nothing more than a series of boolean tests. In many cases, we can optimize the tests by creating lookup tables and such, compressing multiple tests into a single jump calculation so that they may not "look like" sequential boolean tests, but nevertheless that's what they are. The controversy over ABCs is really a controversy over what kinds of tests we should use for dispatching. Should the tests be limited to "is-a" (isinstance) tests, or should we also allow "has-a" (hasattr) tests as well? But generic functions can easily use either kind of test, or any other kind of boolean test for that matter. So nothing about ABCs is incompatible with the use of generic functions. Now, you bring up the point that in some cases, you might want to use tests in a way that allows you to override the classification of an object. In other words, the object itself may have some "native" classification (e.g. 'this is a list, this is a string') that is inherent in its construction, but instead you want to might want to impose a different set of categories (e.g. 'this is a marshall-by-reference, this is a marshall-by-value'), so that objects which are unrelated get grouped together, and vice versa. It seems to me that this is a good argument to have - it's really the crux of much of the previous discussion - but it will need more concrete examples and use cases. You see, even if this "override" feature isn't supported in the ABC system, there's nothing to prevent someone from adding it to the generic dispatch mechanism. Generic functions need not be limited to only using 'isinstance' tests and need not be limited to only using ABCs. An independent taxonomy of types and categories can be built using an entirely parallel data structure, without touching the original classes at all, and dispatching can be based on that. The real question to ask is - what is the 90% solution? In other words, what is the simplest mechanism that covers 90% of the use cases, while allowing the other 10% of use cases to handled by allowing the application programmer to extend the system or add a layer on top of it? Here's how the debate should go: Various proponents will submit designs for an object inspection/classification system that cover those 90% of cases. Ideally, the simplest design will be used as the starting point. Various people will then put forward use cases that aren't included by the candidate design. However, merely putting forward a use case isn't enough, one has to make a convincing argument that such a use case is significant - meaning that it is likely to be commonly occurring, and has no easy workaround. Thus, if someone has a use case that requires that objects be dynamically reclassified at runtime, they should be prepared to show that (a) its a common use case, and (b) there's no obvious way to do it other than to build that feature into the system from the beginning. > use a 'getitem' generic function ... > > Need to implement it for classes you didn't write (or even for your > own) : > @method(FooWidget, int) > def getitem(widget, pos): > # just do it in a FooWidget-y- way > > Well, I just don't dare post this on the py3k list ... I also just > don't get why people are so reluctant to understand what generic > functions are, and how they would solve all these non-problems :) > > Of course, for py3k there should be some bridge between old-style > generic functions (__len__, __getitem__, ...) and something more > general ; something that, above all, DECOUPLES method definition from > class definition. We already know that Python is not about > encapsulation (in the classical dogmatic way), but merely about puting > stuff into namespaces. So let's drop entirely the pseudo encapsulation > thing, and let's go with the one truly modular way. Generic functions > are namespaces, of course, and can be further put into any modules or > namespace kind you want to. > > Cheers, > Aur?lien. > > PS : note how such a getitem gf could generalize also to > hashtable/dict-like access, with something like : > > @method(dict, str): > def getitem(adict, astr): > # just do it ... > > Double dispatch is not just a fancy complicated feature. It allows to > build much more leagible, maintainable code. Ok that might be a little > harder for the folks that maintain the language runtime. But these are > supposed the real men, not the day-to-day users that currently > struggle with a broken 'object model'. > > Wich is broken because inspired by the aforementioned, mindless, > dogmas. Too many people in Python-land have been exposed to Java, alas > ... > > hum, I stop now. By the way, there's something odd about the "reply-to" headers on your email, when I hit "reply all", it replied only to you, not the list. -- Talin From talin at acm.org Tue Apr 17 09:23:39 2007 From: talin at acm.org (Talin) Date: Tue, 17 Apr 2007 00:23:39 -0700 Subject: [Python-3000] An introduction to ABC's In-Reply-To: <462474AD.2050807@acm.org> References: <46211608.7010604@acm.org> <20070416150746.GE17621@crater.logilab.fr> <462474AD.2050807@acm.org> Message-ID: <462475FB.6060405@acm.org> Talin wrote: > By the way, there's something odd about the "reply-to" headers on your > email, when I hit "reply all", it replied only to you, not the list. Oh, duh. I'm an idiot - I didn't notice that the reply was only to me, not the list. Aur?lien, my apologies for publicly posting material that you sent to me privately, that was my fault. However, I still think that the questions you raise are worth discussing, and I don't think you need be concerned about raising them in public. At least, you won't look as dumb as I do right now :) -- Talin From aurelien.campeas at logilab.fr Tue Apr 17 10:02:31 2007 From: aurelien.campeas at logilab.fr (=?iso-8859-1?Q?Aur=E9lien_Camp=E9as?=) Date: Tue, 17 Apr 2007 10:02:31 +0200 Subject: [Python-3000] An introduction to ABC's In-Reply-To: <462475FB.6060405@acm.org> References: <46211608.7010604@acm.org> <20070416150746.GE17621@crater.logilab.fr> <462474AD.2050807@acm.org> <462475FB.6060405@acm.org> Message-ID: <20070417080231.GA4600@crater.logilab.fr> On Tue, Apr 17, 2007 at 12:23:39AM -0700, Talin wrote: > Talin wrote: > > By the way, there's something odd about the "reply-to" headers on your > > email, when I hit "reply all", it replied only to you, not the list. > > Oh, duh. I'm an idiot - I didn't notice that the reply was only to me, > not the list. > > Aur?lien, my apologies for publicly posting material that you sent to me > privately, that was my fault. No problem. > > However, I still think that the questions you raise are worth > discussing, and I don't think you need be concerned about raising them > in public. At least, you won't look as dumb as I do right now :) Well, sorry :). Two points: my feeling is that right now these discussions belong to python-ideas ; also I lack the time to write clean and well-laid contributions that are up to level/quality of these lists (py3k, python-ideas). But the unconclusiveness of all the discussions related to ABCs, interfaces and generic functions only show the immaturity of many on these topics. Including mine, of course, as in: I lack the experience required to back up all my claims about the generic function school of OO. ... (although P. Eby did a great work on this front, he looks a bit lonely imho). > > -- Talin > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/aurelien.campeas%40logilab.fr From nnorwitz at gmail.com Tue Apr 17 10:49:02 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 17 Apr 2007 01:49:02 -0700 Subject: [Python-3000] some test failures on 64-bit Message-ID: test_cpickle (probably all the same issue here): test_xpickle, too ====================================================================== FAIL: test_ints (test.test_cpickle.cPickleTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/neal/python/dev/py3k/Lib/test/pickletester.py", line 503, in test_ints self.assertEqual(expected, n2) AssertionError: -9223372036854775807 != 1 ====================================================================== FAIL: test_ints (test.test_cpickle.cPicklePicklerTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/neal/python/dev/py3k/Lib/test/pickletester.py", line 503, in test_ints self.assertEqual(expected, n2) AssertionError: -9223372036854775807 != 1 ====================================================================== FAIL: test_ints (test.test_cpickle.cPickleListPicklerTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/neal/python/dev/py3k/Lib/test/pickletester.py", line 503, in test_ints self.assertEqual(expected, n2) AssertionError: -9223372036854775807 != 1 ====================================================================== FAIL: test_ints (test.test_cpickle.cPickleFastPicklerTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/neal/python/dev/py3k/Lib/test/pickletester.py", line 503, in test_ints self.assertEqual(expected, n2) AssertionError: -9223372036854775807 != 1 test_gzip: test test_gzip failed -- Traceback (most recent call last): File "/home/neal/python/dev/py3k/Lib/test/test_gzip.py", line 77, in test_many_append ztxt = zgfile.read(8192) File "/home/neal/python/dev/py3k/Lib/gzip.py", line 235, in read self._read(readsize) File "/home/neal/python/dev/py3k/Lib/gzip.py", line 300, in _read self._read_eof() File "/home/neal/python/dev/py3k/Lib/gzip.py", line 316, in _read_eof crc32 = read32(self.fileobj) File "/home/neal/python/dev/py3k/Lib/gzip.py", line 40, in read32 return struct.unpack(" I whacked almost everything in the string (and strop) module. Here are some things left to do: * capwords is defined and used only in one place: ./idlelib/testcode.py ISTM, this can go. * maketrans remains (also implemented in stropmodule.c). It is used 11 times. * ascii_letters is used 17 times * ascii_lowercase is used 2 times * digits is used 15 times * hexdigits is used 2 times * letters is used 9 times * whitespace is used 4 times Some of these uses are in test code. There are a bunch of other constants that aren't used at all in the stdlib. Below is the full list from my grep (which probably missed a few uses due to from string import). n -- Lib/idlelib/UndoDelegator.py: string.ascii_letters + string.digits Lib/idlelib/AutoComplete.py: string.ascii_letters + string.digits Lib/idlelib/AutoComplete.py: string.ascii_letters + string.digits Lib/idlelib/HyperParser.py: string.ascii_letters + string.digits Lib/idlelib/HyperParser.py: string.ascii_letters Lib/idlelib/keybindingDialog.py: string.ascii_lowercase + string.digits Lib/idlelib/keybindingDialog.py: string.ascii_lowercase Lib/idlelib/PyShell.py: string.ascii_letters + string.digits Lib/idlelib/ScriptBinding.py: string.ascii_letters + string.digits Lib/idlelib/AutoExpand.py: string.ascii_letters + string.digits Lib/distutils/util.py: string.whitespace Lib/distutils/util.py: string.whitespace Lib/distutils/fancy_getopt.py: string.maketrans('-', '_') Lib/distutils/fancy_getopt.py: string.maketrans(string.whitespace, ' ' * len(string.whitespace)) Lib/bsddb/test/test_dbshelve.py: string.letters Lib/bsddb/test/test_get_none.py: string.letters Lib/bsddb/test/test_get_none.py: string.letters Lib/bsddb/test/test_basics.py: string.letters Lib/bsddb/test/test_queue.py: string.letters Lib/bsddb/test/test_queue.py: string.letters Lib/msilib/__init__.py: string.digits Lib/plat-riscos/riscospath.py: string.ascii_letters Lib/plat-riscos/rourl2path.py: string.maketrans("/.", "./") Lib/test/test_csv.py: string.letters Lib/test/test_pkgimport.py: string.letters Lib/test/test_pkgimport.py: string.letters Lib/test/test_md5.py: string.hexdigits Lib/test/test_mimetools.py: string.ascii_letters + string.digits Lib/test/test_pkg.py: "print(__name__, 'loading'); from . import string; print(string.spam)"), Lib/test/test_hashlib.py: string.hexdigits Lib/test/string_tests.py: string.maketrans('abc', 'xyz') Lib/test/string_tests.py: string.maketrans Lib/test/string_tests.py: string.maketrans('abc', 'xyz') Lib/test/string_tests.py: string.maketrans('a', 'A') Lib/test/test_urllib2.py: string.maketrans("/.", "./")) Lib/test/test_bigmem.py: string.maketrans('.aZ', '-!$') Lib/plat-mac/gensuitemodule.py: string.ascii_letters + '_' Lib/plat-mac/gensuitemodule.py: string.digits Lib/nturl2path.py: string.ascii_letters Lib/pipes.py: string.ascii_letters + string.digits Lib/textwrap.py: string.maketrans(_whitespace, ' ' * len(_whitespace)) Lib/textwrap.py: string.lowercase Lib/tokenize.py: string.ascii_letters Lib/ntpath.py: string.ascii_letters + string.digits Lib/Cookie.py: string.ascii_letters + string.digits Lib/urllib.py: string.maketrans("", "") Lib/cmd.py: string.ascii_letters + string.digits From barry at python.org Tue Apr 17 14:28:04 2007 From: barry at python.org (Barry Warsaw) Date: Tue, 17 Apr 2007 08:28:04 -0400 Subject: [Python-3000] string module trimming In-Reply-To: References: Message-ID: <173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 17, 2007, at 5:03 AM, Neal Norwitz wrote: > * maketrans remains (also implemented in stropmodule.c). It is > used 11 times. > * ascii_letters is used 17 times > * ascii_lowercase is used 2 times > * digits is used 15 times > * hexdigits is used 2 times > * letters is used 9 times > * whitespace is used 4 times You're not planning on removing these too are you? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRiS9VHEjvBPtnXfVAQJUIQP9Fn+5jclYPtaRwp+W/j78XARk5iIKIi2j EEaOYrtRiJtzsQX743L8hT3x8MGnHWN4D803DFTE7n+gaooGdhBzzHw6jQNIwuIv ZYEjRn7+1kUAYpONB9iepqVhk7KuLMOwRe+VyBRftnxh1bIaXPQxGRCU7ZRM6R+2 s7CiFH0Elc0= =zsiL -----END PGP SIGNATURE----- From guido at python.org Tue Apr 17 15:53:13 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Apr 2007 06:53:13 -0700 Subject: [Python-3000] Draft PEP: Using UTF-8 as the default source encoding In-Reply-To: References: <4621F48A.1090308@v.loewis.de> Message-ID: I don't see how I can fight this overwhelming support, so I'll give it +1 too. On 4/16/07, Travis E. Oliphant wrote: > Martin v. L?wis wrote: > > I'd like to submit the following PEP for Python 3. > > > > Regards, > > Martin > > > > > > PEP: xxx > > Title: Using UTF-8 as the default source encoding > > Version: $Revision $ > > Last-Modified: $Date $ > > Author: Martin v. L?wis > > Status: Draft > > Type: Standards Track > > Content-Type: text/x-rst > > Created: 15-Apr-2007 > > Python-Version: 3.0 > > Post-History: > > > > +1 > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Tue Apr 17 16:02:58 2007 From: barry at python.org (Barry Warsaw) Date: Tue, 17 Apr 2007 10:02:58 -0400 Subject: [Python-3000] i18n prefix In-Reply-To: <662B0B21-F14D-4EFC-BB57-9ACC22E879E6@digi.com.br> References: <662B0B21-F14D-4EFC-BB57-9ACC22E879E6@digi.com.br> Message-ID: <51F9EA46-4562-44DD-8B4E-440E206B0697@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 17, 2007, at 1:35 AM, Humberto Diogenes wrote: > This is my first post to python-3000. After Guido PEPs-deadline > announcement, I decided to discuss an old idea while it still has > some chance of being accepted. :) > > No, I do not want i18n identifiers. What I'd like to propose is a > standard syntax for i18n strings. Something like: > print i"my string" > > My points are: > * i18n is everywhere (or is it the other way around? :-) > * Python has a tradition of incorporating good practices > * _ = gettext.gettext; print _("my string") isn't very nice > > So, do you think that something like i"my string" would be better > than _("my string")? Would it add unnecessary complexity to Python? I've never understood the desire for an i-prefix, though it comes up often. In py3k you won't have u-prefixes, but you'll still have r- prefixes (I'm assuming), so while you'll have less prefix-explosion, I still don't see much benefit in adding a prefix for something that is perfectly well handled by a function call. Besides, if you used a prefix you'd have to explain how the run-time aspects of translations will get handled. Definitely PEP material. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRiTTknEjvBPtnXfVAQKfiwP/WIxeq+iG1hXVYikH4vy6pkHHT96zsxRv tJ9aYuhLOFDmWkhsXpiRsSnPzPMXvth3UHiHCYo+D1aOEUyHyqCAZ97iHcvD+N7T PDycmWGl5dEaOBJ3ieXVV+0Bbv3i+J6QpA1Aj1QdS5o6pkB31STdt+3OPG2eKEiz SQKG+YrpgAI= =cR8m -----END PGP SIGNATURE----- From guido at python.org Tue Apr 17 17:39:19 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Apr 2007 08:39:19 -0700 Subject: [Python-3000] An introduction to ABC's In-Reply-To: <46211608.7010604@acm.org> References: <46211608.7010604@acm.org> Message-ID: Have you seen the (*very* *early* *experimental*) set of ABCs that I started in sandbox/abc/abc.py ? On 4/14/07, Talin wrote: > Part of the reason why I haven't volunteered to write a PEP for ABC's is > because I don't feel that I understand the various proposals and the > background discussion well enough. However, it occurs to me that writing > the rationale section of a PEP can often be the hardest part, and I > think I understand the issues well enough to write that preface. So > here's my contribution: > > --- > > In the domain of object-oriented programming, the usage patterns for > interacting with an object can be divided into two basic categories, > which are 'invocation' and 'inspection'. > > Invocation means interacting with an object by invoking its methods. > Usually this is combined with polymorphism, so that invoking a given > method may run different code depending on the type of an object. > > Inspection means the ability for external code (outside of the object's > methods) to examine the type or properties of that object, and make > decisions on how to treat that object based on that information. > > Both usage patterns serve the same general end, which is to be able to > support the processing of diverse and potentially novel objects in a > uniform way, but at the same time allowing processing decisions to be > customized for each different type of object. > > In classical OOP theory, invocation is the preferred usage pattern, and > inspection is actively discouraged, being considered a relic of an > earlier, procedural programming style. However, in practice this view is > simply too dogmatic and inflexible, and leads to a kind of design > rigidity that is very much at odds with the dynamic nature of a language > like Python. > > In particular, there is often a need to process objects in a way that > wasn't anticipated by the creator of the object class. It is not always > the best solution to build in to every object methods that satisfy the > needs of every possible user of that object. Moreover, there are many > powerful dispatch philosophies that are in direct contrast to the > classic OOP requirement of behavior being strictly encapsulated within > an object, examples being rule or pattern-match driven logic. > > On the the other hand, one of the criticisms of inspection by classic > OOP theorists is the lack of formalisms and the ad hoc nature of what is > being inspected. In a language such as Python, in which almost any > aspect of an object can be reflected and directly accessed by external > code, there are many different ways to test whether an object conforms > to a particular protocol or not. For example, if asking 'is this object > a mutable sequence container?', one can look for a base class of 'list', > or one can look for a method named '__getitem__'. But note that although > these tests may seem obvious, neither of them are correct, as one > generates false negatives, and the other false positives. > > The generally agreed-upon remedy is to standardize the tests, and group > them into a formal arrangement. This is most easily done by associating > with each class a set of standard testable properties, either via the > inheritance mechanism or some other means. Each test carries with it a > set of promises: it contains a promise about the general behavior of the > class, and a promise as to what other class methods will be available. > > This PEP proposes a particular strategy for organizing these tests known > as Abstract Base Classes, or ABC. ABCs are simply Python classes that > are added into an object's inheritance tree to signal certain features > of that object to an external inspector. Tests are done using > isinstance(), and the presence of a particular ABC means that the test > has passed. > > Like all other things in Python, these promises are in the nature of a > gentlemen's agreement - which means that the language does not attempt > to enforce that these promises are kept. > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Tue Apr 17 18:18:38 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 17 Apr 2007 12:18:38 -0400 Subject: [Python-3000] An introduction to ABC's In-Reply-To: <20070417103337.C1402@tribble.ilrt.bris.ac.uk> References: <20070417080231.GA4600@crater.logilab.fr> <46211608.7010604@acm.org> <20070416150746.GE17621@crater.logilab.fr> <462474AD.2050807@acm.org> <462475FB.6060405@acm.org> <20070417080231.GA4600@crater.logilab.fr> Message-ID: <5.1.1.6.0.20070417115834.02b207e8@sparrow.telecommunity.com> At 10:47 AM 4/17/2007 +0100, Jan Grant wrote: >On Tue, 17 Apr 2007, Aur?lien Camp?as wrote: > > > But the unconclusiveness of all the discussions related to ABCs, > > interfaces and generic functions only show the immaturity of many on > > these topics. Including mine, of course, as in: I lack the experience > > required to back up all my claims about the generic function school of > > OO. ... (although P. Eby did a great work on this front, he looks a bit > > lonely imho). > >FWIW Phillip's GFs look more and more like being the pythonic way to do >things. ABCs and interfaces work in a java world, where arguments to a >method arrive with a "preferred" type (ie, the type in the parameter >declaraion) and that type is used to interpret method calls when it >comes to method overloading. Hence Phillip's comment that interfaces are >"receiver-oriented". Yes... and in *Java*, it tends to work out that way because a lot of factors conspire to make it work, like the fact that mere method names aren't enough to make an object acceptable, combined with overloading, lack of really useful generic types, etc. >We don't have that in python, which makes me question the utility of >creating a complex ABC universe prior to having a mechanism for method >dispatch. > >Of course, GF implementations _do_ exist which would give the ABCs some >utility. It's just a shame that there doesn't appear to be one that's >going to be the default way to do it in py3k. FYI, there's actually one tucked away in the pkgutil module since Python 2.5... single dispatch only, but it was the easiest way to support asking future PEP 302 importers what modules are in a package. It's more primitive than the "simplegeneric" package on the Cheeseshop, but would certainly be functional enough to work with ABC's, if it were documented (at the moment it's technically an implementation detail of pkgutil. However, if there's interest, I'd be willing to create a *small* (i.e. about 1000 lines, give or take a few hundred) Py3K/2.6 multi-dispatch/interface/adapters module, written such that it would be extensible for third-party dispatch and interface systems. I'm not sure I could deliver a complete PEP before the deadline, however, and my impression is that Guido feels (or at least felt) that such a thing would be premature. If that has changed, however, I'm willing to write a PEP and take a stab at getting a draft in this week. Anyway, if we do that, we would then have interfaces, adapters, generic functions, *and* ABCs in Py3K, so everybody should be happy... except for those who want only one Obvious Way, of course. :) From pje at telecommunity.com Tue Apr 17 18:46:38 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 17 Apr 2007 12:46:38 -0400 Subject: [Python-3000] An introduction to ABC's In-Reply-To: <462474AD.2050807@acm.org> References: <20070416150746.GE17621@crater.logilab.fr> <46211608.7010604@acm.org> <20070416150746.GE17621@crater.logilab.fr> Message-ID: <5.1.1.6.0.20070417122259.02b206a0@sparrow.telecommunity.com> At 12:18 AM 4/17/2007 -0700, Talin wrote: >The real question to ask is - what is the 90% solution? In other words, >what is the simplest mechanism that covers 90% of the use cases, while >allowing the other 10% of use cases to handled by allowing the >application programmer to extend the system or add a layer on top of it? That's easy, actually. If you have a generic function, then you can add an overload for your specific type that does whatever you need it to do. In other words, the proper discipline for inspection is to place the inspection within a generic function that your clients can add overloads to. Note, by the way, that even fairly sophisticated generic function implementations are pretty trivial in Python. Take a look at Guido's multi-dispatch prototype for Py3K, or the simplegeneric package on the Cheeseshop, or even the generic function implementation in pkgutil (Python 2.5 and up). That makes them a pretty good 90% solution, especially since they don't even *need* ABC's for this to work. It's quite okay to define methods for concrete types and let the user decide whether their implementation suffices, doing something like: thefunction[mytype] = thefunction[othertype] to say that *in the context of thefunction*, mytype is an adequate replacement for othertype. (ABCs and interfaces have the problem, however, of being context-free. Thus, the proper discipline for interfaces is to use adaptation and to have "one interface per use case", rather than "one interface per concept".) >Thus, if someone has a use case that requires that objects be >dynamically reclassified at runtime, Dynamic reclassification isn't the real problem, it's just as much a problem for static classifications that *disagree with the receiver's choice of inspection*. Pydoc and earlier Zope versions have this problem all over the place, no dynamic reclassification necessary. The key principle here is that the *client* should get to decide what happens to their objects, because the author of the client is the only one who knows what he or she *intends*. The author of the library cannot *know* the use cases of his or her users, and should therefore refuse the temptation to guess. Generic functions allow library authors to provide *overrideable defaults*, in place of *unchangeable guesses*. (Which is what if/then tests are, regardless of whether they use ABCs, concrete types, interfaces, or even hasattr().) From nnorwitz at gmail.com Tue Apr 17 19:27:44 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 17 Apr 2007 10:27:44 -0700 Subject: [Python-3000] string module trimming In-Reply-To: <173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org> References: <173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org> Message-ID: On 4/17/07, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Apr 17, 2007, at 5:03 AM, Neal Norwitz wrote: > > > * maketrans remains (also implemented in stropmodule.c). It is > > used 11 times. > > * ascii_letters is used 17 times > > * ascii_lowercase is used 2 times > > * digits is used 15 times > > * hexdigits is used 2 times > > * letters is used 9 times > > * whitespace is used 4 times > > You're not planning on removing these too are you? I don't have any plans, just considering options. Move them somewhere? Perhaps, trim the ones that are unused. In a unicode world, I'm not sure how much some of these make sense. letters stands out more than others. I don't know enough about unicode to know if digits or whitespace can be diff. The Template class is in string, which seems pretty appropriate. Although with the naming re-org that will be coming, should string be renamed to stringlib (or whatever)? What would you like to see for things like maketrans, capwords, etc? n From barry at python.org Tue Apr 17 22:58:19 2007 From: barry at python.org (Barry Warsaw) Date: Tue, 17 Apr 2007 16:58:19 -0400 Subject: [Python-3000] string module trimming In-Reply-To: References: <173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 17, 2007, at 1:27 PM, Neal Norwitz wrote: > On 4/17/07, Barry Warsaw wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Apr 17, 2007, at 5:03 AM, Neal Norwitz wrote: >> >> > * maketrans remains (also implemented in stropmodule.c). It is >> > used 11 times. >> > * ascii_letters is used 17 times >> > * ascii_lowercase is used 2 times >> > * digits is used 15 times >> > * hexdigits is used 2 times >> > * letters is used 9 times >> > * whitespace is used 4 times >> >> You're not planning on removing these too are you? > > I don't have any plans, just considering options. Move them > somewhere? Perhaps, trim the ones that are unused. In a unicode > world, I'm not sure how much some of these make sense. letters stands > out more than others. I don't know enough about unicode to know if > digits or whitespace can be diff. Certainly it makes sense to keep the ascii_* constants. The docs say letters and lowercase are locale-aware, so I guess it makes sense to keep those too. > The Template class is in string, which seems pretty appropriate. > Although with the naming re-org that will be coming, should string be > renamed to stringlib (or whatever)? I don't know, I think 'string' still works fine for me. > What would you like to see for things like maketrans, capwords, etc? I've used them both a few times, so I guess I'd be fine with keeping them where they are. capwords() seems easy enough to write the one or two times I'd need it, so I don't have strong feelings about them. I know at one point there was talk about a 'text' or 'textlib' package, which might serve as a way to organize such other modules as textwrap, *StringIO, etc. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRiU07HEjvBPtnXfVAQKcOQQAt/1r05QGcDJ5VYKQ6xo9v5oA7vGpxtCp oAZJJxreQOlXbxYjcupEzaX9pfFOTng3hGBmrSSspxivOK5v8kcKZvBfxf3Ndt2j DKAH8rH79IoZO4TCQ8ehuEBxA6v912YUJtLmcwIWxsgO+ZgEBeTDV8UaddxEWRWS mQRow+zP7ew= =CkXk -----END PGP SIGNATURE----- From lists at cheimes.de Tue Apr 17 22:12:10 2007 From: lists at cheimes.de (Christian Heimes) Date: Tue, 17 Apr 2007 22:12:10 +0200 Subject: [Python-3000] string module trimming In-Reply-To: References: <173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org> Message-ID: Neal Norwitz schrieb: > I don't have any plans, just considering options. Move them > somewhere? Perhaps, trim the ones that are unused. In a unicode > world, I'm not sure how much some of these make sense. letters stands > out more than others. I don't know enough about unicode to know if > digits or whitespace can be diff. What do you think about replacing the definitions by information from the unicode character properties database. The information are available somewhere in Python: http://docs.python.org/lib/re-syntax.html \w ... With LOCALE, it will match the set [0-9_] plus whatever characters are defined as alphanumeric for the current locale. If UNICODE is set, this will match the characters [0-9_] plus whatever is classified as alphanumeric in the Unicode character properties database. Christian From guido at python.org Tue Apr 17 23:23:15 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Apr 2007 14:23:15 -0700 Subject: [Python-3000] string module trimming In-Reply-To: References: <173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org> Message-ID: On 4/17/07, Barry Warsaw wrote: > Certainly it makes sense to keep the ascii_* constants. The docs say > letters and lowercase are locale-aware, so I guess it makes sense to > keep those too. Really? Remember that Py3k won't be using the locale for anything character/string-related -- strings will all be unicode and the islower() methods and friends just use the Unicode database. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Tue Apr 17 23:27:25 2007 From: barry at python.org (Barry Warsaw) Date: Tue, 17 Apr 2007 17:27:25 -0400 Subject: [Python-3000] string module trimming In-Reply-To: References: <173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 17, 2007, at 5:23 PM, Guido van Rossum wrote: > On 4/17/07, Barry Warsaw wrote: >> Certainly it makes sense to keep the ascii_* constants. The docs say >> letters and lowercase are locale-aware, so I guess it makes sense to >> keep those too. > > Really? Remember that Py3k won't be using the locale for anything > character/string-related -- strings will all be unicode and the > islower() methods and friends just use the Unicode database. Well then in in that case, zap 'em! :) - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRiU7vXEjvBPtnXfVAQJO6gQAkZgv+nKJgZI0Vkf5Ykw7k0fLKWjPi+mp bJdkoyZaD/vcS5Katy+okoDCvYy/aK2q4xt1zhppJjhn1wxOPA+2xYKNOU9vvLhP RaH/aZwd09fiJAQLsXiaPeb21KWbW6kPoinVV+V5DokOXGdvZ3AnoC/BPERwkOc2 6Q6ocAKjcvU= =wTup -----END PGP SIGNATURE----- From guido at python.org Tue Apr 17 23:28:00 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Apr 2007 14:28:00 -0700 Subject: [Python-3000] string module trimming In-Reply-To: References: <173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org> Message-ID: On 4/17/07, Christian Heimes wrote: > Neal Norwitz schrieb: > > I don't have any plans, just considering options. Move them > > somewhere? Perhaps, trim the ones that are unused. In a unicode > > world, I'm not sure how much some of these make sense. letters stands > > out more than others. I don't know enough about unicode to know if > > digits or whitespace can be diff. > > What do you think about replacing the definitions by information from > the unicode character properties database. The information are available > somewhere in Python: > > http://docs.python.org/lib/re-syntax.html > > \w ... With LOCALE, it will match the set [0-9_] plus whatever > characters are defined as alphanumeric for the current locale. If > UNICODE is set, this will match the characters [0-9_] plus whatever is > classified as alphanumeric in the Unicode character properties database. Yes, unicode.islower() and friends have this information. It would be silly to set e.g. letters to a string of all unicode letters -- that would be a string of 46618 characters! Similar, there are 304 unicode digits. (And this is in a narrow Unicode build, only supporting the basic Unicode plane, 0--2**16!) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jimjjewett at gmail.com Wed Apr 18 00:55:46 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 17 Apr 2007 18:55:46 -0400 Subject: [Python-3000] string module trimming In-Reply-To: References: <173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org> Message-ID: On 4/17/07, Christian Heimes wrote: > Neal Norwitz schrieb: > > I don't have any plans, just considering options. Move them > > somewhere? Perhaps, trim the ones that are unused. In a unicode > > world, I'm not sure how much some of these make sense. letters stands > > out more than others. I don't know enough about unicode to know if > > digits or whitespace can be diff. There are several additional characters in both sets, and plenty of reasons that a given program might want to use a restricted set. (Probably those already in string, or else a letters grouping set by locale.) > What do you think about replacing the definitions by information from > the unicode character properties database. The information are available > somewhere in Python: > http://docs.python.org/lib/re-syntax.html > \w ... With LOCALE, it will match the set [0-9_] plus whatever > characters are defined as alphanumeric for the current locale. If > UNICODE is set, this will match the characters [0-9_] plus whatever is > classified as alphanumeric in the Unicode character properties database. There are reasons to want exactly ASCII. There are also reasons to want only "local" letters. For example, in a French interface, I might want to include the extra French letters, but not the Greek. Also note that regex isn't quite the only use of those letters groupings. -jJ From guido at python.org Wed Apr 18 01:07:09 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Apr 2007 16:07:09 -0700 Subject: [Python-3000] string module trimming In-Reply-To: References: <173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org> Message-ID: On 4/17/07, Jim Jewett wrote: > There are also reasons to want only "local" letters. For example, in > a French interface, I might want to include the extra French letters, > but not the Greek. The Unicode world doesn't support this directly AFAIK. The locale module doesn't deal with Unicode, only with 8-bit characters (not multi-byte characters). You'll lose this anyway. Certainly string.letters is not going to provide this functionality. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Wed Apr 18 01:53:48 2007 From: lists at cheimes.de (Christian Heimes) Date: Wed, 18 Apr 2007 01:53:48 +0200 Subject: [Python-3000] removing functions from string module In-Reply-To: References: Message-ID: Neal Norwitz schrieb: > I'm tempted to ask the same thing about the types module. Don't forget the 'new' module. It's a candidate for 'svn rm'. """Create new objects of various types. Deprecated. This module is no longer required except for backward compatibility. Objects of most types can now be created by calling the type object. """ Christian From jcarlson at uci.edu Wed Apr 18 03:19:04 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 17 Apr 2007 18:19:04 -0700 Subject: [Python-3000] removing functions from string module In-Reply-To: References: Message-ID: <20070417181112.6320.JCARLSON@uci.edu> Christian Heimes wrote: > > Neal Norwitz schrieb: > > I'm tempted to ask the same thing about the types module. > > Don't forget the 'new' module. It's a candidate for 'svn rm'. > > """Create new objects of various types. Deprecated. > > This module is no longer required except for backward compatibility. > Objects of most types can now be created by calling the type object. > """ I would generally agree with you, except that when monkey patching instances... self.method = new.instancemethod(other.method.im_func, self, self.__class__) ... is a bit more understandable than... self.metbod = type(other.method)(other.method.im_func, self, self.__class__) ... at least in the examples that I've seen. Never mind that new is documented in the standard library documentation, but the types are not. - Josiah From lists at cheimes.de Wed Apr 18 03:49:16 2007 From: lists at cheimes.de (Christian Heimes) Date: Wed, 18 Apr 2007 03:49:16 +0200 Subject: [Python-3000] removing functions from string module In-Reply-To: <20070417181112.6320.JCARLSON@uci.edu> References: <20070417181112.6320.JCARLSON@uci.edu> Message-ID: Josiah Carlson schrieb: > I would generally agree with you, except that when monkey patching > instances... > self.method = new.instancemethod(other.method.im_func, > self, > self.__class__) > > .... is a bit more understandable than... > self.metbod = type(other.method)(other.method.im_func, > self, > self.__class__) > > .... at least in the examples that I've seen. Never mind that new is > documented in the standard library documentation, but the types are not. I strongly emphasize on "a bit more". One needs a lot of inside knowledge about internal representation to rebind a method to another instance. The following idea is much easier to understand: self.method = boundmethod(self, method_or_function) Christian From steven.bethard at gmail.com Wed Apr 18 04:26:05 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 17 Apr 2007 20:26:05 -0600 Subject: [Python-3000] removing functions from string module In-Reply-To: <20070417181112.6320.JCARLSON@uci.edu> References: <20070417181112.6320.JCARLSON@uci.edu> Message-ID: On 4/17/07, Josiah Carlson wrote: > > Christian Heimes wrote: > > > > Neal Norwitz schrieb: > > > I'm tempted to ask the same thing about the types module. > > > > Don't forget the 'new' module. It's a candidate for 'svn rm'. > > > > """Create new objects of various types. Deprecated. > > > > This module is no longer required except for backward compatibility. > > Objects of most types can now be created by calling the type object. > > """ > > I would generally agree with you, except that when monkey patching > instances... > self.method = new.instancemethod(other.method.im_func, > self, > self.__class__) > > ... is a bit more understandable than... > self.metbod = type(other.method)(other.method.im_func, > self, > self.__class__) > > ... at least in the examples that I've seen. Yes, but you could just as easily write this as:: self.method = other.method.im_func.__get__(self) which I'd contend is better than the original new.instancemethod version. Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From guido at python.org Wed Apr 18 05:45:18 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Apr 2007 20:45:18 -0700 Subject: [Python-3000] An introduction to ABC's In-Reply-To: <5.1.1.6.0.20070417115834.02b207e8@sparrow.telecommunity.com> References: <46211608.7010604@acm.org> <20070416150746.GE17621@crater.logilab.fr> <462474AD.2050807@acm.org> <462475FB.6060405@acm.org> <20070417080231.GA4600@crater.logilab.fr> <20070417103337.C1402@tribble.ilrt.bris.ac.uk> <5.1.1.6.0.20070417115834.02b207e8@sparrow.telecommunity.com> Message-ID: On 4/17/07, Phillip J. Eby wrote: > At 10:47 AM 4/17/2007 +0100, Jan Grant wrote: > >On Tue, 17 Apr 2007, Aur?lien Camp?as wrote: > > > > > But the unconclusiveness of all the discussions related to ABCs, > > > interfaces and generic functions only show the immaturity of many on > > > these topics. Including mine, of course, as in: I lack the experience > > > required to back up all my claims about the generic function school of > > > OO. ... (although P. Eby did a great work on this front, he looks a bit > > > lonely imho). > > > >FWIW Phillip's GFs look more and more like being the pythonic way to do > >things. ABCs and interfaces work in a java world, where arguments to a > >method arrive with a "preferred" type (ie, the type in the parameter > >declaraion) and that type is used to interpret method calls when it > >comes to method overloading. Hence Phillip's comment that interfaces are > >"receiver-oriented". > > Yes... and in *Java*, it tends to work out that way because a lot of > factors conspire to make it work, like the fact that mere method names > aren't enough to make an object acceptable, combined with overloading, lack > of really useful generic types, etc. > > > >We don't have that in python, which makes me question the utility of > >creating a complex ABC universe prior to having a mechanism for method > >dispatch. > > > >Of course, GF implementations _do_ exist which would give the ABCs some > >utility. It's just a shame that there doesn't appear to be one that's > >going to be the default way to do it in py3k. > > FYI, there's actually one tucked away in the pkgutil module since Python > 2.5... single dispatch only, but it was the easiest way to support asking > future PEP 302 importers what modules are in a package. It's more > primitive than the "simplegeneric" package on the Cheeseshop, but would > certainly be functional enough to work with ABC's, if it were documented > (at the moment it's technically an implementation detail of pkgutil. > > However, if there's interest, I'd be willing to create a *small* (i.e. > about 1000 lines, give or take a few hundred) Py3K/2.6 > multi-dispatch/interface/adapters module, written such that it would be > extensible for third-party dispatch and interface systems. I'm not sure I > could deliver a complete PEP before the deadline, however, and my > impression is that Guido feels (or at least felt) that such a thing would > be premature. If that has changed, however, I'm willing to write a PEP and > take a stab at getting a draft in this week. > > Anyway, if we do that, we would then have interfaces, adapters, generic > functions, *and* ABCs in Py3K, so everybody should be happy... except for > those who want only one Obvious Way, of course. :) I'd like to see that PEP and that implementation. I am planning to make time to write my own PEP promoting ABCs. I think it would be great if your code and mine could coexist and cooperate instead of competing -- it's much easier to accept two PEPs than to have to choose between two competing proposals. I don't take the TOOWTDI concern too seriously; surely while there will be some overlap, for many use cases it will be obvious which approach to take, and covering more use cases means fewer times where applications or frameworks will feel the need to roll their own. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jcarlson at uci.edu Wed Apr 18 06:41:49 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 17 Apr 2007 21:41:49 -0700 Subject: [Python-3000] removing functions from string module In-Reply-To: References: <20070417181112.6320.JCARLSON@uci.edu> Message-ID: <20070417213402.6325.JCARLSON@uci.edu> "Steven Bethard" wrote: > On 4/17/07, Josiah Carlson wrote: > > Christian Heimes wrote: > > > Neal Norwitz schrieb: > > > > I'm tempted to ask the same thing about the types module. > > > > > > Don't forget the 'new' module. It's a candidate for 'svn rm'. > > > > > > """Create new objects of various types. Deprecated. > > > > > > This module is no longer required except for backward compatibility. > > > Objects of most types can now be created by calling the type object. > > > """ > > > > I would generally agree with you, except that when monkey patching > > instances... > > self.method = new.instancemethod(other.method.im_func, > > self, > > self.__class__) > > Yes, but you could just as easily write this as:: > > self.method = other.method.im_func.__get__(self) > > which I'd contend is better than the original new.instancemethod version. It is certainly easier to read, but I don't believe it is as self-documenting as the new.instancemethod variant. What do you get when calling 'new.instancemethod'? A new instance method. Is func.__get__ documented anywhere? I've never seen it before (which is why I use new). Christian Heimes wrote: > I strongly emphasize on "a bit more". One needs a lot of inside > knowledge about internal representation to rebind a method to another > instance. The following idea is much easier to understand: > > self.method = boundmethod(self, method_or_function) I agree that the above is easier to understand, but boundmethod is not a builtin in 2.3, 2.4, or 2.5. Even with my own use-case of monkey patching instances with functions or methods, I would resist its inclusion as a builtin. - Josiah From steven.bethard at gmail.com Wed Apr 18 07:11:20 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 17 Apr 2007 23:11:20 -0600 Subject: [Python-3000] removing functions from string module In-Reply-To: <20070417213402.6325.JCARLSON@uci.edu> References: <20070417181112.6320.JCARLSON@uci.edu> <20070417213402.6325.JCARLSON@uci.edu> Message-ID: On 4/17/07, Josiah Carlson wrote: > > "Steven Bethard" wrote: > > On 4/17/07, Josiah Carlson wrote: > > > Christian Heimes wrote: > > > > Neal Norwitz schrieb: > > > > > I'm tempted to ask the same thing about the types module. > > > > > > > > Don't forget the 'new' module. It's a candidate for 'svn rm'. > > > > > > > > """Create new objects of various types. Deprecated. > > > > > > > > This module is no longer required except for backward compatibility. > > > > Objects of most types can now be created by calling the type object. > > > > """ > > > > > > I would generally agree with you, except that when monkey patching > > > instances... > > > self.method = new.instancemethod(other.method.im_func, > > > self, > > > self.__class__) > > > > Yes, but you could just as easily write this as:: > > > > self.method = other.method.im_func.__get__(self) > > > > which I'd contend is better than the original new.instancemethod version. > > It is certainly easier to read, but I don't believe it is as > self-documenting as the new.instancemethod variant. What do you get > when calling 'new.instancemethod'? A new instance method. > > Is func.__get__ documented anywhere? In the Reference: http://docs.python.org/ref/descriptor-invocation.html But I agree it's not as prominent or as clear as it should be. Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From erno at iki.fi Wed Apr 18 11:23:40 2007 From: erno at iki.fi (Erno Kuusela) Date: 18 Apr 2007 12:23:40 +0300 Subject: [Python-3000] IPv6 address tuple References: <4602E9D8.6080803@v.loewis.de> <20070323095711.GO14803@erno.iki.fi> <46050350.600@v.loewis.de> Message-ID: "Martin v. L?wis" writes: > > I first suggested just chopping off the flowid and scopeid parts of the > > tuple. Itojun's reply seems to indicate we could potentially get away > > with merging the scopeid to the IP address part in a standard fashion, > > and discarding the flowid part. > > Thanks for this research? As a next step, it would be interesting to > see how it is done elsewhere (besides C and Python). > > As you may see, I would be really uncomfortable changing this in > a way that locks out interesting applications, or deviates too far > from the IPv6 community. So compatibility with the rest of the > world is of utmost importance. > > If you could collect options, established tradition, pros and cons > in a document, that would be good. I've not managed to get a survey of other environments done so I'd like to ask others on this list to do a little digging on other languages they are familiar with. Hackers with knowledge about socket interfaces in Ruby, Perl, Java and C# or any other high-level socket mapping, I'd like answers to the following questions: * How is the socket addres (or equivalent) abstracted, what mechanisms are there to make it convenient for code to handle both IPv4 and IPv6 sockets? * If there's a high-level semi-automatic DNS interface, how does that deal with v4 vs v6? Eg, can I ask "make a TCP connection to www.ripe.net port 80, I don't care v4 or v6" * In what way (if any) are the flowinfo and scopeid parts carried along the with the object doing the part of struct sockaddr? -- erno From jimjjewett at gmail.com Wed Apr 18 21:24:54 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 18 Apr 2007 15:24:54 -0400 Subject: [Python-3000] string module trimming In-Reply-To: References: <173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org> Message-ID: On 4/17/07, Guido van Rossum wrote: > On 4/17/07, Jim Jewett wrote: > > There are also reasons to want only "local" letters. For example, in > > a French interface, I might want to include the extra French letters, > > but not the Greek. > The Unicode world doesn't support this directly AFAIK. Alphabets do tend to be contiguous, but ... not directly, no. > The locale module doesn't deal with Unicode, only with 8-bit characters (not > multi-byte characters). You'll lose this anyway. Certainly > string.letters is not going to provide this functionality. But for languages in Latin1, 8-bit characters are sufficient -- anything with more than 8 bits is by definition not a (local) letter. I won't swear that localizations currently replace string.letters with the appropriately ordered (slight) superset, but it is a valid use case, and string* (or text*) is clearly the right place. -jJ From guido at python.org Wed Apr 18 23:18:47 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 18 Apr 2007 14:18:47 -0700 Subject: [Python-3000] string module trimming In-Reply-To: References: <173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org> Message-ID: On 4/18/07, Jim Jewett wrote: > On 4/17/07, Guido van Rossum wrote: > > The locale module doesn't deal with Unicode, only with 8-bit characters (not > > multi-byte characters). You'll lose this anyway. Certainly > > string.letters is not going to provide this functionality. > > But for languages in Latin1, 8-bit characters are sufficient -- > anything with more than 8 bits is by definition not a (local) letter. Latin-1 is just another encoding (and not a very useful one given that it can't encode all of Unicode). I don't want to define a feature that only works for Latin-1. > I won't swear that localizations currently replace string.letters with > the appropriately ordered (slight) superset, but it is a valid use > case, and string* (or text*) is clearly the right place. The right solution for locale-dependent collation for sure isn't having a string containing all the letters in the right order. There are plenty of languages where that approach doesn't even work. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Apr 18 23:57:57 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 18 Apr 2007 14:57:57 -0700 Subject: [Python-3000] Fixing super anyone? Message-ID: Is anyone available to write up a PEP on how to turn super into a keyword? Inside regular and class methods, super.foo(args) should be equivalent to super(ThisClass, self).foo(args). I think there are ways to make the old syntax work too, but maybe that's only necessary for 2.6. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jimjjewett at gmail.com Wed Apr 18 23:59:40 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 18 Apr 2007 17:59:40 -0400 Subject: [Python-3000] string module trimming In-Reply-To: References: <173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org> Message-ID: On 4/18/07, Guido van Rossum wrote: > On 4/18/07, Jim Jewett wrote: > > On 4/17/07, Guido van Rossum wrote: > > > The locale module doesn't deal with Unicode, only with 8-bit characters (not > > > multi-byte characters). You'll lose this anyway. Certainly > > > string.letters is not going to provide this functionality. > > But for languages in Latin1, 8-bit characters are sufficient -- > > anything with more than 8 bits is by definition not a (local) letter. > Latin-1 is just another encoding (and not a very useful one given that > it can't encode all of Unicode). I don't want to define a feature that > only works for Latin-1. Today, string.letters works most easily with ASCII supersets, and is effectively limited to 8-bit encodings. Once everything is unicode, I don't think that 8-bit restriction should apply any more. > > I won't swear that localizations currently replace string.letters with > > the appropriately ordered (slight) superset, but it is a valid use > > case, and string* (or text*) is clearly the right place. > The right solution for locale-dependent collation for sure isn't > having a string containing all the letters in the right order. There > are plenty of languages where that approach doesn't even work. Theoretically, English is one of those non-working languages. (Names in bibliographic entries are supposed to be alphabetized according to language of origin.) In practice, ordered-list-of-chars works well enough, often enough. It often works better than sorting by code point, which is the only obvious alternative. Unless I missed it (and I may have), unicode itself sort of ducks the question about how to sort strings. Python really needs to provide *an* answer, but I'm not sure it is possible to provide the (single) correct answer. string.letters is one workaround, and I don't think we should remove it until a better solution (or workaround) is available. -jJ From jimjjewett at gmail.com Thu Apr 19 00:03:47 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 18 Apr 2007 18:03:47 -0400 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: Message-ID: On 4/18/07, Guido van Rossum wrote: > Is anyone available to write up a PEP on how to turn super into a > keyword? Inside regular and class methods, super.foo(args) should be > equivalent to super(ThisClass, self).foo(args). I think there are ways > to make the old syntax work too, but maybe that's only necessary for > 2.6. Does this mean it should find the super of the current class (which isn't yet defined when the method is being defined)? That would be a slight change (though probably an improvement) against today's lookup-by-name. If that change is OK, and no one else volunteers, I'll try to take a go at it this weekend. -jJ From guido at python.org Thu Apr 19 00:08:29 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 18 Apr 2007 15:08:29 -0700 Subject: [Python-3000] string module trimming In-Reply-To: References: <173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org> Message-ID: On 4/18/07, Jim Jewett wrote: > On 4/18/07, Guido van Rossum wrote: > > On 4/18/07, Jim Jewett wrote: > > > On 4/17/07, Guido van Rossum wrote: > > > > The locale module doesn't deal with Unicode, only with 8-bit characters (not > > > > multi-byte characters). You'll lose this anyway. Certainly > > > > string.letters is not going to provide this functionality. > > > > But for languages in Latin1, 8-bit characters are sufficient -- > > > anything with more than 8 bits is by definition not a (local) letter. > > > Latin-1 is just another encoding (and not a very useful one given that > > it can't encode all of Unicode). I don't want to define a feature that > > only works for Latin-1. > > Today, string.letters works most easily with ASCII supersets, and is > effectively limited to 8-bit encodings. Once everything is unicode, I > don't think that 8-bit restriction should apply any more. But we already went over this. There are over 40K letters in Unicode. It simply makes no sense to have a string.letters approaching that size. > > > I won't swear that localizations currently replace string.letters with > > > the appropriately ordered (slight) superset, but it is a valid use > > > case, and string* (or text*) is clearly the right place. > > > The right solution for locale-dependent collation for sure isn't > > having a string containing all the letters in the right order. There > > are plenty of languages where that approach doesn't even work. > > Theoretically, English is one of those non-working languages. (Names > in bibliographic entries are supposed to be alphabetized according to > language of origin.) > > In practice, ordered-list-of-chars works well enough, often enough. > It often works better than sorting by code point, which is the only > obvious alternative. > > Unless I missed it (and I may have), unicode itself sort of ducks the > question about how to sort strings. Python really needs to provide > *an* answer, but I'm not sure it is possible to provide the (single) > correct answer. The Unicode standard certainly has a solution, but it is complicated and I don't believe it is currently implemented in core Python. > string.letters is one workaround, and I don't think we should remove > it until a better solution (or workaround) is available. I disagree. The correct solution is to implement the Unicode support for locale-specific sorting. Remember that the locale module supports only a single, global locale at a time. This renders it totally useless in many apps requiring locale support (such as web servers). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Apr 19 00:15:49 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 18 Apr 2007 15:15:49 -0700 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: Message-ID: On 4/18/07, Jim Jewett wrote: > On 4/18/07, Guido van Rossum wrote: > > Is anyone available to write up a PEP on how to turn super into a > > keyword? Inside regular and class methods, super.foo(args) should be > > equivalent to super(ThisClass, self).foo(args). I think there are ways > > to make the old syntax work too, but maybe that's only necessary for > > 2.6. > > Does this mean it should find the super of the current class (which > isn't yet defined when the method is being defined)? > > That would be a slight change (though probably an improvement) against > today's lookup-by-name. Right. It should definitely use the current class and not the name of the current class. I hope you'll be able to find a way to implement this! Perhaps the class could be gotten from a hidden cell reference, where the cell is filled in by the metaclass whe the class is created. (The cell can't represent the result of the super call, because that also incorporates the current value of 'self', for which you can use the first argument.) So the syntax "super" could be translated into "super(__thisclass__, self)" where __thisclass__ is the cell reference, self refers to the (current or original?) value of the first argument, and super is exactly the same as the 2.x super function. (Though it could be referenced via an opcode.) > If that change is OK, and no one else volunteers, I'll try to take a > go at it this weekend. Thanks! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Thu Apr 19 00:22:56 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 19 Apr 2007 10:22:56 +1200 Subject: [Python-3000] string module trimming In-Reply-To: References: <173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org> Message-ID: <46269A40.3030602@canterbury.ac.nz> Christian Heimes wrote: > What do you think about replacing the definitions by information from > the unicode character properties database. I'm not sure I like the idea of pulling a rather large database into the basic string module. Also, even in a unicode world, it can be useful to deal just with ascii letters and digits, e.g. when processing a programming language. -- Greg From guido at python.org Thu Apr 19 00:34:06 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 18 Apr 2007 15:34:06 -0700 Subject: [Python-3000] string module trimming In-Reply-To: <46269A40.3030602@canterbury.ac.nz> References: <173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org> <46269A40.3030602@canterbury.ac.nz> Message-ID: On 4/18/07, Greg Ewing wrote: > Christian Heimes wrote: > > > What do you think about replacing the definitions by information from > > the unicode character properties database. > > I'm not sure I like the idea of pulling a rather large > database into the basic string module. Agreed. Though most of the database is loaded as part of the unicode string object anyway, to support isupper() etc. > Also, even in a unicode world, it can be useful to > deal just with ascii letters and digits, e.g. when > processing a programming language. Right, I like having ascii_letters etc. (though I don't care about where it lives or this particular name -- only about the functionality). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Thu Apr 19 00:41:40 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 19 Apr 2007 10:41:40 +1200 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: Message-ID: <46269EA4.3070500@canterbury.ac.nz> Guido van Rossum wrote: > Is anyone available to write up a PEP on how to turn super into a > keyword? Inside regular and class methods, super.foo(args) should be > equivalent to super(ThisClass, self).foo(args). As I'm still not convinced that super() is very useful in the first place, I'm not sure I want it made into a keyword at all. I'd rather see syntactic support for making a normal inherited method call without danger of using the wrong class, e.g. if you change a base class but forget to update inherited calls to match. -- Greg From guido at python.org Thu Apr 19 00:54:09 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 18 Apr 2007 15:54:09 -0700 Subject: [Python-3000] Fixing super anyone? In-Reply-To: <46269EA4.3070500@canterbury.ac.nz> References: <46269EA4.3070500@canterbury.ac.nz> Message-ID: On 4/18/07, Greg Ewing wrote: > Guido van Rossum wrote: > > Is anyone available to write up a PEP on how to turn super into a > > keyword? Inside regular and class methods, super.foo(args) should be > > equivalent to super(ThisClass, self).foo(args). > > As I'm still not convinced that super() is very > useful in the first place, I'm not sure I want > it made into a keyword at all. Well, I and others *do* find it useful (indispensible, in fact) for writing classes that support cooperative multiple inheritance. That's not a sport for casual users, but if you need it, super is the only game in town. > I'd rather see syntactic support for making a > normal inherited method call without danger of > using the wrong class, e.g. if you change a > base class but forget to update inherited calls > to match. You will get that too with the proposal. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jimjjewett at gmail.com Thu Apr 19 01:08:59 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 18 Apr 2007 19:08:59 -0400 Subject: [Python-3000] string module trimming In-Reply-To: References: Message-ID: On 4/18/07, Guido van Rossum wrote: > On 4/18/07, Jim Jewett wrote: > > Today, string.letters works most easily with ASCII supersets, and is > > effectively limited to 8-bit encodings. Once everything is unicode, I > > don't think that 8-bit restriction should apply any more. > But we already went over this. There are over 40K letters in Unicode. > It simply makes no sense to have a string.letters approaching that > size. Agreed. But there aren't 40K (alphabetic) letters in any particular locale. Most individual languages will have less than 100. As a proxy for measuring "local" characters, I'll note that during some optimization drives for Pango (e.g., http://primates.ximian.com/~federico/news-2005-11.html#04 ) it turned out that there were only two non C-J-K languages that needed more than 256 cache positions in their character glyph tables. > > Unless I missed it (and I may have), unicode itself sort of ducks the > > question about how to sort strings. Python really needs to provide > > *an* answer, but I'm not sure it is possible to provide the (single) > > correct answer. > The Unicode standard certainly has a solution, but it is complicated > and I don't believe it is currently implemented in core Python. I guess you're right; I saw too many alternatives the last time I looked, and must have stopped reading http://unicode.org/reports/tr10/ after section 1, where it becomes obvious that there is no context-free right answer. > > string.letters is one workaround, and I don't think we should remove > > it until a better solution (or workaround) is available. > I disagree. The correct solution is to implement the Unicode support > for locale-specific sorting. And set-inclusion. I'm not convinced that waiting for such a heavyweight solution is really the best choice, particularly since the spec itself warns against using the strictest forms (too inefficient). > Remember that the locale module supports only a single, global locale > at a time. This renders it totally useless in many apps requiring > locale support (such as web servers). Fair enough. -jJ From guido at python.org Thu Apr 19 01:16:43 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 18 Apr 2007 16:16:43 -0700 Subject: [Python-3000] string module trimming In-Reply-To: References: Message-ID: On 4/18/07, Jim Jewett wrote: > On 4/18/07, Guido van Rossum wrote: > > On 4/18/07, Jim Jewett wrote: > > > > Today, string.letters works most easily with ASCII supersets, and is > > > effectively limited to 8-bit encodings. Once everything is unicode, I > > > don't think that 8-bit restriction should apply any more. > > > But we already went over this. There are over 40K letters in Unicode. > > It simply makes no sense to have a string.letters approaching that > > size. > > Agreed. But there aren't 40K (alphabetic) letters in any particular > locale. Most individual languages will have less than 100. Isn't that excluding the written language of half the world population (at least China, Korea and Japan)? > As a proxy for measuring "local" characters, I'll note that during > some optimization drives for Pango (e.g., > http://primates.ximian.com/~federico/news-2005-11.html#04 ) it turned > out that there were only two non C-J-K languages that needed more than > 256 cache positions in their character glyph tables. But here we're talking features, not optimizations. I really don't think it's a good idea to propose a feature that can't be used reasonably for CJK languages. > > > Unless I missed it (and I may have), unicode itself sort of ducks the > > > question about how to sort strings. Python really needs to provide > > > *an* answer, but I'm not sure it is possible to provide the (single) > > > correct answer. > > > The Unicode standard certainly has a solution, but it is complicated > > and I don't believe it is currently implemented in core Python. > > I guess you're right; I saw too many alternatives the last time I > looked, and must have stopped reading http://unicode.org/reports/tr10/ > after section 1, where it becomes obvious that there is no > context-free right answer. > > > > string.letters is one workaround, and I don't think we should remove > > > it until a better solution (or workaround) is available. > > > I disagree. The correct solution is to implement the Unicode support > > for locale-specific sorting. > > And set-inclusion. For set-inclusion we already have isalpha() etc. That should be enough. I really don't see much of a use case for inquiries of the type "is this a letter in my locale" -- by the time you are doing that, you probably are only thinking of one specific locale, and then you should just reject non-locale charaters altogether rather than treating them as punctuation. > I'm not convinced that waiting for such a heavyweight solution is > really the best choice, particularly since the spec itself warns > against using the strictest forms (too inefficient). > > > Remember that the locale module supports only a single, global locale > > at a time. This renders it totally useless in many apps requiring > > locale support (such as web servers). > > Fair enough. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jyasskin at gmail.com Thu Apr 19 08:19:26 2007 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Wed, 18 Apr 2007 23:19:26 -0700 Subject: [Python-3000] string module trimming In-Reply-To: References: Message-ID: <5d44f72f0704182319j124b4743he67b669630d303ad@mail.gmail.com> On 4/18/07, Jim Jewett wrote: > On 4/18/07, Guido van Rossum wrote: > > On 4/18/07, Jim Jewett wrote: > > But we already went over this. There are over 40K letters in Unicode. > > It simply makes no sense to have a string.letters approaching that > > size. > > Agreed. But there aren't 40K (alphabetic) letters in any particular > locale. Most individual languages will have less than 100. I missed the beginning of this discussion, so sorry if you've already covered this. Are you saying that in your app, just because I've set the en_US locale, I won't be able to type "????"? Or that those characters won't be recognized as letters? The Unicode character database (http://www.unicode.org/ucd/) seems like the obvious way to handle character properties if you want to get the right answers. Namast?, Jeffrey Yasskin From jcarlson at uci.edu Thu Apr 19 08:50:17 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 18 Apr 2007 23:50:17 -0700 Subject: [Python-3000] string module trimming In-Reply-To: <5d44f72f0704182319j124b4743he67b669630d303ad@mail.gmail.com> References: <5d44f72f0704182319j124b4743he67b669630d303ad@mail.gmail.com> Message-ID: <20070418233622.6344.JCARLSON@uci.edu> "Jeffrey Yasskin" wrote: > On 4/18/07, Jim Jewett wrote: > > On 4/18/07, Guido van Rossum wrote: > > > On 4/18/07, Jim Jewett wrote: > > > But we already went over this. There are over 40K letters in Unicode. > > > It simply makes no sense to have a string.letters approaching that > > > size. > > > > Agreed. But there aren't 40K (alphabetic) letters in any particular > > locale. Most individual languages will have less than 100. > > I missed the beginning of this discussion, so sorry if you've already > covered this. Are you saying that in your app, just because I've set > the en_US locale, I won't be able to type "????"? Or that those > characters won't be recognized as letters? If I understand the conversation correctly, the discussion is what will be in string.letters, and what will be handled in str.upper(), etc., when a locale is set. > The Unicode character database (http://www.unicode.org/ucd/) seems > like the obvious way to handle character properties if you want to get > the right answers. Certainly, but having 40k characters in string.letters seems like a bit of overkill, for *any* locale. It seems as though it only makes sense to include the letters for the current locale as string.letters, and to handle str.upper(), etc., as determined by the locale. In terms of sorting, since all (unicode) strings should be comparable to one another, using the unicode-specified ordering would seem to make sense, unless it is something other than code point values. If it isn't code point values (which seems to be the implication), then we need to decide if we want to check a 128kbyte table (for UCS-2 builds) in order to sort strings (though cache lookup locality may make this a moot point for most comparisons). - Josiah From jyasskin at gmail.com Thu Apr 19 17:14:15 2007 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Thu, 19 Apr 2007 08:14:15 -0700 Subject: [Python-3000] string module trimming In-Reply-To: <20070418233622.6344.JCARLSON@uci.edu> References: <5d44f72f0704182319j124b4743he67b669630d303ad@mail.gmail.com> <20070418233622.6344.JCARLSON@uci.edu> Message-ID: <5d44f72f0704190814t729d4149p43847449db523f14@mail.gmail.com> On 4/18/07, Josiah Carlson wrote: > "Jeffrey Yasskin" wrote: > > I missed the beginning of this discussion, so sorry if you've already > > covered this. Are you saying that in your app, just because I've set > > the en_US locale, I won't be able to type "????"? Or that those > > characters won't be recognized as letters? > > If I understand the conversation correctly, the discussion is what will > be in string.letters, and what will be handled in str.upper(), etc., > when a locale is set. string.letters should go away because I don't know of any correct uses of it, and as you say 40K letters is too long. Searching a list is the wrong way to decide whether a character is a letter, and case transformations don't work a character at a time (consider what happens with "?".upper() (That is, U+00DF, German Small Sharp S)). http://www.unicode.org/Public/UNIDATA/SpecialCasing.txt defines the mappings that aren't 1-1. There are some that are locale-specific, but you can do a pretty good job ignoring the language, as long as you allow strings to change length. > > The Unicode character database (http://www.unicode.org/ucd/) seems > > like the obvious way to handle character properties if you want to get > > the right answers. > > Certainly, but having 40k characters in string.letters seems like a bit > of overkill, for *any* locale. It seems as though it only makes sense > to include the letters for the current locale as string.letters, and to > handle str.upper(), etc., as determined by the locale. As far as I understand, "letters for the current locale" is the same as "letters" in Unicode. Can you point me to a character that is a letter in one locale but not in another? (The third column of http://www.unicode.org/Public/UNIDATA/UnicodeData.txt defines the character's category, and http://www.unicode.org/Public/UNIDATA/UCD.html#General_Category_Values says what it means.) > In terms of sorting, since all (unicode) strings should be comparable to > one another, using the unicode-specified ordering would seem to make > sense, unless it is something other than code point values. If it isn't > code point values (which seems to be the implication), then we need to > decide if we want to check a 128kbyte table (for UCS-2 builds) in order > to sort strings (though cache lookup locality may make this a moot point > for most comparisons). If you just need to store strings in an order-based data structure (which I guess is moot for python with its hashes), then codepoint order is fine. If you intend to show users a sorted list, then you have to use the real collation algorithm or you'll produce the wrong answer. I don't understand the algorithm's details, but ICU has an implementation, and http://icu-project.org/charts/icu4c_footprint.html claims that the data for all languages fits in 354K. UCS-2 is an old and broken fixed-width encoding that cannot represent characters above U+FFFF. Nobody should ever use it. You probably meant UTF-16. -- Namast?, Jeffrey Yasskin From jcarlson at uci.edu Thu Apr 19 19:22:13 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu, 19 Apr 2007 10:22:13 -0700 Subject: [Python-3000] string module trimming In-Reply-To: <5d44f72f0704190814t729d4149p43847449db523f14@mail.gmail.com> References: <20070418233622.6344.JCARLSON@uci.edu> <5d44f72f0704190814t729d4149p43847449db523f14@mail.gmail.com> Message-ID: <20070419091246.6347.JCARLSON@uci.edu> "Jeffrey Yasskin" wrote: > On 4/18/07, Josiah Carlson wrote: > > "Jeffrey Yasskin" wrote: > > > I missed the beginning of this discussion, so sorry if you've already > > > covered this. Are you saying that in your app, just because I've set > > > the en_US locale, I won't be able to type "????"? Or that those > > > characters won't be recognized as letters? > > > > If I understand the conversation correctly, the discussion is what will > > be in string.letters, and what will be handled in str.upper(), etc., > > when a locale is set. > > string.letters should go away because I don't know of any correct uses > of it, and as you say 40K letters is too long. Searching a list is the > wrong way to decide whether a character is a letter, and case > transformations don't work a character at a time (consider what > happens with "??".upper() (That is, U+00DF, German Small Sharp S)). > http://www.unicode.org/Public/UNIDATA/SpecialCasing.txt defines the > mappings that aren't 1-1. There are some that are locale-specific, but > you can do a pretty good job ignoring the language, as long as you > allow strings to change length. Because we aren't mutating unicode strings, this isn't an issue. I respond below regarding string.letters . > > > The Unicode character database (http://www.unicode.org/ucd/) seems > > > like the obvious way to handle character properties if you want to get > > > the right answers. > > > > Certainly, but having 40k characters in string.letters seems like a bit > > of overkill, for *any* locale. It seems as though it only makes sense > > to include the letters for the current locale as string.letters, and to > > handle str.upper(), etc., as determined by the locale. > > As far as I understand, "letters for the current locale" is the same > as "letters" in Unicode. Can you point me to a character that is a > letter in one locale but not in another? (The third column of > http://www.unicode.org/Public/UNIDATA/UnicodeData.txt defines the > character's category, and > http://www.unicode.org/Public/UNIDATA/UCD.html#General_Category_Values > says what it means.) Neither I, nor I believe Python mean 'letters' in the general sense, but the 'alphabet' of a particular locale. For example, en_US compared to sv_SE . > > In terms of sorting, since all (unicode) strings should be comparable to > > one another, using the unicode-specified ordering would seem to make > > sense, unless it is something other than code point values. If it isn't > > code point values (which seems to be the implication), then we need to > > decide if we want to check a 128kbyte table (for UCS-2 builds) in order > > to sort strings (though cache lookup locality may make this a moot point > > for most comparisons). > > If you just need to store strings in an order-based data structure > (which I guess is moot for python with its hashes), then codepoint > order is fine. If you intend to show users a sorted list, then you > have to use the real collation algorithm or you'll produce the wrong > answer. I don't understand the algorithm's details, but ICU has an > implementation, and http://icu-project.org/charts/icu4c_footprint.html > claims that the data for all languages fits in 354K. It could probably even be reduced lower than 354K with two tables and a comparison function that knows how to handle surrogates. > UCS-2 is an old and broken fixed-width encoding that cannot represent > characters above U+FFFF. Nobody should ever use it. You probably meant > UTF-16. You are more or less right. Earlier versions of Windows were limited to UCS-2, and I believe earlier versions of Python on Windows were also limited to UCS-2. For narrow builds we use UTF-16, with surrogate pairs and everything (though a unicode string consisting of a single surrogate pair will have length 2, not 1 as would be expected). - Josiah From guido at python.org Thu Apr 19 19:40:41 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 19 Apr 2007 10:40:41 -0700 Subject: [Python-3000] [Python-Dev] python3k change to slicing In-Reply-To: References: Message-ID: [+python-3000; followups please remove python-dev] -1 While this may be theoretically preferable, I believe that in practice changing this would be a major pain for very little gain. I don't recall ever finding a bug related to this feature, and I believe it's occasionally useful. Here's something that would be much more cumbersome with your proposed change: suppose I have a string of unknown length and I want to get the first three characters, or less if it's not that long. Today I can write s[:3]. With your proposal I would have to write s[:min(3, len(s))]. --Guido On 4/19/07, Neal Becker wrote: > There is one thing I'd like to see changed in a future python. I always > found it surprising, that > >>> x = [1,2,3,4,5] > >>> x[1:10] > [2, 3, 4, 5] > > is not an error. This is perhaps the only case (but a fundamental one!) > where an error is silently ignored. > > I really can't think of a good justification for it. If I really meant > x[1:] > I would have said so. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jason.orendorff at gmail.com Thu Apr 19 20:51:24 2007 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Thu, 19 Apr 2007 14:51:24 -0400 Subject: [Python-3000] string module trimming In-Reply-To: References: Message-ID: On 4/18/07, Jim Jewett wrote: > On 4/18/07, Guido van Rossum wrote: > > On 4/18/07, Jim Jewett wrote: > > > > Today, string.letters works most easily with ASCII supersets, and is > > > effectively limited to 8-bit encodings. Once everything is unicode, I > > > don't think that 8-bit restriction should apply any more. > > > But we already went over this. There are over 40K letters in Unicode. > > It simply makes no sense to have a string.letters approaching that > > size. > > Agreed. But there aren't 40K (alphabetic) letters in any particular > locale. Most individual languages will have less than 100. But isn't the hip thing these days to use UTF-8 as the character set, as in "LC_ALL=en_US.UTF-8"? I picked a random Linux machine and this happened to be the default LANG. A quick C test revealed that iswalpha() thinks there are 45,974 letters in that locale. Anyway, let's discuss your use cases. One: UIs that involve displaying all the letters. > There are also reasons to want only "local" letters. For example, in > a French interface, I might want to include the extra French letters, > but not the Greek. But POSIX locales don't claim to provide this information, as far as I can tell. It seems like for a quick throwaway program, you're better off ignoring languages. And for even a slightly serious program, string.letters is wrong in a dozen ways. Seriously, a table of alphabets that's saner than string.letters is pretty trivial to write: alphabets = { 'en': list("ABCDEFGHIJKLMNOPQRSTUVWXYZ"), 'es': ("A B C Ch D E F G H I J K L Ll M " + "N \u00d1 O P Q R S T U V W X Y Z").split(), ... } ...and you can go from there. Two: Collation. Collation can be done right: provide a function text.sort_key() that converts a str into an opaque thing that has the desired ordering. You would use it like so: names.sort(key=text.sort_key) records.sort(key=lambda rec: text.sort_key(rec.title)) Jython can implement this using java.text.Collator. IronPython can use CultureInfo.CompareInfo.GetSortKey(). CPython can call LCMapString() on Windows and wcsxfrm() on POSIX, falling back on just returning the string itself. -j From jimjjewett at gmail.com Thu Apr 19 20:52:00 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 19 Apr 2007 14:52:00 -0400 Subject: [Python-3000] string module trimming In-Reply-To: <20070418233622.6344.JCARLSON@uci.edu> References: <5d44f72f0704182319j124b4743he67b669630d303ad@mail.gmail.com> <20070418233622.6344.JCARLSON@uci.edu> Message-ID: On 4/19/07, Josiah Carlson wrote: > > "Jeffrey Yasskin" wrote: > > On 4/18/07, Jim Jewett wrote: > > > On 4/18/07, Guido van Rossum wrote: > > > > On 4/18/07, Jim Jewett wrote: > > > Agreed. But there aren't 40K (alphabetic) letters in any particular > > > locale. Most individual languages will have less than 100. > > ... Are you saying that in your app, just because I've set > > the en_US locale, I won't be able to type "????"? Or that those > > characters won't be recognized as letters? The latter. Some applications may reject them for that reason; for example some domain registrars have policies to prevent domain name spoofing with similar-looking characters. One way to do that is to say that a character used in a domain name (under that registrar) is limited to those letters used by the appropriate national language. > In terms of sorting, since all (unicode) strings should be comparable to > one another, using the unicode-specified ordering would seem to make > sense, unless it is something other than code point values. It is definately something other than code-point values. In particular, see section 1.8 (common misconceptions) of http://unicode.org/reports/tr10/ The sorting isn't fully defined without locale-specific tailoring and a Unicode Element Collation Table (default 4 bytes/char, though compressible). There is a default tailoring and default Unicode Element Collation Table; it looks (but I haven't proven to myself) as if these defaults are sufficient for most use, but certainly not all usage. Unicode sorting (even with your own collation table) definately requires normalization, which is something Python has been careful not to promise. (There were some arguments over whether normalization was even possible to do in a strictly correct fashion. I didn't understand them well enough to remember the summary.) Unless the "repetoire of supported character sequences" is (unnaturally) restricted, normalization is only an intermediate step; a third representation is constructed for the actual comparison. This third form can be done a few characters at a time, but then you have to redo it for the next comparison. As best I can easily tell about the default settings, there are distinct strings which are equal, unequal strings which are not ordered, and strings for which you must compare multiple characters at once ("x"<"y", but "xz">"yz") -jJ From jimjjewett at gmail.com Thu Apr 19 21:34:25 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 19 Apr 2007 15:34:25 -0400 Subject: [Python-3000] string module trimming In-Reply-To: References: Message-ID: On 4/19/07, Jason Orendorff wrote: > On 4/18/07, Jim Jewett wrote: > > On 4/18/07, Guido van Rossum wrote: > > > On 4/18/07, Jim Jewett wrote: > Seriously, a table of alphabets that's saner than string.letters > is pretty trivial to write: > alphabets = { > 'en': list("ABCDEFGHIJKLMNOPQRSTUVWXYZ"), > 'es': ("A B C Ch D E F G H I J K L Ll M " > + "N \u00d1 O P Q R S T U V W X Y Z").split(), > ... > } I suspect you could do a bit better with properties already in the unicode database -- but offhand, I'm not sure how. If setting locale put the correct one in string.letters for me, that would be great. > Two: Collation. > Collation can be done right: provide a function text.sort_key() > that converts a str into an opaque thing that has the desired > ordering. If this function is context-free (depending on only the input string), it will violate the unicode standard (and, apparently, do the wrong thing for some languages -- usually including French). Also, that key is probably larger than the original string, and they warn against trying to create it in a single pass. I'm not saying relying strictly on unicode properties *can't* be done right -- I'm just saying that it would be very difficult and very inefficient, so it probably won't happen soon -- which is an argument for keeping the half-measures around meanwhile. -jJ From jimjjewett at gmail.com Thu Apr 19 21:48:00 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 19 Apr 2007 15:48:00 -0400 Subject: [Python-3000] Fixing super anyone? In-Reply-To: <46269EA4.3070500@canterbury.ac.nz> References: <46269EA4.3070500@canterbury.ac.nz> Message-ID: On 4/18/07, Greg Ewing wrote: > I'd rather see syntactic support for making a > normal inherited method call without danger of > using the wrong class, e.g. if you change a > base class but forget to update inherited calls > to match. Could you give an example? My first three readings of this were (1) Huh? (2) You mean, if the __bases__ are changed, super doesn't get updated? (It does, as I just verified.) (3) You mean, with an explicit call, such as B.meth1(self), you want some sort of warning if class B changes the signature of meth1? (Presumably at compile-time; an incompatible change will already be caught at run-time.) -jJ From greg.ewing at canterbury.ac.nz Fri Apr 20 00:43:17 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 20 Apr 2007 10:43:17 +1200 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <46269EA4.3070500@canterbury.ac.nz> Message-ID: <4627F085.7000301@canterbury.ac.nz> Guido van Rossum wrote: > On 4/18/07, Greg Ewing wrote: >> I'd rather see syntactic support for making a >> normal inherited method call without danger of >> using the wrong class, e.g. if you change a >> base class but forget to update inherited calls >> to match. > > You will get that too with the proposal. Is there an actual proposal yet? I'd like to see how this would work. -- Greg From guido at python.org Fri Apr 20 00:55:40 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 19 Apr 2007 15:55:40 -0700 Subject: [Python-3000] Need help completing ABC pep Message-ID: I've started a PEP on Abstract Base Classes (ABCs), PEP 3119: http://www.python.org/dev/peps/pep-3119/ While I'm not ready yet to answer tough questions about this compared to alternative proposals, I *am* ready for feedback on the various open issues sprinkled throughout the current text, especially concerning decisions regarding exactly which operations to include in the various ABCs, and also regarding the level of detail required in the PEP. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Fri Apr 20 03:41:39 2007 From: brett at python.org (Brett Cannon) Date: Thu, 19 Apr 2007 18:41:39 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: Message-ID: On 4/19/07, Guido van Rossum wrote: > I've started a PEP on Abstract Base Classes (ABCs), PEP 3119: > > http://www.python.org/dev/peps/pep-3119/ > > While I'm not ready yet to answer tough questions about this compared > to alternative proposals, I *am* ready for feedback on the various > open issues sprinkled throughout the current text, especially > concerning decisions regarding exactly which operations to include in > the various ABCs, and also regarding the level of detail required in > the PEP. Since there is no embedded copy of the PEP I am just quoting the various open issues and such. That way a simple search of the PEP wil find what am talking about. At the end I have some random comments that do not directly mention an open issue. * "perhaps abstractmethod and AbstractInstantiationError should become built-ins, Abstract's functionality should be subsumed by object, and AbstractClass's functionality should be merged into type" Maybe, but what is the performance cost of doing the checking? If it's too high I don't want to have to pay for it if I am not using anything that uses ABCs. Then again, if the decision is made to run with this then a whole lot of stdlib code is going to end up with ABCs and thus the cost will be unavoidable. * "Should we also implement the issubset and issuperset methods found on the set type in Python 2? As these are just aliases for __le__ and __ge__, I'm tempted to leave these out." Leave them out. Not terribly needed plus it is better to start out small. They can easily be added later if called for. * Discussing sets: "Open issue: Should this class also implement the operators and/or methods that compute union, intersection, symmetric and asymmetric difference?" If you are willing to let them just return the built-in set type (since one could argue that 'set' is the base type that almost any set will inherit from), then go ahead and add them as concrete methods. Otherwise leave them out. * "Should we support all the operations implemented by the Python 2 set type?" No, only core methods should be directly supported. Everything else is just convenience functions. If these can be implemented simply as concrete methods then fine, but otherwise leave them out. * "Do we need BasicMapping and IterableMapping? Perhaps we should just start with Mapping" Just Mapping. Doing two checks for Mapping and Iterable is not exactly complex. =) * "Also pop, popitem, setdefault?" Eh, my gut reaction is "no". But you do have 'pop' on MutableSequence. * Discussing sequences: "Other candidate methods, which can all have default concrete implementations that only depend on __len__ and __getitem__ with an integer argument: __reversed__, index, count, __add__, __mul__, __eq__, __lt__, __le__." I say go ahead and include the concrete implementation. * Discussing sequences: "What about += and *=?" No. Only worth it when a special gain is available by the implementation. * "Do we have a use case for Cardinal (Integer >= 0)?" No. * "Do we need Indexable (converts to Integer using __index__)? Or is that just subsumed into Integer and should we use __index__ only at the C level?" I actually immediately thought of this as a use-case for ABCs when I started reading this PEP. So my vote is for Indexable, although the name doesn't fit as the object doesn't get indexed but can be used as an index. So I would rather have Index or IndexItem or something. * "Should we strive to use ABCs for all areas of Python?" No. As you said, Orderable is probably the only thing really missing. But if we want some inspiration we can look at what Haskell has. * "Perhaps the numeric classes could be moved to a separate PEP" Well, one could argue then that we should have a PEP on how ABCs should work and then individual PEPs on each group of defined ABCs. I suspect that if we go with ABCs we will not take their definitions lightly and end up haggling over definitions a lot. Non-open issue comments: About Iterator, does it subclass Iterable? It seems to meet the definition. I really don't like the name of Finite. When I read the name before knowing what it represented I really had no clue what it represented. In hindsight it makes sense, but it not immediately obvious. What about 'append' on MutableSequence? -Brett From guido at python.org Fri Apr 20 04:35:59 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 19 Apr 2007 19:35:59 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: Message-ID: On 4/19/07, Brett Cannon wrote: > On 4/19/07, Guido van Rossum wrote: > > I've started a PEP on Abstract Base Classes (ABCs), PEP 3119: > > > > http://www.python.org/dev/peps/pep-3119/ > > > > While I'm not ready yet to answer tough questions about this compared > > to alternative proposals, I *am* ready for feedback on the various > > open issues sprinkled throughout the current text, especially > > concerning decisions regarding exactly which operations to include in > > the various ABCs, and also regarding the level of detail required in > > the PEP. > > Since there is no embedded copy of the PEP I am just quoting the > various open issues and such. That way a simple search of the PEP wil > find what am talking about. At the end I have some random comments > that do not directly mention an open issue. Thanks! > * "perhaps abstractmethod and AbstractInstantiationError should become > built-ins, Abstract's functionality should be subsumed by object, and > AbstractClass's functionality should be merged into type" > > Maybe, but what is the performance cost of doing the checking? If > it's too high I don't want to have to pay for it if I am not using > anything that uses ABCs. Then again, if the decision is made to run > with this then a whole lot of stdlib code is going to end up with ABCs > and thus the cost will be unavoidable. Good point. Now that you mention this, and I don't understand why I didn't already do it this way in sandbox/abc, the check could be made at *class creation time*, and at instantiation time it should just have to make one tiny check. > * "Should we also implement the issubset and issuperset methods found > on the set type in Python 2? As these are just aliases for __le__ and > __ge__, I'm tempted to leave these out." > > Leave them out. Not terribly needed plus it is better to start out > small. They can easily be added later if called for. Already gone. I think you read the penultimate version. :-) > * Discussing sets: "Open issue: Should this class also implement the > operators and/or methods that compute union, intersection, symmetric > and asymmetric difference?" > > If you are willing to let them just return the built-in set type > (since one could argue that 'set' is the base type that almost any set > will inherit from), then go ahead and add them as concrete methods. > Otherwise leave them out. I doubt that 'set' will be inherited from much (except for trivial stuff) but returning a 'set' does make some sense. Although for trivial subclasses (e.g. adding a different repr or a brand new method) you'd like the results also to be an instance of the subclass. If we could find a way to do that it would be best. > * "Should we support all the operations implemented by the Python 2 set type?" > > No, only core methods should be directly supported. Everything else > is just convenience functions. If these can be implemented simply as > concrete methods then fine, but otherwise leave them out. Sure. But one can differ about whether union() is the convenience and __or__() the core, or vice versa. :-) > * "Do we need BasicMapping and IterableMapping? Perhaps we should just > start with Mapping" > > Just Mapping. Doing two checks for Mapping and Iterable is not > exactly complex. =) OK, I'll think about this. > * "Also pop, popitem, setdefault?" > > Eh, my gut reaction is "no". But you do have 'pop' on MutableSequence. But for sequences it trivially maps on core methods. pop() for mappings is a little more convoluted (how to pick a random key? Just pick the first one I guess?) > * Discussing sequences: "Other candidate methods, which can all have > default concrete implementations that only depend on __len__ and > __getitem__ with an integer argument: __reversed__, index, count, > __add__, __mul__, __eq__, __lt__, __le__." > > I say go ahead and include the concrete implementation. OK. I was just being lazy. > * Discussing sequences: "What about += and *=?" > > No. Only worth it when a special gain is available by the implementation. OTOH not requiring these gives them fuzzy semantics -- some mutable sequences will return self, others a new object. That's not good. I'd like to standardize self-modifying +=, *=. > * "Do we have a use case for Cardinal (Integer >= 0)?" > > No. > > > * "Do we need Indexable (converts to Integer using __index__)? Or is > that just subsumed into Integer and should we use __index__ only at > the C level?" > > I actually immediately thought of this as a use-case for ABCs when I > started reading this PEP. So my vote is for Indexable, although the > name doesn't fit as the object doesn't get indexed but can be used as > an index. So I would rather have Index or IndexItem or something. Index is a good one. > * "Should we strive to use ABCs for all areas of Python?" > > No. As you said, Orderable is probably the only thing really missing. > But if we want some inspiration we can look at what Haskell has. Pointer? (I'm lazy. :-) > * "Perhaps the numeric classes could be moved to a separate PEP" > > Well, one could argue then that we should have a PEP on how ABCs > should work and then individual PEPs on each group of defined ABCs. I > suspect that if we go with ABCs we will not take their definitions > lightly and end up haggling over definitions a lot. Yeah, but IMO the collections are the most contentious area, and doing these right is a motivation for the whole idea of ABCs -- it makes more concrete "what ABCs can do for you." > Non-open issue comments: > > About Iterator, does it subclass Iterable? It seems to meet the definition. It does, and it says so. > I really don't like the name of Finite. When I read the name before > knowing what it represented I really had no clue what it represented. > In hindsight it makes sense, but it not immediately obvious. I have been playing with Sizeable and Lengthy, but rejected both as being too cute. Other suggestions? > What about 'append' on MutableSequence? Oh, duh. It's listed as add. Will fix. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Fri Apr 20 05:53:26 2007 From: brett at python.org (Brett Cannon) Date: Thu, 19 Apr 2007 20:53:26 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: Message-ID: On 4/19/07, Guido van Rossum wrote: > On 4/19/07, Brett Cannon wrote: > > On 4/19/07, Guido van Rossum wrote: > > > I've started a PEP on Abstract Base Classes (ABCs), PEP 3119: > > > > > > http://www.python.org/dev/peps/pep-3119/ > > > > > > While I'm not ready yet to answer tough questions about this compared > > > to alternative proposals, I *am* ready for feedback on the various > > > open issues sprinkled throughout the current text, especially > > > concerning decisions regarding exactly which operations to include in > > > the various ABCs, and also regarding the level of detail required in > > > the PEP. > > > > Since there is no embedded copy of the PEP I am just quoting the > > various open issues and such. That way a simple search of the PEP wil > > find what am talking about. At the end I have some random comments > > that do not directly mention an open issue. > > Thanks! > No problem. Gave me something to do while I waited for my girlfriend to get off of work. =) > > * "perhaps abstractmethod and AbstractInstantiationError should become > > built-ins, Abstract's functionality should be subsumed by object, and > > AbstractClass's functionality should be merged into type" > > > > Maybe, but what is the performance cost of doing the checking? If > > it's too high I don't want to have to pay for it if I am not using > > anything that uses ABCs. Then again, if the decision is made to run > > with this then a whole lot of stdlib code is going to end up with ABCs > > and thus the cost will be unavoidable. > > Good point. Now that you mention this, and I don't understand why I > didn't already do it this way in sandbox/abc, the check could be made > at *class creation time*, and at instantiation time it should just > have to make one tiny check. > I thought you were already doing that. =) I was just worried that it would be prohibitively expensive at class creation time for some short script that used mostly stdlib code that happened to use ABCs. > > * "Should we also implement the issubset and issuperset methods found > > on the set type in Python 2? As these are just aliases for __le__ and > > __ge__, I'm tempted to leave these out." > > > > Leave them out. Not terribly needed plus it is better to start out > > small. They can easily be added later if called for. > > Already gone. I think you read the penultimate version. :-) > Quite possibly. Did this all offline. > > * Discussing sets: "Open issue: Should this class also implement the > > operators and/or methods that compute union, intersection, symmetric > > and asymmetric difference?" > > > > If you are willing to let them just return the built-in set type > > (since one could argue that 'set' is the base type that almost any set > > will inherit from), then go ahead and add them as concrete methods. > > Otherwise leave them out. > > I doubt that 'set' will be inherited from much (except for trivial > stuff) but returning a 'set' does make some sense. Although for > trivial subclasses (e.g. adding a different repr or a brand new > method) you'd like the results also to be an instance of the subclass. > If we could find a way to do that it would be best. > Well, self.__class__() would work, right? That should resolve properly. Or maybe we need that super keyword. =) > > * "Should we support all the operations implemented by the Python 2 set type?" > > > > No, only core methods should be directly supported. Everything else > > is just convenience functions. If these can be implemented simply as > > concrete methods then fine, but otherwise leave them out. > > Sure. But one can differ about whether union() is the convenience and > __or__() the core, or vice versa. :-) > So I say base it on ease of implementation. If it can be done trivially in five lines of Python code then include it, otherwise let the user do it. > > * "Do we need BasicMapping and IterableMapping? Perhaps we should just > > start with Mapping" > > > > Just Mapping. Doing two checks for Mapping and Iterable is not > > exactly complex. =) > > OK, I'll think about this. > > > * "Also pop, popitem, setdefault?" > > > > Eh, my gut reaction is "no". But you do have 'pop' on MutableSequence. > > But for sequences it trivially maps on core methods. pop() for > mappings is a little more convoluted (how to pick a random key? Just > pick the first one I guess?) > Right, which is why my gut reaction was "no". I don't know how you choose. But if you are popping off of a mapping object you should know the order will be undefined. I would take your latter suggestion: get a list of keys, take the first one and use that as what gets popped off. > > * Discussing sequences: "Other candidate methods, which can all have > > default concrete implementations that only depend on __len__ and > > __getitem__ with an integer argument: __reversed__, index, count, > > __add__, __mul__, __eq__, __lt__, __le__." > > > > I say go ahead and include the concrete implementation. > > OK. I was just being lazy. > =) > > * Discussing sequences: "What about += and *=?" > > > > No. Only worth it when a special gain is available by the implementation. > > OTOH not requiring these gives them fuzzy semantics -- some mutable > sequences will return self, others a new object. That's not good. I'd > like to standardize self-modifying +=, *=. > Then I guess you answered your own question. If you were asking about semantics I would just follow what lists do. > > * "Do we have a use case for Cardinal (Integer >= 0)?" > > > > No. > > > > > > * "Do we need Indexable (converts to Integer using __index__)? Or is > > that just subsumed into Integer and should we use __index__ only at > > the C level?" > > > > I actually immediately thought of this as a use-case for ABCs when I > > started reading this PEP. So my vote is for Indexable, although the > > name doesn't fit as the object doesn't get indexed but can be used as > > an index. So I would rather have Index or IndexItem or something. > > Index is a good one. > > > * "Should we strive to use ABCs for all areas of Python?" > > > > No. As you said, Orderable is probably the only thing really missing. > > But if we want some inspiration we can look at what Haskell has. > > Pointer? (I'm lazy. :-) > Damn, I guess I have to do a little extra work now.... See http://haskell.org/onlinereport/prelude-index.html for the Haskell 98 report and the prelude (Haskell's version of built-ins). You care about the type classes (which they just call "classes" on the page). If you follow the links for any of the type classes they will tell you exactly what functions must be defined for something to meet that type class. Some of them are Haskell-specific, but not all. > > * "Perhaps the numeric classes could be moved to a separate PEP" > > > > Well, one could argue then that we should have a PEP on how ABCs > > should work and then individual PEPs on each group of defined ABCs. I > > suspect that if we go with ABCs we will not take their definitions > > lightly and end up haggling over definitions a lot. > > Yeah, but IMO the collections are the most contentious area, and doing > these right is a motivation for the whole idea of ABCs -- it makes > more concrete "what ABCs can do for you." > Gotcha. > > Non-open issue comments: > > > > About Iterator, does it subclass Iterable? It seems to meet the definition. > > It does, and it says so. > Ah, see it now. > > I really don't like the name of Finite. When I read the name before > > knowing what it represented I really had no clue what it represented. > > In hindsight it makes sense, but it not immediately obvious. > > I have been playing with Sizeable and Lengthy, but rejected both as > being too cute. Other suggestions? > Umm, Sized? =) Basically, no, I don't have a better suggestion. > > What about 'append' on MutableSequence? > > Oh, duh. It's listed as add. Will fix. Ah. =) I just figured 'add' was some method to specify where to insert a value or something. -Brett From talin at acm.org Fri Apr 20 06:54:13 2007 From: talin at acm.org (Talin) Date: Thu, 19 Apr 2007 21:54:13 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: Message-ID: <46284775.8020806@acm.org> Brett Cannon wrote: > On 4/19/07, Guido van Rossum wrote: >> On 4/19/07, Brett Cannon wrote: >>> I really don't like the name of Finite. When I read the name before >>> knowing what it represented I really had no clue what it represented. >>> In hindsight it makes sense, but it not immediately obvious. >> I have been playing with Sizeable and Lengthy, but rejected both as >> being too cute. Other suggestions? >> > > Umm, Sized? =) Basically, no, I don't have a better suggestion. Hmmm. "Countable" or "Enumerable" come to mind. "Enumerable" might be confusing, however, since some languages use that word to mean "Iterable". Other words that come to mind are "Dimension" and "Extent", although I'm not sure how to turn them into adjectives. -- Talin From jyasskin at gmail.com Fri Apr 20 09:48:28 2007 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Fri, 20 Apr 2007 00:48:28 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: Message-ID: <5d44f72f0704200048p2d410de9x516e78a888747278@mail.gmail.com> On 4/19/07, Guido van Rossum wrote: > I've started a PEP on Abstract Base Classes (ABCs), PEP 3119: > > http://www.python.org/dev/peps/pep-3119/ > > While I'm not ready yet to answer tough questions about this compared > to alternative proposals, I *am* ready for feedback on the various > open issues sprinkled throughout the current text, especially > concerning decisions regarding exactly which operations to include in > the various ABCs, and also regarding the level of detail required in > the PEP. I have a bunch of comments that aren't all related to the open issues. I'm more familiar with statically typed languages, so sorry if some of these comments don't make sense in a Python context. Hashable might want to specify the exception that should be thrown for mutable objects. I wouldn't have guessed what Finite was from the name, but I don't mind it now that I know. I do prefer 'Sized'. A possible name for the __contains__ ABC for sequences and strings is 'Searchable'. On the other hand, an annotation of some sort for the running time of the method might do as well, but I don't know of any prior art for that. On Sets: Do you have an example of a data structure that is a Set but not a ComposableSet? Set should inherit from a PartiallyOrdered ABC that defines the comparison operators. The final specification of Set should spell out the invariants, but they're clear enough that the PEP probably doesn't need to. It's odd, though, that you mention optimizations to __eq__ and __le__ but not their semantics. ComposableSet's open issues: > Should we define an API for creating new instances (e.g. a class method or a fixed constructor signature)? There are some set types that can't be constructed from arbitrary sequences. For example, Patricia Tries (http://www.haskell.org/ghc/docs/latest/html/libraries/base/Data-IntSet.html) can be used to make very efficient sets of integers, but only integers. > Should we just pick a concrete return type (e.g. set)? For set compositions, you should say whether the arguments need to be uniform. (Is (IntSet(1,2,3) | set('a','b','c')) allowed?) If they do, then you don't need a concrete return type; if they don't, then you do, and the default implementations should use it. Even this may be a problem if someone writes a TreeSet which requires TotallyOrdered-but-not-Hashable elements. In any case, the ABC should encourage implementors to return an instance of the same type as the arguments when the arguments are of the same type. > Should we add the copy method? Not on ComposableSet, but maybe in a Copyable ABC, which it might inherit from. MutableSet: > Should we support more operations implemented by the Python 2 set type? You have groups of equivalent operations (add, update, __ior__), (discard, difference_update, __isub__), (??, intersection_update, __iand__), and (??, symmetric_difference_update, __ixor__) It would be nice to make these all uniform. Intersection is hard to support one element at a time, but symmetric difference would work with a toggle() method. The operations that work a group at a time are likely be more efficient on Sized arguments, and much more efficient on arguments of the same type. How much are you trying to support these kinds of optimizations to implementations? > Should we unify remove and discard, a la Java (which has a single method returning a boolean indicating whether it was removed or not)? Yes. :) Mappings: Set-like unions, intersections, and differences are useful on maps too, but you usually have to provide a function to say what to do with colliding values. This is probably a different PEP though. Sequence: I guess you've already dealt with this, but allowing __getitem__ with integer indices on str objects forces you to use UTF-32 or play interesting games with caching. Most string algorithms are quite happy with (bidirectional? multipass?) iterators, but programmers aren't used to using them. Numbers: Haskell might be good to use for inspiration on the numeric classes: http://www.haskell.org/ghc/docs/latest/html/libraries/base/Prelude.html#7. Note that Haskell's "Real" doesn't imply that the number is fractional, just that it's not complex. Some differences I see from Bill Janssen's proposal: * A Haskell Num is constructible from any Integer (python's long). * A Haskell Fractional (including Complex) is constructible from any Rational (which can be constructed from a float) * The operations are spread into more classes. I think that primarily helps Complex and Fixed-point types fit into the hierarchy better. The Haskell folks actually complain that the operations aren't spread into _enough_ classes. They mostly want more group-theory related classes below Num. * The bitwise operators are segregated into a Bits class, but I don't see any benefit to that. * In a possible use for Cardinal, Integral**Cardinal=>Integral, Fractional**Integral=>Fractional, and Floating**Floating=>Floating. * Complex isn't Orderable Fortress (http://research.sun.com/projects/plrg/fortress.pdf#page=270) has an extremely detailed hierarchy of algebraic traits, and it looks like they plan a similarly detailed hierarchy of numbers, but the numbers aren't yet specified beyond integers and rationals. Cardinal is also useful as the argument to (Sequence * Cardinal). Index (Ix) is used in Haskell primarily in the form of tuples of Ints for multi-dimensional arrays. -- Namast?, Jeffrey Yasskin From jan.grant at bristol.ac.uk Fri Apr 20 11:21:39 2007 From: jan.grant at bristol.ac.uk (Jan Grant) Date: Fri, 20 Apr 2007 10:21:39 +0100 (BST) Subject: [Python-3000] Need help completing ABC pep In-Reply-To: <46284775.8020806@acm.org> References: <46284775.8020806@acm.org> Message-ID: <20070420101924.B1402@tribble.ilrt.bris.ac.uk> On Thu, 19 Apr 2007, Talin wrote: > Brett Cannon wrote: > > On 4/19/07, Guido van Rossum wrote: > >> On 4/19/07, Brett Cannon wrote: > >>> I really don't like the name of Finite. When I read the name before > >>> knowing what it represented I really had no clue what it represented. > >>> In hindsight it makes sense, but it not immediately obvious. > >> I have been playing with Sizeable and Lengthy, but rejected both as > >> being too cute. Other suggestions? > >> > > > > Umm, Sized? =) Basically, no, I don't have a better suggestion. > > Hmmm. > > "Countable" or "Enumerable" come to mind. "Enumerable" might be > confusing, however, since some languages use that word to mean "Iterable". > > Other words that come to mind are "Dimension" and "Extent", although I'm > not sure how to turn them into adjectives. "Bounded?" I was going to suggest that all reifications of "infinite sequences" in a running program are, in practice, finite but unbounded. -- jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/ Tel +44 (0)117 3317661 http://ioctl.org/jan/ Whose kung-fu is the best? From ncoghlan at gmail.com Fri Apr 20 11:49:54 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 20 Apr 2007 19:49:54 +1000 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: Message-ID: <46288CC2.30304@gmail.com> Guido van Rossum wrote: > On 4/19/07, Brett Cannon wrote: >> On 4/19/07, Guido van Rossum wrote: >> * "Also pop, popitem, setdefault?" >> >> Eh, my gut reaction is "no". But you do have 'pop' on MutableSequence. > > But for sequences it trivially maps on core methods. pop() for > mappings is a little more convoluted (how to pick a random key? Just > pick the first one I guess?) pop() on dict requires you to say which key to pop, so a concrete Mapping.pop might look something like: _sentinel=object() def pop(self, k, d=_sentinel): try: v = self[k] except KeyError: if d is _sentinel: raise v = d else: del self[k] return v >> I really don't like the name of Finite. When I read the name before >> knowing what it represented I really had no clue what it represented. >> In hindsight it makes sense, but it not immediately obvious. > > I have been playing with Sizeable and Lengthy, but rejected both as > being too cute. Other suggestions? Sized? Still not wonderful, but probably easier to figure out and remember than Finite. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Fri Apr 20 11:59:45 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 20 Apr 2007 19:59:45 +1000 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: Message-ID: <46288F11.3030505@gmail.com> Guido van Rossum wrote: > I've started a PEP on Abstract Base Classes (ABCs), PEP 3119: > > http://www.python.org/dev/peps/pep-3119/ > > While I'm not ready yet to answer tough questions about this compared > to alternative proposals, I *am* ready for feedback on the various > open issues sprinkled throughout the current text, especially > concerning decisions regarding exactly which operations to include in > the various ABCs, and also regarding the level of detail required in > the PEP. > It's not listed as an open issue, but when looking at the list of ABCs, I noticed a potential solution to a different problem we have occasionally wrestled with: the unfortunate side-effects of having a default __hash__ method defined on object. If the details of collections.Hashable.__hash__ are declared to be implementation dependent rather than having it always return 0, then we can get rid of object.__hash__. For CPython, collections.Hashable.__hash__ would then just invoke id() on the object, and other implementations can also do whatever they currently do for object.__hash__ Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From michele.simionato at gmail.com Fri Apr 20 12:07:55 2007 From: michele.simionato at gmail.com (Michele Simionato) Date: Fri, 20 Apr 2007 10:07:55 +0000 (UTC) Subject: [Python-3000] Fixing super anyone? References: Message-ID: Guido van Rossum python.org> writes: > On 4/18/07, Jim Jewett gmail.com> wrote: > > Does this mean it should find the super of the current class (which > > isn't yet defined when the method is being defined)? > Right. It should definitely use the current class and not the name of > the current class. I hope you'll be able to find a way to implement > this! Perhaps the class could be gotten from a hidden cell reference, > where the cell is filled in by the metaclass whe the class is created. > (The cell can't represent the result of the super call, because that > also incorporates the current value of 'self', for which you can use > the first argument.) > > So the syntax "super" could be translated into "super(__thisclass__, > self)" where __thisclass__ is the cell reference, self refers to the > (current or original?) value of the first argument, and super is > exactly the same as the 2.x super function. (Though it could be > referenced via an opcode.) Here is an implentation based on bytecode hacks that works in current Python: http://groups.google.com/group/comp.lang.python/browse_frm/thread/a6010c7494871bb1/62a2da68961caeb6?lnk=gst&q=simionato+challenge&rnum=1&hl=en#62a2da68961caeb6 Michele Simionato From thomas at python.org Fri Apr 20 13:56:36 2007 From: thomas at python.org (Thomas Wouters) Date: Fri, 20 Apr 2007 13:56:36 +0200 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: Message-ID: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> On 4/19/07, Jim Jewett wrote: > > On 4/18/07, Guido van Rossum wrote: > > Is anyone available to write up a PEP on how to turn super into a > > keyword? Inside regular and class methods, super.foo(args) should be > > equivalent to super(ThisClass, self).foo(args). I think there are ways > > to make the old syntax work too, but maybe that's only necessary for > > 2.6. > > Does this mean it should find the super of the current class (which > isn't yet defined when the method is being defined)? > > That would be a slight change (though probably an improvement) against > today's lookup-by-name. > > If that change is OK, and no one else volunteers, I'll try to take a > go at it this weekend. I've had this near the top of my (Python) TODO list for a while, but I haven't been able to find the time. I've considered it while doing the dishes and such, though. I can think of two ways of doing the underlying magic "properly", and one way that's too ugly to think about: 1) Add a new type of descriptor-hook, for associating an object with the class it is an attribute of. After class creation (in the metaclass __init__), any object with this __associate__ (or whatever) hook gets it called. It's only called for objects that are attributes of *that* class, not of any superclasses (since they will already be associated with the superclass.) I'm sure there are other uses for this hook, just not methods that want to use super() -- like zope interfaces ;) 2) Add a new argument to __get__ (or a new method that acts like __get__ but with the extra argument, and is called instead of __get__ if it is defined) where the extra argument is the class that the retrieved descriptor is actually an attribute of. It currently (correctly) gets the class the attribute was retrieved through, which could be a subclass. In both these cases, the (un)bound method object would end up with knowledge of the class it is defined in, and create a local variable with a super-object for the 'super' keyword to use. There still are some grey areas to solve -- what if classes share functions, what if functions share code-objects, etc. #2 is probably a better way to go about it than #1, in that regard. (and 3) the fugly method: don't do anything special with methods, but have super() search through __mro__ for the first class to have the current function as an attribute, then pick the next class from that. Yecch, and more grey areas than the other two methods.) There's also the question of what the super keyword itself should look like, e.g. # declaration-style, no dot res = super currentmethod(arg, arg) # treat super like self res = super.currentmethod(arg, arg) # treat super like self.currentmethod res = super(arg, arg) I think super.currentmethod(arg, arg) makes the most sense, but that may be because it most closely resembles the current practice. However, it may call the "wrong" supermethod when the class does, for instance, '__repr__ = __str__'. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070420/f3dd83c8/attachment.html From jason.orendorff at gmail.com Fri Apr 20 17:10:54 2007 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Fri, 20 Apr 2007 11:10:54 -0400 Subject: [Python-3000] string module trimming In-Reply-To: References: Message-ID: On 4/19/07, Jim Jewett wrote: > On 4/19/07, Jason Orendorff wrote: > > Collation can be done right: provide a function text.sort_key() > > that converts a str into an opaque thing that has the desired > > ordering. > > If this function is context-free (depending on only the input string), > it will violate the unicode standard (and, apparently, do the wrong > thing for some languages -- usually including French). I meant this to be a function of the string and the locale,[*] implemented as a thin wrapper around wcsxfrm() on posix, LCMapString() on Win32, Collator.getCollationKey() in Java, CompareInfo.GetSortKey() in .NET. Whether these are Unicode-compliant is out of our hands. We're not Sun or IBM. I don't think we're going to implement and maintain this ourselves. So I see two options: (1) swallow a hard dependency on a particular implementation, maybe ICU; (2) use whatever the system happens to provide. Either one is fine with me. > I'm not saying relying strictly on unicode properties *can't* be done > right -- I'm just saying that it would be very difficult and very > inefficient, so it probably won't happen soon -- which is an argument > for keeping the half-measures around meanwhile. This would be true if the only option were to implement it all ourselves. -j [*] I would prefer a context-free function really takes both the string and the locale as arguments... but the posix API doesn't support that. :-P From jyasskin at gmail.com Fri Apr 20 17:19:08 2007 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Fri, 20 Apr 2007 08:19:08 -0700 Subject: [Python-3000] string module trimming In-Reply-To: References: <5d44f72f0704182319j124b4743he67b669630d303ad@mail.gmail.com> <20070418233622.6344.JCARLSON@uci.edu> Message-ID: <5d44f72f0704200819m111c6ff2i94e1fa78d0fc318e@mail.gmail.com> On 4/19/07, Jim Jewett wrote: > > "Jeffrey Yasskin" wrote: > > > ... Are you saying that in your app, just because I've set > > > the en_US locale, I won't be able to type "????"? Or that those > > > characters won't be recognized as letters? > > The latter. Some applications may reject them for that reason; for > example some domain registrars have policies to prevent domain name > spoofing with similar-looking characters. One way to do that is to > say that a character used in a domain name (under that registrar) is > limited to those letters used by the appropriate national language. Filtering to a certain language's alphabet is neither necessary nor sufficient for a domain registrar. They're going to implement their own system paying attention to http://www.unicode.org/reports/tr36/ (see particularly section 2.3 for why even a restriction to one script doesn't keep them safe), so they don't justify keeping an attractive nuisance in the standard library. -- Namast?, Jeffrey Yasskin From guido at python.org Fri Apr 20 19:53:23 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 20 Apr 2007 10:53:23 -0700 Subject: [Python-3000] Fixing super anyone? In-Reply-To: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> Message-ID: On 4/20/07, Thomas Wouters wrote: > On 4/19/07, Jim Jewett wrote: > > On 4/18/07, Guido van Rossum wrote: > > > Is anyone available to write up a PEP on how to turn super into a > > > keyword? Inside regular and class methods, super.foo(args) should be > > > equivalent to super(ThisClass, self).foo(args). I think there are ways > > > to make the old syntax work too, but maybe that's only necessary for > > > 2.6. > > > > Does this mean it should find the super of the current class (which > > isn't yet defined when the method is being defined)? > > > > That would be a slight change (though probably an improvement) against > > today's lookup-by-name. > > > > If that change is OK, and no one else volunteers, I'll try to take a > > go at it this weekend. > > I've had this near the top of my (Python) TODO list for a while, but I > haven't been able to find the time. I've considered it while doing the > dishes and such, though. (Not in the shower? :-) > I can think of two ways of doing the underlying > magic "properly", and one way that's too ugly to think about: > > 1) Add a new type of descriptor-hook, for associating an object with the > class it is an attribute of. After class creation (in the metaclass > __init__), any object with this __associate__ (or whatever) hook gets it > called. It's only called for objects that are attributes of *that* class, > not of any superclasses (since they will already be associated with the > superclass.) I'm sure there are other uses for this hook, just not methods > that want to use super() -- like zope interfaces ;) Nice and elegant. Though what would the hook *do*? I.e. where does it store the class reference for super to find? Perhaps this is where my idea of a designated cell comes in handy (although you'd end up with a cell per method instead of all methods sharing one cell). Also, what arguments are passed to the hook? For this purpose the class object is the only thing needed -- but is that always sufficient? > 2) Add a new argument to __get__ (or a new method that acts like __get__ > but with the extra argument, and is called instead of __get__ if it is > defined) where the extra argument is the class that the retrieved descriptor > is actually an attribute of. It currently (correctly) gets the class the > attribute was retrieved through, which could be a subclass. I'm not so keen on this; calling an existing API with an extra argument is fraught with transitional problems (2to3 notwithstanding). And checking for two different special methods slows things down. I'd propose to pass the class where it's found as the 3rd arg always, since (almost) nobody uses this argument, but I'm not sure of the consequences for class methods. Anyway, this also still requires us to answer how that extra argument gets passed on to the executing code -- but now a cell won't do, as the argument may be different (theoretically anyway) on each call to __get__. So that also makes me wary because this adds overhead per call (technically: per lookup that binds a method) while the thing we need to pass on typically only changes if the class changes -- so doing some magic at class creation time makes more sense to me. (Of course, there's the issue what to do if we dynamically define a new function, and poke it into a class as a method, and that function needs to use super. IMO it's fine to require such a function to use the underlying super() machinery -- this must be an exceedignly rare case!) > In both these cases, the (un)bound method object would end up with knowledge > of the class it is defined in, and create a local variable with a > super-object for the 'super' keyword to use. Ah, so you propose to store it in a local variable. But where is it stored before the function is called? I guess on the "bound method" object. But how is the value passed on as part of the call? I like a cell a lot better because we have an established way to pass cells into calls. > There still are some grey areas > to solve -- what if classes share functions, what if functions share > code-objects, etc. #2 is probably a better way to go about it than #1, in > that regard. Sharing functions is very rare and IMO it's okay if the mechanism doesn't work in that case -- as long as it fails loudly. Sharing code objects shouldn't be a problem -- code objects are immutable anyway, cells are associated with the function object, not with the code object. > (and 3) the fugly method: don't do anything special with methods, but have > super() search through __mro__ for the first class to have the current > function as an attribute, then pick the next class from that. Yecch, and > more grey areas than the other two methods.) This does not get my vote. > There's also the question of what the super keyword itself should look like, > e.g. > > # declaration-style, no dot > res = super currentmethod(arg, arg) > # treat super like self > res = super.currentmethod (arg, arg) > # treat super like self.currentmethod > res = super(arg, arg) > > I think super.currentmethod(arg, arg) makes the most sense, but that may be > because it most closely resembles the current practice. However, it may call > the "wrong" supermethod when the class does, for instance, '__repr__ = > __str__'. Depends on how you define "wrong". :-) I am strongly in favor of super.currentmethod(...) if only because that's closest to how we've always done it, and there may even be a use case for callong some *other* super method. -- --Guido van Rossum (home page: http://www.python.org/~guido/) My signature virus killed your signature virus! From pje at telecommunity.com Fri Apr 20 20:26:53 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 20 Apr 2007 14:26:53 -0400 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> Message-ID: <5.1.1.6.0.20070420141539.02bdf0b0@sparrow.telecommunity.com> At 10:53 AM 4/20/2007 -0700, Guido van Rossum wrote: >Also, what arguments are passed to the hook? For this purpose the >class object is the only thing needed -- but is that always >sufficient? I'd want to have the attribute name as an argument, too, at least for other uses of the hook. PEAK and Chandler both have special metaclasses to do this sort of attribute-class association, and I believe Paste does too (it might be some other project(s) by Ian Bicking, e.g. SQLObject or FormEncode). It comes up pretty much anywhere that people have some sort of object-attribute system. Having a built-in hook for this would be nice. Ideally, the return value of the hook should then be used as the descriptor, rather than the original one; PEAK uses an approach like this to do things like: class Whatever: foo = bar = DelegateTo('otherattribute') where the foo and bar attributes are being set up to delegate to the foo and bar attributes of self.someattribute. (i.e., foo and bar end up with separate descriptors, because the DelegateTo descriptor's association hook is called twice, once with 'foo' and once with 'bar', and the return values become the actual descriptors used in the class.) One question, however, would be whether it should refer to the class *after* decoration, or before. For the super() use case, it would presumably need to be the *decorated* class, which might make things a bit more complicated. I also wonder if this hook mightn't be usable to implement abstractmethod as well? From guido at python.org Fri Apr 20 20:43:20 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 20 Apr 2007 11:43:20 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: Message-ID: On 4/19/07, Brett Cannon wrote: > > Good point. Now that you mention this, and I don't understand why I > > didn't already do it this way in sandbox/abc, the check could be made > > at *class creation time*, and at instantiation time it should just > > have to make one tiny check. > > I thought you were already doing that. =) I am now. > I was just worried that it > would be prohibitively expensive at class creation time for some short > script that used mostly stdlib code that happened to use ABCs. The built-in types can and will use certain shortcuts. :) > > I doubt that 'set' will be inherited from much (except for trivial > > stuff) but returning a 'set' does make some sense. Although for > > trivial subclasses (e.g. adding a different repr or a brand new > > method) you'd like the results also to be an instance of the subclass. > > If we could find a way to do that it would be best. > > Well, self.__class__() would work, right? That should resolve > properly. Or maybe we need that super keyword. =) No, self.__class__() emphatically does *not* work. The PEP specifically claims not to define the constructor signatures -- it's quite possible to implement an object that conforms to the set ABC but whose constructor does not take a sequence of values. Assuming an empty constructor always works is unwarranted too. This is why I don't quite know what to do, except perhaps always returning a (built-in) set instance. > > > * "Should we support all the operations implemented by the Python 2 set type?" > > > > > > No, only core methods should be directly supported. Everything else > > > is just convenience functions. If these can be implemented simply as > > > concrete methods then fine, but otherwise leave them out. > > > > Sure. But one can differ about whether union() is the convenience and > > __or__() the core, or vice versa. :-) > > So I say base it on ease of implementation. If it can be done > trivially in five lines of Python code then include it, otherwise let > the user do it. OK, for now I'm including |= and the other operators and excluding union and the other methods. I'm thinking the argument to |= and friends should be any iterable though. (But not the argument to the non-inline | operator!) Concrete implementations are trivial. > > > * "Also pop, popitem, setdefault?" > > > > > > Eh, my gut reaction is "no". But you do have 'pop' on MutableSequence. > > > > But for sequences it trivially maps on core methods. pop() for > > mappings is a little more convoluted (how to pick a random key? Just > > pick the first one I guess?) > > Right, which is why my gut reaction was "no". I don't know how you > choose. But if you are popping off of a mapping object you should > know the order will be undefined. I would take your latter > suggestion: get a list of keys, take the first one and use that as > what gets popped off. OK, pop and popitem are back in, as concrete methods. > > > * Discussing sequences: "What about += and *=?" > > > > > > No. Only worth it when a special gain is available by the implementation. > > > > OTOH not requiring these gives them fuzzy semantics -- some mutable > > sequences will return self, others a new object. That's not good. I'd > > like to standardize self-modifying +=, *=. > > Then I guess you answered your own question. If you were asking about > semantics I would just follow what lists do. OK, they're in, as concrete operators. > See http://haskell.org/onlinereport/prelude-index.html for the Haskell > 98 report and the prelude (Haskell's version of built-ins). You care > about the type classes (which they just call "classes" on the page). > If you follow the links for any of the type classes they will tell you > exactly what functions must be defined for something to meet that type > class. Some of them are Haskell-specific, but not all. One for my copious spare time... > Umm, Sized? =) Basically, no, I don't have a better suggestion. I like it. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Apr 20 20:52:55 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 20 Apr 2007 11:52:55 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: <46288F11.3030505@gmail.com> References: <46288F11.3030505@gmail.com> Message-ID: On 4/20/07, Nick Coghlan wrote: > It's not listed as an open issue, but when looking at the list of ABCs, > I noticed a potential solution to a different problem we have > occasionally wrestled with: the unfortunate side-effects of having a > default __hash__ method defined on object. > > If the details of collections.Hashable.__hash__ are declared to be > implementation dependent rather than having it always return 0, then we > can get rid of object.__hash__. > > For CPython, collections.Hashable.__hash__ would then just invoke id() > on the object, and other implementations can also do whatever they > currently do for object.__hash__ I don't like that at all. I want to force folks to implement a __hash__ that matches their __eq__. As soon as you implement an __eq__ you need to redefine __hash__. I'd much rather have the default __hash__ be correct but insanely slow than fast but incorrect -- it being incorrect is much harder to debug, and it smells of premature optimization. As Hashable.__hash__ is abstract, you are *forced* to define your own hash. I do see your point though: object.__hash__ makes it possible to use all sorts of objects as dict keys by default, as long as they don't implement __eq__. But perhaps this is not such a good idea and we should just get rid of it and add __hash__ back to specific objects that are useful to use as keys? (E.g. classes but not iterators, to take some extreme examples.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Apr 20 20:56:46 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 20 Apr 2007 11:56:46 -0700 Subject: [Python-3000] Fixing super anyone? In-Reply-To: <5.1.1.6.0.20070420141539.02bdf0b0@sparrow.telecommunity.com> References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> <5.1.1.6.0.20070420141539.02bdf0b0@sparrow.telecommunity.com> Message-ID: If you are serious about this, it'd need a PEP, clarifying the use cases etc. On 4/20/07, Phillip J. Eby wrote: > At 10:53 AM 4/20/2007 -0700, Guido van Rossum wrote: > >Also, what arguments are passed to the hook? For this purpose the > >class object is the only thing needed -- but is that always > >sufficient? > > I'd want to have the attribute name as an argument, too, at least for other > uses of the hook. PEAK and Chandler both have special metaclasses to do > this sort of attribute-class association, and I believe Paste does too (it > might be some other project(s) by Ian Bicking, e.g. SQLObject or > FormEncode). It comes up pretty much anywhere that people have some sort > of object-attribute system. Having a built-in hook for this would be nice. > > Ideally, the return value of the hook should then be used as the > descriptor, rather than the original one; PEAK uses an approach like this > to do things like: > > class Whatever: > foo = bar = DelegateTo('otherattribute') > > where the foo and bar attributes are being set up to delegate to the foo > and bar attributes of self.someattribute. (i.e., foo and bar end up with > separate descriptors, because the DelegateTo descriptor's association hook > is called twice, once with 'foo' and once with 'bar', and the return values > become the actual descriptors used in the class.) > > One question, however, would be whether it should refer to the class > *after* decoration, or before. For the super() use case, it would > presumably need to be the *decorated* class, which might make things a bit > more complicated. > > I also wonder if this hook mightn't be usable to implement abstractmethod > as well? > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From janssen at parc.com Fri Apr 20 21:07:04 2007 From: janssen at parc.com (Bill Janssen) Date: Fri, 20 Apr 2007 12:07:04 PDT Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: Message-ID: <07Apr20.120713pdt."57996"@synergy1.parc.xerox.com> Guido, A good start. A few comments: Where do strings and bytes fit in? They seem to be sequences. Are they sub-typable? If so, shouldn't their base interface be defined? I'd either add numeric types to this, or split it in two pieces: one about ABCs in general (perhaps defining Comparable and Object), and another for collection types. Then a third for numeric types. I don't like the name of the "pop()" method on Sets. It's too much associated with stacks. People reading code that uses "pop" on a value will naturally assume that the value is a stack. I'd prefer "remove_and_return_random_item()". > Python will not require that a class derives from BasicMapping or > Sequence when it defines a __getitem__ method, Good. > nor will the x[y] syntax require that x is an instance of either ABC. Good. > You will still be able to assign any "file-like" object to > sys.stdout, as long as it has a write method. Well, yes, but... I think there should be an ABC for "file-like", which the standard "file" and "StringIO" implementations inherit from. This could even be a fourth PEP. Bill From talin at acm.org Fri Apr 20 21:13:22 2007 From: talin at acm.org (Talin) Date: Fri, 20 Apr 2007 12:13:22 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: <07Apr20.120713pdt."57996"@synergy1.parc.xerox.com> References: <07Apr20.120713pdt."57996"@synergy1.parc.xerox.com> Message-ID: <462910D2.7060301@acm.org> Bill Janssen wrote: >> You will still be able to assign any "file-like" object to >> sys.stdout, as long as it has a write method. > > Well, yes, but... I think there should be an ABC for "file-like", > which the standard "file" and "StringIO" implementations inherit from. > This could even be a fourth PEP. I suggest the ABCs for stream objects should be incorporated into the PEP for the new IO library, using this PEP as a prerequisite. The reason for this is that the stream ABCs will naturally follow the same class decomposition and methods as are defined by the new library (i.e. input stream, output stream, and so on.) -- Talin From brett at python.org Fri Apr 20 21:21:18 2007 From: brett at python.org (Brett Cannon) Date: Fri, 20 Apr 2007 12:21:18 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: Message-ID: On 4/19/07, Guido van Rossum wrote: > I've started a PEP on Abstract Base Classes (ABCs), PEP 3119: > > http://www.python.org/dev/peps/pep-3119/ > > While I'm not ready yet to answer tough questions about this compared > to alternative proposals, I *am* ready for feedback on the various > open issues sprinkled throughout the current text, especially > concerning decisions regarding exactly which operations to include in > the various ABCs, and also regarding the level of detail required in > the PEP. > Just noticed your latest checkin asks whether to unify 'remove' and 'discard' for sets. I say yes. If people care about whether the item pre-exists or not they can check if the item is contained in the set since it implements Container. So either have it throw an exception if the item does not exist or return a boolean based on whether something was remove ala Java. But I don't see a need for both when the check is as simple as:: set.remove(item) if item in set else None or something along those lines. -Brett From talin at acm.org Fri Apr 20 21:33:42 2007 From: talin at acm.org (Talin) Date: Fri, 20 Apr 2007 12:33:42 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: Message-ID: <46291596.1020105@acm.org> Guido van Rossum wrote: > I've started a PEP on Abstract Base Classes (ABCs), PEP 3119: > > http://www.python.org/dev/peps/pep-3119/ > > While I'm not ready yet to answer tough questions about this compared > to alternative proposals, I *am* ready for feedback on the various > open issues sprinkled throughout the current text, especially > concerning decisions regarding exactly which operations to include in > the various ABCs, and also regarding the level of detail required in > the PEP. One thing that I noticed is that you say "ABCs are not intrinsically incompatible with Interfaces", but you don't define what interfaces are. I'm not sure my understanding of the interface proposals is correct, but here is what I would write based on my current understanding: --- "Interfaces" in this context refers to a set of proposals for additional metadata elements attached to a class which are not part of the regular class hierarchy, but do allow for certain types of inheritance testing. Such metadata would be designed, at least in some proposals, so as to be easily mutable by an application, allowing application writers to override the normal classification of an object. The drawback to this idea of attaching mutable metadata to a class is that classes are shared state, and mutating them may lead to conflicts of intent. Additionally, the need to override the classification of an object can be done more cleanly using generic functions: In the simplest case, one can define a "category membership" generic function that simply returns False in the base implementation, and then provide overrides that return True for any classes of interest. -- Talin From thomas at python.org Fri Apr 20 22:06:56 2007 From: thomas at python.org (Thomas Wouters) Date: Fri, 20 Apr 2007 22:06:56 +0200 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> Message-ID: <9e804ac0704201306q38eaf531k9ff0a83b633b3c8b@mail.gmail.com> On 4/20/07, Guido van Rossum wrote: > > On 4/20/07, Thomas Wouters wrote: > > I've had this near the top of my (Python) TODO list for a while, but I > > haven't been able to find the time. I've considered it while doing the > > dishes and such, though. > > (Not in the shower? :-) Yes, there too, when I shower alone; I was going to spare everyone the visual though :) > I can think of two ways of doing the underlying > > magic "properly", and one way that's too ugly to think about: > > > > 1) Add a new type of descriptor-hook, for associating an object with > the > > class it is an attribute of. After class creation (in the metaclass > > __init__), any object with this __associate__ (or whatever) hook gets it > > called. It's only called for objects that are attributes of *that* > class, > > not of any superclasses (since they will already be associated with the > > superclass.) I'm sure there are other uses for this hook, just not > methods > > that want to use super() -- like zope interfaces ;) > > Nice and elegant. Though what would the hook *do*? I.e. where does it > store the class reference for super to find? Perhaps this is where my > idea of a designated cell comes in handy (although you'd end up with a > cell per method instead of all methods sharing one cell). > > Also, what arguments are passed to the hook? For this purpose the > class object is the only thing needed -- but is that always > sufficient? The hook would 'bind' the function to the class -- in some unspecified way. Unspecified because I haven't thought that far ahead yet, and I don't forsee any particular difficulties. Either the function gets wrapped in an 'associated function' type that knows which class it was defined in (so somewhat different from the current 'unbound method' type, but not much) or we just store the associated-class in the existing function object (probably the better option.). In either case we need some new magic to pass this class-object to the actual code object, or give the code object a way to refer to the function object (sometimes also useful for other reasons; it's not an uncommon request, in newbie areas.) As for the arguments to this hook -- I guess passing the attribute name as well as the class object makes sense. It's not much more overhead, and there isn't much more we could sensibly pass that isn't easily available given the class object and attribute name. Phillip mentioned class decorators -- I'd forgotten about those. I don't see much use for them myself, as metaclasses make much more sense to me, in particular because they are naturally inherited, whereas class decorators are (I assume) not inherited. I'm not sure how they should -- or could -- work together with super. It would only be a problem if the class decorator returns a wrapper class, rather than mutate the class. I guess we'd need a way to tell associated-functions "instead of ever visiting this class, visit this wrapper class instead. But don't mess with MRO"... I think. I'm not entirely sure how class decorators would affect MRO, really. > 2) Add a new argument to __get__ (or a new method that acts like __get__ > > but with the extra argument, and is called instead of __get__ if it is > > defined) where the extra argument is the class that the retrieved > descriptor > > is actually an attribute of. It currently (correctly) gets the class the > > attribute was retrieved through, which could be a subclass. > > I'm not so keen on this; calling an existing API with an extra > argument is fraught with transitional problems (2to3 notwithstanding). > And checking for two different special methods slows things down. I'd > propose to pass the class where it's found as the 3rd arg always, > since (almost) nobody uses this argument, but I'm not sure of the > consequences for class methods. The consequence for classmethods would be bad. They would always get the class they were defined on as first argument, making them completely pointless. So, let's not do that :) (Of course, there's the issue what to do if we dynamically define a > new function, and poke it into a class as a method, and that function > needs to use super. IMO it's fine to require such a function to use > the underlying super() machinery -- this must be an exceedignly rare > case!) It's easy to handle such cases if we flag super-syntax-using-functions specially (which is also easy.) type's __setattr__ can complain if such a function gets assigned to an attribute, or it could do the binding magic right then. Functions that don't use the super syntax will continue to work like usual. And super-syntax-using-functions can scream and shout when they get 'associated' with more than one class, of course. > In both these cases, the (un)bound method object would end up with > knowledge > > of the class it is defined in, and create a local variable with a > > super-object for the 'super' keyword to use. > > Ah, so you propose to store it in a local variable. But where is it > stored before the function is called? I guess on the "bound method" > object. But how is the value passed on as part of the call? I like a > cell a lot better because we have an established way to pass cells > into calls. As I mentioned above, I didn't really think about this part. By 'local variable' I just mean that we don't use a global registry or some such ugly hack -- it's a reference, stored wherever, specific to that call. > There still are some grey areas > > to solve -- what if classes share functions, what if functions share > > code-objects, etc. #2 is probably a better way to go about it than #1, > in > > that regard. > > Sharing functions is very rare and IMO it's okay if the mechanism > doesn't work in that case -- as long as it fails loudly. Sharing code > objects shouldn't be a problem -- code objects are immutable anyway, > cells are associated with the function object, not with the code > object. Agreed. > There's also the question of what the super keyword itself should look > like, > > e.g. > > > > # declaration-style, no dot > > res = super currentmethod(arg, arg) > > # treat super like self > > res = super.currentmethod (arg, arg) > > # treat super like self.currentmethod > > res = super(arg, arg) > > > > I think super.currentmethod(arg, arg) makes the most sense, but that may > be > > because it most closely resembles the current practice. However, it may > call > > the "wrong" supermethod when the class does, for instance, '__repr__ = > > __str__'. > > Depends on how you define "wrong". :-) That's why it's in "quotes" :) No matter what we do, it will always be wrong for someone somewhere, I'm sure. I am strongly in favor of super.currentmethod(...) if only because > that's closest to how we've always done it, and there may even be a > use case for callong some *other* super method. I agree. The big downside of thinking of these things off-line (like in the shower) is that it's hard to visualize the actual code. The moment I wrote down the three alternatives I'd imagined, I realized 'super.currentmethod()' is the only sane spelling ;) -- Thomas Wouters Hi! I'm a signature-virus-eating-signature-virus-resistant .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070420/cb041827/attachment-0001.html From pje at telecommunity.com Fri Apr 20 23:15:19 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 20 Apr 2007 17:15:19 -0400 Subject: [Python-3000] Fixing super anyone? In-Reply-To: <9e804ac0704201306q38eaf531k9ff0a83b633b3c8b@mail.gmail.com > References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> Message-ID: <5.1.1.6.0.20070420170331.04af8cf8@sparrow.telecommunity.com> At 10:06 PM 4/20/2007 +0200, Thomas Wouters wrote: >It would only be a problem if the class decorator returns a wrapper class, >rather than mutate the class. I guess we'd need a way to tell >associated-functions "instead of ever visiting this class, visit this >wrapper class instead. But don't mess with MRO"... I think. I'm not >entirely sure how class decorators would affect MRO, really. This depends somewhat, I think, on whether the conceptual translation of super.foo is "super(ClassNameHere, self)" or "super(__thisclass__, self)". If it is ClassNameHere, then the decorated class should be used; if it's __thisclass__, there's a miniscule amount more argument for it being the original class -- but even then, I think practicality probably beats purity. Meanwhile, I still think that cell (closure) variables are the One Obvious Way to implement this. We can implement the super.foo syntax as an AST transform to "super(..., firstarg)" where ... is either __thisclass__ or ClassName, and then say that __thisclass__ or ClassName is then treated as a cell variable defined in the surrounding scope. In either case, we simply have the language definition say that the class name or __thisclass__ (when used in a function inside a class) always refers to the (decorated) class. (Notice that this means there really isn't a need for the descriptor association protocol, at least for Python's own use; I missed that in my previous post, due to my enthusiasm for the independent merits of the __associate__ idea.) From guido at python.org Fri Apr 20 23:33:01 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 20 Apr 2007 14:33:01 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: <6558863970457745851@unknownmsgid> References: <6558863970457745851@unknownmsgid> Message-ID: On 4/20/07, Bill Janssen wrote: > Where do strings and bytes fit in? They seem to be sequences. Are > they sub-typable? If so, shouldn't their base interface be defined? Yes, yes and yes. I haven't gotten to them yet. :-) > I'd either add numeric types to this, or split it in two pieces: one > about ABCs in general (perhaps defining Comparable and Object), and > another for collection types. Then a third for numeric types. Yeah, I'm thinking of at least separating the numeric types. I'd like to keep the collections in the same PEP as the general topics just so that it shows "what can ABCs do for you". > I don't like the name of the "pop()" method on Sets. It's too much > associated with stacks. People reading code that uses "pop" on a value > will naturally assume that the value is a stack. I'd prefer > "remove_and_return_random_item()". Yeah, but it has a long history in Python. Python 2's sets have a pop() method, and dicts have an equivalent popitem() method (as well as a pop() that requires you to specify a key). > > You will still be able to assign any "file-like" object to > > sys.stdout, as long as it has a write method. > > Well, yes, but... I think there should be an ABC for "file-like", > which the standard "file" and "StringIO" implementations inherit from. > This could even be a fourth PEP. PEP 3116 already documents ABCs like RawIOBase, BufferedIOBase, TextIOBase. (It needs to be updated; the implementation adds a fourth, the base of them all, IOBase.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Apr 21 00:24:26 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 20 Apr 2007 15:24:26 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: <5d44f72f0704200048p2d410de9x516e78a888747278@mail.gmail.com> References: <5d44f72f0704200048p2d410de9x516e78a888747278@mail.gmail.com> Message-ID: On 4/20/07, Jeffrey Yasskin wrote: > On 4/19/07, Guido van Rossum wrote: > > I've started a PEP on Abstract Base Classes (ABCs), PEP 3119: > > > > http://www.python.org/dev/peps/pep-3119/ > > > > While I'm not ready yet to answer tough questions about this compared > > to alternative proposals, I *am* ready for feedback on the various > > open issues sprinkled throughout the current text, especially > > concerning decisions regarding exactly which operations to include in > > the various ABCs, and also regarding the level of detail required in > > the PEP. > > I have a bunch of comments that aren't all related to the open issues. > I'm more familiar with statically typed languages, so sorry if some of > these comments don't make sense in a Python context. Thanks for the feedback! You're doing fine. :-) > Hashable might want to specify the exception that should be thrown for > mutable objects. Thanks -- done. > I wouldn't have guessed what Finite was from the name, but I don't > mind it now that I know. I do prefer 'Sized'. Sized it is. > A possible name for the __contains__ ABC for sequences and strings is > 'Searchable'. On the other hand, an annotation of some sort for the > running time of the method might do as well, but I don't know of any > prior art for that. I've added the name suggestion to the open issues; I like it. I'm not sure I want to break new ground specifying running time; it's a tricky business. > On Sets: > Do you have an example of a data structure that is a Set but not a > ComposableSet? Take for example a set that is a contiguous range of integers, represented by a (lo, hi) tuple. In many cases the return type of union and symmetric difference cannot be represented as an object of the same type. I don't want to force the implementer of such a set to provide implementations for operations they don't need. A different POV could be to move the concrete composition operations back into Set and let the default implementations return built-in ``set`` instances. I'm not sure how useful that is. > Set should inherit from a PartiallyOrdered ABC that defines the > comparison operators. I guess then there would also be a TotallyOrdered ABC deriving from PartiallyOrdered which adds no operations but does add a constraint. I've added both, and made Set derive from PartiallyOrdered. > The final specification of Set should spell out the invariants, but > they're clear enough that the PEP probably doesn't need to. It's odd, > though, that you mention optimizations to __eq__ and __le__ but not > their semantics. The semantics are kind of obvious. :-) Text spelling out the invariants in more detail that I can just paste into the PEP is most welcome! > ComposableSet's open issues: > > Should we define an API for creating new instances (e.g. a class method or a fixed constructor signature)? > > There are some set types that can't be constructed from arbitrary > sequences. For example, Patricia Tries > (http://www.haskell.org/ghc/docs/latest/html/libraries/base/Data-IntSet.html) > can be used to make very efficient sets of integers, but only > integers. Right. That's why I'm separating ComposableSet from Set. > > Should we just pick a concrete return type (e.g. set)? > > For set compositions, you should say whether the arguments need to be > uniform. (Is (IntSet(1,2,3) | set('a','b','c')) allowed?) If they do, > then you don't need a concrete return type; if they don't, then you > do, and the default implementations should use it. You may be misunderstanding what I meant. I certainly don't want to constrain the potential return type of the composition operations (except that they should themselves be composable sets). The question about picking a specific return type was meant as a way out to providing a concrete default implementation. I'm still on the fence of this, as it could be a big waste of time (I imagine that composing Patricia trees can be done more efficiently than iterating over the elements). > Even this may be a > problem if someone writes a TreeSet which requires > TotallyOrdered-but-not-Hashable elements. In any case, the ABC should > encourage implementors to return an instance of the same type as the > arguments when the arguments are of the same type. Another good argument against having a default concrete implementation that always returns a built-in set. > > Should we add the copy method? > Not on ComposableSet, but maybe in a Copyable ABC, which it might inherit from. I'd rather leave copying APIs out of this completely. There's a separate copying protocol in Python already, you can define __copy__() for a shallow copy and __deepcopy__() for a deep copy (copies the elements recursively). This is also tied into pickling (serialization). > MutableSet: > > Should we support more operations implemented by the Python 2 set type? > > You have groups of equivalent operations (add, update, __ior__), > (discard, difference_update, __isub__), (??, intersection_update, > __iand__), and (??, symmetric_difference_update, __ixor__) > > It would be nice to make these all uniform. Intersection is hard to > support one element at a time, but symmetric difference would work > with a toggle() method. Cool, I've added toggle(). FWIW, I've remoevd the named methods (update etc.) in favor of stretching the accepted args for the operators (__ior__ etc.). > The operations that work a group at a time are > likely be more efficient on Sized arguments, and much more efficient > on arguments of the same type. How much are you trying to support > these kinds of optimizations to implementations? Haven't thought much about it. Implementations can just override the concrete methods if they have a faster way, and fall back on the concrete base class methods if their faster way doesn't apply for a particular argument. > > Should we unify remove and discard, a la Java (which has a single method returning a boolean indicating whether it was removed or not)? > > Yes. :) Done. > Mappings: > Set-like unions, intersections, and differences are useful on maps > too, but you usually have to provide a function to say what to do with > colliding values. This is probably a different PEP though. Right. My dream is to unify the set and dict types, solving the problem of how to specify an empty set ({} would do it), but this was rejected by Raymond Hettinger for valid pragmatic reasons. > Sequence: > I guess you've already dealt with this, but allowing __getitem__ with > integer indices on str objects forces you to use UTF-32 or play > interesting games with caching. Most string algorithms are quite happy > with (bidirectional? multipass?) iterators, but programmers aren't > used to using them. Yeah, we've had our fill of discussions about alternative implementations. I want the *semantics* to be those of UTF-32 or UTF-16, and the initial implementation will use that, since these semantics are most familiar to Python users. At some point we may have competing implementations. > Numbers: Would you mind helping to write a PEP for numbers? I think I'm running out of round tuits just for the collections and the ABC infrastructure. > Haskell might be good to use for inspiration on the numeric classes: > http://www.haskell.org/ghc/docs/latest/html/libraries/base/Prelude.html#7. > Note that Haskell's "Real" doesn't imply that the number is > fractional, just that it's not complex. Some differences I see from > Bill Janssen's proposal: > * A Haskell Num is constructible from any Integer (python's long). > * A Haskell Fractional (including Complex) is constructible from any > Rational (which can be constructed from a float) > * The operations are spread into more classes. I think that primarily > helps Complex and Fixed-point types fit into the hierarchy better. The > Haskell folks actually complain that the operations aren't spread into > _enough_ classes. They mostly want more group-theory related classes > below Num. > * The bitwise operators are segregated into a Bits class, but I don't > see any benefit to that. Right. Python has a long tradition of using ints (and longs) for that. > * In a possible use for Cardinal, Integral**Cardinal=>Integral, > Fractional**Integral=>Fractional, and Floating**Floating=>Floating. That can be (and is currently) done dynamically (see below). > * Complex isn't Orderable > > Fortress (http://research.sun.com/projects/plrg/fortress.pdf#page=270) > has an extremely detailed hierarchy of algebraic traits, and it looks > like they plan a similarly detailed hierarchy of numbers, but the > numbers aren't yet specified beyond integers and rationals. > > Cardinal is also useful as the argument to (Sequence * Cardinal). I think there's little value for this in a dynamically typed language; the implementation can raise ValueError or treat negative numbers as zero (which is what Python currently does). > Index (Ix) is used in Haskell primarily in the form of tuples of Ints > for multi-dimensional arrays. Right, that's where we use it too (especially the NumPy community). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Apr 21 00:40:18 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 20 Apr 2007 15:40:18 -0700 Subject: [Python-3000] Fixing super anyone? In-Reply-To: <9e804ac0704201306q38eaf531k9ff0a83b633b3c8b@mail.gmail.com> References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> <9e804ac0704201306q38eaf531k9ff0a83b633b3c8b@mail.gmail.com> Message-ID: No time to respond in kind but feeling general agreement. I hope you & Jim Jewett (who first volunteered) can work together on the details for a PEP? On 4/20/07, Thomas Wouters wrote: > > > On 4/20/07, Guido van Rossum wrote: > > On 4/20/07, Thomas Wouters wrote: > > > I've had this near the top of my (Python) TODO list for a while, but I > > > haven't been able to find the time. I've considered it while doing the > > > dishes and such, though. > > > > (Not in the shower? :-) > > Yes, there too, when I shower alone; I was going to spare everyone the > visual though :) > > > > I can think of two ways of doing the underlying > > > magic "properly", and one way that's too ugly to think about: > > > > > > 1) Add a new type of descriptor-hook, for associating an object with > the > > > class it is an attribute of. After class creation (in the metaclass > > > __init__), any object with this __associate__ (or whatever) hook gets it > > > called. It's only called for objects that are attributes of *that* > class, > > > not of any superclasses (since they will already be associated with the > > > superclass.) I'm sure there are other uses for this hook, just not > methods > > > that want to use super() -- like zope interfaces ;) > > > > Nice and elegant. Though what would the hook *do*? I.e. where does it > > store the class reference for super to find? Perhaps this is where my > > idea of a designated cell comes in handy (although you'd end up with a > > cell per method instead of all methods sharing one cell). > > > > Also, what arguments are passed to the hook? For this purpose the > > class object is the only thing needed -- but is that always > > sufficient? > > > The hook would 'bind' the function to the class -- in some unspecified way. > Unspecified because I haven't thought that far ahead yet, and I don't forsee > any particular difficulties. Either the function gets wrapped in an > 'associated function' type that knows which class it was defined in (so > somewhat different from the current 'unbound method' type, but not much) or > we just store the associated-class in the existing function object (probably > the better option.). In either case we need some new magic to pass this > class-object to the actual code object, or give the code object a way to > refer to the function object (sometimes also useful for other reasons; it's > not an uncommon request, in newbie areas.) > > As for the arguments to this hook -- I guess passing the attribute name as > well as the class object makes sense. It's not much more overhead, and there > isn't much more we could sensibly pass that isn't easily available given the > class object and attribute name. > > Phillip mentioned class decorators -- I'd forgotten about those. I don't see > much use for them myself, as metaclasses make much more sense to me, in > particular because they are naturally inherited, whereas class decorators > are (I assume) not inherited. I'm not sure how they should -- or could -- > work together with super. It would only be a problem if the class decorator > returns a wrapper class, rather than mutate the class. I guess we'd need a > way to tell associated-functions "instead of ever visiting this class, visit > this wrapper class instead. But don't mess with MRO"... I think. I'm not > entirely sure how class decorators would affect MRO, really. > > > > 2) Add a new argument to __get__ (or a new method that acts like > __get__ > > > but with the extra argument, and is called instead of __get__ if it is > > > defined) where the extra argument is the class that the retrieved > descriptor > > > is actually an attribute of. It currently (correctly) gets the class the > > > attribute was retrieved through, which could be a subclass. > > > > I'm not so keen on this; calling an existing API with an extra > > argument is fraught with transitional problems (2to3 notwithstanding). > > And checking for two different special methods slows things down. I'd > > propose to pass the class where it's found as the 3rd arg always, > > since (almost) nobody uses this argument, but I'm not sure of the > > consequences for class methods. > > The consequence for classmethods would be bad. They would always get the > class they were defined on as first argument, making them completely > pointless. So, let's not do that :) > > > (Of course, there's the issue what to do if we dynamically define a > > new function, and poke it into a class as a method, and that function > > needs to use super. IMO it's fine to require such a function to use > > the underlying super() machinery -- this must be an exceedignly rare > > case!) > > It's easy to handle such cases if we flag super-syntax-using-functions > specially (which is also easy.) type's __setattr__ can complain if such a > function gets assigned to an attribute, or it could do the binding magic > right then. Functions that don't use the super syntax will continue to work > like usual. And super-syntax-using-functions can scream and shout when they > get 'associated' with more than one class, of course. > > > > In both these cases, the (un)bound method object would end up with > knowledge > > > of the class it is defined in, and create a local variable with a > > > super-object for the 'super' keyword to use. > > > > Ah, so you propose to store it in a local variable. But where is it > > stored before the function is called? I guess on the "bound method" > > object. But how is the value passed on as part of the call? I like a > > cell a lot better because we have an established way to pass cells > > into calls. > > As I mentioned above, I didn't really think about this part. By 'local > variable' I just mean that we don't use a global registry or some such ugly > hack -- it's a reference, stored wherever, specific to that call. > > > > There still are some grey areas > > > to solve -- what if classes share functions, what if functions share > > > code-objects, etc. #2 is probably a better way to go about it than #1, > in > > > that regard. > > > > Sharing functions is very rare and IMO it's okay if the mechanism > > doesn't work in that case -- as long as it fails loudly. Sharing code > > objects shouldn't be a problem -- code objects are immutable anyway, > > cells are associated with the function object, not with the code > > object. > > Agreed. > > > > There's also the question of what the super keyword itself should look > like, > > > e.g. > > > > > > # declaration-style, no dot > > > res = super currentmethod(arg, arg) > > > # treat super like self > > > res = super.currentmethod (arg, arg) > > > # treat super like self.currentmethod > > > res = super(arg, arg) > > > > > > I think super.currentmethod(arg, arg) makes the most sense, but that may > be > > > because it most closely resembles the current practice. However, it may > call > > > the "wrong" supermethod when the class does, for instance, '__repr__ = > > > __str__'. > > > > Depends on how you define "wrong". :-) > > That's why it's in "quotes" :) No matter what we do, it will always be wrong > for someone somewhere, I'm sure. > > > I am strongly in favor of super.currentmethod(...) if only because > > that's closest to how we've always done it, and there may even be a > > use case for callong some *other* super method. > > I agree. The big downside of thinking of these things off-line (like in the > shower) is that it's hard to visualize the actual code. The moment I wrote > down the three alternatives I'd imagined, I realized ' > super.currentmethod()' is the only sane spelling ;) > > -- > Thomas Wouters > > Hi! I'm a signature-virus-eating-signature-virus-resistant > .signature virus! copy me into your .signature file to help me spread! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jyasskin at gmail.com Sat Apr 21 09:30:16 2007 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sat, 21 Apr 2007 00:30:16 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: <5d44f72f0704200048p2d410de9x516e78a888747278@mail.gmail.com> Message-ID: <5d44f72f0704210030r4fc572f0ta07b6de6d4ed08cd@mail.gmail.com> On 4/20/07, Guido van Rossum wrote: > On 4/20/07, Jeffrey Yasskin wrote: > > On Sets: > > Do you have an example of a data structure that is a Set but not a > > ComposableSet? > > Take for example a set that is a contiguous range of integers, > represented by a (lo, hi) tuple. In many cases the return type of > union and symmetric difference cannot be represented as an object of > the same type. I don't want to force the implementer of such a set to > provide implementations for operations they don't need. Ah, good example. I think that PEPs that propose ABCs should be encouraged to describe a few possible implementations, and should be particularly encouraged to give examples of the implementations that made them split two classes, or omit a method from a particular class. > > Set should inherit from a PartiallyOrdered ABC that defines the > > comparison operators. > > I guess then there would also be a TotallyOrdered ABC deriving from > PartiallyOrdered which adds no operations but does add a constraint. > I've added both, and made Set derive from PartiallyOrdered. Cool. Here's some language for their invariants (checking myself with the Fortress spec, and stealing its properties nearly verbatim): PartiallyOrdered: This ABC defines the 5 inequality operations <, <=, >=, >, and cmp(). (Note that == and != are defined by object.) Classes deriving from this ABC should ensure that '<' and '>' form strict partial orders, and '<=' and '>=' form weak partial orders, in the sense of http://en.wikipedia.org/wiki/Partially_ordered_set. These should be compatible with each other and with ==: Where PO is a subtype of PartiallyOrdered, and a and b are instances of PO, cmp(a, b) < 0 ? cmp(b, a) > 0 cmp(a, b) == 0 ? cmp(b, a) == 0 cmp(a, b) == None ? cmp(b, a) == None a < b ? cmp(a, b) < 0 a <= b ? cmp(a, b) <= 0 a == b ? cmp(a, b) == 0 a >=b ? cmp(a, b) >= 0 a > b ? cmp(a, b) > 0 Subclasses of PartiallyOrdered must define one of cmp() or <=. The other operators are defaulted as follows: def <=(a,b): return cmp(a,b) <= 0 def cmp(a,b): if a<=b: if b<=a: return 0 else: return -1 else: if b<=a: return 1 else: return None def ==(a,b): return a<=b and b<=a def <(a,b): return a<=b and not b<=a def >=(a,b): return b<=a def >(a,b): return b, and >= are total orders in the sense of http://en.wikipedia.org/wiki/Totally_ordered_set, and that cmp(a,b) never returns None. Given this additional restriction, subclasses only need to define one of <, <=, or cmp(), and the rest of the methods can be defaulted. Open Issue: How does the TotallyOrdered property inherit? If Real (an abstract class) inherits from TotallyOrdered, do different subclasses need to be totally comparable with each other? If long (a concrete class) inherits from TotallyOrdered, and I subclass long, do I have to ensure that my instances are totally comparable with other longs? I think 'yes' to the second, to obey the Liskov substitution principle. I don't know about the first: for Real it seems fine, but it might inhibit refactoring the ABC hierarchy. > > The final specification of Set should spell out the invariants, but > > they're clear enough that the PEP probably doesn't need to. It's odd, > > though, that you mention optimizations to __eq__ and __le__ but not > > their semantics. > > The semantics are kind of obvious. :-) > > Text spelling out the invariants in more detail that I can just paste > into the PEP is most welcome! Bah! ;) Set: Sets should implement an efficient (no worse than O(log N)) __contains__ operation. Sets are defined by their extensions. Subclasses may override the following default definitions of __eq__ and __le__ with more efficient versions, but they must respect the same semantics: def __eq__(a, b): return len(a) == len(b) and all(x in a for x in b) def __le__(a, b): return len(a) <= len(b) and all(x in a for x in b) ComposableSet: ... mathematical meaning of set composition: Where a and b are instances of ComposableSet and __op__(a,b) is defined: x in (a | b) ? x in a or x in b x in (a & b) ? x in a and x in b x in (a ^ b) ? x in a != x in b x in (a - b) ? x in a and not x in b > > ComposableSet's open issues: > > > Should we just pick a concrete return type (e.g. set)? > > > > For set compositions, you should say whether the arguments need to be > > uniform. (Is (IntSet(1,2,3) | set('a','b','c')) allowed?) If they do, > > then you don't need a concrete return type; if they don't, then you > > do, and the default implementations should use it. > > You may be misunderstanding what I meant. I certainly don't want to > constrain the potential return type of the composition operations > (except that they should themselves be composable sets). The question > about picking a specific return type was meant as a way out to > providing a concrete default implementation. I'm still on the fence of > this, as it could be a big waste of time (I imagine that composing > Patricia trees can be done more efficiently than iterating over the > elements). Borrowing some syntax from Java generics, which of: ComposableSet __or__(ComposableSet a, ComposableSet b); ComposableSet __or__(CS a, CS b); CS __or__(CS a, CS b); do you mean to define? I think they have different implications about what defaults you need to provide. The second is probably out because it would mean (a|b|c) would likely break. This gets even more complex if we consider the types of the contents of the sets... Here's a possible overloaded signature for __or__. (Do you already have a better notation for this?) ComposableSet __or__(ComposableSet a, ComposableSet b); extends ComposableSet)> CS __or__(CS a, CS b); The first would probably be optimized when its arguments are of the same type, but could fall back on the abstract methods from ComposableSet, which would return a built-in (frozen?)set. > > Numbers: > > Would you mind helping to write a PEP for numbers? I think I'm running > out of round tuits just for the collections and the ABC > infrastructure. Bah! again. Yes, I'll try to throw something together with some help from Neal. Who else has been thinking about the Numbers PEP? > > Cardinal is also useful as the argument to (Sequence * Cardinal). > > I think there's little value for this in a dynamically typed language; > the implementation can raise ValueError or treat negative numbers as > zero (which is what Python currently does). True, and any hypothetical type checkers that want to use such information can define their own subclasses of Integer. -- Namast?, Jeffrey Yasskin From ncoghlan at gmail.com Sat Apr 21 11:17:15 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 21 Apr 2007 19:17:15 +1000 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: <46288F11.3030505@gmail.com> Message-ID: <4629D69B.9010508@gmail.com> Guido van Rossum wrote: > I do see your point though: object.__hash__ makes it possible to use > all sorts of objects as dict keys by default, as long as they don't > implement __eq__. But perhaps this is not such a good idea and we > should just get rid of it and add __hash__ back to specific objects > that are useful to use as keys? (E.g. classes but not iterators, to > take some extreme examples.) We've tried to do this before - the point where we got stuck was the fact that Jython needed a separate location to store the default hash method (unlike CPython, Jython's id() and object.__hash__() do different things), and we couldn't come up with a reasonable location for it. I agree that moving the default method from object to Hashable doesn't eliminate the errors that arise from redefining __eq__ without redefining __hash__, but at least it restricts them to those classes which have Hashable somewhere in their MRO. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From michele.cella at gmail.com Sat Apr 21 16:07:13 2007 From: michele.cella at gmail.com (Michele Cella) Date: Sat, 21 Apr 2007 16:07:13 +0200 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> Message-ID: Guido van Rossum wrote: > >> There's also the question of what the super keyword itself should look like, >> e.g. >> >> # declaration-style, no dot >> res = super currentmethod(arg, arg) >> # treat super like self >> res = super.currentmethod (arg, arg) >> # treat super like self.currentmethod >> res = super(arg, arg) >> >> I think super.currentmethod(arg, arg) makes the most sense, but that may be >> because it most closely resembles the current practice. However, it may call >> the "wrong" supermethod when the class does, for instance, '__repr__ = >> __str__'. > > Depends on how you define "wrong". :-) > > I am strongly in favor of super.currentmethod(...) if only because > that's closest to how we've always done it, and there may even be a > use case for callong some *other* super method. > If super is going to be a keyword (+1) shouldn't it behave like any other keyword [1]? I mean, it's there any keyword (ATM) that uses an attribute access syntax? Personally, I really like this syntax: res = super currentmethod(arg, arg) Why? because it's very consistent (pythonic?) with the way I'm used to use any other keyword. Anyway, thanks for fixing super in python 3000, it's probably the only python thing I really hate. PS Excuse me if I sound like a noob, that's the first time I post to a python dev list. [1] http://docs.python.org/ref/keywords.html From guido at python.org Sat Apr 21 16:37:37 2007 From: guido at python.org (Guido van Rossum) Date: Sat, 21 Apr 2007 07:37:37 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: <4629D69B.9010508@gmail.com> References: <46288F11.3030505@gmail.com> <4629D69B.9010508@gmail.com> Message-ID: On 4/21/07, Nick Coghlan wrote: > Guido van Rossum wrote: > > I do see your point though: object.__hash__ makes it possible to use > > all sorts of objects as dict keys by default, as long as they don't > > implement __eq__. But perhaps this is not such a good idea and we > > should just get rid of it and add __hash__ back to specific objects > > that are useful to use as keys? (E.g. classes but not iterators, to > > take some extreme examples.) > > We've tried to do this before - the point where we got stuck was the > fact that Jython needed a separate location to store the default hash > method (unlike CPython, Jython's id() and object.__hash__() do different > things), and we couldn't come up with a reasonable location for it. I don't understand. Why does the default hash method need to be stored in a location? Why couldn't it be a method on PyObj or whatever their base object it? Or are you talking about a place to store the hash *value*? But why would that need to be stored? > I agree that moving the default method from object to Hashable doesn't > eliminate the errors that arise from redefining __eq__ without > redefining __hash__, but at least it restricts them to those classes > which have Hashable somewhere in their MRO. Actually it would significantly reduce the errors because Hashable.__hash__ is abstract, *forcing* you to redefine it. My concern is more with the amount of broken code if we remove object.__hash__. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From michele.cella at gmail.com Sat Apr 21 16:56:26 2007 From: michele.cella at gmail.com (Michele Cella) Date: Sat, 21 Apr 2007 16:56:26 +0200 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> Message-ID: Michele Cella wrote: > > Personally, I really like this syntax: > > res = super currentmethod(arg, arg) > Sorry for replying to myself, what about: def mymethod(self, arg): super self.mymethod(arg) this syntax resembles very closely the way you actually invoke a *self* method, the only difference is that the super keyword clearly denotes that you want to call the super implementation of that method (very much like the global keyword). So you get something like: class Example(): def example(self): # do it def mymethod(self, arg): super self.mymethod(arg) self.example() that's all very consistent and explicit IMHO. Ciao Michele From pje at telecommunity.com Sat Apr 21 17:01:55 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 21 Apr 2007 11:01:55 -0400 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: <46291596.1020105@acm.org> References: Message-ID: <5.1.1.6.0.20070421105729.04c77840@sparrow.telecommunity.com> At 12:33 PM 4/20/2007 -0700, Talin wrote: >Additionally, the need to override the classification of an >object can be done more cleanly using generic functions: In the simplest >case, one can define a "category membership" generic function that >simply returns False in the base implementation, and then provide >overrides that return True for any classes of interest. Yes, but that's sort of like saying it's better to point a pistol at your foot than a shotgun. :) Explicitly testing category membership is a bug, not a feature; you're far better off defining an overload to do whatever it was you were going to make the if-then test do. From guido at python.org Sat Apr 21 17:01:41 2007 From: guido at python.org (Guido van Rossum) Date: Sat, 21 Apr 2007 08:01:41 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: <5d44f72f0704210030r4fc572f0ta07b6de6d4ed08cd@mail.gmail.com> References: <5d44f72f0704200048p2d410de9x516e78a888747278@mail.gmail.com> <5d44f72f0704210030r4fc572f0ta07b6de6d4ed08cd@mail.gmail.com> Message-ID: On 4/21/07, Jeffrey Yasskin wrote: > On 4/20/07, Guido van Rossum wrote: > > On 4/20/07, Jeffrey Yasskin wrote: > > > On Sets: > > > Do you have an example of a data structure that is a Set but not a > > > ComposableSet? > > > > Take for example a set that is a contiguous range of integers, > > represented by a (lo, hi) tuple. In many cases the return type of > > union and symmetric difference cannot be represented as an object of > > the same type. I don't want to force the implementer of such a set to > > provide implementations for operations they don't need. > > Ah, good example. I think that PEPs that propose ABCs should be > encouraged to describe a few possible implementations, and should be > particularly encouraged to give examples of the implementations that > made them split two classes, or omit a method from a particular class. Yeah, the PEP is short on examples. > > > Set should inherit from a PartiallyOrdered ABC that defines the > > > comparison operators. > > > > I guess then there would also be a TotallyOrdered ABC deriving from > > PartiallyOrdered which adds no operations but does add a constraint. > > I've added both, and made Set derive from PartiallyOrdered. > > Cool. Here's some language for their invariants (checking myself with > the Fortress spec, and stealing its properties nearly verbatim): > > PartiallyOrdered: > This ABC defines the 5 inequality operations <, <=, >=, >, and cmp(). Actually, I'm hoping to eliminate cmp() completely. Also, even if it doesn't get eliminated, the existence of cmp() suggests a total ordering. > (Note that == and != are defined by object.) Classes deriving from > this ABC should ensure that '<' and '>' form strict partial orders, > and '<=' and '>=' form weak partial orders, in the sense of > http://en.wikipedia.org/wiki/Partially_ordered_set. These should be > compatible with each other and with ==: > > Where PO is a subtype of PartiallyOrdered, and a and b are instances of PO, > cmp(a, b) < 0 ? cmp(b, a) > 0 > cmp(a, b) == 0 ? cmp(b, a) == 0 > cmp(a, b) == None ? cmp(b, a) == None > a < b ? cmp(a, b) < 0 > a <= b ? cmp(a, b) <= 0 > a == b ? cmp(a, b) == 0 > a >=b ? cmp(a, b) >= 0 > a > b ? cmp(a, b) > 0 > Subclasses of PartiallyOrdered must define one of cmp() or <=. Actually, Python typically uses < as the single comparison operator that must be defined (e.g. list.sort() uses < exclusively). > The other operators are defaulted as follows: > def <=(a,b): return cmp(a,b) <= 0 > def cmp(a,b): > if a<=b: > if b<=a: return 0 > else: return -1 > else: > if b<=a: return 1 > else: return None > def ==(a,b): return a<=b and b<=a > def <(a,b): return a<=b and not b<=a > def >=(a,b): return b<=a > def >(a,b): return b > Open Issues: 1) Is it reasonable to let cmp() return None for > unordered pairs, or should we instead define a 4-value enumeration for > it to return? As I said, I'd rather kill it or at least ignore it for this part. > 2) The @abstractmethod decorator isn't enough to specify > that subclasses must override at least one of a group of methods, and > there are classes with even more complex requirements like "subclasses > must override a or (b and (c or d))". I guess those requirements will have to be expressed in English. I don't see the point to invent a decorator mini-language or other mechanism that's powerful enough to express such possibilities. If someone *really* wants they can implement a new metaclass that checks these, but IMO it's a rather silly waste of time since the metaclass can't check that any of the methods are implemented correcttly, so the exercise is kind of pointless (super-strict in one area while 100% lenient in another). > 3) http://docs.python.org/ref/customization.html#l2h-187 still implies > that < defines a total order over all python objects. Has that changed > for py3k? 4) How should we handle the reflected forms of these > operators? Yes, in Py3k < <= > >= are not defined by default. We may not have updated the docs though. > TotallyOrdered: > This ABC derives from PartiallyOrdered. It adds no new operations but > implies a promise that <, <=, >, and >= are total orders in the sense > of http://en.wikipedia.org/wiki/Totally_ordered_set, and that cmp(a,b) > never returns None. Given this additional restriction, subclasses only > need to define one of <, <=, or cmp(), and the rest of the methods can > be defaulted. > > Open Issue: How does the TotallyOrdered property inherit? If Real (an > abstract class) inherits from TotallyOrdered, do different subclasses > need to be totally comparable with each other? If long (a concrete > class) inherits from TotallyOrdered, and I subclass long, do I have to > ensure that my instances are totally comparable with other longs? I > think 'yes' to the second, to obey the Liskov substitution principle. > I don't know about the first: for Real it seems fine, but it might > inhibit refactoring the ABC hierarchy. Perhaps this is more appropriate for the Numeric ABCs PEP. IMO the only concern is the realities of floating point arithmetic (whether binary or decimal); I would be fine with some language that explains that numeric types computed/represented with limited precision may violate the rules. The only alternative I see is to force float and decimal (and hence Real, Complex and Number) to inherit from PartiallyOrdered instead of TotallyOrdered, which sounds like a shame -- or is it? > > > The final specification of Set should spell out the invariants, but > > > they're clear enough that the PEP probably doesn't need to. It's odd, > > > though, that you mention optimizations to __eq__ and __le__ but not > > > their semantics. > > > > The semantics are kind of obvious. :-) > > > > Text spelling out the invariants in more detail that I can just paste > > into the PEP is most welcome! > > Bah! ;) > > Set: > Sets should implement an efficient (no worse than O(log N)) > __contains__ operation. Sets are defined by their extensions. What's an extension? > Subclasses may override the following default definitions of __eq__ > and __le__ with more efficient versions, but they must respect the > same semantics: > def __eq__(a, b): return len(a) == len(b) and all(x in a for x in b) > def __le__(a, b): return len(a) <= len(b) and all(x in a for x in b) > > ComposableSet: > ... mathematical meaning of set composition: > Where a and b are instances of ComposableSet and __op__(a,b) is defined: > x in (a | b) ? x in a or x in b > x in (a & b) ? x in a and x in b > x in (a ^ b) ? x in a != x in b > x in (a - b) ? x in a and not x in b > > > > ComposableSet's open issues: > > > > Should we just pick a concrete return type (e.g. set)? > > > > > > For set compositions, you should say whether the arguments need to be > > > uniform. (Is (IntSet(1,2,3) | set('a','b','c')) allowed?) If they do, > > > then you don't need a concrete return type; if they don't, then you > > > do, and the default implementations should use it. > > > > You may be misunderstanding what I meant. I certainly don't want to > > constrain the potential return type of the composition operations > > (except that they should themselves be composable sets). The question > > about picking a specific return type was meant as a way out to > > providing a concrete default implementation. I'm still on the fence of > > this, as it could be a big waste of time (I imagine that composing > > Patricia trees can be done more efficiently than iterating over the > > elements). > > Borrowing some syntax from Java generics, which of: > ComposableSet __or__(ComposableSet a, ComposableSet b); > ComposableSet __or__(CS a, CS b); > CS __or__(CS a, CS b); > do you mean to define? Can't say I understand that. The minimum requirement is that the return type is a subclass of ComposableSet. *Some* subclasses CS may themselves guarantee that the result is always a subclass of CS, but I don't want to force that. Returning a 'set' in the default implementation satisfies the former but not the latter. This is in itself fine (CS can override __or__ to return a CS). But it could be expensive -- even if the author of CS doesn't care about the return type being a CS itself, they might still care about th efficiency. > I think they have different implications about > what defaults you need to provide. The second is probably out because > it would mean (a|b|c) would likely break. This gets even more complex > if we consider the types of the contents of the sets... Here's a > possible overloaded signature for __or__. (Do you already have a > better notation for this?) I think this is more appropriate for other languages. > ComposableSet > __or__(ComposableSet a, ComposableSet b); > extends > ComposableSet)> CS __or__(CS a, CS b); It's Saturday AM, I'm not even going to try to parse that. :-) > The first would probably be optimized when its arguments are of the > same type, but could fall back on the abstract methods from > ComposableSet, which would return a built-in (frozen?)set. > > > > Numbers: > > > > Would you mind helping to write a PEP for numbers? I think I'm running > > out of round tuits just for the collections and the ABC > > infrastructure. > > Bah! again. Yes, I'll try to throw something together with some help > from Neal. Who else has been thinking about the Numbers PEP? > > > > Cardinal is also useful as the argument to (Sequence * Cardinal). > > > > I think there's little value for this in a dynamically typed language; > > the implementation can raise ValueError or treat negative numbers as > > zero (which is what Python currently does). > > True, and any hypothetical type checkers that want to use such > information can define their own subclasses of Integer. > > -- > Namast?, > Jeffrey Yasskin > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Apr 21 17:06:37 2007 From: guido at python.org (Guido van Rossum) Date: Sat, 21 Apr 2007 08:06:37 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: <5.1.1.6.0.20070421105729.04c77840@sparrow.telecommunity.com> References: <46291596.1020105@acm.org> <5.1.1.6.0.20070421105729.04c77840@sparrow.telecommunity.com> Message-ID: On 4/21/07, Phillip J. Eby wrote: > At 12:33 PM 4/20/2007 -0700, Talin wrote: > >Additionally, the need to override the classification of an > >object can be done more cleanly using generic functions: In the simplest > >case, one can define a "category membership" generic function that > >simply returns False in the base implementation, and then provide > >overrides that return True for any classes of interest. > > Yes, but that's sort of like saying it's better to point a pistol at your > foot than a shotgun. :) Explicitly testing category membership is a bug, > not a feature; you're far better off defining an overload to do whatever it > was you were going to make the if-then test do. I don't believe that's always true. I can imagine use cases where, if I were writing it in an if-then-else fashion, I'd be writing the *same* if-test several times as part of a particular algorithm, without having an easy way to refactor the algorithm so that there's only one place testing it. Turning each place where the if-test is repeated into a different GF might be a wasted of time; turning just the test into a GF seems just right. (IN THAT PARTICULAR CASE.) Note that I don't object against the idea that usually a GF is better; I just don't think it always is so. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Sat Apr 21 17:21:02 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 22 Apr 2007 01:21:02 +1000 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: <46288F11.3030505@gmail.com> <4629D69B.9010508@gmail.com> Message-ID: <462A2BDE.5030101@gmail.com> Guido van Rossum wrote: > On 4/21/07, Nick Coghlan wrote: >> We've tried to do this before - the point where we got stuck was the >> fact that Jython needed a separate location to store the default hash >> method (unlike CPython, Jython's id() and object.__hash__() do different >> things), and we couldn't come up with a reasonable location for it. > > I don't understand. Why does the default hash method need to be stored > in a location? Why couldn't it be a method on PyObj or whatever their > base object it? Or are you talking about a place to store the hash > *value*? But why would that need to be stored? I believe the point was that currently object.__hash__ combines with object.__eq__ to give identity based comparison by default. If object.__hash__ is removed, then an appropriate hash function for identity-based comparison needs to be available somewhere else in the standard library - it was pointed out (by Samuele?) that id() is too expensive in Jython to be used directly as such a hash function. I think the thread then devolved into a bikeshed discussion about possible different names (object.default_hash, sys.id_hash, etc, etc, ...), which is why nothing was ever actually done about it. Cheers, Nick. P.S. This discussion happened quite some time ago, and I haven't trawled through the list archive to check I'm remembering the details correctly. But I'm pretty sure I have the gist correct. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From pje at telecommunity.com Sat Apr 21 18:43:17 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 21 Apr 2007 12:43:17 -0400 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: <5.1.1.6.0.20070421105729.04c77840@sparrow.telecommunity.com> <46291596.1020105@acm.org> <5.1.1.6.0.20070421105729.04c77840@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070421122647.02b67e70@sparrow.telecommunity.com> At 08:06 AM 4/21/2007 -0700, Guido van Rossum wrote: >On 4/21/07, Phillip J. Eby wrote: > > At 12:33 PM 4/20/2007 -0700, Talin wrote: > > >Additionally, the need to override the classification of an > > >object can be done more cleanly using generic functions: In the simplest > > >case, one can define a "category membership" generic function that > > >simply returns False in the base implementation, and then provide > > >overrides that return True for any classes of interest. > > > > Yes, but that's sort of like saying it's better to point a pistol at your > > foot than a shotgun. :) Explicitly testing category membership is a bug, > > not a feature; you're far better off defining an overload to do whatever it > > was you were going to make the if-then test do. > >I don't believe that's always true. I can imagine use cases where, if >I were writing it in an if-then-else fashion, I'd be writing the >*same* if-test several times as part of a particular algorithm, >without having an easy way to refactor the algorithm so that there's >only one place testing it. Turning each place where the if-test is >repeated into a different GF might be a wasted of time; turning just >the test into a GF seems just right. (IN THAT PARTICULAR CASE.) I understand the motivation; it's just been my personal experience that when I encounter such a case, there are always other parts of the library that will need those "different" GF's, because they end up being expressive and relevant to the problem domain of the library as a whole. Those "different" GF's then become a clean specification of the interface(s) required by the library. For example, pydoc frequently asks whether something 'isroutine()' or 'ismethod()', but if it were factored to use elementary GF's instead, we would have things like 'getsignature()' and 'getmembers()' instead, and it would then be straightforward to document how to make a new kind of object work with it. (i.e. if you have members, implement a getmembers() overload, if you're callable, implement a getsignature() overload, etc.) So, if you're writing any sort of library, you will usually get a solid payoff from doing factoring these primitives, given the high probability that your own library will use them more than once, and the ease of documenting the idea of having a call signature vs. trying to explain (or making your users figure out) the nine jillion consequences of having 'isroutine()' return True. I suppose you could say it's a tradeoff of "wasting" your time, vs. wasting your users' time. >Note that I don't object against the idea that usually a GF is better; >I just don't think it always is so. I can certainly conceive that there might be an exception, I just have yet to encounter it. But then, my focus is nearly always on library development, where my "customer" is the library's user. Within that niche, I think there's a much bigger case to be made that it is at least *nearly* always so. From talin at acm.org Sat Apr 21 22:51:57 2007 From: talin at acm.org (Talin) Date: Sat, 21 Apr 2007 13:51:57 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: Message-ID: <462A796D.50700@acm.org> John Reese wrote: > "This PEP proposes a particular strategy for organizing these tests > known as Abstract Base Classes, or ABC. ABCs are simply Python classes > that are added into an object's inheritance tree to signal certain > features of that object to an external inspector. Tests are done using > isinstance(), and the presence of a particular ABC means that the test > has passed." > > I'd like to see another few lines in the rationale section explaining > the intent of the concrete methods. For example, inheriting from > abc.Mapping would have a similar effect to inheriting from > UserDict.DictMixin -- by overriding it's few abstract methods you get > close to the full complement of dict's convenience methods. But from > the open issues and other discussion in this thread, I don't think the > intention is to provide full coverage of the convenience methods in > all of the builtin classes in their corresponding ABCs, but merely to > expand the number of methods you can assume given an ABC. > > I'd like an explanation of exactly where the line is being drawn in > terms of what convenience methods ABCs will provide through concrete > methods. OK how about this. Replace the paragraph immediately following the one you quote with these two paragraphs: In addition, the ABCs define a minimal set of methods that establish the characteristic behavior of the type. Code that discriminates objects based on their ABC type can trust that those methods will always be present. Each of these methods are accompanied by an generalized abstract semantic definition that is described in the documentation for the ABC. These standard semantic definitions are not enforced, but are strongly recommended. Like all other things in Python, these promises are in the nature of a gentlemen's agreement, which in this case means that while the language does enforce some of the promises made in the ABC, it is up to the implementer of the concrete class to insure that the remaining ones are kept. -- Talin From jimjjewett at gmail.com Sun Apr 22 06:20:43 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 22 Apr 2007 00:20:43 -0400 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: Message-ID: On 4/19/07, Guido van Rossum wrote: > On 4/19/07, Brett Cannon wrote: > > On 4/19/07, Guido van Rossum wrote: > > > I've started a PEP on Abstract Base Classes (ABCs), PEP 3119: > > * "Should we also implement the issubset and issuperset methods found > > on the set type in Python 2? As these are just aliases for __le__ and > > __ge__, I'm tempted to leave these out." > > Leave them out. Not terribly needed plus it is better to start out > > small. They can easily be added later if called for. I think the names are more sensible than repurposing the numeric operators. The "concrete" implementation can just forward to other name, so it doesn't cost much. > I doubt that 'set' will be inherited from much (except for trivial > stuff) but returning a 'set' does make some sense. Although for > trivial subclasses (e.g. adding a different repr or a brand new > method) you'd like the results also to be an instance of the subclass. > If we could find a way to do that it would be best. You could use a variant of the numeric operator resolution. The left class wins, unless the right class is a subclass of the left. If this class does not have a constructor that accepts an iterable, (maybe fall back to the other class, then) fall back to a builtin set. Going back to the PEP itself, in the ABC support framework, I would change: "We define the following four new built-in objects that help defining ABCs:" -> "We define the following four new objects (in module abc) that help defining ABCs:" and in the @abstractmethod description, "Such a method may be called" -> "These abstract methods may still be called" In the Hashable section: > Another constraint is that hashable objects, once created, > should never change their value (as compared by ==) or their hash > value. If a class cannot guarantee this, it should not derive from > Hashable; if it cannot guarantee this for certain instances only, > __hash__ for those instances should raise a TypeError exception. Why not just return -1 (unless/until the value is stable)? Is the -1 special case being phased out too? ... I would say that Searchable might be worth separating into a Container subclass, but don't worry about the speed differences. Many containers are small enough that constant effects are more important than big-O. When big-O does matter, then the user may well want to distinguish between the O(1) and O(log N) case. I think this gets detailed enough that it isn't worth putting in the common library. Why must a HashableSet or MutableSet be composable? If this is because you figure any useful set has those properties (and I don't think so, when doing uniquification), then Set and ComposableSet should become BasicSet and Set? ... Note that in the current sandbox sandbox (which doesn't have *Ordered), the name BasicSet is used for a Set which isn't Sized. (Except that Set doesn't actually inherit from BasicSet, as I assume it should do.) Continuing with the current sandbox Why must @abstractmethod come last, when mixed with other decorators? IterableMapping needs to override __iter__; as written, it returns the empty iterator. Should Sequence.index take optional start and stop arguments? def index(self, value, start=0, stop=None): if stop is None: stop = len(self) for i, elem in enumerate(self): if i < start: continue if stop < i: break if elem == value: return i raise ValueError -jJ From jtr at miskatonic.nu Sat Apr 21 22:21:26 2007 From: jtr at miskatonic.nu (John Reese) Date: Sat, 21 Apr 2007 13:21:26 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: Message-ID: On 4/19/07, Guido van Rossum wrote: > I've started a PEP on Abstract Base Classes (ABCs), PEP 3119: > > http://www.python.org/dev/peps/pep-3119/ > > While I'm not ready yet to answer tough questions about this compared > to alternative proposals, I *am* ready for feedback on the various > open issues sprinkled throughout the current text, especially > concerning decisions regarding exactly which operations to include in > the various ABCs, and also regarding the level of detail required in > the PEP. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/john.reese%40gmail.com > "This PEP proposes a particular strategy for organizing these tests known as Abstract Base Classes, or ABC. ABCs are simply Python classes that are added into an object's inheritance tree to signal certain features of that object to an external inspector. Tests are done using isinstance(), and the presence of a particular ABC means that the test has passed." I'd like to see another few lines in the rationale section explaining the intent of the concrete methods. For example, inheriting from abc.Mapping would have a similar effect to inheriting from UserDict.DictMixin -- by overriding it's few abstract methods you get close to the full complement of dict's convenience methods. But from the open issues and other discussion in this thread, I don't think the intention is to provide full coverage of the convenience methods in all of the builtin classes in their corresponding ABCs, but merely to expand the number of methods you can assume given an ABC. I'd like an explanation of exactly where the line is being drawn in terms of what convenience methods ABCs will provide through concrete methods. From jyasskin at gmail.com Sun Apr 22 10:12:33 2007 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sun, 22 Apr 2007 01:12:33 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: <5d44f72f0704200048p2d410de9x516e78a888747278@mail.gmail.com> <5d44f72f0704210030r4fc572f0ta07b6de6d4ed08cd@mail.gmail.com> Message-ID: <5d44f72f0704220112j5cc5bc53u82171e86779c6368@mail.gmail.com> On 4/21/07, Guido van Rossum wrote: > On 4/21/07, Jeffrey Yasskin wrote: > > PartiallyOrdered: > > This ABC defines the 5 inequality operations <, <=, >=, >, and cmp(). > > Actually, I'm hoping to eliminate cmp() completely. Also, even if it > doesn't get eliminated, the existence of cmp() suggests a total > ordering. Ok. It's more efficient when comparing large structures, but it's not absolutely needed. In some languages it can be more convenient too, but that's because you take advantage of their case statements. > > Subclasses of PartiallyOrdered must define one of cmp() or <=. > > Actually, Python typically uses < as the single comparison operator > that must be defined (e.g. list.sort() uses < exclusively). With only a partial order, you need both < and == to define the other operators, or you can do it with just <=. A total order allows you to do it with either < or <= alone. sort(), at least in C++, needs at least a "strict weak ordering" (http://www.sgi.com/tech/stl/StrictWeakOrdering.html), which lets you define equivalence but not equality using just <. > > > > The final specification of Set should spell out the invariants, but > > > > they're clear enough that the PEP probably doesn't need to. It's odd, > > > > though, that you mention optimizations to __eq__ and __le__ but not > > > > their semantics. > > > > > > The semantics are kind of obvious. :-) > > > > > > Text spelling out the invariants in more detail that I can just paste > > > into the PEP is most welcome! > > > > Bah! ;) > > > > Set: > > Sets should implement an efficient (no worse than O(log N)) > > __contains__ operation. Sets are defined by their extensions. > > What's an extension? Oops. It's the collection of objects 'x' for which 'x in set' is true. We can probably just skip that sentence. > > Subclasses may override the following default definitions of __eq__ > > and __le__ with more efficient versions, but they must respect the > > same semantics: > > def __eq__(a, b): return len(a) == len(b) and all(x in a for x in b) > > def __le__(a, b): return len(a) <= len(b) and all(x in a for x in b) > > > > ComposableSet: > > ... mathematical meaning of set composition: > > Where a and b are instances of ComposableSet and __op__(a,b) is defined: > > x in (a | b) ? x in a or x in b > > x in (a & b) ? x in a and x in b > > x in (a ^ b) ? x in a != x in b > > x in (a - b) ? x in a and not x in b > > > > > > ComposableSet's open issues: > > > > > Should we just pick a concrete return type (e.g. set)? > > > > > > > > For set compositions, you should say whether the arguments need to be > > > > uniform. (Is (IntSet(1,2,3) | set('a','b','c')) allowed?) If they do, > > > > then you don't need a concrete return type; if they don't, then you > > > > do, and the default implementations should use it. > > > > > > You may be misunderstanding what I meant. I certainly don't want to > > > constrain the potential return type of the composition operations > > > (except that they should themselves be composable sets). The question > > > about picking a specific return type was meant as a way out to > > > providing a concrete default implementation. I'm still on the fence of > > > this, as it could be a big waste of time (I imagine that composing > > > Patricia trees can be done more efficiently than iterating over the > > > elements). > > > > Borrowing some syntax from Java generics, which of: > > ComposableSet __or__(ComposableSet a, ComposableSet b); > > ComposableSet __or__(CS a, CS b); > > CS __or__(CS a, CS b); > > do you mean to define? > > Can't say I understand that. The minimum requirement is that the > return type is a subclass of ComposableSet. *Some* subclasses CS may > themselves guarantee that the result is always a subclass of CS, but I > don't want to force that. Returning a 'set' in the default > implementation satisfies the former but not the latter. This is in > itself fine (CS can override __or__ to return a CS). But it could be > expensive -- even if the author of CS doesn't care about the return > type being a CS itself, they might still care about th efficiency. I think you want my first option then. How about: "The default implementation ensures that __or__, __and__, __sub__, and __xor__ do the right thing for any ComposableSet with Hashable elements, but because it iterates over both arguments and returns a built-in set, it may be slow. Subclasses should optimize these operations for types they know about but continue to support heterogenous operations by delegating to ComposableSet. "This should be rare: If you define a ComposableSet that can hold non-Hashable elements, your users should expect that they can compose sets with identical types, but not necessarily sets with different types, unless, of course, you give them extra guarantees." > > I think they have different implications about > > what defaults you need to provide. The second is probably out because > > it would mean (a|b|c) would likely break. This gets even more complex > > if we consider the types of the contents of the sets... Here's a > > possible overloaded signature for __or__. (Do you already have a > > better notation for this?) > > I think this is more appropriate for other languages. Having the compiler check the signature is definitely more appropriate for other languages, or at least a third-party module. Knowing what types we handle as arguments and what that type the result can be is important for any language. My notation is, of course, probably not the best we could come up with. :) -- Namast?, Jeffrey Yasskin From pedronis at openendsystems.com Sun Apr 22 12:35:47 2007 From: pedronis at openendsystems.com (Samuele Pedroni) Date: Sun, 22 Apr 2007 12:35:47 +0200 Subject: [Python-3000] question Re: Need help completing ABC pep In-Reply-To: References: <46291596.1020105@acm.org> <5.1.1.6.0.20070421105729.04c77840@sparrow.telecommunity.com> Message-ID: <462B3A83.1060204@openendsystems.com> Guido van Rossum wrote: > > > I don't believe that's always true. I can imagine use cases where, if > I were writing it in an if-then-else fashion, I'd be writing the > *same* if-test several times as part of a particular algorithm, > without having an easy way to refactor the algorithm so that there's > only one place testing it. Turning each place where the if-test is > repeated into a different GF might be a wasted of time; turning just > the test into a GF seems just right. (IN THAT PARTICULAR CASE.) > > Note that I don't object against the idea that usually a GF is better; > I just don't think it always is so. > > If we keep s % d (do we? reading the list archives and the peps didn't give an answer, I suppose it depends if PEP 3101 is accepted): will it check if d is a Mapping or continue to use the current heuristics? I suppose this is a legitimate question for other corners of the language, std lib. From greg.ewing at canterbury.ac.nz Mon Apr 23 00:07:38 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 23 Apr 2007 10:07:38 +1200 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> Message-ID: <462BDCAA.7020501@canterbury.ac.nz> Michele Cella wrote: > Personally, I really like this syntax: > > res = super currentmethod(arg, arg) > > Why? because it's very consistent (pythonic?) with the way I'm used to > use any other keyword. But it's inconsistent with the way every other method is called. -- Greg From aahz at pythoncraft.com Mon Apr 23 00:47:08 2007 From: aahz at pythoncraft.com (Aahz) Date: Sun, 22 Apr 2007 15:47:08 -0700 Subject: [Python-3000] Fixing super anyone? In-Reply-To: <462BDCAA.7020501@canterbury.ac.nz> References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> <462BDCAA.7020501@canterbury.ac.nz> Message-ID: <20070422224708.GA7208@panix.com> On Mon, Apr 23, 2007, Greg Ewing wrote: > Michele Cella wrote: >> >> Personally, I really like this syntax: >> >> res = super currentmethod(arg, arg) >> >> Why? because it's very consistent (pythonic?) with the way I'm used to >> use any other keyword. > > But it's inconsistent with the way every other method is called. But we also don't have any keywords with methods "attached". I don't strongly favor Michele's approach, but it doesn't seem to me that there are any strong arguments in favor of super.method(). -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "...string iteration isn't about treating strings as sequences of strings, it's about treating strings as sequences of characters. The fact that characters are also strings is the reason we have problems, but characters are strings for other good reasons." --Aahz From brett at python.org Mon Apr 23 03:45:18 2007 From: brett at python.org (Brett Cannon) Date: Sun, 22 Apr 2007 18:45:18 -0700 Subject: [Python-3000] PEP to change how the main module is delineated Message-ID: This PEP is to change the ``if __name__ == "__main__": ...`` idiom to ``if __name__ == sys.main: ...`` so that you at least have a chance to execute module in a package that use relative imports. Ran this PEP past python-ideas. Stopped the discussion there when too many new ideas were being proposed. =) I have listed all of them in the Rejected Ideas section, although if overwhelming support for one comes forward the PEP can shift to one of them. --------------------------------------------- PEP: XXX Title: Delineation of the main module Version: $Revision: 52916 $ Last-Modified: $Date: 2006-12-04 11:59:42 -0800 (Mon, 04 Dec 2006) $ Author: Brett Cannon Status: Draft Type: Standards Track Content-Type: text/x-rst Created: XXX-Apr-2007 Abstract ======== Because of how name resolution works for relative imports in a world where PEP 328 is implemented, the ability to execute modules within a package ceases being possible. This failing stems from the fact that the module being executed as the "main" module replaces its ``__name__`` attribute with ``"__main__"`` instead of leaving it as the absolute name of the module. This breaks import's ability to resolve relative imports from the main module into absolute names. In order to resolve this issue, this PEP proposes to change how the main module is delineated. By leaving the ``__name__`` attribute in a module alone and setting ``sys.main`` to the name of the main module this will allow at least some instances of executing a module within a package that uses relative imports. This PEP does not address the idea of introducing a module-level function that is automatically executed like PEP 299 proposes. The Problem =========== With the introduction of PEP 328, relative imports became dependent on the ``__name__`` attribute of the module performing the import. This is because the use of dots in a relative import are used to strip away parts of the calling module's name to calculate where in the package hierarchy an import should fall (prior to PEP 328 relative imports could fail and would fall back on absolute imports which had a chance of succeeding). For instance, consider the import ``from .. import spam`` made from the ``bacon.ham.beans`` module (``bacon.ham.beans`` is not a package itself, i.e., does not define ``__path__``). Name resolution of the relative import takes the caller's name (``bacon.ham.beans``), splits on dots, and then slices off the last n parts based on the level (which is 2). In this example both ``ham`` and ``beans`` are dropped and ``spam`` is joined with what is left (``bacon``). This leads to the proper import of the module ``bacon.spam``. This reliance on the ``__name__`` attribute of a module when handling relative imports becomes an issue when executing a script within a package. Because the executing script has its name set to ``'__main__'``, import cannot resolve any relative imports, leading to an ``ImportError``. For example, assume we have a package named ``bacon`` with an ``__init__.py`` file containing:: from . import spam Also create a module named ``spam`` within the ``bacon`` package (it can be an empty file). Now if you try to execute the ``bacon`` package (either through ``python bacon/__init__.py`` or ``python -m bacon``) you will get an ``ImportError`` about trying to do a relative import from within a non-package. Obviously the import is valid, but because of the setting of ``__name__`` to ``'__main__'`` import thinks that ``bacon/__init__.py`` is not in a package since no dots exist in ``__name__``. To see how the algorithm works in more detail, see ``importlib.Import._resolve_name()`` in the sandbox [#importlib]_. Currently a work-around is to remove all relative imports in the module being executed and make them absolute. This is unfortunate, though, as one should not be required to use a specific type of resource in order to make a module in a package be able to be executed. The Solution ============ The solution to the problem is to not change the value of ``__name__`` in modules. But there still needs to be a way to let executing code know it is being executed as a script. This is handled with a new attribute in the ``sys`` module named ``main``. When a module is being executed as a script, ``sys.main`` will be set to the name of the module. This changes the current idiom of:: if __name__ == '__main__': ... to:: import sys if __name__ == sys.main: ... The newly proposed solution does introduce an added line of boilerplate which is a module import. But as the solution does not introduce a new built-in or module attribute (as discussed in `Rejected Ideas`_) it has been deemed worth the extra line. Another issue with the proposed solution (which also applies to all rejected ideas as well) is that it does not directly solve the problem of discovering the name of a file. Consider ``python bacon/spam.py``. By the file name alone it is not obvious whether ``bacon`` is a package. In order to properly find this out both the current direction must exist on ``sys.path`` as well as ``bacon/__init__.py`` existing. But this is the simple example. Consider ``python ../spam.py``. From the file name alone it is not at all clear if ``spam.py`` is in a package or not. One possible solution is to find out what the absolute name of ``..``, check if a file named ``__init__.py`` exists, and then look if the directory is on ``sys.path``. If it is not, then continue to walk up the directory until no more ``__init__.py`` files are found or the directory is found on ``sys.path``. This could potentially be an expensive process. If the package depth happens to be deep then it could require a large amount of disk access to discover where the package is anchored on ``sys.path``, if at all. The stat calls alone can be expensive if the file system the executed script is on is something like NFS. Because of these issues, only when the ``-m`` command-line argument (introduced by PEP 338) is used will ``__name__`` be set. Otherwise the fallback semantics of setting ``__name__`` to ``"__main__"`` will occur. ``sys.main`` will still be set to the proper value, regardless of what ``__name__`` is set to. Implementation ============== When the ``-m`` option is used, ``sys.main`` will be set to the argument passed in. ``sys.argv`` will be adjusted as it is currently. Then the equivalent of ``__import__(self.main)`` will occur. This differs from current semantics as the ``runpy`` module fetches the code object for the file specified by the module name in order to explicitly set ``__name__`` and other attributes. This is no longer needed as import can perform its normal operation in this situation. If a file name is specified, then ``sys.main`` will be set to ``"__main__"``. The specified file will then be read and have a code object created and then be executed with ``__name__`` set to ``"__main__"``. This mirrors current semantics. Transition Plan =============== In order for Python 2.6 to be able to support both the current semantics and the proposed semantics, ``sys.main`` will always be set to ``"__main__"``. Otherwise no change will occur for Python 2.6. This unfortunately means that no benefit from this change will occur in Python 2.6, but it maximizes compatibility for code that is to work as much as possible with 2.6 and 3.0. To help transition to the new idiom, 2to3 [#2to3]_ will gain a rule to transform the current ``if __name__ == '__main__': ...`` idiom to the new one. This will not help with code that checks ``__name__`` outside of the idiom, though. Rejected Ideas ============== ``__main__`` built-in --------------------- A counter-proposal to introduce a built-in named ``__main__``. The value of the built-in would be the name of the module being executed (just like the proposed ``sys.main``). This would lead to a new idiom of:: if __name__ == __main__: ... A drawback is that the syntactic difference is subtle; the dropping of quotes around "__main__". Some believe that for existing Python programmers bugs will be introduced where the quotation marks will be put on by accident. But one could argue that the bug would be discovered quickly through testing as it is a very shallow bug. While the name of built-in could obviously be different (e.g., ``main``) the other drawback is that it introduces a new built-in. With a simple solution such as ``sys.main`` being possible without adding another built-in to Python, this proposal was rejected. ``__main__`` module attribute ----------------------------- Another proposal was to add a ``__main__`` attribute to every module. For the one that was executing as the main module, the attribute would have a true value while all other modules had a false value. This has a nice consequence of simplify the main module idiom to:: if __main__: ... The drawback was the introduction of a new module attribute. It also required more integration with the import machinery than the proposed solution. Use ``__file__`` instead of ``__name__`` ---------------------------------------- Any of the proposals could be changed to use the ``__file__`` attribute on modules instead of ``__name__``, including the current semantics. The problem with this is that with the proposed solutions there is the issue of modules having no ``__file__`` attribute defined or having the same value as other modules. The problem that comes up with the current semantics is you still have to try to resolve the file path to a module name for the import to work. Special string subclass for ``__name__`` that overrides ``__eq__`` ------------------------------------------------------------------ One proposal was to define a subclass of ``str`` that overrode the ``__eq__`` method so that it would compare equal to ``"__main__"`` as well as the actual name of the module. In all other respects the subclass would be the same as ``str``. This was rejected as it seemed like too much of a hack. References ========== .. [#2to3] 2to3 tool (http://svn.python.org/view/sandbox/trunk/2to3/) [ViewVC] .. [#importlib] importlib (http://svn.python.org/view/sandbox/trunk/import_in_py/importlib.py?view=markup) [ViewVC] Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From lists at cheimes.de Mon Apr 23 04:20:08 2007 From: lists at cheimes.de (Christian Heimes) Date: Mon, 23 Apr 2007 04:20:08 +0200 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: Message-ID: Brett Cannon wrote: > Implementation > ============== > > When the ``-m`` option is used, ``sys.main`` will be set to the > argument passed in. ``sys.argv`` will be adjusted as it is currently. > Then the equivalent of ``__import__(self.main)`` will occur. This > differs from current semantics as the ``runpy`` module fetches the > code object for the file specified by the module name in order to > explicitly set ``__name__`` and other attributes. This is no longer > needed as import can perform its normal operation in this situation. __import__(self.main)? I guess you wanted to write __import__(sys.main) :) > The newly proposed solution does introduce an added line of > boilerplate which is a module import. But as the solution does not > introduce a new built-in or module attribute (as discussed in > `Rejected Ideas`_) it has been deemed worth the extra line. Someone on python-idea brought up the idea to add the sys module to builtins. The feature would remove the requirement for an extra line and the costs should be minimal or even zero since sys is a special builtin module. Some people including me like to resurrect the PEP 299 __main__() function idea. http://www.python.org/dev/peps/pep-0299/ Christian From jimjjewett at gmail.com Mon Apr 23 05:05:14 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 22 Apr 2007 23:05:14 -0400 Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function Currently Being Defined (this) Message-ID: (Please note that several groups were Cc'd. For now, please limit followups to python-3000. This would *probably* be backported to 2.6, but that wouldn't be decided until the implementation strategy was settled.) PEP: 30XX Title: Access to Module/Class/Function Currently Being Defined (this) Version: $Revision$ Last-Modified: $Date$ Author: Jim J. Jewett Status: Draft Type: Standards Track Content-Type: text/plain Created: 22-Apr-2007 Python-Version: 3.0 Post-History: 22-Apr-2007 Abstract It is common to need a reference to the current module, class, or function, but there is currently no entirely correct way to do this. This PEP proposes adding the keywords __module__, __class__, and __function__. Rationale Many modules export various functions, classes, and other objects, but will perform additional activities (such as running unit tests) when run as a script. The current idiom is to test whether the module's name has been set to magic value. if __name__ == "__main__": ... More complicated introspection requires a module to (attempt to) import itself. If importing the expected name actually produces a different module, there is no good workaround. Proposal: Add a __module__ keyword which refers to the module currently being defined (executed). (But see open issues.) if __module__ is sys.main: ... # assuming PEP 3020, Cannon Class methods are passed the current instance; from this they can determine self.__class__ (or cls, for classmethods). Unfortunately, this reference is to the object's actual class, which may be a subclass of the defining class. The current workaround is to repeat the name of the class, and assume that the name will not be rebound. class C(B): def meth(self): super(C, self).meth() # Hope C is never rebound. class D(C): def meth(self): super(C, self).meth() # ?!? issubclass(D,C), so it "works" Proposal: Add a __class__ keyword which refers to the class currently being defined (executed). (But see open issues.) class C(B): def meth(self): super(__class__, self).meth() Note that super calls may be further simplified by PEP 30XX, Jewett. The __class__ (or __this_class__) attribute came up in attempts to simplify the explanation and/or implementation of that PEP, but was separated out as an independent decision. Note that __class__ (or __this_class__) is not quite the same as the __thisclass__ property on bound super objects. The existing super.__thisclass__ property refers to the class from which the Method Resolution Order search begins. In the above class D, it would refer to (the current reference of name) C. Functions (including methods) often want access to themselves, usually for a private storage location. While there are several workarounds, all have their drawbacks. def counter(_total=[0]): # _total shouldn't really appear in the _total[0] += 1 # signature at all; the list wrapping and return _total[0] # [0] unwrapping obscure the code @annotate(total=0) def counter(): counter.total += 1 # Assume name counter is never rebound return counter.total class _wrap(object): # class exists only to provide storage __total=0 def f(self): self.__total += 1 return self.__total accum=_wrap().f # set module attribute to a bound method Proposal: Add a __function__ keyword which refers to the function (or method) currently being defined (executed). (But see open issues.) @annotate(total=0) def counter(): __function__.total += 1 # Always refers to this function obj return __function__.total Backwards Compatibility While a user could be using these names already, __anything__ names are explicitly reserved to the interpreter. It is therefore acceptable to introduce special meaning to these names within a single feature release. Implementation Ideally, these names would be keywords treated specially by the bytecode compiler. Guido has suggested [1] using a cell variable filled in by the metaclass. Michele Simionato has provided a prototype using bytecode hacks [2]. Open Issues - Are __module__, __class__, and __function__ the right names? In particular, should the names include the word "this", either as __this_module__, __this_class__, and __this_function__, (format discussed on the python-3000 and python-ideas lists) or as __thismodule__, __thisclass__, and __thisfunction__ (inspired by, but conflicting with, current usage of super.__thisclass__). - Are all three keywords needed, or should this enhancement be limited to a subset of the objects? Should methods be treated separately from other functions? References [1] Fixing super anyone? Guido van Rossum http://mail.python.org/pipermail/python-3000/2007-April/006671.html [2] Descriptor/Decorator challenge, Michele Simionato http://groups.google.com/group/comp.lang.python/browse_frm/thread/a6010c7494871bb1/62a2da68961caeb6?lnk=gst&q=simionato+challenge&rnum=1&hl=en#62a2da68961caeb6 Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From nnorwitz at gmail.com Mon Apr 23 05:12:40 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 22 Apr 2007 20:12:40 -0700 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: Message-ID: Can you add refs to all the PEPs? Will this go into 2.6 also? If so, you should send to python-dev too. If this is accepted, can you update the 2.6 PEP 361. On 4/22/07, Brett Cannon wrote: > > Implementation > ============== > > When the ``-m`` option is used, ``sys.main`` will be set to the > argument passed in. ``sys.argv`` will be adjusted as it is currently. > Then the equivalent of ``__import__(self.main)`` will occur. This > differs from current semantics as the ``runpy`` module fetches the > code object for the file specified by the module name in order to > explicitly set ``__name__`` and other attributes. This is no longer > needed as import can perform its normal operation in this situation. > > If a file name is specified, then ``sys.main`` will be set to > ``"__main__"``. The specified file will then be read and have a code > object created and then be executed with ``__name__`` set to > ``"__main__"``. This mirrors current semantics. What happens when no file is passed in (ie, interactive) or when -c is used? Will sys.main be set? If so, to what? n From steven.bethard at gmail.com Mon Apr 23 05:34:54 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 22 Apr 2007 21:34:54 -0600 Subject: [Python-3000] [Python-Dev] PEP 30xx: Access to Module/Class/Function Currently Being Defined (this) In-Reply-To: References: Message-ID: On 4/22/07, Jim Jewett wrote: > PEP: 30XX > Title: Access to Module/Class/Function Currently Being Defined (this) [snip] > > Abstract > > It is common to need a reference to the current module, class, > or function, but there is currently no entirely correct way to > do this. This PEP proposes adding the keywords __module__, > __class__, and __function__. +1 on the general idea, if it can be implemented. ;-) One thing that isn't clear... You're saying "keywords". So that means "__module__" would not be a normal attribute looked up on in the globals or builtins, but instead would be associated with its own op-code? STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From guido at python.org Mon Apr 23 05:36:04 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 22 Apr 2007 20:36:04 -0700 Subject: [Python-3000] question Re: Need help completing ABC pep In-Reply-To: <462B3A83.1060204@openendsystems.com> References: <46291596.1020105@acm.org> <5.1.1.6.0.20070421105729.04c77840@sparrow.telecommunity.com> <462B3A83.1060204@openendsystems.com> Message-ID: On 4/22/07, Samuele Pedroni wrote: > If we keep s % d (do we? reading the list archives and the peps didn't > give an answer, I suppose it depends if PEP 3101 is accepted): > > will it check if d is a Mapping or continue to use the current heuristics? > > I suppose this is a legitimate question for other corners of the > language, std lib. Any place that currently uses a heuristic should be changed to check for Mapping instead (actually, in this particular case BasicMapping, as the % operator doesn't need to iterate over the keys or items). And most places that currently check for a dict should probably check for a Mapping or BasicMapping as well. BUT BUT BUT -- this is only for places that already check (like s%d). Places that just plunge in and call __getitem__ should continue to do so and should not be modified to require a flavor Mapping. Similarly, assignment to sys.stdout shouldn't be constrained to subclasses of io.IOBase; we can and should continue to use duck typing where it makes sense. PS. I fully intend to be dropping s%d in favor of the PEP 3101 solution. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Mon Apr 23 05:52:03 2007 From: brett at python.org (Brett Cannon) Date: Sun, 22 Apr 2007 20:52:03 -0700 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: Message-ID: On 4/22/07, Christian Heimes wrote: > Brett Cannon wrote: > > Implementation > > ============== > > > > When the ``-m`` option is used, ``sys.main`` will be set to the > > argument passed in. ``sys.argv`` will be adjusted as it is currently. > > Then the equivalent of ``__import__(self.main)`` will occur. This > > differs from current semantics as the ``runpy`` module fetches the > > code object for the file specified by the module name in order to > > explicitly set ``__name__`` and other attributes. This is no longer > > needed as import can perform its normal operation in this situation. > > __import__(self.main)? I guess you wanted to write __import__(sys.main) :) > Yep, that's what I meant. > > The newly proposed solution does introduce an added line of > > boilerplate which is a module import. But as the solution does not > > introduce a new built-in or module attribute (as discussed in > > `Rejected Ideas`_) it has been deemed worth the extra line. > > Someone on python-idea brought up the idea to add the sys module to > builtins. The feature would remove the requirement for an extra line and > the costs should be minimal or even zero since sys is a special builtin > module. > That's a separate PEP. > Some people including me like to resurrect the PEP 299 __main__() > function idea. http://www.python.org/dev/peps/pep-0299/ As I have already said multiple times on python-ideas, I don't personally like that idea. If someone wants to do the work to try to push that view then they can. -Brett From brett at python.org Mon Apr 23 05:56:54 2007 From: brett at python.org (Brett Cannon) Date: Sun, 22 Apr 2007 20:56:54 -0700 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: Message-ID: On 4/22/07, Neal Norwitz wrote: > Can you add refs to all the PEPs? > Sure, although considering my laptop died within minutes of my last commit to 25-maint it might be a little while (using my girlfriend's laptop to check email). > Will this go into 2.6 also? It's covered in the transition plan. Basically sys.main just becomes equal to "__main__" and nothing else changes. Figured didn't have to involve python-dev with such a minor change. Plus Guido has to go for this first. =) > If so, you should send to python-dev too. > If this is accepted, can you update the 2.6 PEP 361. > If I remember, sure. =) > On 4/22/07, Brett Cannon wrote: > > > > Implementation > > ============== > > > > When the ``-m`` option is used, ``sys.main`` will be set to the > > argument passed in. ``sys.argv`` will be adjusted as it is currently. > > Then the equivalent of ``__import__(self.main)`` will occur. This > > differs from current semantics as the ``runpy`` module fetches the > > code object for the file specified by the module name in order to > > explicitly set ``__name__`` and other attributes. This is no longer > > needed as import can perform its normal operation in this situation. > > > > If a file name is specified, then ``sys.main`` will be set to > > ``"__main__"``. The specified file will then be read and have a code > > object created and then be executed with ``__name__`` set to > > ``"__main__"``. This mirrors current semantics. > > What happens when no file is passed in (ie, interactive) or when -c is used? > Will sys.main be set? If so, to what? Good question. I would think None would be a good value. Otherwise the empty string if people don't want to have a possibly different type other than str in the attribute. -Brett From pje at telecommunity.com Mon Apr 23 06:09:57 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 23 Apr 2007 00:09:57 -0400 Subject: [Python-3000] Generic function PEP won't make it in time Message-ID: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> I got off to a good start in terms of figuring out what needs to be in the PEP... good enough to realize that a complete specification of what I have in mind is probably going to dwarf the WSGI spec. There are just too many details that have to be explained; the roughly 1000 lines of Python code I have in mind is translating to quite a *lot* more words of English, to explain intentions, ramifications, etc. In particular, when you implement generic functions using generic functions, it reads a lot like trying to explain the relationships between "type" and "object" (i.e. type being its own type and an instance of object, and object being a subclass of type). You can say what needs to be said in a fairly short space, but saying it in a way that lets people actually wrap their heads around it takes a bit more text. :) So, there is no way I could get it done in time for anybody to *read* it before the deadline, let alone discuss it (and there are at least a few things about it I think would really *need* some discussion before finalizing the proposal). On the plus side, it was interesting to think through how I'd take the current peak.rules.core and adapt it to Python 3.0, with argument annotations and no classic classes to worry about. It was also interesting to look at how certain things might overlap with other Py3K PEP's; for example, the peak.rules.core API has an '@abstract' decorator that's used in basically the same way as '@abstractmethod'... except that you can add overloads to an @abstract function. (Functions not declared @abstract are considered to be a 1-method generic function whose default rule is the function's original body code.) Anyway, I'm not abandoning the idea, but it's beginning to seem to me that it might be a lot easier to close the open issues by actually working on an implementation instead of first trying to write an adequate explanation of what the issues are. However, I was *definitely* being delusional about how much time it would take to write this up -- probably because how to make systems like this is now so deeply embedded in my subconscious that I don't think about it much any more. Like "type" and "object", it's all very simple... until you try to explain it. :) (It occurs to me that I should mention that, just like the relationship between 'type' and 'object' rarely matters unless you delve into metaclasses or other wizardry, so too am I referring here only to the wizard-level aspects of a generic function module, where my design allows for people to write their own dispatchers or interface implementations and plug them into the base module, using the same decorators and basic machinery. Merely *using* generic functions and interfaces would of course be at least as simple as interfaces and static overloads in Java or C++ -- that's *not* the part of the PEP that's hard to write.) From tjreedy at udel.edu Mon Apr 23 06:17:23 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 23 Apr 2007 00:17:23 -0400 Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function CurrentlyBeing Defined (this) References: Message-ID: "Jim Jewett" wrote in message news:fb6fbf560704222005le4798a4j5daa5e71e644f069 at mail.gmail.com... | Functions (including methods) often want access to themselves, | usually for a private storage location. Or for true recursion. At present, a function has to 'hope' that its definition name is not rebound. tjr From steven.bethard at gmail.com Mon Apr 23 06:43:21 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 22 Apr 2007 22:43:21 -0600 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: Message-ID: On 4/22/07, Brett Cannon wrote: > Implementation > ============== > > When the ``-m`` option is used, ``sys.main`` will be set to the > argument passed in. ``sys.argv`` will be adjusted as it is currently. > Then the equivalent of ``__import__(self.main)`` will occur. This > differs from current semantics as the ``runpy`` module fetches the > code object for the file specified by the module name in order to > explicitly set ``__name__`` and other attributes. This is no longer > needed as import can perform its normal operation in this situation. > > If a file name is specified, then ``sys.main`` will be set to > ``"__main__"``. The specified file will then be read and have a code > object created and then be executed with ``__name__`` set to > ``"__main__"``. This mirrors current semantics. So __name__ will still sometimes be "__main__". That's disappointing. To clarify, assuming that the foo.bar module contains something like "from . import baz", this PEP only addresses the following situation:: python -mfoo.bar and all of the following will still raise ImportErrors:: ~> python foo/bar ~/grok> python ../foo/bar ~/foo> python bar Right? If that's right, I'm -1 on the proposal. It's complicating the standard "am I the main module?" idiom to solve a tiny little problem with "-m". The "am I the main module?" idiom has been around much longer than the "-m" flag, and I'd prefer to see a more compelling reason to change it. Two reasons that would be compelling for me: * Simplifying the "am I the main module?" idiom, e.g. with the rejected ``if __main__`` proposal. * Getting rid of the "__main__" value for __name__ entirely. This would require code like the following to determine the name of a module given at the command line:: def get_name(path): sys_path_set = set(sys.path) path, _ = os.path.splitext(path) path = os.path.abspath(path) parts = path.split(os.path.sep) for i in range(1, len(parts)): sub_path = os.path.sep.join(parts[:i]) if sub_path in sys_path_set: return '.'.join(parts[i:]) return parts[-1] Even if we can't do the name resolution perfectly, if the flaws are documented (e.g. when '..' and symbolic links are combined) I think that would be enough better than the current situation to merit changing the standard "am I the current module?" idiom. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From michele.cella at gmail.com Mon Apr 23 08:48:28 2007 From: michele.cella at gmail.com (Michele Cella) Date: Mon, 23 Apr 2007 08:48:28 +0200 Subject: [Python-3000] Fixing super anyone? In-Reply-To: <462BDCAA.7020501@canterbury.ac.nz> References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> <462BDCAA.7020501@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Michele Cella wrote: > >> Personally, I really like this syntax: >> >> res = super currentmethod(arg, arg) >> >> Why? because it's very consistent (pythonic?) with the way I'm used to >> use any other keyword. > > But it's inconsistent with the way every other > method is called. > Yeah, you're absolutely right. :-) Have you seen my second proposal? Calling a method: self.method(arg) Calling a super method: super self.method(arg) That's consistent with: * the way you call any other method * the way you use any other keyword Thanks for the attention. Ciao Michele From ncoghlan at gmail.com Mon Apr 23 11:31:39 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 23 Apr 2007 19:31:39 +1000 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: Message-ID: <462C7CFB.5060607@gmail.com> Brett Cannon wrote: > This PEP is to change the ``if __name__ == "__main__": ...`` idiom to > ``if __name__ == sys.main: ...`` so that you at least have a chance > to execute module in a package that use relative imports. > > Ran this PEP past python-ideas. Stopped the discussion there when too > many new ideas were being proposed. =) I have listed all of them in > the Rejected Ideas section, although if overwhelming support for one > comes forward the PEP can shift to one of them. -1 I still prefer the "__module_name__" fallback that was actually briefly in the tree for 2.5 (the change was made really late in the 2.5 cycle, so it was backed out with the intention of revisiting the idea for 2.6 - check the svn log for lib/runpy.py). The trick with that approach was that runpy.run_module() would *always* set "__module_name__" to the real name of the module being executed, regardless of what it was told to set "__name__" to. Then the import machinery was modified such that if it came across an explicit relative import being made from a module called "__main__", it would check to see if "__module_name__" was set, and if so, use that instead. The issue discussed in PEP 338 (and this new PEP) gets fixed, with zero impact on anything else (even normal import speed is unaffected, as the check for "__module_name__" would only occur on an error handling path that currently raises an exception). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From thomas at python.org Mon Apr 23 15:07:05 2007 From: thomas at python.org (Thomas Wouters) Date: Mon, 23 Apr 2007 15:07:05 +0200 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> <462BDCAA.7020501@canterbury.ac.nz> Message-ID: <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> On 4/23/07, Michele Cella wrote: > Calling a method: > > self.method(arg) > > Calling a super method: > > super self.method(arg) > > That's consistent with: > * the way you call any other method > * the way you use any other keyword But how would it *work*? I assume you want 'super' to take a single expression, and be an expression itself. How does it know which instance you want to 'super'? Does it specialcase the variable named 'self'? Threat 'everything up to the first dot' as 'the instance'? Specialcase the first argument to the method? (That's what 'super.foo()' would do.) How would you do things like: # call the supermethod 'register' with the nonsuper 'callback': super.register(self.callback) # call the nonsuper 'register' with the supermethod 'callback' self.register(super.callback) # call the 'frominstance' method of the 'innerclass' attribute on the superclass super.innerclass.frominstance(self) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070423/e1da0542/attachment.html From lists at cheimes.de Mon Apr 23 15:07:33 2007 From: lists at cheimes.de (Christian Heimes) Date: Mon, 23 Apr 2007 15:07:33 +0200 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: Message-ID: Brett Cannon wrote: >> Someone on python-idea brought up the idea to add the sys module to >> builtins. The feature would remove the requirement for an extra line and >> the costs should be minimal or even zero since sys is a special builtin >> module. >> > > That's a separate PEP. Do you remember the PEP number? I can't find the PEP. >> Some people including me like to resurrect the PEP 299 __main__() >> function idea. http://www.python.org/dev/peps/pep-0299/ > > As I have already said multiple times on python-ideas, I don't > personally like that idea. If someone wants to do the work to try to > push that view then they can. I know. I mentioned it because I "wanted to push that view". O:-) Christian From lists at cheimes.de Mon Apr 23 15:21:01 2007 From: lists at cheimes.de (Christian Heimes) Date: Mon, 23 Apr 2007 15:21:01 +0200 Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function Currently Being Defined (this) In-Reply-To: References: Message-ID: > Proposal: Add a __module__ keyword which refers to the module > currently being defined (executed). (But see open issues.) > > if __module__ is sys.main: ... # assuming PEP 3020, Cannon > -1 on __module__ I understand PEP 3020 correctly then sys.main will contain the dotted name of the module as *string*. Also __module__ is already used in objects like classes. >>> class C: pass ... >>> C.__module__ '__main__' >>> type(C.__module__) I don't like the fact that __module__ is going to be a string in some cases and a module object in other cases. It's too confusing. class C: def egg(self): # string from class C or an object from module? # confused ... nonlocal __module__ It might work with __this_module__ but I don't see the point in getting the local module. I'v used it about 5 times in my many years as Python developer when I had to deal with a legacy product. The module and package names were badly chosen (Foo/Foo.py: import Foo). Christian From barry at python.org Mon Apr 23 15:36:20 2007 From: barry at python.org (Barry Warsaw) Date: Mon, 23 Apr 2007 09:36:20 -0400 Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function Currently Being Defined (this) In-Reply-To: References: Message-ID: <88B9B3C2-D226-48C1-9C03-8028BC8390C6@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 23, 2007, at 9:21 AM, Christian Heimes wrote: > -1 on __module__ > > I understand PEP 3020 correctly then sys.main will contain the dotted > name of the module as *string*. > > Also __module__ is already used in objects like classes. I'm not sure whether a general mechanism is necessary or not, but I've always wished I could just do something like "import __me__" to get the current module. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRiy2VHEjvBPtnXfVAQKgdwQAmuAVdKvz+rZdOPHHSyk03bOM6wkUQ9FU +qhhThUPl9yZIexJP5DvE/fEo6b3hsU87kkSBmTVynCCdVzzhwYDqILdGvCaJ8wP L+K3oD9mq69BJ8bVi1mGu8st9XmK/jwWuq6d6Fo3pMKGehuk38EtW3N6dzHHFG0g oECLki6Fc1I= =yw83 -----END PGP SIGNATURE----- From steven.bethard at gmail.com Mon Apr 23 15:53:44 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 23 Apr 2007 07:53:44 -0600 Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function Currently Being Defined (this) In-Reply-To: <88B9B3C2-D226-48C1-9C03-8028BC8390C6@python.org> References: <88B9B3C2-D226-48C1-9C03-8028BC8390C6@python.org> Message-ID: On 4/23/07, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Apr 23, 2007, at 9:21 AM, Christian Heimes wrote: > > > -1 on __module__ > > > > I understand PEP 3020 correctly then sys.main will contain the dotted > > name of the module as *string*. > > > > Also __module__ is already used in objects like classes. > > I'm not sure whether a general mechanism is necessary or not, but > I've always wished I could just do something like "import __me__" to > get the current module. Well, you can always do:: __import__(__name__) That's not much more verbose. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From steven.bethard at gmail.com Mon Apr 23 16:19:12 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 23 Apr 2007 08:19:12 -0600 Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function Currently Being Defined (this) In-Reply-To: References: <88B9B3C2-D226-48C1-9C03-8028BC8390C6@python.org> Message-ID: On 4/23/07, Steven Bethard wrote: > On 4/23/07, Barry Warsaw wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA1 > > > > On Apr 23, 2007, at 9:21 AM, Christian Heimes wrote: > > > > > -1 on __module__ > > > > > > I understand PEP 3020 correctly then sys.main will contain the dotted > > > name of the module as *string*. > > > > > > Also __module__ is already used in objects like classes. > > > > I'm not sure whether a general mechanism is necessary or not, but > > I've always wished I could just do something like "import __me__" to > > get the current module. > > Well, you can always do:: > > __import__(__name__) > > That's not much more verbose. Of course, to make it work with packages, you have to write something like:: mod = __import__(__name__) for sub_mod_name in __name__.split('.')[1:]: mod = getattr(mod, sub_mod_name) I guess that's not so simple after all. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From lists at cheimes.de Mon Apr 23 16:41:42 2007 From: lists at cheimes.de (Christian Heimes) Date: Mon, 23 Apr 2007 16:41:42 +0200 Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function Currently Being Defined (this) In-Reply-To: References: <88B9B3C2-D226-48C1-9C03-8028BC8390C6@python.org> Message-ID: Steven Bethard wrote: > Of course, to make it work with packages, you have to write something like:: > > mod = __import__(__name__) > for sub_mod_name in __name__.split('.')[1:]: > mod = getattr(mod, sub_mod_name) > > I guess that's not so simple after all. > > STeVe Do you want to do something like: import sys mod = sys.modules[__name__] mod = sys.modules[self.__class__.__module__] ? Christian From exarkun at divmod.com Mon Apr 23 16:53:32 2007 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Mon, 23 Apr 2007 10:53:32 -0400 Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function Currently Being Defined (this) In-Reply-To: Message-ID: <20070423145332.19381.2022886003.divmod.quotient.4334@ohm> On Mon, 23 Apr 2007 08:19:12 -0600, Steven Bethard wrote: >On 4/23/07, Steven Bethard wrote: >> On 4/23/07, Barry Warsaw wrote: >> > -----BEGIN PGP SIGNED MESSAGE----- >> > Hash: SHA1 >> > >> > On Apr 23, 2007, at 9:21 AM, Christian Heimes wrote: >> > >> > > -1 on __module__ >> > > >> > > I understand PEP 3020 correctly then sys.main will contain the dotted >> > > name of the module as *string*. >> > > >> > > Also __module__ is already used in objects like classes. >> > >> > I'm not sure whether a general mechanism is necessary or not, but >> > I've always wished I could just do something like "import __me__" to >> > get the current module. >> >> Well, you can always do:: >> >> __import__(__name__) >> >> That's not much more verbose. > >Of course, to make it work with packages, you have to write something like:: > > mod = __import__(__name__) > for sub_mod_name in __name__.split('.')[1:]: > mod = getattr(mod, sub_mod_name) > >I guess that's not so simple after all. Maybe this indicates a problem with __import__. Twisted has something called namedAny, and I always use that, because it just works, unlike __import__, which seems to take too many arguments and has weird behavior for dotted names. me = namedAny(__name__) The inverse is also available (qual(me) == __name__), and is another function I've always thought it strange was missing from the stdlib. Jean-Paul From guido at python.org Mon Apr 23 16:57:04 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Apr 2007 07:57:04 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> Message-ID: That's okay. We won't be able to consider GFs for 3.0a1, but the whole library reorg thing will be done later, so I expect adding GFs later (to the library) will also be okay. I do hope that you'll stay with us while I barrel forward with the ABC PEP. I see no serious incompatibilities between GFs and ABCs, so I don't believe I should hold it up just because your proposal is late; but there is time to tweak the details to avoid gratuitous friction between the two mechanisms. --Guido On 4/22/07, Phillip J. Eby wrote: > I got off to a good start in terms of figuring out what needs to be in the > PEP... good enough to realize that a complete specification of what I have > in mind is probably going to dwarf the WSGI spec. There are just too many > details that have to be explained; the roughly 1000 lines of Python code I > have in mind is translating to quite a *lot* more words of English, to > explain intentions, ramifications, etc. > > In particular, when you implement generic functions using generic > functions, it reads a lot like trying to explain the relationships between > "type" and "object" (i.e. type being its own type and an instance of > object, and object being a subclass of type). You can say what needs to be > said in a fairly short space, but saying it in a way that lets people > actually wrap their heads around it takes a bit more text. :) > > So, there is no way I could get it done in time for anybody to *read* it > before the deadline, let alone discuss it (and there are at least a few > things about it I think would really *need* some discussion before > finalizing the proposal). > > On the plus side, it was interesting to think through how I'd take the > current peak.rules.core and adapt it to Python 3.0, with argument > annotations and no classic classes to worry about. It was also interesting > to look at how certain things might overlap with other Py3K PEP's; for > example, the peak.rules.core API has an '@abstract' decorator that's used > in basically the same way as '@abstractmethod'... except that you can add > overloads to an @abstract function. (Functions not declared @abstract are > considered to be a 1-method generic function whose default rule is the > function's original body code.) > > Anyway, I'm not abandoning the idea, but it's beginning to seem to me that > it might be a lot easier to close the open issues by actually working on an > implementation instead of first trying to write an adequate explanation of > what the issues are. > > However, I was *definitely* being delusional about how much time it would > take to write this up -- probably because how to make systems like this is > now so deeply embedded in my subconscious that I don't think about it much > any more. Like "type" and "object", it's all very simple... until you try > to explain it. :) > > (It occurs to me that I should mention that, just like the relationship > between 'type' and 'object' rarely matters unless you delve into > metaclasses or other wizardry, so too am I referring here only to the > wizard-level aspects of a generic function module, where my design allows > for people to write their own dispatchers or interface implementations and > plug them into the base module, using the same decorators and basic > machinery. Merely *using* generic functions and interfaces would of course > be at least as simple as interfaces and static overloads in Java or C++ -- > that's *not* the part of the PEP that's hard to write.) > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Mon Apr 23 17:26:23 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 23 Apr 2007 11:26:23 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070423111608.00a04748@sparrow.telecommunity.com> At 07:57 AM 4/23/2007 -0700, Guido van Rossum wrote: >That's okay. We won't be able to consider GFs for 3.0a1, but the whole >library reorg thing will be done later, so I expect adding GFs later >(to the library) will also be okay. I do hope that you'll stay with us >while I barrel forward with the ABC PEP. I see no serious >incompatibilities between GFs and ABCs, so I don't believe I should >hold it up just because your proposal is late; but there is time to >tweak the details to avoid gratuitous friction between the two >mechanisms. Yeah, I don't see any issues there at all; the only places I can think of right now where there's even any overlap with non-library stuff is GF @abstract with ABC @abstractmethod, and the possibility of having the default metaclass (i.e., 'type') support an "implements" keyword argument. And all of the stuff I'm proposing (except for argument annotation support) has been implemented before in either PEAK-Rules or PyProtocols, so it's mostly a matter of things like getting rid of bytecode hacks, and making sure the spec can be implemented in other Python interpreters. (PyPy should have no problems, but I don't know enough about Jython or IronPython to be able to say the same for them.) From p.f.moore at gmail.com Mon Apr 23 17:35:22 2007 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 23 Apr 2007 16:35:22 +0100 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> Message-ID: <79990c6b0704230835h715f799au32ee4ceb73637784@mail.gmail.com> On 23/04/07, Phillip J. Eby wrote: > (It occurs to me that I should mention that, just like the relationship > between 'type' and 'object' rarely matters unless you delve into > metaclasses or other wizardry, so too am I referring here only to the > wizard-level aspects of a generic function module, where my design allows > for people to write their own dispatchers or interface implementations and > plug them into the base module, using the same decorators and basic > machinery. Merely *using* generic functions and interfaces would of course > be at least as simple as interfaces and static overloads in Java or C++ -- > that's *not* the part of the PEP that's hard to write.) I wonder whether starting off with a PEP that omits or skims over the wizard-level stuff would not actually be more use. If you already have most of that PEP, why not publish it, with a section called something like "Advanced Use" which for now is simply a "to be completed" placeholder. If the basic stuff really is that simple, I'd love to see just that for now. It would put the advanced stuff into context when it finally arrived. Paul From jimjjewett at gmail.com Mon Apr 23 17:51:37 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 23 Apr 2007 11:51:37 -0400 Subject: [Python-3000] [Python-Dev] PEP 30xx: Access to Module/Class/Function Currently Being Defined (this) In-Reply-To: References: Message-ID: On 4/22/07, Steven Bethard wrote: > On 4/22/07, Jim Jewett wrote: > > PEP: 30XX > > Title: Access to Module/Class/Function Currently Being Defined (this) > One thing that isn't clear... You're saying "keywords". So that means > "__module__" would not be a normal attribute looked up on in the > globals or builtins, but instead would be associated with its own > op-code? My assumption would be something closer to Michele's solution -- the parser would create a new type of AST node, and the compiler would emit slightly different bytecode, but no new op-codes would be needed. -jJ From daniel at stutzbachenterprises.com Mon Apr 23 17:56:54 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 23 Apr 2007 10:56:54 -0500 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical Message-ID: I have two proposals for Python 3000 that I'll write PEP(s) for if there's interest. The first proposal is, I think, a modest one. The second proposal is related, but more radical. I fully expect the second proposal to be rejected for that alone, especially since I am a relatively an outsider to the Python developer community (though it was great to meet some of you at PyCon this year). I bring up the more radical proposal primarily for completeness. The Modest Proposal ------------------------------- As a pet project, I've written a container type that looks, acts, and quacks like a Python list(), but has better asymptotic performance. Specifically, O(log n) time for inserts and deletes in the middle of the list. I'd like to offer it for inclusion in the collections module. Probably, you have encountered some other proposals for types that meet this description. Here are some of the unique features of mine, called a BList, which is based on B+Trees. I'm open to suggestions for a better name. - just as fast as a Python list() when the list is small - getslice runs in O(log n) time - making a shallow copy runs in O(1) time - setslice runs in O(log n + log k) time if the inserted slice is a BList of length k - multipling a BList by k takes O(log k) time and O(log k) memory Example: >>> from blist import * >>> x = BList([0]) >>> x *= 2**29 >>> x.append(5) >>> y = x[4:-234234] >>> del x[3:1024] None of the above operations have a noticeable delay, even though the lists have over 500 million elements due to line 3. The BList has two key features that allow it to pull this off this performance: 1) The BList type features transparent copy-on-write. If a non-root node needs to be copied (as part of a getslice, copy, setslice, etc.), the node is shared between multiple parents instead of being copied. If it needs to be modified later, it will be copied at that time. This is completely behind-the-scenes; from the user's point of view, the BList works just like a regular Python list. 2) A B+Tree is a wide, squat tree. The maximum number of children per node is defined by a compile-time parameter (LIMIT). If the list is shorter than LIMIT, then there is only one node, which simply contains an array of the objects in the list. In other words, for short lists, a BList works just like Python's array-based list() type. Thus, it has the same good performance on small lists. So you can see the performance of the BList in more detail, I've made several performance graphs available at the following link: http://stutzbachenterprises.com/blist/ Each webpage shows two graphs: - The raw execution time for the operation as a function of the list size - The execution time for the operation, normalized by the execution time of Python's list() These graphs were made using LIMIT=128. The only operation where list() is much faster for small lists is getitem through the x[5] syntax. There's a special case for that operation deep inside the Python bytecode interpreter. My extension module can't compete with that. :-) BList is just as fast as list() for small lists via .__getitem__(). The source code for the type is available for compilation as an external module: http://stutzbachenterprises.com/blist/blist.tar.gz I wrote it against Python 2.5. The Radical Proposal ------------------------------- Replace list() with the BList. There are only a few use-cases (that I can think of) where Python's list() regularly outperforms the BList. These are: 1. A large LIFO stack, where there are many .append() and .pop(-1) operations. These are O(1) for a Python list, but O(log n) for the BList(). (BList's are just as fast for small lists) 2. An large, unchanging list, where there are many getitem() calls, and none of the inserts or deletes where the BList() shines. Again, these are O(1) for a Python list, but O(log n) for the BList(), and the BList() is just as fast if the list is small. Use case #1 is covered by collections.deque. Use case #2 is covered by tuples. When I first started learning Python, I often ended up inadvertently writing O(n**2) algorithms by putting a O(n) list operation inside a loop (which is a problem when n=100,000). Since nothing in the signature of list() informs the user when an operation is going to be slow, it took me a while to learn how to write my code to work around this constraint. In fact, I initially assumed "list" meant "linked list" so most of my assumptions about which operations would be fast vs. slow were backwards. I think it would be more Pythonic if the user can trust that the implementation will always be fast (at least O(log n)). I recognize that the Python developer community is understandably very attached to the current array-based list, so I expect this to get shot down. I hope this doesn't reflect badly on the more modest proposal of including a new type in the collections module. Also, please don't kill the messenger. :-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From pje at telecommunity.com Mon Apr 23 18:09:37 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 23 Apr 2007 12:09:37 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <79990c6b0704230835h715f799au32ee4ceb73637784@mail.gmail.co m> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> At 04:35 PM 4/23/2007 +0100, Paul Moore wrote: >If the basic stuff really is that simple, I'd love to see just that >for now. It would put the advanced stuff into context when it finally >arrived. Here you go: from overloading import overload, abstract def foo(bar, baz): """Default implementation here""" @overload def foo(bar:int, baz:str): """Overloaded implementation for int/str here""" @abstract def spam(fizz): """This function has no default implementation, and raises a "no applicable methods" error if called...""" @overload def spam(fizz:list): """...unless an appropriate overload is added first""" Any questions? :) Miscellaneous notes/points for discussion: * Names/locations of errors to be raised when methods aren't found or are ambiguous (e.g. should they be (or subclass) NotImplementedError? TypeError?) * Notice that it's not necessary to specially declare a function as being "generic"; you can therefore overload an existing function, classmethod, staticmethod, etc., and the existing function object is just modified in-place. * CLOS/AspectJ-style method combination is supported, using @before, @after, and @around decorators. That's pretty much it, for the generic function part. The interface part looks like the "recombinable interfaces" one I previously posted, where you simply subclass Interface, and you don't have to write any adapter classes, because the interface is its own adapter class. You just register methods for stuff. I would also propose an 'implements' class keyword argument that asks the interface objects its been given in order to register the class. In the case of overloading.Interface, that would be done by registering the class' methods of matching names with the generic functions embedded in the Interface, but other interface implementations might work differently. There are perhaps a few more details or features visible at this user level, but all the "interesting" stuff (i.e. wizardry and defense against the dark arts) takes place under the hood. Specifically, how to make all this such that advanced or customized dispatch systems and interfaces can play in the same playground, being able to use the *same decorators* and "implements" keyword, so that systems like PEAK-Rules and zope.interface won't appear to be second-class citizens. From pje at telecommunity.com Mon Apr 23 18:11:30 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 23 Apr 2007 12:11:30 -0400 Subject: [Python-3000] [Python-Dev] PEP 30xx: Access to Module/Class/Function Currently Being Defined (this) In-Reply-To: References: Message-ID: <5.1.1.6.0.20070423121019.04426ca8@sparrow.telecommunity.com> At 11:51 AM 4/23/2007 -0400, Jim Jewett wrote: >On 4/22/07, Steven Bethard wrote: > > On 4/22/07, Jim Jewett wrote: > > > PEP: 30XX > > > Title: Access to Module/Class/Function Currently Being Defined (this) > > > One thing that isn't clear... You're saying "keywords". So that means > > "__module__" would not be a normal attribute looked up on in the > > globals or builtins, but instead would be associated with its own > > op-code? > >My assumption would be something closer to Michele's solution -- the >parser would create a new type of AST node, and the compiler would >emit slightly different bytecode, but no new op-codes would be needed. Why would a new type of AST node be required? Wouldn't these just be a special symbol, like __debug__? Or does __debug__ have its own AST node type now? From janssen at parc.com Mon Apr 23 18:31:36 2007 From: janssen at parc.com (Bill Janssen) Date: Mon, 23 Apr 2007 09:31:36 PDT Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: References: Message-ID: <07Apr23.093138pdt."57996"@synergy1.parc.xerox.com> > Replace list() with the BList. Generally, I like the idea. But how about extension modules which make use of the current implementation details? Bill From jarausch at igpm.rwth-aachen.de Mon Apr 23 18:30:57 2007 From: jarausch at igpm.rwth-aachen.de (Helmut Jarausch) Date: Mon, 23 Apr 2007 18:30:57 +0200 (CEST) Subject: [Python-3000] a plea : invalid comparisons should raise an exception Message-ID: Some years ago, I was allured to convert from Perl to Python. The main argument was that Python is much more secure. That turned out to be true with one big exception. The following lines would run just fine (as equivalent Perl code) from cStringIO import StringIO Inp= StringIO('Guido 10') for line in Inp: (Name,Age)= line.split() if Age > 64 : print Name,"has retired" But with Python it returns a very unexpected result without any evidence of a problem. So, please make comparisons raise an exception (at least optionally) if "uncompatible" types are compared. -- Helmut Jarausch Lehrstuhl fuer Numerische Mathematik RWTH - Aachen University D 52056 Aachen, Germany From jimjjewett at gmail.com Mon Apr 23 18:46:59 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 23 Apr 2007 12:46:59 -0400 Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function Currently Being Defined (this) In-Reply-To: References: Message-ID: On 4/23/07, Christian Heimes wrote: > > Proposal: Add a __module__ keyword which refers to the module > > currently being defined (executed). (But see open issues.) > -1 on __module__ > Also __module__ is already used in objects like classes. Yes, I worried that it might cause some confusion, but I also worry that it might cause confusion *not* to reuse the name. X.__module__ is the __module__ attribute of X X.__class__ is the __class__ attribute of X An (unqualified) __module__ would represent the current module. An (unqualified) __class__ would represent the current class. That said, the choice between reused short names (__module__) and longer names (__this_module__, __current_module__, __thismodule__) is indeed one of the open issues. I'll count this as one vote against the short forms. -jJ From p.f.moore at gmail.com Mon Apr 23 18:50:06 2007 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 23 Apr 2007 17:50:06 +0100 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> Message-ID: <79990c6b0704230950secc75aflb8fc92c7ea950877@mail.gmail.com> On 23/04/07, Phillip J. Eby wrote: > Here you go: [...] > Any questions? :) No :-) I really do think that putting this in a PEP as it is, would be a good start. > That's pretty much it, for the generic function part. The interface part > looks like the "recombinable interfaces" one I previously posted, where you > simply subclass Interface, and you don't have to write any adapter classes, > because the interface is its own adapter class. You just register methods > for stuff. Again, I'd argue for getting it in a PEP. > There are perhaps a few more details or features visible at this user > level, but all the "interesting" stuff (i.e. wizardry and defense against the dark > arts) takes place under the hood. I don't see the harm in putting that in later - particularly if it means that the basic end user story is documented formally from the start in a PEP. If nothing else, it makes it easier for you to remind people that GFs are easy, without having to repeatedly post that example... :-) Paul. From jimjjewett at gmail.com Mon Apr 23 18:52:59 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 23 Apr 2007 12:52:59 -0400 Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function Currently Being Defined (this) In-Reply-To: References: <88B9B3C2-D226-48C1-9C03-8028BC8390C6@python.org> Message-ID: On 4/23/07, Christian Heimes wrote: > Steven Bethard wrote: > > Of course, to make it work with packages, you have to write something like:: > > mod = __import__(__name__) > > for sub_mod_name in __name__.split('.')[1:]: > > mod = getattr(mod, sub_mod_name) > > I guess that's not so simple after all. > > > > STeVe > Do you want to do something like: > import sys > mod = sys.modules[__name__] > mod = sys.modules[self.__class__.__module__] Sure. But I also want it to work, even if module.__name__ has been changed, or module has been removed from sys.modules (perhaps for security reasons), or some other module is already using that name. That said, I won't be shocked if Guido eventually decides to support only one or two of the magic words, just as he initially limited decorators to functions. -jJ From nevillegrech at gmail.com Mon Apr 23 18:58:25 2007 From: nevillegrech at gmail.com (Neville Grech Neville Grech) Date: Mon, 23 Apr 2007 18:58:25 +0200 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: References: Message-ID: I like the implementation of the BList, however I'm not so sure whether replacing a list with BList is a good idea since there are many instances where python scripts will iterate through endless lists of items. In such case time complexity (to iterate a whole list) as you have mentioned is O (nlog(n)) instead of O(n). You will probably find that such cases (iterating through long lists) are very common and at the same time the developer still wants his lists to be mutable. If a developer wants to optimise his lists for insertion and deletion speed he can then use an optimised implementation like the one you're proposing anyway. What's the performance of concatenation in these BLists? O(n) ? Are you suggesting BList as part of the standard library? -Neville On 4/23/07, Daniel Stutzbach wrote: > > I have two proposals for Python 3000 that I'll write PEP(s) for if > there's interest. The first proposal is, I think, a modest one. The > second proposal is related, but more radical. I fully expect the > second proposal to be rejected for that alone, especially since I am a > relatively an outsider to the Python developer community (though it > was great to meet some of you at PyCon this year). I bring up the > more radical proposal primarily for completeness. > > The Modest Proposal > ------------------------------- > > As a pet project, I've written a container type that looks, acts, and > quacks like a Python list(), but has better asymptotic performance. > Specifically, O(log n) time for inserts and deletes in the middle of > the list. I'd like to offer it for inclusion in the collections > module. > > Probably, you have encountered some other proposals for types that > meet this description. Here are some of the unique features of mine, > called a BList, which is based on B+Trees. I'm open to suggestions > for a better name. > > - just as fast as a Python list() when the list is small > - getslice runs in O(log n) time > - making a shallow copy runs in O(1) time > - setslice runs in O(log n + log k) time if the inserted slice is a > BList of length k > - multipling a BList by k takes O(log k) time and O(log k) memory > > Example: > > >>> from blist import * > >>> x = BList([0]) > >>> x *= 2**29 > >>> x.append(5) > >>> y = x[4:-234234] > >>> del x[3:1024] > > None of the above operations have a noticeable delay, even though the > lists have over 500 million elements due to line 3. > > The BList has two key features that allow it to pull this off this > performance: > > 1) The BList type features transparent copy-on-write. If a non-root > node needs to be copied (as part of a getslice, copy, setslice, etc.), > the node is shared between multiple parents instead of being copied. > If it needs to be modified later, it will be copied at that time. > This is completely behind-the-scenes; from the user's point of view, > the BList works just like a regular Python list. > > 2) A B+Tree is a wide, squat tree. The maximum number of children per > node is defined by a compile-time parameter (LIMIT). If the list is > shorter than LIMIT, then there is only one node, which simply contains > an array of the objects in the list. In other words, for short lists, > a BList works just like Python's array-based list() type. Thus, it > has the same good performance on small lists. > > So you can see the performance of the BList in more detail, I've made > several performance graphs available at the following link: > http://stutzbachenterprises.com/blist/ > > Each webpage shows two graphs: > - The raw execution time for the operation as a function of the list size > - The execution time for the operation, normalized by the execution > time of Python's list() > > These graphs were made using LIMIT=128. > > The only operation where list() is much faster for small lists is > getitem through the x[5] syntax. There's a special case for that > operation deep inside the Python bytecode interpreter. My extension > module can't compete with that. :-) BList is just as fast as list() > for small lists via .__getitem__(). > > The source code for the type is available for compilation as an external > module: > http://stutzbachenterprises.com/blist/blist.tar.gz > > I wrote it against Python 2.5. > > The Radical Proposal > ------------------------------- > > Replace list() with the BList. > > There are only a few use-cases (that I can think of) where Python's > list() regularly outperforms the BList. These are: > > 1. A large LIFO stack, where there are many .append() and .pop(-1) > operations. These are O(1) for a Python list, but O(log n) for the > BList(). (BList's are just as fast for small lists) > > 2. An large, unchanging list, where there are many getitem() calls, > and none of the inserts or deletes where the BList() shines. Again, > these are O(1) for a Python list, but O(log n) for the BList(), and > the BList() is just as fast if the list is small. > > Use case #1 is covered by collections.deque. Use case #2 is covered by > tuples. > > When I first started learning Python, I often ended up inadvertently > writing O(n**2) algorithms by putting a O(n) list operation inside a > loop (which is a problem when n=100,000). Since nothing in the > signature of list() informs the user when an operation is going to be > slow, it took me a while to learn how to write my code to work around > this constraint. In fact, I initially assumed "list" meant "linked > list" so most of my assumptions about which operations would be fast > vs. slow were backwards. I think it would be more Pythonic if the > user can trust that the implementation will always be fast (at least > O(log n)). > > I recognize that the Python developer community is understandably very > attached to the current array-based list, so I expect this to get shot > down. I hope this doesn't reflect badly on the more modest proposal > of including a new type in the collections module. Also, please don't > kill the messenger. :-) > > -- > Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/nevillegrech%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070423/f0bd8136/attachment.html From daniel at stutzbachenterprises.com Mon Apr 23 18:57:18 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 23 Apr 2007 11:57:18 -0500 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: <2410590390749666520@unknownmsgid> References: <2410590390749666520@unknownmsgid> Message-ID: On 4/23/07, Bill Janssen wrote: > > Replace list() with the BList. > > Generally, I like the idea. But how about extension modules > which make use of the current implementation details? PyListObject could be moved to the collections module so that C extension modules can continue to use it. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From collinw at gmail.com Mon Apr 23 19:12:03 2007 From: collinw at gmail.com (Collin Winter) Date: Mon, 23 Apr 2007 10:12:03 -0700 Subject: [Python-3000] a plea : invalid comparisons should raise an exception In-Reply-To: References: Message-ID: <43aa6ff70704231012q4f6f4cb7i75997d8850f52b2f@mail.gmail.com> On 4/23/07, Helmut Jarausch wrote: > Some years ago, I was allured to convert from Perl to Python. > The main argument was that Python is much more secure. > That turned out to be true with one big exception. > The following lines would run just fine (as equivalent > Perl code) > > from cStringIO import StringIO > Inp= StringIO('Guido 10') > for line in Inp: > (Name,Age)= line.split() > if Age > 64 : > print Name,"has retired" > > But with Python it returns a very unexpected result > without any evidence of a problem. > > So, please make comparisons raise an exception (at least > optionally) if "uncompatible" types are compared. This is already going to happen; see http://www.python.org/dev/peps/pep-3100/#core-language, 12th bullet point. Collin Winter From jcarlson at uci.edu Mon Apr 23 19:16:29 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 23 Apr 2007 10:16:29 -0700 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: References: <2410590390749666520@unknownmsgid> Message-ID: <20070423101315.63C8.JCARLSON@uci.edu> "Daniel Stutzbach" wrote: > On 4/23/07, Bill Janssen wrote: > > > Replace list() with the BList. > > > > Generally, I like the idea. But how about extension modules > > which make use of the current implementation details? > > PyListObject could be moved to the collections module so that C > extension modules can continue to use it. Unless the implementation of a module is #included from Python.h, it is not going to be available to extension modules; either its implementation or its interface. - Josiah From jcarlson at uci.edu Mon Apr 23 19:23:23 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 23 Apr 2007 10:23:23 -0700 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: References: Message-ID: <20070423101749.63CB.JCARLSON@uci.edu> "Neville Grech Neville Grech" wrote: > I like the implementation of the BList, however I'm not so sure whether > replacing a list with BList is a good idea since there are many instances > where python scripts will iterate through endless lists of items. In such > case time complexity (to iterate a whole list) as you have mentioned is O > (nlog(n)) instead of O(n). You will probably find that such cases (iterating > through long lists) are very common and at the same time the developer still > wants his lists to be mutable. If a developer wants to optimise his lists > for insertion and deletion speed he can then use an optimised implementation > like the one you're proposing anyway. A naive implementation of iteration would be O(nlogn), with a minimal amount of work, it could be optimized to O(nlogn) worst case (mutations all the time everywhere) but O(n) when no mutations occur (the majority of iteration cases seen in the wild). > What's the performance of concatenation in these BLists? O(n) ? It would seem to be O(1), because one could trivially add a single new node that points to the copy-on-write root nodes of each input tree. At worst, it would take O(log n + log m) to combine the rightmost branch of the left addend and the leftmost branch of the right addend. > Are you suggesting BList as part of the standard library? He is, in fact, suggesting it as a replacement for list(). - Josiah From jcarlson at uci.edu Mon Apr 23 19:45:26 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 23 Apr 2007 10:45:26 -0700 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: References: Message-ID: <20070423102411.63CE.JCARLSON@uci.edu> "Daniel Stutzbach" wrote: > I have two proposals for Python 3000 that I'll write PEP(s) for if > there's interest. The first proposal is, I think, a modest one. The > second proposal is related, but more radical. I fully expect the > second proposal to be rejected for that alone, especially since I am a > relatively an outsider to the Python developer community (though it > was great to meet some of you at PyCon this year). I bring up the > more radical proposal primarily for completeness. [snip] > I recognize that the Python developer community is understandably very > attached to the current array-based list, so I expect this to get shot > down. I hope this doesn't reflect badly on the more modest proposal > of including a new type in the collections module. Also, please don't > kill the messenger. :-) I would be +1 on including the object in the collections module in 2.6 and 3.0 . I've implemented variants of this particular structure using binary treaps a few different times. In terms of 3rd party extensions that rely on the interface of a list, one could probably use the extended buffer protocol being discussed, but it really isn't nearly as simple as just traversing the array of PyObject* that a list exposes. -1 on replacing lists (generally) with your tree implementation. Base types should be as simple as possible. - Josiah From pje at telecommunity.com Mon Apr 23 19:54:53 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 23 Apr 2007 13:54:53 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <2f2e5f950704230937x4b8116cbmd3cd95ebfd96c924@mail.gmail.co m> References: <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070423134904.045c1e10@sparrow.telecommunity.com> At 01:37 PM 4/23/2007 -0300, Leonardo Santagada wrote: >i've thought that the overloading in GF was more for rules like (bar > >100) and not for diferent types. will both be suported or only the example >that you gave? The idea is that it should be possible for third-party implementations to add support for more sophisticated rules. The base system would support type annotations on arguments only -- and *maybe* interfaces. This would be done by implementing the generic function implementation using generic functions -- something I've already done in PEAK-Rules. Basically, the dispatch machinery is decoupled from the mechanism for specifying methods, and if you implement a new dispatcher (e.g. one supporting rules like "bar>100"), you simply also register methods with the overloading module's generic functions to support your new dispatcher. That way, people can use your "rule" objects with @when, @around, @before, @after, etc., without having to learn anything but how to create and use other kinds of rules. >ps: I didn't respond to python-3000 because I was afraid of disturbing >serious discussions over there. This is as serious a discussion as any; if you thought of the question, chances are good that somebody else had this question also. Thus, I'm cc:ing this back to the list, before any of them decide to ask me, too. :) From agthorr at barsoom.org Mon Apr 23 19:53:34 2007 From: agthorr at barsoom.org (Daniel Stutzbach) Date: Mon, 23 Apr 2007 12:53:34 -0500 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: References: Message-ID: On 4/23/07, Neville Grech Neville Grech wrote: > I like the implementation of the BList, however I'm not so sure whether > replacing a list with BList is a good idea since there are many instances > where python scripts will iterate through endless lists of items. In such > case time complexity (to iterate a whole list) as you have mentioned is O > (nlog(n)) instead of O(n). Iteration using an iterator object is O(n). For example, see the left-hand graph at: http://stutzbachenterprises.com/blist/forloop.html Of course, the following idiom is still O(n log n): for i in range(len(x)): do_something(x[i]) (but the base of the logarithm is large, so for n < 100 or so, ceil(log n) = 1) > You will probably find that such cases (iterating > through long lists) are very common and at the same time the developer still > wants his lists to be mutable. If a developer wants to optimise his lists > for insertion and deletion speed he can then use an optimised implementation > like the one you're proposing anyway. > > What's the performance of concatenation in these BLists? O(n) ? Concatenating a BList to a BList is O(log n + log m) where n and m are the size of the two BLists. Concatenating any other iterable to a BList is (log n + m) where n is the size of the BList and m is the length of the iterable. > Are you suggesting BList as part of the standard library? I'm making two mutually exclusive but related suggestions: 1) Add BList to the standard library as part of the collections module (which I hope will be accepted), or 2) Replace list() with BList (which I expect to be rejected) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From daniel at stutzbachenterprises.com Mon Apr 23 19:58:20 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 23 Apr 2007 12:58:20 -0500 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: <20070423101315.63C8.JCARLSON@uci.edu> References: <2410590390749666520@unknownmsgid> <20070423101315.63C8.JCARLSON@uci.edu> Message-ID: On 4/23/07, Josiah Carlson wrote: > "Daniel Stutzbach" wrote: > > On 4/23/07, Bill Janssen wrote: > > > > Replace list() with the BList. > > > > > > Generally, I like the idea. But how about extension modules > > > which make use of the current implementation details? > > > > PyListObject could be moved to the collections module so that C > > extension modules can continue to use it. > > Unless the implementation of a module is #included from Python.h, it is > not going to be available to extension modules; either its > implementation or its interface. So, I was unclear. I was picturing PyListObject still being built-in to the interpreter and exposed via Python.h (in C). That way, extension modules could still use it. Within the Python language, it could be made available via collections.list. Or it could continue to be __buitins__.list. By "replacing list", I'm mostly concerned about the return type of things like "[1,2,3,4]" and list comprehensions. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From guido at python.org Mon Apr 23 20:38:07 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Apr 2007 11:38:07 -0700 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: References: Message-ID: On 4/23/07, Daniel Stutzbach wrote: > I have two proposals for Python 3000 that I'll write PEP(s) for if > there's interest. The first proposal is, I think, a modest one. The > second proposal is related, but more radical. I fully expect the > second proposal to be rejected for that alone, especially since I am a > relatively an outsider to the Python developer community (though it > was great to meet some of you at PyCon this year). I bring up the > more radical proposal primarily for completeness. > > The Modest Proposal > ------------------------------- > > As a pet project, I've written a container type that looks, acts, and > quacks like a Python list(), but has better asymptotic performance. > Specifically, O(log n) time for inserts and deletes in the middle of > the list. I'd like to offer it for inclusion in the collections > module. [...] > The Radical Proposal > ------------------------------- > > Replace list() with the BList. Some meta-comments: (1) You really need Raymond Hettinger's review. Nothing else matters much. (2) I'm against proliferation of data types, because it makes the choices for developers harder. For that reason, I'd be against the modest proposal without the radical proposal. But I'd be in favor of the radical proposal if the pragmatics (mostly of existing core code and 3rd party extensions using the PyList_* APIs) can be solved, and if the performance is uniformly better. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From mike.klaas at gmail.com Mon Apr 23 20:42:02 2007 From: mike.klaas at gmail.com (Mike Klaas) Date: Mon, 23 Apr 2007 11:42:02 -0700 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: References: Message-ID: <3d2ce8cb0704231142n4448fd0apb3eb7f3dd350b5a3@mail.gmail.com> On 4/23/07, Daniel Stutzbach wrote: > So you can see the performance of the BList in more detail, I've made > several performance graphs available at the following link: > http://stutzbachenterprises.com/blist/ Very cool. These detailed timings are very useful. Might it be possible to include variance bars? A few of the graphs show your data structure faster than the list even for small N, and it would be great to see if it is meaningfully faster in these cases (if so, perhaps there are optimization possibilities for the current list data structure). A quite glance at your results shows a few more cases where the blist faces problems, such no longer using Timsort, and so suffering in performance for sorted/reversed cases. That would be painful to give up in the built-in list implementation. -Mike From guido at python.org Mon Apr 23 20:43:12 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Apr 2007 11:43:12 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> Message-ID: On 4/23/07, Phillip J. Eby wrote: > from overloading import overload, abstract > > def foo(bar, baz): > """Default implementation here""" > > > @overload > def foo(bar:int, baz:str): > """Overloaded implementation for int/str here""" > > > @abstract > def spam(fizz): > """This function has no default implementation, and raises > a "no applicable methods" error if called...""" (a) What's the point of having a separate keyword for this, as opposed to just raising an exception from the body? (b) I'm proposing an @abstractmethod, which sets its argument's __isabstractmethod__ attribute and then returns it, otherwise unchanged. Would this be good enough for you, or do you need @abstract to do more? My point is, I could rename mine to @abstract so that it would serve your purpose as well -- but only if it would serve. > @overload > def spam(fizz:list): > """...unless an appropriate overload is added first""" > > Any questions? :) I'd like the examples even better if they used ABCs from PEP 3119. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nevillegrech at gmail.com Mon Apr 23 20:58:45 2007 From: nevillegrech at gmail.com (Neville Grech Neville Grech) Date: Mon, 23 Apr 2007 20:58:45 +0200 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: References: Message-ID: You're right (I was seeing the statistics). Seems that it will outperform the built in list in most cases then. I would be +1 for both proposals if there are no underlying integration issues. Great work. On 4/23/07, Daniel Stutzbach wrote: > > On 4/23/07, Neville Grech Neville Grech wrote: > > I like the implementation of the BList, however I'm not so sure whether > > replacing a list with BList is a good idea since there are many > instances > > where python scripts will iterate through endless lists of items. In > such > > case time complexity (to iterate a whole list) as you have mentioned is > O > > (nlog(n)) instead of O(n). > > Iteration using an iterator object is O(n). For example, see the > left-hand graph at: > http://stutzbachenterprises.com/blist/forloop.html > > Of course, the following idiom is still O(n log n): > > for i in range(len(x)): > do_something(x[i]) > > (but the base of the logarithm is large, so for n < 100 or so, ceil(log n) > = 1) > > > You will probably find that such cases (iterating > > through long lists) are very common and at the same time the developer > still > > wants his lists to be mutable. If a developer wants to optimise his > lists > > for insertion and deletion speed he can then use an optimised > implementation > > like the one you're proposing anyway. > > > > What's the performance of concatenation in these BLists? O(n) ? > > Concatenating a BList to a BList is O(log n + log m) where n and m are > the size of the two BLists. > > Concatenating any other iterable to a BList is (log n + m) where n is > the size of the BList and m is the length of the iterable. > > > Are you suggesting BList as part of the standard library? > > I'm making two mutually exclusive but related suggestions: > > 1) Add BList to the standard library as part of the collections module > (which I hope will be accepted), or > 2) Replace list() with BList (which I expect to be rejected) > > -- > Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070423/2a4efde8/attachment.html From brett at python.org Mon Apr 23 21:05:24 2007 From: brett at python.org (Brett Cannon) Date: Mon, 23 Apr 2007 12:05:24 -0700 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: Message-ID: On 4/22/07, Steven Bethard wrote: > On 4/22/07, Brett Cannon wrote: > > Implementation > > ============== > > > > When the ``-m`` option is used, ``sys.main`` will be set to the > > argument passed in. ``sys.argv`` will be adjusted as it is currently. > > Then the equivalent of ``__import__(self.main)`` will occur. This > > differs from current semantics as the ``runpy`` module fetches the > > code object for the file specified by the module name in order to > > explicitly set ``__name__`` and other attributes. This is no longer > > needed as import can perform its normal operation in this situation. > > > > If a file name is specified, then ``sys.main`` will be set to > > ``"__main__"``. The specified file will then be read and have a code > > object created and then be executed with ``__name__`` set to > > ``"__main__"``. This mirrors current semantics. > > So __name__ will still sometimes be "__main__". That's disappointing. > Yeah, I know. But I can't force people to only run Python scripts that happen to be on sys.path. > To clarify, assuming that the foo.bar module contains something like > "from . import baz", this PEP only addresses the following situation:: > > python -mfoo.bar > > and all of the following will still raise ImportErrors:: > > ~> python foo/bar > ~/grok> python ../foo/bar > ~/foo> python bar > > Right? It could handle the last one no problem. The foo/bar it could if it was deemed worth it to infer the module name. The middle one *might* be solvable with some checking on sys.path and walking the directory. In other words they are possible if we want to pay the startup cost of trying to infer the module's name. > > If that's right, I'm -1 on the proposal. It's complicating the > standard "am I the main module?" idiom to solve a tiny little problem > with "-m". It's not a '-m' problem, it's a relative import problem. This issue occurs no matter what solution you go with, even with the specially named main() function idea some people are tossing around. > The "am I the main module?" idiom has been around much > longer than the "-m" flag, and I'd prefer to see a more compelling > reason to change it. > When you start using relative imports exclusively and find you have to tweak your imports to execute certain modules you might change your mind. =) > Two reasons that would be compelling for me: > > * Simplifying the "am I the main module?" idiom, e.g. with the > rejected ``if __main__`` proposal. > > * Getting rid of the "__main__" value for __name__ entirely. This > would require code like the following to determine the name of a > module given at the command line:: > > def get_name(path): > sys_path_set = set(sys.path) > path, _ = os.path.splitext(path) > path = os.path.abspath(path) > parts = path.split(os.path.sep) > for i in range(1, len(parts)): > sub_path = os.path.sep.join(parts[:i]) > if sub_path in sys_path_set: > return '.'.join(parts[i:]) > return parts[-1] > > Even if we can't do the name resolution perfectly, if the flaws are > documented (e.g. when '..' and symbolic links are combined) I think > that would be enough better than the current situation to merit > changing the standard "am I the current module?" idiom. > As I said in the PEP, I rejected this because of startup cost. If people are willing to pay for it then we can go with it and just assume that if the current directory is not a prefix for the module's path that we just use the filename to set the module's name and sys.main, that's fine as well. It just is not as accurate as it might be. So the questions for people to weigh in on are: * For spam/bacon.py w/ with spam/__init__.py existing: - Should we infer it as spam.bacon? Startup cost, but accurate name allows for relative imports. - Should we infer as bacon? Cheap but breaks relative imports. - Should we infer the name as __main__? Status quo and breaks relative imports. * For spam/bacon.py with *no* spam/__init__.py: - Should we infer the name as bacon? No relative imports possible since not in a package. - Should we infer the name as __main__? Status quo and no relative import issue as not in a package. * For ../bacon.py - With ../__init__.py defined and ../../ on sys.path: + Should we infer the name as bacon? Simple, but breaks relative imports. + Should we walk up the directories until __init__.py is not found and check for that directory on sys.path? Costly startup, but allows for relative imports to work. + Should we infer the name as __main__? Status quo and relative imports don't work. - Without ../__init__.py + Infer as bacon? No relative imports possible so fine. + Infer as __main__? Status quo, and no relative imports even possible. If people want a simple inference rule we can ditch __main__ entirely and just go with a simple rule that would be more costly for startup. Otherwise we can stick with __main__ as the PEP has proposed and have a partial solution. Or we can ditch the PEP. -Brett From janssen at parc.com Mon Apr 23 21:19:23 2007 From: janssen at parc.com (Bill Janssen) Date: Mon, 23 Apr 2007 12:19:23 PDT Subject: [Python-3000] Principles (was: Two proposals for a new list-like type: one modest, one radical) In-Reply-To: <20070423102411.63CE.JCARLSON@uci.edu> References: <20070423102411.63CE.JCARLSON@uci.edu> Message-ID: <07Apr23.121924pdt."57996"@synergy1.parc.xerox.com> > Base types should be as simple as possible. Is that really a good idea? Wouldn't we wind up with BLISS (unsigned integers)? I think it's always desirable to provide more useful base types, but that it has to be balanced against conversion and maintenance costs. Bill From pje at telecommunity.com Mon Apr 23 21:28:14 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 23 Apr 2007 15:28:14 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> At 11:43 AM 4/23/2007 -0700, Guido van Rossum wrote: >On 4/23/07, Phillip J. Eby wrote: > > from overloading import overload, abstract > > > > def foo(bar, baz): > > """Default implementation here""" > > > > > > @overload > > def foo(bar:int, baz:str): > > """Overloaded implementation for int/str here""" > > > > > > @abstract > > def spam(fizz): > > """This function has no default implementation, and raises > > a "no applicable methods" error if called...""" > >(a) What's the point of having a separate keyword for this, as opposed >to just raising an exception from the body? EIBTI, mainly. Specifically, it's clear from the beginning that you're looking at an empty function -- i.e., one that *can't* be called successfully except for explicitly registered patterns -- versus one that has a truly "generic" base implementation. For example, a "pprint" generic function whose implementation falls back to repr() is a bit different than say, a "get_stream()" function that can only be called on path objects, stringIOs, files, and sockets. The latter has no "generic" implementation. Using the decorator also makes it easier to be consistent about what error is raised -- @abstract is easier for me to remember than which exception to raise. :) I believe there was also another, somewhat more esoteric reason that gets involved when you build more sophisticated rulesystems on top of the base machinery, but the exact reason is escaping me at this moment. I remember there being something that bothered me about your original Py3K overloading prototype having to always use the original function body as the default condition, but I don't remember specifically what, except that it was sort of like doing math without being able to write "zero": you can work around it, but it's kind of weird. :) >(b) I'm proposing an @abstractmethod, which sets its argument's >__isabstractmethod__ attribute and then returns it, otherwise >unchanged. Would this be good enough for you, or do you need @abstract >to do more? My point is, I could rename mine to @abstract so that it >would serve your purpose as well -- but only if it would serve. Assuming that: 1. If you call such a function, it will raise some error, like NotImplementedError 2. It's a normal function object (i.e., can have its func_code repointed later) 3. The __isabstractmethod__ can be set back to False once there is other code registered Then yes, it's perfect. > > @overload > > def spam(fizz:list): > > """...unless an appropriate overload is added first""" > > > > Any questions? :) > >I'd like the examples even better if they used ABCs from PEP 3119. :-) I'll keep that in mind for the actual PEP. :) From steven.bethard at gmail.com Mon Apr 23 21:49:01 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 23 Apr 2007 13:49:01 -0600 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: Message-ID: On 4/23/07, Brett Cannon wrote: > On 4/22/07, Steven Bethard wrote: > > On 4/22/07, Brett Cannon wrote: > > > Implementation > > > ============== > > > > > > When the ``-m`` option is used, ``sys.main`` will be set to the > > > argument passed in. ``sys.argv`` will be adjusted as it is currently. > > > Then the equivalent of ``__import__(self.main)`` will occur. This > > > differs from current semantics as the ``runpy`` module fetches the > > > code object for the file specified by the module name in order to > > > explicitly set ``__name__`` and other attributes. This is no longer > > > needed as import can perform its normal operation in this situation. > > > > > > If a file name is specified, then ``sys.main`` will be set to > > > ``"__main__"``. The specified file will then be read and have a code > > > object created and then be executed with ``__name__`` set to > > > ``"__main__"``. This mirrors current semantics. [snip] > > If that's right, I'm -1 on the proposal. It's complicating the > > standard "am I the main module?" idiom to solve a tiny little problem > > with "-m". > > It's not a '-m' problem, it's a relative import problem. Right, but the PEP was only solving the relative import problem for '-m'. > > Two reasons that would be compelling for me: > > > > * Simplifying the "am I the main module?" idiom, e.g. with the > > rejected ``if __main__`` proposal. > > > > * Getting rid of the "__main__" value for __name__ entirely. This > > would require code like the following to determine the name of a > > module given at the command line:: [snip] > > Even if we can't do the name resolution perfectly, if the flaws are > > documented (e.g. when '..' and symbolic links are combined) I think > > that would be enough better than the current situation to merit > > changing the standard "am I the current module?" idiom. > > As I said in the PEP, I rejected this because of startup cost. If > people are willing to pay for it then we can go with it I'm personally willing to pay for it because it's only going to apply at most once each time ``python`` is started. > So the questions for people to weigh in on are: I'm basically +1 on all the versions that do the full lookup. That means scanning up the parent directories until we encounter either a directory with no __init__.py or a directory on sys.path. > * For spam/bacon.py w/ with spam/__init__.py existing: Infer 'spam.bacon': * parent dir of 'bacon.py', 'spam', has an '__init__.py', so 'spam.bacon' * parent dir of 'spam' is on sys.path (it's the current directory), so stop > * For spam/bacon.py with *no* spam/__init__.py: Infer 'bacon': * parent dir of 'bacon.py' has no '__init__.py', so stop > * For ../bacon.py > > - With ../__init__.py defined and ../../ on sys.path: Infer 'spam.bacon' (assuming parent directory is 'spam'): * parent dir of 'bacon.py', 'spam', has an '__init__.py', so 'spam.bacon' * parent dir of 'spam' is on sys.path, so stop > - Without ../__init__.py Infer 'bacon' * parent dir of 'bacon.py' has no '__init__.py', so stop. The above were pretty easy. Here's a harder one: foo/ __init__.py bar/ __init__.py baz/ __init__.py frobble.py where the 'foo' directory is on sys.path. Should this infer 'bar.baz.frobble' or 'foo.bar.baz.frobble'? The former assumes that sys.path takes precedence over the presence of an '__init__.py'. The latter assumes the reverse. (I lean towards the fomer FWIW, though it might be worth giving the person who organized their code this way some sort of bodily threat.) ;-) > If people want a simple inference rule we can ditch __main__ entirely > and just go with a simple rule that would be more costly for startup. This is my preference, but I guess I'd like to know how big the startup costs would be. Clearly if Python takes twice as long to start, it's not worth it. ;-) STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From brett at python.org Mon Apr 23 22:00:18 2007 From: brett at python.org (Brett Cannon) Date: Mon, 23 Apr 2007 13:00:18 -0700 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: <462C7CFB.5060607@gmail.com> References: <462C7CFB.5060607@gmail.com> Message-ID: On 4/23/07, Nick Coghlan wrote: > Brett Cannon wrote: > > This PEP is to change the ``if __name__ == "__main__": ...`` idiom to > > ``if __name__ == sys.main: ...`` so that you at least have a chance > > to execute module in a package that use relative imports. > > > > Ran this PEP past python-ideas. Stopped the discussion there when too > > many new ideas were being proposed. =) I have listed all of them in > > the Rejected Ideas section, although if overwhelming support for one > > comes forward the PEP can shift to one of them. > > -1 > > I still prefer the "__module_name__" fallback that was actually briefly > in the tree for 2.5 (the change was made really late in the 2.5 cycle, > so it was backed out with the intention of revisiting the idea for 2.6 - > check the svn log for lib/runpy.py). > > The trick with that approach was that runpy.run_module() would *always* > set "__module_name__" to the real name of the module being executed, > regardless of what it was told to set "__name__" to. > > Then the import machinery was modified such that if it came across an > explicit relative import being made from a module called "__main__", it > would check to see if "__module_name__" was set, and if so, use that > instead. > I don't like having to tweak the import machinery for this. It feels like working around a problem instead of trying to solve it. Plus it doesn't address Steve's concerns of when modules are still named '__main__' because there absolute name cannot be figured out. -Brett From brett at python.org Mon Apr 23 22:01:55 2007 From: brett at python.org (Brett Cannon) Date: Mon, 23 Apr 2007 13:01:55 -0700 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: Message-ID: On 4/23/07, Christian Heimes wrote: > Brett Cannon wrote: > >> Someone on python-idea brought up the idea to add the sys module to > >> builtins. The feature would remove the requirement for an extra line and > >> the costs should be minimal or even zero since sys is a special builtin > >> module. > >> > > > > That's a separate PEP. > > Do you remember the PEP number? I can't find the PEP. > 299. > >> Some people including me like to resurrect the PEP 299 __main__() > >> function idea. http://www.python.org/dev/peps/pep-0299/ > > > > As I have already said multiple times on python-ideas, I don't > > personally like that idea. If someone wants to do the work to try to > > push that view then they can. > > I know. I mentioned it because I "wanted to push that view". O:-) That's fine, but this view in no way addresses the problem I am tackling; how to allow a module to be executed that has a relative import. With the change you want you don't even address the issue I am trying to tackle. -Brett From jcarlson at uci.edu Mon Apr 23 22:13:23 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 23 Apr 2007 13:13:23 -0700 Subject: [Python-3000] Principles (was: Two proposals for a new list-like type: one modest, one radical) In-Reply-To: <07Apr23.121924pdt."57996"@synergy1.parc.xerox.com> References: <20070423102411.63CE.JCARLSON@uci.edu> <07Apr23.121924pdt."57996"@synergy1.parc.xerox.com> Message-ID: <20070423123456.63D7.JCARLSON@uci.edu> Bill Janssen wrote: > > Base types should be as simple as possible. > Is that really a good idea? Wouldn't we wind up with BLISS (unsigned > integers)? With everything, there is a balance. I do not advocate *removing* functionality to make an object simpler; that would be absurd. On the other hand, when we already have types that are necessarily as simple as possible to implement a reasonable set of functionality (lists, tuples, integers, floats, dictionaries, str, etc.), I think that's a good starting point. Now, do we replace a structure which is very easy to understand and maintain with a structure that necessarily isn't as easy to understand and maintain? What if it breaks 3rd party code that expects the old implementation? At what point do we say that certain performance improvements are or are not significant enough to balance API breakage, performance assumptions, etc.? I believe that no performance improvement warrants such large scale breakage of 3rd party extensions. This is the same reason why I didn't support the string view+concatenation mechanism that was offered by Larry Hastings. > I think it's always desirable to provide more useful base > types, but that it has to be balanced against conversion and > maintenance costs. Indeed. As a base type, I don't believe a BTree implementation replacing the array implementation of a list is a good thing. Guido seems to be against the addition of yet another type into collections (which has defaultdict and deque in 2.5). Me, I'd be happy to have this object in collections (as I said, I've implemented this kind of thing before). Depending on my mood, I may even support replacing the deque implementation with the Daniel's BTree, as the timings that Daniel has provided show that inserting and deleting is quite fast for moderately sized sequences (10k elements). When people ask, "I want a list-like thing that is able to do X", we can tell them, "use a deque, it's really fast for just about everything". Later when people invariably ask, "if it's fast for most everything, why doesn't it replace list?" we can tell them, "it breaks too many 3rd party extensions that rely on the array-based implementation of list." - Josiah From jcarlson at uci.edu Mon Apr 23 22:20:46 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 23 Apr 2007 13:20:46 -0700 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: Message-ID: <20070423122833.63D4.JCARLSON@uci.edu> "Brett Cannon" wrote: [snip] > So the questions for people to weigh in on are: [snip] > * For ../bacon.py [snip] > + Should we walk up the directories until __init__.py is not > found and check for that directory on sys.path? > Costly startup, but allows for relative imports to work. +.99, but should be applied regardless of ../bacon.py, bacon.py, or spam/bacon.py . Generally, I believe that we should advance up the file system/whatever paths until we run out of __init__.py . That would give every package, module, etc., a unique name that is the same regardless of whether it is run as bacon.py, ../bacon.py, or spam/bacon.py . I also realize that this idea may not be popular. > If people want a simple inference rule we can ditch __main__ entirely > and just go with a simple rule that would be more costly for startup. Ditch __main__, at least check for path/__init__.py when path/bacon.py is run. Whether to advance up the parent paths, and/or how deep to advance up the parent paths I don't know. I know what I want, but I understand that it may not be what others want (especially when confronted with NFS, database imports, etc.). - Josiah From jimjjewett at gmail.com Mon Apr 23 22:17:35 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 23 Apr 2007 16:17:35 -0400 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: Message-ID: On 4/23/07, Brett Cannon wrote: > On 4/22/07, Steven Bethard wrote: > In other words they are possible if we want to pay the startup cost of > trying to infer the module's name. This doesn't need to be a startup cost for python; it can be delayed until an actual relative import is requested. (Though I suppose that might mean making __name__ a lazy attribute, which itself seems like overkill.) Call me cynical, but the problem seems to crop up only when *all* of the following are true: (a) Your helper modules are not on sys.path. (and you won't add them -- or at the very least, you want to ensure using a local copy instead of one that might be earlier on sys.path.) (b) You helper modules are not in the same directory as your main code, or even a subdirectory of it. (Maybe someone else is supplying a black-box API, but you can't tell them where to install/symlink it. Maybe your code is so complex that nothing is at the top level.) (c) You insist on running as __main__, instead of getting called as a library by a better-placed wrapper script. (Because doing it that way is more convenient, for a quick-and-dirty version? This conflicts with (b).) (d) You have either a *lot* of imports or a very slow filesystem, and can't afford much slop. This also conflicts with (c). I won't say this use case doesn't matter, but I'm not sure it is worth any extra complexity in the import rules. I *am* convinced that people needing that much complexity should be the only ones to pay the extra startup cost -- but extending it to all uses of -m might be OK. The choice for those people who actually use it is really (1) Do it right, despite the cost or (2) Accept that you don't really need such a comprehensive install/config framework, and simplify your code. > * For spam/bacon.py w/ with spam/__init__.py existing: > - Should we infer it as spam.bacon? > Startup cost, but accurate name allows for relative imports. uh ... how much cost? To me it looks like one extra file check -- less than the cost of having more than one directory in sys.path. > - Should we infer as bacon? > Cheap but breaks relative imports. > - Should we infer the name as __main__? > Status quo and breaks relative imports. But still probably good enough most of the time ... just put "." on sys.path. If you can't do that, you're in the special-case zone. > * For spam/bacon.py with *no* spam/__init__.py: > - Should we infer the name as bacon? > No relative imports possible since not in a package. > - Should we infer the name as __main__? > Status quo and no relative import issue as not in a package. So does it matter which you choose? Would it really be all that awful even if you did allow relative imports based strictly on the directory structure? > * For ../bacon.py > > - With ../__init__.py defined and ../../ on sys.path: So you can't change directory before launching, *and* you are launching with a relative filename instead of an absolute ... which suggests quick-hack. You may not need the fancy inferences, nor should you object to the price. > + Should we infer the name as bacon? > Simple, but breaks relative imports. > + Should we walk up the directories until __init__.py is not > found and check for that directory on sys.path? > Costly startup, but allows for relative imports to work. So if you care about startup costs, launch from the right directory... -jJ From jimjjewett at gmail.com Mon Apr 23 22:52:57 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 23 Apr 2007 16:52:57 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> Message-ID: On 4/23/07, Phillip J. Eby wrote: > At 11:43 AM 4/23/2007 -0700, Guido van Rossum wrote: > >On 4/23/07, Phillip J. Eby wrote: > > > @abstract > > > def spam(fizz): > > > """This function has no default implementation, and raises > > > a "no applicable methods" error if called...""" > >(a) What's the point of having a separate keyword for this, as > >opposed to just raising an exception from the body? > EIBTI, mainly. Specifically, it's clear from the beginning that you're > looking at an empty function -- i.e., one that *can't* be called > successfully except for explicitly registered patterns -- versus one that > has a truly "generic" base implementation. erm... not to me. To me it looks like any short function until I look carefully. Once I do notice that the function body is only a string, I get a bad reaction, until I remember that is is really just a strange abbreviation for "pass". > I believe there was also another, somewhat more esoteric > reason that gets involved when you build more sophisticated > rulesystems on top of the base machinery, but the exact reason > is escaping me at this moment. ... sort > of like doing math without being able to write "zero": I think it had to do with next_method, and was the same problem super has. If you aren't sure the job is done -- no matter what you got mixed with -- then you have to call the next method. 99% of the time, it won't do anything useful, but you have to call it just in case. My favorite example is __del__(), or at least close(). I can be sure that I've cleaned up *my* scarce resources, but I don't know what subclasses might have mixed me in with. So I call super __del__, and catch the Exception when it turns out I wasn't mixed in at all. A "pass" default implementation is much nicer to subclasses than an Exception that they have to try...except. > > My point is, I could rename mine to @abstract so that it > >would serve your purpose as well -- but only if it would serve. > Assuming that: > 1. If you call such a function, it will raise some error, like > NotImplementedError Note that your default methods don't do this either, except as part of the wrapper. Guido's current implementation is that (a) An error is raised as soon as you try to instantiate a class that *could* call the method directly, even if you don't plan to ever *actually* call it. (b) The method itself is annotated, so that the rules engine could raise an error if it so chose. > 2. It's a normal function object (i.e., can have its func_code > repointed later) > 3. The __isabstractmethod__ can be set back to False once there > is other code registered I'm not sure that actually *doing* this would play well with ABC assumptions, but, yes, it is possible. > Then yes, it's perfect. -jJ From michele.cella at gmail.com Mon Apr 23 23:08:03 2007 From: michele.cella at gmail.com (Michele Cella) Date: Mon, 23 Apr 2007 23:08:03 +0200 Subject: [Python-3000] Fixing super anyone? In-Reply-To: <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> <462BDCAA.7020501@canterbury.ac.nz> <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> Message-ID: Thomas Wouters wrote: > > But how would it *work*? I assume you want 'super' to take a single > expression, and be an expression itself. How does it know which instance > you want to 'super'? Does it specialcase the variable named 'self'? > Threat 'everything up to the first dot' as 'the instance'? Specialcase > the first argument to the method? (That's what ' super.foo()' would do.) > How would you do things like: > > # call the supermethod 'register' with the nonsuper 'callback': > super.register(self.callback) > # call the nonsuper 'register' with the supermethod 'callback' > self.register(super.callback) > # call the 'frominstance' method of the 'innerclass' attribute on the > superclass > super.innerclass.frominstance(self) > Hi Thomas, Yes, you raise good points. My solution is nice (IMHO) for the simplest use case (and, probably, the most frequent) but it's behavior is really unclear for these complex use cases. Thanks for pointing that out. As far as I'm concerned, I can only confirm that having a super keyword with methods attached (like Aahz said) when no other keyword has (and there are many) doesn't feel right (to me) and can be confusing at first. That's unfortunate as I rarely found python confusing up to now (contrary to other languages). The only other option that comes to my mind is using a special attribute in the instance itself (like __class__, __dict__, ...): self.__super__.mymethod(arg) I'm sure this (again) introduces other problems I simply overlooked, in this case, forgive me please. ;-) Thanks again for your attention. Ciao Michele From ncoghlan at gmail.com Mon Apr 23 23:22:00 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 24 Apr 2007 07:22:00 +1000 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: <462C7CFB.5060607@gmail.com> Message-ID: <462D2378.3020006@gmail.com> Brett Cannon wrote: > On 4/23/07, Nick Coghlan wrote: >> Then the import machinery was modified such that if it came across an >> explicit relative import being made from a module called "__main__", it >> would check to see if "__module_name__" was set, and if so, use that >> instead. >> > > I don't like having to tweak the import machinery for this. It feels > like working around a problem instead of trying to solve it. As you yourself said, it's a relative import problem, more so than a main module execution problem. I think addressing it on the import side (with some cooperation from the main module execution code) reflects that. The other nice thing about this solution, is that you can explicitly enable main relative imports without using "-m", simply by setting __module_name__ appropriately (an alternative would be to call the additional attribute __package_name__, to save a small amount of string fiddling in the relative import code) > Plus it > doesn't address Steve's concerns of when modules are still named > '__main__' because there absolute name cannot be figured out. That problem is never going to go away, IMO - if you run a file directly, Python has no clue where in the package hierarchy you intend it to live. We provide a mechanism (the -m switch) that allows people to be explicit if they want to - other than that, I say we resist the temptation to guess (take a look at PEP 302 to get some idea of just how wildly wrong attempts to guess a package name from a file name may be). Now, when it comes to use cases for executing packages inside a module: ./python -m test.regrtest (I use this one all the time) python -m pychecker.checker (I should use this more than I do...) Test modules embedded inside a package [1] I forget who first described the use case in [1] to me, but it was instrumental in convincing me the current behaviour was a bug that should be fixed. SF Bug #1510172 is currently sitting open, pending a resolution of this question (i.e. am I correct in thinking that this is a bug that should be fixed, or are we better off keeping main module behaviour consistent between direct execution and the -m switch?). Cheers, Nick. [1] http://mail.python.org/pipermail/python-dev/2006-June/066680.html -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From pje at telecommunity.com Mon Apr 23 23:25:04 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 23 Apr 2007 17:25:04 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070423170905.02ebd3d0@sparrow.telecommunity.com> At 04:52 PM 4/23/2007 -0400, Jim Jewett wrote: >On 4/23/07, Phillip J. Eby wrote: >>At 11:43 AM 4/23/2007 -0700, Guido van Rossum wrote: >> >On 4/23/07, Phillip J. Eby wrote: > >> > > @abstract >> > > def spam(fizz): >> > > """This function has no default implementation, and raises >> > > a "no applicable methods" error if called...""" > >> >(a) What's the point of having a separate keyword for this, as >> >opposed to just raising an exception from the body? > >>EIBTI, mainly. Specifically, it's clear from the beginning that you're >>looking at an empty function -- i.e., one that *can't* be called >>successfully except for explicitly registered patterns -- versus one that >>has a truly "generic" base implementation. > >erm... not to me. To me it looks like any short function until I look >carefully. Which is precisely why @abstract is useful. Are you saying you *don't* want @abstract explicitly indicating that the function has no behavior? >>I believe there was also another, somewhat more esoteric >>reason that gets involved when you build more sophisticated >>rulesystems on top of the base machinery, but the exact reason >>is escaping me at this moment. ... sort >>of like doing math without being able to write "zero": > >I think it had to do with next_method, and was the same problem super >has. If you aren't sure the job is done -- no matter what you got >mixed with -- then you have to call the next method. 99% of the time, >it won't do anything useful, but you have to call it just in case. > >My favorite example is __del__(), or at least close(). I can be sure >that I've cleaned up *my* scarce resources, but I don't know what >subclasses might have mixed me in with. So I call super __del__, and >catch the Exception when it turns out I wasn't mixed in at all. > >A "pass" default implementation is much nicer to subclasses than an >Exception that they have to try...except. Right. If you're defining a generic function as a hook, you would just define the function as a no-op -- i.e. NOT an @abstract function. (e.g., the "foo()" function in my previous post.) >> > My point is, I could rename mine to @abstract so that it >> >would serve your purpose as well -- but only if it would serve. > >>Assuming that: > >>1. If you call such a function, it will raise some error, like >>NotImplementedError > >Note that your default methods don't do this either, except as part of >the wrapper. I don't understand what the sentence above means. I don't even understand it enough to misunderstand it. :) > Guido's current implementation is that > > (a) An error is raised as soon as you try to instantiate a class >that *could* call the method directly, even if you don't plan to ever >*actually* call it. > > (b) The method itself is annotated, so that the rules engine >could raise an error if it so chose. Ah. I'm suggesting that a function object marked as abstract should also raise an exception (perhaps NotImplementedError) when it's actually called. @abstract could either change the function object's type, or set its func_code, or whatever else makes sense. >>2. It's a normal function object (i.e., can have its func_code >>repointed later) >>3. The __isabstractmethod__ can be set back to False once there >>is other code registered > >I'm not sure that actually *doing* this would play well with ABC >assumptions, but, yes, it is possible. Since @overload requires a function of the same name to alread exist in the current namespace, you'll normally be doing this: class Concrete(Abstract): def my_implementation_of_something_abstract(self, ...): """Default implementation""" @overload def my_implementation_of_something_abstract(self, ...): """This overloads *this* class' definition, not the ABC's""" In any case, I don't really see any reason to add overloads to an ABC's abstract methods. The GF usage of @abstract is really only useful for standalone functions, at least as far as I can see. From pje at telecommunity.com Mon Apr 23 23:31:31 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 23 Apr 2007 17:31:31 -0400 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> <462BDCAA.7020501@canterbury.ac.nz> <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> Message-ID: <5.1.1.6.0.20070423172551.04c4e5f0@sparrow.telecommunity.com> At 11:08 PM 4/23/2007 +0200, Michele Cella wrote: >My solution is nice (IMHO) for the simplest use case (and, probably, the >most frequent) but it's behavior is really unclear for these complex use >cases. Thanks for pointing that out. > >As far as I'm concerned, I can only confirm that having a super keyword >with methods attached (like Aahz said) when no other keyword has (and >there are many) doesn't feel right (to me) and can be confusing at >first. That's unfortunate as I rarely found python confusing up to now >(contrary to other languages). > >The only other option that comes to my mind is using a special attribute >in the instance itself (like __class__, __dict__, ...): > > self.__super__.mymethod(arg) > >I'm sure this (again) introduces other problems I simply overlooked, in >this case, forgive me please. ;-) One other option that's been used more and more lately are parentheses, as in: (super self).foo() But I can't say I particularly like this idea, compared to "super.foo" or even "super(self).foo". In fact, the latter invocation doesn't even require a keyword -- it just means the compiler needs to include a cell variable for the current class whenever it thinks you might be using super(). From daniel at stutzbachenterprises.com Tue Apr 24 00:05:12 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 23 Apr 2007 17:05:12 -0500 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: <3d2ce8cb0704231142n4448fd0apb3eb7f3dd350b5a3@mail.gmail.com> References: <3d2ce8cb0704231142n4448fd0apb3eb7f3dd350b5a3@mail.gmail.com> Message-ID: On 4/23/07, Mike Klaas wrote: > On 4/23/07, Daniel Stutzbach wrote: > > So you can see the performance of the BList in more detail, I've made > > several performance graphs available at the following link: > > http://stutzbachenterprises.com/blist/ > > Very cool. These detailed timings are very useful. > > Might it be possible to include variance bars? I'm not really sure what variance bars would reveal. Performing exactly the same operations follows exactly the same code path, so there's not a lot of variance to be found. Let me explain my methodology for generating these figures. Right now I'm generating the timings using timeit.py, where the setup step is run once, then the actual operation is run enough times to exceed 10 milliseconds. I repeat the process 3 times and take the lowest value (to try to factor out noise introduced by other processes and other OS issues). The machine I run the tests on is generally unloaded. Once I had an initial working implementation, I've been using theses experiments and graphs to figure out where I needed to do optimization work. I can testify that when I rerun the experiments with the same code, the results would look *very* similar. > A few of the graphs > show your data structure faster than the list even for small N, and it > would be great to see if it is meaningfully faster in these cases (if > so, perhaps there are optimization possibilities for the current list > data structure). Yes, in some cases the optimizations I've made could be ported back to the current list data structure. I will try to hunt these down and send patches via SourceForge. > A quite glance at your results shows a few more cases where the blist > faces problems, such no longer using Timsort, and so suffering in > performance for sorted/reversed cases. That would be painful to give > up in the built-in list implementation. I plan on implementing Timsort for BLists; I just haven't had time yet. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From guido at python.org Tue Apr 24 00:05:38 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Apr 2007 15:05:38 -0700 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: Message-ID: On 4/22/07, Brett Cannon wrote: > This PEP is to change the ``if __name__ == "__main__": ...`` idiom to > ``if __name__ == sys.main: ...`` so that you at least have a chance > to execute module in a package that use relative imports. > > Ran this PEP past python-ideas. Stopped the discussion there when too > many new ideas were being proposed. =) I have listed all of them in > the Rejected Ideas section, although if overwhelming support for one > comes forward the PEP can shift to one of them. I'm -1 on this and on any other proposed twiddlings of the __main__ machinery. The only use case seems to be running scripts that happen to be living inside a module's directory, which I've always seen as an antipattern. To make me change my mind you'd have to convince me that it isn't. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Tue Apr 24 00:08:03 2007 From: brett at python.org (Brett Cannon) Date: Mon, 23 Apr 2007 15:08:03 -0700 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: Message-ID: On 4/23/07, Guido van Rossum wrote: > On 4/22/07, Brett Cannon wrote: > > This PEP is to change the ``if __name__ == "__main__": ...`` idiom to > > ``if __name__ == sys.main: ...`` so that you at least have a chance > > to execute module in a package that use relative imports. > > > > Ran this PEP past python-ideas. Stopped the discussion there when too > > many new ideas were being proposed. =) I have listed all of them in > > the Rejected Ideas section, although if overwhelming support for one > > comes forward the PEP can shift to one of them. > > I'm -1 on this and on any other proposed twiddlings of the __main__ > machinery. The only use case seems to be running scripts that happen > to be living inside a module's directory, which I've always seen as an > antipattern. To make me change my mind you'd have to convince me that > it isn't. With Guido not liking it and people not agreeing on a solution with me not feeling like fighting for this one I consider the PEP rejected. When I get my laptop back (end of the week hopefully) I will check the PEP in as rejected so that we don't have to go through this again. Thanks to everyone to contributed to the conversation. -Brett From mike.klaas at gmail.com Tue Apr 24 00:15:59 2007 From: mike.klaas at gmail.com (Mike Klaas) Date: Mon, 23 Apr 2007 15:15:59 -0700 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: References: <3d2ce8cb0704231142n4448fd0apb3eb7f3dd350b5a3@mail.gmail.com> Message-ID: <3d2ce8cb0704231515k3c75246eud35c5c5a3a2fb8f4@mail.gmail.com> On 4/23/07, Daniel Stutzbach wrote: > On 4/23/07, Mike Klaas wrote: > > Might it be possible to include variance bars? > > I'm not really sure what variance bars would reveal. Performing > exactly the same operations follows exactly the same code path, so > there's not a lot of variance to be found. Let me explain my > methodology for generating these figures. You;re right. I was interested in whether some of the jagged graphs were due to noise or underlying behaviour. On closer inspection it is clear that the jumps tend to happen near the same place on most ops (esp the first at ~128), which makes it clear that it is underlying behaviour. > > A few of the graphs > > show your data structure faster than the list even for small N, and it > > would be great to see if it is meaningfully faster in these cases (if > > so, perhaps there are optimization possibilities for the current list > > data structure). > > Yes, in some cases the optimizations I've made could be ported back to > the current list data structure. I will try to hunt these down and > send patches via SourceForge. The init-from iterable is especially interesting: a 20% speed increase for list comps is not to be scoffed at. Have you analyzed the memory consumption of the two data structures? -Mike From guido at python.org Tue Apr 24 00:16:02 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Apr 2007 15:16:02 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> Message-ID: [Guido] > >(b) I'm proposing an @abstractmethod, which sets its argument's > >__isabstractmethod__ attribute and then returns it, otherwise > >unchanged. Would this be good enough for you, or do you need @abstract > >to do more? My point is, I could rename mine to @abstract so that it > >would serve your purpose as well -- but only if it would serve. On 4/23/07, Phillip J. Eby wrote: > Assuming that: > > 1. If you call such a function, it will raise some error, like NotImplementedError That would be up to the author of the function; they would have to explicitly raise NotImplementedError in the body. The ABC proposal allows meaningful abstract methods that can be called (only) via "super"; the use case for this is that the abstract method might be the one that decides which exception should be thrown (e.g. __getitem__ and __next__ do this), or perhaps it could provide a default implementation for certain types. (Potential example of the latter: Sequence.__getitem__() could raise IndexError when the argument is an Integer but handle the case where the argument is a slice instance, assuming there's a suitable factory which could be a designated class method.) > 2. It's a normal function object (i.e., can have its func_code repointed later) Yes. > 3. The __isabstractmethod__ can be set back to False once there is other > code registered Yes. > Then yes, it's perfect. I guess it falls short. I'll stick with @abstractmethod, which rhymes with classmethod and staticmethod. You can have @abstract. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rhamph at gmail.com Tue Apr 24 00:21:43 2007 From: rhamph at gmail.com (Adam Olsen) Date: Mon, 23 Apr 2007 16:21:43 -0600 Subject: [Python-3000] Fixing super anyone? In-Reply-To: <5.1.1.6.0.20070423172551.04c4e5f0@sparrow.telecommunity.com> References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> <462BDCAA.7020501@canterbury.ac.nz> <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> <5.1.1.6.0.20070423172551.04c4e5f0@sparrow.telecommunity.com> Message-ID: On 4/23/07, Phillip J. Eby wrote: > But I can't say I particularly like this idea, compared to "super.foo" or > even "super(self).foo". In fact, the latter invocation doesn't even > require a keyword -- it just means the compiler needs to include a cell > variable for the current class whenever it thinks you might be using super(). +1 on super(self).foo. It's SomeLongClassName we want to get rid of, not self. As a bonus, super() and super(cls) have obvious semantics. -- Adam Olsen, aka Rhamphoryncus From krstic at solarsail.hcs.harvard.edu Tue Apr 24 00:21:06 2007 From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?B?SXZhbiBLcnN0acSH?=) Date: Mon, 23 Apr 2007 18:21:06 -0400 Subject: [Python-3000] Principles In-Reply-To: <20070423123456.63D7.JCARLSON@uci.edu> References: <20070423102411.63CE.JCARLSON@uci.edu> <07Apr23.121924pdt."57996"@synergy1.parc.xerox.com> <20070423123456.63D7.JCARLSON@uci.edu> Message-ID: <462D3152.8050702@solarsail.hcs.harvard.edu> Josiah Carlson wrote: > I believe that no performance improvement warrants such large scale > breakage of 3rd party extensions. If there's an implementation that's deemed superior to what's there now, then 3.0 is exactly the right time to break 3rd party extensions. Of course 3.0 isn't a blank check for gratuitous breakage, but being conservative *solely* for the purpose of not breaking compatibility defeats the purpose of having a 3.0 separate from the 2.x series, IMO. -- Ivan Krsti? | GPG: 0x147C722D From guido at python.org Tue Apr 24 00:38:21 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Apr 2007 15:38:21 -0700 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> <462BDCAA.7020501@canterbury.ac.nz> <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> <5.1.1.6.0.20070423172551.04c4e5f0@sparrow.telecommunity.com> Message-ID: On 4/23/07, Adam Olsen wrote: > On 4/23/07, Phillip J. Eby wrote: > > But I can't say I particularly like this idea, compared to "super.foo" or > > even "super(self).foo". In fact, the latter invocation doesn't even > > require a keyword -- it just means the compiler needs to include a cell > > variable for the current class whenever it thinks you might be using super(). > > +1 on super(self).foo. It's SomeLongClassName we want to get rid of, not self. > As a bonus, super() and super(cls) have obvious semantics. At least +0 from me too. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rhamph at gmail.com Tue Apr 24 00:41:37 2007 From: rhamph at gmail.com (Adam Olsen) Date: Mon, 23 Apr 2007 16:41:37 -0600 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: Message-ID: On 4/23/07, Guido van Rossum wrote: > I'm -1 on this and on any other proposed twiddlings of the __main__ > machinery. The only use case seems to be running scripts that happen > to be living inside a module's directory, which I've always seen as an > antipattern. To make me change my mind you'd have to convince me that > it isn't. So ./python Lib/test/regrtest.py and ./python Lib/test/test_something.py are both wrong? I note there's 26 files marked executable and 50 with #!/usr/bin/env python, so there's quite a bit of prior art in the "old way". Besides correcting the stdlib, adding a warning when running a script from a dir with __init__.py would help change the momentum. -- Adam Olsen, aka Rhamphoryncus From daniel at stutzbachenterprises.com Tue Apr 24 00:54:00 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 23 Apr 2007 17:54:00 -0500 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: <3d2ce8cb0704231515k3c75246eud35c5c5a3a2fb8f4@mail.gmail.com> References: <3d2ce8cb0704231142n4448fd0apb3eb7f3dd350b5a3@mail.gmail.com> <3d2ce8cb0704231515k3c75246eud35c5c5a3a2fb8f4@mail.gmail.com> Message-ID: On 4/23/07, Mike Klaas wrote: > The init-from iterable is especially interesting: a 20% speed increase > for list comps is not to be scoffed at. That one is because list()'s tp_new method is PyType_GenericNew instead of a wrapper around the more efficient PyList_New. Probably list comprehensions and the [] syntax already call PyList_New, but I haven't checked. > Have you analyzed the memory consumption of the two data structures? I haven't done any empirical studies, but I can give you a pretty good picture. For small lists, a BList uses a fixed amount of memory rather than aiming for the "just large enough" for Python lists. So, for very small lists, BLists are currently rather wasteful. The BList code could be changed so that for small lists they also do the "just large enough" thing, at the expense of a bit more code complexity. For large lists, the *worst* case for BLists is around twice the memory requirements of a regular Python list (each BList node is required to be at least half-full). I know that sounds horrible, but hear me out. :-) The *best* case for BLists is when each node is completely full, in which case they use negligibly more memory than a regular Python list. If a user creates a BList from whole-cloth (e.g., BList(iterable)), they will get a best-case BList. This would probably be the common case for user's who don't plan to modify their list much. If a user is doing lots of inserts and deletes, the BList's node's will typically be three-quarters full, translating into 33% extra memory above a regular Python list. However (and this is a big however!), these users will greatly appreciate the BList's additional speed for these operations. I hasten to add that these memory estimates are only about the space required to store the list structure as well as pointers to the objects in the list. The objects themselves (e.g., integers, class instances, whatever) will take up substantially more memory in any case. For example, for the regular 32-bit integer type, a single object takes up 12 bytes. A regular python list will spend an extra 4 bytes per integer (16 bytes total). A worst-case BList spends an extra 8 bytes per integer (20 bytes total). So, worst case is only a 25% increase. If the user makes use of operations that leverage the copy-on-write feature, then BLists are *much* more memory efficient that regular lists. __getslice__ is probably the most commonly used function in this category. A slice of a large BList uses only O(log n) memory in the worst case. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From guido at python.org Tue Apr 24 00:56:49 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Apr 2007 15:56:49 -0700 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: Message-ID: On 4/23/07, Adam Olsen wrote: > On 4/23/07, Guido van Rossum wrote: > > I'm -1 on this and on any other proposed twiddlings of the __main__ > > machinery. The only use case seems to be running scripts that happen > > to be living inside a module's directory, which I've always seen as an > > antipattern. To make me change my mind you'd have to convince me that > > it isn't. > > So ./python Lib/test/regrtest.py and ./python > Lib/test/test_something.py are both wrong? I'll happily admit they are bad examples, though as long as they work I don't care (and since they do work, I don't see them as motivating the PEP :-). This occasionally causes problems when someone adds a new test module that imports stuff using implicit relative imports. Fortunately those will be gone in py3k. > I note there's 26 files > marked executable and 50 with #!/usr/bin/env python, so there's quite > a bit of prior art in the "old way". Where are these files? If they are directly in Lib, that's not a problem. The PEP only addresses what happens if these are in a subpackage (like test). > Besides correcting the stdlib, adding a warning when running a script > from a dir with __init__.py would help change the momentum. Patch anyone? We could move regrtest out of the test package into Lib. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Tue Apr 24 01:13:12 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 23 Apr 2007 19:13:12 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> At 03:16 PM 4/23/2007 -0700, Guido van Rossum wrote: >On 4/23/07, Phillip J. Eby wrote: >>Assuming that: >> >>1. If you call such a function, it will raise some error, like >>NotImplementedError > >That would be up to the author of the function; they would have to >explicitly raise NotImplementedError in the body. The ABC proposal >allows meaningful abstract methods that can be called (only) via >"super"; the use case for this is that the abstract method might be >the one that decides which exception should be thrown (e.g. >__getitem__ and __next__ do this), or perhaps it could provide a >default implementation for certain types. (Potential example of the >latter: Sequence.__getitem__() could raise IndexError when the >argument is an Integer but handle the case where the argument is a >slice instance, assuming there's a suitable factory which could be a >designated class method.) Ah... interesting. This is different from what I understood to be "abstract" methods in other languages where an abstract method is always one that does not have an actual implementation. I guess I skimmed PEP 3119 a little too quickly. It sounds like your proposal is to mark methods as "abstract" even if they have a useful "null" implementation. I guess I don't see what this adds, at least for the examples in the sandbox, except for making the class non-instantiable as a side-effect. Of course, if @abstract were a class decorator as well as a function decorator, then it could have a single meaning in both contexts: "this thing shouldn't be callable". That is to say, this: @abstract class Iterator(Iterable): ... would simply mean calling "Iterator()" would result in a NotImplementedError, just like marking a function @abstract means that calling it would result in a NotImplementedError. And, as far as I can see, the only ABC method I'd mark @abstract individually would be Hashable.__hash__: everything else here looks to me like a perfectly valid "empty" implementation of the method(s) in question. I suppose there is some value in requiring people to override @abstract methods to make a subclass instantiable, versus merely using the lack of an @abstract class decorator to indicate that a subclass is concrete. But I wonder if being able to have just one @abstract decorator (that always means "you can't call this by default") mightn't be worth giving up that tiny bit of extra type checking? From rhamph at gmail.com Tue Apr 24 01:32:45 2007 From: rhamph at gmail.com (Adam Olsen) Date: Mon, 23 Apr 2007 17:32:45 -0600 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: References: <3d2ce8cb0704231142n4448fd0apb3eb7f3dd350b5a3@mail.gmail.com> <3d2ce8cb0704231515k3c75246eud35c5c5a3a2fb8f4@mail.gmail.com> Message-ID: On 4/23/07, Daniel Stutzbach wrote: > If a user creates a BList from whole-cloth (e.g., BList(iterable)), > they will get a best-case BList. This would probably be the common > case for user's who don't plan to modify their list much. > > If a user is doing lots of inserts and deletes, the BList's node's > will typically be three-quarters full, translating into 33% extra > memory above a regular Python list. However (and this is a big > however!), these users will greatly appreciate the BList's additional > speed for these operations. For contrast, the existing list seems to over-allocate by 12.5%, for an average of 6.25% wasted. However, when deleting it only resizes when it drops below half full, leaving it with a worst case of 100%, the same as BList. The usage patterns can make a big difference there though. A list's worst-case is only reached if you have a spike, then drop to just above half-full. A BList's worst-case is reached through random slicing. Daniel, does repeated getitem/setitem (as in random.shuffle()) increase memory usage, or does it have no effect? A random.shuffle() benchmark might be a better a better demonstration of the overall costs. -- Adam Olsen, aka Rhamphoryncus From jimjjewett at gmail.com Tue Apr 24 01:42:21 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 23 Apr 2007 19:42:21 -0400 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> <462BDCAA.7020501@canterbury.ac.nz> <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> Message-ID: On 4/23/07, Michele Cella wrote: > ... having a super keyword with methods attached ... doesn't feel right Agreed. The goal is to *fix* super, not to make it a keyword. Making it a keyword might -- or might not -- be the way to do that. A keyword with attributes is probably not the right answer. For What Its Worth, the longer I think about this, the more convinced I am that the right answer is to keep it an object. The problem is that it has to grow even more magical than it already is, unless we break a lot of backwards compatibility. If we're willing to put up with the magic, then it would work to make super syntactic sugar for super(__this_class__, self) At the moment, I can't see anything wrong with this, but I have a feeling I'm missing something about how the super object should behave on its own. Is there a good way (other than "being inside a method"?) to distinguish between: super.__str__() # I want to call the super-class' str method and super.__str__() # I want to call str(super) > The only other option that comes to my mind is using a special attribute > in the instance itself (like __class__, __dict__, ...): > self.__super__.mymethod(arg) __class__ and __dict__ are (sort of) ordinary attributes -- they point to a specific object. super would probably need to be an active property. class A: ... class B1(A): ... class C1(B1): ... class B2(A): ... class C2(B1): ... class D(C1, C2): ... class E(D): ... When an instance of E is created, the super call may well be made in D, so you don't know the instance's class when compiling the code. When methods in B1 make a super call for "normal" instances of C1, they go straight to class A. But if the instance is really an instance of E, then they first have to loop back through C2 and C1. So when compiling the class, you don't know which class will be next in the mro. So you can't do it from the class -- but you also can't do it only from the instance. For an instance of E, super(C1, self) and super(B2, self) should go to different next-classes. So you need both the instance's own true class, and the class where the current method was defined -- but even that isn't quite enough. You can't quite say "next in the instance's mro", because if the method happens to be something that C2 doesn't override, then super(D, self) should go straight to C1 -- but it still needs to look at C2 for other methods. -jJ From guido at python.org Tue Apr 24 01:49:52 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Apr 2007 16:49:52 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> Message-ID: On 4/23/07, Phillip J. Eby wrote: > At 03:16 PM 4/23/2007 -0700, Guido van Rossum wrote: > >On 4/23/07, Phillip J. Eby wrote: > >>Assuming that: > >> > >>1. If you call such a function, it will raise some error, like > >>NotImplementedError > > > >That would be up to the author of the function; they would have to > >explicitly raise NotImplementedError in the body. The ABC proposal > >allows meaningful abstract methods that can be called (only) via > >"super"; the use case for this is that the abstract method might be > >the one that decides which exception should be thrown (e.g. > >__getitem__ and __next__ do this), or perhaps it could provide a > >default implementation for certain types. (Potential example of the > >latter: Sequence.__getitem__() could raise IndexError when the > >argument is an Integer but handle the case where the argument is a > >slice instance, assuming there's a suitable factory which could be a > >designated class method.) > > Ah... interesting. This is different from what I understood to be > "abstract" methods in other languages where an abstract method is always > one that does not have an actual implementation. I guess I skimmed PEP > 3119 a little too quickly. > > It sounds like your proposal is to mark methods as "abstract" even if they > have a useful "null" implementation. Not exactly. Some of the proposed ABCs have concrete methods. These are the kinds you'd find in mixin classes for Python 2. (Example: Iterator.__iter__() is a concrete method returning self. You almost never have a need to override it.) The abstract ones have null implementations that aren't all that useful for two reasons: (a) they implement an utter edge case (e.g. an empty iterator) (b) they must be overridden. Their main purpose is to provide an example -- either they show which exception to raise (__next__, __getitem__) or they show the type of value to return (__hash__). > I guess I don't see what this adds, > at least for the examples in the sandbox, except for making the class > non-instantiable as a side-effect. I guess I exaggerated the usefulness of the null implementations. It's perfectly fine not to call the null implementation but to inline it (e.g. raise StopIteration instead of return super(self).__next__()). > Of course, if @abstract were a class decorator as well as a function > decorator, then it could have a single meaning in both contexts: "this > thing shouldn't be callable". > > That is to say, this: > > @abstract > class Iterator(Iterable): > ... > > would simply mean calling "Iterator()" would result in a > NotImplementedError, just like marking a function @abstract means that > calling it would result in a NotImplementedError. Yeah, but it would be less specific because (in the case of ABCs that define multiple methods) it wouldn't tell you which mwethods you have to override. > And, as far as I can see, the only ABC method I'd mark @abstract > individually would be Hashable.__hash__: everything else here looks to me > like a perfectly valid "empty" implementation of the method(s) in question. Actually, Hashable.__hash__ is also perfectly valid. :-) > I suppose there is some value in requiring people to override @abstract > methods to make a subclass instantiable, versus merely using the lack of an > @abstract class decorator to indicate that a subclass is concrete. But I > wonder if being able to have just one @abstract decorator (that always > means "you can't call this by default") mightn't be worth giving up that > tiny bit of extra type checking? I prefer to follow the lead of C++ here -- an abstract class is abstract by virtue of having at least one abstract method. That the abstract methods are still somewhat useful implementations is mostly to provide a valid (if not necessarily useful) end point for super-calling in cooperative MI schemes. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jimjjewett at gmail.com Tue Apr 24 01:50:39 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 23 Apr 2007 19:50:39 -0400 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> <462BDCAA.7020501@canterbury.ac.nz> <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> <5.1.1.6.0.20070423172551.04c4e5f0@sparrow.telecommunity.com> Message-ID: On 4/23/07, Adam Olsen wrote: > On 4/23/07, Phillip J. Eby wrote: > > But I can't say I particularly like this idea, compared to "super.foo" or > > even "super(self).foo". In fact, the latter invocation doesn't even > > require a keyword -- it just means the compiler needs to include a cell > > variable for the current class whenever it thinks you might be using super(). > +1 on super(self).foo. It's SomeLongClassName we want to get rid of, not self. It won't normally be for an attribute. Typically, it would look like: super(self).foo(arg1, arg2) or even super(self).foo() I'm not sure that is much improvement over explicitly showing the class, though it is certainly an improvement over using the (rebindable) class name. > As a bonus, super() and super(cls) have obvious semantics. What would they be? Are you assuming that the two-argument forms would go away? Or that the *first* positional argument could be defaulted to __this_class__, but the second would be mandatory? -jJ From rhamph at gmail.com Tue Apr 24 02:00:42 2007 From: rhamph at gmail.com (Adam Olsen) Date: Mon, 23 Apr 2007 18:00:42 -0600 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: Message-ID: On 4/23/07, Guido van Rossum wrote: > On 4/23/07, Adam Olsen wrote: > > I note there's 26 files > > marked executable and 50 with #!/usr/bin/env python, so there's quite > > a bit of prior art in the "old way". > > Where are these files? If they are directly in Lib, that's not a > problem. The PEP only addresses what happens if these are in a > subpackage (like test). $ ls -l Lib/test/*.py | grep -c '^...x' 26 $ head -qn 1 Lib/test/*.py | grep -c /usr/bin/env 50 > > Besides correcting the stdlib, adding a warning when running a script > > from a dir with __init__.py would help change the momentum. > > Patch anyone? We could move regrtest out of the test package into Lib. -- Adam Olsen, aka Rhamphoryncus From daniel at stutzbachenterprises.com Tue Apr 24 02:37:12 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 23 Apr 2007 19:37:12 -0500 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: References: <3d2ce8cb0704231142n4448fd0apb3eb7f3dd350b5a3@mail.gmail.com> <3d2ce8cb0704231515k3c75246eud35c5c5a3a2fb8f4@mail.gmail.com> Message-ID: On 4/23/07, Adam Olsen wrote: > The usage patterns can make a big difference there though. A list's > worst-case is only reached if you have a spike, then drop to just > above half-full. A BList's worst-case is reached through random > slicing. > > Daniel, does repeated getitem/setitem (as in random.shuffle()) > increase memory usage, or does it have no effect? getitem and setitem have no effect on the BList's internal structure (and thus no effect on memory usage). Only operations that add or remove elements do. > A random.shuffle() benchmark might be a better a better demonstration > of the overall costs. Here you go: http://stutzbachenterprises.com/fig/shuffle.html The regular list's has an advantage due to the special casing of x[k] within the Python bytecode interpreter. After that, the O(log n) cost of BList's item is visible in the right-hand graph via the step at n=128. The next step would be at n=16384. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From rhamph at gmail.com Tue Apr 24 02:41:44 2007 From: rhamph at gmail.com (Adam Olsen) Date: Mon, 23 Apr 2007 18:41:44 -0600 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <462BDCAA.7020501@canterbury.ac.nz> <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> <5.1.1.6.0.20070423172551.04c4e5f0@sparrow.telecommunity.com> Message-ID: On 4/23/07, Jim Jewett wrote: > On 4/23/07, Adam Olsen wrote: > > On 4/23/07, Phillip J. Eby wrote: > > > But I can't say I particularly like this idea, compared to "super.foo" or > > > even "super(self).foo". In fact, the latter invocation doesn't even > > > require a keyword -- it just means the compiler needs to include a cell > > > variable for the current class whenever it thinks you might be using super(). > > > +1 on super(self).foo. It's SomeLongClassName we want to get rid of, not self. > > It won't normally be for an attribute. Typically, it would look like: > > super(self).foo(arg1, arg2) > > or even > > super(self).foo() > > I'm not sure that is much improvement over explicitly showing the > class, though it is certainly an improvement over using the > (rebindable) class name. > > > As a bonus, super() and super(cls) have obvious semantics. > > What would they be? Are you assuming that the two-argument forms > would go away? Or that the *first* positional argument could be > defaulted to __this_class__, but the second would be mandatory? Err.. yes? Actually I assumed that there'd be both the old two-argument forms and the new implicit __this_class__ form, but in hindsight that's obvious ambiguous. Although I don't understand the purpose of the old super(type) option, so maybe that can be deleted to remove the ambiguity. -- Adam Olsen, aka Rhamphoryncus From collinw at gmail.com Tue Apr 24 03:03:31 2007 From: collinw at gmail.com (Collin Winter) Date: Mon, 23 Apr 2007 18:03:31 -0700 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> <462BDCAA.7020501@canterbury.ac.nz> <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> Message-ID: <43aa6ff70704231803r7eff65f0j823814422b5a1ab9@mail.gmail.com> On 4/23/07, Jim Jewett wrote: [snip] > If we're willing to put up with the magic, then it would work to make > super syntactic sugar for > > super(__this_class__, self) > > At the moment, I can't see anything wrong with this, but I have a > feeling I'm missing something about how the super object should behave > on its own. You mean, have "super.method_name(*args)" expanded by the AST compiler into super(__this_class__, self).method_name(*args)"? The object reference -> function call change is too much magic; -1. However, having the AST compiler expand class A: def m(self): blah() super(self).m() blah() to class A: def m(self): super = super_factory(A) blah() super(self).m() blah() strikes me as an "appropriate" amount of magic. Restricting the magic to a source transformation should enable other Python implementations to implement this relatively easily. Collin Winter From pje at telecommunity.com Tue Apr 24 03:09:53 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 23 Apr 2007 21:09:53 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> At 04:49 PM 4/23/2007 -0700, Guido van Rossum wrote: >>I suppose there is some value in requiring people to override @abstract >>methods to make a subclass instantiable, versus merely using the lack of an >>@abstract class decorator to indicate that a subclass is concrete. But I >>wonder if being able to have just one @abstract decorator (that always >>means "you can't call this by default") mightn't be worth giving up that >>tiny bit of extra type checking? > >I prefer to follow the lead of C++ here -- an abstract class is >abstract by virtue of having at least one abstract method. It's been way too long since I did any C++, but isn't it the case that an abstract (aka "pure virtual"?) method in C++ is one that *can't be invoked*? [pause to Google it...] Well I'll be darned. I didn't know you could actually provide an implementation for a pure virtual function. Hm. Learn something new every day... except while I was writing C++ evidently. :) But are there any other languages besides C++ that have this idiom? I'm not familiar with any that do (at least, not that I know of!), and the fact that C++ allows it seems a bit non-obvious to me. Java and C# pretty much do @abstract the way I proposed it, for example: Java: http://java.sun.com/docs/books/tutorial/java/IandI/abstract.html C#: http://www.codeproject.com/useritems/Abstract_CLS_MTHD.asp That is, they both separate class-abstraction from method-abstraction, even though the latter implies the former. Thus, you can have an abstract class 'Iterator' even if all its methods are non-abstract. (However, if any of its methods are abstract, the class itself is required to be abstract.) As someone with more recent background in Java than C++, I find the idea of abstract methods having an executable implementation to be quite confusing, and suspect that other people with that Java or C# background will do the same thing. That is, skim the explanation and miss the significant difference between the C++ way and what they're used to. >That the abstract methods are still somewhat useful implementations is >mostly to provide a valid (if not necessarily useful) end point for >super-calling in cooperative MI schemes. Right; I guess my point is that if those "somewhat" useful implementations are useful, they're useful, and there's no need to treat the corresponding class as "abstract" (in the "non-instantiable" sense) in that case. For example, I could pass an Iterable() to something that expected an iterable. From jcarlson at uci.edu Tue Apr 24 03:16:47 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 23 Apr 2007 18:16:47 -0700 Subject: [Python-3000] Principles In-Reply-To: <462D3152.8050702@solarsail.hcs.harvard.edu> References: <20070423123456.63D7.JCARLSON@uci.edu> <462D3152.8050702@solarsail.hcs.harvard.edu> Message-ID: <20070423174934.63E0.JCARLSON@uci.edu> Ivan Krstic wrote: > Josiah Carlson wrote: > > I believe that no performance improvement warrants such large scale > > breakage of 3rd party extensions. > > If there's an implementation that's deemed superior to what's there now, > then 3.0 is exactly the right time to break 3rd party extensions. Of > course 3.0 isn't a blank check for gratuitous breakage, but being > conservative *solely* for the purpose of not breaking compatibility > defeats the purpose of having a 3.0 separate from the 2.x series, IMO. The question is whether or not the breakage would be "gratuitous". The only use of lists that wouldn't be broken is if people relied solely on the use of PyList_* functions and not lst->ob_item[] references. A quick google code search offers about 5-10 3rd party examples against 40 varaints of the Python C source. The lack of results seems a bit fishy, but maybe my search for \-\>ob_item\[ was too restrictive. Searching for \-\>ob_item gets more results, some of which are things like PyLucene, guppy, etc. This also seems a bit slim (Google doesn't seem to index all projects on sourceforge). - Josiah From jcarlson at uci.edu Tue Apr 24 05:19:40 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 23 Apr 2007 20:19:40 -0700 Subject: [Python-3000] PEP to change how the main module is delineated In-Reply-To: References: Message-ID: <20070423195834.63E9.JCARLSON@uci.edu> "Guido van Rossum" wrote: > > On 4/22/07, Brett Cannon wrote: > > This PEP is to change the ``if __name__ == "__main__": ...`` idiom to > > ``if __name__ == sys.main: ...`` so that you at least have a chance > > to execute module in a package that use relative imports. > > > > Ran this PEP past python-ideas. Stopped the discussion there when too > > many new ideas were being proposed. =) I have listed all of them in > > the Rejected Ideas section, although if overwhelming support for one > > comes forward the PEP can shift to one of them. > > I'm -1 on this and on any other proposed twiddlings of the __main__ > machinery. The only use case seems to be running scripts that happen > to be living inside a module's directory, which I've always seen as an > antipattern. To make me change my mind you'd have to convince me that > it isn't. "Guido's Decision" on PEP 328 offers a module/package hierarchy where 'from ..subpackage2.moduleZ import eggs' is offered as a valid use-case. It does not specify the name of subpackage1/moduleX.py (that is performing the import), but there is the somewhat hidden implication that if it were executed directly from the command line, it would not be allowed (but this is only revealed in understanding the implications of __name__ == '__main__' with regards to the PEP). Personally, I would much prefer a change in __name__ == sys.main than having to hack relative imports to offer __name__s like __main__..subpackage.moduleZ, or even 'import sys;sys.path.insert(1, os.path.basename(os.path.basename(__file__)))', the latter of which I already do in every script in a codebase laid out like... /.../packageX/ __init__.py application1/ __init__.py script1.py ... tests/ ... application2/ __init__.py script5.py ... tests/ ... ... shared/ __init__.py shared1.py ... The preamble to properly deal with error conditions in the current path became tiresome to place in every application path after the first few. I can't believe that the above is an antipattern; seems to be the simplest way to update and distribute a group of related applications with a simple 'svn co packageX' or 'svn up packageX'. - Josiah From ironfroggy at gmail.com Tue Apr 24 06:11:01 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Tue, 24 Apr 2007 00:11:01 -0400 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> <462BDCAA.7020501@canterbury.ac.nz> <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> Message-ID: <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> On 4/23/07, Jim Jewett wrote: > On 4/23/07, Michele Cella wrote: > > > ... having a super keyword with methods attached ... doesn't feel right > > Agreed. The goal is to *fix* super, not to make it a keyword. Making > it a keyword might -- or might not -- be the way to do that. A > keyword with attributes is probably not the right answer. > > For What Its Worth, the longer I think about this, the more convinced > I am that the right answer is to keep it an object. The problem is > that it has to grow even more magical than it already is, unless we > break a lot of backwards compatibility. > > If we're willing to put up with the magic, then it would work to make > super syntactic sugar for > > super(__this_class__, self) > > At the moment, I can't see anything wrong with this, but I have a > feeling I'm missing something about how the super object should behave > on its own. Is there a good way (other than "being inside a method"?) > to distinguish between: > > super.__str__() # I want to call the super-class' str method > > and > > super.__str__() # I want to call str(super) > > > > > The only other option that comes to my mind is using a special attribute > > in the instance itself (like __class__, __dict__, ...): > > > self.__super__.mymethod(arg) > > __class__ and __dict__ are (sort of) ordinary attributes -- they point > to a specific object. super would probably need to be an active > property. > > class A: ... > class B1(A): ... > class C1(B1): ... > class B2(A): ... > class C2(B1): ... > > class D(C1, C2): ... > class E(D): ... > > When an instance of E is created, the super call may well be made in > D, so you don't know the instance's class when compiling the code. > > When methods in B1 make a super call for "normal" instances of C1, > they go straight to class A. But if the instance is really an > instance of E, then they first have to loop back through C2 and C1. > So when compiling the class, you don't know which class will be next > in the mro. > > So you can't do it from the class -- but you also can't do it only > from the instance. For an instance of E, super(C1, self) and > super(B2, self) should go to different next-classes. > > So you need both the instance's own true class, and the class where > the current method was defined -- but even that isn't quite enough. > You can't quite say "next in the instance's mro", because if the > method happens to be something that C2 doesn't override, then super(D, > self) should go straight to C1 -- but it still needs to look at C2 for > other methods. > > -jJ > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/ironfroggy%40gmail.com > I will +1 on the self.__super__ suggestion. Hey, its very doable and I even whipped it up with a simple metaclass, so it would be a tiny change to 'type' in order to actually implement it as a standard feature. The demonstration is as follows: class _superdesc(object): def __get__(self, obj, objcls): return super(cls, obj) class autosuper(type): def __init__(cls, name, bases, clsdict): cls.__super__ = _superdesc() class A(object): __metaclass__ = autosuper x = 1 class B(A): x = 2 assert B().__super__.x == 1 -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From steven.bethard at gmail.com Tue Apr 24 06:22:44 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 23 Apr 2007 22:22:44 -0600 Subject: [Python-3000] Fixing super anyone? In-Reply-To: <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> <462BDCAA.7020501@canterbury.ac.nz> <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> Message-ID: On 4/23/07, Calvin Spealman wrote: > I will +1 on the self.__super__ suggestion. Hey, its very doable and I > even whipped it up with a simple metaclass, so it would be a tiny > change to 'type' in order to actually implement it as a standard > feature. The demonstration is as follows: > > class _superdesc(object): > def __get__(self, obj, objcls): > return super(cls, obj) > > class autosuper(type): > def __init__(cls, name, bases, clsdict): > cls.__super__ = _superdesc() > > class A(object): > __metaclass__ = autosuper > x = 1 > > class B(A): > x = 2 > > assert B().__super__.x == 1 Does that really work? There's a typo in _superdesc (I don't know where 'cls' comes from) but if you meant 'objcls', here's what I get:: >>> class _superdesc(object): ... def __get__(self, obj, cls): ... return super(cls, obj) ... >>> class autosuper(type): ... def __init__(cls, name, bases, clsdict): ... cls.__super__ = _superdesc() ... >>> class A: ... __metaclass__ = autosuper ... def f(self): ... print 'A' ... >>> class B(A): ... def f(self): ... print 'B' ... self.__super__.f() ... >>> class C(A): ... def f(self): ... print 'C' ... self.__super__.f() ... >>> class D(B, C): ... def f(self): ... print 'D' ... self.__super__.f() ... >>> D().f() D B B B ... Traceback (most recent call last): File "", line 1, in ... RuntimeError: maximum recursion depth exceeded STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From guido at python.org Tue Apr 24 06:25:01 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Apr 2007 21:25:01 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> Message-ID: On 4/23/07, Phillip J. Eby wrote: > At 04:49 PM 4/23/2007 -0700, Guido van Rossum wrote: > >>I suppose there is some value in requiring people to override @abstract > >>methods to make a subclass instantiable, versus merely using the lack of an > >>@abstract class decorator to indicate that a subclass is concrete. But I > >>wonder if being able to have just one @abstract decorator (that always > >>means "you can't call this by default") mightn't be worth giving up that > >>tiny bit of extra type checking? > > > >I prefer to follow the lead of C++ here -- an abstract class is > >abstract by virtue of having at least one abstract method. > > It's been way too long since I did any C++, but isn't it the case that an > abstract (aka "pure virtual"?) method in C++ is one that *can't be > invoked*? [pause to Google it...] Well I'll be darned. I didn't know you > could actually provide an implementation for a pure virtual > function. Hm. Learn something new every day... except while I was > writing C++ evidently. :) Actually I just meant to follow its lead in defining that a class is abstract as soon as one or more of its methods are abstract. In Java you need to repeat this by making the class abstract; that seems unpythonic. > But are there any other languages besides C++ that have this idiom? I'm > not familiar with any that do (at least, not that I know of!), and the fact > that C++ allows it seems a bit non-obvious to me. Java and C# pretty much > do @abstract the way I proposed it, for example: > > Java: http://java.sun.com/docs/books/tutorial/java/IandI/abstract.html > C#: http://www.codeproject.com/useritems/Abstract_CLS_MTHD.asp > > That is, they both separate class-abstraction from method-abstraction, even > though the latter implies the former. Thus, you can have an abstract class > 'Iterator' even if all its methods are non-abstract. (However, if any of > its methods are abstract, the class itself is required to be abstract.) I don't see the point of having an abstract class without abstract methods. The main reasons I came up (without knowing C++ allows this too!) with the idea of giving abstract methods a valid implementation wer (a) there's got to be *something* in the body; (b) to provide an end point for cooperative-MI-style code. > As someone with more recent background in Java than C++, I find the idea of > abstract methods having an executable implementation to be quite confusing, > and suspect that other people with that Java or C# background will do the > same thing. That is, skim the explanation and miss the significant > difference between the C++ way and what they're used to. Well, too bad. After a day of coding in Java I start typing curly braces and semicolons too. But that doesn't mean Python should adopt these. > >That the abstract methods are still somewhat useful implementations is > >mostly to provide a valid (if not necessarily useful) end point for > >super-calling in cooperative MI schemes. > > Right; I guess my point is that if those "somewhat" useful implementations > are useful, they're useful, and there's no need to treat the corresponding > class as "abstract" (in the "non-instantiable" sense) in that case. For > example, I could pass an Iterable() to something that expected an iterable. Perhaps. Though I think it would defeat the purpose of being required to provide an implementation. If forced to choose, I'd rather remove the meaningful bodies (and replace them with "raise NotImplementedError") than remove the @abstractmethod decorators. But I prefer the current proposal. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nnorwitz at gmail.com Tue Apr 24 06:25:09 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 23 Apr 2007 21:25:09 -0700 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: References: Message-ID: On 4/23/07, Daniel Stutzbach wrote: > > The Radical Proposal > ------------------------------- > > Replace list() with the BList. I looked over this patch and have various questions/comments. In rough priority order: I'm concerned about the moderately heavy use of macros wrt to debugging. I noticed the use of alloca(). This isn't used in the core anywhere (except a few uses on Windows). That might be an issue. I didn't check how it was used. I noticed that calculating the height was recursive. Not sure where this function was called from, but does it handle blists which contain themselves? Have you tried running this code under regrtest.py with the -R flag to check for memory leaks? There were C++ (//) style comments and C99 declarations of variables. These would have to be changed if included in the core. Also, size_t and Py_ssize_t should be used throughout. (ssize_t should be Py_ssize_t). I noticed a bunch of ints and unsigneds. Although I also noticed you used PyIndex_Check(). In general I thought the code was pretty good. Had a decent amount of error checking/handling. There were some cases I wasn't sure, but in the paths that I traced, I didn't find any obvious problems. n From tom at vector-seven.com Tue Apr 24 03:34:32 2007 From: tom at vector-seven.com (Thomas Lee) Date: Tue, 24 Apr 2007 11:34:32 +1000 Subject: [Python-3000] Fixing super anyone? In-Reply-To: <43aa6ff70704231803r7eff65f0j823814422b5a1ab9@mail.gmail.com> References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> <462BDCAA.7020501@canterbury.ac.nz> <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> <43aa6ff70704231803r7eff65f0j823814422b5a1ab9@mail.gmail.com> Message-ID: <462D5EA8.4070902@vector-seven.com> Oops, here's the (broken) patch. -------------- next part -------------- A non-text attachment was scrubbed... Name: super_python-3000.patch Type: text/x-patch Size: 1401 bytes Desc: not available Url : http://mail.python.org/pipermail/python-3000/attachments/20070424/4120a433/attachment.bin From tom at vector-seven.com Tue Apr 24 03:32:59 2007 From: tom at vector-seven.com (Thomas Lee) Date: Tue, 24 Apr 2007 11:32:59 +1000 Subject: [Python-3000] Fixing super anyone? In-Reply-To: <43aa6ff70704231803r7eff65f0j823814422b5a1ab9@mail.gmail.com> References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com> <462BDCAA.7020501@canterbury.ac.nz> <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> <43aa6ff70704231803r7eff65f0j823814422b5a1ab9@mail.gmail.com> Message-ID: <462D5E4B.2000604@vector-seven.com> Collin Winter wrote: > However, having the AST compiler expand > > class A: > def m(self): > blah() > super(self).m() > blah() > > to > > class A: > def m(self): > super = super_factory(A) > blah() > super(self).m() > blah() > I'm sold on the super(self) syntax and whacked together a quick & dirty patch. I made my changes in Objects/super.c, rather than the AST as you suggest. Determining __this_class__ ('A' in your example) without touching the AST is obviously going to be problematic, but even *with* the AST how would you propose we determine the "current" class? I've attached my patch irrespective. All relevant changes are within super_init. Forgive the ignorance, I'm relatively new to this stuff :) Cheers, Tom From talin at acm.org Tue Apr 24 08:17:44 2007 From: talin at acm.org (Talin) Date: Mon, 23 Apr 2007 23:17:44 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> Message-ID: <462DA108.8090307@acm.org> Guido van Rossum wrote: > On 4/23/07, Phillip J. Eby wrote: >> At 04:49 PM 4/23/2007 -0700, Guido van Rossum wrote: >>>> I suppose there is some value in requiring people to override @abstract >>>> methods to make a subclass instantiable, versus merely using the lack of an >>>> @abstract class decorator to indicate that a subclass is concrete. But I >>>> wonder if being able to have just one @abstract decorator (that always >>>> means "you can't call this by default") mightn't be worth giving up that >>>> tiny bit of extra type checking? >>> I prefer to follow the lead of C++ here -- an abstract class is >>> abstract by virtue of having at least one abstract method. >> It's been way too long since I did any C++, but isn't it the case that an >> abstract (aka "pure virtual"?) method in C++ is one that *can't be >> invoked*? [pause to Google it...] Well I'll be darned. I didn't know you >> could actually provide an implementation for a pure virtual >> function. Hm. Learn something new every day... except while I was >> writing C++ evidently. :) I'm not sure what Phillip is talking about here - C++ does not allow you to instantiate a class that has non-overridden pure virtual methods, and no pure virtual method can have a body. You cannot provide an implementation for a pure virtual function. Now, there are occasions in C++ where it is possible for buggy code to "call" a pure virtual function - but the result is always an exception. Recall that in C++, when an instance is destructed, as each base class's destructor is called the vtable pointer is modified to point to the vtable for that base class, not the vtable for the actual class of the instance. (I've always had problems with this rule, but there it is.) If you attempt to call a subclass method after that subclass's destructor has been called (which is, of course, an error), you may end up invoking the little stub function which a pure virtual vtable entry points to. This is about as "legal" as dereferencing memory after it has been freed. > Actually I just meant to follow its lead in defining that a class is > abstract as soon as one or more of its methods are abstract. In Java > you need to repeat this by making the class abstract; that seems > unpythonic. > >> But are there any other languages besides C++ that have this idiom? I'm >> not familiar with any that do (at least, not that I know of!), and the fact >> that C++ allows it seems a bit non-obvious to me. Java and C# pretty much >> do @abstract the way I proposed it, for example: >> >> Java: http://java.sun.com/docs/books/tutorial/java/IandI/abstract.html >> C#: http://www.codeproject.com/useritems/Abstract_CLS_MTHD.asp >> >> That is, they both separate class-abstraction from method-abstraction, even >> though the latter implies the former. Thus, you can have an abstract class >> 'Iterator' even if all its methods are non-abstract. (However, if any of >> its methods are abstract, the class itself is required to be abstract.) > > I don't see the point of having an abstract class without abstract methods. > > The main reasons I came up (without knowing C++ allows this too!) with > the idea of giving abstract methods a valid implementation wer (a) > there's got to be *something* in the body; (b) to provide an end point > for cooperative-MI-style code. > >> As someone with more recent background in Java than C++, I find the idea of >> abstract methods having an executable implementation to be quite confusing, >> and suspect that other people with that Java or C# background will do the >> same thing. That is, skim the explanation and miss the significant >> difference between the C++ way and what they're used to. > > Well, too bad. After a day of coding in Java I start typing curly > braces and semicolons too. But that doesn't mean Python should adopt > these. I think I am in (weak) agreement with Phillip here. In every language that I am familiar with, the adjective "abstract" when applied to a method implies two things: (1) that the method does not have a body, and (2) the method must be implemented by a subclass if that subclass is able to be instantiated. In fact, (2) is merely a consequence of (1), when combined with the additional constraint that no instantiated class can have methods with no body. However, I think what is going on is that you're using the word "abstract" in a slightly difference sense. Normally, an abstract method provides merely an interface, in other words it defines the method name, argument types, and return type of a method - and nothing more. What you're doing, I think, is saying that an abstract method provides all those things, and in addition it provides a kind of sketch or outline of what the functionality ought to be. I don't have a problem with that part so much. But in addition there is a sense that you also want these methods to be useful helper functions for base classes to call, but you don't want them to be called directly. In languages such as Java and C++, this is what "protected" is for - methods that aren't available to the outside world, but which can be called by subclasses. So it seems to me -- at least from the standpoint of someone steeped in the mindset of Java / C++ / C# et al -- that you are conflating "abstract" and "protected". In what is considered "good style" for such languages, protected helper methods ought to have different names than the methods which invoke them. So for example, you might have the following: /// Abstract base class for hashable object class Hashable { public: /// Declare abstract hash method virtual int GetHash() const = 0; protected: /// Protected helper method int DefaultHashFunc() { return 0; } }; // A concrete class derived from hashable class MyConcreteClass : public Hashable { public: /// Declare abstract hash method virtual int GetHash() const { return DefaultHashFunc(); } }; Granted, in this particular case the decision to provide a helper function is rather odd. Why go through the trouble of defining DefaultHash() in the base class that simply returns 0, when any non-trivial subclass will provide a real implementation of GetHash() instead of merely forwarding the call to DefaultHash()? But the solution isn't to give Hashable::GetHash() a body. From a C++ programmer's view, there's only two choices: Either Hashable::GetHash() is abstract, or it's not. If it's abstract, then every subclass is required to re-implement it, and giving it a body has no purpose whatsoever; If it's not, then subclasses are allowed to default to its implementation. >>> That the abstract methods are still somewhat useful implementations is >>> mostly to provide a valid (if not necessarily useful) end point for >>> super-calling in cooperative MI schemes. >> Right; I guess my point is that if those "somewhat" useful implementations >> are useful, they're useful, and there's no need to treat the corresponding >> class as "abstract" (in the "non-instantiable" sense) in that case. For >> example, I could pass an Iterable() to something that expected an iterable. > > Perhaps. Though I think it would defeat the purpose of being required > to provide an implementation. If forced to choose, I'd rather remove > the meaningful bodies (and replace them with "raise > NotImplementedError") than remove the @abstractmethod decorators. But > I prefer the current proposal. -- Talin From daniel at stutzbachenterprises.com Tue Apr 24 13:10:03 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 24 Apr 2007 06:10:03 -0500 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: References: Message-ID: On 4/23/07, Neal Norwitz wrote: > On 4/23/07, Daniel Stutzbach wrote: > > Replace list() with the BList. > > I looked over this patch and have various questions/comments. In > rough priority order: Thanks for taking the time to look over the code. I greatly appreciate it. > I'm concerned about the moderately heavy use of macros wrt to debugging. Yes, some of the debugging code is a little hairy and needs a clean-up. I originally had written a prototype for BLists in Python and used function decorators to indicate which invariants the function must maintain. This was immensely helpful during development. C doesn't have function decorators, so I went with some very hairy macros. Perhaps it would be better to go with something more like this: PyObject *some_function(PyObject *self, some_other_args) { PyObject *ret; check_invariants(self); // do some stuff if (some_condition) { ret = case1; goto done; } // do some more stuff done: check_return_invariants(self, VALID_RW, VALID_PARENT) return ret; } > I noticed the use of alloca(). This isn't used in the core anywhere > (except a few uses on Windows). That might be an issue. I didn't > check how it was used. These were the result of converting code that was doing malloc/free's to be more efficient. They could all be converted to ordinary variable declarations now. > I noticed that calculating the height was recursive. Not sure where > this function was called from, but does it handle blists which contain > themselves? Yes. If the ->leaf flag is true, the BList node contains user objects which will not be recursed on (ever if they are also BList nodes). If the ->flag is false, the BList node contains other BList nodes that are part of the same data structure. Also, FWIW, the maximum height (and therefore recursion depth) of a BList on a 32-bit machine is 16 (assuming the maximum children per node is 128, as I have used). > Have you tried running this code under regrtest.py with the -R flag to > check for memory leaks? I hadn't heard of regrtest.py, so I re-invented the wheel. I ran the code through the list and sequence unit tests, as well as several additional unit tests that I wrote. For each test, I setup things up so that when compiled in debug mode it runs the test, counts the total number of references in the interpreter, runs the test again, and confirms that the total number of references have not changed. > There were C++ (//) style comments and C99 declarations of variables. > These would have to be changed if included in the core. Also, size_t > and Py_ssize_t should be used throughout. (ssize_t should be > Py_ssize_t). I noticed a bunch of ints and unsigneds. Although I > also noticed you used PyIndex_Check(). Is there a document that describes the right times to use int vs size_t vs Py_ssize_t? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From pje at telecommunity.com Tue Apr 24 13:17:22 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 24 Apr 2007 07:17:22 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <462DA108.8090307@acm.org> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070424071405.04effb30@sparrow.telecommunity.com> At 11:17 PM 4/23/2007 -0700, Talin wrote: >I'm not sure what Phillip is talking about here - C++ does not allow you >to instantiate a class that has non-overridden pure virtual methods, and >no pure virtual method can have a body. You cannot provide an >implementation for a pure virtual function. I've never done it myself, but at least these guys say you can: http://www.parashift.com/c++-faq-lite/abcs.html#faq-22.4 http://en.wikipedia.org/wiki/Virtual_function#Abstract_classes_and_pure_virtual_functions But clearly, it's pretty obscure. From pje at telecommunity.com Tue Apr 24 13:51:47 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 24 Apr 2007 07:51:47 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com> At 09:25 PM 4/23/2007 -0700, Guido van Rossum wrote: >On 4/23/07, Phillip J. Eby wrote: > > At 04:49 PM 4/23/2007 -0700, Guido van Rossum wrote: > > >>I suppose there is some value in requiring people to override @abstract > > >>methods to make a subclass instantiable, versus merely using the lack > of an > > >>@abstract class decorator to indicate that a subclass is concrete. But I > > >>wonder if being able to have just one @abstract decorator (that always > > >>means "you can't call this by default") mightn't be worth giving up that > > >>tiny bit of extra type checking? > > > > > >I prefer to follow the lead of C++ here -- an abstract class is > > >abstract by virtue of having at least one abstract method. > > > > It's been way too long since I did any C++, but isn't it the case that an > > abstract (aka "pure virtual"?) method in C++ is one that *can't be > > invoked*? [pause to Google it...] Well I'll be darned. I didn't know you > > could actually provide an implementation for a pure virtual > > function. Hm. Learn something new every day... except while I was > > writing C++ evidently. :) > >Actually I just meant to follow its lead in defining that a class is >abstract as soon as one or more of its methods are abstract. In Java >you need to repeat this by making the class abstract; that seems >unpythonic. Right - but as I understand it, Java and C# allow a class to be abstract without having *any* of its methods being abstract. I wasn't proposing that a redudnant class level @abstract be required; just noting that it seems useful to declare a class @abstract regardless of whether it has any abstract methods. >I don't see the point of having an abstract class without abstract methods. Well, by my definition of "abstract", none of the ABCs in the sandbox *have* any abstract methods (i.e. methods without a usable implementation), so apparently you *do* see the point. We just have different definitions of "abstract". :) Mine (courtesy of other languages) is, "a method without an implementation". Yours appears to be "a method that must be overridden". An abstract class, to me, is simply one that is not instantiable. Is that your definition as well? (By the way, I'm not continuing this thread to push for my definitions -- I'm seeking clarity so that your PEP can better explain just what it is that you mean to do. E.g. "While in other languages, abstract methods are X, Python abstract methods are Y". I'm clear now about what the *behavior* of Y is, but am trying to understand the *reasons* of Y.) >The main reasons I came up (without knowing C++ allows this too!) with >the idea of giving abstract methods a valid implementation wer (a) >there's got to be *something* in the body; (b) to provide an end point >for cooperative-MI-style code. But "b", as far as I can tell, doesn't have anything to do with abstractness as I understand it. If there's a useful end-point, why does it need to be abstract? What do we gain by making someone override it? It seems like making somebody jump through a Java-style typechecking hoop for no purpose. ISTM that "consenting adults" means I should be able to just use that code without overriding it, as long as there is something there to use. >Well, too bad. After a day of coding in Java I start typing curly >braces and semicolons too. But that doesn't mean Python should adopt >these. Of course not -- but those differences are something you're not likely to miss by skimming the docs! ;) My point here is to make sure that this gets called out clearly and boldly, since it is a Python-only concept that nonetheless is using a term that means something completely different in other languages. We might, perhaps, consider an alternative term for it than 'abstractmethod', since it is quite different from what anybody else means by "abstract method", as far as I can tell. > > >That the abstract methods are still somewhat useful implementations is > > >mostly to provide a valid (if not necessarily useful) end point for > > >super-calling in cooperative MI schemes. > > > > Right; I guess my point is that if those "somewhat" useful implementations > > are useful, they're useful, and there's no need to treat the corresponding > > class as "abstract" (in the "non-instantiable" sense) in that case. For > > example, I could pass an Iterable() to something that expected an iterable. > >Perhaps. Though I think it would defeat the purpose of being required >to provide an implementation. And what *is* the purpose of being required to provide an implementation, if a useful one exists? This is the bit I guess I'm still missing. >If forced to choose, I'd rather remove >the meaningful bodies (and replace them with "raise >NotImplementedError") than remove the @abstractmethod decorators. But >I prefer the current proposal. Right; so let's work on making the rationale for it a bit clearer. (i.e., I've already stopped arguing to change the behavior -- I just want to *understand* it better.) From ironfroggy at gmail.com Tue Apr 24 16:11:21 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Tue, 24 Apr 2007 10:11:21 -0400 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <462BDCAA.7020501@canterbury.ac.nz> <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> Message-ID: <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> I must have miscopied then because it worked perfectly here. Yes, I meant to have the _superdesc defined inside the metaclass __init__, but thought I could pull it out to make it cleaner. I forgot it actually had to be there! Here is the metaclass that works. class autosuper(type): def __init__(cls, name, bases, clsdict): class _superdesc(object): def __get__(self, obj, objcls): return super(cls, obj) cls.__super__ = _superdesc() On 4/24/07, Steven Bethard wrote: > On 4/23/07, Calvin Spealman wrote: > > I will +1 on the self.__super__ suggestion. Hey, its very doable and I > > even whipped it up with a simple metaclass, so it would be a tiny > > change to 'type' in order to actually implement it as a standard > > feature. The demonstration is as follows: > > > > class _superdesc(object): > > def __get__(self, obj, objcls): > > return super(cls, obj) > > > > class autosuper(type): > > def __init__(cls, name, bases, clsdict): > > cls.__super__ = _superdesc() > > > > class A(object): > > __metaclass__ = autosuper > > x = 1 > > > > class B(A): > > x = 2 > > > > assert B().__super__.x == 1 > > Does that really work? There's a typo in _superdesc (I don't know > where 'cls' comes from) but if you meant 'objcls', here's what I get:: > > >>> class _superdesc(object): > ... def __get__(self, obj, cls): > ... return super(cls, obj) > ... > >>> class autosuper(type): > ... def __init__(cls, name, bases, clsdict): > ... cls.__super__ = _superdesc() > ... > >>> class A: > ... __metaclass__ = autosuper > ... def f(self): > ... print 'A' > ... > >>> class B(A): > ... def f(self): > ... print 'B' > ... self.__super__.f() > ... > >>> class C(A): > ... def f(self): > ... print 'C' > ... self.__super__.f() > ... > >>> class D(B, C): > ... def f(self): > ... print 'D' > ... self.__super__.f() > ... > >>> D().f() > D > B > B > B > ... > Traceback (most recent call last): > File "", line 1, in > ... > RuntimeError: maximum recursion depth exceeded > > STeVe > -- > I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a > tiny blip on the distant coast of sanity. > --- Bucky Katt, Get Fuzzy > -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From emin.shopper at gmail.com Tue Apr 24 16:39:45 2007 From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper) Date: Tue, 24 Apr 2007 10:39:45 -0400 Subject: [Python-3000] An introduction to ABC's Message-ID: <32e43bb70704240739x53d80ff3j863e64d9a11b5f@mail.gmail.com> Dear Guido, Talin, and other Python Experts, I would like to register my strong support/encouragement for some type of abstract base class (ABC) mechanism in python. While I think dynamism, reflection, duck-typing, unit tests, etc. are wonderful, I often come across situations where ABCs are very useful. For example, when writing programs which take a long time to run (e.g., numerical work or large simulations), it is very painful to run a unit test that takes many minutes or even hours to run only to discover that a newly defined class is missing a crucial method. By using ABCs appropriately, I can make sure that all classes have the required methods as soon as a class is imported instead of part-way through the simulation. Below are links to the ABC implementation based on meta-classes and decorators which I have been using for quite a while. http://mail.python.org/pipermail/python-list/2006-December/419941.html http://alum.mit.edu/www/emin/source_code/py_abc/abc.py Some features of this module include: * Allows you to declare an ABC by inheriting from AbstractBaseClass. * Allows you to declare abstract methods using the @Abstract decorator. * Works with multiple inheritance (i.e., can inherit from multiple ABCs). * Allows you to turn off ABC checks on a per-class basis by setting klass.__allow_abstract__=True. * Allows partial ABCs (i.e., an ABC which inherits from another ABC but is also abstract itself) * Relatively simple implementation (only a few hundred lines of pure python including comments and doctests). * Documentation and doctests to illustrate/test features. Sincerely, -Emin Martinian -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070424/d538d5be/attachment.html From guido at python.org Tue Apr 24 18:52:26 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Apr 2007 09:52:26 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <462DA108.8090307@acm.org> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462DA108.8090307@acm.org> Message-ID: On 4/23/07, Talin wrote: > I think I am in (weak) agreement with Phillip here. In every language > that I am familiar with, the adjective "abstract" when applied to a > method implies two things: (1) that the method does not have a body, and > (2) the method must be implemented by a subclass if that subclass is > able to be instantiated. In fact, (2) is merely a consequence of (1), > when combined with the additional constraint that no instantiated class > can have methods with no body. > > However, I think what is going on is that you're using the word > "abstract" in a slightly difference sense. Normally, an abstract method > provides merely an interface, in other words it defines the method name, > argument types, and return type of a method - and nothing more. > > What you're doing, I think, is saying that an abstract method provides > all those things, and in addition it provides a kind of sketch or > outline of what the functionality ought to be. I don't have a problem > with that part so much. Right. That's was the original goal. > But in addition there is a sense that you also want these methods to be > useful helper functions for base classes to call, but you don't want > them to be called directly. In languages such as Java and C++, this is > what "protected" is for - methods that aren't available to the outside > world, but which can be called by subclasses. But Python doesn't have protected, and isn't about to get it. > So it seems to me -- at least from the standpoint of someone steeped in > the mindset of Java / C++ / C# et al -- that you are conflating > "abstract" and "protected". > > In what is considered "good style" for such languages, protected helper > methods ought to have different names than the methods which invoke > them. Unfortunately then the ABC would need to invent names for those helpers. > So for example, you might have the following: > > /// Abstract base class for hashable object > class Hashable { > public: > /// Declare abstract hash method > virtual int GetHash() const = 0; > > protected: > /// Protected helper method > int DefaultHashFunc() { return 0; } > }; > > // A concrete class derived from hashable > class MyConcreteClass : public Hashable { > public: > /// Declare abstract hash method > virtual int GetHash() const { return DefaultHashFunc(); } > }; > > Granted, in this particular case the decision to provide a helper > function is rather odd. Why go through the trouble of defining > DefaultHash() in the base class that simply returns 0, when any > non-trivial subclass will provide a real implementation of GetHash() > instead of merely forwarding the call to DefaultHash()? Hashable.__hash__ is probably not the best example of the idea; its implementation is valid but useless, so it really serves more as a "sketch or outline of the functionality". > But the solution isn't to give Hashable::GetHash() a body. From a C++ > programmer's view, there's only two choices: Either Hashable::GetHash() > is abstract, or it's not. If it's abstract, then every subclass is > required to re-implement it, and giving it a body has no purpose > whatsoever; If it's not, then subclasses are allowed to default to its > implementation. I'm not afraid of breaking new ground. Since the fact of the matter is that we can't prevent abstract methods from having a body (that would require syntactic support for abstract methods which is out of scope here) we might as well make good use of the body. There's also the use case of providing an end-point for cooperative super-calls, which I've brought up repeatedly but which nobody seems to pick up. Perhaps because cooperative super-calls are not supported by either Java or C++. But they do exist in other languages (Lisp, and the C++ extension described in "Putting Metaclasses to Work: A New Dimension in Object-Oriented Programming", by Ira R. Forman and Scott H. Danforth (which formed the basis of much of my thinking on metaclasses, and which I reference in "Unifying types and classes in Python 2.2": http://www.python.org/download/releases/2.2.3/descrintro/). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From daniel at stutzbachenterprises.com Tue Apr 24 19:18:15 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 24 Apr 2007 12:18:15 -0500 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462DA108.8090307@acm.org> Message-ID: On 4/24/07, Guido van Rossum wrote: > I'm not afraid of breaking new ground. Since the fact of the matter is > that we can't prevent abstract methods from having a body (that would > require syntactic support for abstract methods which is out of scope > here) we might as well make good use of the body. Sure, we could. class SomeABC: some_abstract_method = AbstractMethod where "AbstractMethod" is a singleton. This is similar to C++'s "= 0" syntax for pure virtual functions. Unfortunately, that would leave no place for docstrings, so I like the existing proposal better (@abstract). -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From jimjjewett at gmail.com Tue Apr 24 19:35:08 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 24 Apr 2007 13:35:08 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <462DA108.8090307@acm.org> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462DA108.8090307@acm.org> Message-ID: On 4/24/07, Talin wrote: > In what is considered "good style" for such languages, protected helper > methods ought to have different names than the methods which invoke > them. So for example, you might have the following: > /// Abstract base class for hashable object > class Hashable { > public: > /// Declare abstract hash method > virtual int GetHash() const = 0; > > protected: > /// Protected helper method > int DefaultHashFunc() { return 0; } > }; This is exactly why diamond inheritance is tricky. Try this with a cooperative method, such as save_state. def save_state(self): # Save my own state, then... # Do call the helper method if no one else will... # Do not call the helper method in case someone else does... The solution is to avoid making the "top" of the hierarchy a singleton. Calling it (through super) is OK. It might even be helpful, or it might be a no-op -- but it is safe to call through inheritance. The reason to mark it @abstract is to say that the implementation isn't complete yet. For hashable, you may never call super at all; for save_state, you should always call it. -jJ From steven.bethard at gmail.com Tue Apr 24 19:35:51 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 24 Apr 2007 11:35:51 -0600 Subject: [Python-3000] Fixing super anyone? In-Reply-To: <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> References: <462BDCAA.7020501@canterbury.ac.nz> <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> Message-ID: On 4/24/07, Calvin Spealman wrote: > I must have miscopied then because it worked perfectly here. Yes, I > meant to have the _superdesc defined inside the metaclass __init__, > but thought I could pull it out to make it cleaner. I forgot it > actually had to be there! Here is the metaclass that works. > > class autosuper(type): > def __init__(cls, name, bases, clsdict): > class _superdesc(object): > def __get__(self, obj, objcls): > return super(cls, obj) > cls.__super__ = _superdesc() I still get the same error. Take a look at what the 'self' and '__super__' objects are at each level:: >>> class A: ... __metaclass__ = autosuper ... def f(self): ... print 'A', self ... >>> class B(A): ... def f(self): ... print 'B', self, self.__super__ ... self.__super__.f() ... >>> class C(A): ... def f(self): ... print 'C', self, self.__super__ ... self.__super__.f() ... >>> class D(B, C): ... def f(self): ... print 'D', self, self.__super__ ... self.__super__.f() ... >>> D().f() D <__main__.D object at 0x00E88530> , > B <__main__.D object at 0x00E88530> , > B <__main__.D object at 0x00E88530> , > ... Traceback (most recent call last): File "", line 1, in File "", line 4, in f File "", line 4, in f ... RuntimeError: maximum recursion depth exceeded >>> Notice that the 'self' object is always the 'D()' object, and therefore the '__super__' object is always the 'B' wrapper. If the 'self' object were first 'D()', then 'D().__super__', then 'D().__super__.__super__', etc. your code might work:: >>> D().__super__ , > >>> D().__super__.__super__ , > >>> D().__super__.__super__.__super__ , > >>> D().__super__.__super__.__super__.__super__ , > STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From brett at python.org Tue Apr 24 19:40:11 2007 From: brett at python.org (Brett Cannon) Date: Tue, 24 Apr 2007 10:40:11 -0700 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: References: Message-ID: On 4/24/07, Daniel Stutzbach wrote: [SNIP] > Is there a document that describes the right times to use int vs > size_t vs Py_ssize_t? Try PEP 353: http://www.python.org/dev/peps/pep-0353/ -Brett From guido at python.org Tue Apr 24 19:50:16 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Apr 2007 10:50:16 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com> Message-ID: On 4/24/07, Phillip J. Eby wrote: > Right - but as I understand it, Java and C# allow a class to be abstract > without having *any* of its methods being abstract. I wasn't proposing > that a redudnant class level @abstract be required; just noting that it > seems useful to declare a class @abstract regardless of whether it has any > abstract methods. OK. It's a minor feature, but I have a use case: I've seen code that says raise Exception("some error message") and that's really a mistake -- if they are to lazy to define their own exception subclass they should raise RuntimeError() or another appropriate standard error. So if we had this feature I'd make BaseException, Exception, StandardError, EnvironmentError and Warning abstract (or at least the first 2-3 of those). > >I don't see the point of having an abstract class without abstract methods. > > Well, by my definition of "abstract", none of the ABCs in the sandbox > *have* any abstract methods (i.e. methods without a usable implementation), > so apparently you *do* see the point. We just have different definitions > of "abstract". :) In my definition they are abstract if they have an @abstractmethod decorator. :) > Mine (courtesy of other languages) is, "a method without an > implementation". Yours appears to be "a method that must be > overridden". That's a very subtle distinction, since the *goal* in other languages is certainly to force it to be overridden. > An abstract class, to me, is simply one that is not > instantiable. Is that your definition as well? As the PEP stands today it's a class that has at least one abstract method (that isn't overridden). > (By the way, I'm not continuing this thread to push for my definitions -- > I'm seeking clarity so that your PEP can better explain just what it is > that you mean to do. E.g. "While in other languages, abstract methods are > X, Python abstract methods are Y". I'm clear now about what the *behavior* > of Y is, but am trying to understand the *reasons* of Y.) Frankly, I don't care all that much about the reasons -- you can read into it what you want. I am just trying to define mechanisms that I think are useful. > >The main reasons I came up (without knowing C++ allows this too!) with > >the idea of giving abstract methods a valid implementation wer (a) > >there's got to be *something* in the body; (b) to provide an end point > >for cooperative-MI-style code. > > But "b", as far as I can tell, doesn't have anything to do with > abstractness as I understand it. If there's a useful end-point, why does > it need to be abstract? It doesn't have to be. Being a useful end-point and being abstract are separate concepts. They are intertwined only because IMO abstract-ness shouldn't prevent something from being a useful end-point. The cooperative-MI theory makes a big distinction between defining and overriding of methods; abstract methods (if not by definition then by convention) are defining new methods, not overriding them, so that pretty much requires them to be useful end-points -- but being an end-point doesn't mean they can't be abstract. > What do we gain by making someone override it? It > seems like making somebody jump through a Java-style typechecking hoop for > no purpose. ISTM that "consenting adults" means I should be able to just > use that code without overriding it, as long as there is something there to > use. The only practical concrete use I see of these abstract classes would be to have a handy "empty sequence/set/mapping/iterator/etc." implementation. Since an empty list/set/dict/etc. serves just as well for that purpose I don't think this is a strong use case. What I'm *trying* to do with the @abstractmethod decorator is telling potential implementors of these classes: the minimum set of operations you need to provide in order to be an X is this one (or these two or three) methods (the ones that have the @abstractmethod decorator). > >Well, too bad. After a day of coding in Java I start typing curly > >braces and semicolons too. But that doesn't mean Python should adopt > >these. > > Of course not -- but those differences are something you're not likely to > miss by skimming the docs! ;) My point here is to make sure that this > gets called out clearly and boldly, since it is a Python-only concept that > nonetheless is using a term that means something completely different in > other languages. We might, perhaps, consider an alternative term for it > than 'abstractmethod', since it is quite different from what anybody else > means by "abstract method", as far as I can tell. I don't expect confusion (except from folks who tend to over-analyze things), since the practical consequences are pretty much the same. You could happily ignore the fact that some abstract methods are callable (via super). Remember Python makes a business out of taking concepts and terms from other languages and giving them new meaning. E.g. assignment -> name binding, classes and imports as run-time features, etc. Python's MI is another big one. > And what *is* the purpose of being required to provide an implementation, > if a useful one exists? This is the bit I guess I'm still missing. What is the point of claiming to be a Sequence *without* overriding __getitem__? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jimjjewett at gmail.com Tue Apr 24 20:00:18 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 24 Apr 2007 14:00:18 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com> Message-ID: On 4/24/07, Phillip J. Eby wrote: > At 09:25 PM 4/23/2007 -0700, Guido van Rossum wrote: > >I don't see the point of having an abstract class without abstract methods. Phillip is used to writing classes with (unbased) mutally recursive functions. Concrete example: class Eq(object): def __eq__(self, other): return not self.__ne__(other) def __ne__(self, other): return not self.__eq__(other) There isn't a single method that needs to be overridden, so he doesn't want to mark either method as abstract. You need to override one of them, so he would like to mark the class abstract. > Mine (courtesy of other languages) is, "a method without an > implementation". Yours appears to be "a method that must be > overridden". + " or extended" I still call it overriding, but Guido sometimes distinguishes the two -- and methods which must be extended are important cooperative super calls. > >... (b) to provide an end point for cooperative-MI-style code. > But "b", as far as I can tell, doesn't have anything to do with > abstractness as I understand it. If there's a useful end-point, why does > it need to be abstract? Usefor for cooperative-MI-style might mean "useful to the programmer, who doesn't need to add boilerplate catching the exception just in case his class was the last *concrete* class" Though in this case, I think he is also referring to partial implementations. Look at Sequence.__getitem__ in http://svn.python.org/view/sandbox/trunk/abc/abc.py?rev=54894&view=markup An implementation does need to override the method, but the override could just be an extension that handles only integer indices, and defers slice indices. -jJ From jimjjewett at gmail.com Tue Apr 24 20:22:27 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 24 Apr 2007 14:22:27 -0400 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <462BDCAA.7020501@canterbury.ac.nz> <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> Message-ID: On 4/24/07, Steven Bethard wrote: > > class autosuper(type): > I still get the same error. No doubt it is fiddly, but he did remind me to look at some of the older recipes. In the stdlib test\test_descr.py (inside function metaclass), there is an autosuper class that comes pretty close to what we want. I'm not sure we can just add it to type, though, because it relies on name mangling. self.__super.meth() I think for default behavior, we want either either no underscores, or matched underscores. # too magical? self.super.meth() self.super().meth() # Should a double_underscore method ever "normally" need to be called # by name, outside of another double_underscore method? self.__super__ .meth() # __attribute__ rather than method __this_class__.super.meth() -jJ From pje at telecommunity.com Tue Apr 24 20:35:10 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 24 Apr 2007 14:35:10 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com> <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070424141334.02ed6d00@sparrow.telecommunity.com> At 10:50 AM 4/24/2007 -0700, Guido van Rossum wrote: >On 4/24/07, Phillip J. Eby wrote: >>Mine (courtesy of other languages) is, "a method without an >>implementation". Yours appears to be "a method that must be >>overridden". > >That's a very subtle distinction, since the *goal* in other languages >is certainly to force it to be overridden. But this just goes right back around to the original point... if you *have* an implementation, you don't really *need* to force it to be overridden. Or do you? As far as I can tell, you still haven't said what the point of *forcing* somebody to override perfectly usable methods is. You've merely said you don't see the point of inheriting or using them directly, but that's not the same thing as a rationale for *forcing* them not to. >Being a useful end-point and being abstract >are separate concepts. They are intertwined only because IMO >abstract-ness shouldn't prevent something from being a useful >end-point. But that's the very definition of "abstract" -- i.e., that it's *not* useful. >The only practical concrete use I see of these abstract classes would >be to have a handy "empty sequence/set/mapping/iterator/etc." >implementation. Since an empty list/set/dict/etc. serves just as well >for that purpose I don't think this is a strong use case. That's certainly true for the Python-supplied ABCs, but it's not so for many other use cases. I frequently write base classes that have various default implementations, combined with a few methods that have to be overridden. On the other hand, specifying which methods >What I'm *trying* to do with the @abstractmethod decorator is telling >potential implementors of these classes: the minimum set of operations >you need to provide in order to be an X is this one (or these two or >three) methods (the ones that have the @abstractmethod decorator). Then let's call it @required or @requiredmethod, and all is well. :) I've just been trying to get clarity on your intent, so I could suggest a name that reflects that intent. At this point, it seems clear your intent is to say, "this method is required in a subclass, even if there's an implementation here". So let's have a name that reflects that intention. (I still don't see the point of marking a method as required if it has an actual implementation, since to my way of thinking there is no harm in someone *not* overriding it. They simply don't get as rich of a behavior in that case. But that's a separate issue.) >>And what *is* the purpose of being required to provide an implementation, >>if a useful one exists? This is the bit I guess I'm still missing. > >What is the point of claiming to be a Sequence *without* overriding >__getitem__? I didn't say there was a point to that. However, as Jim Jewett pointed out, you can have mutually recursive default implementations of a protocol, and probably at least *one* of the methods needs to be overridden. Really, the patterns of what needs to be overridden in subclasses is complex enough that we don't really want to get into a bunch of ways to specify how those things should be overridden. That's why I don't see the "make sure you override everything" testing as being all that useful; it seems too much like adding empty methods and extra "try" blocks in Java just to get your code past the compiler. From lcaamano at gmail.com Tue Apr 24 20:56:21 2007 From: lcaamano at gmail.com (Luis P Caamano) Date: Tue, 24 Apr 2007 14:56:21 -0400 Subject: [Python-3000] Generic function PEP won't make it in time Message-ID: On 4/24/07, "Guido van Rossum" wrote: > > > > But Python doesn't have protected, and isn't about to get it. > We use methods names with one underscore as a convention for protected methods and I think we learned that from somebody else: class Eggs: def thePublicMethod(self): things = self._myProtected() return [ "thing_%s" % str(thing) for thing in things] def _myProtected(self): raise RuntimeError("missing base method") class RealEggs(AbstractEggs): def _myProtected(self): return ["one", "two"] -- Luis P Caamano Atlanta, GA USA From guido at python.org Tue Apr 24 21:00:59 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Apr 2007 12:00:59 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <5.1.1.6.0.20070424141334.02ed6d00@sparrow.telecommunity.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com> <5.1.1.6.0.20070424141334.02ed6d00@sparrow.telecommunity.com> Message-ID: Let's agree to disagree. I am getting tired from going around in circles about this. On 4/24/07, Phillip J. Eby wrote: > At 10:50 AM 4/24/2007 -0700, Guido van Rossum wrote: > >On 4/24/07, Phillip J. Eby wrote: > >>Mine (courtesy of other languages) is, "a method without an > >>implementation". Yours appears to be "a method that must be > >>overridden". > > > >That's a very subtle distinction, since the *goal* in other languages > >is certainly to force it to be overridden. > > But this just goes right back around to the original point... if you > *have* an implementation, you don't really *need* to force it to be > overridden. Or do you? > > As far as I can tell, you still haven't said what the point of *forcing* > somebody to override perfectly usable methods is. You've merely said you > don't see the point of inheriting or using them directly, but that's not > the same thing as a rationale for *forcing* them not to. > > > >Being a useful end-point and being abstract > >are separate concepts. They are intertwined only because IMO > >abstract-ness shouldn't prevent something from being a useful > >end-point. > > But that's the very definition of "abstract" -- i.e., that it's *not* useful. > > > >The only practical concrete use I see of these abstract classes would > >be to have a handy "empty sequence/set/mapping/iterator/etc." > >implementation. Since an empty list/set/dict/etc. serves just as well > >for that purpose I don't think this is a strong use case. > > That's certainly true for the Python-supplied ABCs, but it's not so for > many other use cases. I frequently write base classes that have various > default implementations, combined with a few methods that have to be > overridden. On the other hand, specifying which methods > > > >What I'm *trying* to do with the @abstractmethod decorator is telling > >potential implementors of these classes: the minimum set of operations > >you need to provide in order to be an X is this one (or these two or > >three) methods (the ones that have the @abstractmethod decorator). > > Then let's call it @required or @requiredmethod, and all is well. :) I've > just been trying to get clarity on your intent, so I could suggest a name > that reflects that intent. At this point, it seems clear your intent is to > say, "this method is required in a subclass, even if there's an > implementation here". So let's have a name that reflects that intention. > > (I still don't see the point of marking a method as required if it has an > actual implementation, since to my way of thinking there is no harm in > someone *not* overriding it. They simply don't get as rich of a behavior > in that case. But that's a separate issue.) > > > >>And what *is* the purpose of being required to provide an implementation, > >>if a useful one exists? This is the bit I guess I'm still missing. > > > >What is the point of claiming to be a Sequence *without* overriding > >__getitem__? > > I didn't say there was a point to that. However, as Jim Jewett pointed > out, you can have mutually recursive default implementations of a protocol, > and probably at least *one* of the methods needs to be overridden. > > Really, the patterns of what needs to be overridden in subclasses is > complex enough that we don't really want to get into a bunch of ways to > specify how those things should be overridden. That's why I don't see the > "make sure you override everything" testing as being all that useful; it > seems too much like adding empty methods and extra "try" blocks in Java > just to get your code past the compiler. > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Tue Apr 24 21:54:22 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 24 Apr 2007 15:54:22 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <5.1.1.6.0.20070424141334.02ed6d00@sparrow.telecommunity.com> <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com> <5.1.1.6.0.20070424141334.02ed6d00@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070424154919.02d9b5d0@sparrow.telecommunity.com> At 12:00 PM 4/24/2007 -0700, Guido van Rossum wrote: >Let's agree to disagree. I am getting tired from going around in >circles about this. Does that mean you're -1 on renaming @abstractmethod to @required or @requiredmethod, or did you not see that proposal? Other than that question, I'm done on this also. For that matter, I'll assume a non-reply means -1, so there's no need to reply if you did in fact see it. From janssen at parc.com Wed Apr 25 01:30:37 2007 From: janssen at parc.com (Bill Janssen) Date: Tue, 24 Apr 2007 16:30:37 PDT Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com> Message-ID: <07Apr24.163038pdt."57996"@synergy1.parc.xerox.com> Just to chime in... My preference would be to have "abstract" just mark classes that define an API, that define the "abstraction" of that particular interface. Not carry over meanings from Java and C++ that "abstract" somehow always means "partial" or "incomplete". I'd envision having "abstract" classes that are in fact also useful and complete implementations. "Abstract" methods don't really make sense (to me) in Python. This sounds very close to what Guido has already. Maybe "abstraction" is a better keyword than "abstract". Maybe it makes sense to have "abstraction" classes, and an "@ignorethismethod" decorator to mark methods that are *not* part of the abstraction, instead of "@abstractmethod" decorators to mark the methods that *are* part of the abstraction. Bill From guido at python.org Wed Apr 25 02:01:10 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Apr 2007 17:01:10 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <3003523447303727199@unknownmsgid> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com> <3003523447303727199@unknownmsgid> Message-ID: On 4/24/07, Bill Janssen wrote: > Just to chime in... > > My preference would be to have "abstract" just mark classes that > define an API, that define the "abstraction" of that particular > interface. Not carry over meanings from Java and C++ that "abstract" > somehow always means "partial" or "incomplete". I'd envision having > "abstract" classes that are in fact also useful and complete > implementations. "Abstract" methods don't really make sense (to me) > in Python. This sounds very close to what Guido has already. > > Maybe "abstraction" is a better keyword than "abstract". > > Maybe it makes sense to have "abstraction" classes, and an > "@ignorethismethod" decorator to mark methods that are *not* part of > the abstraction, instead of "@abstractmethod" decorators to mark the > methods that *are* part of the abstraction. Hm, but sometimes characteristic methods are abstract, and in other cases they are concrete. For exampe: class Iterator(Iterable): @abstract def __next__(self): raise StopIteration def __iter__(self): # This is abstract in Iterable, but concrete here return self Both methods are part of the ABC. But only __next__ is abstract. Once could define a perfectly valid empty iterator like this: class EmptyIterator(Iterator): def __next__(self): return super(EmptyIterator, self).__next__() -- --Guido van Rossum (home page: http://www.python.org/~guido/) From janssen at parc.com Wed Apr 25 02:23:51 2007 From: janssen at parc.com (Bill Janssen) Date: Tue, 24 Apr 2007 17:23:51 PDT Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com> <3003523447303727199@unknownmsgid> Message-ID: <07Apr24.172353pdt."57996"@synergy1.parc.xerox.com> > > Maybe "abstraction" is a better keyword than "abstract". > > > > Maybe it makes sense to have "abstraction" classes, and an > > "@ignorethismethod" decorator to mark methods that are *not* part of > > the abstraction, instead of "@abstractmethod" decorators to mark the > > methods that *are* part of the abstraction. > > class Iterator(Iterable): > @abstract > def __next__(self): > raise StopIteration > def __iter__(self): # This is abstract in Iterable, but concrete here > return self > > Both methods are part of the ABC. But only __next__ is abstract. Once > could define a perfectly valid empty iterator like this: If I'm following you correctly, you are saying that "Iterator" is an abstraction class, but the only method which is part of the abstraction is "__next__". So it could be marked up as @abstraction class Iterator(Iterable): @abstractionmethod def __next__(self): raise StopIteration def __iter__(self): # This is abstract in Iterable, but concrete here return self or @abstraction class Iterator(Iterable): def __next__(self): raise StopIteration @ignorethismethod def __iter__(self): # This is abstract in Iterable, but concrete here return self "Iterator" inherits "__iter__" from "Iterable", so I think I prefer the idea of explicitly marking the methods which form part of the abstraction (the first form), because it gives you a chance to define new methods with the same name: @abstraction class Iterator(Iterable): @abstractionmethod def __next__(self): raise StopIteration @abstractionmethod def __iter__(self, n): # This is *not* the __iter__ from Iterable, but a new method return self * n Bill From greg.ewing at canterbury.ac.nz Wed Apr 25 03:05:07 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 25 Apr 2007 13:05:07 +1200 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: References: <3d2ce8cb0704231142n4448fd0apb3eb7f3dd350b5a3@mail.gmail.com> Message-ID: <462EA943.1020403@canterbury.ac.nz> Daniel Stutzbach wrote: > I'm not really sure what variance bars would reveal. > I can testify that when I rerun the experiments with the same > code, the results would look *very* similar. If you're running real-world timing tests, then there will be some variance in the results, so you need to plot error bars. If the variance is indeed small, as you claim, then the error bars will show that. -- Greg From tcdelaney at optusnet.com.au Wed Apr 25 03:13:05 2007 From: tcdelaney at optusnet.com.au (Tim Delaney) Date: Wed, 25 Apr 2007 11:13:05 +1000 Subject: [Python-3000] Fixing super anyone? Message-ID: <003501c786d6$e222e710$0201a8c0@ryoko> I've been off sick from work for over a week - I've been following this discussion, but now I think I've got something which matches all the criteria we've been discussing, so I've changed my subscription address temporarily to my home one. It involves bytecode hacking for now, as well as a metaclass. The code is attached. It's based partially off my 'self.super' recipe, but doesn't need to use sys._getframe(). Recipe: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286195 Basically, the metaclass modifies the bytecode to include some setup code to any function that uses 'super'. The setup code generates the correct 'super' object, and stores it as a local variable. All uses of LOAD_GLOBAL(super) are changed to LOAD_FAST(super). The one thing I would like to add is something I've got in my 'self.super' recipe - I've got my own 'super' object that is callable. Calling it invokes the super method of the current method i.e. if you've got the following code: class A(autosuper): def f(self): pass class B(A): def f(self): super() it would be functionally identical to the following: class B(A): def f(self): super.f() This could be easily implemented by having the 'super' constructor take a 'name' parameter (which would be passed the name of current method in the setup code). Then when super() is called, it would perform the name lookup. Tim Delaney -------------- next part -------------- A non-text attachment was scrubbed... Name: autosuper.py Type: application/octet-stream Size: 5330 bytes Desc: not available Url : http://mail.python.org/pipermail/python-3000/attachments/20070425/6b9124d7/attachment-0001.obj From tcdelaney at optusnet.com.au Wed Apr 25 03:20:54 2007 From: tcdelaney at optusnet.com.au (Tim Delaney) Date: Wed, 25 Apr 2007 11:20:54 +1000 Subject: [Python-3000] Fixing super anyone? References: <003501c786d6$e222e710$0201a8c0@ryoko> Message-ID: <004701c786d7$f8f22d60$0201a8c0@ryoko> From: "Tim Delaney" > I've been off sick from work for over a week - I've been following this > discussion, but now I think I've got something which matches all the > criteria we've been discussing, so I've changed my subscription address > temporarily to my home one. > > It involves bytecode hacking for now, as well as a metaclass. The code is > attached. It's based partially off my 'self.super' recipe, but doesn't > need > to use sys._getframe(). Oh - some output ... >>> class autosuper(object): ... __metaclass__ = _autosuper ... >>> class A(autosuper): ... def f(self): ... print 'A:', super ... >>> class B(A): ... def f(self): ... print 'B:', super ... super.f() ... >>> class C(A): ... def f(self): ... print 'C:', super ... super.f() ... >>> class D(B, C): ... def f(self): ... print 'D:', super ... super.f() ... >>> D().f() D: , > B: , > C: , > A: , > Tim Delaney From greg.ewing at canterbury.ac.nz Wed Apr 25 04:06:02 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 25 Apr 2007 14:06:02 +1200 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> Message-ID: <462EB78A.207@canterbury.ac.nz> Do we really need to have B&D-ish enforcement of abstract method implementation? It doesn't seem pythonic to me. I might want to leave some methods of an ABC unimplemented because I'm not intending to use them. I'd be annoyed if I were prevented from doing that because of some theoretical notion that they "should" be implemented. -- Greg From steven.bethard at gmail.com Wed Apr 25 04:29:20 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 24 Apr 2007 20:29:20 -0600 Subject: [Python-3000] adding @abstractmethods after class creation Message-ID: On 4/24/07, guido.van.rossum wrote: > +We define a new built-in decorator, ``@abstractmethod`` [snip] > +**Implementation:** The ``@abstractmethod`` decorator sets the > +function attribute ``__isabstractmethod__`` to the value ``True``. > +The ``type.__new__`` method computes the type attribute > +``__abstractmethods__`` as the set of all method names that have an > +``__isabstractmethod__`` attribute whose value is true. It does this > +by combining the ``__abstractmethods__` attributes of the base > +classes, adding the names of all methods in the new class dict that > +have a true ``__isabstractmethod__`` attribute, and removing the names > +of all methods in the new class dict that don't have a true > +``__isabstractmethod__`` attribute. If the resulting > +``__abstractmethods__`` set is non-empty, the class is considered > +abstract, and attempts to instantiate it will raise ``TypeError``. (Hope this wasn't covered in that really long thread. I couldn't see an answer in the PEP.) So what happens in a situation like:: class C: pass C.foo = abstractmethod(foo) It seems from the description above like ``C.__abstractmethods__`` would be empty, and therefore C could be instantiated even though it has an abstract method. I don't mind at all if we just say "don't do that", but then @abstractmethod should probably be documented explicitly as only being usable within a class body. (Note that this is different from @classmethod and @staticmethod which can be used after the fact.) I guess the other option would be to have __setattr__ on classes append to the __abstractmethods__ list when passed a value with __isabstractmethod__ == True. But this would probably make setting attributes on classes (though not on other objects) slower. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From ironfroggy at gmail.com Wed Apr 25 04:42:26 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Tue, 24 Apr 2007 22:42:26 -0400 Subject: [Python-3000] adding @abstractmethods after class creation In-Reply-To: References: Message-ID: <76fd5acf0704241942o19d24029q7472713fcf7b05d3@mail.gmail.com> On 4/24/07, Steven Bethard wrote: > On 4/24/07, guido.van.rossum wrote: > > +We define a new built-in decorator, ``@abstractmethod`` > [snip] > > +**Implementation:** The ``@abstractmethod`` decorator sets the > > +function attribute ``__isabstractmethod__`` to the value ``True``. > > +The ``type.__new__`` method computes the type attribute > > +``__abstractmethods__`` as the set of all method names that have an > > +``__isabstractmethod__`` attribute whose value is true. It does this > > +by combining the ``__abstractmethods__` attributes of the base > > +classes, adding the names of all methods in the new class dict that > > +have a true ``__isabstractmethod__`` attribute, and removing the names > > +of all methods in the new class dict that don't have a true > > +``__isabstractmethod__`` attribute. If the resulting > > +``__abstractmethods__`` set is non-empty, the class is considered > > +abstract, and attempts to instantiate it will raise ``TypeError``. > > (Hope this wasn't covered in that really long thread. I couldn't see > an answer in the PEP.) > > So what happens in a situation like:: > > class C: > pass > C.foo = abstractmethod(foo) > > It seems from the description above like ``C.__abstractmethods__`` > would be empty, and therefore C could be instantiated even though it > has an abstract method. > > I don't mind at all if we just say "don't do that", but then > @abstractmethod should probably be documented explicitly as only being > usable within a class body. (Note that this is different from > @classmethod and @staticmethod which can be used after the fact.) > > I guess the other option would be to have __setattr__ on classes > append to the __abstractmethods__ list when passed a value with > __isabstractmethod__ == True. But this would probably make setting > attributes on classes (though not on other objects) slower. > > STeVe > -- > I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a > tiny blip on the distant coast of sanity. > --- Bucky Katt, Get Fuzzy > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/ironfroggy%40gmail.com > Or could __abstractmethods__ be created dynamically? Which would be used less, __abstractmethods__ or setting class attributes, that it would be OK to be a bit slower? -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From steven.bethard at gmail.com Wed Apr 25 04:56:10 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 24 Apr 2007 20:56:10 -0600 Subject: [Python-3000] adding @abstractmethods after class creation In-Reply-To: <76fd5acf0704241942o19d24029q7472713fcf7b05d3@mail.gmail.com> References: <76fd5acf0704241942o19d24029q7472713fcf7b05d3@mail.gmail.com> Message-ID: On 4/24/07, Calvin Spealman wrote: > Or could __abstractmethods__ be created dynamically? Which would be > used less, __abstractmethods__ or setting class attributes, that it > would be OK to be a bit slower? I considered that, but __abstractmethods__ gets checked at every class instantiation (as I understand it). That means that __abstractmethods__ has to be fast. I assumed that was why Guido introduced it in the first place... STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From greg.ewing at canterbury.ac.nz Wed Apr 25 04:58:38 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 25 Apr 2007 14:58:38 +1200 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com> Message-ID: <462EC3DE.4010407@canterbury.ac.nz> Guido van Rossum wrote: > So if we had this feature I'd make > BaseException, Exception, StandardError, EnvironmentError and Warning > abstract (or at least the first 2-3 of those). I'm not sure I follow the reasoning here. I've just written some code which raises EnvironmentError in numerous places. This works perfectly well for the application at hand, because I don't need to distinguish between them any more finely. If I need to do so later, I'll have to make some changes -- but that would happen anyway even if I had created a subclass of EnvironmentError such as MyEnvironmentError at the outset. For library code, it's probably advisable not to use the standard exception classes directly -- but I don't think that guideline applies to *all* code. -- Greg From guido at python.org Wed Apr 25 06:29:22 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Apr 2007 21:29:22 -0700 Subject: [Python-3000] adding @abstractmethods after class creation In-Reply-To: References: <76fd5acf0704241942o19d24029q7472713fcf7b05d3@mail.gmail.com> Message-ID: On 4/24/07, Steven Bethard wrote: > On 4/24/07, Calvin Spealman wrote: > > Or could __abstractmethods__ be created dynamically? Which would be > > used less, __abstractmethods__ or setting class attributes, that it > > would be OK to be a bit slower? > > I considered that, but __abstractmethods__ gets checked at every class > instantiation (as I understand it). That means that > __abstractmethods__ has to be fast. I assumed that was why Guido > introduced it in the first place... Check the patch on SF (referenced by the PEP now). The instantiation code only checks a bit in tp_flags, which is set at class creation time iff __abstractmethods__ is non-empty. I'm not sure what the point would be of supporting dynamically modifying ABCs -- it looks like you're just exploring the corners of the semantics rather than providing actual use cases. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Wed Apr 25 06:50:54 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 24 Apr 2007 22:50:54 -0600 Subject: [Python-3000] adding @abstractmethods after class creation In-Reply-To: References: <76fd5acf0704241942o19d24029q7472713fcf7b05d3@mail.gmail.com> Message-ID: On 4/24/07, Guido van Rossum wrote: > I'm not sure what the point would be of supporting dynamically > modifying ABCs -- it looks like you're just exploring the corners of > the semantics rather than providing actual use cases. Yep, I'm just checking the corner case semantics. Like I said:: I don't mind at all if we just say "don't do that", but then @abstractmethod should probably be documented explicitly as only being usable within a class body. (Note that this is different from @classmethod and @staticmethod which can be used after the fact.) I just want to make sure it's intended and documented. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From steven.bethard at gmail.com Wed Apr 25 07:26:37 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 24 Apr 2007 23:26:37 -0600 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <462BDCAA.7020501@canterbury.ac.nz> <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> Message-ID: On 4/24/07, Steven Bethard wrote: > On 4/24/07, Calvin Spealman wrote: > > I must have miscopied then because it worked perfectly here. Yes, I > > meant to have the _superdesc defined inside the metaclass __init__, > > but thought I could pull it out to make it cleaner. I forgot it > > actually had to be there! Here is the metaclass that works. > > > > class autosuper(type): > > def __init__(cls, name, bases, clsdict): > > class _superdesc(object): > > def __get__(self, obj, objcls): > > return super(cls, obj) > > cls.__super__ = _superdesc() > > I still get the same error. I played around with this for a while. Below is some code that does work. The trick is to redefine super() so that when you're calling __get__ to bind a method to an instance, you bind it to the Super() instance, not the original object instance. class Super(object): def __init__(self, type, obj=None): if isinstance(obj, Super): obj = obj.__obj__ self.__type__ = type self.__obj__ = obj def __get__(self, obj, cls=None): if obj is None: raise Exception('only supports instances') else: return Super(self.__type__, obj) def __getattr__(self, attr): mro = iter(self.__obj__.__class__.__mro__) for cls in mro: if cls is self.__type__: break for cls in mro: if attr in cls.__dict__: x = cls.__dict__[attr] if hasattr(x, '__get__'): x = x.__get__(self, cls) return x raise AttributeError, attr class autosuper(type): def __init__(cls, name, bases, clsdict): cls.__super__ = Super(cls) class A: __metaclass__ = autosuper def f(self): return 'A' class B(A): def f(self): return 'B' + self.__super__.f() class C(A): def f(self): return 'C' + self.__super__.f() class D(B, C): def f(self): return 'D' + self.__super__.f() assert D().f() == 'DBCA' See ma, no bytecode hacks! ;-) STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From nnorwitz at gmail.com Wed Apr 25 07:48:37 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 24 Apr 2007 22:48:37 -0700 Subject: [Python-3000] An introduction to ABC's In-Reply-To: <32e43bb70704240739x53d80ff3j863e64d9a11b5f@mail.gmail.com> References: <32e43bb70704240739x53d80ff3j863e64d9a11b5f@mail.gmail.com> Message-ID: On 4/24/07, Emin.shopper Martinian.shopper wrote: > > Below are links to the ABC implementation based on meta-classes and > decorators which I have been using for quite a while. Emin, You can get similar functionality by using tools such as pychecker. PyLint and pyflakes may also have this functionality. If a method's body is: raise NotImplemenedError and a class with such a method is instantiated, a warning will be generated. Cheers, n From python at rcn.com Wed Apr 25 08:04:27 2007 From: python at rcn.com (Raymond Hettinger) Date: Tue, 24 Apr 2007 23:04:27 -0700 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical References: Message-ID: <00c601c786ff$95839700$f101a8c0@RaymondLaptop1> [Daniel Stutzbach] > As a pet project, I've written a container type that looks, acts, and > quacks like a Python list(), but has better asymptotic performance. > Specifically, O(log n) time for inserts and deletes in the middle of > the list. I'd like to offer it for inclusion in the collections > module. In principle, I think this new container would be a fine addition to the collections module. That is a perfect place for alternate implementations of standard types. By matching the list-api, you've reduced the learning curve to near zero while still offering intermediate-level users an alternative data structure with different performance trade-offs. In practice, the code for BList is somewhat complex, and its desirability and performance in actual applications is unproven. Fortunately, Py2.6 is still a long way off. My recommendation is that you release it right away as a third-party module and let the hordes of Pythonistas test it in battle. With the code base thoroughly exercised and some positive user feedback, it would be hard to say no to this going into Py2.6. > I'm open to suggestions for a better name. I'm hoping for a better name too. Hope you enjoy the impending naming debate -- very few people will read your code, more than a few will read the proposal, but *everyone* will have an opinion on the name ;-) Perhaps start-off with something romantic and enigmatic like "treelist" ;-) > - just as fast as a Python list() when the list is small > - getslice runs in O(log n) time > - making a shallow copy runs in O(1) time > - setslice runs in O(log n + log k) time if the inserted slice is a > BList of length k > - multipling a BList by k takes O(log k) time and O(log k) memory How does BList's memory consumption compare with the current list over-allocation scheme? Does it take longer to instantiate an emtpy list? What are the malloc/free patterns? > In other words, for short lists, > a BList works just like Python's array-based list() type. Thus, it > has the same good performance on small lists. While this is a good implementation idea (similar ideas are used to speed-up quicksort for instance), it makes the first listed advantage seem like a straw-man designed to mislead proposal reviewers into thinking that the blist structure is more efficient than is actually is. Perhaps rewrite it as: "To keep the performance acceptable for short lists, btreelists fall back to the existing implementation." > The Radical Proposal > Replace list() with the BList. FWIW, I prefer that the current list implementation remain in-place. Because what we have is simple, it is not hard to maintain, its performance characteristics are easily understood, and we have a straight-forward C-API that allows module writers to have fast, intuitive direct access to the structure. OTOH, if your btreelist goes into Py2.6's collections module and users flock to it instead of regular lists, then there may be some basis for further discussion. BTW, please offer-up a pure python version (like we do for the heapq module). That will make it easier for the curious to find-out how the magic is done. And, it will make life easier for Jython, IronPython, and PyPy implementer so they won't have to do any extra work to support Py2.6. > There are only a few use-cases (that I can think of) where Python's > list() regularly outperforms the BList. These are: > > 1. A large LIFO stack, where there are many .append() and .pop(-1) > operations. These are O(1) for a Python list, but O(log n) for the > BList(). This is a somewhat important use-case (we devote two methods to it). > 2. An large, unchanging list, where there are many getitem() calls, > and none of the inserts or deletes where the BList() shines. Again, > these are O(1) for a Python list, but O(log n) for the BList(), and > the BList() is just as fast if the list is small. This is also a critical use-case. > When I first started learning Python, I often ended up inadvertently > writing O(n**2) algorithms by putting a O(n) list operation inside a > loop (which is a problem when n=100,000). Since nothing in the > signature of list() informs the user when an operation is going to be > slow, it took me a while to learn how to write my code to work around > this constraint. In fact, I initially assumed "list" meant "linked > list" so most of my assumptions about which operations would be fast > vs. slow were backwards. You've come a long way since then. Congrats. Thanks for your contribution and thoughtful discussion. This weekend, I look forward to reading you source code in detail. Raymond From p.f.moore at gmail.com Wed Apr 25 10:05:42 2007 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 25 Apr 2007 09:05:42 +0100 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <462EB78A.207@canterbury.ac.nz> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> Message-ID: <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> On 25/04/07, Greg Ewing wrote: > Do we really need to have B&D-ish enforcement of abstract > method implementation? It doesn't seem pythonic to me. > I might want to leave some methods of an ABC unimplemented > because I'm not intending to use them. I'd be annoyed if > I were prevented from doing that because of some theoretical > notion that they "should" be implemented. I agree - this whole discussion still sounds like things are getting overly strict. OTOH, in Greg's case maybe he should just not bother inheriting from the ABC. From what I hear, that shouldn't cause him any problems ("duck typing isn't going away" is the message I've heard). Of course, it's difficult to be sure without specific examples (on either side of the debate). Paul. From jyasskin at gmail.com Wed Apr 25 10:36:06 2007 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Wed, 25 Apr 2007 01:36:06 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) Message-ID: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> Here's a draft of the numbers ABCs PEP. The most up to date version will live in the darcs repository at http://jeffrey.yasskin.info/darcs/PEPs/pep-3141.txt (unless the number changes) for now. Naming a PEP about numbers 3.141 seems cute, but of course, I don't get to pick the number. :) This is my first PEP, so apologies for any obvious mistakes. I'd particularly like the numpy people's input on whether I've gotten floating-point support right. Thanks, Jeffrey Yasskin ------------------------------- PEP: 3141 Title: A Type Hierarchy for Numbers (and other algebraic entities) Version: $Revision: 54928 $ Last-Modified: $Date: 2007-04-23 16:37:29 -0700 (Mon, 23 Apr 2007) $ Author: Jeffrey Yasskin Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 23-Apr-2007 Post-History: Not yet posted Abstract ======== This proposal defines a hierarchy of Abstract Base Classes (ABCs) [#pep3119] to represent numbers and other algebraic entities similar to numbers. It proposes: * A hierarchy of algebraic concepts, including monoids, groups, rings, and fields with successively more operators and constraints on their operators. This will be added as a new library module named "algebra". * A hierarchy of specifically numeric types, which can be converted to and from the native Python types. This will be added as a new library module named "numbers". Rationale ========= Functions that take numbers as arguments should be able to determine the properties of those numbers, and if and when overloading based on types is added to the language, should be overloadable based on the types of the arguments. This PEP defines some abstract base classes that are useful in numerical calculations. A function can check that variable is an instance of one of these classes and then rely on the properties specified for them. Of course, the language cannot check these properties, so where I say something is "guaranteed", I really just mean that it's one of those properties a user should be able to rely on. This PEP tries to find a balance between providing fine-grained distinctions and specifying types that few people will ever use. Specification ============= Although this PEP uses terminology from PEP3119, the hierarchy is meaningful for any systematic method of defining sets of classes. **Todo:** link to the Interfaces PEP when it's ready. I'm also using the extra notation from [#pep3107] (annotations) to specify some types. Object oriented systems have a general problem in constraining functions that take two arguments. To take addition as an example, ``int(3) + int(4)`` is defined, and ``vector(1,2,3) + vector(3,4,5)`` is defined, but ``int(3) + vector(3,4,5)`` doesn't make much sense. So ``a + b`` is not guaranteed to be defined for any two instances of ``AdditiveGroup``, but it is guaranteed to be defined when ``type(a) == type(b)``. On the other hand, ``+`` does make sense for any sorts of numbers, so the ``Complex`` ABC refines the properties for plus so that ``a + b`` is defined whenever ``isinstance(a,Complex) and isinstance(b,Complex)``, even if ``type(a) != type(b)``. Monoids (http://en.wikipedia.org/wiki/Monoid) consist of a set with an associative operation, and an identity element under that operation. **Open issue**: Is a @classmethod the best way to define constants that depend only on the type?:: class MonoidUnderPlus(Abstract): """+ is associative but not necessarily commutative and has an identity given by plus_identity(). Subclasses follow the laws: a + (b + c) === (a + b) + c a.plus_identity() + a === a === a + a.plus_identity() Sequences are monoids under plus (in Python) but are not AdditiveGroups. """ @abstractmethod def __add__(self, other): raise NotImplementedError @classmethod @abstractmethod def plus_identity(cls): raise NotImplementedError I skip ordinary non-commutative groups here because I don't have any common examples of groups that use ``+`` as their operator but aren't commutative. If we find some, the class can be added later.:: class AdditiveGroup(MonoidUnderPlus): """Defines a commutative group whose operator is +, and whose inverses are produced by -x. See http://en.wikipedia.org/wiki/Abelian_group. Where a, b, and c are instances of the same subclass of AdditiveGroup, the operations should follow these laws, where 'zero' is a.__class__.zero(). a + b === b + a (a + b) + c === a + (b + c) zero + a === a a + (-a) === zero a - b === a + -b Some abstract subclasses, such as Complex, may extend the definition of + to heterogenous subclasses, but AdditiveGroup only guarantees it's defined on arguments of exactly the same types. Vectors are AdditiveGroups but are not Rings. """ @abstractmethod def __add__(self, other): """Associative commutative operation, whose inverse is negation.""" raise NotImplementedError **Open issue:** Do we want to give people a choice of which of the following to define, or should we pick one arbitrarily?:: def __neg__(self): """Must define this or __sub__().""" return self.zero() - self def __sub__(self, other): """Must define this or __neg__().""" return self + -other @classmethod @abstractmethod def zero(cls): """A better name for +'s identity as we move into more mathematical domains.""" raise NotImplementedError @classmethod def plus_identity(cls): return cls.zero() Including Semiring (http://en.wikipedia.org/wiki/Semiring) would help a little with defining a type for the natural numbers. That can be split out once someone needs it (see ``IntegralDomain`` for how).:: class Ring(AdditiveGroup): """A mathematical ring over the operations + and *. See http://en.wikipedia.org/wiki/Ring_%28mathematics%29. In addition to the requirements of the AdditiveGroup superclass, a Ring has an associative but not necessarily commutative multiplication operation with identity (one) that distributes over addition. A Ring can be constructed from any integer 'i' by adding 'one' to itself 'i' times. When R is a subclass of Ring, the additive identity is R(0), and the multiplicative identity is R(1). Matrices are Rings but not Commutative Rings or Division Rings. The quaternions are a Division Ring but not a Field. The integers are a Commutative Ring but not a Field. """ @abstractmethod def __init__(self, i:int): """An instance of a Ring may be constructed from an integer. This may be a lossy conversion, as in the case of the integers modulo N.""" pass @abstractmethod def __mul__(self, other): """Satisfies: a * (b * c) === (a * b) * c one * a === a a * one === a a * (b + c) === a * b + a * c where one == a.__class__(1) """ raise NotImplementedError @classmethod def zero(cls): return cls(0) @classmethod def one(cls): return cls(1) I'm skipping both CommutativeRing and DivisionRing here. class Field(Ring): """The class Field adds to Ring the requirement that * be a commutative group operation except that zero does not have an inverse. See http://en.wikipedia.org/wiki/Field_%28mathematics%29. Practically, that means we can define division on a Field. The additional laws are: a * b === b * a a / a === a.__class_(1) # when a != a.__class__(0) Division lets us construct a Field from any Python float, although the conversion is likely to be lossy. Some Fields include the real numbers, rationals, and integers mod a prime. Python's ``float`` resembles a Field closely. """ def __init__(self, f:float): """A Field should be constructible from any rational number, which includes Python floats.""" pass @abstractmethod def __div__(self, divisor): raise NotImplementedError Division is somewhat complicated in Python. You have both __floordiv__ and __div__, and ints produce floats when they're divided. For the purposes of this hierarchy, ``__floordiv__(a, b)`` is defined by ``floor(__div__(a, b))``, and, since int is not a subclass of Field, it's allowed to do whatever it wants with __div__. There are four more reasonable classes that I'm skipping here in the interest of keeping the initial library simple. They are: ``Algebraic`` Rational powers of its elements are defined (and maybe a few other operations) (http://en.wikipedia.org/wiki/Algebraic_number). Complex numbers are the most well-known algebraic set. Real numbers are _not_ algebraic, but Python does define these operations on floats, which makes defining this class somewhat difficult. ``Trancendental`` The elementary functions (http://en.wikipedia.org/wiki/Elementary_function) are defined. These are basically arbitrary powers, trig functions, and logs, the contents of ``cmath``. The following two classes can be reasonably combined with ``Integral`` for now. ``IntegralDomain`` Defines __divmod__. (http://darcs.haskell.org/numericprelude/docs/html/Algebra-IntegralDomain.html#t%3AC) ``PrincipalIdealDomain`` Defines gcd and lcm. (http://darcs.haskell.org/numericprelude/docs/html/Algebra-PrincipalIdealDomain.html#t%3AC) If someone needs to split them later, they can use code like:: import numbers class IntegralDomain(Ring): ... numbers.Integral.__bases__ = (IntegralDomain,) + numbers.Integral.__bases__ Finally, we get to numbers. This is where we switch from the "algebra" module to the "numbers" module.:: class Complex(Ring, Hashable): """The ``Complex`` ABC indicates that the value lies somewhere on the complex plane, not that it in fact has a complex component: ``int`` is a subclass of ``Complex``. Because these actually represent complex numbers, they can be converted to the ``complex`` type. ``Complex`` finally gets around to requiring its subtypes to be immutable so they can be hashed in a standard way. ``Complex`` also requires its operations to accept heterogenous arguments. Subclasses should override the operators to be more accurate when they can, but should fall back on the default definitions to handle arguments of different (Complex) types. **Open issue:** __abs__ doesn't fit here because it doesn't exist for the Gaussian integers (http://en.wikipedia.org/wiki/Gaussian_integer). In fact, it only exists for algebraic complex numbers and real numbers. We could define it in both places, or leave it out of the ``Complex`` classes entirely and let it be a custom extention of the ``complex`` type. The Gaussian integers are ``Complex`` but not a ``Field``. """ @abstractmethod def __complex__(self): """Any Complex can be converted to a native complex object.""" raise NotImplementedError def __hash__(self): return hash(complex(self)) @abstractmethod def real(self) => Real: raise NotImplementedError @abstractmethod def imag(self) => Real: raise NotImplementedError @abstractmethod def __add__(self, other): """The other Ring operations should be implemented similarly.""" if isinstance(other, Complex): return complex(self) + complex(other) else: return NotImplemented ``FractionalComplex(Complex, Field)`` might fit here, except that it wouldn't give us any new operations. class Real(Complex, TotallyOrdered): """Numbers along the real line. Some subclasses of this class may contain NaNs that are not ordered with the rest of the instances of that type. Oh well. **Open issue:** what problems will that cause? Is it worth it in order to get a straightforward type hierarchy? """ @abstractmethod def __float__(self): raise NotImplementedError def __complex__(self): return complex(float(self)) def real(self) => self.__class__: return self def imag(self) => self.__class__: return self.__class__(0) def __abs__(self) => self.__class__: if self < 0: return -self else: return self class FractionalReal(Real, Field): """Rationals and floats. This class provides concrete definitions of the other four methods from properfraction and allows you to convert fractional reals to integers in a disciplined way. """ @abstractmethod def properfraction(self) => (int, self.__class__): """Returns a pair (n,f) such that self == n+f, and: * n is an integral number with the same sign as self; and * f is a fraction with the same type and sign as self, and with absolute value less than 1. """ raise NotImplementedError def floor(self) => int: n, r = self.properfraction() if r < 0 then n - 1 else n def ceiling(self) => int: ... def __trunc__(self) => int: ... def round(self) => int: ... **Open issue:** What's the best name for this class? RealIntegral? Integer?:: class Integral(Real): """Integers!""" @abstractmethod def __int__(self): raise NotImplementedError def __float__(self): return float(int(self)) @abstractmethod def __or__(self, other): raise NotImplementedError @abstractmethod def __xor__(self, other): raise NotImplementedError @abstractmethod def __and__(self, other): raise NotImplementedError @abstractmethod def __lshift__(self, other): raise NotImplementedError @abstractmethod def __rshift__(self, other): raise NotImplementedError @abstractmethod def __invert__(self): raise NotImplementedError Floating point values may not exactly obey several of the properties you would expect from their superclasses. For example, it is possible for ``(large_val + -large_val) + 3 == 3``, but ``large_val + (-large_val + 3) == 0``. On the values most functions deal with this isn't a problem, but it is something to be aware of. Types like this inherit from ``FloatingReal`` so that functions that care can know to use a numerically stable algorithm on them. **Open issue:** Is this the proper way to handle floating types?:: class FloatingReal: """A "floating" number is one that is represented as ``mantissa * radix**exponent`` where mantissa, radix, and exponent are all integers. Subclasses of FloatingReal don't follow all the rules you'd expect numbers to follow. If you really care about the answer, you have to use numerically stable algorithms, whatever those are. **Open issue:** What other operations would be useful here? These include floats and Decimals. """ @classmethod @abstractmethod def radix(cls) => int: raise NotImplementedError @classmethod @abstractmethod def digits(cls) => int: """The number of significant digits of base cls.radix().""" raise NotImplementedError @classmethod @abstractmethod def exponentRange(cls) => (int, int): """A pair of the (lowest,highest) values possible in the exponent.""" raise NotImplementedError @abstractmethod def decode(self) => (int, int): """Returns a pair (mantissa, exponent) such that mantissa*self.radix()**exponent == self.""" raise NotImplementedError Inspiration =========== http://hackage.haskell.org/trac/haskell-prime/wiki/StandardClasses http://repetae.net/john/recent/out/classalias.html References ========== .. [#pep3119] Introducing Abstract Base Classes (http://www.python.org/dev/peps/pep-3119/) .. [#pep3107] Function Annotations (http://www.python.org/dev/peps/pep-3107/) .. [3] Possible Python 3K Class Tree?, wiki page created by Bill Janssen (http://wiki.python.org/moin/AbstractBaseClasses) .. [#numericprelude] NumericPrelude: An experimental alternative hierarchy of numeric type classes (http://darcs.haskell.org/numericprelude/docs/html/index.html) Acknowledgements ---------------- Thanks to Neil Norwitz for helping me through the PEP process. The Haskell Numeric Prelude [#numericprelude] nicely condensed a lot of experience with the Haskell numeric hierarchy into a form that was relatively easily adaptable to Python. Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From ironfroggy at gmail.com Wed Apr 25 14:52:50 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Wed, 25 Apr 2007 08:52:50 -0400 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> Message-ID: <76fd5acf0704250552p19bb3c51sb66cf569beebfda6@mail.gmail.com> On 4/25/07, Steven Bethard wrote: > On 4/24/07, Steven Bethard wrote: > > On 4/24/07, Calvin Spealman wrote: > > > I must have miscopied then because it worked perfectly here. Yes, I > > > meant to have the _superdesc defined inside the metaclass __init__, > > > but thought I could pull it out to make it cleaner. I forgot it > > > actually had to be there! Here is the metaclass that works. > > > > > > class autosuper(type): > > > def __init__(cls, name, bases, clsdict): > > > class _superdesc(object): > > > def __get__(self, obj, objcls): > > > return super(cls, obj) > > > cls.__super__ = _superdesc() > > > > I still get the same error. > > I played around with this for a while. Below is some code that does > work. The trick is to redefine super() so that when you're calling > __get__ to bind a method to an instance, you bind it to the Super() > instance, not the original object instance. > > class Super(object): > def __init__(self, type, obj=None): > if isinstance(obj, Super): > obj = obj.__obj__ > self.__type__ = type > self.__obj__ = obj > def __get__(self, obj, cls=None): > if obj is None: > raise Exception('only supports instances') > else: > return Super(self.__type__, obj) > def __getattr__(self, attr): > mro = iter(self.__obj__.__class__.__mro__) > for cls in mro: > if cls is self.__type__: > break > for cls in mro: > if attr in cls.__dict__: > x = cls.__dict__[attr] > if hasattr(x, '__get__'): > x = x.__get__(self, cls) > return x > raise AttributeError, attr > > class autosuper(type): > def __init__(cls, name, bases, clsdict): > cls.__super__ = Super(cls) > > class A: > __metaclass__ = autosuper > def f(self): > return 'A' > > class B(A): > def f(self): > return 'B' + self.__super__.f() > > class C(A): > def f(self): > return 'C' + self.__super__.f() > > class D(B, C): > def f(self): > return 'D' + self.__super__.f() > > assert D().f() == 'DBCA' > > See ma, no bytecode hacks! ;-) > > STeVe > -- > I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a > tiny blip on the distant coast of sanity. > --- Bucky Katt, Get Fuzzy Awesomeness. The 'super keyword' brought up in the original post would also be possible via the same route and some stack pokin' to get at self from the super object in uses like super.foo(args) and such. Jim, you said you'd take a crack "if no one else volunteers", so perhaps I will, if you didn't have your heart set on it. I would like more opportunities to contribute and this wouldn't be a terribly long PEP for my first foray into writing one. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From emin.shopper at gmail.com Wed Apr 25 15:16:24 2007 From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper) Date: Wed, 25 Apr 2007 09:16:24 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> Message-ID: <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> I disagree. I think that B&D-ish enforcement of ABCs is the most important feature of an ABC (with the option of being able to turn off enforcement on a per-class basis). Imagine you have a program that takes a long time to run. If you use duck-typing or similar ideas you may watch the program run for hours or days and then raise an AttributeError or NotImplemented error because a derived class didn't implement a required method. With B&D-ish enforcement of ABCs, you can automatically catch such problems as soon as the program starts. This is a problem that I see no other good solution to. For long running programs, unit-tests will also take a long time to run so the argument that your unit-tests should catch such errors is no longer valid. Also, while pylint and pychecker are great tools that could be used to catch such errors, they must be run manually by the programmer and generally take a long time to run (at least longer than the ABC implementation I have). To reiterate, while I think the documentation and reflection benefits of ABCs are nice, I don't think they are nearly as essential as enforcement. For most of the use cases I've seen, you could just as easily read the documentation or the source code or try a simple example to determine how to implement an iterable or hashable that will play nice with other classes. In contrast, ABC enforcement provides the real benefit of catching errors before many hours of program runtime are wasted. Consequently, I'm +1 on B&D in this case. -Emin Martinian On 4/25/07, Paul Moore wrote: > > On 25/04/07, Greg Ewing wrote: > > Do we really need to have B&D-ish enforcement of abstract > > method implementation? It doesn't seem pythonic to me. > > I might want to leave some methods of an ABC unimplemented > > because I'm not intending to use them. I'd be annoyed if > > I were prevented from doing that because of some theoretical > > notion that they "should" be implemented. > > I agree - this whole discussion still sounds like things are getting > overly strict. > > OTOH, in Greg's case maybe he should just not bother inheriting from > the ABC. From what I hear, that shouldn't cause him any problems > ("duck typing isn't going away" is the message I've heard). Of course, > it's difficult to be sure without specific examples (on either side of > the debate). > > Paul. > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/emin.shopper%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070425/860a297b/attachment.html From emin.shopper at gmail.com Wed Apr 25 15:19:07 2007 From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper) Date: Wed, 25 Apr 2007 09:19:07 -0400 Subject: [Python-3000] An introduction to ABC's In-Reply-To: References: <32e43bb70704240739x53d80ff3j863e64d9a11b5f@mail.gmail.com> Message-ID: <32e43bb70704250619o7cda8942ka9a76457ea9c52ea@mail.gmail.com> On 4/25/07, Neal Norwitz wrote: You can get similar functionality by using tools such as pychecker. > PyLint and pyflakes may also have this functionality. I like PyLint and pychecker a lot, but the funtionality is not similar. Both take a lot longer to run (at least longer than my ABC implementation) and you have to remember to run them each time you change your program. Using an ABC adds very little overhead and gets checked automatically. Thanks, -Emin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070425/9733d63c/attachment.html From jimjjewett at gmail.com Wed Apr 25 15:52:35 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 25 Apr 2007 09:52:35 -0400 Subject: [Python-3000] Fixing super anyone? In-Reply-To: <76fd5acf0704250552p19bb3c51sb66cf569beebfda6@mail.gmail.com> References: <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <76fd5acf0704250552p19bb3c51sb66cf569beebfda6@mail.gmail.com> Message-ID: On 4/25/07, Calvin Spealman wrote: > Jim, you said you'd take a crack "if no one else volunteers", so > perhaps I will, if you didn't have your heart set on it. I would like > more opportunities to contribute and this wouldn't be a terribly long > PEP for my first foray into writing one. I would be thrilled if you took it; I posted the __this_class__ PEP as a separate piece, but I still haven't figured out the full super -- and if you have time, that would be great. -jJ From jjb5 at cornell.edu Wed Apr 25 15:21:28 2007 From: jjb5 at cornell.edu (Joel Bender) Date: Wed, 25 Apr 2007 09:21:28 -0400 Subject: [Python-3000] Fixing super anyone? In-Reply-To: <76fd5acf0704250552p19bb3c51sb66cf569beebfda6@mail.gmail.com> References: <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <76fd5acf0704250552p19bb3c51sb66cf569beebfda6@mail.gmail.com> Message-ID: <462F55D8.5000100@cornell.edu> I mucked around with this, and here is my version: class _super(property): def __init__(self): property.__init__(self, self.get_super, None, None) def get_super(self, klass): def dong(obj): class wrapper: def __getattr__(self, fn): save = obj.__class__ obj.__class__ = klass.__mro__[1] f = getattr(obj, fn) obj.__class__ = save return f return wrapper() return dong class _superable(type): __super__ = _super() class W(object): __metaclass__ = _superable def f(self): print "W" class X(W): def f(self): print "X" X.__super__(self).f() class Y(W): def f(self): print "Y" Y.__super__(self).f() class Z(X, Y): def f(self): print "Z" Z.__super__(self).f() I would rather the end result look more like this: class Z(X, Y): def f(self): print "Z" Z.__super__.f(self) But I haven't figured out how to do that yet, and the self.__super__.f() pattern that Steven Bethard posted trumps it anyway! Joel From p.f.moore at gmail.com Wed Apr 25 16:04:22 2007 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 25 Apr 2007 15:04:22 +0100 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> Message-ID: <79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com> On 25/04/07, Emin.shopper Martinian.shopper wrote: > I disagree. I think that B&D-ish enforcement of ABCs is the most important > feature of an ABC (with the option of being able to turn off enforcement on > a per-class basis). OK. We can agree to disagree. My point was that as far as I am aware, Guido's current position on ABCs is that they are, and will remain, optional - people like myself (and Greg, from the sound of it) who don't want to derive from ABCs will not be penalised by being excluded from anything in the Python core or the stdlib. Example - I recall Guido saying explicitly that there will be no requirement for objects assigned to sys.stdout to derive from any of the IO ABCs. Paul. From jjb5 at cornell.edu Wed Apr 25 16:44:15 2007 From: jjb5 at cornell.edu (Joel Bender) Date: Wed, 25 Apr 2007 10:44:15 -0400 Subject: [Python-3000] Fixing super anyone? (redux) In-Reply-To: <462F55D8.5000100@cornell.edu> References: <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <76fd5acf0704250552p19bb3c51sb66cf569beebfda6@mail.gmail.com> <462F55D8.5000100@cornell.edu> Message-ID: <462F693F.90306@cornell.edu> > But I haven't figured out how to do that yet... It turns out to be easier than I thought, and it avoids changing the object __class__, which is ugly. class _super(property): def __init__(self): property.__init__(self, self.get_super, None, None) def get_super(self, klass): class wrapper: def __getattr__(self, fn): for cls in klass.__mro__[1:]: f = getattr(cls, fn, None) if f: return f raise AttributeError, fn return wrapper() class _superable(type): __super__ = _super() class W(object): __metaclass__ = _superable def f(self): print "W" class X(W): def f(self): print "X" X.__super__.f(self) class Y(W): def f(self): print "Y" Y.__super__.f(self) class Z(X, Y): def f(self): print "Z" Z.__super__.f(self) I'll go back to lurking now... Joel From bingham at cenix-bioscience.com Wed Apr 25 16:58:04 2007 From: bingham at cenix-bioscience.com (Aaron Bingham) Date: Wed, 25 Apr 2007 16:58:04 +0200 Subject: [Python-3000] An introduction to ABC's In-Reply-To: <32e43bb70704250619o7cda8942ka9a76457ea9c52ea@mail.gmail.com> References: <32e43bb70704240739x53d80ff3j863e64d9a11b5f@mail.gmail.com> <32e43bb70704250619o7cda8942ka9a76457ea9c52ea@mail.gmail.com> Message-ID: <462F6C7C.5090608@cenix-bioscience.com> Emin.shopper Martinian.shopper wrote: > On 4/25/07, *Neal Norwitz* > wrote: > > You can get similar functionality by using tools such as pychecker. > PyLint and pyflakes may also have this functionality. > > > I like PyLint and pychecker a lot, but the funtionality is not > similar. Both take a lot longer to run (at least longer than my ABC > implementation) and you have to remember to run them each time you > change your program. Using an ABC adds very little overhead and gets > checked automatically. You could use an editor that automatically runs PyLint and/or pychecker. The PyDev plugin for Eclipse runs PyLint in the background and highlights any lines that produced errors or warnings. Now that I've worked with PyDev for several months, I wouldn't want to work without the immediate feedback it provides. Regards, -- Aaron Bingham Senior Software Engineer bingham at cenix-bioscience.com Tel. +49 (0)351 4173-146 Fax +49 (0)351 4173-109 Cenix BioScience GmbH Tatzberg 47 01307 Dresden, Germany www.cenix-bioscience.com --------------------------------------------------------- Sitz der Gesellschaft (Place of Business): Dresden Gesch?ftsf?hrer (CEO): Dr. Christophe J. Echeverri Amtsgericht (Local Court): Dresden, HRB 19964 Ust-ID (VAT-No.): DE205824437 --------------------------------------------------------- From jcarlson at uci.edu Wed Apr 25 18:03:14 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 25 Apr 2007 09:03:14 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> Message-ID: <20070425085350.6408.JCARLSON@uci.edu> "Jeffrey Yasskin" wrote: > Here's a draft of the numbers ABCs PEP. The most up to date version > will live in the darcs repository at > http://jeffrey.yasskin.info/darcs/PEPs/pep-3141.txt (unless the number > changes) for now. Naming a PEP about numbers 3.141 seems cute, but of > course, I don't get to pick the number. :) This is my first PEP, so > apologies for any obvious mistakes. > > I'd particularly like the numpy people's input on whether I've gotten > floating-point support right. > > Thanks, > Jeffrey Yasskin > > ------------------------------- > PEP: 3141 > Title: A Type Hierarchy for Numbers (and other algebraic entities) > Version: $Revision: 54928 $ > Last-Modified: $Date: 2007-04-23 16:37:29 -0700 (Mon, 23 Apr 2007) $ > Author: Jeffrey Yasskin > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 23-Apr-2007 > Post-History: Not yet posted > > > Abstract > ======== > > This proposal defines a hierarchy of Abstract Base Classes (ABCs) > [#pep3119] to represent numbers and other algebraic entities similar > to numbers. It proposes: [snip] Are the base number operations in Python all that difficult to understand? Do we really need to add mathematical formalism into Python's type system before people understand the meaning of X * Y? Because all I really see this doing is confusing the hell out of people who aren't mathematicians; I'm a theoretical computer scientist and I had to try to think for a moment to verify that all of those were really necessary to separate the cases. I really do understand wanting to be able to ask "is operation X supported for values Y and Z", but is this really necessary for numbers? - Josiah From jyasskin at gmail.com Wed Apr 25 18:50:20 2007 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Wed, 25 Apr 2007 09:50:20 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <20070425085350.6408.JCARLSON@uci.edu> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <20070425085350.6408.JCARLSON@uci.edu> Message-ID: <5d44f72f0704250950j2c2f6c77i61560a0d60ae7da4@mail.gmail.com> [bcc'ing numpy-discussion. Comments should probably try to stay on the python-3000 list.] On 4/25/07, Josiah Carlson wrote: > Are the base number operations in Python all that difficult to > understand? Do we really need to add mathematical formalism into > Python's type system before people understand the meaning of X * Y? > Because all I really see this doing is confusing the hell out of people > who aren't mathematicians; I'm a theoretical computer scientist and I > had to try to think for a moment to verify that all of those were really > necessary to separate the cases. > > I really do understand wanting to be able to ask "is operation X > supported for values Y and Z", but is this really necessary for numbers? If you don't want to use the full power of the system, nothing forces you to. If someone doesn't know what a Ring or a complex number is, they can just check for Real and rely on the same operations, at the small cost of overconstraining their inputs. One reason I'm suggesting splitting the classes into the "algebra" and "numbers" modules is so that people who don't know or care about the algebra can rely on just "numbers" and still get nearly all of the benefits. However, I think it's important to provide a framework for people who want to do more powerful things. If you write a function that can take an arbitrary ring, it should Just Work on a python int, or any other Complex subclass, without you having to check for a lot of operations separately. The Haskell98 numerical hierarchy actually does start at Num, ignoring the algebraic structures underneath, and it draws a lot of criticism for it. That does mean that the names and documentation in the numbers module need to be as clear as we can make them for non-mathematical audiences. I'm certainly open to suggestions on that front. -- Namast?, Jeffrey Yasskin From collinw at gmail.com Wed Apr 25 18:54:20 2007 From: collinw at gmail.com (Collin Winter) Date: Wed, 25 Apr 2007 09:54:20 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> Message-ID: <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> The following things mean absolutely nothing to me: - Monoid - MonoidUnderPlus - Group - Ring - Semiring - Field So, most of the terminology in the PEP. I can see use-cases for this level of formalism, but I'm a strong -1 on making any part of the stdlib effectively off-limits for people without advanced math degrees. Why can't this be shipped as a third-party module? Collin Winter From daniel at stutzbachenterprises.com Wed Apr 25 18:54:33 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Wed, 25 Apr 2007 11:54:33 -0500 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: <00c601c786ff$95839700$f101a8c0@RaymondLaptop1> References: <00c601c786ff$95839700$f101a8c0@RaymondLaptop1> Message-ID: On 4/25/07, Raymond Hettinger wrote: > In practice, the code for BList is somewhat complex, and its desirability and performance in actual applications is unproven. > Fortunately, Py2.6 is still a long way off. My recommendation is that you release it right away as a third-party module and let the > hordes of Pythonistas test it in battle. With the code base thoroughly exercised and some positive user feedback, it would be hard > to say no to this going into Py2.6. Will do. What's the best way to release a module to make it easy for people to find, download, and install? (a pointer to a relevant document would be great) > How does BList's memory consumption compare with the current list over-allocation scheme? > > Does it take longer to instantiate an emtpy list? What are the malloc/free patterns? See these earlier message in the thread: http://mail.python.org/pipermail/python-3000/2007-April/006801.html http://mail.python.org/pipermail/python-3000/2007-April/006804.html http://mail.python.org/pipermail/python-3000/2007-April/006809.html The short summary is that BList and the current list have very similar best-case and worst-case memory consumption for a single list. (If copy-on-write comes into play, BList is much more memory efficient than a regular list) Malloc/free operations occur when the list is large and growing or shrinking (approximately when 64 elements have been added or removed). They occur more often than for list(), but if you're doing a lot of growing and shrinking the O(log n) vs O(n) performance boost easily pays for the extra memory operations. All the blocks are the same size. The BList code keeps a small list of recently freed nodes to recycle, which speeds things up in certain common cases where additions and removals are interchanged. > > In other words, for short lists, > > a BList works just like Python's array-based list() type. Thus, it > > has the same good performance on small lists. > > While this is a good implementation idea (similar ideas are used to speed-up quicksort for instance), it makes the first listed > advantage seem like a straw-man designed to mislead proposal reviewers into thinking that the blist structure is more efficient than > is actually is. Perhaps rewrite it as: "To keep the performance acceptable for short lists, btreelists fall back to the existing > implementation." I fear that you have misunderstood me, as the BList does not fall back on the existing implementation. I'm sorry that you may have gotten the impression that I was attempting to mislead. I admit openly that the performance for small lists is not due to any fancy data structure. However, it's not array-like as a special case just for small lists, either. Each BList node contains an array of up to 128 children. If the BList node is a leaf, these children are pointers to items stored by the user, otherwise they are pointers to other BList nodes. If the list contains 128 or fewer children, the BList can have one node which is both the root and a leaf. This node just has an array of the user's items, so it ends up having the same performance as Python's array-based list. It is true that you end up with array-operations when you have a small list and as the base case for many operations implemented via recursion. In some sense that is similar to a QuickSort that uses InsertionSort for n < threshold. Unlike QuickSort, I couldn't rewrite it *without* array operations as the base case, and array operations also occur in the middle of the tree. They're fundamental, not a performance tweak. So I don't think the QuickSort with InsertionSort analogy holds. How about if I include a brief description up-front of how it works like: "The BList internally uses a tree structure where each node contains an array of up to 128 children." Then the first performance bullet can be something like "just as fast as a Python list() when the list is small, since BList's are also based on arrays"? > FWIW, I prefer that the current list implementation remain in-place. Because what we have is simple, it is not hard to maintain, its > performance characteristics are easily understood, and we have a straight-forward C-API that allows module writers to have fast, > intuitive direct access to the structure. > > OTOH, if your btreelist goes into Py2.6's collections module and users flock to it instead of regular lists, then there may be some > basis for further discussion. This sounds like a good plan to me, though how does that square with Guido's desire to avoid a proliferation of data types? http://mail.python.org/pipermail/python-3000/2007-April/006774.html > BTW, please offer-up a pure python version (like we do for the heapq module). That will make it easier for the curious to find-out > how the magic is done. And, it will make life easier for Jython, IronPython, and PyPy implementer so they won't have to do any > extra work to support Py2.6. Will do. > > There are only a few use-cases (that I can think of) where Python's > > list() regularly outperforms the BList. These are: > > > > 1. A large LIFO stack, where there are many .append() and .pop(-1) > > operations. These are O(1) for a Python list, but O(log n) for the > > BList(). > > This is a somewhat important use-case (we devote two methods to it). > > > 2. An large, unchanging list, where there are many getitem() calls, > > and none of the inserts or deletes where the BList() shines. Again, > > these are O(1) for a Python list, but O(log n) for the BList(), and > > the BList() is just as fast if the list is small. > > This is also a critical use-case. I don't disagree. I think the key question is this: Given that there's no performance difference for small lists, would you rather have a type that for large lists: 1) Gives you O(1) speed for some critical use-cases, but O(n) for others, or 2) Gives you O(log n) across the board? Put another way, for a 10,000 element list how would you feel about taking a small performance hit (50% increase in execution time for LIFO; 5% increase in execution time for getitem) in exchange for large performance improvements elsewhere (< 1% of the old execution time for getslice, setslice, FIFO, etc.)? I obviously favor #2, but I acknowledge that this is a subjective question that depends on which use-cases you care most about. Your incremental deployment plan (extension module, possibly in collections for 2.6, go from there) may be a good way to evaluate how the community as a whole feels. > You've come a long way since then. Congrats. > > Thanks for your contribution and thoughtful discussion. This weekend, I look forward to reading you source code in detail. Thanks! I've just adding my Python prototype implementation to the .tar.gz. It has a lengthy text section at the top that describes the structure of the BList and the invariants that it must maintain. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From collinw at gmail.com Wed Apr 25 19:05:25 2007 From: collinw at gmail.com (Collin Winter) Date: Wed, 25 Apr 2007 10:05:25 -0700 Subject: [Python-3000] Suggested PEP 3108 addition Message-ID: <43aa6ff70704251005i4912e6d6s7efc6346483a142b@mail.gmail.com> I'd like to propose that the bsddb185 module be added to PEP 3108's list of modules slated for removal. Reasons to kill it: - The module isn't built by default; manual tweaking of setup.py is required. - The following scare-paragraph is included at the bottom of http://docs.python.org/lib/module-bsddb.html: """ Beginning in 2.3 some Unix versions of Python may have a bsddb185 module. This is present only to allow backwards compatibility with systems which ship with the old Berkeley DB 1.85 database library. The bsddb185 module should never be used directly in new code. """ Collin Winter From jyasskin at gmail.com Wed Apr 25 19:09:45 2007 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Wed, 25 Apr 2007 10:09:45 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <20070425085350.6408.JCARLSON@uci.edu> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <20070425085350.6408.JCARLSON@uci.edu> Message-ID: <5d44f72f0704251009w12eddd82l971f5d06b35b22fb@mail.gmail.com> [bcc'ing numpy-discussion. Comments should probably try to stay on the python-3000 list.] On 4/25/07, Josiah Carlson wrote: > Are the base number operations in Python all that difficult to > understand? Do we really need to add mathematical formalism into > Python's type system before people understand the meaning of X * Y? > Because all I really see this doing is confusing the hell out of people > who aren't mathematicians; I'm a theoretical computer scientist and I > had to try to think for a moment to verify that all of those were really > necessary to separate the cases. > > I really do understand wanting to be able to ask "is operation X > supported for values Y and Z", but is this really necessary for numbers? If you don't want to use the full power of the system, nothing forces you to. If someone doesn't know what a Ring or a complex number is, they can just check for Real and rely on the same operations, at the small cost of overconstraining their inputs. One reason I'm suggesting splitting the classes into the "algebra" and "numbers" modules is so that people who don't know or care about the algebra can rely on just "numbers" and still get nearly all of the benefits. However, I think it's important to provide a framework for people who want to do more powerful things. If you write a function that can take an arbitrary ring, it should Just Work on a python int, or any other Complex subclass, without you having to check for a lot of operations separately. The Haskell98 numerical hierarchy actually does start at Num, ignoring the algebraic structures underneath, and it draws a lot of criticism for it. That does mean that the names and documentation in the numbers module need to be as clear as we can make them for non-mathematical audiences. I'm certainly open to suggestions on that front. -- Namast?, Jeffrey Yasskin From jason.orendorff at gmail.com Wed Apr 25 19:27:31 2007 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Wed, 25 Apr 2007 13:27:31 -0400 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> Message-ID: On 4/25/07, Collin Winter wrote: > I can see use-cases for this level of formalism, but I'm a strong -1 > on making any part of the stdlib effectively off-limits for people > without advanced math degrees. Why can't this be shipped as a > third-party module? I agree. That last question says something interesting about ABCs, though. You and I don't care about Rings, but some people do, and they would probably like this as a third-party library... if it were possible. In Haskell, you can define a typeclass (which is a lot like an ABC): class Ring r where zero :: r add :: r -> r -> r negate :: r -> r mul :: r -> r -> r Then you can declare that certain existing types, even builtin types, conform to it: -- tell Haskell that "the Integer type is a Ring" instance Ring Integer where zero = 0 add x y = x + y negate x = -x mul x y = x * y This feature makes stuff like NumericPrelude possible. My understanding is that Guido's ABCs won't support this, so in fact you *couldn't* ship Jeffrey's numeric classes as a third-party library. (My perspective is a little different from Jeffrey's. Maybe Haskell does draw fire for not having a ton of numeric typeclasses; I don't know. But it seems to me Haskell gets it Just Right: the standard typeclasses are simple, practical, and loose enough to accomodate Int and Float; and the language lets you roll your own crazy mathematical mojo if that's what you like.) -j From ironfroggy at gmail.com Wed Apr 25 19:37:47 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Wed, 25 Apr 2007 13:37:47 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> Message-ID: <76fd5acf0704251037s17761b66t8fcbe7a62a3b31ff@mail.gmail.com> On 4/25/07, Emin.shopper Martinian.shopper wrote: > I disagree. I think that B&D-ish enforcement of ABCs is the most important > feature of an ABC (with the option of being able to turn off enforcement on > a per-class basis). > > Imagine you have a program that takes a long time to run. If you use > duck-typing or similar ideas you may watch the program run for hours or days > and then raise an AttributeError or NotImplemented error because a derived > class didn't implement a required method. With B&D-ish enforcement of ABCs, > you can automatically catch such problems as soon as the program starts. > This is a problem that I see no other good solution to. For long running > programs, unit-tests will also take a long time to run so the argument that > your unit-tests should catch such errors is no longer valid. Also, while > pylint and pychecker are great tools that could be used to catch such > errors, they must be run manually by the programmer and generally take a > long time to run (at least longer than the ABC implementation I! have). > > To reiterate, while I think the documentation and reflection benefits of > ABCs are nice, I don't think they are nearly as essential as enforcement. > For most of the use cases I've seen, you could just as easily read the > documentation or the source code or try a simple example to determine how to > implement an iterable or hashable that will play nice with other classes. In > contrast, ABC enforcement provides the real benefit of catching errors > before many hours of program runtime are wasted. Consequently, I'm +1 on B&D > in this case. > > -Emin Martinian It sounds like you just need better unit tests. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From lists at cheimes.de Wed Apr 25 20:07:03 2007 From: lists at cheimes.de (Christian Heimes) Date: Wed, 25 Apr 2007 20:07:03 +0200 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <20070425085350.6408.JCARLSON@uci.edu> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <20070425085350.6408.JCARLSON@uci.edu> Message-ID: Josiah Carlson wrote: > I really do understand wanting to be able to ask "is operation X > supported for values Y and Z", but is this really necessary for numbers? It's not required for ordinary people who deal with ints and floats. Python is pushing into the scientific market. The math guys need it when they have to deal with vectors, matrix operation and other advanced stuff. I'm seeing the proposal as an important step to make Python a professional tool for scientific research. From guido at python.org Wed Apr 25 20:19:56 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2007 11:19:56 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> Message-ID: On 4/25/07, Collin Winter wrote: > The following things mean absolutely nothing to me: > > - Monoid > - MonoidUnderPlus > - Group > - Ring > - Semiring > - Field > > So, most of the terminology in the PEP. > > I can see use-cases for this level of formalism, but I'm a strong -1 > on making any part of the stdlib effectively off-limits for people > without advanced math degrees. Why can't this be shipped as a > third-party module? As a math major I have no excuse, but I have to confess I'm also really rusty on these things. (Nothing a quick look at wikipedia can't refresh though.) Jeffrey, is there any way you can drop the top of the tree and going straight from Number to Complex -> Real -> Rational -> Integer? These are the things that everyone with high school math will know. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Apr 25 20:33:59 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2007 11:33:59 -0700 Subject: [Python-3000] Fixing super anyone? In-Reply-To: References: <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> Message-ID: On 4/24/07, Steven Bethard wrote: > class Super(object): > def __init__(self, type, obj=None): > if isinstance(obj, Super): > obj = obj.__obj__ > self.__type__ = type > self.__obj__ = obj > def __get__(self, obj, cls=None): > if obj is None: > raise Exception('only supports instances') > else: > return Super(self.__type__, obj) > def __getattr__(self, attr): > mro = iter(self.__obj__.__class__.__mro__) > for cls in mro: > if cls is self.__type__: > break > for cls in mro: > if attr in cls.__dict__: > x = cls.__dict__[attr] > if hasattr(x, '__get__'): > x = x.__get__(self, cls) > return x > raise AttributeError, attr > > class autosuper(type): > def __init__(cls, name, bases, clsdict): > cls.__super__ = Super(cls) > > class A: > __metaclass__ = autosuper > def f(self): > return 'A' > > class B(A): > def f(self): > return 'B' + self.__super__.f() > > class C(A): > def f(self): > return 'C' + self.__super__.f() > > class D(B, C): > def f(self): > return 'D' + self.__super__.f() > > assert D().f() == 'DBCA' But: class E(D): pass print E().f() This prints DDBCA which surely isn't right. Sounds like the classic bug in such attempts. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Wed Apr 25 20:41:34 2007 From: brett at python.org (Brett Cannon) Date: Wed, 25 Apr 2007 11:41:34 -0700 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical In-Reply-To: References: <00c601c786ff$95839700$f101a8c0@RaymondLaptop1> Message-ID: On 4/25/07, Daniel Stutzbach wrote: > On 4/25/07, Raymond Hettinger wrote: > > In practice, the code for BList is somewhat complex, and its desirability and performance in actual applications is unproven. > > Fortunately, Py2.6 is still a long way off. My recommendation is that you release it right away as a third-party module and let the > > hordes of Pythonistas test it in battle. With the code base thoroughly exercised and some positive user feedback, it would be hard > > to say no to this going into Py2.6. > > Will do. What's the best way to release a module to make it easy for > people to find, download, and install? (a pointer to a relevant > document would be great) > Create a Distutils package or egg, get it up on the Cheeseshop (PyPI), and then send an announcement to comp.lang.python.announce. -Brett From brett at python.org Wed Apr 25 20:44:44 2007 From: brett at python.org (Brett Cannon) Date: Wed, 25 Apr 2007 11:44:44 -0700 Subject: [Python-3000] Suggested PEP 3108 addition In-Reply-To: <43aa6ff70704251005i4912e6d6s7efc6346483a142b@mail.gmail.com> References: <43aa6ff70704251005i4912e6d6s7efc6346483a142b@mail.gmail.com> Message-ID: On 4/25/07, Collin Winter wrote: > I'd like to propose that the bsddb185 module be added to PEP 3108's > list of modules slated for removal. Reasons to kill it: > > - The module isn't built by default; manual tweaking of setup.py is required. > > - The following scare-paragraph is included at the bottom of > http://docs.python.org/lib/module-bsddb.html: > > """ > Beginning in 2.3 some Unix versions of Python may have a bsddb185 > module. This is present only to allow backwards compatibility with > systems which ship with the old Berkeley DB 1.85 database library. The > bsddb185 module should never be used directly in new code. > """ That's enough by me to kill it and fits the primary desire to kill old modules that have been replaced (in this case bsddb3). Unless someone speaks up I will add the module to the PEP when I get my laptop back or if some kind soul beats me to it. -Brett From emin.shopper at gmail.com Wed Apr 25 20:49:29 2007 From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper) Date: Wed, 25 Apr 2007 14:49:29 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <76fd5acf0704251037s17761b66t8fcbe7a62a3b31ff@mail.gmail.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> <76fd5acf0704251037s17761b66t8fcbe7a62a3b31ff@mail.gmail.com> Message-ID: <32e43bb70704251149l3acca60ak70d30da2b4dd4395@mail.gmail.com> On 4/25/07, Calvin Spealman wrote: > > It sounds like you just need better unit tests. Only if you believe you can unit test everything within a few seconds. If you believe that there exist data structures or algorithms where a meaningful unit test takes many minutes or hours, then there is value in an ABC enforcement mechanism. -Emin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070425/46d39207/attachment.htm From jcarlson at uci.edu Wed Apr 25 21:22:07 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 25 Apr 2007 12:22:07 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <32e43bb70704251149l3acca60ak70d30da2b4dd4395@mail.gmail.com> References: <76fd5acf0704251037s17761b66t8fcbe7a62a3b31ff@mail.gmail.com> <32e43bb70704251149l3acca60ak70d30da2b4dd4395@mail.gmail.com> Message-ID: <20070425121618.6417.JCARLSON@uci.edu> "Emin.shopper Martinian.shopper" wrote: > On 4/25/07, Calvin Spealman wrote: > > > > It sounds like you just need better unit tests. > > Only if you believe you can unit test everything within a few seconds. If > you believe that there exist data structures or algorithms where a > meaningful unit test takes many minutes or hours, then there is value in an > ABC enforcement mechanism. If your unit tests take hours to run, then you aren't going to get anything useful from the ABC requirements for hours either (unless you are pre-instantiating everything you are going to use during program execution, in which case this could become your unittest). Please remember that this isn't about Java style type checking and verification during compilation. This is, strictly speaking, ABC checking on object instantiation, method invocation, and possibly even argument verification. This isn't the magic bullet you are looking for (you'll have to go to one of those static typed languages for that). - Josiah From jimjjewett at gmail.com Wed Apr 25 22:03:32 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 25 Apr 2007 16:03:32 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <20070425121618.6417.JCARLSON@uci.edu> References: <76fd5acf0704251037s17761b66t8fcbe7a62a3b31ff@mail.gmail.com> <32e43bb70704251149l3acca60ak70d30da2b4dd4395@mail.gmail.com> <20070425121618.6417.JCARLSON@uci.edu> Message-ID: On 4/25/07, Josiah Carlson wrote: > Please remember that this isn't about Java style type checking and > verification during compilation. This is, strictly speaking, ABC > checking on object instantiation, method invocation, and possibly even > argument verification. This isn't the magic bullet you are looking for > (you'll have to go to one of those static typed languages for that). Yes and no. With duck typing, you call the method when you're ready to use it. At best, he could pre-check his objects to ensure that they have the methods he'll need later. There will inevitably be times when the "wrong" object slips by. With an ABC, he gets a quick check that the object at least *claims* to do the right thing (as opposed to having unrelated methods of the same name), and that they have taken *some* effort to fulfill this pledge. The upside is that there won't be as many wrong objects getting by. The downside is that there will be some objects which get wrongly rejected. (gmpy.mpz is probably a good enough integer, even before it gets updated to say so.) For Ermin's use case, a few false rejections is cheaper than a few false hits. -jJ From jyasskin at gmail.com Wed Apr 25 22:04:18 2007 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Wed, 25 Apr 2007 13:04:18 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> Message-ID: <5d44f72f0704251304o15f2e080wcb9ade83be56403@mail.gmail.com> On 4/25/07, Guido van Rossum wrote: > Jeffrey, is there any way you can drop the top of the tree and going > straight from Number to Complex -> Real -> Rational -> Integer? These > are the things that everyone with high school math will know. I think yes, if you can confirm that the import numbers class Ring(AdditiveGroup): ... numbers.Complex.__bases__ = (Ring,) + numbers.Complex.__bases__ hack I mention under IntegralDomain will make isinstance(int, Ring) return True. Do you want "Number" to be equivalent to Ring+Hashable (or a Haskell-like Ring+Hashable+__abs__+__str__)? I don't think a "Number" class is strictly necessary since people can check for Complex or Real directly, and one of those two is probably what they'll expect after knowing that something's a number. Also, I don't see much point in putting Rational between Real and Integer. The current hierarchy is Real (int, float, Decimal, rational) :> Integer (int) and Real :> FractionalReal (float, Decimal, rational), but Integer and FractionalReal are just siblings. I'm wondering how many people are writing new numeric types who don't know the abstract algebra. I think we should be able to insulate people who are just using the types from the complexities underneath, and the people writing new types will benefit. Or will seeing "Ring" in a list of superclasses be enough to confuse people? -- Namast?, Jeffrey Yasskin From guido at python.org Wed Apr 25 22:14:17 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2007 13:14:17 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> <79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com> Message-ID: On 4/25/07, Paul Moore wrote: > On 25/04/07, Emin.shopper Martinian.shopper wrote: > > I disagree. I think that B&D-ish enforcement of ABCs is the most important > > feature of an ABC (with the option of being able to turn off enforcement on > > a per-class basis). > > OK. We can agree to disagree. My point was that as far as I am aware, > Guido's current position on ABCs is that they are, and will remain, > optional - people like myself (and Greg, from the sound of it) who > don't want to derive from ABCs will not be penalised by being excluded > from anything in the Python core or the stdlib. > > Example - I recall Guido saying explicitly that there will be no > requirement for objects assigned to sys.stdout to derive from any of > the IO ABCs. Yes, that's actually in the PEP. But while I don't like Emin Shopper's argument (it sounds like too much like the classic static type checking argument), I am not about to remove the @abstractmethod decorator or its semantics (that it must be overridden in order to be allowed to instantiate the class). You don't need to use it (and I recommend thinking twice before you use it). But it makes sense in some cases to insist that someone who goes through the trouble of inheriting from an ABC gets an error if they forget to implement one of the methods that are deemed 'essential' to that ABC. Concerning the use case where you'd like to claim to implement an ABC but happen to know that a particular method will never be called, I suspect that this is the exception of ABC usage rather than the common case, so I'd rather cater to the common case. ESNPS. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tcdelaney at optusnet.com.au Wed Apr 25 22:29:43 2007 From: tcdelaney at optusnet.com.au (Tim Delaney) Date: Thu, 26 Apr 2007 06:29:43 +1000 Subject: [Python-3000] Fixing super anyone? References: <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com><76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com><76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> Message-ID: <002401c78778$75fb7eb0$0201a8c0@ryoko> From: "Guido van Rossum" > class E(D): pass > > print E().f() > > This prints DDBCA which surely isn't right. > > Sounds like the classic bug in such attempts. Yep - missing overridden methods tend to either do the above, or end up in infinite recursion. My bytecode hacking version doesn't suffer from these problems ... >>> class autosuper(object): ... __metaclass__ = _autosuper ... >>> class A(autosuper): ... def f(self): ... print 'A:', super ... >>> class B(A): ... def f(self): ... print 'B:', super ... super.f() ... >>> class C(A): ... def f(self): ... print 'C:', super ... super.f() ... >>> class D(B, C): ... pass ... >>> class E(D): ... def f(self): ... print 'E:', super ... super.f() ... >>> class F(E, A): ... pass ... >>> F().f() E: , > B: , > C: , > A: , > What I haven't worked out yet is if you should be able to do the following: class A(autosuper): def f(self): print 'A:', super class B(A): def f(self): def inner(): print 'B:', super super.f() inner() Should the call to inner() result in a call to A.f? Currently my bytecode version doesn't do this. I think this should be addressed in the PEP, as well as my proposal to have super(args) work as super.func(args) when called inside func. Tim Delaney From emin.shopper at gmail.com Wed Apr 25 22:44:49 2007 From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper) Date: Wed, 25 Apr 2007 16:44:49 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <20070425121618.6417.JCARLSON@uci.edu> References: <76fd5acf0704251037s17761b66t8fcbe7a62a3b31ff@mail.gmail.com> <32e43bb70704251149l3acca60ak70d30da2b4dd4395@mail.gmail.com> <20070425121618.6417.JCARLSON@uci.edu> Message-ID: <32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com> On 4/25/07, Josiah Carlson wrote: > > > If your unit tests take hours to run, then you aren't going to get > anything useful from the ABC requirements for hours either (unless you > are pre-instantiating everything you are going to use during program > execution, in which case this could become your unittest). No, the ABC requirements are checked as soon as a module is imported (i.e., classes are checked at definition time not when an instance is created). I have been using my own ABC implementation for quite a while and it has repeatedly proven it's value when I create a new derived class and forget to implement a necessary method. Please remember that this isn't about Java style type checking and > verification during compilation. This is, strictly speaking, ABC > checking on object instantiation, method invocation, and possibly even > argument verification. This isn't the magic bullet you are looking for > (you'll have to go to one of those static typed languages for that). Again, I disagree. With metaclasses it is easy and useful to have checking at class definition (see http://web.mit.edu/~emin/www/source_code/py_abc/abc.py for an example of what I mean). Generally I don't want static checking, but occasionally it is nice to have something like that and ABCs provide a good way to do that. I'm not claiming that is the only benefit of ABCs. I'm just pointing out what I find to be a very useful feature of ABCs that I hope makes into python. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070425/43c2b864/attachment.htm From guido at python.org Wed Apr 25 22:58:45 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2007 13:58:45 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <5d44f72f0704251304o15f2e080wcb9ade83be56403@mail.gmail.com> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <5d44f72f0704251304o15f2e080wcb9ade83be56403@mail.gmail.com> Message-ID: On 4/25/07, Jeffrey Yasskin wrote: > On 4/25/07, Guido van Rossum wrote: > > Jeffrey, is there any way you can drop the top of the tree and going > > straight from Number to Complex -> Real -> Rational -> Integer? These > > are the things that everyone with high school math will know. > > I think yes, if you can confirm that the > import numbers > class Ring(AdditiveGroup): ... > numbers.Complex.__bases__ = (Ring,) + numbers.Complex.__bases__ > hack I mention under IntegralDomain will make isinstance(int, Ring) return True. Hm... That would require built-in types to be mutable (by mere mortals) which would kill the idea of multiple interpreters. AFAIK the only user of multiple interpreters is mod_python, which is a pretty popular Apache module... (Google claims 1.8M hits, vs. 5.66M for mod_perl, the market leader.) I guess the alternative would be to keep the mathematical ones in but make it so that mere mortals (among which I count myself :) never need to know about them. Sort of like metaclasses, for most people. But this still presents a bit of a problem. If we insist that built-in types are immutable, whenever a mathematician needs a new kind of algebraic category (you mention a few in the PEP), they have no way to declare that int or float are members of that category. They could write things like class MyFloat(float, DivisionRing): pass but this would be fairly painful. I would like to let users modify the built-ins, but such changes ought to be isolated from other interpreters running in the same address space. I don't want to be the one to tell the mod_python community they won't be able to upgrade to 3.0... > Do you want "Number" to be equivalent to Ring+Hashable (or a > Haskell-like Ring+Hashable+__abs__+__str__)? I don't think a "Number" > class is strictly necessary since people can check for Complex or Real > directly, and one of those two is probably what they'll expect after > knowing that something's a number. Fair enough, you could start with Complex. > Also, I don't see much point in putting Rational between Real and > Integer. The current hierarchy is Real (int, float, Decimal, rational) > :> Integer (int) and Real :> FractionalReal (float, Decimal, > rational), but Integer and FractionalReal are just siblings. Why? Why not have a properfraction() on Integer that always returns 0 for the fraction? The current math.modf() works just fine on ints (returning a tuple of floats). And why distinguish between Real and FractionalReal? What's the use case for reals that don't know how to compute their fractional part? > I'm wondering how many people are writing new numeric types who don't > know the abstract algebra. It's easy to know the algebra without recalling the terminologies. Lots of programmers (myself included!) are largely self-taught. > I think we should be able to insulate > people who are just using the types from the complexities underneath, > and the people writing new types will benefit. Or will seeing "Ring" > in a list of superclasses be enough to confuse people? Shouldn't only mathematicians be tempted to write classes deriving from Ring? Everyone else is likely to subclass one of the more familiar types, from Complex onwards. PS. I've checked your PEP in, as PEP3141. We might as well keep a fairly complete record in svn, even if we decide to cut out the algebraic foundational classes. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jcarlson at uci.edu Wed Apr 25 23:11:47 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 25 Apr 2007 14:11:47 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com> References: <20070425121618.6417.JCARLSON@uci.edu> <32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com> Message-ID: <20070425140134.641D.JCARLSON@uci.edu> "Emin.shopper Martinian.shopper" wrote: > On 4/25/07, Josiah Carlson wrote: > > If your unit tests take hours to run, then you aren't going to get > > anything useful from the ABC requirements for hours either (unless you > > are pre-instantiating everything you are going to use during program > > execution, in which case this could become your unittest). > > No, the ABC requirements are checked as soon as a module is imported (i.e., > classes are checked at definition time not when an instance is created). I > have been using my own ABC implementation for quite a while and it has > repeatedly proven it's value when I create a new derived class and forget to > implement a necessary method. But that is not part of the ABC PEP (according to my most recent reading). See the "ABC Support Framework" portion. Only instantiation triggers the "you didn't implement method X". Certainly you can add metaclasses to handle checking for "does class X implement method Y", but it isn't going to be included out of the box. > I'm not claiming that is the only benefit of ABCs. I'm just pointing out > what I find to be a very useful feature of ABCs that I hope makes into > python. I doubt this particular feature will make it in. It seems too "B&D"-static typing to me (and others). - Josiah From guido at python.org Wed Apr 25 23:19:27 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2007 14:19:27 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <20070425140134.641D.JCARLSON@uci.edu> References: <20070425121618.6417.JCARLSON@uci.edu> <32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com> <20070425140134.641D.JCARLSON@uci.edu> Message-ID: On 4/25/07, Josiah Carlson wrote: > "Emin.shopper Martinian.shopper" wrote: > > On 4/25/07, Josiah Carlson wrote: > > > If your unit tests take hours to run, then you aren't going to get > > > anything useful from the ABC requirements for hours either (unless you > > > are pre-instantiating everything you are going to use during program > > > execution, in which case this could become your unittest). > > > > No, the ABC requirements are checked as soon as a module is imported (i.e., > > classes are checked at definition time not when an instance is created). I > > have been using my own ABC implementation for quite a while and it has > > repeatedly proven it's value when I create a new derived class and forget to > > implement a necessary method. > > But that is not part of the ABC PEP (according to my most recent reading). > See the "ABC Support Framework" portion. Only instantiation triggers > the "you didn't implement method X". Certainly you can add metaclasses > to handle checking for "does class X implement method Y", but it isn't > going to be included out of the box. > > > I'm not claiming that is the only benefit of ABCs. I'm just pointing out > > what I find to be a very useful feature of ABCs that I hope makes into > > python. > > I doubt this particular feature will make it in. It seems too > "B&D"-static typing to me (and others). IIUC, Emin's framework requires you to explicitly declare any class as abstract that has abstract methods. IMO this is a requirement to redundancy which I find unpythonic. So indeed I am not about to add this to the PEP. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jimjjewett at gmail.com Wed Apr 25 23:23:25 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 25 Apr 2007 17:23:25 -0400 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> Message-ID: On 4/25/07, Jeffrey Yasskin wrote: > class MonoidUnderPlus(Abstract): Is this useful? Just because two things are both Monoid instances doesn't mean I can add them -- they have to be part of the same Monoid. By the time you do assert isinstance(a, MonoidUnderPlus) assert isinstance(b, MonoidUnderPlus) assert isinstance(a, b.__class__) or isinstance(b, a.__class__) I'm not sure how much you've really saved. > **Open issue:** Do we want to give people a choice of which of the > following to define, or should we pick one arbitrarily?:: > def __neg__(self): > """Must define this or __sub__().""" > return self.zero() - self > def __sub__(self, other): > """Must define this or __neg__().""" > return self + -other Probably better to pick one; then it is clear that they have to override something, and there won't be as many accidental infinite loops. If they really want to define the other, they can just do def __neg__(self): super(_this_class__, self).__neg__() The decorator doesn't know it isn't a real override. -jJ From emin.shopper at gmail.com Wed Apr 25 23:56:13 2007 From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper) Date: Wed, 25 Apr 2007 17:56:13 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <20070425121618.6417.JCARLSON@uci.edu> <32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com> <20070425140134.641D.JCARLSON@uci.edu> Message-ID: <32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com> On 4/25/07, Guido van Rossum wrote: > IIUC, Emin's framework requires you to explicitly declare any class as > abstract that has abstract methods. I believe that is an easily fixable implementation issue, which I would be happy to remedy. I think a bigger issue is whether to check at definition time or check at instantiation time. It seems to me the former has the benefit of catching errors earlier and potentially saving time. I don't immediately see the advantage of the latter (probably that is due to my lack of imagination). I would like to see the PEP include the ability to (optionally) enforce ABC requirements at definition time. For example, you could set an attribute ( e.g., __check_abstract__) or inherit from something special if you want to turn enforcement on (if not checking at def time is the default) or turn it off (if checking at def time is the default). I am not recommending that anyone be forced to conform to this to use anything in the standard library. I think this would be useful, not conflict with the other goals of the PEP, and be relatively easy to implement in pure python (as illustrated by my example implementation). If others find this useful (or at least not objectionable), I would be happy to help with the implementation or writing additional material for the PEP or even propose it separately if people think this distracts from the point of the original ABC PEP. Thanks, -Emin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070425/ebfe0ef7/attachment.html From jimjjewett at gmail.com Thu Apr 26 00:10:23 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 25 Apr 2007 18:10:23 -0400 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) Message-ID: The current ABC proposal is to use isinstance as the test; Jeffrey Yaskin's numbers PEP highlighted the weakness there with a concrete example. If you need to an abstraction less powerful than an existing ABC, you're out of luck; you can't just assert that the existing class is already sufficient, nor can you expect everyone else to use multiple annotations. Short of allowing more __bases__ surgery, we need a function parallel to isinstance, which at least makes 3rd-party registration possible. I suspect Phillip will say that we really need to make the ABCs generic functions... but I disagree; I'm not sure that 3rd-party adapters should even be allowed by default, but it *should* be simple to create an ABC that does take them. Perhaps def isexample(obj, ABC): for cls in obj.__class__.__mro__: result = ABC.meets(obj, cls) if result: return result return False class Abstract... # override this with a dictionary to allow 3rd-party registration _good_enough = () @classmethod def meets(cls, obj=None, objclass=None): if objclass is cls: # covers isinstance return obj if objclass in cls._good_enough: # Nothing is, by default return cls._good_enough.[objclass](obj) @classmethod def assert_sufficient(cls, objclass, adapter): cls[objclass]=adapter On 4/25/07, Jeffrey Yasskin wrote: > If someone needs to split them later, they can use code like:: > import numbers > class IntegralDomain(Ring): ... > numbers.Integral.__bases__ = (IntegralDomain,) + numbers.Integral.__bases__ This only works with old-style classes, which are going away. >>> class Abstract1(object): pass >>> class Abstract2(object): pass >>> Abstract1.__bases__ = (Abstract2,) + Abstract1.__bases__ Traceback (most recent call last): File "", line 1, in Abstract1.__bases__ = (Abstract2,) + Abstract1.__bases__ TypeError: __bases__ assignment: 'Abstract2' deallocator differs from 'object' From jimjjewett at gmail.com Thu Apr 26 00:20:18 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 25 Apr 2007 18:20:18 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com> References: <20070425121618.6417.JCARLSON@uci.edu> <32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com> <20070425140134.641D.JCARLSON@uci.edu> <32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com> Message-ID: On 4/25/07, Emin.shopper Martinian.shopper wrote: > I think a bigger issue is whether to check at definition time or check at > instantiation time. It seems to me the former has the benefit of catching > errors earlier and potentially saving time. I don't immediately see the > advantage of the latter (probably that is due to my lack of imagination). The advantage of waiting is that you can *have* (partially) abstract classes. So long as you never instantiate them directly (super calls are OK), everything is fine. If you want an earlier check, then just create an instance of your class at import time (or, alternatively, in your unit tests). If it can be created, it passed the test. -jJ From guido at python.org Thu Apr 26 00:27:48 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2007 15:27:48 -0700 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: Message-ID: (Forgot to answer this promptly.) On 4/21/07, Jim Jewett wrote: [PEP 3119] > > > * "Should we also implement the issubset and issuperset methods found > > > on the set type in Python 2? As these are just aliases for __le__ and > > > __ge__, I'm tempted to leave these out." [Brett] > > > Leave them out. Not terribly needed plus it is better to start out > > > small. They can easily be added later if called for. > > I think the names are more sensible than repurposing the numeric > operators. The "concrete" implementation can just forward to other > name, so it doesn't cost much. But the numeric operators are what everybody uses. So it makes more sense to define these, and only these. It probably isn't even necessary to implement the 'r' variants (__ror__ etc.) since these only work if the other argument is also a Set. [Guido] > > I doubt that 'set' will be inherited from much (except for trivial > > stuff) but returning a 'set' does make some sense. Although for > > trivial subclasses (e.g. adding a different repr or a brand new > > method) you'd like the results also to be an instance of the subclass. > > If we could find a way to do that it would be best. > > You could use a variant of the numeric operator resolution. > > The left class wins, unless the right class is a subclass of the left. > If this class does not have a constructor that accepts an iterable, > (maybe fall back to the other class, then) fall back to a builtin set. The problem isn't which side wins. The problem is what API to use to create a new set. Since this is an *abstract* class, it's wrong to just calling self.__class__() and assume its constructor takes an iterable of values. > In the Hashable section: > > > Another constraint is that hashable objects, once created, > > should never change their value (as compared by ==) or their hash > > value. If a class cannot guarantee this, it should not derive from > > Hashable; if it cannot guarantee this for certain instances only, > > __hash__ for those instances should raise a TypeError exception. > > Why not just return -1 (unless/until the value is stable)? Is the -1 > special case being phased out too? I don't recall that -1 was ever a special case at the Python level; that was only done at the C level to make error checking easier. Returning any hash value at all implies a guarantee that the object value won't ever change (not just the hash value!); if you can't guarantee that (e.g. for hash(([], []))) then you shouldn't return a hash value. I don't think that hashing all objects with an unstable value together on -1 supports a valid use case. > ... > I would say that Searchable might be worth separating into a Container > subclass, but don't worry about the speed differences. Many > containers are small enough that constant effects are more important > than big-O. When big-O does matter, then the user may well want to > distinguish between the O(1) and O(log N) case. I think this gets > detailed enough that it isn't worth putting in the common library. Right. What is your specific proposal here? Adding Searchable(Container) with the implication that __contains__ takes a sequence as well as a single value? How should the type of the argument be described? Should it be a Sequence? a Searchable? something that is an instance of self.__class__? > Why must a HashableSet or MutableSet be composable? If this is > because you figure any useful set has those properties (and I don't > think so, when doing uniquification), then Set and ComposableSet > should become BasicSet and Set? Hmm, good point! This is in fact listed as an open issue for HashableSet. If we can find a good use case for non-composable sets that should nevertheless be hashable, we may end up with four classes: set, composable set, hashable set, composable hashable set. (I think you're hinting at an example; can you work it out a bit more?) For mutable sets I think the case is weaker; these already support in-place operations (|= etc.) which can easily and (usually efficiently) be implemented using the primitives add() and discard(), and it would be somewhat weird to have |= without |. > ... > > Note that in the current sandbox sandbox (which doesn't have > *Ordered), the name BasicSet is used for a Set which isn't Sized. > (Except that Set doesn't actually inherit from BasicSet, as I assume > it should do.) The sandbox/abc/abc.py code is way out of date. I'll try to update it. > Continuing with the current sandbox > > Why must @abstractmethod come last, when mixed with other decorators? Because it sets a function attribute. It is conceivable that other decorators would make funtion attributes readonly. However the example given is currently wrong -- @classmethod doesn't pass on function attributes at all! > IterableMapping needs to override __iter__; as written, it returns the > empty iterator. No, it is still an abstract method. > Should Sequence.index take optional start and stop arguments? > > def index(self, value, start=0, stop=None): > if stop is None: > stop = len(self) > for i, elem in enumerate(self): > if i < start: > continue > if stop < i: > break > if elem == value: > return i > raise ValueError Good question. I'm tempted to say no, and consider that a list-specific extension. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From emin.shopper at gmail.com Thu Apr 26 00:32:35 2007 From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper) Date: Wed, 25 Apr 2007 18:32:35 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <20070425121618.6417.JCARLSON@uci.edu> <32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com> <20070425140134.641D.JCARLSON@uci.edu> <32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com> Message-ID: <32e43bb70704251532o6370254cyf574d0d3c2b19465@mail.gmail.com> On 4/25/07, Jim Jewett wrote: > > On 4/25/07, Emin.shopper Martinian.shopper wrote: > > > I think a bigger issue is whether to check at definition time or check > at > > instantiation time. It seems to me the former has the benefit of > catching > > errors earlier and potentially saving time. I don't immediately see the > > advantage of the latter (probably that is due to my lack of > imagination). > > The advantage of waiting is that you can *have* (partially) abstract > classes. So long as you never instantiate them directly (super calls > are OK), everything is fine. Good point. However, you can do this without precluding def time checks. For example, you can have any class which has abstract methods not be forced to satisfy parent ABC rules since it is still abstract, or you can provide another way to mark the class as partially abstract (my initial implemenation does this by having the partially abstract class inherit from both the parent abstract class and the class that all abstract classes inherit from). You could reasonably argue that the benefit of def time enforcement is outweighed by the need to do something special for a class that inherits from an abstract base class but has no abstract methods itself. If you want an earlier check, then just create an instance of your > class at import time (or, alternatively, in your unit tests). If it > can be created, it passed the test. Yes, I could do that assuming that the inputs to the class __init__ method can be constructed quickly. Personally, I would find it preferable to have to do something slightly different for partially abstract classes with no abstract methods, but I guess it's a matter of taste. -Emin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070425/f6271f55/attachment.html From exarkun at divmod.com Thu Apr 26 00:40:25 2007 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Wed, 25 Apr 2007 18:40:25 -0400 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: Message-ID: <20070425224025.19381.1937422068.divmod.quotient.5365@ohm> On Wed, 25 Apr 2007 18:10:23 -0400, Jim Jewett wrote: >The current ABC proposal is to use isinstance as the test; Jeffrey >Yaskin's numbers PEP highlighted the weakness there with a concrete >example. > >If you need to an abstraction less powerful than an existing ABC, >you're out of luck; you can't just assert that the existing class is >already sufficient, nor can you expect everyone else to use multiple >annotations. I'm sure everyone is already aware of the behavior of the classImplements and directlyProvides functions available in zope.interface, which exactly satisfy this use-case in the interface world. Jean-Paul From jimjjewett at gmail.com Thu Apr 26 00:48:13 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 25 Apr 2007 18:48:13 -0400 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <5d44f72f0704251304o15f2e080wcb9ade83be56403@mail.gmail.com> Message-ID: On 4/25/07, Guido van Rossum wrote: > ... If we insist that built-in types are immutable ... They > could write things like > class MyFloat(float, DivisionRing): pass > but this would be fairly painful. Worse than that once you get 3rd parties involved. try: from gmpy import mpf except ImportError: class mpf(object): pass class MyFloat(float, DivisionRing, gmpy): pass # err ... will this even work at all, with two concrete types? > On 4/25/07, Jeffrey Yasskin wrote: > > will seeing "Ring" > > in a list of superclasses be enough to confuse people? Yes. That is part of the problem with twisted -- there are so many layers of abstraction that it takes a while to find something. If users ever see "Ring" outside of a footnote or the advanced math section, then something went wrong. -jJ From guido at python.org Thu Apr 26 01:32:03 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2007 16:32:03 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com> References: <20070425121618.6417.JCARLSON@uci.edu> <32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com> <20070425140134.641D.JCARLSON@uci.edu> <32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com> Message-ID: On 4/25/07, Emin.shopper Martinian.shopper wrote: > On 4/25/07, Guido van Rossum wrote: > > > IIUC, Emin's framework requires you to explicitly declare any class as > > abstract that has abstract methods. > > I believe that is an easily fixable implementation issue, which I would be > happy to remedy. > > I think a bigger issue is whether to check at definition time or check at > instantiation time. It seems to me the former has the benefit of catching > errors earlier and potentially saving time. I don't immediately see the > advantage of the latter (probably that is due to my lack of imagination). I'm confused. Can you show a simple example of something that can be checked at definition time? Using the machinery currently in the PEP, any class that inherits an abstract method and doesn't override it is automatically an abstract class itself. There's nothing to complain about at definition time. > I would like to see the PEP include the ability to (optionally) enforce ABC > requirements at definition time. For example, you could set an attribute > (e.g., __check_abstract__) or inherit from something special if you want to > turn enforcement on (if not checking at def time is the default) or turn it > off (if checking at def time is the default). I am not recommending that > anyone be forced to conform to this to use anything in the standard library. So I'm confused about what you can enforce. (And I do hope you will read the PEP before replying.) > I think this would be useful, not conflict with the other goals of the PEP, > and be relatively easy to implement in pure python (as illustrated by my > example implementation). If others find this useful (or at least not > objectionable), I would be happy to help with the implementation or writing > additional material for the PEP or even propose it separately if people > think this distracts from the point of the original ABC PEP. Your example implementation was a bit too much code for me to read. :-) Please show some usage examples instead. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Apr 26 01:41:40 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2007 16:41:40 -0700 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <20070425224025.19381.1937422068.divmod.quotient.5365@ohm> References: <20070425224025.19381.1937422068.divmod.quotient.5365@ohm> Message-ID: On 4/25/07, Jean-Paul Calderone wrote: > I'm sure everyone is already aware of the behavior of the classImplements > and directlyProvides functions available in zope.interface, which exactly > satisfy this use-case in the interface world. Don't be so sure; I wasn't (aware, that is). At this point it doesn't look like there will be an Interfaces PEP. If you want one, or if you want to borrow specific ideas from zope.interface, now would be the time to bring them up and enlighten the rest of us about them. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Apr 26 01:43:45 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2007 16:43:45 -0700 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: Message-ID: This is a very good point. Perhaps we can come up with a way to make isinstance and issubclass into something like GFs (without requiring the whole GF machinery). I'll think about it some more. --Guido On 4/25/07, Jim Jewett wrote: > The current ABC proposal is to use isinstance as the test; Jeffrey > Yaskin's numbers PEP highlighted the weakness there with a concrete > example. > > If you need to an abstraction less powerful than an existing ABC, > you're out of luck; you can't just assert that the existing class is > already sufficient, nor can you expect everyone else to use multiple > annotations. > > Short of allowing more __bases__ surgery, we need a function parallel > to isinstance, which at least makes 3rd-party registration possible. > I suspect Phillip will say that we really need to make the ABCs > generic functions... but I disagree; I'm not sure that 3rd-party > adapters should even be allowed by default, but it *should* be simple > to create an ABC that does take them. > > Perhaps > > def isexample(obj, ABC): > for cls in obj.__class__.__mro__: > result = ABC.meets(obj, cls) > if result: > return result > return False > > class Abstract... > > # override this with a dictionary to allow 3rd-party registration > _good_enough = () > > @classmethod > def meets(cls, obj=None, objclass=None): > if objclass is cls: # covers isinstance > return obj > if objclass in cls._good_enough: # Nothing is, by default > return cls._good_enough.[objclass](obj) > > @classmethod > def assert_sufficient(cls, objclass, adapter): > cls[objclass]=adapter > > > On 4/25/07, Jeffrey Yasskin wrote: > > > If someone needs to split them later, they can use code like:: > > import numbers > > class IntegralDomain(Ring): ... > > numbers.Integral.__bases__ = (IntegralDomain,) + numbers.Integral.__bases__ > > This only works with old-style classes, which are going away. > > >>> class Abstract1(object): pass > >>> class Abstract2(object): pass > >>> Abstract1.__bases__ = (Abstract2,) + Abstract1.__bases__ > > Traceback (most recent call last): > File "", line 1, in > Abstract1.__bases__ = (Abstract2,) + Abstract1.__bases__ > TypeError: __bases__ assignment: 'Abstract2' deallocator differs from 'object' > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jimjjewett at gmail.com Thu Apr 26 02:19:03 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 25 Apr 2007 20:19:03 -0400 Subject: [Python-3000] Need help completing ABC pep In-Reply-To: References: Message-ID: On 4/25/07, Guido van Rossum wrote: > On 4/21/07, Jim Jewett wrote: > [PEP 3119] > > > > * "Should we also implement the issubset and issuperset methods found > > > > on the set type in Python 2? As these are just aliases for __le__ and > > > > __ge__, I'm tempted to leave these out." > [Brett] > > > > Leave them out. Not terribly needed plus it is better to start out > > > > small. They can easily be added later if called for. > > I think the names are more sensible than repurposing the numeric > > operators. The "concrete" implementation can just forward to other > > name, so it doesn't cost much. > But the numeric operators are what everybody uses. So it makes more > sense to define these, and only these. ahh... because you want to ensure that implementors override the correct one. If you defined def union(self, other): return self.__or__(other) and I overrode union but not __or__, things would be inconsistent. But that's still true even if you don't define union in the ABC. Maybe add a sanity check in the actually-doing-something default implementation? def __or__(self, other): # If they overrode union but not __or__, honor the override. if self.union is not __this_class__.union: return self.union(other) new = set(self) new.update(other) return frozenset(new) > > In the Hashable section: > > > Another constraint is that hashable objects, once created, > > > should never change their value (as compared by ==) or their hash > > > value. If a class cannot guarantee this, it should not derive from > > > Hashable; if it cannot guarantee this for certain instances only, > > > __hash__ for those instances should raise a TypeError exception. > > Why not just return -1 (unless/until the value is stable)? Is the -1 > > special case being phased out too? > I don't recall that -1 was ever a special case at the Python level; > that was only done at the C level to make error checking easier. > Returning any hash value at all implies a guarantee that the object > value won't ever change (not just the hash value!); if you can't > guarantee that (e.g. for hash(([], []))) then you shouldn't return a > hash value. I don't think that hashing all objects with an unstable > value together on -1 supports a valid use case. If you want to enforce that, it is worth documenting. As nearly as I can tell, hash is used only by dict and set, and they always treat a hash of -1 as unhashable, so it won't get into the dictionary (or set). To me, this suggested it was OK to change value (if both values have the same hash), and it was OK to (once per object) change hash from -1 to a real value. I have used this for objects where: (a) The most significant attributes are immutable and known at creation (b) Additional attributes are not known at creation, so they may change from "None" to a value. (c) Equality depends on all attributes, and therefore could -- theoretically -- change. (d) In practice, several objects might pass through the same (partially completed) value, but only one would be in that state at a time. (In other words, the result of a==b was stable, even though the result of a What is your specific proposal here? Adding > Searchable(Container) with the implication that __contains__ takes a > sequence as well as a single value? Yes. > How should the type of the argument be described? Sequence would be fine if you want to leave yourself room to change your mind later. I wouldn't bother to describe it at all. def __contains__(self, item): atomic = super(Searchable, self).__contains__(item) # or __this_class__ if atomic: return atomic try: it = iter(item) except TypeError: return False # It isn't iterable, so give up # Note that this doesn't assume Sequence. It would say "bbbbaa" in "ab" return all(super(Searchable, self).__contains__(elt) for elt in it) > > Why must a HashableSet or MutableSet be composable? If this is > > because you figure any useful set has those properties (and I don't > > think so, when doing uniquification), then Set and ComposableSet > > should become BasicSet and Set? > Hmm, good point! This is in fact listed as an open issue for > HashableSet. If we can find a good use case for non-composable sets > that should nevertheless be hashable, we may end up with four classes: > set, composable set, hashable set, composable hashable set. (I think > you're hinting at an example; can you work it out a bit more?) Yes, but not immediately. :D > > Should Sequence.index take optional start and stop arguments? > Good question. I'm tempted to say no, and consider that a > list-specific extension. It is also a pretty important string method, unless it has been deprecated for find. -jJ From pje at telecommunity.com Thu Apr 26 02:31:10 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 25 Apr 2007 20:31:10 -0400 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: Message-ID: <5.1.1.6.0.20070425202858.04ee1ee8@sparrow.telecommunity.com> At 06:10 PM 4/25/2007 -0400, Jim Jewett wrote: >I suspect Phillip will say that we really need to make the ABCs >generic functions... Nope; they're *already* generic functions: hash(), len(), iter(), various operator.* functions, and I believe we're adding next(). I've got nothing to add to that list. :) From jimjjewett at gmail.com Thu Apr 26 02:31:39 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 25 Apr 2007 20:31:39 -0400 Subject: [Python-3000] ABC PEP -- dropping set.clear Message-ID: On 4/25/07, guido.van.rossum wrote: > + implementation.) **Open issues:** Forcing every mutable set > + to implement this may be a pain for such a fairly > + non-essential method. Perhaps just drop it? There are also comments worrying that an abstract implementation would be horribly slow -- but so what? Alex Martelli posted some stats (for dicts, I think) showing that (even today) clear was slower than just creating a new object. It still makes sense if you care about the "is" relation Since you are keeping "pop", the abstract implementation can at least approximate iteration. def clear(self): while self.pop(): pass -jJ From pje at telecommunity.com Thu Apr 26 02:35:01 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 25 Apr 2007 20:35:01 -0400 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: Message-ID: <5.1.1.6.0.20070425203455.05062ae8@sparrow.telecommunity.com> At 04:43 PM 4/25/2007 -0700, Guido van Rossum wrote: >This is a very good point. Perhaps we can come up with a way to make >isinstance and issubclass into something like GFs (without requiring >the whole GF machinery). > >I'll think about it some more. __special__ methods are how Python currently defines built-in GFs, so perhaps __isinstance__ and __issubclass__ slots (and their __r*__ counterparts) should suffice. OTOH, the old PEP 246 (implemented by both PyProtocols and zope.interface) just uses __conform__ and __adapt__ and basically works the same way as an __isinstance__/__risinstance__ pair... From pje at telecommunity.com Thu Apr 26 02:40:14 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 25 Apr 2007 20:40:14 -0400 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <20070425224025.19381.1937422068.divmod.quotient.5365@ohm> References: Message-ID: <5.1.1.6.0.20070425203654.04ee17d8@sparrow.telecommunity.com> At 06:40 PM 4/25/2007 -0400, Jean-Paul Calderone wrote: >On Wed, 25 Apr 2007 18:10:23 -0400, Jim Jewett wrote: > >The current ABC proposal is to use isinstance as the test; Jeffrey > >Yaskin's numbers PEP highlighted the weakness there with a concrete > >example. > > > >If you need to an abstraction less powerful than an existing ABC, > >you're out of luck; you can't just assert that the existing class is > >already sufficient, nor can you expect everyone else to use multiple > >annotations. > >I'm sure everyone is already aware of the behavior of the classImplements >and directlyProvides functions available in zope.interface, which exactly >satisfy this use-case in the interface world. I'm either misunderstanding Jim or you, because I don't see the relationship here. If I understand Jim correctly, he's actually asking for something like PyProtocols' "protocolIsSubsetOf" declaration -- something that would be like dynamically adding a superclass to an existing interface in zope.interface. Whereas the features you're talking about sound like declaring that a class object itself implements an interface -- something apparently unrelated to the question at hand. From guido at python.org Thu Apr 26 02:39:38 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2007 17:39:38 -0700 Subject: [Python-3000] ABC PEP -- dropping set.clear In-Reply-To: References: Message-ID: On 4/25/07, Jim Jewett wrote: > On 4/25/07, guido.van.rossum wrote: > > + implementation.) **Open issues:** Forcing every mutable set > > + to implement this may be a pain for such a fairly > > + non-essential method. Perhaps just drop it? > > There are also comments worrying that an abstract implementation would > be horribly slow -- but so what? True. In the discussion of __hash__ returning 0 by default I even mention that I'd rather be slow and correct than fast and wrong. :-) > Alex Martelli posted some stats (for dicts, I think) showing that > (even today) clear was slower than just creating a new object. :) > It still makes sense if you care about the "is" relation > > Since you are keeping "pop", the abstract implementation can at least > approximate iteration. > > def clear(self): > while self.pop(): pass That would have to be def clear(self): while True: try: self.pop() except KeyError: break But yeah, I'll do this. Thanks for the reality check! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Thu Apr 26 02:38:52 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 26 Apr 2007 12:38:52 +1200 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> Message-ID: <462FF49C.3060503@canterbury.ac.nz> Paul Moore wrote: > OTOH, in Greg's case maybe he should just not bother inheriting from > the ABC. From what I hear, that shouldn't cause him any problems > ("duck typing isn't going away" is the message I've heard). Of course, > it's difficult to be sure without specific examples (on either side of > the debate). I gathered that some ABCs would be providing useful default implementations of some methods. That would be a reason for me to use one of the supplied ABCs instead of doing it all myself. -- Greg From greg.ewing at canterbury.ac.nz Thu Apr 26 02:47:19 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 26 Apr 2007 12:47:19 +1200 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> Message-ID: <462FF697.2060801@canterbury.ac.nz> Emin.shopper Martinian.shopper wrote: > Imagine you have a program that takes a long time to run. If you use > duck-typing or similar ideas you may watch the program run for hours or > days and then raise an AttributeError or NotImplemented error because a > derived class didn't implement a required method. You should have a way of exercising the program with a set of small test cases before using it for big runs. You need that anyway, because there are many other ways the program could fail at run-time that wouldn't be caught by this feature. This is just another version of the argument for static type checking, and all the same counter-arguments apply to it. -- Greg From guido at python.org Thu Apr 26 02:50:54 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2007 17:50:54 -0700 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <5.1.1.6.0.20070425203455.05062ae8@sparrow.telecommunity.com> References: <5.1.1.6.0.20070425203455.05062ae8@sparrow.telecommunity.com> Message-ID: On 4/25/07, Phillip J. Eby wrote: > At 04:43 PM 4/25/2007 -0700, Guido van Rossum wrote: > >This is a very good point. Perhaps we can come up with a way to make > >isinstance and issubclass into something like GFs (without requiring > >the whole GF machinery). > > > >I'll think about it some more. > > __special__ methods are how Python currently defines built-in GFs, so > perhaps __isinstance__ and __issubclass__ slots (and their __r*__ > counterparts) should suffice. Yeah. The pragmatic issues are (a) how easy is it to add this to the metaclass; (b) whether to keep the existing "abstract isinstance/issubclass" code which looks for __class__ and __bases__ attributes and believes whatever they return. > OTOH, the old PEP 246 (implemented by both PyProtocols and zope.interface) > just uses __conform__ and __adapt__ and basically works the same way as an > __isinstance__/__risinstance__ pair... Are pairs needed? I was hoping that isinstance(x, C) would only need to be overridden on the class C, and issubclass(D, C) would only need to be overridden on C as well. In particular, the use case brought up here is isinstance(42, Ring) and issubclass(int, Ring). So we'd want to add something to Ring. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Thu Apr 26 03:07:13 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 26 Apr 2007 13:07:13 +1200 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> <79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com> Message-ID: <462FFB41.8060507@canterbury.ac.nz> Paul Moore wrote: > My point was that as far as I am aware, > Guido's current position on ABCs is that they are, and will remain, > optional - people like myself (and Greg, from the sound of it) who > don't want to derive from ABCs will not be penalised by being excluded > from anything in the Python core or the stdlib. Maybe not in the core or stdlib, but it seems like there will be cases where one *is* penalised for not using the ABCs, if they get used the way the PEP proposes. From the PEP, it seems there are two purposes for these proposed ABCs: 1) To document what is meant when we say that something is a "sequence", "mapping", etc. This could be done simply by writing documentation. There is no need for anything embodied in the language. 2) To mark a class for the purposes of introspection. I thought we had generally agreed that this is an anti-pattern in Python. If I want my object to interoperate with someone else's code that tests for the presence of some ABC, then I *am* forced to use that ABC, even if it doesn't entirely suit my purposes. I'm -1 on including anything in the language, stdlib or docs that appears to officially sanction type testing as a normal style of programming. -- Greg From greg.ewing at canterbury.ac.nz Thu Apr 26 03:14:55 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 26 Apr 2007 13:14:55 +1200 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <5d44f72f0704251009w12eddd82l971f5d06b35b22fb@mail.gmail.com> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <20070425085350.6408.JCARLSON@uci.edu> <5d44f72f0704251009w12eddd82l971f5d06b35b22fb@mail.gmail.com> Message-ID: <462FFD0F.4040305@canterbury.ac.nz> Jeffrey Yasskin wrote: > If you write a function that can take > an arbitrary ring, it should Just Work on a python int, or any other > Complex subclass, There's no reason it shouldn't "just work", unless it's being anal and trying to make sure it's been given things of what it considers to be the proper type. -- Greg From greg.ewing at canterbury.ac.nz Thu Apr 26 03:27:13 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 26 Apr 2007 13:27:13 +1200 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <32e43bb70704251149l3acca60ak70d30da2b4dd4395@mail.gmail.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> <76fd5acf0704251037s17761b66t8fcbe7a62a3b31ff@mail.gmail.com> <32e43bb70704251149l3acca60ak70d30da2b4dd4395@mail.gmail.com> Message-ID: <462FFFF1.3040307@canterbury.ac.nz> Emin.shopper Martinian.shopper wrote: > If you believe that there exist data structures or algorithms where a > meaningful unit test takes many minutes or hours, I wouldn't be worried about spending minutes to save hours later. As for unit tests taking hours -- I'm skeptical. Do you know of algorithms that it's provably impossible to test without it taking hours? Thirdly -- your program can *still* fail after many hours due to one of the many things your static checks don't and can't cover. -- Greg From exarkun at divmod.com Thu Apr 26 04:06:05 2007 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Wed, 25 Apr 2007 22:06:05 -0400 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <5.1.1.6.0.20070425203654.04ee17d8@sparrow.telecommunity.com> Message-ID: <20070426020605.19381.476607338.divmod.quotient.5402@ohm> On Wed, 25 Apr 2007 20:40:14 -0400, "Phillip J. Eby" wrote: >At 06:40 PM 4/25/2007 -0400, Jean-Paul Calderone wrote: >>On Wed, 25 Apr 2007 18:10:23 -0400, Jim Jewett >>wrote: >> >The current ABC proposal is to use isinstance as the test; Jeffrey >> >Yaskin's numbers PEP highlighted the weakness there with a concrete >> >example. >> > >> >If you need to an abstraction less powerful than an existing ABC, >> >you're out of luck; you can't just assert that the existing class is >> >already sufficient, nor can you expect everyone else to use multiple >> >annotations. >> >>I'm sure everyone is already aware of the behavior of the classImplements >>and directlyProvides functions available in zope.interface, which exactly >>satisfy this use-case in the interface world. > >I'm either misunderstanding Jim or you, because I don't see the relationship >here. If I understand Jim correctly, he's actually asking for something >like PyProtocols' "protocolIsSubsetOf" declaration -- something that would >be like dynamically adding a superclass to an existing interface in >zope.interface. Whereas the features you're talking about sound like >declaring that a class object itself implements an interface -- something >apparently unrelated to the question at hand. > Ugh, you're at least half right. I didn't read the implementation of ABC.meets carefully enough to notice that it's actually an adapter registry, not just a predicate (meets seems like a funny name for that method). classImplements would have satisfied the adapterless use-case by letting Jim declare that someone else's class definition actually implements the feature set that some unit of code he wants to use that class with is expecting. With the adapter registry, I don't see why this isn't just PEP 246 adapt() (if I am reading one of your later mails correctly though, you think it is too). I'm not particularly familiar with protocolIsSubsetOf, but it wouldn't surprise me at all if it could also satisfy this use-case. There are lots of ways to do this, many of them fairly simple. Using inheritance is the part that makes it hard, since inheritance is controlled by the wrong party in most cases and comes with unrelated features that are, at best, irrelevant to the particular use case and at worst actively detrimental. I'm sure a way around this can be invented, I just don't see why it should be. Jean-Paul From emin.shopper at gmail.com Thu Apr 26 04:21:43 2007 From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper) Date: Wed, 25 Apr 2007 22:21:43 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <20070425121618.6417.JCARLSON@uci.edu> <32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com> <20070425140134.641D.JCARLSON@uci.edu> <32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com> Message-ID: <32e43bb70704251921n4fb470c2uf9154180a75c593c@mail.gmail.com> On 4/25/07, Guido van Rossum wrote: > I'm confused. Can you show a simple example of something that can be > checked at definition time? Sure. Below is a slightly modified example from the doctest: >>> class AbstractCar(AbstractBaseClass): # inherit from AbstractBaseClass to make something abstract ... @Abstract ... def Drive(self,x): pass To make a class abstract, you inherit from AbstractBaseClass in order to get the metaclass set properly and then use a function decorator to declare methods abstract. This is similar to the proposed implementation in your abc.py and equally pythonic or unpythonic. The following shows what happens when you define a class that doesn't override the abstract methods: >>> try: # illustrate what happens when you don't implement @Abstract methods ... class fails(AbstractCar): # an erroneous implementation of AbstractCar ... pass ... except AssertionError, e: # doctest: +ELLIPSIS ... print e Class must override Drive to implement: []. The exception gets raised at definition time because the __call__ method of the metaclass inherited from AbstractBaseClass checks whether abstract methods have been implemented. In your implementation, you use the __new__ method of the metaclass to do the check at instantiation. If desired you could enable/disable checking at definition and instantiation separately. A slightly tricky point is that if something inherits DIRECTLY from AbstractBaseClass (e.g., AbstractCar) then it is allowed to have abstract methods that have not been overridden. This allows you to define a partial abstraction as shown below: >>> class AbstractFastCar(AbstractCar,AbstractBaseClass):# inherit from ABC ... @Abstract # so that interpreter ... def DriveFast(self): pass # doesn't worry about ... # Drive being abstract My implementation will consider this a valid definition because AbstractFastCar inherits DIRECTLY from AbstractBaseClass and is therefore allowed to have abstract methods that have not been overridden. In an earlier post you claimed that my implementation forces you to declare a class abstract redundantly (and therefore unpythonic-ly) when this should be inferred from the class having abstract methods. I believe you were referring to the partially abstract class example above. I agree that this is redundant and unpythonic, but I believe this could be fixed by having the metaclass __call__ method check whether the class itself defines any abstract methods so you wouldn't need partial abstractions which define abstract methods to still inherit from AbstractBaseClass directly. So far I believe that the main reasonable objection raised to definition time checks is in how the following example would be handled: >>> # Define a class that provides a helper function but is still >>> # abstract in the sense that children must implement Drive. >>> class AbstractCarHelper(AbstractCar,AbstractBaseClass): ... def DriveHelper(self): ... print 'Vroom!' The example above has no abstract methods, but if you want to make this kind of partial abstraction a legal definition you need to tell the interpreter that this is still an abstract class by inheriting directly from AbstractBaseClass. If we want def time checking, I don't see a way to make the above definition legal without somehow telling the interpreter to allow it to be abstract. I don't think this is unpythonic since inheriting from AbstractBaseClass is non-redundant. In fact, one could argue that it is good since it lets the programmer that AbstractCarHelper is still an abstract class even though it does not directly define any abstract methods. If you ignore the last example, I don't think having def time checks for ABC enforcement has any disadvantages. You can easily turn off the def time checking or have it off by default and turn it on if desired. Since def time checks are done by __call__ and instantiation checks are done by __new__, this shouldn't interfere with anything in the original ABC PEP. So in my mind, the only reason to object to def time checks is if it bothers you that the last example must inherit directly from AbstractBaseClass. > Using the machinery currently in the PEP, > any class that inherits an abstract method and doesn't override it is > automatically an abstract class itself. There's nothing to complain > about at definition time. Yes, I see that. What I'm proposing is to have __call__ in the metaclass complain if a class does not inherit directly from AbstractBaseClass and does not have any abstract methods. >From your point of view, if you have def time checks turned off by default then the difference in using my proposal vs. the original ABC proposal all comes from the last example. For example, your Iterator class in abc.pyinherits from Iterable (which is abstract) and declares its own abstract method. Under my current implementation this would raise an error unless Iterator also inherits from AbstractBaseClass. This is easily fixable by having my implementation allow classes which declare their own abstract methods to be abstract. The incompatibly I pointed out in my last example is illustrated by the Set class in your abc.py. I believe that you want Set to be an ABC which requires children to support the ABC requirements of its parents while providing useful implementations of __le__, __lt__, and __eq__. In my implementation you would need to have Set either inherit directly from AbstractBaseClass or define at least one abstract method to prevent an error at def time (if def time checks are on at all). Note that I don't think it would be unpythonic to have Set inherit from AbstractBaseClass since it provides someone reading the source code with the useful information that this is an ABC and not a concrete implementation. In summary, let me try to clarify what I see as the only incompatibility between the current PEP and an ABC implementation: with def time checks turned on you would need partial abstractions like Set to somehow indicate that they are abstract by either inheriting directly from AbstractBaseClass or defining at least one abstract method. If people don't like that, then I don't have any response. But if people are worried about B&D and other things, I think those are non-issues. B&D at runtime is no better than B&D at def time (I think B&D at runtime is worse). > > Your example implementation was a bit too much code for me to read. > :-) No problem. Unfortunately, I think I might have made the example usage in this post a little too long to read (but hopefully not). :) -Emin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070425/761c38c7/attachment.html From emin.shopper at gmail.com Thu Apr 26 04:29:13 2007 From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper) Date: Wed, 25 Apr 2007 22:29:13 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <462FFFF1.3040307@canterbury.ac.nz> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> <76fd5acf0704251037s17761b66t8fcbe7a62a3b31ff@mail.gmail.com> <32e43bb70704251149l3acca60ak70d30da2b4dd4395@mail.gmail.com> <462FFFF1.3040307@canterbury.ac.nz> Message-ID: <32e43bb70704251929s27714009r6ed6c9487559d9c3@mail.gmail.com> On 4/25/07, Greg Ewing wrote: > > Emin.shopper Martinian.shopper wrote: > > > If you believe that there exist data structures or algorithms where a > > meaningful unit test takes many minutes or hours, > > I wouldn't be worried about spending minutes to save hours > later. OK, but I would rather not have to spend 10 minutes waiting for a unit test each time I edit a file when I can easily have the test done in seconds at import time. As for unit tests taking hours -- I'm skeptical. Do you > know of algorithms that it's provably impossible to test > without it taking hours? Imagine you have to query a database and do something with the resulting data. Or imagine some complicated network interaction among p2p clients. Or imagine numerical simulation over a very fine mesh. I've worked on all of these and the unit-tests can easily take long enough that you don't want to run them every time you make any changes. Thirdly -- your program can *still* fail after many hours > due to one of the many things your static checks don't and > can't cover. Of course. I'm not saying ABCs will solve all bugs. What I am saying is that ABCs provide an easy way to catch simple bugs that can sometimes save you a lot of time. Sure you could produce fake data for input to your unit-tests or otherwise contrive things so that you have very quick unit-tests that really only check if things have the right methods. But if you want a quick check that derived classes implement the right methods, I think ABCs are a good solution. At least, they're a solution I find useful. I'm not trying to advocate that you should use them only that they should be available. -Emin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070425/9cca2a15/attachment.htm From emin.shopper at gmail.com Thu Apr 26 04:37:41 2007 From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper) Date: Wed, 25 Apr 2007 22:37:41 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <32e43bb70704251921n4fb470c2uf9154180a75c593c@mail.gmail.com> References: <20070425121618.6417.JCARLSON@uci.edu> <32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com> <20070425140134.641D.JCARLSON@uci.edu> <32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com> <32e43bb70704251921n4fb470c2uf9154180a75c593c@mail.gmail.com> Message-ID: <32e43bb70704251937k6d81bb0ao3ee752806d8a453a@mail.gmail.com> References to __call__ in my post should have been to __init__. Apologies for the error. -Emin On 4/25/07, Emin.shopper Martinian.shopper wrote: > > On 4/25/07, Guido van Rossum wrote: > > > I'm confused. Can you show a simple example of something that can be > > checked at definition time? > > > Sure. Below is a slightly modified example from the doctest: > > >>> class AbstractCar(AbstractBaseClass): # inherit from AbstractBaseClass > to make something abstract > ... @Abstract > ... def Drive(self,x): pass > > To make a class abstract, you inherit from AbstractBaseClass in order to > get the metaclass set properly and then use a function decorator to declare > methods abstract. This is similar to the proposed implementation in your > abc.py and equally pythonic or unpythonic. The following shows what > happens when you define a class that doesn't override the abstract methods: > > >>> try: # illustrate what happens when you don't implement @Abstract > methods > ... class fails(AbstractCar): # an erroneous implementation of > AbstractCar > ... pass > ... except AssertionError, e: # doctest: +ELLIPSIS > ... print e > Class must override Drive to implement: > []. > > The exception gets raised at definition time because the __call__ method > of the metaclass inherited from AbstractBaseClass checks whether abstract > methods have been implemented. In your implementation, you use the __new__ > method of the metaclass to do the check at instantiation. If desired you > could enable/disable checking at definition and instantiation separately. > > A slightly tricky point is that if something inherits DIRECTLY from > AbstractBaseClass (e.g., AbstractCar) then it is allowed to have abstract > methods that have not been overridden. This allows you to define a partial > abstraction as shown below: > > >>> class AbstractFastCar(AbstractCar,AbstractBaseClass):# inherit from > ABC > ... @Abstract # so that > interpreter > ... def DriveFast(self): pass # doesn't worry > about > ... # Drive being > abstract > > My implementation will consider this a valid definition because > AbstractFastCar inherits DIRECTLY from AbstractBaseClass and is therefore > allowed to have abstract methods that have not been overridden. In an > earlier post you claimed that my implementation forces you to declare a > class abstract redundantly (and therefore unpythonic-ly) when this should be > inferred from the class having abstract methods. I believe you were > referring to the partially abstract class example above. I agree that this > is redundant and unpythonic, but I believe this could be fixed by having the > metaclass __call__ method check whether the class itself defines any > abstract methods so you wouldn't need partial abstractions which define > abstract methods to still inherit from AbstractBaseClass directly. > > So far I believe that the main reasonable objection raised to definition > time checks is in how the following example would be handled: > > >>> # Define a class that provides a helper function but is still > >>> # abstract in the sense that children must implement Drive. > >>> class AbstractCarHelper(AbstractCar,AbstractBaseClass): > ... def DriveHelper(self): > ... print 'Vroom!' > > The example above has no abstract methods, but if you want to make this > kind of partial abstraction a legal definition you need to tell the > interpreter that this is still an abstract class by inheriting directly from > AbstractBaseClass. If we want def time checking, I don't see a way to make > the above definition legal without somehow telling the interpreter to allow > it to be abstract. I don't think this is unpythonic since inheriting from > AbstractBaseClass is non-redundant. In fact, one could argue that it is good > since it lets the programmer that AbstractCarHelper is still an abstract > class even though it does not directly define any abstract methods. > > If you ignore the last example, I don't think having def time checks for > ABC enforcement has any disadvantages. You can easily turn off the def time > checking or have it off by default and turn it on if desired. Since def time > checks are done by __call__ and instantiation checks are done by __new__, > this shouldn't interfere with anything in the original ABC PEP. So in my > mind, the only reason to object to def time checks is if it bothers you that > the last example must inherit directly from AbstractBaseClass. > > > > > Using the machinery currently in the PEP, > > any class that inherits an abstract method and doesn't override it is > > automatically an abstract class itself. There's nothing to complain > > about at definition time. > > > Yes, I see that. What I'm proposing is to have __call__ in the metaclass > complain if a class does not inherit directly from AbstractBaseClass and > does not have any abstract methods. > > From your point of view, if you have def time checks turned off by default > then the difference in using my proposal vs. the original ABC proposal all > comes from the last example. For example, your Iterator class in abc.pyinherits from Iterable (which is abstract) and declares its own abstract > method. Under my current implementation this would raise an error unless > Iterator also inherits from AbstractBaseClass. This is easily fixable by > having my implementation allow classes which declare their own abstract > methods to be abstract. The incompatibly I pointed out in my last example is > illustrated by the Set class in your abc.py. I believe that you want Set > to be an ABC which requires children to support the ABC requirements of its > parents while providing useful implementations of __le__, __lt__, and > __eq__. In my implementation you would need to have Set either inherit > directly from AbstractBaseClass or define at least one abstract method to > prevent an error at def time (if def time checks are on at all). Note that I > don't think it would be unpythonic to have Set inherit from > AbstractBaseClass since it provides someone reading the source code with the > useful information that this is an ABC and not a concrete implementation. > > In summary, let me try to clarify what I see as the only incompatibility > between the current PEP and an ABC implementation: with def time checks > turned on you would need partial abstractions like Set to somehow indicate > that they are abstract by either inheriting directly from AbstractBaseClass > or defining at least one abstract method. If people don't like that, then I > don't have any response. But if people are worried about B&D and other > things, I think those are non-issues. B&D at runtime is no better than B&D > at def time (I think B&D at runtime is worse). > > > > > > > Your example implementation was a bit too much code for me to read. > > :-) > > > No problem. Unfortunately, I think I might have made the example usage in > this post a little too long to read (but hopefully not). :) > > -Emin > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070425/01dfd8a3/attachment-0001.html From guido at python.org Thu Apr 26 05:49:15 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2007 20:49:15 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <462FFB41.8060507@canterbury.ac.nz> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> <79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com> <462FFB41.8060507@canterbury.ac.nz> Message-ID: On 4/25/07, Greg Ewing wrote: > From the PEP, it seems there are two purposes for > these proposed ABCs: > > 1) To document what is meant when we say that something > is a "sequence", "mapping", etc. > > This could be done simply by writing documentation. There > is no need for anything embodied in the language. > > 2) To mark a class for the purposes of introspection. > > I thought we had generally agreed that this is an > anti-pattern in Python. Then how do you explain the popularity of zope.interfaces in both the Zope and the Twisted world? While it often is an anti-pattern, it isn't always, and I don't see why we should forever have to tell people who need this functionality "go download this obfuscated piece of 3rd party code." > If I want my object to > interoperate with someone else's code that tests for > the presence of some ABC, then I *am* forced to use > that ABC, even if it doesn't entirely suit my > purposes. No different than if you want to interoperate with someone who uses zope.interfaces, except you won't have to use 3rd party code, so the cost to you of conforming is many times lower. > I'm -1 on including anything in the language, stdlib > or docs that appears to officially sanction type > testing as a normal style of programming. Read PEP 3119's Rationale. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Apr 26 06:07:58 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2007 21:07:58 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <32e43bb70704251921n4fb470c2uf9154180a75c593c@mail.gmail.com> References: <20070425121618.6417.JCARLSON@uci.edu> <32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com> <20070425140134.641D.JCARLSON@uci.edu> <32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com> <32e43bb70704251921n4fb470c2uf9154180a75c593c@mail.gmail.com> Message-ID: On 4/25/07, Emin.shopper Martinian.shopper wrote: > Sure. Below is a slightly modified example from the doctest: > > >>> class AbstractCar(AbstractBaseClass): # inherit from AbstractBaseClass > to make something abstract > ... @Abstract > ... def Drive(self,x): pass > > To make a class abstract, you inherit from AbstractBaseClass in order to get > the metaclass set properly and then use a function decorator to declare > methods abstract. This is similar to the proposed implementation in your > abc.py and equally pythonic or unpythonic. The following shows what happens > when you define a class that doesn't override the abstract methods: > > >>> try: # illustrate what happens when you don't implement @Abstract > methods > ... class fails(AbstractCar): # an erroneous implementation of > AbstractCar > ... pass > ... except AssertionError, e: # doctest: +ELLIPSIS > ... print e > Class must override Drive to implement: > []. > > The exception gets raised at definition time because the __call__ method of > the metaclass inherited from AbstractBaseClass checks whether abstract > methods have been implemented. In your implementation, you use the __new__ > method of the metaclass to do the check at instantiation. If desired you > could enable/disable checking at definition and instantiation separately. Well I like the PEP's approach better (and AFAIK it matches what C++ does). In the PEP, it is not an error to inherit from an abstract class -- the resulting class itself is abstract. In your version one has to repeat "AbstractBaseClass" each time a class contains (or inherits) abstract methods. I find that an unpythonic requirement, bordering on the repetition of the same information that is so common to statically typed languages. (Note that the PEP now proposes @abstractmethod as a builtin and the Abstract[Class] functionality merged into object and type; I already have an implementation, referenced in the PEP's References section.) [...] > In summary, let me try to clarify what I see as the only incompatibility > between the current PEP and an ABC implementation: with def time checks > turned on you would need partial abstractions like Set to somehow indicate > that they are abstract by either inheriting directly from AbstractBaseClass > or defining at least one abstract method. If people don't like that, then I > don't have any response. But if people are worried about B&D and other > things, I think those are non-issues. B&D at runtime is no better than B&D > at def time (I think B&D at runtime is worse). You summarize the differences clearly. Let's agree to disagree. I think that having to have an abstraction marker on the class *and* on the abstract methods is asking the user to repeat (nearly) the same information twice, and I really don't think that a (partially) abstract class needs to be re-marked as abstract. I find it hard to believe that the definition-time error saves you much development time at all compared to a instantiation-time error. Others have already responded to your attitude towards unit tests -- if your unit tests take 10 minutes to run, you're doing something else wrong. May I suggest you read this: http://googletesting.blogspot.com/2007/01/introducing-testing-on-toilet.html PS. My implementation costs one flag bit check and jump on each object allocation, which surely vanishes compared to the rest of the allocation cost (and if not, it could be shrunk even further by substituting a different function for tp_alloc). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Apr 26 06:10:54 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Apr 2007 21:10:54 -0700 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <20070426020605.19381.476607338.divmod.quotient.5402@ohm> References: <5.1.1.6.0.20070425203654.04ee17d8@sparrow.telecommunity.com> <20070426020605.19381.476607338.divmod.quotient.5402@ohm> Message-ID: On 4/25/07, Jean-Paul Calderone wrote: > On Wed, 25 Apr 2007 20:40:14 -0400, "Phillip J. Eby" wrote: > >At 06:40 PM 4/25/2007 -0400, Jean-Paul Calderone wrote: > >>On Wed, 25 Apr 2007 18:10:23 -0400, Jim Jewett > >>wrote: > >> >The current ABC proposal is to use isinstance as the test; Jeffrey > >> >Yaskin's numbers PEP highlighted the weakness there with a concrete > >> >example. > >> > > >> >If you need to an abstraction less powerful than an existing ABC, > >> >you're out of luck; you can't just assert that the existing class is > >> >already sufficient, nor can you expect everyone else to use multiple > >> >annotations. > >> > >>I'm sure everyone is already aware of the behavior of the classImplements > >>and directlyProvides functions available in zope.interface, which exactly > >>satisfy this use-case in the interface world. > > > >I'm either misunderstanding Jim or you, because I don't see the relationship > >here. If I understand Jim correctly, he's actually asking for something > >like PyProtocols' "protocolIsSubsetOf" declaration -- something that would > >be like dynamically adding a superclass to an existing interface in > >zope.interface. Whereas the features you're talking about sound like > >declaring that a class object itself implements an interface -- something > >apparently unrelated to the question at hand. > > > > Ugh, you're at least half right. I didn't read the implementation of > ABC.meets carefully enough to notice that it's actually an adapter > registry, not just a predicate (meets seems like a funny name for that > method). > > classImplements would have satisfied the adapterless use-case by letting > Jim declare that someone else's class definition actually implements the > feature set that some unit of code he wants to use that class with is > expecting. > > With the adapter registry, I don't see why this isn't just PEP 246 adapt() > (if I am reading one of your later mails correctly though, you think it is > too). > > I'm not particularly familiar with protocolIsSubsetOf, but it wouldn't > surprise me at all if it could also satisfy this use-case. There are > lots of ways to do this, many of them fairly simple. Using inheritance > is the part that makes it hard, since inheritance is controlled by the > wrong party in most cases and comes with unrelated features that are, > at best, irrelevant to the particular use case and at worst actively > detrimental. > > I'm sure a way around this can be invented, I just don't see why it > should be. JP, would you mind dropping this and instead considering how we could turn isinstance and issubclass into (poor-man's) GFs by allowing classes to define class methods __isinstance__ and __issubclass__? (Though the arguments are reversed so to avoid confusion perhaps the names should be different, similar as with the 'in' operator and the __contains__ magic method.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Thu Apr 26 07:16:22 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 26 Apr 2007 01:16:22 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <462FFB41.8060507@canterbury.ac.nz> <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> <79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com> <462FFB41.8060507@canterbury.ac.nz> Message-ID: <5.1.1.6.0.20070426010727.02d2b898@sparrow.telecommunity.com> At 08:49 PM 4/25/2007 -0700, Guido van Rossum wrote: >Then how do you explain the popularity of zope.interfaces in both the >Zope and the Twisted world? In a word: adaptation. (Caveat: it's been a while since I looked at very much of Twisted, and even longer for Zope; but ISTR that Twisted hardly ever checked to see what things implemented, compared to how often it just adapted. Zope 3, meanwhile, has major infrastructure built on adaptation, even though it does do some introspection and typechecking.) From jyasskin at gmail.com Thu Apr 26 08:13:55 2007 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Wed, 25 Apr 2007 23:13:55 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> Message-ID: <5d44f72f0704252313n15c86257n3a480b2d534e04a@mail.gmail.com> On 4/25/07, Guido van Rossum wrote: > Jeffrey, is there any way you can drop the top of the tree and going > straight from Number to Complex -> Real -> Rational -> Integer? These > are the things that everyone with high school math will know. Backing up a bit, what kinds of types do you want this hierarchy to support out of the box? Here are all of the concrete numeric types I can think of. (Adding to this list is where the numpy people would be really helpful.) complex(Real) (that is, pairs of any type of real number, treated as complex. So complex(float) is another spelling for the built-in complex, and complex(int) is the gaussian integers.) float, Rational, and Decimal, int, IntsMod(n) (for any n. prime n's are special. n==2^8 is a C unsigned char. Also written as "Z/nZ") Interval(Real) ([3,5] * [1,3] == [1,15]) Range (I'm not sure how this should work, but the idea is that range(2,5) <: range(0,10).) I'm inclined to exclude Interval because it can't be converted into an int, and Range because operators will return results with types different from their arguments, so it breaks my static typing and mathematical prejudices. Given the rest of the types, I think the following hierarchy is minimal: class Complex(Abstract) # complex({int, rational}) + AlgebraicComplex + Real class AlgebraicComplex(Complex) # complex({float,Decimal}) class Real(Complex) # int + FractionalReal class FractionalReal(Real) # float, rational, Decimal We don't want an Integer abstract class because there is only one type in it, ``int``, so we'd have no guidance on picking the right methods. Why not include IntsMod(n), you ask? Their arithmetic, particularly division, is so different from true integers that I think they'd be likely to violate the Liskov substitution principle. (IntsMod(11)(7) / IntsMod(11)(3) == IntsMod(11)(6), whee!) I'm not completely satisfied that there's no way to distinguish the gaussian integers from Reals with a single check, but since none of you are probably using them, the simpler hierarchy is probably worth it. Maybe we should even combine Real with FractionalReal. Are there any functions that work on floats but not ints or do different things depending on the type? The numpy people do need support for the mathematical ABCs, but if they can be implemented in a separate module that most people don't need to know about, I think they should be (I'll move the classes I currently have into algebra.py to serve as a jumping-off point). And I think that for ABCs to be viable, it must be possible to add "superclasses" to an existing abstract class from a separate module. Hopefully Jim's thread will be fruitful. I haven't updated the PEP to correspond to this yet, in case it's even worse than what I have. :) -- Namast?, Jeffrey Yasskin From python at rcn.com Thu Apr 26 08:29:41 2007 From: python at rcn.com (Raymond Hettinger) Date: Wed, 25 Apr 2007 23:29:41 -0700 Subject: [Python-3000] Two proposals for a new list-like type: one modest, one radical References: <00c601c786ff$95839700$f101a8c0@RaymondLaptop1> Message-ID: <014501c787cc$463d2940$f101a8c0@RaymondLaptop1> [Raymond Hettinger on treelist being added to the collections module] >> In practice, the code for BList is somewhat complex, and its desirability and performance in actual applications is unproven. >> Fortunately, Py2.6 is still a long way off. My recommendation is that you release it right away as a third-party module and let >> the >> hordes of Pythonistas test it in battle. With the code base thoroughly exercised and some positive user feedback, it would be >> hard >> to say no to this going into Py2.6. [Daniel Stutzbach] > Will do. I look forward to the announcement. Raymond From oliphant.travis at ieee.org Thu Apr 26 12:28:14 2007 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Thu, 26 Apr 2007 04:28:14 -0600 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> Message-ID: Forgive my ignorance, but I'm not really sure what this PEP is trying to do. I don't want to sound negative, I really just don't understand the purpose. I've just never encountered a problem this that I can see how this PEP will help me solve. That doesn't mean it's not worthwhile, I just don't understand it. > > Abstract > ======== > > This proposal defines a hierarchy of Abstract Base Classes (ABCs) > [#pep3119] to represent numbers and other algebraic entities similar > to numbers. It proposes: > > * A hierarchy of algebraic concepts, including monoids, groups, rings, > and fields with successively more operators and constraints on their > operators. This will be added as a new library module named > "algebra". > The SAGE people may be interested in this, but I doubt there will more than a handful of users of these algebraic base classes. > * A hierarchy of specifically numeric types, which can be converted to > and from the native Python types. This will be added as a new > library module named "numbers". > I could see having a hierarchy of "numbers" we have one in NumPy. All the NumPy array scalars fall under a hierarchy of types so that it is easy to check whether or not you have signed or unsigned integers, or inexact numbers. > > Object oriented systems have a general problem in constraining > functions that take two arguments. To take addition as an example, > ``int(3) + int(4)`` is defined, and ``vector(1,2,3) + vector(3,4,5)`` > is defined, but ``int(3) + vector(3,4,5)`` doesn't make much sense. In NumPy, this kind of operation makes sense we just broadcast int(3) to the equivalent vector(3,3,3) and do the element-by-element operation. This is how most array packages outside of Python do it as well. I think adding the abstract base-classes for "algebras" is more complicated than has been discussed if you are going to start treating arrays as elements in their own algebra. There is a lot of flexibility in how arrays can be viewed in an algebraic context. The author gives a valiant go at defining base classes for generic algebras, but I didn't see everything I've encountered in my travels through algebra, and I don't understand the motivation for trying to include these things in Python specifically. Therefore, I would stick with defining a hierarchy of numbers and leave it at that for now (if I even went that far). For numbers, several suggestions have been offered. In NumPy we use abstract base classes that look like this Number Integer SignedInteger UnsignedInteger Inexact Floating ComplexFloating But, this is because the array scalars in NumPy are based on C-data-types and not on general-purpose integers or floats. In NumPy it is useful to check if you have an integer-typed array and so the base-classes become useful. For general purpose Python, I would do something like Complex Rational_Complex Integer_Complex Floating_Complex # uses hardware float Decimal_Complex Real Rational Integer Floating # uses hardware float Decimal I don't like the "copied" structure which occurs because complex numbers can are isomorphic to a pair of reals but don't know else how to specify a rational-complex number. I also don't know if the decimal base-class includes un-limited precision floats (the analog of Python long integers) or if they must be fixed-precision. -Travis From emin.shopper at gmail.com Thu Apr 26 14:20:04 2007 From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper) Date: Thu, 26 Apr 2007 08:20:04 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <20070425121618.6417.JCARLSON@uci.edu> <32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com> <20070425140134.641D.JCARLSON@uci.edu> <32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com> <32e43bb70704251921n4fb470c2uf9154180a75c593c@mail.gmail.com> Message-ID: <32e43bb70704260520v1095d2f6o3be0e0924beac4c4@mail.gmail.com> On 4/26/07, Guido van Rossum wrote: > > > You summarize the differences clearly. Let's agree to disagree. I > think that having to have an abstraction marker on the class *and* on > the abstract methods is asking the user to repeat (nearly) the same > information twice, and I really don't think that a (partially) > abstract class needs to be re-marked as abstract. OK. I still believe that this slight flaw is worth the benefit of def time checks, but accept your judgement that the use cases I have in mind are not common enough to justify it. Consider my proposal withdrawn. I find it hard to believe that the definition-time error saves you > much development time at all compared to a instantiation-time error. > Others have already responded to your attitude towards unit tests -- > if your unit tests take 10 minutes to run, you're doing something else > wrong. May I suggest you read this: > > http://googletesting.blogspot.com/2007/01/introducing-testing-on-toilet.html Sigh. I think unit tests are great. I write unit tests for all of my code. I have a script that runs all the unit tests on our development branch overnight and emails me about the things that fails. I'm not motivated by a dislike of unit-tests. If anything, I am too much a believer in testing which is why I advocated the option to have more testing facilities (i.e., def time tests). I think having more options available for testing is a good thing and I don't understand the vehement objections people have to anything that vaguely resembles static checks. Thanks, -Emin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070426/aaec9bbf/attachment.htm From bjourne at gmail.com Thu Apr 26 15:33:08 2007 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Thu, 26 Apr 2007 06:33:08 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> Message-ID: <740c3aec0704260633u64ab70ffy2e68cbe5429d0566@mail.gmail.com> On 4/22/07, Phillip J. Eby wrote: > I got off to a good start in terms of figuring out what needs to be in the > PEP... good enough to realize that a complete specification of what I have > in mind is probably going to dwarf the WSGI spec. There are just too many > details that have to be explained; the roughly 1000 lines of Python code I > have in mind is translating to quite a *lot* more words of English, to > explain intentions, ramifications, etc. "If the implementation is hard to explain, it's a bad idea." -- mvh Bj?rn From goodger at python.org Thu Apr 26 16:39:47 2007 From: goodger at python.org (David Goodger) Date: Thu, 26 Apr 2007 10:39:47 -0400 Subject: [Python-3000] call for PEPs Message-ID: <4335d2c40704260739k1b9fc46fs1701ecabd9eb7d4e@mail.gmail.com> I'm one of the PEP editors at the end of peps-at-python-dot-org; Barry Warsaw is the other. Since before PyCon, I haven't had time for PEP number assignments, editing, and checkins. I don't follow python-dev or python-3000 closely, so I only recently learned of the April 30 deadline for PEPs. I'd like to get all the outstanding PEPs checked in, but I need help. There are a bunch of draft PEPs in my inbox, but I can't trust it because some have already been checked in, and others have since been updated or become obsolete. Please re-send the latest drafts (as attachments!) of any outstanding PEPs to peps-at-python-dot-org. Include the dates of any postings of the PEP to python-dev, python-3000, or python-list (in the Post-History header). Be sure to read and follow PEP 1 (PEP Purpose and Guidelines; http://www.python.org/dev/peps/pep-0001/) for details on what to include. Posting a draft PEP to a python-* mailing list IS NOT SUFFICIENT. It must be sent to peps-at-python-dot-org. Committers (those with commit access to Python's SVN) are welcome to add PEPs themselves. Just take the next available number from PEP 0 (which must be updated). BTW, with PSF and PyCon duties (I'm chairing PyCon 2008), I don't see my availability for PEP editing increasing any time soon. A new PEP Editor would be welcome! -- David Goodger From pje at telecommunity.com Thu Apr 26 17:30:23 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 26 Apr 2007 11:30:23 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <740c3aec0704260633u64ab70ffy2e68cbe5429d0566@mail.gmail.co m> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070426112300.0515f820@sparrow.telecommunity.com> At 06:33 AM 4/26/2007 -0700, BJ?rn Lindqvist wrote: >On 4/22/07, Phillip J. Eby wrote: > > I got off to a good start in terms of figuring out what needs to be in the > > PEP... good enough to realize that a complete specification of what I have > > in mind is probably going to dwarf the WSGI spec. There are just too many > > details that have to be explained; the roughly 1000 lines of Python code I > > have in mind is translating to quite a *lot* more words of English, to > > explain intentions, ramifications, etc. > >"If the implementation is hard to explain, it's a bad idea." The implementation is *easy* to explain. Here's an equally easy explanation of the new-style type hierarchy: 'type' is a subclass of 'object'. 'object' is an instance of 'type'. 'type' is an instance of 'type'. The method resolution order (mro) of a type is a sequence such that the types bases appear in the same relative order in the mro as they do in the bases list, and such that the types in the mros of the bases maintain the same relative order in the subclass' mro as they do in that base's mro. If no such sequence can be constructed, the type is invalid. Short, sweet, and beautiful in its simplicity. It's the *rationale* and *consequences* that take a lot of explaining, not the implementation. In the case of the explanation above for new-style types, the rationale and consequences fill most of a book: the "Putting Metaclasses to Work" book that Guido used as inspiration. From talin at acm.org Thu Apr 26 18:00:45 2007 From: talin at acm.org (Talin) Date: Thu, 26 Apr 2007 09:00:45 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> Message-ID: <4630CCAD.1040400@acm.org> Phillip J. Eby wrote: > In particular, when you implement generic functions using generic > functions, it reads a lot like trying to explain the relationships between > "type" and "object" (i.e. type being its own type and an instance of > object, and object being a subclass of type). You can say what needs to be > said in a fairly short space, but saying it in a way that lets people > actually wrap their heads around it takes a bit more text. :) I meant to comment on this earlier, and forgot. I kind of wonder about the idea of implementing generic functions using generic functions. It's one of those ideas that is so obviously elegant and clever, that it's very easy for a programmer to fall in love with. As someone who frequently finds himself in a similar situation, I often discover later that the clever idea I loved wasn't the most practical. I've occasionally been accused of being an "architecture astronaut", someone whose designs have a theoretical purity that is far above the heads of most of its users, to the point where it is hard for them to wrap their head around the design: http://www.joelonsoftware.com/articles/fog0000000018.html Related to this is my tendency towards "dual-duty design", in which every line of code is serving multiple agendas (like the way that all of the major components on an airplane serve more than one functions - the wings are not only the primary lift surfaces, they are also the fuel tanks and the mounting points for landing gear, engines, running lights, and so on.) The advantage of this is that it makes the code very compact and dense, with lots of ideas being embodied in a relatively few lines of code. The problem with this is that someone who comes along and reads my code may not be able to make any sense of it, especially if they are only aware of the primary agenda that is informing the code. Now, I may be talking out of my butt here - I don't know much about your situation, or what rationales you have for designing things the way that you have. However, I would suggest that you might want to consider what your actual, practical needs are, and if there is a way to get them that is more straightforward and easy to explain. -- Talin From jdc at uwo.ca Thu Apr 26 18:07:20 2007 From: jdc at uwo.ca (Dan Christensen) Date: Thu, 26 Apr 2007 12:07:20 -0400 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> Message-ID: <873b2n5dc7.fsf@uwo.ca> [sage list added] "Travis E. Oliphant" writes: > The SAGE people may be interested in this, but I doubt there will more > than a handful of users of these algebraic base classes. SAGE has quite a sophisticated type hierarchy, and a sophisticated set of coercion methods. What is done in SAGE should definitely be consulted, since it is probably the most complex set of mathematical types yet written in python. The SAGE tutorial is at http://www.sagemath.org/doc/html/tut/tut.html and Section 2.2 gives a brief introduction to numbers: http://www.sagemath.org/doc/html/tut/node9.html The SAGE reference manual is at http://www.sagemath.org/doc/html/ref/index.html Chapter 20: http://www.sagemath.org/doc/html/ref/node198.html and nearby chapters are quite relevant for this discussion. > For general purpose Python, I would do something like > > Complex > Rational_Complex > Integer_Complex > Floating_Complex # uses hardware float > Decimal_Complex > Real > Rational > Integer > Floating # uses hardware float > Decimal Note also that double-precision reals are a subset of the rationals, since each double precision real is exactly representable as a rational number, but many rational numbers are not exactly representable as double precision reals. Not sure if this means that reals should be a subclass of the rationals. I believe that in SAGE these relationships aren't expressed using subclassing. Dan From jimjjewett at gmail.com Thu Apr 26 18:59:30 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 26 Apr 2007 12:59:30 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <20070425121618.6417.JCARLSON@uci.edu> <32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com> <20070425140134.641D.JCARLSON@uci.edu> <32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com> <32e43bb70704251921n4fb470c2uf9154180a75c593c@mail.gmail.com> Message-ID: On 4/26/07, Guido van Rossum wrote: > In [Emin's] version one has to repeat "AbstractBaseClass" each > time a class contains (or inherits) abstract methods. His modified proposal is better, though perhaps still too complicated. You must declare a class abstract *either* by explicitly marking it, *or* by including an abstract method. (Maybe it should be any attribute, not just methods.) This would make it more obvious that, say, an IterableMapping still has some abstract methods that you need to override before you're done. On the other hand, what I would really want to see is the list of missing methods (next and getitem; possibly iter); maybe this sort of de-abstraction should be left to an IDE for now. -jJ From janssen at parc.com Thu Apr 26 19:11:27 2007 From: janssen at parc.com (Bill Janssen) Date: Thu, 26 Apr 2007 10:11:27 PDT Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <20070425085350.6408.JCARLSON@uci.edu> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <20070425085350.6408.JCARLSON@uci.edu> Message-ID: <07Apr26.101132pdt."57996"@synergy1.parc.xerox.com> > Are the base number operations in Python all that difficult to > understand? Well, they're a little tricky. But the basic problem with "number" support, in almost every programming language, is that they are too low-level an abstraction. A number in a program is never *just* an integer or a float. It's always an integer or float (or complex, OK) deployed in support of a very specific purpose. It's a loop index, or a weight in kilos, or the number of calendar entries for a particular day. It often has units, and a fixed range of values or fixed set of values, neither of which are taken into account in most programming languages. Strings really suffer from the same lack of focus. A string is never just a string. It's a book title, or a filename, or a person's surname, or an email address. It's usually got a particular human language (or computer OS) associated with it. Each of these usages has particular limitations and patterns which limit both its range of values, and the operations which can be successfully applied to it. Bill From exarkun at divmod.com Thu Apr 26 19:26:02 2007 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Thu, 26 Apr 2007 13:26:02 -0400 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <07Apr26.101132pdt."57996"@synergy1.parc.xerox.com> Message-ID: <20070426172602.19381.679137565.divmod.quotient.5553@ohm> On Thu, 26 Apr 2007 10:11:27 PDT, Bill Janssen wrote: >> Are the base number operations in Python all that difficult to >> understand? > >Well, they're a little tricky. > >But the basic problem with "number" support, in almost every >programming language, is that they are too low-level an abstraction. >A number in a program is never *just* an integer or a float. It's >always an integer or float (or complex, OK) deployed in support of a >very specific purpose. It's a loop index, or a weight in kilos, or >the number of calendar entries for a particular day. It often has >units, and a fixed range of values or fixed set of values, neither of >which are taken into account in most programming languages. > >Strings really suffer from the same lack of focus. A string is never >just a string. It's a book title, or a filename, or a person's >surname, or an email address. It's usually got a particular human >language (or computer OS) associated with it. Each of these usages >has particular limitations and patterns which limit both its range of >values, and the operations which can be successfully applied to it. I definitely agree with this, and resolving it would probably address a large class of mistakes made when doing any programming involving numbers and strings (anyone done much of that lately? :). Does the introduction of a lot of base classes for `int' and `float' do anything to address this, though? What you really want is a bunch of new subclasses of them, the opposite. Jean-Paul From janssen at parc.com Thu Apr 26 19:26:18 2007 From: janssen at parc.com (Bill Janssen) Date: Thu, 26 Apr 2007 10:26:18 PDT Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> Message-ID: <07Apr26.102627pdt."57996"@synergy1.parc.xerox.com> > Jeffrey, is there any way you can drop the top of the tree and going > straight from Number to Complex -> Real -> Rational -> Integer? These > are the things that everyone with high school math will know. I think knowledge of the concepts of group, ring, and field is supposed to be standard knowledge for any high-school senior -- isn't this what the "new math" was all about?. But they're pretty fundamental to computer science; anyone trying to do serious work in the field (that is, anyone with a reason to read the PEP) should have a nodding acquaintance with them. Bill From janssen at parc.com Thu Apr 26 19:31:14 2007 From: janssen at parc.com (Bill Janssen) Date: Thu, 26 Apr 2007 10:31:14 PDT Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> <79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com> Message-ID: <07Apr26.103122pdt."57996"@synergy1.parc.xerox.com> > But it makes sense in some cases to insist that someone who goes > through the trouble of inheriting from an ABC gets an error if they > forget to implement one of the methods that are deemed 'essential' to > that ABC. This is the problem with abstract methods. What should happen is that the default method (the one defined in the ABC class) gets called. Otherwise you're back in the morass of Java types, without effective mixin capability. In Python, we need to be able to define an abstract class with real implementation methods -- a class that both defines an interface, and provides an implementation. That means that anyone providing a subclass need only implement the methods that they override. Now, we can still have a way of marking the methods of an ABC as being "not implemented", which would in fact force the subclass designer to implement them. But this concept should be carefully separated from the concept of "this method is part of the interface defined by this class". Bill From guido at python.org Thu Apr 26 20:32:33 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Apr 2007 11:32:33 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <1251171308604668819@unknownmsgid> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> <79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com> <1251171308604668819@unknownmsgid> Message-ID: On 4/26/07, Bill Janssen wrote: [Guido] > > But it makes sense in some cases to insist that someone who goes > > through the trouble of inheriting from an ABC gets an error if they > > forget to implement one of the methods that are deemed 'essential' to > > that ABC. > > This is the problem with abstract methods. What should happen is that > the default method (the one defined in the ABC class) gets called. > Otherwise you're back in the morass of Java types, without effective > mixin capability. In Python, we need to be able to define an abstract > class with real implementation methods -- a class that both defines an > interface, and provides an implementation. That means that anyone > providing a subclass need only implement the methods that they > override. > > Now, we can still have a way of marking the methods of an ABC as being > "not implemented", which would in fact force the subclass designer to > implement them. But this concept should be carefully separated from > the concept of "this method is part of the interface defined by this > class". Yeah. I like @abstractmethod to mean "not implemented" (except perhaps as an end point for cooperative super calls). For the latter concept, I propose that all methods present in an apparent ABC are considered part of the interface, unless their name starts with a single underscore (the latter should be considered implementation details, or perhaps definitional details). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jjb5 at cornell.edu Thu Apr 26 20:55:14 2007 From: jjb5 at cornell.edu (Joel Bender) Date: Thu, 26 Apr 2007 14:55:14 -0400 Subject: [Python-3000] Fixing super anyone? (reflux) In-Reply-To: References: <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com> <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> Message-ID: <4630F592.9090806@cornell.edu> I've come late to this thread, and misunderstood what was wrong with super(), so a thousand pardons please. But since that's never stopped me before... Guido van Rossum wrote: > But: > > class E(D): pass > > print E().f() > > This prints DDBCA which surely isn't right. > > Sounds like the classic bug in such attempts. About about this? class _super(property): def __init__(self): property.__init__(self, self.get_super, None, None) def get_super(self, klass): class wrapper: def __getattr__(self, fn): self.fn = fn return self def __call__(self, obj, *args, **kwargs): mro = iter(obj.__class__.__mro__) for cls in mro: if cls is klass: break for cls in mro: f = getattr(cls, self.fn) if f: return f(obj, *args, **kwargs) raise AttributeError, self.fn return wrapper() class _superable(type): __super__ = _super() class A(object): __metaclass__ = _superable def f(self): return "A" class B(A): def f(self): return "B" + B.__super__.f(self) class C(A): def f(self): return "C" + C.__super__.f(self) class D(B, C): def f(self): return "D" + D.__super__.f(self) class E(D): pass assert E().f() == "DBCA" Tim Delaney wrote: > What I haven't worked out yet is if you should be able to do > the following: > > class A(autosuper): > def f(self): > print 'A:', super > > class B(A): > def f(self): > def inner(): > print 'B:', super > super.f() > inner() And here's my version: class A(object): __metaclass__ = _superable def f(self): return "A" class B(A): def f(self): def inner(): return "B" + B.__super__.f(self) inner() assert B().f() == "BA" Now, back to the original request, I came up with this: def super(klass, obj=None): class wrapper: def __init__(self): self.klass = klass self.obj = obj def __getattr__(self, fn): self.fn = fn return self def __call__(self, *args, **kwargs): if not self.obj: self.obj = args[0] args = args[1:] mro = iter(self.obj.__class__.__mro__) for cls in mro: if cls is self.klass: break for cls in mro: f = getattr(cls, self.fn) if f: return f(self.obj, *args, **kwargs) raise AttributeError, self.fn return wrapper() class A(object): def f(self): return "A" class B(A): def f(self): return "B" + super(B).f(self) class C(A): def f(self): return "C" + super(C, self).f() class D(B, C): def f(self): return "D" + super(D, self).f() class E(D): pass assert E().f() == "DBCA" I prefer the version in C, but B works as well. Comments? Is this ground that has already been covered? Joel From guido at python.org Thu Apr 26 21:06:45 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Apr 2007 12:06:45 -0700 Subject: [Python-3000] Fixing super anyone? (reflux) In-Reply-To: <4630F592.9090806@cornell.edu> References: <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <4630F592.9090806@cornell.edu> Message-ID: Writing B.__super__.f(self) instead of super(B, self).f() is not an improvement. On 4/26/07, Joel Bender wrote: > I've come late to this thread, and misunderstood what was wrong with > super(), so a thousand pardons please. But since that's never stopped > me before... > > Guido van Rossum wrote: > > > But: > > > > class E(D): pass > > > > print E().f() > > > > This prints DDBCA which surely isn't right. > > > > Sounds like the classic bug in such attempts. > > About about this? > > class _super(property): > def __init__(self): > property.__init__(self, self.get_super, None, None) > def get_super(self, klass): > class wrapper: > def __getattr__(self, fn): > self.fn = fn > return self > def __call__(self, obj, *args, **kwargs): > mro = iter(obj.__class__.__mro__) > for cls in mro: > if cls is klass: > break > for cls in mro: > f = getattr(cls, self.fn) > if f: > return f(obj, *args, **kwargs) > raise AttributeError, self.fn > return wrapper() > > class _superable(type): > __super__ = _super() > > class A(object): > __metaclass__ = _superable > def f(self): > return "A" > > class B(A): > def f(self): > return "B" + B.__super__.f(self) > > class C(A): > def f(self): > return "C" + C.__super__.f(self) > > class D(B, C): > def f(self): > return "D" + D.__super__.f(self) > > class E(D): > pass > > assert E().f() == "DBCA" > > > Tim Delaney wrote: > > > What I haven't worked out yet is if you should be able to do > > the following: > > > > class A(autosuper): > > def f(self): > > print 'A:', super > > > > class B(A): > > def f(self): > > def inner(): > > print 'B:', super > > super.f() > > inner() > > And here's my version: > > class A(object): > __metaclass__ = _superable > def f(self): > return "A" > > class B(A): > def f(self): > def inner(): > return "B" + B.__super__.f(self) > inner() > > assert B().f() == "BA" > > > Now, back to the original request, I came up with this: > > > def super(klass, obj=None): > class wrapper: > def __init__(self): > self.klass = klass > self.obj = obj > def __getattr__(self, fn): > self.fn = fn > return self > def __call__(self, *args, **kwargs): > if not self.obj: > self.obj = args[0] > args = args[1:] > mro = iter(self.obj.__class__.__mro__) > for cls in mro: > if cls is self.klass: > break > for cls in mro: > f = getattr(cls, self.fn) > if f: > return f(self.obj, *args, **kwargs) > raise AttributeError, self.fn > return wrapper() > > class A(object): > def f(self): > return "A" > > class B(A): > def f(self): > return "B" + super(B).f(self) > > class C(A): > def f(self): > return "C" + super(C, self).f() > > class D(B, C): > def f(self): > return "D" + super(D, self).f() > > class E(D): > pass > > assert E().f() == "DBCA" > > > I prefer the version in C, but B works as well. Comments? Is this > ground that has already been covered? > > > Joel > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Thu Apr 26 22:08:25 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 26 Apr 2007 16:08:25 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <4630CCAD.1040400@acm.org> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070426153850.02ce0038@sparrow.telecommunity.com> At 09:00 AM 4/26/2007 -0700, Talin wrote: >I kind of wonder about the idea of implementing generic functions using >generic functions. It's one of those ideas that is so obviously elegant >and clever, that it's very easy for a programmer to fall in love with. >As someone who frequently finds himself in a similar situation, I often >discover later that the clever idea I loved wasn't the most practical. In this case, it's the reverse; I previously implemented generic functions the other way around, in RuleDispatch. PEAK-Rules is considerably simpler; in fact I've been realizing lately that the more "object orientation" I *remove* from it, the easier it gets to work with. For example, it's recently become clear that my "TreeBuilder" class (used to assemble efficient dispatch trees by figuring out what tests come next) is nothing more than a memoized generic function. Having the class exist just adds to the number of things you have to subclass when you change anything, and makes it more complex because you also have to change any code that *creates* a TreeBuilder to create an instance of your subclass instead. Making it a generic function makes a bunch of extra bookkeeping just disappear. >Now, I may be talking out of my butt here - I don't know much about your >situation, or what rationales you have for designing things the way that >you have. Pretty simple, really -- RuleDispatch's heavily object-based internal structure made it brittle and complex to refactor or extend, compared to the outer layers where it used generic functions. The obvious solution was to make the inner layers generic also. The real key, though, was looking at Guido's "overloading" prototype for Py3K and realizing I didn't need to settle for single-dispatch generics for the core, or have to try and bootstrap the engine on itself; I could use a simple implementation like his that did multi-dispatch instead. And so I did. After some experimentation, I found that you can make any generic function self-bootstrapping (i.e., usable in its own implementation) as long as your implementation is re-entrant. That is, if a generic function has to call itself while a method is being added to it, it must simply call the *last stable version* of itself, rather than the version it's modifying. This is actually quite simple to arrange, even in the presence of threads. >However, I would suggest that you might want to consider what >your actual, practical needs are, and if there is a way to get them that >is more straightforward and easy to explain. Well, I could use some other language of course, but I prefer Python. :) From guido at python.org Thu Apr 26 22:25:28 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Apr 2007 13:25:28 -0700 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: Message-ID: On 4/25/07, Jim Jewett wrote: > The current ABC proposal is to use isinstance as the test; Jeffrey > Yaskin's numbers PEP highlighted the weakness there with a concrete > example. > > If you need to an abstraction less powerful than an existing ABC, > you're out of luck; you can't just assert that the existing class is > already sufficient, nor can you expect everyone else to use multiple > annotations. I now have a proposal to allow overloading isinstance() and issubclass(), by defining special (class) methods on the second argument. See http://python.org/sf/1708353. Does this need a PEP? The unit test shows that it can be used to support the use case described above: class ABC(type): def __instancecheck__(cls, inst): """Implement isinstance(inst, cls).""" return any(cls.__subclasscheck__(c) for c in {type(inst), inst.__class__}) def __subclasscheck__(cls, sub): """Implement issubclass(sub, cls).""" candidates = cls.__dict__.get("__subclass__", set()) return any(c in candidates for c in sub.mro()) class Integer(metaclass=ABC): __subclass__ = {int} -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Apr 26 22:36:46 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Apr 2007 13:36:46 -0700 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: Message-ID: On 4/26/07, Guido van Rossum wrote: > I now have a proposal to allow overloading isinstance() and > issubclass(), by defining special (class) methods on the second > argument. See http://python.org/sf/1708353. Does this need a PEP? The > unit test shows that it can be used to support the use case described > above: > > > class ABC(type): > > def __instancecheck__(cls, inst): > """Implement isinstance(inst, cls).""" > return any(cls.__subclasscheck__(c) > for c in {type(inst), inst.__class__}) > > def __subclasscheck__(cls, sub): > """Implement issubclass(sub, cls).""" > candidates = cls.__dict__.get("__subclass__", set()) > return any(c in candidates for c in sub.mro()) > > > class Integer(metaclass=ABC): > > __subclass__ = {int} That example has a bug. Try this instead: class ABC(type): def __instancecheck__(cls, inst): """Implement isinstance(inst, cls).""" return any(cls.__subclasscheck__(c) for c in {type(inst), inst.__class__}) def __subclasscheck__(cls, sub): """Implement issubclass(sub, cls).""" candidates = cls.__dict__.get("__subclass__", set()) | {cls} return any(c in candidates for c in sub.mro()) class Integer(metaclass=ABC): __subclass__ = {int} To complete the example: isinstance(42, Integer) and issubclass(int, Integer) now both return True, while isinstance(3.14, Integer) and issubclass(float, Integer) return False. Of course, isinstance(Integer(), Integer) and issubclass(Integer, Integer) are both True. If you subclass Integer, int won't be considered a subclass of it! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jimjjewett at gmail.com Thu Apr 26 23:13:59 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 26 Apr 2007 17:13:59 -0400 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: Message-ID: On 4/26/07, Guido van Rossum wrote: > I now have a proposal to allow overloading isinstance() and > issubclass(), by defining special (class) methods on the second > argument. See http://python.org/sf/1708353. Does this need a PEP? It needs to be mentioned in the ABC PEP; whether it needs its own PEP is probably up to you. -jJ From skip at pobox.com Thu Apr 26 19:18:44 2007 From: skip at pobox.com (skip at pobox.com) Date: Thu, 26 Apr 2007 12:18:44 -0500 Subject: [Python-3000] [Python-Dev] call for PEPs In-Reply-To: <4335d2c40704260739k1b9fc46fs1701ecabd9eb7d4e@mail.gmail.com> References: <4335d2c40704260739k1b9fc46fs1701ecabd9eb7d4e@mail.gmail.com> Message-ID: <17968.57076.311577.775834@montanaro.dyndns.org> David> ... I only recently learned of the April 30 deadline for David> PEPs. This is the first I heard of any sort of deadline... Skip From talin at acm.org Fri Apr 27 04:27:57 2007 From: talin at acm.org (Talin) Date: Thu, 26 Apr 2007 19:27:57 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> <79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com> <1251171308604668819@unknownmsgid> Message-ID: <46315FAD.8050702@acm.org> Guido van Rossum wrote: > Yeah. I like @abstractmethod to mean "not implemented" (except perhaps > as an end point for cooperative super calls). For the latter concept, > I propose that all methods present in an apparent ABC are considered > part of the interface, unless their name starts with a single > underscore (the latter should be considered implementation details, or > perhaps definitional details). I kind of wish that it was called "incomplete" rather than abstract, for reasons that we've already discussed. -- Talin From guido at python.org Fri Apr 27 06:12:51 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Apr 2007 21:12:51 -0700 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <46315FAD.8050702@acm.org> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> <79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com> <1251171308604668819@unknownmsgid> <46315FAD.8050702@acm.org> Message-ID: On 4/26/07, Talin wrote: > I kind of wish that it was called "incomplete" rather than abstract, for > reasons that we've already discussed. Can't remember the details though I vaguely recall some grumblings about that. But why invent new terminology when we can leverage existing terminology, even if the semantics aren't quite the same? Python has a long history of doing that -- e.g. list, int, assignment, class, method, multiple inheritance, all come from other languages but Python gives them a new twist -- sometimes intentional (assignment, list, int), sometimes because the simplistic, dynamic implementation just can't provide what a typical compiler can do. I find that the current meanings of "abstract method" and "abstract class" are still close enough to C++ or Java that reusing the name is helpful rather than harmful. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Fri Apr 27 09:25:50 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 27 Apr 2007 19:25:50 +1200 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <4630CCAD.1040400@acm.org> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <4630CCAD.1040400@acm.org> Message-ID: <4631A57E.1000301@canterbury.ac.nz> Talin wrote: > I kind of wonder about the idea of implementing generic functions using > generic functions. I'd be worried about efficiency. Generic functions on top of generic functions sounds like you would have two layers of dynamic lookup to go through instead of one. -- Greg From guido at python.org Thu Apr 26 20:50:59 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Apr 2007 11:50:59 -0700 Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes Message-ID: After a fair amount of pre-discussion, I'm ready for the first official review of this PEP. The PEP is online at http://www.python.org/dev/peps/pep-3119/ . Here's a summary of open issues on which I could use more help (more details in the full text of the PEP below): - Where should PartiallyOrdered and TotallyOrdered live? - Should we support comparison of different concrete set types? - Ditto for mapping types? - Ditto for sequence types? - Should Sequence derive from TotallyOrdered? - Should ComposableSet.__or__ and friends be abstract or concrete? - If concrete, how should they create the result? - Do we need a non-composable hashable set type? - Ditto for a non-composable mutable set type? - Should we require that the iteration order for keys, values and items of a mapping are always consistent? - Which standard methods should sequences have? Of course, feel free to discuss any issues not marked as "open" as well. Full text of the PEP: PEP: 3119 Title: Introducing Abstract Base Classes Version: $Revision: 54986 $ Last-Modified: $Date: 2007-04-26 11:24:07 -0700 (Thu, 26 Apr 2007) $ Author: Guido van Rossum , Talin Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 18-Apr-2007 Post-History: 26-Apr-2007 Abstract ======== This is a proposal to add Abstract Base Class (ABC) support to Python 3000. It proposes: * An "ABC support framework" which defines a built-in decorator that can be used to define abstract methods. A class containing an abstract method that isn't overridden cannot be instantiated. * Specific ABCs for containers and iterators, to be added to the collections module. Much of the thinking that went into the proposal is not about the specific mechanism of ABCs, as contrasted with Interfaces or Generic Functions (GFs), but about clarifying philosophical issues like "what makes a set", "what makes a mapping" and "what makes a sequence". Acknowledgements ---------------- Talin wrote the Rationale below [1]_ as well as most of the section on ABCs vs. Interfaces. For that alone he deserves co-authorship. The rest of the PEP uses "I" referring to the first author. Rationale ========= In the domain of object-oriented programming, the usage patterns for interacting with an object can be divided into two basic categories, which are 'invocation' and 'inspection'. Invocation means interacting with an object by invoking its methods. Usually this is combined with polymorphism, so that invoking a given method may run different code depending on the type of an object. Inspection means the ability for external code (outside of the object's methods) to examine the type or properties of that object, and make decisions on how to treat that object based on that information. Both usage patterns serve the same general end, which is to be able to support the processing of diverse and potentially novel objects in a uniform way, but at the same time allowing processing decisions to be customized for each different type of object. In classical OOP theory, invocation is the preferred usage pattern, and inspection is actively discouraged, being considered a relic of an earlier, procedural programming style. However, in practice this view is simply too dogmatic and inflexible, and leads to a kind of design rigidity that is very much at odds with the dynamic nature of a language like Python. In particular, there is often a need to process objects in a way that wasn't anticipated by the creator of the object class. It is not always the best solution to build in to every object methods that satisfy the needs of every possible user of that object. Moreover, there are many powerful dispatch philosophies that are in direct contrast to the classic OOP requirement of behavior being strictly encapsulated within an object, examples being rule or pattern-match driven logic. On the the other hand, one of the criticisms of inspection by classic OOP theorists is the lack of formalisms and the ad hoc nature of what is being inspected. In a language such as Python, in which almost any aspect of an object can be reflected and directly accessed by external code, there are many different ways to test whether an object conforms to a particular protocol or not. For example, if asking 'is this object a mutable sequence container?', one can look for a base class of 'list', or one can look for a method named '__getitem__'. But note that although these tests may seem obvious, neither of them are correct, as one generates false negatives, and the other false positives. The generally agreed-upon remedy is to standardize the tests, and group them into a formal arrangement. This is most easily done by associating with each class a set of standard testable properties, either via the inheritance mechanism or some other means. Each test carries with it a set of promises: it contains a promise about the general behavior of the class, and a promise as to what other class methods will be available. This PEP proposes a particular strategy for organizing these tests known as Abstract Base Classes, or ABC. ABCs are simply Python classes that are added into an object's inheritance tree to signal certain features of that object to an external inspector. Tests are done using isinstance(), and the presence of a particular ABC means that the test has passed. In addition, the ABCs define a minimal set of methods that establish the characteristic behavior of the type. Code that discriminates objects based on their ABC type can trust that those methods will always be present. Each of these methods are accompanied by an generalized abstract semantic definition that is described in the documentation for the ABC. These standard semantic definitions are not enforced, but are strongly recommended. Like all other things in Python, these promises are in the nature of a gentlemen's agreement, which in this case means that while the language does enforce some of the promises made in the ABC, it is up to the implementer of the concrete class to insure that the remaining ones are kept. Specification ============= The specification follows the categories listed in the abstract: * An "ABC support framework" which defines a built-in decorator that make it easy to define ABCs, and mechanisms to support it. * Specific ABCs for containers and iterators, to be added to the collections module. ABC Support Framework --------------------- We define a new built-in decorator, ``@abstractmethod``, to be used to declare abstract methods. A class containing at least one method declared with this decorator that hasn't been overridden yet cannot be instantiated. Such a methods may be called from the overriding method in the subclass (using ``super`` or direct invocation). For example:: class A: @abstractmethod def foo(self): pass A() # raises TypeError class B(A): pass B() # raises TypeError class C(A): def foo(self): print(42) C() # works **Note:** The ``@abstractmethod`` decorator should only be used inside a class body. Dynamically adding abstract methods to a class, or attempting to modify the abstraction status of a method or class once it is created, are not supported. **Implementation:** The ``@abstractmethod`` decorator sets the function attribute ``__isabstractmethod__`` to the value ``True``. The ``type.__new__`` method computes the type attribute ``__abstractmethods__`` as the set of all method names that have an ``__isabstractmethod__`` attribute whose value is true. It does this by combining the ``__abstractmethods__`` attributes of the base classes, adding the names of all methods in the new class dict that have a true ``__isabstractmethod__`` attribute, and removing the names of all methods in the new class dict that don't have a true ``__isabstractmethod__`` attribute. If the resulting ``__abstractmethods__`` set is non-empty, the class is considered abstract, and attempts to instantiate it will raise ``TypeError``. (CPython can uses an internal flag ``Py_TPFLAGS_ABSTRACT`` to speed up this check [6]_.) **Discussion:** Unlike C++ or Java, abstract methods as defined here may have an implementation. This implementation can be called via the ``super`` mechanism from the class that overrides it. This could be useful as an end-point for a super-call in framework using a cooperative multiple-inheritance [7]_, [8]_. ABCs for Containers and Iterators --------------------------------- The ``collections`` module will define ABCs necessary and sufficient to work with sets, mappings, sequences, and some helper types such as iterators and dictionary views. The ABCs provide implementations of their abstract methods that are technically valid but fairly useless; e.g. ``__hash__`` returns 0, and ``__iter__`` returns an empty iterator. In general, the abstract methods represent the behavior of an empty container of the indicated type. Some ABCs also provide concrete (i.e. non-abstract) methods; for example, the ``Iterator`` class has an ``__iter__`` method returning itself, fulfilling an important invariant of iterators (which in Python 2 has to be implemented anew by each iterator class). No ABCs override ``__init__``, ``__new__``, ``__str__`` or ``__repr__``. Defining a standard constructor signature would unnecessarily constrain custom container types, for example Patricia trees or gdbm files. Defining a specific string representation for a collection is similarly left up to individual implementations. Ordering ABCs ''''''''''''' These ABCs are closer to ``object`` in the ABC hierarchy. ``PartiallyOrdered`` This ABC defines the 4 inequality operations ``<``, ``<=``, ``>=``, ``>``. (Note that ``==`` and ``!=`` are defined by ``object``.) Classes deriving from this ABC should implement a partial order as defined in mathematics. [9]_ ``TotallyOrdered`` This ABC derives from ``PartiallyOrdered``. It adds no new operations but implies a promise of stronger invariants. Classes deriving from this ABC should implement a total order as defined in mathematics. [10]_ **Open issues:** Where should these live? The ``collections`` module doesn't seem right, but making them built-ins seems a slippery slope too. One Trick Ponies '''''''''''''''' These abstract classes represent single methods like ``__iter__`` or ``__len__``. ``Hashable`` The base class for classes defining ``__hash__``. The ``__hash__`` method should return an ``Integer`` (see "Numbers" below). The abstract ``__hash__`` method always returns 0, which is a valid (albeit inefficient) implementation. **Invariant:** If classes ``C1`` and ``C2`` both derive from ``Hashable``, the condition ``o1 == o2`` must imply ``hash(o1) == hash(o2)`` for all instances ``o1`` of ``C1`` and all instances ``o2`` of ``C2``. IOW, two objects shouldn't compare equal but have different hash values. Another constraint is that hashable objects, once created, should never change their value (as compared by ``==``) or their hash value. If a class cannot guarantee this, it should not derive from ``Hashable``; if it cannot guarantee this for certain instances only, ``__hash__`` for those instances should raise a ``TypeError`` exception. **Note:** being an instance of this class does not imply that an object is immutable; e.g. a tuple containing a list as a member is not immutable; its ``__hash__`` method raises ``TypeError``. ``Iterable`` The base class for classes defining ``__iter__``. The ``__iter__`` method should always return an instance of ``Iterator`` (see below). The abstract ``__iter__`` method returns an empty iterator. ``Iterator`` The base class for classes defining ``__next__``. This derives from ``Iterable``. The abstract ``__next__`` method raises ``StopIteration``. The concrete ``__iter__`` method returns ``self``. ``Sized`` The base class for classes defining ``__len__``. The ``__len__`` method should return an ``Integer`` (see "Numbers" below) >= 0. The abstract ``__len__`` method returns 0. **Invariant:** If a class ``C`` derives from ``Sized`` as well as from ``Iterable``, the invariant ``sum(1 for x in o) == len(o)`` should hold for any instance ``o`` of ``C``. ``Container`` The base class for classes defining ``__contains__``. The ``__contains__`` method should return a ``bool``. The abstract ``__contains__`` method returns ``False``. **Invariant:** If a class ``C`` derives from ``Container`` as well as from ``Iterable``, then ``(x in o for x in o)`` should be a generator yielding only True values for any instance ``o`` of ``C``. **Note:** strictly speaking, there are three variants of this method's semantics. The first one is for sets and mappings, which is fast: O(1) or O(log N). The second one is for membership checking on sequences, which is slow: O(N). The third one is for subsequence checking on (character or byte) strings, which is also slow: O(N). Would it make sense to distinguish these? The signature of the third variant is different, since it takes a sequence (typically of the same type as the method's target) intead of an element. For now, I'm using the same type for all three. This means that is is possible for ``x in o`` to be True even though ``x`` is never yielded by ``iter(o)``. A suggested name for the third form is ``Searchable``. Sets '''' These abstract classes represent various stages of "set-ness". The most fundamental set operation is the membership test, written as ``x in s`` and implemented by ``s.__contains__(x)``. This is already taken care of by the `Container`` class defined above. Therefore, we define a set as a sized, iterable container for which certain invariants from mathematical set theory hold. The built-in type ``set`` derives from ``MutableSet``. The built-in type ``frozenset`` derives from ``HashableSet``. You might wonder why we require a set to be sized -- surely certain infinite sets can be represented just fine in Python. For example, the set of even integers could be defined like this:: class EvenIntegers(Container): def __contains__(self, x): return x % 2 == 0 However, such sets have rather limited practical value, and deciding whether one such set is a subset of another would be difficult in general without using a symbolic algebra package. So I consider this out of the scope of a pragmatic proposal like this. ``Set`` This is a sized, iterable, partially ordered container, i.e. a subclass of ``Sized``, ``Iterable``, ``Container`` and ``PartiallyOrdered``. Not every subset of those three classes is a set though! Sets have the additional invariant that each element occurs only once (as can be determined by iteration), and in addition sets define concrete operators that implement the inequality operations as subclass/superclass tests. In general, the invariants for finite sets in mathematics hold. [11]_ Sets with different implementations can be compared safely, (usually) efficiently and correctly using the mathematical definitions of the subclass/superclass operations for finite sets. The ordering operations have concrete implementations; subclasses may override these for speed but should maintain the semantics. Because ``Set`` derives from ``Sized``, ``__eq__`` may take a shortcut and returns ``False`` immediately if two sets of unequal length are compared. Similarly, ``__le__`` may return ``False`` immediately if the first set has more members than the second set. Note that set inclusion implements only a partial ordering; e.g. ``{1, 2}`` and ``{1, 3}`` are not ordered (all three of ``<``, ``==`` and ``>`` return ``False`` for these arguments). Sets cannot be ordered relative to mappings or sequences, but they can be compared to those for equality (and then they always compare unequal). **Note:** the ``issubset`` and ``issuperset`` methods found on the set type in Python 2 are not supported, as these are mostly just aliases for ``__le__`` and ``__ge__``. **Open issues:** should we define comparison of instances of different concrete set types this way? ``ComposableSet`` This is a subclass of ``Set`` that defines abstract operators to compute union, intersection, symmetric and asymmetric difference, respectively ``__or__``, ``__and__``, ``__xor__`` and ``__sub__``. These operators should return instances of ``ComposableSet``. The abstract implementations return no meaningful values but raise ``NotImplementedError``; this is because any generic implementation would have to create new instances but the ABCs don't (and shouldn't, IMO) provide an API for creating new instances. The implementations of these operators should ensure that the results match the mathematical definition of set composition. [11]_ **Open issues:** Should ``__or__`` and friends be abstract or concrete methods? Making them abstract means that every ComposableSet implementation must reimplement all of them. But making them concrete begs the question of the actual return type: since the ABC doesn't (and IMO shouldn't) define the constructor signature for subclasses, the concrete implementations in the ABC don't have an API to construct a new instance given an iterable. Perhaps the right choice is to have a static concrete factory function ``fromiterable`` which takes an iterable and returns a ``ComposableSet`` instance. Subclasses can override this and benefit from the default implementations of ``__or__`` etc.; or they can override ``__or__`` if they want to. ``HashableSet`` This is a subclass of both ``ComposableSet`` and ``Hashable``. It implements a concrete ``__hash__`` method that subclasses should not override; or if they do, the subclass should compute the same hash value. This is so that sets with different implementations still hash to the same value, so they can be used interchangeably as dictionary keys. (A similar constraint exists on the hash values for different types of numbers and strings.) **Open issues:** Spell out the hash algorithm. Should there be another ABC that derives from Set and Hashable, but not from Composable? ``MutableSet`` This is a subclass of ``ComposableSet`` implementing additional operations to add and remove elements. The supported methods have the semantics known from the ``set`` type in Python 2 (except for ``discard``, which is modeled after Java): ``.add(x)`` Abstract method returning a ``bool`` that adds the element ``x`` if it isn't already in the set. It should return ``True`` if ``x`` was added, ``False`` if it was already there. The abstract implementation raises ``NotImplementedError``. ``.discard(x)`` Abstract method returning a ``bool`` that removes the element ``x`` if present. It should return ``True`` if the element was present and ``False`` if it wasn't. The abstract implementation raises ``NotImplementedError``. ``.pop()`` Concrete method that removes an arbitrary item. If the set is empty, it raises ``KeyError``. The default implementation removes the first item returned by the set's iterator. ``.toggle(x)`` Concrete method returning a ``bool`` that adds x to the set if it wasn't there, but removes it if it was there. It should return ``True`` if ``x`` was added, ``False`` if it was removed. ``.clear()`` Concrete method that empties the set. The default implementation repeatedly calls ``self.pop()`` until ``KeyError`` is caught. (**Note:** this is likely much slower than simply creating a new set, even if an implementation overrides it with a faster approach; but in some cases object identity is important.) This also supports the in-place mutating operations ``|=``, ``&=``, ``^=``, ``-=``. These are concrete methods whose right operand can be an arbitrary ``Iterable``, except for ``&=``, whose right operand must be a ``Container``. This ABC does not support the named methods present on the built-in concrete ``set`` type that perform (almost) the same operations. Mappings '''''''' These abstract classes represent various stages of mapping-ness. The ``Mapping`` class represents the most common read-only mapping API. However, code *accepting* a mapping is encouraged to check for the ``BasicMapping`` ABC when iteration is not used. This allows for certain "black-box" implementations that can look up values by key but don't provide a convenient iteration API. A hypothetical example would be an interface to a hierarchical filesystem, where keys are pathnames relative to some root directory. Iterating over all pathnames would presumably take forever, as would counting the number of valid pathnames. The built-in type ``dict`` derives from ``MutableMapping``. ``BasicMapping`` A subclass of ``Container`` defining the following methods: ``.__getitem__(key)`` Abstract method that returns the value corresponding to ``key``, or raises ``KeyError``. The implementation always raises ``KeyError``. ``.get(key, default=None)`` Concrete method returning ``self[key]`` if this does not raise ``KeyError``, and the ``default`` value if it does. ``.__contains__()`` Concrete method returning ``True`` if ``self[key]`` does not raise ``KeyError``, and ``False`` if it does. ``Mapping`` A subclass of ``BasicMapping``, ``Iterable`` and ``Sized``. The keys of a mapping naturally form a set. The (key, value) pairs are also referred to as items. The items also form a set. Methods: ``__len__`` Abstract method returning the length of the key set. ``__iter__`` Abstract method returning each key in the key set exactly once. ``__eq__`` Concrete method for comparing mappings. Two mappings, even with different implementations, can be compared for equality, and are considered equal if and only iff their item sets are equal. **Open issues:** should we define comparison of instances of different concrete mapping types this way? ``keys`` Concrete method returning the key set as a ``Set``. The default concrete implementation returns a "view" on the key set (meaning if the underlying mapping is modified, the view's value changes correspondingly); subclasses are not required to return a view but they should return a ``Set``. ``items`` Concrete method returning the items as a ``Set``. The default concrete implementation returns a "view" on the item set; subclasses are not required to return a view but they should return a ``Set``. ``values`` Concrete method returning the values as a sized, iterable container (not a set!). The default concrete implementation returns a "view" on the values of the mapping; subclasses are not required to return a view but they should return a sized, iterable container. The following invariant should hold for any mapping ``m``:: set(m.items()) == set(zip(m.keys(), m.values())) i.e. iterating over the keys and the values in parallel should return *corresponding* keys and values. **Open issues:** Should this always be required? How about the stronger invariant using ``list()`` instead of ``set()``? ``HashableMapping`` A subclass of ``Mapping`` and ``Hashable``. The values should be instances of ``Hashable``. The concrete ``__hash__`` method should be equal to ``hash(m.items())``. ``MutableMapping`` A subclass of ``Mapping`` that also implements some standard mutating methods. Abstract methods include ``__setitem__``, ``__delitem__``. Concrete methods include ``pop``, ``popitem``, ``clear``, ``update``. **Note:** ``setdefault`` is *not* included. **Open issues:** Write out the specs for the methods. Sequences ''''''''' These abstract classes represent various stages of sequence-ness. The built-in ``list`` and ``bytes`` types derive from ``MutableSequence``. The built-in ``tuple`` and ``str`` types derive from ``HashableSequence``. ``Sequence`` A subclass of ``Iterable``, ``Sized``, ``Container``. It defines a new abstract method ``__getitem__`` that has a somewhat complicated signature: when called with an integer, it returns an element of the sequence or raises ``IndexError``; when called with a ``slice`` object, it returns another ``Sequence``. The concrete ``__iter__`` method iterates over the elements using ``__getitem__`` with integer arguments 0, 1, and so on, until ``IndexError`` is raised. The length should be equal to the number of values returned by the iterator. **Open issues:** Other candidate methods, which can all have default concrete implementations that only depend on ``__len__`` and ``__getitem__`` with an integer argument: __reversed__, index, count, __add__, __mul__, __eq__, __lt__, __le__. ``HashableSequence`` A subclass of ``Sequence`` and ``Hashable``. The concrete ``__hash__`` method should implements the hashing algorithms used by tuples in Python 2. ``MutableSequence`` A subclass of ``Sequence`` adding some standard mutating methods. Abstract mutating methods: ``__setitem__`` (for integer indices as well as slices), ``__delitem__`` (ditto), ``insert``, ``append``, ``reverse``. Concrete mutating methods: ``extend``, ``pop``, ``remove``. Concrete mutating operators: ``+=``, ``*=`` (these mutate the object in place). **Note:** this does not define ``sort()`` -- that is only required to exist on genuine ``list`` instances. **Open issues:** If all the elements of a sequence are totally ordered, the sequence itself can be totally ordered with respect to other sequences containing corresponding items of the same type. Should we reflect this by making ``Sequence`` derive from ``TotallyOrdered``? Or ``Partiallyordered``? Also, we could easily define comparison of sequences of different types, so that e.g. ``(1, 2, 3) == [1, 2, 3]`` and ``(1, 2) < [1, 2, 3]``. Should we? (It might imply ``["a", "b"] == "ab"`` and ``[1, 2] == b"\1\2"``.) Strings ------- Python 3000 has two built-in string types: byte strings (``bytes``), deriving from ``MutableSequence``, and (Unicode) character strings (``str``), deriving from ``HashableSequence``. They also derive from ``TotallyOrdered``. If we were to introduce ``Searchable``, they would also derive from that. **Open issues:** define the base interfaces for these so alternative implementations and subclasses know what they are in for. This may be the subject of a new PEP or PEPs (PEP 358 should be co-opted for the ``bytes`` type). Numbers ------- ABCs for numerical types are defined in PEP 3141. Guidelines for Writing ABCs --------------------------- Some suggestions for writing ABCs: * Use the ``@abstractmethod`` decorator. * Define abstract methods that could be useful as an end point when called via a super chain. * Define concrete methods that are very simple permutations of abstract methods (e.g. ``Mapping.get``). * Keep abstract classes small, one per use case instead of one per concept. ABCs vs. Alternatives ===================== In this section I will attempt to compare and contrast ABCs to other approaches that have been proposed. ABCs vs. Duck Typing -------------------- Does the introduction of ABCs mean the end of Duck Typing? I don't think so. Python will not require that a class derives from ``BasicMapping`` or ``Sequence`` when it defines a ``__getitem__`` method, nor will the ``x[y]`` syntax require that ``x`` is an instance of either ABC. You will still be able to assign any "file-like" object to ``sys.stdout``, as long as it has a ``write`` method. Of course, there will be some carrots to encourage users to derive from the appropriate base classes; these vary from default implementations for certain functionality to an improved ability to distinguish between mappings and sequences. But there are no sticks. If ``hasattr(x, __len__)`` works for you, great! ABCs are intended to solve problems that don't have a good solution at all in Python 2, such as distinguishing between mappings and sequences. ABCs vs. Generic Functions -------------------------- ABCs are compatible with Generic Functions (GFs). For example, my own Generic Functions implementation [4]_ uses the classes (types) of the arguments as the dispatch key, allowing derived classes to override base classes. Since (from Python's perspective) ABCs are quite ordinary classes, using an ABC in the default implementation for a GF can be quite appropriate. For example, if I have an overloaded ``prettyprint`` function, it would make total sense to define pretty-printing of sets like this:: @prettyprint.register(Set) def pp_set(s): return "{" + ... + "}" # Details left as an exercise and implementations for specific subclasses of Set could be added easily. I believe ABCs also won't present any problems for RuleDispatch, Phillip Eby's GF implementation in PEAK [5]_. Of course, GF proponents might claim that GFs (and concrete, or implementation, classes) are all you need. But even they will not deny the usefulness of inheritance; and one can easily consider the ABCs proposed in this PEP as optional implementation base classes; there is no requirement that all user-defined mappings derive from ``BasicMapping``. ABCs vs. Interfaces ------------------- ABCs are not intrinsically incompatible with Interfaces, but there is considerable overlap. For now, I'll leave it to proponents of Interfaces to explain why Interfaces are better. I expect that much of the work that went into e.g. defining the various shades of "mapping-ness" and the nomenclature could easily be adapted for a proposal to use Interfaces instead of ABCs. "Interfaces" in this context refers to a set of proposals for additional metadata elements attached to a class which are not part of the regular class hierarchy, but do allow for certain types of inheritance testing. Such metadata would be designed, at least in some proposals, so as to be easily mutable by an application, allowing application writers to override the normal classification of an object. The drawback to this idea of attaching mutable metadata to a class is that classes are shared state, and mutating them may lead to conflicts of intent. Additionally, the need to override the classification of an object can be done more cleanly using generic functions: In the simplest case, one can define a "category membership" generic function that simply returns False in the base implementation, and then provide overrides that return True for any classes of interest. References ========== .. [1] An Introduction to ABC's, by Talin (http://mail.python.org/pipermail/python-3000/2007-April/006614.html) .. [2] Incomplete implementation prototype, by GvR (http://svn.python.org/view/sandbox/trunk/abc/) .. [3] Possible Python 3K Class Tree?, wiki page created by Bill Janssen (http://wiki.python.org/moin/AbstractBaseClasses) .. [4] Generic Functions implementation, by GvR (http://svn.python.org/view/sandbox/trunk/overload/) .. [5] Charming Python: Scaling a new PEAK, by David Mertz (http://www-128.ibm.com/developerworks/library/l-cppeak2/) .. [6] Implementation of @abstractmethod (http://python.org/sf/1706989) .. [7] Unifying types and classes in Python 2.2, by GvR (http://www.python.org/download/releases/2.2.3/descrintro/) .. [8] Putting Metaclasses to Work: A New Dimension in Object-Oriented Programming, by Ira R. Forman and Scott H. Danforth (http://www.amazon.com/gp/product/0201433052) .. [9] Partial order, in Wikipedia (http://en.wikipedia.org/wiki/Partial_order) .. [10] Total order, in Wikipedia (http://en.wikipedia.org/wiki/Total_order) .. [11] Finite set, in Wikipedia (http://en.wikipedia.org/wiki/Finite_set) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Fri Apr 27 10:21:04 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 27 Apr 2007 20:21:04 +1200 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> <79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com> <462FFB41.8060507@canterbury.ac.nz> Message-ID: <4631B270.7070900@canterbury.ac.nz> Guido van Rossum wrote: > Then how do you explain the popularity of zope.interfaces in both the > Zope and the Twisted world? I don't know -- from my point of view, the fact that Zope and Twisted rely on them so much seems like a strangely unpythonic aberration. They don't seem to have caught on in a big way outside those two packages. > Read PEP 3119's Rationale. I did read it, and it seemed to be nothing more than a proposal to provide a way of doing something that shouldn't be done in the first place, i.e. large-scale use of LYBL. I see that you're now talking about providing some way of declaring a type as conforming to an ABC after the fact, which does mitigate things somewhat. But I still have large misgivings about the comefromish nature of such a thing, and its close cousins, adaptation and generic functions. From my point of view, it still seems like a rather elaborate solution in search of a deservingly pressing problem. -- Greg From martin at v.loewis.de Fri Apr 27 10:30:05 2007 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 27 Apr 2007 10:30:05 +0200 Subject: [Python-3000] PEP 3120 added Message-ID: <4631B48D.3080806@v.loewis.de> I added PEP 3120 (Using UTF-8 as the default source encoding, http://www.python.org/dev/peps/pep-3120/) to the repository. Should I set it to the accepted state? Martin From martin at v.loewis.de Fri Apr 27 10:33:52 2007 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 27 Apr 2007 10:33:52 +0200 Subject: [Python-3000] PEP 3121: Module Initialization and finalization Message-ID: <4631B570.7030307@v.loewis.de> Continuing a discussion from last April, I added PEP 3121, included below for convenience. Please comment. Regards, Martin PEP: 3121 Title: Module Initialization and finalization Version: $Revision: 54998 $ Last-Modified: $Date: 2007-04-27 10:31:58 +0200 (Fr, 27 Apr 2007) $ Author: Martin v. L?wis Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 27-Apr-2007 Python-Version: 3.0 Post-History: Abstract ======== Module initialization currently has a few deficiencies. There is no cleanup for modules, the entry point name might give naming conflicts, the entry functions don't follow the usual calling convention, and multiple interpreters are not supported well. This PEP addresses these issues. Module Finalization =================== Currently, C modules are initialized usually once and then "live" forever. The only exception is when Py_Finalize() is called: then the initialization routine is invoked a second time. This is bad from a resource management point of view: memory and other resources might get allocated each time initialization is called, but there is no way to reclaim them. As a result, there is currently no way to completely release all resources Python has allocated. Entry point name conflicts ========================== The entry point is currently called init. This might conflict with other symbols also called init. In particular, initsocket is known to have conflicted in the past (this specific problem got resolved as a side effect of renaming the module to _socket). Entry point signature ===================== The entry point is currently a procedure (returning void). This deviates from the usual calling conventions; callers can find out whether there was an error during initialization only by checking PyErr_Occurred. The entry point should return a PyObject*, which will be the module created, or NULL in case of an exception. Multiple Interpreters ===================== Currently, extension modules share their state across all interpreters. This allows for undesirable information leakage across interpreters: one script could permanently corrupt objects in an extension module, possibly breaking all scripts in other interpreters. Specification ============= The module initialization routines change their signature to:: PyObject *PyInit_() The initialization routine will be invoked once per interpreter, when the module is imported. It should return a new module object each time. In order to store per-module state in C variables, each module object will contain a block of memory that is interpreted only by the module. The amount of memory used for the module is specified at the point of creation of the module. In addition to the initialization function, a module may implement a number of additional callback function, which are invoked when the module's tp_traverse, tp_clear, and tp_free functions are invoked, and when the module is reloaded. The entire module definition is combined in a struct PyModuleDef:: struct PyModuleDef{ PyModuleDef_Base m_base; /* To be filled out by the interpreter */ Py_ssize_t m_size; /* Size of per-module data */ PyMethodDef *m_methods; inquiry m_reload; traverseproc m_traverse; inquiry m_clear; freefunc m_free; }; Creation of a module is changed to expect an optional PyModuleDef*. The module state will be null-initialized. Each module method with be passed the module object as the first parameter. To access the module data, a function:: void* PyModule_GetData(PyObject*); will be provided. In addition, to lookup a module more efficiently than going through sys.modules, a function:: PyObject* PyState_FindModule(struct PyModuleDef*); will be provided. This lookup function will use an index located in the m_base field, to find the module by index, not by name. As all Python objects should be controlled through the Python memory management, usage of "static" type objects is discouraged, unless the type object itself has no memory-managed state. To simplify definition of heap types, a new method:: PyTypeObject* PyType_Copy(PyTypeObject*); is added. Example ======= xxmodule.c would be changed to remove the initxx function, and add the following code instead:: struct xxstate{ PyObject *ErrorObject; PyObject *Xxo_Type; }; #define xxstate(o) ((struct xxstate*)PyModule_GetState(o)) static int xx_traverse(PyObject *m, visitproc v, void *arg) { Py_VISIT(xxstate(m)->ErrorObject); Py_VISIT(xxstate(m)->Xxo_Type); return 0; } static int xx_clear(PyObject *m) { Py_CLEAR(xxstate(m)->ErrorObject); Py_CLEAR(xxstate(m)->Xxo_Type); return 0; } static struct PyModuleDef xxmodule = { {}, /* m_base */ sizeof(struct xxstate), &xx_methods, 0, /* m_reload */ xx_traverse, xx_clear, 0, /* m_free - not needed, since all is done in m_clear */ } PyObject* PyInit_xx() { PyObject *res = PyModule_New("xx", &xxmodule); if (!res) return NULL; xxstate(res)->ErrorObject = PyErr_NewException("xx.error, NULL, NULL); if (!xxstate(res)->ErrorObject) { Py_DECREF(res); return NULL; } xxstate(res)->XxoType = PyType_Copy(&Xxo_Type); if (!xxstate(res)->Xxo_Type) { Py_DECREF(res); return NULL; } return res; } Discussion ========== Tim Peters reports in [1]_ that PythonLabs considered such a feature at one point, and lists the following additional hooks which aren't currently supported in this PEP: * when the module object is deleted from sys.modules * when Py_Finalize is called * when Python exits * when the Python DLL is unloaded (Windows only) References ========== .. [1] Tim Peters, reporting earlier conversation about such a feature http://mail.python.org/pipermail/python-3000/2006-April/000726.html Copyright ========= This document has been placed in the public domain. From jan.grant at bristol.ac.uk Fri Apr 27 15:47:59 2007 From: jan.grant at bristol.ac.uk (Jan Grant) Date: Fri, 27 Apr 2007 14:47:59 +0100 (BST) Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <873b2n5dc7.fsf@uwo.ca> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <873b2n5dc7.fsf@uwo.ca> Message-ID: <20070427144537.T1402@tribble.ilrt.bris.ac.uk> On Thu, 26 Apr 2007, Dan Christensen wrote: > Note also that double-precision reals are a subset of the rationals, > since each double precision real is exactly representable as a > rational number, but many rational numbers are not exactly > representable as double precision reals. Not sure if this means > that reals should be a subclass of the rationals. Not quite all: the space of doubles include a small number of things that aren't representable by a rational (+/- inf, for instance). -- jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/ Tel +44 (0)117 3317661 http://ioctl.org/jan/ Spreadsheet through network. Oh yeah. From guido at python.org Fri Apr 27 16:14:16 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 27 Apr 2007 07:14:16 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <20070427144537.T1402@tribble.ilrt.bris.ac.uk> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <873b2n5dc7.fsf@uwo.ca> <20070427144537.T1402@tribble.ilrt.bris.ac.uk> Message-ID: On 4/27/07, Jan Grant wrote: > On Thu, 26 Apr 2007, Dan Christensen wrote: > > > Note also that double-precision reals are a subset of the rationals, > > since each double precision real is exactly representable as a > > rational number, but many rational numbers are not exactly > > representable as double precision reals. Not sure if this means > > that reals should be a subclass of the rationals. > > Not quite all: the space of doubles include a small number of things > that aren't representable by a rational (+/- inf, for instance). This suddenly makes me think of a new idea -- perhaps we could changes the type of Inf and NaNs to some *other* numeric type? We could then reserve a place in the numeric hierarchy for its abstract base class. Though I don't know if this extends to complex numbers with one or both parts NaN/Inf or not. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Fri Apr 27 16:16:55 2007 From: barry at python.org (Barry Warsaw) Date: Fri, 27 Apr 2007 10:16:55 -0400 Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes In-Reply-To: References: Message-ID: <04A4F15C-38C3-4727-875D-82803F4FB974@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 26, 2007, at 2:50 PM, Guido van Rossum wrote: > After a fair amount of pre-discussion, I'm ready for the first > official review of this PEP. The PEP is online at > http://www.python.org/dev/peps/pep-3119/ Thanks for posting the PEP Guido. I haven't had time to follow the mailing list discussions, so this PEP was a great way to understand where you're heading with this proposal. Here are some comments. I think there's a typo in the paragraph that introduces __instancecheck__: "The call isinstance(x, C) first checks whether C.__instancecheck__ exists, and if so, calls C.__subclasscheck__(x) instead of its normal implementation." I think the C.__subclasscheck__ there ought to be C.__instancecheck__, right? After reading the PEP, it's pretty clear to me that ABCs are orthogonal to what /I've/ started using interfaces for, and that's fine. If they can live together, as I think is the case, then cool. There are a few things that I think are part of interfaces that aren't part of ABCs: - - Attributes. Interfaces allow you to make assertions about attributes, not just methods, while ABCs necessarily cover only methods. - - Documentation. Sure, you can docstring the abstract methods in an ABC, but it's not the emphasis of the ABC. For me, the main strength of interfaces is the emphasis on documenting the semantics of the interfaces, methods, and attributes that contribute to the contract you're asserting. Not just documenting them in off-line web pages, but right there in the code. There are of course plenty of other differences that people with more experience can hash out. Some that draw me to using interfaces include: - - The separation of inheritance and interface. ABCs conflate these concepts while interfaces keep them separate. I like the separation because it's clearer when reading the class definition which is which. If I care about what the class promises, I look at its interfaces. If I care about how the class keeps those promises, I look at its base classes. With ABCs I think I'd have to look at everything and suss out which is which. - - With interfaces, you can make assertions about individual objects which may be different than what their classes assert. Interface proponents seem to care a lot about this and it seems there are valid uses cases for it. For example, I've seen classes which map to a database table, where certain instances might assert additional properties (by adding interfaces) based on the value of a column. Another example of separating inheritance and interface comes up when you want to derive a subclass to share implementation details, but you want to subtly change the semantics, which would invalidate an ABC claim by the base class. Something like a GrowOnlyDictionary that derived from dict for implementation purposes, but didn't want to implement __delitem__ as required by the MutableMapping ABC. Finally, I'm concerned with the "weight" of adding ABCs to all the built-in types. There are two weights I'm thinking about: cognitive weight for people (newbies, casual programmers) trying to understand Python's built-in type system, and the weightiness of the additional implementation details (new modules, lots of new classes and objects) on applications that don't care about things like ABCs and interfaces. Not everything is a framework! :) I wonder if it would make sense or be possible to leave the built-in types lightweight and ABC-free. The Python implementation itself shouldn't care about ABCs, but any application, framework, or library that does would be able to define their own MutableMappingDict which derived from dict and MutableMapping. Or to promote consistency, there could be an abclib module which provides these subclasses for those applications and frameworks that care about them. Or we could have a standard set of GFs, adaptors, whatever, which hide the lightweightedness of the built-in types. Cheers, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRjIF3XEjvBPtnXfVAQLySQP+MRhTdbZzYqZVfUQu1Pu++QmLu5+Tfiu0 DHoF6EBxeaghPpLT6XElYXG7AkFOWk18XajtJWUikaaSlgckujVpEJ7nEUh6dfue rko3hiyy8XblHN4FjbGPkIja/+vgAdpFVpDIGBcNQouK1TPJ1pj7qfGPqoYpYdU2 7Woq2y9kIPU= =sfLK -----END PGP SIGNATURE----- From barry at python.org Fri Apr 27 16:26:05 2007 From: barry at python.org (Barry Warsaw) Date: Fri, 27 Apr 2007 10:26:05 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <4631B270.7070900@canterbury.ac.nz> References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> <462EB78A.207@canterbury.ac.nz> <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com> <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com> <79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com> <462FFB41.8060507@canterbury.ac.nz> <4631B270.7070900@canterbury.ac.nz> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 27, 2007, at 4:21 AM, Greg Ewing wrote: > Guido van Rossum wrote: > >> Then how do you explain the popularity of zope.interfaces in both the >> Zope and the Twisted world? > > I don't know -- from my point of view, the fact that > Zope and Twisted rely on them so much seems like a > strangely unpythonic aberration. They don't seem to > have caught on in a big way outside those two packages. Part of the reason may be that interfaces, ABCs, or whatever become more important as applications turn into frameworks. I sometimes think of my code along this axis of progression: quick hack < script < library < application < framework You can quibble about the order . I've almost never cared about interfaces for anything smaller than an application, and even for most of those I don't care. But as you approach frameworkiness, you start to care more because other people have to know where -- and how -- they can hook in. I think there are just few Python applications complex enough to warrant the additional weight of interfaces. For most Python code, keeping things light and malleable far outweighs any benefit that could be attained through interfaces. But get far enough to the right of that scale and the weight of dealing with complexity through interfaces becomes more cost effective. Cheers, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRjIH/XEjvBPtnXfVAQJ0CAP/Qbk6dXnNqpML2ujemL3uT7jT34veNb2g kvxsfeDq4ibaJ/wDLR/ToMUaltBxmO7Y8GoCd455uZJ7JvCNlx7UanjjICp5MA/S SZYV97W2fxZKTnlcATd1YEDCA7l+Um+ysP/aDQNle9QmyrdlW81DzG8GFU9lsn6W GaR1k+/s/4s= =HXQ1 -----END PGP SIGNATURE----- From jyasskin at gmail.com Fri Apr 27 16:58:43 2007 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Fri, 27 Apr 2007 07:58:43 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <873b2n5dc7.fsf@uwo.ca> <20070427144537.T1402@tribble.ilrt.bris.ac.uk> Message-ID: <5d44f72f0704270758n27f3c20y8e2a33c24d8128dd@mail.gmail.com> On 4/27/07, Guido van Rossum wrote: > On 4/27/07, Jan Grant wrote: > > On Thu, 26 Apr 2007, Dan Christensen wrote: > > > > > Note also that double-precision reals are a subset of the rationals, > > > since each double precision real is exactly representable as a > > > rational number, but many rational numbers are not exactly > > > representable as double precision reals. Not sure if this means > > > that reals should be a subclass of the rationals. > > > > Not quite all: the space of doubles include a small number of things > > that aren't representable by a rational (+/- inf, for instance). > > This suddenly makes me think of a new idea -- perhaps we could changes > the type of Inf and NaNs to some *other* numeric type? We could then > reserve a place in the numeric hierarchy for its abstract base class. > Though I don't know if this extends to complex numbers with one or > both parts NaN/Inf or not. From the Fortress spec: "The trait Q ( QQ ) encompasses all finite rational numbers, the results of dividing any integer by any nonzero integer. The trait Q? ( QQ_star ) is Q with two extra elements, + ? and ?? . The trait Q# ( QQ_splat ) is Q? with one additional element, the indefinite rational (written 0/0 ), which is used as the result of dividing zero by zero or of adding ?? to +?." So separating Inf and NaN into other types has some precedent. I'd also point out that A being a subset of B doesn't make it a subtype also. If operations on A behave differently than the specification of the operations on B, then As aren't substitutable for Bs and A isn't a subtype. Because doubles have finite precision and rationals don't, I don't think doubles are a subtype of the rationals, even if you juggle Nan/Inf to make them a subset. Then again, doubles aren't a group either because of this imprecision, and I'm suggesting claiming they're a subclass of that, so maybe there's room in a practical language to make them a subclass of the rationals too. -- Namast?, Jeffrey Yasskin From pje at telecommunity.com Fri Apr 27 17:22:07 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 27 Apr 2007 11:22:07 -0400 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <4631A57E.1000301@canterbury.ac.nz> References: <4630CCAD.1040400@acm.org> <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <4630CCAD.1040400@acm.org> Message-ID: <5.1.1.6.0.20070427112016.02bf4710@sparrow.telecommunity.com> At 07:25 PM 4/27/2007 +1200, Greg Ewing wrote: >Talin wrote: > > > I kind of wonder about the idea of implementing generic functions using > > generic functions. > >I'd be worried about efficiency. Generic functions on >top of generic functions sounds like you would have two >layers of dynamic lookup to go through instead of one. No, it only affects method addition and index updating (which only occurs when new types are encountered or new methods are added). You don't need the generic functions in order to dispatch generic functions, only to build the dispatch tables. From jimjjewett at gmail.com Fri Apr 27 19:05:29 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 27 Apr 2007 13:05:29 -0400 Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes In-Reply-To: References: Message-ID: On 4/26/07, Guido van Rossum wrote: > - Where should PartiallyOrdered and TotallyOrdered live? The abc module seems reasonable, even though the rest are containers. > - Should we support comparison of different concrete set types? > - Ditto for mapping types? > - Ditto for sequence types? > - Should Sequence derive from TotallyOrdered? I think either choice would have been reasonable once upon a time, but it is too late for "abc" = ("a", "b", "c"), because they can appear as keys in the same dict. I don't know if this constraint applies to mappings or sets. > - Do we need a non-composable hashable set type? > - Ditto for a non-composable mutable set type? Given iterable, non-composable is at best rare. So long as it is possible to override the isinstance and issubclass checks, I think they can be left out of the stdlib. (Though I think you use one as a test case, which is a good idea.) > No ABCs override ``__init__``, ``__new__``, ``__str__`` or > ``__repr__``. Defining a standard constructor signature would > unnecessarily constrain custom container types, for example Patricia > trees or gdbm files. Defining a specific string representation for a > collection is similarly left up to individual implementations. Is this a style guide, or an actual constraint that should apply to user-created ABCs? > ``Hashable`` > The base class for classes defining ``__hash__``. The > ``__hash__`` method should return an ``Integer`` (see "Numbers" > below). The abstract ``__hash__`` method always returns 0, which > is a valid (albeit inefficient) implementation. **Invariant:** If > classes ``C1`` and ``C2`` both derive from ``Hashable``, the > condition ``o1 == o2`` must imply ``hash(o1) == hash(o2)`` for all > instances ``o1`` of ``C1`` and all instances ``o2`` of ``C2``. > IOW, two objects shouldn't compare equal but have different hash > values. Is this really restricted to classes implementing Hashable? I thought it was an existing general python rule that for all non-buggy objects a and b (a == b) ==> (hash(a) == hash(b)) unless at least one of hash(a), hash(b) returned -1 or raised an error. > Another constraint is that hashable objects, once created, should > never change their value (as compared by ``==``) or their hash > value. I've had a use case where objects were placed in a dict before they were finalized. Their value would change only in that certain attributes could go (once) from undefined to defined. Several different objects might pass through the same indetermininate stage, but only one would be in that state at a time. (Yes, there was probably a better architecture if I weren't dealing with legacy formats.) Basing hash on only the attributes always known from the start meant that the hash would never change, and there weren't any equality tests whose results could change, though the value as a whole (for complete ordering) could change. > **Note:** the ``issubset`` and ``issuperset`` methods found on the > set type in Python 2 are not supported, as these are mostly just > aliases for ``__le__`` and ``__ge__``. And I'll again lobby to include them, if only as concrete implementations that say forward to self.__le__(other). > Strings > ------- > > Python 3000 has two built-in string types: byte strings (``bytes``), > deriving from ``MutableSequence``, and (Unicode) character strings > (``str``), deriving from ``HashableSequence``. They also derive from > ``TotallyOrdered``. If we were to introduce ``Searchable``, they > would also derive from that. > > **Open issues:** define the base interfaces for these so alternative > implementations and subclasses know what they are in for. This may be > the subject of a new PEP or PEPs (PEP 358 should be co-opted for the > ``bytes`` type). By Monday...? -jJ From jimjjewett at gmail.com Fri Apr 27 19:10:24 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 27 Apr 2007 13:10:24 -0400 Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes In-Reply-To: <04A4F15C-38C3-4727-875D-82803F4FB974@python.org> References: <04A4F15C-38C3-4727-875D-82803F4FB974@python.org> Message-ID: On 4/27/07, Barry Warsaw wrote: > - - Attributes. Interfaces allow you to make assertions about > attributes, not just methods, while ABCs necessarily cover only methods. Why can't they have data attributes as well? > - - With interfaces, you can make assertions about individual objects > which may be different than what their classes assert. Interface > proponents seem to care a lot about this and it seems there are valid > uses cases for it. Isn't this something that could be handled by overriding isinstance? > Another example of separating inheritance and interface comes up when > you want to derive a subclass to share implementation details, but > you want to subtly change the semantics, which would invalidate an > ABC claim by the base class. Something like a GrowOnlyDictionary > that derived from dict for implementation purposes, but didn't want > to implement __delitem__ as required by the MutableMapping ABC. OK, that makes the isubclass override trickier, so there should be an example, but I think it can still be done. > Finally, I'm concerned with the "weight" of adding ABCs to all the > built-in types. What if the builtin types did not initially derive from any ABC, but were added (through an issubclass override) when the abc module was imported? -jJ From janssen at parc.com Fri Apr 27 19:40:02 2007 From: janssen at parc.com (Bill Janssen) Date: Fri, 27 Apr 2007 10:40:02 PDT Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes In-Reply-To: References: <04A4F15C-38C3-4727-875D-82803F4FB974@python.org> Message-ID: <07Apr27.104005pdt."57996"@synergy1.parc.xerox.com> Jim Jewett wrote: > On 4/27/07, Barry Warsaw wrote: > > > - - Attributes. Interfaces allow you to make assertions about > > attributes, not just methods, while ABCs necessarily cover only methods. > > Why can't they have data attributes as well? I agree with Jim. It seems to me that documenting standard attributes of a type is one thing that ABC's should be able to do. Perhaps a "@abstractattribute" decorator of some sort? Perhaps a class decorator "@attributes(name1, name2, ...)"? Attributes with those names would be automatically added to an instance, initialized to None, before the first __init__ method would be called. Bill From lists at cheimes.de Fri Apr 27 19:28:00 2007 From: lists at cheimes.de (Christian Heimes) Date: Fri, 27 Apr 2007 19:28:00 +0200 Subject: [Python-3000] Generic function PEP won't make it in time In-Reply-To: <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> References: <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com> <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com> <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com> <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com> <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com> Message-ID: Phillip J. Eby wrote: > As someone with more recent background in Java than C++, I find the idea of > abstract methods having an executable implementation to be quite confusing, > and suspect that other people with that Java or C# background will do the > same thing. That is, skim the explanation and miss the significant > difference between the C++ way and what they're used to. From the naive point of view from somebody with less background knowledge about ABCs, abstract methods and generics I expect that * an abstract class can't be instantiated. An abstract class must be sub-classed before it can be instantiated. * an abstract method has no implementation and simply raises a NonImplementedError based exception. * a class with abstract methods is automatically an abstract class unless generics are entering the game of class creation. When a generic overwrites an abstract method the class is a concrete class but when the abstract method is called an exception is raised. class AbstractEgg(abstract=True): """Abstract class, cannot be instantiated """ def egg(self, foo): pass or: @abstract class AbstractEgg: pass class Egg(AbstractEgg): """Subclass can be instantiated """ class AbstractSpam: """Abstract class because it contains an abstract method """ @abstract def egg(self, foo): pass class Spam(AbstractSpam): """Still an abstract method because it doesn't overwrite egg() """ class GenericSpam(AbstractSpam): """Subclass can be instantiated I'm not sure about the @generic syntax but it's just an example """ @generic(int) def egg(self, foo): pass >>> gs = GenericSpam() This works: >>> gs.egg(1) This raises an AbstractMethodException(NotImplementedException) >>> gs.egg('foo') Just my 5 cents from an ordinary Python user who is trying to learn ABCs and generics. Christian From brett at python.org Fri Apr 27 20:23:49 2007 From: brett at python.org (Brett Cannon) Date: Fri, 27 Apr 2007 11:23:49 -0700 Subject: [Python-3000] PEP 3121: Module Initialization and finalization In-Reply-To: <4631B570.7030307@v.loewis.de> References: <4631B570.7030307@v.loewis.de> Message-ID: On 4/27/07, "Martin v. L?wis" wrote: > Continuing a discussion from last April, I added > PEP 3121, included below for convenience. Please > comment. > > Regards, > Martin > > PEP: 3121 > Title: Module Initialization and finalization > Version: $Revision: 54998 $ > Last-Modified: $Date: 2007-04-27 10:31:58 +0200 (Fr, 27 Apr 2007) $ > Author: Martin v. L?wis > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 27-Apr-2007 > Python-Version: 3.0 > Post-History: > > Abstract > ======== > > Module initialization currently has a few deficiencies. There is no > cleanup for modules, the entry point name might give naming conflicts, > the entry functions don't follow the usual calling convention, and > multiple interpreters are not supported well. This PEP addresses these > issues. > Thanks for trying to solve this, Martin! > Module Finalization > =================== > > Currently, C modules are initialized usually once and then "live" > forever. The only exception is when Py_Finalize() is called: then > the initialization routine is invoked a second time. This is bad > from a resource management point of view: memory and other resources > might get allocated each time initialization is called, but there > is no way to reclaim them. As a result, there is currently no > way to completely release all resources Python has allocated. > > Entry point name conflicts > ========================== > > The entry point is currently called init. This might conflict > with other symbols also called init. In particular, > initsocket is known to have conflicted in the past (this specific > problem got resolved as a side effect of renaming the module to > _socket). > > Entry point signature > ===================== > > The entry point is currently a procedure (returning void). This > deviates from the usual calling conventions; callers can find out > whether there was an error during initialization only by checking > PyErr_Occurred. The entry point should return a PyObject*, which will > be the module created, or NULL in case of an exception. > > Multiple Interpreters > ===================== > > Currently, extension modules share their state across all > interpreters. This allows for undesirable information leakage across > interpreters: one script could permanently corrupt objects in an > extension module, possibly breaking all scripts in other interpreters. > After the intro and up to here, it would seem like changing each section into a sub-section of a "Problems" section of some sort would make the document more organized and easier to read. > Specification > ============= > > The module initialization routines change their signature > to:: > > PyObject *PyInit_() > > The initialization routine will be invoked once per > interpreter, when the module is imported. It should > return a new module object each time. > > In order to store per-module state in C variables, > each module object will contain a block of memory > that is interpreted only by the module. The amount > of memory used for the module is specified at > the point of creation of the module. > > In addition to the initialization function, a module > may implement a number of additional callback > function, which are invoked when the module's > tp_traverse, tp_clear, and tp_free functions are > invoked, and when the module is reloaded. > > The entire module definition is combined in a struct > PyModuleDef:: > > struct PyModuleDef{ > PyModuleDef_Base m_base; /* To be filled out by the interpreter */ > Py_ssize_t m_size; /* Size of per-module data */ > PyMethodDef *m_methods; > inquiry m_reload; > traverseproc m_traverse; > inquiry m_clear; > freefunc m_free; > }; > > Creation of a module is changed to expect an optional > PyModuleDef*. The module state will be > null-initialized. > > Each module method with be passed the module object I think you meant "will", not "with". > as the first parameter. To access the module data, > a function:: > > void* PyModule_GetData(PyObject*); > > will be provided. In addition, to lookup a module > more efficiently than going through sys.modules, > a function:: > > PyObject* PyState_FindModule(struct PyModuleDef*); > > will be provided. This lookup function will use an > index located in the m_base field, to find the > module by index, not by name. > > As all Python objects should be controlled through > the Python memory management, usage of "static" > type objects is discouraged, unless the type object > itself has no memory-managed state. Ooh, I like this side-effect of the proposal! > To simplify > definition of heap types, a new method:: > > PyTypeObject* PyType_Copy(PyTypeObject*); > > is added. > > Example > ======= > > xxmodule.c would be changed to remove the initxx > function, and add the following code instead:: > > struct xxstate{ > PyObject *ErrorObject; > PyObject *Xxo_Type; > }; > > #define xxstate(o) ((struct xxstate*)PyModule_GetState(o)) > > static int xx_traverse(PyObject *m, visitproc v, > void *arg) > { > Py_VISIT(xxstate(m)->ErrorObject); > Py_VISIT(xxstate(m)->Xxo_Type); > return 0; > } > > static int xx_clear(PyObject *m) > { > Py_CLEAR(xxstate(m)->ErrorObject); > Py_CLEAR(xxstate(m)->Xxo_Type); > return 0; > } > > static struct PyModuleDef xxmodule = { > {}, /* m_base */ > sizeof(struct xxstate), > &xx_methods, > 0, /* m_reload */ > xx_traverse, > xx_clear, > 0, /* m_free - not needed, since all is done in m_clear */ > } > > PyObject* > PyInit_xx() > { > PyObject *res = PyModule_New("xx", &xxmodule); > if (!res) return NULL; > xxstate(res)->ErrorObject = PyErr_NewException("xx.error, NULL, NULL); > if (!xxstate(res)->ErrorObject) { > Py_DECREF(res); > return NULL; > } > xxstate(res)->XxoType = PyType_Copy(&Xxo_Type); > if (!xxstate(res)->Xxo_Type) { > Py_DECREF(res); > return NULL; > } > return res; > } > How would I raise xx.error in C code now? I am guessing like this:: PyObject* module = PyState_FindModule(&xxmodule); if (!module) return NULL; PyObject* xx_error = xxstate(module)->ErrorObject; if (!xx_error) { PyErr_SetString(PyExc_SystemError, "xx.error missing"); return NULL; } PyErr_SetString(xx_error, "oops"); return NULL; Since most objects will move to being memory-managed, one needs to worry about checking that the object still exists. I assume I didn't go overboard with the error checking here, right? I guess people are going to end up writing helper functions to access the various data fields as the above would get rather tedious if you had to write it more than twice. > > Discussion > ========== > > Tim Peters reports in [1]_ that PythonLabs considered such a feature > at one point, and lists the following additional hooks which aren't > currently supported in this PEP: > > * when the module object is deleted from sys.modules > > * when Py_Finalize is called > > * when Python exits > Wouldn't the above be covered by the deallocation of the module? Overall I like the idea. I think people will need to get used to the idea of writing more accessor functions for the data field, though, if using static variables to hold things like exceptions becomes discouraged. -Brett From tony at PageDNA.com Fri Apr 27 20:17:41 2007 From: tony at PageDNA.com (Tony Lownds) Date: Fri, 27 Apr 2007 11:17:41 -0700 Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes In-Reply-To: <07Apr27.104005pdt."57996"@synergy1.parc.xerox.com> References: <04A4F15C-38C3-4727-875D-82803F4FB974@python.org> <07Apr27.104005pdt."57996"@synergy1.parc.xerox.com> Message-ID: <217F4CF1-3CC1-48CE-A635-877C22562C78@PageDNA.com> On Apr 27, 2007, at 10:40 AM, Bill Janssen wrote: > Jim Jewett wrote: >> On 4/27/07, Barry Warsaw wrote: >> >>> - - Attributes. Interfaces allow you to make assertions about >>> attributes, not just methods, while ABCs necessarily cover only >>> methods. >> >> Why can't they have data attributes as well? > > I agree with Jim. It seems to me that documenting standard attributes > of a type is one thing that ABC's should be able to do. Perhaps a > "@abstractattribute" decorator of some sort? Perhaps a class > decorator > "@attributes(name1, name2, ...)"? Attributes with those names > would be > automatically added to an instance, initialized to None, before the > first __init__ method would be called. +0 on abstract attributes. Methods seem to dominate most APIs that make use of interfaces, but there are always a few exceptions. Attribute annotations could provide an elegant solution here. class ABC: concrete_attribute : "any expression" = None abstract_attribute : "any expression" I posted about attribute annotations before on python-ideas and there was lukewarm response. If anyone thinks a PEP on attribute annotations is worth submitting, let me know. Thanks -Tony From fdrudol5 at rockwellcollins.com Fri Apr 27 20:25:07 2007 From: fdrudol5 at rockwellcollins.com (fdrudol5 at rockwellcollins.com) Date: Fri, 27 Apr 2007 13:25:07 -0500 Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes Message-ID: The PEP looks good. A few nits and a couple of related questions: MutableSet.pop() "...removes an arbitrary..." # should be "...removes and returns an arbitrary..." BasicMapping .__contains__() # should be .__contains__(key) In Mapping.__eq__, "if and only iff" is redundant. And yes, I think the stated semantics are useful. The invariant set(m.items()) == set(zip(m.keys(), m.values())) is not strong enough and using list wouldn't fix it. A really goofy implementation that iterates values in a different order than keys, then defines items as returning the zip would satisfy the invariant. How about: [value for value in m.values()] == [m[key] for key in m.keys()] [item for item in m.items()] == [(key, m[key]) for key in m.keys()] Do we need to explicitly state the invariant len(m.values()) == len(m.keys())? Is this an invariant? all(isinstance(item,tuple) for item in m.items()) Really minor nit: inconsistent use of "." and "()": ".pop()", ".clear()", but "items", "values", etc. In Sequence, the other method candidates all make sense, should have standard semantics, and have easy default implementations. Might as well do it. Can the standard ABCs be usefully superclassed? For example, in a class for potentially infinite sets that can be unioned and intersected with builtin sets, would __subclass__ = {ComposableSet} be useful? In the context of parameter annotations, having homogeneous containers (sets of strings, sequences of integers) were discussed. Can wrappers be created so that a content invariant can be enforced on a container and easily checked against a constraint on a formal parameter? #Rudy From thomas at thomas-lotze.de Fri Apr 27 20:53:35 2007 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Fri, 27 Apr 2007 20:53:35 +0200 Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes References: Message-ID: Guido van Rossum wrote: > Another constraint is that hashable objects, once created, should > never change their value (as compared by ``==``) or their hash value. > If a class cannot guarantee this, it should not derive from > ``Hashable``; if it cannot guarantee this for certain instances only, > ``__hash__`` for those instances should raise a ``TypeError`` > exception. Shouldn't this rather be a ValueError since it occurs not because of the type of the object in question, but its value (while in general, there are instances of the same type representing other values which are hashable)? > The third one is for subsequence > checking on (character or byte) strings, which is also slow: O(N). > Would it make sense to distinguish these? The signature of the third > variant is different, since it takes a sequence (typically of the same > type as the method's target) intead of an element. For now, I'm using > the same type for all three. This means that is is possible for ``x > in o`` to be True even though ``x`` is never yielded by ``iter(o)``. > A suggested name for the third form is ``Searchable``. To me, 'searchable' isn't associated with subsequences in any way. I'd think of a container that is able to answer the question "where is this element?" A sequence might return an index or a set of indexes, a mapping might return a key or a set of keys in reply to this. If what you're after is really a subsequence containment test, it should just live on sequences instead of getting an ABC of its own, IMO. > **Open issues:** If all the elements of a sequence are totally ordered, > the sequence itself can be totally ordered with respect to other sequences > containing corresponding items of the same type. Should we reflect this by > making ``Sequence`` derive from ``TotallyOrdered``? Or > ``Partiallyordered``? Unless I'm mistaken, since a sequence type in general can't know whether all the elements in a given instance will be totally ordered, deriving Sequence from TotallyOrdered would make promises it can't keep. But I'm +1 on deriving Sequence from PartiallyOrdered since sequences do have an ordering relationship between some of their values. > Also, we could easily define comparison of > sequences of different types, so that e.g. ``(1, 2, 3) == [1, 2, 3]`` and > ``(1, 2) < [1, 2, 3]``. Should we? (It might imply ``["a", "b"] == "ab"`` > and ``[1, 2] == b"\1\2"``.) +1 -- Thomas From jimjjewett at gmail.com Fri Apr 27 21:06:31 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 27 Apr 2007 15:06:31 -0400 Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes In-Reply-To: References: Message-ID: On 4/27/07, Thomas Lotze wrote: > Guido van Rossum wrote: > > ``__hash__`` for those instances should raise a ``TypeError`` > > exception. > Shouldn't this rather be a ValueError since it occurs not because of the > type of the object in question, but its value (while in general, there are > instances of the same type representing other values which are hashable)? Yes, but it is a TypeError today. Is it worth the backwards compatibility? > > [It] is possible for ``x > > in o`` to be True even though ``x`` is never yielded by ``iter(o)``. > To me, 'searchable' isn't associated with subsequences in any way. I'd > think of a container that is able to answer the question "where is this > element?" A sequence might return an index or a set of indexes, a mapping > might return a key or a set of keys in reply to this. If what you're after > is really a subsequence containment test, it should just live on sequences > instead of getting an ABC of its own, IMO. That makes sense. Looking at unordered containers like sets, should {"a", "b"} in {"b", "a", "d"} be able to return true, by deriving from Searchable? If not, then I agree that it is really about sequences. -jJ From thomas at thomas-lotze.de Fri Apr 27 21:47:30 2007 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Fri, 27 Apr 2007 21:47:30 +0200 Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes References: Message-ID: Jim Jewett wrote: > Yes, but it is a TypeError today. Is it worth the backwards > compatibility? I'd say yes for the sake of doing things right, in particular as we are talking about Py3k, but then I don't think I have a good picture of all the things that would be broken by this change. > That makes sense. Looking at unordered containers like sets, should > > {"a", "b"} in {"b", "a", "d"} be able to return true, by deriving from > Searchable? No. {"a", "b"} clearly is a subset of {"b", "a", "d"}, not an element. So {"a", "b"} < {"b", "a", "d"}, but {"a", "b"} not in {"b", "a", "d"}. -- Thomas From jason.orendorff at gmail.com Fri Apr 27 22:24:14 2007 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Fri, 27 Apr 2007 16:24:14 -0400 Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes In-Reply-To: References: Message-ID: On 4/26/07, Guido van Rossum wrote: > - Where should PartiallyOrdered and TotallyOrdered live? Could someone please post some sample code using either of these? I don't see what they're good for. Here's my failed attempt: def my_max(a : TotallyOrdered, b : TotallyOrdered): return a if a >= b else b But this is both too loose (consider my_max(7, "spam")) and too strict (consider my_max([1, 2], [2, 1])). -j From martin at v.loewis.de Sat Apr 28 00:44:25 2007 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Apr 2007 00:44:25 +0200 Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD Message-ID: <46327CC9.4040901@v.loewis.de> I propose the PEP below for Py3k. Regards, Martin PEP: 3122 Title: Dropping PyObject_HEAD Version: $Revision: 54998 $ Last-Modified: $Date: 2007-04-27 10:31:58 +0200 (Fr, 27 Apr 2007) $ Author: Martin v. L?wis Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 27-Apr-2007 Python-Version: 3.0 Post-History: Abstract ======== Python currently relies on undefined C behavior, with its usage of PyObject_HEAD. This PEP proposes to change that into standard C. Rationale ========= Standard C defines that an object must be accessed only through a pointer of its type, and that all other accesses are undefined behavior, with a few exceptions. In particular, the following code has undefined behavior:: struct FooObject{ PyObject_HEAD int data; }; PyObject *foo(struct FooObject*f){ return (PyObject*)f; } int bar(){ struct FooObject *f = malloc(sizeof(struct FooObject)); struct PyObject *o = foo(f); f->ob_refcnt = 0; o->ob_refcnt = 1; return f->ob_refcnt; } The problem here is that the storage is both accessed as if it where struct PyObject, and as struct FooObject. Historically, compilers did not cause any problems with this code. However, modern compiler use that clause as an optimization opportunity, finding that f->ob_refcnt and o->ob_refcnt cannot possibly refer to the same memory, and that therefore the function should return 0, without having to fetch the value of ob_refcnt at all in the return statement. For GCC, Python now uses -fno-strict-aliasing to work around that problem; with other compilers, it may just see undefined behavior. Even with GCC, using -fno-strict-aliasing may pessimize the generated code unnecessarily. Specification ============= Standard C has one specific exception to its aliasing rules precisely designed to support the case of Python: a value of a struct type may also be accessed through a pointer to the first field. E.g. if a struct starts with an int, the struct\* may also be cast to an int\*, allowing to write int values into the first field. For Python, PyObject_HEAD and PyObject_VAR_HEAD will be dropped, and PyObject gets defined to contain all fields explicitly:: typedef struct _object{ _PyObject_HEAD_EXTRA Py_ssize_t ob_refcnt; struct _typeobject *ob_type; }PyObject; typedef struct { PyObject ob_base; Py_ssize_t ob_size; } PyVarObject; Types defined as fixed-size structure will then include PyObject as its first field; variable-sized objects PyVarObject. E.g.:: typedef struct{ PyObject ob_base; PyObject *start, *stop, *step; } PySliceObject; typedef struct{ PyVarObject ob_base; PyObject **ob_item; Py_ssize_t allocated; } PyListObject; As a convention, the base field SHOULD be called ob_base. However, all accesses to ob_refcnt and ob_type MUST cast the object pointer to PyObject* (unless the pointer is already known to have that type), and SHOULD use the respective accessor macros. To simplify access to ob_type, a macro:: #define Py_Type(o) (((PyObject*)o)->ob_type) is added. Copyright ========= This document has been placed in the public domain. From brett at python.org Sat Apr 28 01:30:50 2007 From: brett at python.org (Brett Cannon) Date: Fri, 27 Apr 2007 16:30:50 -0700 Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD In-Reply-To: <46327CC9.4040901@v.loewis.de> References: <46327CC9.4040901@v.loewis.de> Message-ID: Second PEP today. Martin is on a roll! =) On 4/27/07, "Martin v. L?wis" wrote: > I propose the PEP below for Py3k. > > Regards, > Martin > > PEP: 3122 > Title: Dropping PyObject_HEAD > Version: $Revision: 54998 $ > Last-Modified: $Date: 2007-04-27 10:31:58 +0200 (Fr, 27 Apr 2007) $ > Author: Martin v. L?wis > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 27-Apr-2007 > Python-Version: 3.0 > Post-History: > > Abstract > ======== > > Python currently relies on undefined C behavior, with its > usage of PyObject_HEAD. This PEP proposes to change that > into standard C. > > Rationale > ========= > > Standard C defines that an object must be accessed only through a > pointer of its type, and that all other accesses are undefined > behavior, with a few exceptions. In particular, the following > code has undefined behavior:: > > struct FooObject{ > PyObject_HEAD > int data; > }; > > PyObject *foo(struct FooObject*f){ > return (PyObject*)f; > } > > int bar(){ > struct FooObject *f = malloc(sizeof(struct FooObject)); > struct PyObject *o = foo(f); > f->ob_refcnt = 0; > o->ob_refcnt = 1; > return f->ob_refcnt; > } > > The problem here is that the storage is both accessed as > if it where struct PyObject, and as struct FooObject. > > Historically, compilers did not cause any problems with this Reads easier if you replace "cause" with "have". > code. However, modern compiler use that clause as an Probably want to pluralize "compiler". Your use of "clause" really confused me until I realized what you were talking about. > optimization opportunity, finding that f->ob_refcnt and > o->ob_refcnt cannot possibly refer to the same memory, and > that therefore the function should return 0, without having > to fetch the value of ob_refcnt at all in the return > statement. For GCC, Python now uses -fno-strict-aliasing > to work around that problem; with other compilers, it > may just see undefined behavior. Even with GCC, using > -fno-strict-aliasing may pessimize the generated code > unnecessarily. > > Specification > ============= > > Standard C has one specific exception to its aliasing rules precisely > designed to support the case of Python: a value of a struct type may > also be accessed through a pointer to the first field. E.g. if a > struct starts with an int, the struct\* may also be cast to an int\*, > allowing to write int values into the first field. > > For Python, PyObject_HEAD and PyObject_VAR_HEAD will be dropped, and > PyObject gets defined to contain all fields explicitly:: > > typedef struct _object{ > _PyObject_HEAD_EXTRA > Py_ssize_t ob_refcnt; > struct _typeobject *ob_type; > }PyObject; > > typedef struct { > PyObject ob_base; > Py_ssize_t ob_size; > } PyVarObject; > > Types defined as fixed-size structure will then include PyObject > as its first field; variable-sized objects PyVarObject. E.g.:: > > typedef struct{ > PyObject ob_base; > PyObject *start, *stop, *step; > } PySliceObject; > > typedef struct{ > PyVarObject ob_base; > PyObject **ob_item; > Py_ssize_t allocated; > } PyListObject; > > As a convention, the base field SHOULD be called ob_base. However, > all accesses to ob_refcnt and ob_type MUST cast the object pointer > to PyObject* (unless the pointer is already known to have that > type), and SHOULD use the respective accessor macros. To simplify > access to ob_type, a macro:: > > #define Py_Type(o) (((PyObject*)o)->ob_type) > > is added. > An example of how this will change current code would be good. E.g., ``o->ob_type->tp_name`` becomes ``PyType(o)->typ_name`` or ``o->ob_base->ob_type->tp_name``. Otherwise I am all for cleaning up the codebase and thus support this PEP. -Brett From jyasskin at gmail.com Fri Apr 27 21:27:16 2007 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Fri, 27 Apr 2007 12:27:16 -0700 Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes In-Reply-To: References: Message-ID: <5d44f72f0704271227j7e94091m3b2e196e19d59e6a@mail.gmail.com> This PEP says that no ABC defines __init__ or a couple other methods. Is that a prohibition on ABCs in general, or just a comment about the ones defined here? Can properties be abstract? How would one specify it? (Is that @abstractattribute?) On the orderings: If I have a function foo(a, b) and I want to check whether they're totally ordered (perhaps sort() will use a topological sort when its argument's contents are only partially ordered.), how do I do it? "isinstance(a, TotallyOrdered) and isinstance(b, TotallyOrdered)" doesn't work since they may be in two different orders. "isinstance(a, TotallyOrdered) and type(a)==type(b)" doesn't work when you tend to inherit from concrete types. " isinstance(a, TotallyOrdered) and isinstance(b, TotallyOrdered) and (isinstance(a, type(b)) or isinstance(b, type(a))" fails when a and b are sibling subclasses of int. (I didn't think of this option; I think it was Jim Jewett's?) Searching up the hierarchy from a and b for the first concrete type, and checking if those are equal would be pretty reliable if types were singly-inherited, but with multiple inheritance, it would take more thought to get it right. Java's solution is to parametrize Comparable with the type that the subclass is comparable with. TotallyOrdered could specify an attribute on its subclasses for users to check against. On 4/26/07, Guido van Rossum wrote: > After a fair amount of pre-discussion, I'm ready for the first > official review of this PEP. The PEP is online at > http://www.python.org/dev/peps/pep-3119/ . Here's a summary of open > issues on which I could use more help (more details in the full text > of the PEP below): > > - Where should PartiallyOrdered and TotallyOrdered live? > > - Should we support comparison of different concrete set types? > - Ditto for mapping types? > - Ditto for sequence types? > - Should Sequence derive from TotallyOrdered? > > - Should ComposableSet.__or__ and friends be abstract or concrete? > - If concrete, how should they create the result? > > - Do we need a non-composable hashable set type? > - Ditto for a non-composable mutable set type? > > - Should we require that the iteration order for keys, values and > items of a mapping are always consistent? > > - Which standard methods should sequences have? > > Of course, feel free to discuss any issues not marked as "open" as well. > > Full text of the PEP: > > PEP: 3119 > Title: Introducing Abstract Base Classes > Version: $Revision: 54986 $ > Last-Modified: $Date: 2007-04-26 11:24:07 -0700 (Thu, 26 Apr 2007) $ > Author: Guido van Rossum , Talin > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 18-Apr-2007 > Post-History: 26-Apr-2007 > > > Abstract > ======== > > This is a proposal to add Abstract Base Class (ABC) support to Python > 3000. It proposes: > > * An "ABC support framework" which defines a built-in decorator that > can be used to define abstract methods. A class containing an > abstract method that isn't overridden cannot be instantiated. > > * Specific ABCs for containers and iterators, to be added to the > collections module. > > Much of the thinking that went into the proposal is not about the > specific mechanism of ABCs, as contrasted with Interfaces or Generic > Functions (GFs), but about clarifying philosophical issues like "what > makes a set", "what makes a mapping" and "what makes a sequence". > > > Acknowledgements > ---------------- > > Talin wrote the Rationale below [1]_ as well as most of the section on > ABCs vs. Interfaces. For that alone he deserves co-authorship. The > rest of the PEP uses "I" referring to the first author. > > > Rationale > ========= > > In the domain of object-oriented programming, the usage patterns for > interacting with an object can be divided into two basic categories, > which are 'invocation' and 'inspection'. > > Invocation means interacting with an object by invoking its methods. > Usually this is combined with polymorphism, so that invoking a given > method may run different code depending on the type of an object. > > Inspection means the ability for external code (outside of the > object's methods) to examine the type or properties of that object, > and make decisions on how to treat that object based on that > information. > > Both usage patterns serve the same general end, which is to be able to > support the processing of diverse and potentially novel objects in a > uniform way, but at the same time allowing processing decisions to be > customized for each different type of object. > > In classical OOP theory, invocation is the preferred usage pattern, > and inspection is actively discouraged, being considered a relic of an > earlier, procedural programming style. However, in practice this view > is simply too dogmatic and inflexible, and leads to a kind of design > rigidity that is very much at odds with the dynamic nature of a > language like Python. > > In particular, there is often a need to process objects in a way that > wasn't anticipated by the creator of the object class. It is not > always the best solution to build in to every object methods that > satisfy the needs of every possible user of that object. Moreover, > there are many powerful dispatch philosophies that are in direct > contrast to the classic OOP requirement of behavior being strictly > encapsulated within an object, examples being rule or pattern-match > driven logic. > > On the the other hand, one of the criticisms of inspection by classic > OOP theorists is the lack of formalisms and the ad hoc nature of what > is being inspected. In a language such as Python, in which almost any > aspect of an object can be reflected and directly accessed by external > code, there are many different ways to test whether an object conforms > to a particular protocol or not. For example, if asking 'is this > object a mutable sequence container?', one can look for a base class > of 'list', or one can look for a method named '__getitem__'. But note > that although these tests may seem obvious, neither of them are > correct, as one generates false negatives, and the other false > positives. > > The generally agreed-upon remedy is to standardize the tests, and > group them into a formal arrangement. This is most easily done by > associating with each class a set of standard testable properties, > either via the inheritance mechanism or some other means. Each test > carries with it a set of promises: it contains a promise about the > general behavior of the class, and a promise as to what other class > methods will be available. > > This PEP proposes a particular strategy for organizing these tests > known as Abstract Base Classes, or ABC. ABCs are simply Python > classes that are added into an object's inheritance tree to signal > certain features of that object to an external inspector. Tests are > done using isinstance(), and the presence of a particular ABC means > that the test has passed. > > In addition, the ABCs define a minimal set of methods that establish > the characteristic behavior of the type. Code that discriminates > objects based on their ABC type can trust that those methods will > always be present. Each of these methods are accompanied by an > generalized abstract semantic definition that is described in the > documentation for the ABC. These standard semantic definitions are > not enforced, but are strongly recommended. > > Like all other things in Python, these promises are in the nature of a > gentlemen's agreement, which in this case means that while the > language does enforce some of the promises made in the ABC, it is up > to the implementer of the concrete class to insure that the remaining > ones are kept. > > > Specification > ============= > > The specification follows the categories listed in the abstract: > > * An "ABC support framework" which defines a built-in decorator that > make it easy to define ABCs, and mechanisms to support it. > > * Specific ABCs for containers and iterators, to be added to the > collections module. > > > ABC Support Framework > --------------------- > > We define a new built-in decorator, ``@abstractmethod``, to be used to > declare abstract methods. A class containing at least one method > declared with this decorator that hasn't been overridden yet cannot be > instantiated. Such a methods may be called from the overriding method > in the subclass (using ``super`` or direct invocation). For example:: > > class A: > @abstractmethod > def foo(self): pass > > A() # raises TypeError > > class B(A): > pass > > B() # raises TypeError > > class C(A): > def foo(self): print(42) > > C() # works > > **Note:** The ``@abstractmethod`` decorator should only be used inside > a class body. Dynamically adding abstract methods to a class, or > attempting to modify the abstraction status of a method or class once > it is created, are not supported. > > **Implementation:** The ``@abstractmethod`` decorator sets the > function attribute ``__isabstractmethod__`` to the value ``True``. > The ``type.__new__`` method computes the type attribute > ``__abstractmethods__`` as the set of all method names that have an > ``__isabstractmethod__`` attribute whose value is true. It does this > by combining the ``__abstractmethods__`` attributes of the base > classes, adding the names of all methods in the new class dict that > have a true ``__isabstractmethod__`` attribute, and removing the names > of all methods in the new class dict that don't have a true > ``__isabstractmethod__`` attribute. If the resulting > ``__abstractmethods__`` set is non-empty, the class is considered > abstract, and attempts to instantiate it will raise ``TypeError``. > (CPython can uses an internal flag ``Py_TPFLAGS_ABSTRACT`` to speed up > this check [6]_.) > > **Discussion:** Unlike C++ or Java, abstract methods as defined here > may have an implementation. This implementation can be called via the > ``super`` mechanism from the class that overrides it. This could be > useful as an end-point for a super-call in framework using a > cooperative multiple-inheritance [7]_, [8]_. > > > ABCs for Containers and Iterators > --------------------------------- > > The ``collections`` module will define ABCs necessary and sufficient > to work with sets, mappings, sequences, and some helper types such as > iterators and dictionary views. > > The ABCs provide implementations of their abstract methods that are > technically valid but fairly useless; e.g. ``__hash__`` returns 0, and > ``__iter__`` returns an empty iterator. In general, the abstract > methods represent the behavior of an empty container of the indicated > type. > > Some ABCs also provide concrete (i.e. non-abstract) methods; for > example, the ``Iterator`` class has an ``__iter__`` method returning > itself, fulfilling an important invariant of iterators (which in > Python 2 has to be implemented anew by each iterator class). > > No ABCs override ``__init__``, ``__new__``, ``__str__`` or > ``__repr__``. Defining a standard constructor signature would > unnecessarily constrain custom container types, for example Patricia > trees or gdbm files. Defining a specific string representation for a > collection is similarly left up to individual implementations. > > > Ordering ABCs > ''''''''''''' > > These ABCs are closer to ``object`` in the ABC hierarchy. > > ``PartiallyOrdered`` > This ABC defines the 4 inequality operations ``<``, ``<=``, ``>=``, > ``>``. (Note that ``==`` and ``!=`` are defined by ``object``.) > Classes deriving from this ABC should implement a partial order > as defined in mathematics. [9]_ > > ``TotallyOrdered`` > This ABC derives from ``PartiallyOrdered``. It adds no new > operations but implies a promise of stronger invariants. > Classes deriving from this ABC should implement a total order > as defined in mathematics. [10]_ > > **Open issues:** Where should these live? The ``collections`` module > doesn't seem right, but making them built-ins seems a slippery slope > too. > > > One Trick Ponies > '''''''''''''''' > > These abstract classes represent single methods like ``__iter__`` or > ``__len__``. > > ``Hashable`` > The base class for classes defining ``__hash__``. The > ``__hash__`` method should return an ``Integer`` (see "Numbers" > below). The abstract ``__hash__`` method always returns 0, which > is a valid (albeit inefficient) implementation. **Invariant:** If > classes ``C1`` and ``C2`` both derive from ``Hashable``, the > condition ``o1 == o2`` must imply ``hash(o1) == hash(o2)`` for all > instances ``o1`` of ``C1`` and all instances ``o2`` of ``C2``. > IOW, two objects shouldn't compare equal but have different hash > values. > > Another constraint is that hashable objects, once created, should > never change their value (as compared by ``==``) or their hash > value. If a class cannot guarantee this, it should not derive > from ``Hashable``; if it cannot guarantee this for certain > instances only, ``__hash__`` for those instances should raise a > ``TypeError`` exception. > > **Note:** being an instance of this class does not imply that an > object is immutable; e.g. a tuple containing a list as a member is > not immutable; its ``__hash__`` method raises ``TypeError``. > > ``Iterable`` > The base class for classes defining ``__iter__``. The > ``__iter__`` method should always return an instance of > ``Iterator`` (see below). The abstract ``__iter__`` method > returns an empty iterator. > > ``Iterator`` > The base class for classes defining ``__next__``. This derives > from ``Iterable``. The abstract ``__next__`` method raises > ``StopIteration``. The concrete ``__iter__`` method returns > ``self``. > > ``Sized`` > The base class for classes defining ``__len__``. The ``__len__`` > method should return an ``Integer`` (see "Numbers" below) >= 0. > The abstract ``__len__`` method returns 0. **Invariant:** If a > class ``C`` derives from ``Sized`` as well as from ``Iterable``, > the invariant ``sum(1 for x in o) == len(o)`` should hold for any > instance ``o`` of ``C``. > > ``Container`` > The base class for classes defining ``__contains__``. The > ``__contains__`` method should return a ``bool``. The abstract > ``__contains__`` method returns ``False``. **Invariant:** If a > class ``C`` derives from ``Container`` as well as from > ``Iterable``, then ``(x in o for x in o)`` should be a generator > yielding only True values for any instance ``o`` of ``C``. > > **Note:** strictly speaking, there are three variants of this method's > semantics. The first one is for sets and mappings, which is fast: > O(1) or O(log N). The second one is for membership checking on > sequences, which is slow: O(N). The third one is for subsequence > checking on (character or byte) strings, which is also slow: O(N). > Would it make sense to distinguish these? The signature of the > third variant is different, since it takes a sequence (typically > of the same type as the method's target) intead of an element. > For now, I'm using the same type for all three. This means that > is is possible for ``x in o`` to be True even though ``x`` is > never yielded by ``iter(o)``. A suggested name for the third form > is ``Searchable``. > > > Sets > '''' > > These abstract classes represent various stages of "set-ness". The > most fundamental set operation is the membership test, written as ``x > in s`` and implemented by ``s.__contains__(x)``. This is already > taken care of by the `Container`` class defined above. Therefore, we > define a set as a sized, iterable container for which certain > invariants from mathematical set theory hold. > > The built-in type ``set`` derives from ``MutableSet``. The built-in > type ``frozenset`` derives from ``HashableSet``. > > You might wonder why we require a set to be sized -- surely certain > infinite sets can be represented just fine in Python. For example, > the set of even integers could be defined like this:: > > class EvenIntegers(Container): > def __contains__(self, x): > return x % 2 == 0 > > However, such sets have rather limited practical value, and deciding > whether one such set is a subset of another would be difficult in > general without using a symbolic algebra package. So I consider this > out of the scope of a pragmatic proposal like this. > > ``Set`` > This is a sized, iterable, partially ordered container, i.e. a > subclass of ``Sized``, ``Iterable``, ``Container`` and > ``PartiallyOrdered``. Not every subset of those three classes is > a set though! Sets have the additional invariant that each > element occurs only once (as can be determined by iteration), and > in addition sets define concrete operators that implement the > inequality operations as subclass/superclass tests. In general, > the invariants for finite sets in mathematics hold. [11]_ > > Sets with different implementations can be compared safely, > (usually) efficiently and correctly using the mathematical > definitions of the subclass/superclass operations for finite sets. > The ordering operations have concrete implementations; subclasses > may override these for speed but should maintain the semantics. > Because ``Set`` derives from ``Sized``, ``__eq__`` may take a > shortcut and returns ``False`` immediately if two sets of unequal > length are compared. Similarly, ``__le__`` may return ``False`` > immediately if the first set has more members than the second set. > Note that set inclusion implements only a partial ordering; > e.g. ``{1, 2}`` and ``{1, 3}`` are not ordered (all three of > ``<``, ``==`` and ``>`` return ``False`` for these arguments). > Sets cannot be ordered relative to mappings or sequences, but they > can be compared to those for equality (and then they always > compare unequal). > > **Note:** the ``issubset`` and ``issuperset`` methods found on the > set type in Python 2 are not supported, as these are mostly just > aliases for ``__le__`` and ``__ge__``. > > **Open issues:** should we define comparison of instances of > different concrete set types this way? > > ``ComposableSet`` > This is a subclass of ``Set`` that defines abstract operators to > compute union, intersection, symmetric and asymmetric difference, > respectively ``__or__``, ``__and__``, ``__xor__`` and ``__sub__``. > These operators should return instances of ``ComposableSet``. The > abstract implementations return no meaningful values but raise > ``NotImplementedError``; this is because any generic > implementation would have to create new instances but the ABCs > don't (and shouldn't, IMO) provide an API for creating new > instances. The implementations of these operators should ensure > that the results match the mathematical definition of set > composition. [11]_ > > **Open issues:** Should ``__or__`` and friends be abstract or > concrete methods? Making them abstract means that every > ComposableSet implementation must reimplement all of them. But > making them concrete begs the question of the actual return type: > since the ABC doesn't (and IMO shouldn't) define the constructor > signature for subclasses, the concrete implementations in the ABC > don't have an API to construct a new instance given an iterable. > Perhaps the right choice is to have a static concrete factory > function ``fromiterable`` which takes an iterable and returns > a ``ComposableSet`` instance. Subclasses can override this and > benefit from the default implementations of ``__or__`` etc.; or > they can override ``__or__`` if they want to. > > ``HashableSet`` > This is a subclass of both ``ComposableSet`` and ``Hashable``. It > implements a concrete ``__hash__`` method that subclasses should > not override; or if they do, the subclass should compute the same > hash value. This is so that sets with different implementations > still hash to the same value, so they can be used interchangeably > as dictionary keys. (A similar constraint exists on the hash > values for different types of numbers and strings.) > > **Open issues:** Spell out the hash algorithm. Should there be > another ABC that derives from Set and Hashable, but not from > Composable? > > ``MutableSet`` > This is a subclass of ``ComposableSet`` implementing additional > operations to add and remove elements. The supported methods have > the semantics known from the ``set`` type in Python 2 (except > for ``discard``, which is modeled after Java): > > ``.add(x)`` > Abstract method returning a ``bool`` that adds the element > ``x`` if it isn't already in the set. It should return > ``True`` if ``x`` was added, ``False`` if it was already > there. The abstract implementation raises > ``NotImplementedError``. > > ``.discard(x)`` > Abstract method returning a ``bool`` that removes the element > ``x`` if present. It should return ``True`` if the element > was present and ``False`` if it wasn't. The abstract > implementation raises ``NotImplementedError``. > > ``.pop()`` > Concrete method that removes an arbitrary item. If the set is > empty, it raises ``KeyError``. The default implementation > removes the first item returned by the set's iterator. > > ``.toggle(x)`` > Concrete method returning a ``bool`` that adds x to the set if > it wasn't there, but removes it if it was there. It should > return ``True`` if ``x`` was added, ``False`` if it was > removed. > > ``.clear()`` > Concrete method that empties the set. The default > implementation repeatedly calls ``self.pop()`` until > ``KeyError`` is caught. (**Note:** this is likely much slower > than simply creating a new set, even if an implementation > overrides it with a faster approach; but in some cases object > identity is important.) > > This also supports the in-place mutating operations ``|=``, > ``&=``, ``^=``, ``-=``. These are concrete methods whose right > operand can be an arbitrary ``Iterable``, except for ``&=``, whose > right operand must be a ``Container``. This ABC does not support > the named methods present on the built-in concrete ``set`` type > that perform (almost) the same operations. > > > Mappings > '''''''' > > These abstract classes represent various stages of mapping-ness. The > ``Mapping`` class represents the most common read-only mapping API. > However, code *accepting* a mapping is encouraged to check for the > ``BasicMapping`` ABC when iteration is not used. This allows for > certain "black-box" implementations that can look up values by key but > don't provide a convenient iteration API. A hypothetical example > would be an interface to a hierarchical filesystem, where keys are > pathnames relative to some root directory. Iterating over all > pathnames would presumably take forever, as would counting the number > of valid pathnames. > > The built-in type ``dict`` derives from ``MutableMapping``. > > ``BasicMapping`` > A subclass of ``Container`` defining the following methods: > > ``.__getitem__(key)`` > Abstract method that returns the value corresponding to > ``key``, or raises ``KeyError``. The implementation always > raises ``KeyError``. > > ``.get(key, default=None)`` > Concrete method returning ``self[key]`` if this does not raise > ``KeyError``, and the ``default`` value if it does. > > ``.__contains__()`` > Concrete method returning ``True`` if ``self[key]`` does not > raise ``KeyError``, and ``False`` if it does. > > ``Mapping`` > A subclass of ``BasicMapping``, ``Iterable`` and ``Sized``. The > keys of a mapping naturally form a set. The (key, value) pairs > are also referred to as items. The items also form a set. > Methods: > > ``__len__`` > Abstract method returning the length of the key set. > > ``__iter__`` > Abstract method returning each key in the key set exactly once. > > ``__eq__`` > Concrete method for comparing mappings. Two mappings, even > with different implementations, can be compared for equality, > and are considered equal if and only iff their item sets are > equal. **Open issues:** should we define comparison of > instances of different concrete mapping types this way? > > ``keys`` > Concrete method returning the key set as a ``Set``. The > default concrete implementation returns a "view" on the key > set (meaning if the underlying mapping is modified, the view's > value changes correspondingly); subclasses are not required to > return a view but they should return a ``Set``. > > ``items`` > Concrete method returning the items as a ``Set``. The default > concrete implementation returns a "view" on the item set; > subclasses are not required to return a view but they should > return a ``Set``. > > ``values`` > Concrete method returning the values as a sized, iterable > container (not a set!). The default concrete implementation > returns a "view" on the values of the mapping; subclasses are > not required to return a view but they should return a sized, > iterable container. > > The following invariant should hold for any mapping ``m``:: > > set(m.items()) == set(zip(m.keys(), m.values())) > > i.e. iterating over the keys and the values in parallel should > return *corresponding* keys and values. **Open issues:** Should > this always be required? How about the stronger invariant using > ``list()`` instead of ``set()``? > > ``HashableMapping`` > A subclass of ``Mapping`` and ``Hashable``. The values should be > instances of ``Hashable``. The concrete ``__hash__`` method > should be equal to ``hash(m.items())``. > > ``MutableMapping`` > A subclass of ``Mapping`` that also implements some standard > mutating methods. Abstract methods include ``__setitem__``, > ``__delitem__``. Concrete methods include ``pop``, ``popitem``, > ``clear``, ``update``. **Note:** ``setdefault`` is *not* included. > **Open issues:** Write out the specs for the methods. > > > Sequences > ''''''''' > > These abstract classes represent various stages of sequence-ness. > > The built-in ``list`` and ``bytes`` types derive from > ``MutableSequence``. The built-in ``tuple`` and ``str`` types derive > from ``HashableSequence``. > > ``Sequence`` > A subclass of ``Iterable``, ``Sized``, ``Container``. It > defines a new abstract method ``__getitem__`` that has a somewhat > complicated signature: when called with an integer, it returns an > element of the sequence or raises ``IndexError``; when called with > a ``slice`` object, it returns another ``Sequence``. The concrete > ``__iter__`` method iterates over the elements using > ``__getitem__`` with integer arguments 0, 1, and so on, until > ``IndexError`` is raised. The length should be equal to the > number of values returned by the iterator. > > **Open issues:** Other candidate methods, which can all have > default concrete implementations that only depend on ``__len__`` > and ``__getitem__`` with an integer argument: __reversed__, index, > count, __add__, __mul__, __eq__, __lt__, __le__. > > ``HashableSequence`` > A subclass of ``Sequence`` and ``Hashable``. The concrete > ``__hash__`` method should implements the hashing algorithms used > by tuples in Python 2. > > ``MutableSequence`` > A subclass of ``Sequence`` adding some standard mutating methods. > Abstract mutating methods: ``__setitem__`` (for integer indices as > well as slices), ``__delitem__`` (ditto), ``insert``, ``append``, > ``reverse``. Concrete mutating methods: ``extend``, ``pop``, > ``remove``. Concrete mutating operators: ``+=``, ``*=`` (these > mutate the object in place). **Note:** this does not define > ``sort()`` -- that is only required to exist on genuine ``list`` > instances. > > **Open issues:** If all the elements of a sequence are totally > ordered, the sequence itself can be totally ordered with respect to > other sequences containing corresponding items of the same type. > Should we reflect this by making ``Sequence`` derive from > ``TotallyOrdered``? Or ``Partiallyordered``? Also, we could easily > define comparison of sequences of different types, so that e.g. > ``(1, 2, 3) == [1, 2, 3]`` and ``(1, 2) < [1, 2, 3]``. Should we? > (It might imply ``["a", "b"] == "ab"`` and ``[1, 2] == b"\1\2"``.) > > > Strings > ------- > > Python 3000 has two built-in string types: byte strings (``bytes``), > deriving from ``MutableSequence``, and (Unicode) character strings > (``str``), deriving from ``HashableSequence``. They also derive from > ``TotallyOrdered``. If we were to introduce ``Searchable``, they > would also derive from that. > > **Open issues:** define the base interfaces for these so alternative > implementations and subclasses know what they are in for. This may be > the subject of a new PEP or PEPs (PEP 358 should be co-opted for the > ``bytes`` type). > > > Numbers > ------- > > ABCs for numerical types are defined in PEP 3141. > > > Guidelines for Writing ABCs > --------------------------- > > Some suggestions for writing ABCs: > > * Use the ``@abstractmethod`` decorator. > > * Define abstract methods that could be useful as an end point when > called via a super chain. > > * Define concrete methods that are very simple permutations of > abstract methods (e.g. ``Mapping.get``). > > * Keep abstract classes small, one per use case instead of one per > concept. > > > ABCs vs. Alternatives > ===================== > > In this section I will attempt to compare and contrast ABCs to other > approaches that have been proposed. > > > ABCs vs. Duck Typing > -------------------- > > Does the introduction of ABCs mean the end of Duck Typing? I don't > think so. Python will not require that a class derives from > ``BasicMapping`` or ``Sequence`` when it defines a ``__getitem__`` > method, nor will the ``x[y]`` syntax require that ``x`` is an instance > of either ABC. You will still be able to assign any "file-like" > object to ``sys.stdout``, as long as it has a ``write`` method. > > Of course, there will be some carrots to encourage users to derive > from the appropriate base classes; these vary from default > implementations for certain functionality to an improved ability to > distinguish between mappings and sequences. But there are no sticks. > If ``hasattr(x, __len__)`` works for you, great! ABCs are intended to > solve problems that don't have a good solution at all in Python 2, > such as distinguishing between mappings and sequences. > > > ABCs vs. Generic Functions > -------------------------- > > ABCs are compatible with Generic Functions (GFs). For example, my own > Generic Functions implementation [4]_ uses the classes (types) of the > arguments as the dispatch key, allowing derived classes to override > base classes. Since (from Python's perspective) ABCs are quite > ordinary classes, using an ABC in the default implementation for a GF > can be quite appropriate. For example, if I have an overloaded > ``prettyprint`` function, it would make total sense to define > pretty-printing of sets like this:: > > @prettyprint.register(Set) > def pp_set(s): > return "{" + ... + "}" # Details left as an exercise > > and implementations for specific subclasses of Set could be added > easily. > > I believe ABCs also won't present any problems for RuleDispatch, > Phillip Eby's GF implementation in PEAK [5]_. > > Of course, GF proponents might claim that GFs (and concrete, or > implementation, classes) are all you need. But even they will not > deny the usefulness of inheritance; and one can easily consider the > ABCs proposed in this PEP as optional implementation base classes; > there is no requirement that all user-defined mappings derive from > ``BasicMapping``. > > > ABCs vs. Interfaces > ------------------- > > ABCs are not intrinsically incompatible with Interfaces, but there is > considerable overlap. For now, I'll leave it to proponents of > Interfaces to explain why Interfaces are better. I expect that much > of the work that went into e.g. defining the various shades of > "mapping-ness" and the nomenclature could easily be adapted for a > proposal to use Interfaces instead of ABCs. > > "Interfaces" in this context refers to a set of proposals for > additional metadata elements attached to a class which are not part of > the regular class hierarchy, but do allow for certain types of > inheritance testing. > > Such metadata would be designed, at least in some proposals, so as to > be easily mutable by an application, allowing application writers to > override the normal classification of an object. > > The drawback to this idea of attaching mutable metadata to a class is > that classes are shared state, and mutating them may lead to conflicts > of intent. Additionally, the need to override the classification of > an object can be done more cleanly using generic functions: In the > simplest case, one can define a "category membership" generic function > that simply returns False in the base implementation, and then provide > overrides that return True for any classes of interest. > > > References > ========== > > .. [1] An Introduction to ABC's, by Talin > (http://mail.python.org/pipermail/python-3000/2007-April/006614.html) > > .. [2] Incomplete implementation prototype, by GvR > (http://svn.python.org/view/sandbox/trunk/abc/) > > .. [3] Possible Python 3K Class Tree?, wiki page created by Bill Janssen > (http://wiki.python.org/moin/AbstractBaseClasses) > > .. [4] Generic Functions implementation, by GvR > (http://svn.python.org/view/sandbox/trunk/overload/) > > .. [5] Charming Python: Scaling a new PEAK, by David Mertz > (http://www-128.ibm.com/developerworks/library/l-cppeak2/) > > .. [6] Implementation of @abstractmethod > (http://python.org/sf/1706989) > > .. [7] Unifying types and classes in Python 2.2, by GvR > (http://www.python.org/download/releases/2.2.3/descrintro/) > > .. [8] Putting Metaclasses to Work: A New Dimension in Object-Oriented > Programming, by Ira R. Forman and Scott H. Danforth > (http://www.amazon.com/gp/product/0201433052) > > .. [9] Partial order, in Wikipedia > (http://en.wikipedia.org/wiki/Partial_order) > > .. [10] Total order, in Wikipedia > (http://en.wikipedia.org/wiki/Total_order) > > .. [11] Finite set, in Wikipedia > (http://en.wikipedia.org/wiki/Finite_set) > > > Copyright > ========= > > This document has been placed in the public domain. > > > > .. > Local Variables: > mode: indented-text > indent-tabs-mode: nil > sentence-end-double-space: t > fill-column: 70 > coding: utf-8 > End: > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/jyasskin%40gmail.com > -- Namast?, Jeffrey Yasskin http://jeffrey.yasskin.info/ "Religion is an improper response to the Divine." ? "Skinny Legs and All", by Tom Robbins From jyasskin at gmail.com Sat Apr 28 01:59:26 2007 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Fri, 27 Apr 2007 16:59:26 -0700 Subject: [Python-3000] string module trimming In-Reply-To: References: Message-ID: <5d44f72f0704271659u2a991d7cy6bac14bfa5f61518@mail.gmail.com> On 4/18/07, Jim Jewett wrote: > On 4/18/07, Guido van Rossum wrote: > > On 4/18/07, Jim Jewett wrote: > > > > Today, string.letters works most easily with ASCII supersets, and is > > > effectively limited to 8-bit encodings. Once everything is unicode, I > > > don't think that 8-bit restriction should apply any more. > > > But we already went over this. There are over 40K letters in Unicode. > > It simply makes no sense to have a string.letters approaching that > > size. > > Agreed. But there aren't 40K (alphabetic) letters in any particular > locale. Most individual languages will have less than 100. Here's a relevant bunch of data from the CLDR: http://www.unicode.org/cldr/data/charts/by_type/misc.exemplarCharacters.html -- Namast?, Jeffrey Yasskin From ndbecker2 at gmail.com Sat Apr 28 02:32:49 2007 From: ndbecker2 at gmail.com (Neal Becker) Date: Fri, 27 Apr 2007 20:32:49 -0400 Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD References: <46327CC9.4040901@v.loewis.de> Message-ID: "Martin v. L?wis" wrote: > I propose the PEP below for Py3k. > > Regards, > Martin > > PEP: 3122 > Title: Dropping PyObject_HEAD > Version: $Revision: 54998 $ > Last-Modified: $Date: 2007-04-27 10:31:58 +0200 (Fr, 27 Apr 2007) $ > Author: Martin v. L?wis > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 27-Apr-2007 > Python-Version: 3.0 > Post-History: > It sounds like a good thing, but what is the impact? How much code will be broken by this proposal? From jimjjewett at gmail.com Sat Apr 28 03:06:06 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 27 Apr 2007 21:06:06 -0400 Subject: [Python-3000] string module trimming In-Reply-To: <5d44f72f0704271659u2a991d7cy6bac14bfa5f61518@mail.gmail.com> References: <5d44f72f0704271659u2a991d7cy6bac14bfa5f61518@mail.gmail.com> Message-ID: On 4/27/07, Jeffrey Yasskin wrote: > On 4/18/07, Jim Jewett wrote: > > Agreed. But there aren't 40K (alphabetic) letters in any particular > > locale. Most individual languages will have less than 100. > Here's a relevant bunch of data from the CLDR: > http://www.unicode.org/cldr/data/charts/by_type/misc.exemplarCharacters.html http://www.unicode.org/Public/UNIDATA/Scripts.txt is also relevant, but I can't quite interpret it. There are 5020 "Common" code points. These are mostly non-letters, but I suppose they could appear in some langauges. Latin script has 1070 characters; most Latin-script languages use only a small fraction of them. The standard ASCII alphabet is still only 26 lower + 26 capital, but there are plenty of characters that get used in some language or other. (The largest single block is 208 letters from LATIN CAPITAL LETTER DZ WITH CARON to LATIN SMALL LETTER EZH WITH CURL) -jJ From rhamph at gmail.com Sat Apr 28 04:02:18 2007 From: rhamph at gmail.com (Adam Olsen) Date: Fri, 27 Apr 2007 20:02:18 -0600 Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes In-Reply-To: References: Message-ID: On 4/27/07, Thomas Lotze wrote: > Guido van Rossum wrote: > > > Another constraint is that hashable objects, once created, should > > never change their value (as compared by ``==``) or their hash value. > > If a class cannot guarantee this, it should not derive from > > ``Hashable``; if it cannot guarantee this for certain instances only, > > ``__hash__`` for those instances should raise a ``TypeError`` > > exception. > > Shouldn't this rather be a ValueError since it occurs not because of the > type of the object in question, but its value (while in general, there are > instances of the same type representing other values which are hashable)? >>> hash([]) Traceback (most recent call last): File "", line 1, in TypeError: list objects are unhashable Normally the exception is raised because the type is wrong. Requiring both TypeError and ValueError be caught just for this special case would be confusing. A subclass of both could be created, but that seems pedantic with no practical value. -- Adam Olsen, aka Rhamphoryncus From guido at python.org Sat Apr 28 05:31:09 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 27 Apr 2007 20:31:09 -0700 Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes In-Reply-To: References: Message-ID: On 4/27/07, Adam Olsen wrote: > On 4/27/07, Thomas Lotze wrote: > > Guido van Rossum wrote: > > > > > Another constraint is that hashable objects, once created, should > > > never change their value (as compared by ``==``) or their hash value. > > > If a class cannot guarantee this, it should not derive from > > > ``Hashable``; if it cannot guarantee this for certain instances only, > > > ``__hash__`` for those instances should raise a ``TypeError`` > > > exception. > > > > Shouldn't this rather be a ValueError since it occurs not because of the > > type of the object in question, but its value (while in general, there are > > instances of the same type representing other values which are hashable)? > > >>> hash([]) > Traceback (most recent call last): > File "", line 1, in > TypeError: list objects are unhashable > > Normally the exception is raised because the type is wrong. Requiring > both TypeError and ValueError be caught just for this special case > would be confusing. A subclass of both could be created, but that > seems pedantic with no practical value. Right. The hash of a tuple, or any other hashable container, is computed from the hashes of its elements. If any of the elements happens to be not hashable then it will raise a TypeError and that will just be passed along by the toplevel hash() call. Turning this into a ValueError would just be an error-masking accident waiting to happen. If the sub-hash() call raises another exception, that gets passed through too. (At one point in the past I considered making __hash__ a dynamic instance attribute that would be set to None when the value is unhashable, and computed dynmically. But computing whether it should be None is just about as expensive as computing the __hash__ function, so I abandoned this idea. There's little merit to LBYL hashing anyway.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Sat Apr 28 07:37:07 2007 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 28 Apr 2007 07:37:07 +0200 Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD In-Reply-To: References: <46327CC9.4040901@v.loewis.de> Message-ID: <4632DD83.2030301@v.loewis.de> > It sounds like a good thing, but what is the impact? How much code will be > broken by this proposal? Nearly all extension modules will be broken, as PyObject_HEAD goes away. However, it is likely that Py3k breaks all extension modules for a lot of other reasons, as well. Fixing this specific breakage would be easy: Just replace PyObject_HEAD with "PyObject ob_base;". How many modules then still break, I don't know. I expect access to ob_type is frequent in Foo_Check() macros; those need to be rewritten to use Py_Type(op). Martin From rasky at develer.com Sat Apr 28 14:07:55 2007 From: rasky at develer.com (Giovanni Bajo) Date: Sat, 28 Apr 2007 14:07:55 +0200 Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD In-Reply-To: <46327CC9.4040901@v.loewis.de> References: <46327CC9.4040901@v.loewis.de> Message-ID: On 28/04/2007 0.44, Martin v. L?wis wrote: > PEP: 3122 > Title: Dropping PyObject_HEAD > Version: $Revision: 54998 $ > Last-Modified: $Date: 2007-04-27 10:31:58 +0200 (Fr, 27 Apr 2007) $ > Author: Martin v. L?wis > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 27-Apr-2007 > Python-Version: 3.0 > Post-History: Strong +1 from me. This looks like the exact kind of cleanup Py3k has been thought for. It's a one-chance to modify Python's code to adhere to the C standard, we should not miss it. -- Giovanni Bajo From ironfroggy at gmail.com Sat Apr 28 15:13:29 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 28 Apr 2007 09:13:29 -0400 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: Message-ID: <76fd5acf0704280613w38c273b6xd3073d0076caa0d5@mail.gmail.com> On 4/26/07, Guido van Rossum wrote: > On 4/25/07, Jim Jewett wrote: > > The current ABC proposal is to use isinstance as the test; Jeffrey > > Yaskin's numbers PEP highlighted the weakness there with a concrete > > example. > > > > If you need to an abstraction less powerful than an existing ABC, > > you're out of luck; you can't just assert that the existing class is > > already sufficient, nor can you expect everyone else to use multiple > > annotations. > > I now have a proposal to allow overloading isinstance() and > issubclass(), by defining special (class) methods on the second > argument. See http://python.org/sf/1708353. Does this need a PEP? The > unit test shows that it can be used to support the use case described > above: > > > class ABC(type): > > def __instancecheck__(cls, inst): > """Implement isinstance(inst, cls).""" > return any(cls.__subclasscheck__(c) > for c in {type(inst), inst.__class__}) > > def __subclasscheck__(cls, sub): > """Implement issubclass(sub, cls).""" > candidates = cls.__dict__.get("__subclass__", set()) > return any(c in candidates for c in sub.mro()) > > > class Integer(metaclass=ABC): > > __subclass__ = {int} > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/ironfroggy%40gmail.com I'm just going to jump into this and voice a concern that allowing overriding of isinstance and issubclass seems like a Bad Idea. They should be trustworthy and predictable, and allowing classes to change how objects are considered instances of them seems like the wrong thing to do. I'm not saying that I don't think it should be done, but that it feels quite wrong. When I do, rarely, use isinstance or such, and I need to check isinstance(f, Foo), I know exactly what I am saying. But if Foo can change the meaning of that, can I trust it anymore? Not everything should be dynamic. I am not sure which side of the fence this falls on, but right now, in my mind, it teeters right on the fence itself. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From daniel at stutzbachenterprises.com Sat Apr 28 15:57:45 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sat, 28 Apr 2007 08:57:45 -0500 Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD In-Reply-To: <46327CC9.4040901@v.loewis.de> References: <46327CC9.4040901@v.loewis.de> Message-ID: To facilitate migration, I have a few suggestions: - add the new access macros to 2.6 (defined appropriately) - in 3.0, instead of removing PyObject_HEAD and company, redefine them appropriately, .e.g, #define PyObject_HEAD PyObject obtype; -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From guido at python.org Sat Apr 28 16:54:58 2007 From: guido at python.org (Guido van Rossum) Date: Sat, 28 Apr 2007 07:54:58 -0700 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <76fd5acf0704280613w38c273b6xd3073d0076caa0d5@mail.gmail.com> References: <76fd5acf0704280613w38c273b6xd3073d0076caa0d5@mail.gmail.com> Message-ID: On 4/28/07, Calvin Spealman wrote: > On 4/26/07, Guido van Rossum wrote: > > On 4/25/07, Jim Jewett wrote: > > > The current ABC proposal is to use isinstance as the test; Jeffrey > > > Yaskin's numbers PEP highlighted the weakness there with a concrete > > > example. > > > > > > If you need to an abstraction less powerful than an existing ABC, > > > you're out of luck; you can't just assert that the existing class is > > > already sufficient, nor can you expect everyone else to use multiple > > > annotations. > > > > I now have a proposal to allow overloading isinstance() and > > issubclass(), by defining special (class) methods on the second > > argument. See http://python.org/sf/1708353. Does this need a PEP? The > > unit test shows that it can be used to support the use case described > > above: > > > > > > class ABC(type): > > > > def __instancecheck__(cls, inst): > > """Implement isinstance(inst, cls).""" > > return any(cls.__subclasscheck__(c) > > for c in {type(inst), inst.__class__}) > > > > def __subclasscheck__(cls, sub): > > """Implement issubclass(sub, cls).""" > > candidates = cls.__dict__.get("__subclass__", set()) > > return any(c in candidates for c in sub.mro()) > > > > > > class Integer(metaclass=ABC): > > > > __subclass__ = {int} > > > > > > -- > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > _______________________________________________ > > Python-3000 mailing list > > Python-3000 at python.org > > http://mail.python.org/mailman/listinfo/python-3000 > > Unsubscribe: http://mail.python.org/mailman/options/python-3000/ironfroggy%40gmail.com > > I'm just going to jump into this and voice a concern that allowing > overriding of isinstance and issubclass seems like a Bad Idea. They > should be trustworthy and predictable, and allowing classes to change > how objects are considered instances of them seems like the wrong > thing to do. I'm not saying that I don't think it should be done, but > that it feels quite wrong. When I do, rarely, use isinstance or such, > and I need to check isinstance(f, Foo), I know exactly what I am > saying. But if Foo can change the meaning of that, can I trust it > anymore? > > Not everything should be dynamic. I am not sure which side of the > fence this falls on, but right now, in my mind, it teeters right on > the fence itself. Note though that only the second argument to either function can overload the rules. IOW if you write isinstance(x, C), there is no way that x could attempt to lie; but C could. Similar for issubclass(D, C) -- only D can change the outcome. Whether or not a particular class overload these functions ought to be part of its documentation. The builtins won't be overloading them; you will be able to trust e.g. isinstance(x, int). But it might be true that isinstance(42, Ring) if Ring overloads isinstance -- however you will have been duly warned of this possibility. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From exarkun at divmod.com Sat Apr 28 18:12:04 2007 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Sat, 28 Apr 2007 12:12:04 -0400 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: Message-ID: <20070428161204.19381.458566240.divmod.quotient.6050@ohm> On Sat, 28 Apr 2007 07:54:58 -0700, Guido van Rossum wrote: >On 4/28/07, Calvin Spealman wrote: >> On 4/26/07, Guido van Rossum wrote: >> > On 4/25/07, Jim Jewett wrote: >> > > The current ABC proposal is to use isinstance as the test; Jeffrey >> > > Yaskin's numbers PEP highlighted the weakness there with a concrete >> > > example. >> > > >> > > If you need to an abstraction less powerful than an existing ABC, >> > > you're out of luck; you can't just assert that the existing class is >> > > already sufficient, nor can you expect everyone else to use multiple >> > > annotations. >> > >> > I now have a proposal to allow overloading isinstance() and >> > issubclass(), by defining special (class) methods on the second >> > argument. See http://python.org/sf/1708353. Does this need a PEP? The >> > unit test shows that it can be used to support the use case described >> > above: >> > >> > >> > class ABC(type): >> > >> > def __instancecheck__(cls, inst): >> > """Implement isinstance(inst, cls).""" >> > return any(cls.__subclasscheck__(c) >> > for c in {type(inst), inst.__class__}) >> > >> > def __subclasscheck__(cls, sub): >> > """Implement issubclass(sub, cls).""" >> > candidates = cls.__dict__.get("__subclass__", set()) >> > return any(c in candidates for c in sub.mro()) >> > >> > >> > class Integer(metaclass=ABC): >> > >> > __subclass__ = {int} >> > >> > >> > -- >> > --Guido van Rossum (home page: http://www.python.org/~guido/) >> > _______________________________________________ >> > Python-3000 mailing list >> > Python-3000 at python.org >> > http://mail.python.org/mailman/listinfo/python-3000 >> > Unsubscribe: http://mail.python.org/mailman/options/python-3000/ironfroggy%40gmail.com >> >> I'm just going to jump into this and voice a concern that allowing >> overriding of isinstance and issubclass seems like a Bad Idea. They >> should be trustworthy and predictable, and allowing classes to change >> how objects are considered instances of them seems like the wrong >> thing to do. I'm not saying that I don't think it should be done, but >> that it feels quite wrong. When I do, rarely, use isinstance or such, >> and I need to check isinstance(f, Foo), I know exactly what I am >> saying. But if Foo can change the meaning of that, can I trust it >> anymore? >> >> Not everything should be dynamic. I am not sure which side of the >> fence this falls on, but right now, in my mind, it teeters right on >> the fence itself. > >Note though that only the second argument to either function can >overload the rules. IOW if you write isinstance(x, C), there is no way >that x could attempt to lie; but C could. Similar for issubclass(D, C) >-- only D can change the outcome. Whether or not a particular class >overload these functions ought to be part of its documentation. The >builtins won't be overloading them; you will be able to trust e.g. >isinstance(x, int). But it might be true that isinstance(42, Ring) if >Ring overloads isinstance -- however you will have been duly warned of >this possibility. > Aside from the way in which `x' can already lie: >>> class X(object): ... __class__ = property(lambda self: int) ... >>> isinstance(X(), int) True >>> Is this behavior changed/going to be changed in Py3k? Jean-Paul From guido at python.org Sat Apr 28 18:17:38 2007 From: guido at python.org (Guido van Rossum) Date: Sat, 28 Apr 2007 09:17:38 -0700 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <20070428161204.19381.458566240.divmod.quotient.6050@ohm> References: <20070428161204.19381.458566240.divmod.quotient.6050@ohm> Message-ID: On 4/28/07, Jean-Paul Calderone wrote: > Aside from the way in which `x' can already lie: > > >>> class X(object): > ... __class__ = property(lambda self: int) > ... > >>> isinstance(X(), int) > True > >>> > > Is this behavior changed/going to be changed in Py3k? I'm not particularly enamored with it, but I believe it once served a purpose for Zope. Does anyone know if it is still needed? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nicko at nicko.org Sat Apr 28 18:14:41 2007 From: nicko at nicko.org (Nicko van Someren) Date: Sat, 28 Apr 2007 17:14:41 +0100 Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function CurrentlyBeing Defined (this) In-Reply-To: References: Message-ID: <5AE9FDD9-A78D-4C44-80A6-93515819BC84@nicko.org> On 23 Apr 2007, at 05:17, Terry Reedy wrote: > "Jim Jewett" wrote in > message > news:fb6fbf560704222005le4798a4j5daa5e71e644f069 at mail.gmail.com... > | Functions (including methods) often want access to themselves, > | usually for a private storage location. > > Or for true recursion. At present, a function has to 'hope' that its > definition name is not rebound. Indeed. It's much clearing than having to write: Y = lambda f: partial(f,f) fac = Y(lambda f, n: n*f(f,n-1) if n>0 else 1) :-) Seriously though, while I have to confess never, ever having found rebinding of function names a limiting factor in writing recursive code I'm still in favour of the proposal. Personally I think function-local storage is a more likely use. +1 Nicko From nas at arctrix.com Sat Apr 28 20:02:56 2007 From: nas at arctrix.com (Neil Schemenauer) Date: Sat, 28 Apr 2007 18:02:56 +0000 (UTC) Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD References: <46327CC9.4040901@v.loewis.de> Message-ID: Martin v. L?wis wrote: > I propose the PEP below for Py3k. I think it's a good idea. It would be nice if the PEP included something about forwards compatibility. Ideally, there should be a way to write a module that works with both 2.x and 3.x (e.g. using preprocessor macros). Obviously the 3.x design should not be compromised for this sake though. Neil From ironfroggy at gmail.com Sat Apr 28 20:27:30 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 28 Apr 2007 14:27:30 -0400 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <20070428161204.19381.458566240.divmod.quotient.6050@ohm> Message-ID: <76fd5acf0704281127v74294545r6bf9573d3c83f21c@mail.gmail.com> On 4/28/07, Guido van Rossum wrote: > On 4/28/07, Jean-Paul Calderone wrote: > > Aside from the way in which `x' can already lie: > > > > >>> class X(object): > > ... __class__ = property(lambda self: int) > > ... > > >>> isinstance(X(), int) > > True > > >>> > > > > Is this behavior changed/going to be changed in Py3k? > > I'm not particularly enamored with it, but I believe it once served a > purpose for Zope. Does anyone know if it is still needed? > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) Consider this example: >>> class X(object): pass >>> class Y(object): a = 10 >>> x = X() >>> x.__class__ = Y >>> x.a 10 >>> isinstance(x, Y) True Now, I'm not saying we must keep this behavior, but it does have its merits. In particular, the cases of type-changing objects, which I'm not convinced always have a place, but thats not the point. The point I want to make is that x is not lying about its type. It IS a Y, because __class__ determines its type. The breakage may not be that x could lie and say its an int, but that it actually can't its a deception of the class, not the type, because you can't assign x.__class__ = int, for example. Either the behavior should work for any type, thus allowing heap types to be assigned to __class__ attributes, changing the type of objects (but, obviously immutables would disallow __class__ assignment). Or, an objects type should not be determined simply by the value of attributes, __class__, at all. Does that make sense? -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From guido at python.org Sat Apr 28 20:57:37 2007 From: guido at python.org (Guido van Rossum) Date: Sat, 28 Apr 2007 11:57:37 -0700 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <76fd5acf0704281127v74294545r6bf9573d3c83f21c@mail.gmail.com> References: <20070428161204.19381.458566240.divmod.quotient.6050@ohm> <76fd5acf0704281127v74294545r6bf9573d3c83f21c@mail.gmail.com> Message-ID: On 4/28/07, Calvin Spealman wrote: > On 4/28/07, Guido van Rossum wrote: > > On 4/28/07, Jean-Paul Calderone wrote: > > > Aside from the way in which `x' can already lie: > > > > > > >>> class X(object): > > > ... __class__ = property(lambda self: int) > > > ... > > > >>> isinstance(X(), int) > > > True > > > >>> > > > > > > Is this behavior changed/going to be changed in Py3k? > > > > I'm not particularly enamored with it, but I believe it once served a > > purpose for Zope. Does anyone know if it is still needed? > > > > -- > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > Consider this example: > > >>> class X(object): > pass > >>> class Y(object): > a = 10 > >>> x = X() > >>> x.__class__ = Y > >>> x.a > 10 > >>> isinstance(x, Y) > True > > Now, I'm not saying we must keep this behavior, but it does have its > merits. In particular, the cases of type-changing objects, which I'm > not convinced always have a place, but thats not the point. The point > I want to make is that x is not lying about its type. It IS a Y, > because __class__ determines its type. The breakage may not be that x > could lie and say its an int, but that it actually can't its a > deception of the class, not the type, because you can't assign > x.__class__ = int, for example. Either the behavior should work for > any type, thus allowing heap types to be assigned to __class__ > attributes, changing the type of objects (but, obviously immutables > would disallow __class__ assignment). Or, an objects type should not > be determined simply by the value of attributes, __class__, at all. > > Does that make sense? Alas not much. Your code example is about *actually* changing the type of an object, which is a completely different issue, and is constrained by the object lay-out (try changing the class where both classes use different __slots__). Overriding isinstance is about what classes *claim* that the object is an instance of them, not for the purpose of changing the object's behavior or representation, but purely for the sake of API introspection. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pedronis at openendsystems.com Sat Apr 28 21:00:18 2007 From: pedronis at openendsystems.com (Samuele Pedroni) Date: Sat, 28 Apr 2007 21:00:18 +0200 Subject: [Python-3000] *View in abc.py not inheriting Iterable Re: PEP 3119 - Introducing Abstract Base Classes In-Reply-To: References: Message-ID: <463399C2.3090709@openendsystems.com> Shouldn't the various *View in abc.py be Iterables (inherit from Iterable) ? From guido at python.org Sat Apr 28 21:17:29 2007 From: guido at python.org (Guido van Rossum) Date: Sat, 28 Apr 2007 12:17:29 -0700 Subject: [Python-3000] *View in abc.py not inheriting Iterable Re: PEP 3119 - Introducing Abstract Base Classes In-Reply-To: <463399C2.3090709@openendsystems.com> References: <463399C2.3090709@openendsystems.com> Message-ID: On 4/28/07, Samuele Pedroni wrote: > Shouldn't the various *View in abc.py be Iterables (inherit from Iterable) ? Oops, it looks like this is a mess. There are two classes _MappingView. Ignore them for the sake of the PEP. I'll clean them up later. We probably need fewer mapping classes, nor more... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pedronis at openendsystems.com Sat Apr 28 21:28:59 2007 From: pedronis at openendsystems.com (Samuele Pedroni) Date: Sat, 28 Apr 2007 21:28:59 +0200 Subject: [Python-3000] isinstance(., Iterable) vs lookup('__iter__') != None Re: *View in abc.py not inheriting Iterable Re: PEP 3119 - Introducing Abstract Base Classes In-Reply-To: References: <463399C2.3090709@openendsystems.com> Message-ID: <4633A07B.8080706@openendsystems.com> Guido van Rossum wrote: > On 4/28/07, Samuele Pedroni wrote: >> Shouldn't the various *View in abc.py be Iterables (inherit from >> Iterable) ? > > Oops, it looks like this is a mess. There are two classes > _MappingView. Ignore them for the sake of the PEP. I'll clean them up > later. We probably need fewer mapping classes, nor more... > It seems that something that defines __iter__ should also subclass Iterable, in this kind of situation where the special method is unique enough it seems quite repetitive to have to do both. It seems easy to forget to subclass. Shouldn't now that there are isinstance hooks Iterable be defined again in terms of just checking for the presence of '__iter__' lookup-wise? I suppose there are some other similar cases, of course in other situations the ABC carry an intention that goes beyond the presence of methods. From guido at python.org Sat Apr 28 22:30:06 2007 From: guido at python.org (Guido van Rossum) Date: Sat, 28 Apr 2007 13:30:06 -0700 Subject: [Python-3000] isinstance(., Iterable) vs lookup('__iter__') != None Re: *View in abc.py not inheriting Iterable Re: PEP 3119 - Introducing Abstract Base Classes In-Reply-To: <4633A07B.8080706@openendsystems.com> References: <463399C2.3090709@openendsystems.com> <4633A07B.8080706@openendsystems.com> Message-ID: On 4/28/07, Samuele Pedroni wrote: > Guido van Rossum wrote: > > On 4/28/07, Samuele Pedroni wrote: > >> Shouldn't the various *View in abc.py be Iterables (inherit from > >> Iterable) ? > > > > Oops, it looks like this is a mess. There are two classes > > _MappingView. Ignore them for the sake of the PEP. I'll clean them up > > later. We probably need fewer mapping classes, nor more... > > > It seems that something that defines __iter__ should also subclass Iterable, > in this kind of situation where the special method is unique enough it seems > quite repetitive to have to do both. It seems easy to forget to subclass. > > Shouldn't now that there are isinstance hooks Iterable be defined again > in terms > of just checking for the presence of '__iter__' lookup-wise? I suppose > there are some other similar > cases, of course in other situations the ABC carry an intention that > goes beyond the presence > of methods. Hm, this certainly sounds like an interesting avenue. So Iterable could override isinstance and issubclass that just check for the presence of __iter__, and Sized could check for __len__, Hashable could check for __hash__, Container could check for __contains__. I think the others convey more meaning though; not everything iterable sized container is a set (counter-examples are lists and bags). So it would affect the "one trick ponies" section only. (Should Iterator just check for __next__ and leave __iter__ up to the imagination?) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pedronis at openendsystems.com Sat Apr 28 23:01:06 2007 From: pedronis at openendsystems.com (Samuele Pedroni) Date: Sat, 28 Apr 2007 23:01:06 +0200 Subject: [Python-3000] isinstance(., Iterable) vs lookup('__iter__') != None Re: *View in abc.py not inheriting Iterable Re: PEP 3119 - Introducing Abstract Base Classes In-Reply-To: References: <463399C2.3090709@openendsystems.com> <4633A07B.8080706@openendsystems.com> Message-ID: <4633B612.1020405@openendsystems.com> Guido van Rossum wrote: > On 4/28/07, Samuele Pedroni wrote: >> Guido van Rossum wrote: >> > On 4/28/07, Samuele Pedroni wrote: >> >> Shouldn't the various *View in abc.py be Iterables (inherit from >> >> Iterable) ? >> > >> > Oops, it looks like this is a mess. There are two classes >> > _MappingView. Ignore them for the sake of the PEP. I'll clean them up >> > later. We probably need fewer mapping classes, nor more... >> > >> It seems that something that defines __iter__ should also subclass >> Iterable, >> in this kind of situation where the special method is unique enough >> it seems >> quite repetitive to have to do both. It seems easy to forget to >> subclass. >> >> Shouldn't now that there are isinstance hooks Iterable be defined again >> in terms >> of just checking for the presence of '__iter__' lookup-wise? I suppose >> there are some other similar >> cases, of course in other situations the ABC carry an intention that >> goes beyond the presence >> of methods. > > Hm, this certainly sounds like an interesting avenue. So Iterable > could override isinstance and issubclass that just check for the > presence of __iter__, and Sized could check for __len__, Hashable > could check for __hash__, Container could check for __contains__. I > think the others convey more meaning though; not everything iterable > sized container is a set (counter-examples are lists and bags). So it > would affect the "one trick ponies" section only. yes, those are the one I had in mind. I think that designs that avoid to have to repeat oneself unless the repetion really carries some meaning are better. This would make the transation from current idioms to 3.0 ABC idioms smoother and more sensible. > (Should Iterator > just check for __next__ and leave __iter__ up to the imagination?) > I think it would be a reasonable compromise, there's a trade off between encouraging to subclass Iterator to get the default __iter__ for free, vs. having to change current iterators to subclass the right thing to pass future checks. From pje at telecommunity.com Sat Apr 28 23:17:38 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 28 Apr 2007 17:17:38 -0400 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <20070428161204.19381.458566240.divmod.quotient.6050@ohm> <20070428161204.19381.458566240.divmod.quotient.6050@ohm> Message-ID: <5.1.1.6.0.20070428171439.0420b008@sparrow.telecommunity.com> At 09:17 AM 4/28/2007 -0700, Guido van Rossum wrote: >On 4/28/07, Jean-Paul Calderone wrote: > > Aside from the way in which `x' can already lie: > > > > >>> class X(object): > > ... __class__ = property(lambda self: int) > > ... > > >>> isinstance(X(), int) > > True > > >>> > > > > Is this behavior changed/going to be changed in Py3k? > >I'm not particularly enamored with it, but I believe it once served a >purpose for Zope. Does anyone know if it is still needed? AFAIK, it's still used to support proxies in Zope 3, and my own ProxyTypes library on the Cheeseshop (and some other libraries I've written that use it) depend on it. I believe Chandler uses this, too. In all cases, it's to allow proxies to appear to be the object they're proxying. I wouldn't be surprised if weakref.proxy does the same thing. From jyasskin at gmail.com Sat Apr 28 23:55:36 2007 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sat, 28 Apr 2007 14:55:36 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> Message-ID: <5d44f72f0704281455o7c3c2217pa3d599993f3f8018@mail.gmail.com> On 4/25/07, Jim Jewett wrote: > On 4/25/07, Jeffrey Yasskin wrote: > > class MonoidUnderPlus(Abstract): > > Is this useful? Just because two things are both Monoid instances > doesn't mean I can add them -- they have to be part of the same > Monoid. By the time you do > > assert isinstance(a, MonoidUnderPlus) > assert isinstance(b, MonoidUnderPlus) > assert isinstance(a, b.__class__) or isinstance(b, a.__class__) > > I'm not sure how much you've really saved. I brought this up on the PEP 3119 thread, as an issue for TotallyOrdered too. The exact syntax for checking that two objects are "in the same instance of ABC X" should be decided there. As to the benefits of MonoidUnderPlus over just hasattr(a, '__plus__'), by knowing that the operation is associative, a function may be able to speed itself up. A somewhat academic example is a SortedList type: class SortedList(MonoidUnderPlus): def __init__(self, iterable): self.contents = sorted(iterable) def __add__(lhs, rhs): return merge(lhs, rhs) Then (ignoring that sum currently requires numbers) we can define mysorted() as: def mysorted(iterable): return sum(map(SortedList, iterable)).contents With sum() implemented as a simple for loop, this is insertion sort, taking O(n^2) time. However, by noticing that + is associative on its argument, sum can be defined (assume the annotation implies the appropriate checking) roughly as: def sum(lst : SequenceOfMonoidUnderPlus): if len(lst) == 1: return lst[0] else: return sum(lst[:len(lst)//2]) + sum(lst[len(lst)//2+1:] and we've transformed the algorithm into merge sort, taking O(n log n) time. (Hm, the zero element isn't as useful here as in other languages since you can't get a type without an instance. Maybe it should go leave the interface.) This optimization is, of course, incorrect if + is not associative. > > **Open issue:** Do we want to give people a choice of which of the > > following to define, or should we pick one arbitrarily?:: > > > def __neg__(self): > > def __sub__(self, other): > Probably better to pick one Will do. -- Namast?, Jeffrey Yasskin From baptiste13 at altern.org Sun Apr 29 00:01:17 2007 From: baptiste13 at altern.org (Baptiste Carvello) Date: Sun, 29 Apr 2007 00:01:17 +0200 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <07Apr26.102627pdt."57996"@synergy1.parc.xerox.com> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <07Apr26.102627pdt."57996"@synergy1.parc.xerox.com> Message-ID: Bill Janssen a ?crit : >> Jeffrey, is there any way you can drop the top of the tree and going >> straight from Number to Complex -> Real -> Rational -> Integer? These >> are the things that everyone with high school math will know. > > I think knowledge of the concepts of group, ring, and field is > supposed to be standard knowledge for any high-school senior -- isn't > this what the "new math" was all about?. But they're pretty > fundamental to computer science; anyone trying to do serious work in > the field (that is, anyone with a reason to read the PEP) should have > a nodding acquaintance with them. > > Bill > Even knowing the maths, you still have 2 problems: 1) the number types in Python are pretty much defined by the way CPUs work, not by the math (Decimal beeing the exception), which causes some conceptual impedance mismatch. Discussing to what extend floating point numbers can be compared to theoritical real numbers is a can of worms we probably don't want to open :-) 2) In the PEP, the concepts are used *inconsistently*. Complex derives from Ring because the set of complex numbers *is a* ring. Int derives from Complex because integer are complex numbers (or, alternatively, the set of integers *is included in* the set of complex numbers). The consistent way could be to make the Complex class an instance of Ring, not a subclass. Cheers, BC From jyasskin at gmail.com Sun Apr 29 00:19:47 2007 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sat, 28 Apr 2007 15:19:47 -0700 Subject: [Python-3000] [Numpy-discussion] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <873b2n5dc7.fsf@uwo.ca> <20070427144537.T1402@tribble.ilrt.bris.ac.uk> <5d44f72f0704270758n27f3c20y8e2a33c24d8128dd@mail.gmail.com> Message-ID: <5d44f72f0704281519g316a12abl87e53d42b935deb6@mail.gmail.com> On 4/27/07, Alan Isaac wrote: > On Fri, 27 Apr 2007, Jeffrey Yasskin wrote: > > Then again, doubles aren't a group either because of this > > imprecision, and I'm suggesting claiming they're > > a subclass of that, so maybe there's room in a practical > > language to make them a subclass of the rationals too. > > Would using language from the Scheme report be useful when > discussing this? > http://www-swiss.ai.mit.edu/projects/scheme/documentation/scheme_5.html Very much so. Thanks for sending the link. I'm not going to get a chance to update the document for the next several days, so if you want to put together a patch using such language, I'd be happy to see it go in. Or I'll integrate those ideas once I have a bit more spare time. It looks like the primary ideas there are that it was worthwhile for Scheme to have the full Number:>Complex:>Real:>Rational:>Integer tower, and that it wasn't worthwhile to duplicate it entirely for the exact/inexact distinction. Thanks, Jeffrey From rhamph at gmail.com Sun Apr 29 02:06:42 2007 From: rhamph at gmail.com (Adam Olsen) Date: Sat, 28 Apr 2007 18:06:42 -0600 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <5d44f72f0704281455o7c3c2217pa3d599993f3f8018@mail.gmail.com> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <5d44f72f0704281455o7c3c2217pa3d599993f3f8018@mail.gmail.com> Message-ID: On 4/28/07, Jeffrey Yasskin wrote: > On 4/25/07, Jim Jewett wrote: > > On 4/25/07, Jeffrey Yasskin wrote: > > > class MonoidUnderPlus(Abstract): > > > > Is this useful? Just because two things are both Monoid instances > > doesn't mean I can add them -- they have to be part of the same > > Monoid. By the time you do > > > > assert isinstance(a, MonoidUnderPlus) > > assert isinstance(b, MonoidUnderPlus) > > assert isinstance(a, b.__class__) or isinstance(b, a.__class__) > > > > I'm not sure how much you've really saved. > > I brought this up on the PEP 3119 thread, as an issue for > TotallyOrdered too. The exact syntax for checking that two objects are > "in the same instance of ABC X" should be decided there. > > As to the benefits of MonoidUnderPlus over just hasattr(a, > '__plus__'), by knowing that the operation is associative, a function > may be able to speed itself up. A somewhat academic example is a > SortedList type: > class SortedList(MonoidUnderPlus): > def __init__(self, iterable): self.contents = sorted(iterable) > def __add__(lhs, rhs): return merge(lhs, rhs) > Then (ignoring that sum currently requires numbers) we can define mysorted() as: > def mysorted(iterable): return sum(map(SortedList, iterable)).contents > With sum() implemented as a simple for loop, this is insertion sort, > taking O(n^2) time. However, by noticing that + is associative on its > argument, sum can be defined (assume the annotation implies the > appropriate checking) roughly as: > def sum(lst : SequenceOfMonoidUnderPlus): > if len(lst) == 1: return lst[0] > else: return sum(lst[:len(lst)//2]) + sum(lst[len(lst)//2+1:] > and we've transformed the algorithm into merge sort, taking O(n log n) > time. (Hm, the zero element isn't as useful here as in other languages > since you can't get a type without an instance. Maybe it should go > leave the interface.) > > This optimization is, of course, incorrect if + is not associative. Unfortunately, such a dramatic performance difference will mean people will depend on it, then be surprised when it occasionally fails. Since the original O(n**2) complexity is now perceived as a failure mode, it's best to split it into an explicit method. Python does have many optimizations, but they're well hidden, and rarely if ever result in a noticeable complexity difference. -- Adam Olsen, aka Rhamphoryncus From jyasskin at gmail.com Sun Apr 29 03:38:41 2007 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sat, 28 Apr 2007 18:38:41 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> Message-ID: <5d44f72f0704281838j4e8244c4g152df6e3762aa6d4@mail.gmail.com> Thanks for the comments! On 4/26/07, Travis E. Oliphant wrote: > > Forgive my ignorance, but I'm not really sure what this PEP is trying to > do. I don't want to sound negative, I really just don't understand the > purpose. I've just never encountered a problem this that I can see how > this PEP will help me solve. It's trying to do the same thing for numbers as Guido is doing for collections in PEP 3119: let people make the useful distinctions between them for whatever purposes people need to make distinctions. It looks like the community is a lot more comfortable with overloading than static typing, so some of the distinctions I proposed are probably too fine. > That doesn't mean it's not worthwhile, I just don't understand it. > > > > > Abstract > > ======== > > > > This proposal defines a hierarchy of Abstract Base Classes (ABCs) > > [#pep3119] to represent numbers and other algebraic entities similar > > to numbers. It proposes: > > > > * A hierarchy of algebraic concepts, including monoids, groups, rings, > > and fields with successively more operators and constraints on their > > operators. This will be added as a new library module named > > "algebra". > > > > The SAGE people may be interested in this, but I doubt there will more > than a handful of users of these algebraic base classes. I expected that if someone overloads a function between integers and rationals, the numeric people would be happier if the integer overload triggered for square matrices as well, in which case we'd want to encourage people to overload on Ring vs Field instead of Integral vs FractionalReal. You'd know better whether that's actually true. If not, I guess I'd better stop claiming that they'll be useful for the numpy people. :) > > * A hierarchy of specifically numeric types, which can be converted to > > and from the native Python types. This will be added as a new > > library module named "numbers". > > > > I could see having a hierarchy of "numbers" we have one in NumPy. All > the NumPy array scalars fall under a hierarchy of types so that it is > easy to check whether or not you have signed or unsigned integers, or > inexact numbers. > > > Object oriented systems have a general problem in constraining > > functions that take two arguments. To take addition as an example, > > ``int(3) + int(4)`` is defined, and ``vector(1,2,3) + vector(3,4,5)`` > > is defined, but ``int(3) + vector(3,4,5)`` doesn't make much sense. > > In NumPy, this kind of operation makes sense we just broadcast int(3) to > the equivalent vector(3,3,3) and do the element-by-element operation. > > This is how most array packages outside of Python do it as well. Oops; I found a bad example. Does vector(3,3,3) + matrix([4,5],[6,7]) work? Are there other good examples of the addition having to be from the "same" group? > I think adding the abstract base-classes for "algebras" is more > complicated than has been discussed if you are going to start treating > arrays as elements in their own algebra. There is a lot of flexibility > in how arrays can be viewed in an algebraic context. Really? Since the operators work element-wise, I would think that a numpy array would have exactly the same algebraic structure as its elements. One could wrap an array to put it into a particular algebraic context, but the array itself seems pretty simple. > The author gives a valiant go at defining base classes for generic > algebras, but I didn't see everything I've encountered in my travels > through algebra, and I don't understand the motivation for trying to > include these things in Python specifically. > > Therefore, I would stick with defining a hierarchy of numbers and leave > it at that for now (if I even went that far). > > For numbers, several suggestions have been offered. In NumPy we use > abstract base classes that look like this > > Number > Integer > SignedInteger > UnsignedInteger > Inexact > Floating > ComplexFloating > > But, this is because the array scalars in NumPy are based on > C-data-types and not on general-purpose integers or floats. In NumPy it > is useful to check if you have an integer-typed array and so the > base-classes become useful. Do you have an example of when you'd want to check for an integer-typed array? At one point, Guido was asking for reasons to include Cardinal (aka Natural, aka Unsigned) in the hierarchy. Is UnsignedInteger only present in NumPy because it describes some native C types, or does it have some other purpose? > For general purpose Python, I would do something like > > Complex > Rational_Complex > Integer_Complex > Floating_Complex # uses hardware float > Decimal_Complex > Real > Rational > Integer > Floating # uses hardware float > Decimal Are you specifying an abstract Floating type to incorporate both floats and doubles? Glancing over the Decimal spec, and trying a few things with the implementation, it looks like Decimal objects have a specific precision for each thread, which can be changed for future operations at any time. Is there still a reason to separate Floating and Decimal, or will a modification of FloatingReal suffice? Thanks again, Jeffrey From greg.ewing at canterbury.ac.nz Sun Apr 29 03:47:04 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 29 Apr 2007 13:47:04 +1200 Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD In-Reply-To: <4632DD83.2030301@v.loewis.de> References: <46327CC9.4040901@v.loewis.de> <4632DD83.2030301@v.loewis.de> Message-ID: <4633F918.1010205@canterbury.ac.nz> Martin v. L?wis wrote: > I expect access to ob_type is frequent in Foo_Check() macros; > those need to be rewritten to use Py_Type(op). Maybe some of this breakage could be repaired by renaming the real ob_type field and doing something like #define ob_type ob_base._ob_type -- Greg From greg.ewing at canterbury.ac.nz Sun Apr 29 03:57:49 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 29 Apr 2007 13:57:49 +1200 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <20070428161204.19381.458566240.divmod.quotient.6050@ohm> Message-ID: <4633FB9D.9090705@canterbury.ac.nz> Guido van Rossum wrote: > On 4/28/07, Jean-Paul Calderone wrote: > >> >>> class X(object): >> ... __class__ = property(lambda self: int) > > I'm not particularly enamored with it, but I believe it once served a > purpose for Zope. Does anyone know if it is still needed? I thought you wanted this so that proxy objects can pretend to be instances of the class they're proxying. -- Greg From martin at v.loewis.de Sun Apr 29 12:05:13 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Apr 2007 12:05:13 +0200 Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD In-Reply-To: <4633F918.1010205@canterbury.ac.nz> References: <46327CC9.4040901@v.loewis.de> <4632DD83.2030301@v.loewis.de> <4633F918.1010205@canterbury.ac.nz> Message-ID: <46346DD9.1020101@v.loewis.de> Greg Ewing schrieb: > Martin v. L?wis wrote: >> I expect access to ob_type is frequent in Foo_Check() macros; >> those need to be rewritten to use Py_Type(op). > > Maybe some of this breakage could be repaired by renaming > the real ob_type field and doing something like > > #define ob_type ob_base._ob_type That doesn't work. For variable-sized objects, it would need to be ob_base.ob_base.ob_type. Regards, Martin From martin at v.loewis.de Sun Apr 29 12:10:02 2007 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 29 Apr 2007 12:10:02 +0200 Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD In-Reply-To: References: <46327CC9.4040901@v.loewis.de> Message-ID: <46346EFA.1000408@v.loewis.de> >> I propose the PEP below for Py3k. > > I think it's a good idea. It would be nice if the PEP included > something about forwards compatibility. Ideally, there should be a > way to write a module that works with both 2.x and 3.x (e.g. using > preprocessor macros). Obviously the 3.x design should not be > compromised for this sake though. Daniel Stutzbach proposed (in private mail) to preserve PyObject_HEAD, but define it as "PyObject ob_base;". Technically, that would work just fine, and I'll revise the PEP in that direction. I'd still prefer if the core would not use PyObject_HEAD on its own, because the semicolon-missing macro breaks editors that do indentation based on the C syntax, but providing this level of compatibility is easy. Regards, Martin From martin at v.loewis.de Sun Apr 29 15:47:27 2007 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 29 Apr 2007 15:47:27 +0200 Subject: [Python-3000] PEP 3123: Making PyObject_HEAD conform to standard C (Was: Dropping PyObject_HEAD) In-Reply-To: References: <46327CC9.4040901@v.loewis.de> Message-ID: <4634A1EF.6080204@v.loewis.de> > An example of how this will change current code would be good. E.g., > ``o->ob_type->tp_name`` becomes ``PyType(o)->typ_name`` or > ``o->ob_base->ob_type->tp_name``. > > Otherwise I am all for cleaning up the codebase and thus support this PEP. Thanks for the suggestions, I have add them all into PEP 3123. I also changed the PEP title, to reflect that PyObject_HEAD is maintained for compatibility. Regards, Martin From martin at v.loewis.de Sun Apr 29 15:48:33 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Apr 2007 15:48:33 +0200 Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD In-Reply-To: References: <46327CC9.4040901@v.loewis.de> Message-ID: <4634A231.8010700@v.loewis.de> > - add the new access macros to 2.6 (defined appropriately) > - in 3.0, instead of removing PyObject_HEAD and company, redefine them > appropriately, .e.g, #define PyObject_HEAD PyObject obtype; Thanks, I added both of them into PEP 3123 (now retitled to indicate that PyObject_HEAD is not dropped). Regards, Martin From martin at v.loewis.de Sun Apr 29 16:17:01 2007 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 29 Apr 2007 16:17:01 +0200 Subject: [Python-3000] PEP 3121: Module Initialization and finalization In-Reply-To: References: <4631B570.7030307@v.loewis.de> Message-ID: <4634A8DD.7030109@v.loewis.de> Brett, Thanks for the suggestions; I added them to the PEP. > How would I raise xx.error in C code now? I am guessing like this:: Interestingly enough, xxmodule doesn't raise error. > PyObject* module = PyState_FindModule(&xxmodule); Either that, or just use the self parameter if you are in a module-level function > if (!module) > return NULL; > PyObject* xx_error = xxstate(module)->ErrorObject; > if (!xx_error) { > PyErr_SetString(PyExc_SystemError, "xx.error missing"); > return NULL; > } That should not fail. If the module object is still present, the state shouldn't be cleared normally. So this might only occur if the code above is invoked in the "clear cycles" phase of garbage collection. > Since most objects will move to being memory-managed, one needs to > worry about checking that the object still exists. I assume I didn't > go overboard with the error checking here, right? It would be good if the check for the NULL exception object could be folded in PyErr_SetString (i.e. that raising an internal error if the exception class is NULL). > I guess people are > going to end up writing helper functions to access the various data > fields as the above would get rather tedious if you had to write it > more than twice. Over time, we can try to find convenience macro-functions, e.g. PyModule_ObjectField(xxmodule, ErrorObject) which would return a borrowed reference of type PyObject*, or NULL (with an exception set) if there is some kind of error. >> Tim Peters reports in [1]_ that PythonLabs considered such a feature >> at one point, and lists the following additional hooks which aren't >> currently supported in this PEP: >> >> * when the module object is deleted from sys.modules >> >> * when Py_Finalize is called >> >> * when Python exits >> > > Wouldn't the above be covered by the deallocation of the module? No: deletion from sys.modules might not cause module deallocation, if there are other references to the module (although Guido might have meant this mean "module deallocation"). As for finalization/exiting: normally, all modules should be released before Python exits. Again, if there is some garbage reference to the module, it may not have been deallocated; the question then is whether it should get a further opportunity for cleanup. Regards, Martin From jimjjewett at gmail.com Sun Apr 29 17:40:52 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 29 Apr 2007 11:40:52 -0400 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <76fd5acf0704280613w38c273b6xd3073d0076caa0d5@mail.gmail.com> References: <76fd5acf0704280613w38c273b6xd3073d0076caa0d5@mail.gmail.com> Message-ID: On 4/28/07, Calvin Spealman wrote: > I'm just going to jump into this and voice a concern that allowing > overriding of isinstance and issubclass seems like a Bad Idea. I'm also uncomfortable, which is why I originally suggested a parallel predicate. That said, I can't put my finger on why it is wrong, so I'm not certain that a parallel predicate (isa?) would be better. Guido wrote: > Note though that only the second argument to either function can > overload the rules. IOW if you write isinstance(x, C), there is no way > that x could attempt to lie; but C could. As Barry pointed out, this means a class can't easily say "my parent may well implement ABC, but I don't." That special case could be added to the default overriding behavior, but ... it starts to get fragile. -jJ From brett at python.org Sun Apr 29 19:06:37 2007 From: brett at python.org (Brett Cannon) Date: Sun, 29 Apr 2007 10:06:37 -0700 Subject: [Python-3000] PEP 3123: Making PyObject_HEAD conform to standard C (Was: Dropping PyObject_HEAD) In-Reply-To: <4634A1EF.6080204@v.loewis.de> References: <46327CC9.4040901@v.loewis.de> <4634A1EF.6080204@v.loewis.de> Message-ID: On 4/29/07, "Martin v. L?wis" wrote: > > An example of how this will change current code would be good. E.g., > > ``o->ob_type->tp_name`` becomes ``PyType(o)->typ_name`` or > > ``o->ob_base->ob_type->tp_name``. > > > > Otherwise I am all for cleaning up the codebase and thus support this PEP. > > Thanks for the suggestions, I have add them all into PEP 3123. > > I also changed the PEP title, to reflect that PyObject_HEAD is > maintained for compatibility. > Are we going to have a preference over PyObject_HEAD or defining the struct field directly? -Brett From brett at python.org Sun Apr 29 19:12:20 2007 From: brett at python.org (Brett Cannon) Date: Sun, 29 Apr 2007 10:12:20 -0700 Subject: [Python-3000] PEP 3121: Module Initialization and finalization In-Reply-To: <4634A8DD.7030109@v.loewis.de> References: <4631B570.7030307@v.loewis.de> <4634A8DD.7030109@v.loewis.de> Message-ID: On 4/29/07, "Martin v. L?wis" wrote: > Brett, > > Thanks for the suggestions; I added them to the PEP. > > > How would I raise xx.error in C code now? I am guessing like this:: > > Interestingly enough, xxmodule doesn't raise error. > > > PyObject* module = PyState_FindModule(&xxmodule); > > Either that, or just use the self parameter if you are in a > module-level function > OK. > > if (!module) > > return NULL; > > PyObject* xx_error = xxstate(module)->ErrorObject; > > if (!xx_error) { > > PyErr_SetString(PyExc_SystemError, "xx.error missing"); > > return NULL; > > } > > That should not fail. If the module object is still present, > the state shouldn't be cleared normally. So this might > only occur if the code above is invoked in the "clear cycles" > phase of garbage collection. > Maybe it shouldn't, but can it? Is there something preventing the exception object from being cleared beyond people not being stupid? > > Since most objects will move to being memory-managed, one needs to > > worry about checking that the object still exists. I assume I didn't > > go overboard with the error checking here, right? > > It would be good if the check for the NULL exception object could > be folded in PyErr_SetString (i.e. that raising an internal error > if the exception class is NULL). > Sounds reasonable. > > I guess people are > > going to end up writing helper functions to access the various data > > fields as the above would get rather tedious if you had to write it > > more than twice. > > Over time, we can try to find convenience macro-functions, e.g. > > PyModule_ObjectField(xxmodule, ErrorObject) > > which would return a borrowed reference of type PyObject*, or > NULL (with an exception set) if there is some kind of error. > I am sure some will pop up. > >> Tim Peters reports in [1]_ that PythonLabs considered such a feature > >> at one point, and lists the following additional hooks which aren't > >> currently supported in this PEP: > >> > >> * when the module object is deleted from sys.modules > >> > >> * when Py_Finalize is called > >> > >> * when Python exits > >> > > > > Wouldn't the above be covered by the deallocation of the module? > > No: deletion from sys.modules might not cause module deallocation, > if there are other references to the module (although Guido might > have meant this mean "module deallocation"). > You're right. I was not thinking big enough to consider multiple imports of the same module. > As for finalization/exiting: normally, all modules should be > released before Python exits. Again, if there is some garbage > reference to the module, it may not have been deallocated; > the question then is whether it should get a further opportunity > for cleanup. Eh. I say no for now and if people really need it we can add the hook later. -Brett From jyasskin at gmail.com Sun Apr 29 19:25:35 2007 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Sun, 29 Apr 2007 10:25:35 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> Message-ID: <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com> On 4/28/07, Baptiste Carvello wrote: > 2) In the PEP, the concepts are used *inconsistently*. Complex derives from Ring > because the set of complex numbers *is a* ring. Int derives from Complex because > integer are complex numbers (or, alternatively, the set of integers *is included > in* the set of complex numbers). The consistent way could be to make the Complex > class an instance of Ring, not a subclass. Good point. In this structure, isinstance(3, Ring) really means that 3 is a member of some (unspecified) ring, not that 3 isa Ring, but the ambiguity there is probably the root cause of the problem with mixed-mode operations. We should also say that isinstance(3, Complex) means that 3 is a member of some subring of the complex numbers, which preserves the claim that Complex is a subtype of Ring. Up to here, things make sense, but because of how ABCs work, we need issubclass(rational, Complex). I suppose that's true too, since isinstance(3.4, rational) means "3.4 is a member of the rational subring of the complex numbers", which implies that "3.4 is a member of some subring of the complex numbers." There may be better names for these concepts. Perhaps suffixing every numeric ABC with "Element"? Do you have suggestions? Jason Orendorff points out that Haskell typeclasses capture the fact that complex is an instance of Ring. I think imitating them as much as possible would indeed imply making the numeric ABCs into metaclasses (in Haskell terminology, "kinds"). To tell if the arguments to a function were in the same total order, you'd check if they had any common superclasses that were themselves instances of TotallyOrdered. I don't know enough about how metaclasses are typically used to know how that would conflict. -- Namast?, Jeffrey Yasskin From martin at v.loewis.de Sun Apr 29 19:57:45 2007 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 29 Apr 2007 19:57:45 +0200 Subject: [Python-3000] PEP 3123: Making PyObject_HEAD conform to standard C (Was: Dropping PyObject_HEAD) In-Reply-To: References: <46327CC9.4040901@v.loewis.de> <4634A1EF.6080204@v.loewis.de> Message-ID: <4634DC99.2090307@v.loewis.de> > Are we going to have a preference over PyObject_HEAD or defining the > struct field directly? That is an open issue. PyObject_HEAD clearly is more backwards-compatible. However (besides its current non-C-conformance), I never liked it, because of the missing semicolon (I thought it confuses Emacs, but can't reproduce the problem anymore, though - perhaps it did confuse Emacs in the past?) So I personally would phase it out for style reasons, but others may think differently; if I hear opinions in either direction, I'm happy to code them into the PEP. Regards, Martin From martin at v.loewis.de Sun Apr 29 20:15:05 2007 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 29 Apr 2007 20:15:05 +0200 Subject: [Python-3000] PEP 3121: Module Initialization and finalization In-Reply-To: References: <4631B570.7030307@v.loewis.de> <4634A8DD.7030109@v.loewis.de> Message-ID: <4634E0A9.2020900@v.loewis.de> >> > if (!module) >> > return NULL; >> > PyObject* xx_error = xxstate(module)->ErrorObject; >> > if (!xx_error) { >> > PyErr_SetString(PyExc_SystemError, "xx.error missing"); >> > return NULL; >> > } >> >> That should not fail. If the module object is still present, >> the state shouldn't be cleared normally. So this might >> only occur if the code above is invoked in the "clear cycles" >> phase of garbage collection. >> > > Maybe it shouldn't, but can it? Is there something preventing the > exception object from being cleared beyond people not being stupid? I believe it would always be possible to trigger a problem, no matter what we do: if you key invocation of the function to deletion of the exception class (which is always possible through weak references), then you can trigger a problem indeed. I also now think there can't be a solution: E.g. you may associate reference the exception from the method object itself, so that you the exception "should" be alive while the function is still available. However, breaking cycles may decide to tp_clear that reference, making that approach futile. In some cases, it may be possible to statically determine that clearing certain reference will never be necessary to break all cycles (because every such cycle would be broken at a different reference also), however, that is difficult to generalize, and prone to errors that result in uncollectable garbage. Regards, Martin From rasky at develer.com Sun Apr 29 21:16:52 2007 From: rasky at develer.com (Giovanni Bajo) Date: Sun, 29 Apr 2007 21:16:52 +0200 Subject: [Python-3000] PEP 3123: Making PyObject_HEAD conform to standard C (Was: Dropping PyObject_HEAD) In-Reply-To: <4634DC99.2090307@v.loewis.de> References: <46327CC9.4040901@v.loewis.de> <4634A1EF.6080204@v.loewis.de> <4634DC99.2090307@v.loewis.de> Message-ID: On 29/04/2007 19.57, Martin v. L?wis wrote: > That is an open issue. PyObject_HEAD clearly is more > backwards-compatible. However (besides its current non-C-conformance), > I never liked it, because of the missing semicolon (I thought it > confuses Emacs, but can't reproduce the problem anymore, though - > perhaps it did confuse Emacs in the past?) > > So I personally would phase it out for style reasons, but others > may think differently; if I hear opinions in either direction, > I'm happy to code them into the PEP. The fact that it existed made it possible to do this transition though. Haven't we had this macro in the first place, now you would need to break all third party modules without an easy fallback. So, I'm probably +0 on keeping it since history tends to repeat itself :) -- Giovanni Bajo From brett at python.org Sun Apr 29 21:56:58 2007 From: brett at python.org (Brett Cannon) Date: Sun, 29 Apr 2007 12:56:58 -0700 Subject: [Python-3000] PEP 3123: Making PyObject_HEAD conform to standard C (Was: Dropping PyObject_HEAD) In-Reply-To: <4634DC99.2090307@v.loewis.de> References: <46327CC9.4040901@v.loewis.de> <4634A1EF.6080204@v.loewis.de> <4634DC99.2090307@v.loewis.de> Message-ID: On 4/29/07, "Martin v. L?wis" wrote: > > Are we going to have a preference over PyObject_HEAD or defining the > > struct field directly? > > That is an open issue. PyObject_HEAD clearly is more > backwards-compatible. However (besides its current non-C-conformance), > I never liked it, because of the missing semicolon (I thought it > confuses Emacs, but can't reproduce the problem anymore, though - > perhaps it did confuse Emacs in the past?) > > So I personally would phase it out for style reasons, but others > may think differently; if I hear opinions in either direction, > I'm happy to code them into the PEP. > I say phase out the macro. -Brett From brett at python.org Sun Apr 29 22:04:09 2007 From: brett at python.org (Brett Cannon) Date: Sun, 29 Apr 2007 13:04:09 -0700 Subject: [Python-3000] Suggested PEP 3108 addition In-Reply-To: <43aa6ff70704251005i4912e6d6s7efc6346483a142b@mail.gmail.com> References: <43aa6ff70704251005i4912e6d6s7efc6346483a142b@mail.gmail.com> Message-ID: Done. On 4/25/07, Collin Winter wrote: > I'd like to propose that the bsddb185 module be added to PEP 3108's > list of modules slated for removal. Reasons to kill it: > > - The module isn't built by default; manual tweaking of setup.py is required. > > - The following scare-paragraph is included at the bottom of > http://docs.python.org/lib/module-bsddb.html: > > """ > Beginning in 2.3 some Unix versions of Python may have a bsddb185 > module. This is present only to allow backwards compatibility with > systems which ship with the old Berkeley DB 1.85 database library. The > bsddb185 module should never be used directly in new code. > """ > > Collin Winter > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/brett%40python.org > From tcdelaney at optusnet.com.au Sun Apr 29 22:44:45 2007 From: tcdelaney at optusnet.com.au (Tim Delaney) Date: Mon, 30 Apr 2007 06:44:45 +1000 Subject: [Python-3000] Pre-pre PEP for 'super' keyword References: <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com><76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com><76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <002401c78778$75fb7eb0$0201a8c0@ryoko> Message-ID: <00b601c78a9f$38ec9390$0201a8c0@ryoko> I've been intending to write up a PEP for fixing super, but I haven't had time to get to it. I've worked out the semantics I'm going to propose though. I've attached an implementation of those semantics as far as I've been able to get them so far. I think this could be targeted at 2.6, with a __future__ import. Basically, the proposals are: 1. 'super' becomes a keyword, that returns a super object for the instance method currently being executed. 2. 'super' cannot be rebound, but the object produced can be assigned to other names i.e. super = None # SyntaxError, or UnrebindableError (see #7 below) name = super # OK This allows passing the super object to other functions. 3. 'super' is a cell variable i.e. inner functions can use it, and will get the super object of the outer method. class A(object): def f(self): def inner(): print super inner() 4. super objects are callable, and calling them will execute the super method with the same name as the instance method currently being executed. Lookup of this method occurs when the instance method is entered. class A(object): def f(self): pass class B(A): def f(self): super() # Calls A.f(self) If you want name lookup to occur at the time of the call, you can explicitly specify the method name (just like with any other super attribute): class A(object): def f(self): pass class B(A): def f(self): super.f() # Calls A.f(self) 5. The repr of a super object will include the current class, the instance type and the bound method that will be used when called (if present) or an AttributeError: , , >> , , > 6. Methods will have an attribute holding the class where the unbound method was created. This attribute is available on both unbound and bound methods. In the attached code I've called it func_class, but it really should have an im_prefix (can't be im_class though). This is used to make the repr of a bound method actually reflect which class it comes from. 7. Code objects gain a tuple of unrebindable names, which can be local variables, cell variables, etc. For instance methods and class methods, this would initially contain two names - 'super' and the name of the first parameter to the method (normally 'self' or 'cls). Attempting to rebind these names would throw an UnrebindableError (maybe a subclass of SyntaxError). It would be good if these could actually be detected at compile time. Differences in the attached code are: 1. You need to inherit from 'autosuper' (or use the _autosuper metaclass). 2. 'super' is not a keyword. 3. Rebinding 'super' will not throw an exception, but such rebinding will be ignored. Also see the next point. 4. It is *necessary* to rebind 'super' in the method in order to create a cell for use in inner classes. The recommended way in the attached code is: class A(autosuper): def f(self): super = super def a(): print super a() The 'super = super' line above would normally throw an UnboundLocalError, but the bytecode hacking I've done causes it to work. Comments before I start writing up the PEP? Or if anyone has a PEP partially written, do you want to incorporate any of the above into it? Tim Delaney -------------- next part -------------- A non-text attachment was scrubbed... Name: autosuper.py Type: application/octet-stream Size: 13032 bytes Desc: not available Url : http://mail.python.org/pipermail/python-3000/attachments/20070430/6e412e89/attachment-0001.obj From jimjjewett at gmail.com Sun Apr 29 23:15:33 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 29 Apr 2007 17:15:33 -0400 Subject: [Python-3000] New Super PEP - draft 2 Message-ID: (Adding python-3000 to the Cc, since that is where much of the recent discussion occurred) Calvin's full proposal is archived at: http://mail.python.org/pipermail/python-dev/2007-April/072835.html > Abstract > ======== > The PEP defines the proposal to enhance the super builtin to work implicitly > upon the class within which it is used and upon the instance the current > function was called on. The premise of the new super usage suggested is as > follows: > super.foo(1, 2) > to replace the old: > super(Foo, self).foo(1, 2) The alternatives section needs to explain why "super.foo(*args)" is the right level of simplification. Alternatives include (1) No changes Con: fragility in name binding (2) super(__this_class__, self).foo(*args) Con: The "(__this_class__, self)" is boilerplate. Note that the __this_class__ PEP should be referenced, if only as a possible explanation for what is happening under the covers. (3) self.__super__.foo(*args) # or super.foo(*args) Con: Shouldn't need a __double_underscore_name__ in normal functions. Con: __super__ (even if renamed super) is not a simple attribute; it is a property representing a partially applied function (that gets further applied by the "self") (4) super(self, *args) # ? or __super__(self, *args) Actually, I sort of like this one, as it allows the upcall signature to exactly match the method definition signature. Con: super is still not a simple attribute. Con: changing the method name becomes a hassle. (5) super.foo(self, *args) # ? or __super__.foo(self, *args) Con: self isn't really an argument just to super.foo -- it is an argument to super which is used to find/instantiate foo in the first place. (Plus the objections to partially applied function attributes.) (6) super # ? or super() Pro: matches java; doesn't require a repeat of *args Con: doesn't match anything else in python, needs to be a keyword (and would still need object support). In the first example: class A(object): def f(self): return 'A' class B(A): def f(self): return 'B' + super.f() class C(A): def f(self): return 'C' + super.f() class D(B, C): def f(self): return 'D' + super.f() assert D().f() == 'DBCA' You should probably include tests that fail with some of the more obvious (but wrong) solutions, such as class A(object): def f(self): return 'A' class Bempty(A): # Make sure it doesn't call the A or B method twice pass class B(Bempty): def f(self): return 'B' + super.f() class C(A): def f(self): return 'C' + super.f() class D(B, C): def f(self): return 'D' + super.f() assert D().f() == 'DBCA' class E(C, B): # Show that B can point to C as next or vice versa def f(self): return 'E' + super.f() assert E().f() == 'ECBA' class F(D): # show that the instance may not be a direct instance pass assert D().f() == 'DBCA' > The enhancements to the super type will define a new __getattr__ classmethod > of the super type, Did you really mean, it gets the class of builtin.super, but no info on which class is using super? If not, it isn't a classmethod of the super type. It may be a classmethod of the currently-being-defined-type (__this_class__). > which must look backwards to the previous frame and locate > the instance object. frame manipulation is fragile. Even if it didn't cause problems for other implementations, it causes problems for nested functions and callbacks. def f(self, button, *args): def callback(*args): super(__this_class__, self).f(*args) button.callback=callback When this gets called, the appropriate self won't be in the frame, except as a lexically scoped variable (which might have a different name). Also, note that this fails on most of Thomas Wouters' advanced usages. While you say that super shouldn't be called outside a method, you can't really keep the super object itself from getting returned. > "Every class will gain a new special attribute, __super__, which refers to an > instance of the associated super object for that class" In this capacity, the > new super also acts as its own descriptor, create an instance-specific super > upon lookup. If you didn't say instance-specific, this would be equivalent to a class decorator, translating class A(object): ... into class A(object): ... A.__super__=super(A) As is, it gets a bit trickier, since you need to I think the translation is closer to class A(object):... @property def __super__(self): return __this_class__.__supermaker() A.__supermaker=super(A) I don't see a good non-magical way to pass both the instance and the class-at-time-of-writing (as opposed to class of calling instance), which is why the other solutions used bytecode hacks or name mangling. > Much of this was discussed in the thread of the python-dev list, "Fixing super > anyone?" [1]_. I assume you meant the python-3000 list. -jJ From jimjjewett at gmail.com Sun Apr 29 23:25:53 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 29 Apr 2007 17:25:53 -0400 Subject: [Python-3000] Pre-pre PEP for 'super' keyword In-Reply-To: <00b601c78a9f$38ec9390$0201a8c0@ryoko> References: <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <002401c78778$75fb7eb0$0201a8c0@ryoko> <00b601c78a9f$38ec9390$0201a8c0@ryoko> Message-ID: On 4/29/07, Tim Delaney wrote: > I've been intending to write up a PEP for fixing super, but I haven't had > time to get to it. Calvin Spealman has the most recent draft. I hope he will incorporate this into his draft. > 1. 'super' becomes a keyword, that returns a super object for the instance > method currently being executed. So it is a "keyword" in the sense that None is a keyword; not in the stronger sense that "if" is a keyword? > 4. super objects are callable, and calling them will execute the super > method with the same name as the instance method currently being executed. > Lookup of this method occurs when the instance method is entered. > > class A(object): > def f(self): > pass > > class B(A): > def f(self): > super() # Calls A.f(self) > If you want name lookup to occur at the time of the call, you can explicitly > specify the method name (just like with any other super attribute): > > class A(object): > def f(self): > pass > > class B(A): > def f(self): > super.f() # Calls A.f(self) As long as you can be explicit, should the shortcut be a full shortcut? That is, def f(self, a, b=c, *args, **kwargs): super() # passes the exact arglist that f got vs def __init__(self, myvar, passed_var): super.__init__(self, passed_var) # flags that you are changing the args -jJ From ironfroggy at gmail.com Mon Apr 30 00:23:52 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 29 Apr 2007 18:23:52 -0400 Subject: [Python-3000] Pre-pre PEP for 'super' keyword In-Reply-To: References: <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <002401c78778$75fb7eb0$0201a8c0@ryoko> <00b601c78a9f$38ec9390$0201a8c0@ryoko> Message-ID: <76fd5acf0704291523q78a14a26lb421d927252495a8@mail.gmail.com> On 4/29/07, Jim Jewett wrote: > On 4/29/07, Tim Delaney wrote: > > I've been intending to write up a PEP for fixing super, but I haven't had > > time to get to it. > > Calvin Spealman has the most recent draft. I hope he will incorporate > this into his draft. I will incorporate this into my draft, particularly taking care of the issue with inner functions. > > 1. 'super' becomes a keyword, that returns a super object for the instance > > method currently being executed. > > So it is a "keyword" in the sense that None is a keyword; not in the > stronger sense that "if" is a keyword? I would like to say super becomes a constant in the way that None is a constant, and if there is some reason the implementation today or tomorrow can benefit from actually making it a keyword, that won't break anything if its already constant. One problem with an actual keyword, is there is no other part of Python where an actual keyword evaluates to something. > > 4. super objects are callable, and calling them will execute the super > > method with the same name as the instance method currently being executed. > > Lookup of this method occurs when the instance method is entered. > > > > class A(object): > > def f(self): > > pass > > > > class B(A): > > def f(self): > > super() # Calls A.f(self) This might run into the same issue I had to cover, where you get an ambiguous situation trying to distinguish between calling super and calling the __call__ method of the next class in the MRO. We should absolutely avoid a situation in python now where X() differs from X.__call__() > > If you want name lookup to occur at the time of the call, you can explicitly > > specify the method name (just like with any other super attribute): > > > > class A(object): > > def f(self): > > pass > > > > class B(A): > > def f(self): > > super.f() # Calls A.f(self) > > As long as you can be explicit, should the shortcut be a full > shortcut? That is, > > def f(self, a, b=c, *args, **kwargs): > super() # passes the exact arglist that f got I sure wish my previous complaints didn't hinder this, because I really love the idea of being able to this, which would really encourage more compatible method signatures, so you can use the shortcut! I'm desperate for a solution that satisfies all the sides of the equation. > vs > > def __init__(self, myvar, passed_var): > super.__init__(self, passed_var) # flags that you are > changing the args > > -jJ > -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From guido at python.org Mon Apr 30 00:46:58 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 29 Apr 2007 15:46:58 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com> Message-ID: On 4/29/07, Jeffrey Yasskin wrote: > On 4/28/07, Baptiste Carvello wrote: > > 2) In the PEP, the concepts are used *inconsistently*. Complex derives from Ring > > because the set of complex numbers *is a* ring. Int derives from Complex because > > integer are complex numbers (or, alternatively, the set of integers *is included > > in* the set of complex numbers). The consistent way could be to make the Complex > > class an instance of Ring, not a subclass. > > Good point. In this structure, isinstance(3, Ring) really means that 3 > is a member of some (unspecified) ring, not that 3 isa Ring, To ask whether x is a Ring, you'd use issubclass(x, Ring). Now, in a different context (still in Python) you might define a class Ring whose *instances* are rings; but in the current draft of PEP 3141, Ring is a class whose subclasses are rings. [BTW "isa" is not an English word, nor used anywhere in Python. If "isa" means "is a" you might as well write proper English; if it has additional connotations, they're likely lost to this crowd so you can't count on your readers knowing the distinction.] > but the ambiguity there is probably the root cause of the problem with > mixed-mode operations. (Where by mixed-mode operations you mean e.g. trying to add or multiply members of two *different* rings, right?) > We should also say that isinstance(3, Complex) > means that 3 is a member of some subring of the complex numbers, which > preserves the claim that Complex is a subtype of Ring. Hm... it's beginning to look like binary operations in the mathematical sense just don't have an exact equivalent in OO type theory. (Or vice versa.) There really isn't a way to define an operation binop(a: T, b: T) -> T in such a way that it is clear what should happen if a and b are members of two different subtypes of T, named T1 and T2. Classic OO seems to indicate that this must be defined, and there are plenty of examples that seem to agree, e.g. when T1 and T2 are trivial subtypes of T (maybe each adding a different inessential method). OTOH we have the counter-example where T==Ring and T1 and T2 are two different, unrelated Rings, and the result may not be defined. Hmm... Maybe the conclusion to draw from this is that we shouldn't make Ring a class? Maybe it ought to be a metaclass, so we could ask isinstance(Complex, Ring)? Perhaps a similar line of reasoning migtht apply to PartiallyOrdered and TotallyOrdered. > Up to here, things make sense, but because of how ABCs work, we need > issubclass(rational, Complex). I suppose that's true too, since > isinstance(3.4, rational) means "3.4 is a member of the rational > subring of the complex numbers", which implies that "3.4 is a member > of some subring of the complex numbers." > > There may be better names for these concepts. Perhaps suffixing every > numeric ABC with "Element"? Do you have suggestions? Maybe we should stop trying to capture radically different mathematical number systems using classes or types, and limit ourselves to capturing the systems one learns in high school: C, R, Q, Z, and (perhaps) N (really N0). The concrete types would be complex <: C, float<:R, Decimal<:R, int<:Z. NumPy would have many more. One could argue that float and Decimal are <:Q, but I'm not sure if that makes things better pragmatically; I guess I'm coming from the old Algol school where float was actually called real (and in retrospect I wish I'd called it that in Python). I'd rather reserve membership of Q for an infinite precision rational type (which many people have independently implemented). > Jason Orendorff points out that Haskell typeclasses capture the fact > that complex is an instance of Ring. I think imitating them as much as > possible would indeed imply making the numeric ABCs into metaclasses > (in Haskell terminology, "kinds"). To tell if the arguments to a > function were in the same total order, you'd check if they had any > common superclasses that were themselves instances of TotallyOrdered. > I don't know enough about how metaclasses are typically used to know > how that would conflict. The more I think about it, it sounds like the right thing to do. To take PartiallyOrdered (let's say PO for brevity) as an example, the Set class should specify PO as a metaclass. The PO metaclass could require that the class implement __lt__ and __le__. If it found a class that didn't implement them, it could make the class abstract by adding the missing methods to its __abstractmethods__ attribute. Or, if it found that the class implemented one but not the other, it could inject a default implementation of the other in terms of the one and __eq__. This leaves us with the question of how to check whether an object is partially orderable. Though that may really be the wrong question -- perhaps you should ask whether two objects are partially orderable relative to each other. For that, you would first have to find the most derived common base class (if that is even always a defined operation(*)), and then check whether that class is an instance of PO. It seems easier to just try the comparison -- duck typing isn't dead yet! I don't think this is worth introducing a new inspection primitive ('ismetainstance(x, PO)'). The PO class may still be useful for introspection: at the meta-level, it may be useful occasionally to insist that or inquire whether a given *class* is PO. (Or TO, or a Ring, etc.) Now, you could argue that Complex should also be a metaclass. While that may mathematically meaningful (for all I know there are people doing complex number theory using Complex[Z/n]), for Python's numeric classes I think it's better to make Complex a regular class representing all the usual complex numbers (i.e. a pair of Real numbers). I expect that the complex subclasses used in practice are all happy under mixed arithmetic using the usual definition of mixed arithmetic: convert both arguments to a common base class and compute the operation in that domain. (*) consider classes AB derived from (B, A) and BA derived from (A, B). Would A or B be the most derived base class? Or would we have to skip both and continue the search with A's and B's base classes? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From collinw at gmail.com Mon Apr 30 00:49:54 2007 From: collinw at gmail.com (Collin Winter) Date: Sun, 29 Apr 2007 15:49:54 -0700 Subject: [Python-3000] [Python-Dev] Pre-pre PEP for 'super' keyword In-Reply-To: <76fd5acf0704291523q78a14a26lb421d927252495a8@mail.gmail.com> References: <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <002401c78778$75fb7eb0$0201a8c0@ryoko> <00b601c78a9f$38ec9390$0201a8c0@ryoko> <76fd5acf0704291523q78a14a26lb421d927252495a8@mail.gmail.com> Message-ID: <43aa6ff70704291549v50a11ebah1a96cc213d2d5177@mail.gmail.com> [cc:-python-dev] On 4/29/07, Calvin Spealman wrote: > On 4/29/07, Jim Jewett wrote: > > As long as you can be explicit, should the shortcut be a full > > shortcut? That is, > > > > def f(self, a, b=c, *args, **kwargs): > > super() # passes the exact arglist that f got > > I sure wish my previous complaints didn't hinder this, because I > really love the idea of being able to this, which would really > encourage more compatible method signatures, so you can use the > shortcut! I'm desperate for a solution that satisfies all the sides of > the equation. I hate this. super() calls would be completely different from other function calls in that what appears to be an empty argument list is actually a filled-out argument list. Please stick to the original topic of figuring out how to remove the class name from super calls, rather than inventing new magical, spooky-action-at-a-distance toys. Collin Winter From guido at python.org Mon Apr 30 01:01:47 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 29 Apr 2007 16:01:47 -0700 Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <76fd5acf0704280613w38c273b6xd3073d0076caa0d5@mail.gmail.com> Message-ID: On 4/29/07, Jim Jewett wrote: > Guido wrote: > > Note though that only the second argument to either function can > > overload the rules. IOW if you write isinstance(x, C), there is no way > > that x could attempt to lie; but C could. > > As Barry pointed out, this means a class can't easily say "my parent > may well implement ABC, but I don't." That special case could be > added to the default overriding behavior, but ... it starts to get > fragile. That use case is very questionable, and I see it more as an anti-pattern; I suspect that it's a case of inheriting implementation where you really ought to be using containment instead of subclassing. It's why I'm uncomfortable with string.Template inheriting from a string type, and and ditto for the OO os.path proposals that have the path object inherit from a string type (I know not all of them do). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Mon Apr 30 01:17:46 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 29 Apr 2007 17:17:46 -0600 Subject: [Python-3000] [Python-Dev] Pre-pre PEP for 'super' keyword In-Reply-To: <76fd5acf0704291523q78a14a26lb421d927252495a8@mail.gmail.com> References: <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <002401c78778$75fb7eb0$0201a8c0@ryoko> <00b601c78a9f$38ec9390$0201a8c0@ryoko> <76fd5acf0704291523q78a14a26lb421d927252495a8@mail.gmail.com> Message-ID: On 4/29/07, Calvin Spealman wrote: > On 4/29/07, Jim Jewett wrote: > > On 4/29/07, Tim Delaney wrote: > > > 4. super objects are callable, and calling them will execute the super > > > method with the same name as the instance method currently being executed. > > > Lookup of this method occurs when the instance method is entered. > > > > > > class A(object): > > > def f(self): > > > pass > > > > > > class B(A): > > > def f(self): > > > super() # Calls A.f(self) > > This might run into the same issue I had to cover, where you get an > ambiguous situation trying to distinguish between calling super and > calling the __call__ method of the next class in the MRO. Note that it's at least not backwards incompatible because super objects are not currently callable. You have to make the explicit .__call__() invocation:: >>> class C(object): ... def __call__(self): ... return 'C' ... >>> class D(C): ... def __call__(self): ... sup = super(D, self) ... return 'D' + sup() ... >>> d = D() >>> d() Traceback (most recent call last): File "", line 1, in File "", line 4, in __call__ TypeError: 'super' object is not callable >>> class D(C): ... def __call__(self): ... sup = super(D, self) ... return 'D' + sup.__call__() ... >>> d = D() >>> d() 'DC' That said, you're probably right here: > We should absolutely avoid a situation in python now where X() differs > from X.__call__() Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From guido at python.org Mon Apr 30 01:27:07 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 29 Apr 2007 16:27:07 -0700 Subject: [Python-3000] [Python-Dev] Pre-pre PEP for 'super' keyword In-Reply-To: <43aa6ff70704291549v50a11ebah1a96cc213d2d5177@mail.gmail.com> References: <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <002401c78778$75fb7eb0$0201a8c0@ryoko> <00b601c78a9f$38ec9390$0201a8c0@ryoko> <76fd5acf0704291523q78a14a26lb421d927252495a8@mail.gmail.com> <43aa6ff70704291549v50a11ebah1a96cc213d2d5177@mail.gmail.com> Message-ID: On 4/29/07, Collin Winter wrote: > On 4/29/07, Calvin Spealman wrote: > > On 4/29/07, Jim Jewett wrote: > > > As long as you can be explicit, should the shortcut be a full > > > shortcut? That is, > > > > > > def f(self, a, b=c, *args, **kwargs): > > > super() # passes the exact arglist that f got > > > > I sure wish my previous complaints didn't hinder this, because I > > really love the idea of being able to this, which would really > > encourage more compatible method signatures, so you can use the > > shortcut! I'm desperate for a solution that satisfies all the sides of > > the equation. > > I hate this. super() calls would be completely different from other > function calls in that what appears to be an empty argument list is > actually a filled-out argument list. > > Please stick to the original topic of figuring out how to remove the > class name from super calls, rather than inventing new magical, > spooky-action-at-a-distance toys. Amen, brother! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Apr 30 01:30:18 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 29 Apr 2007 16:30:18 -0700 Subject: [Python-3000] [Python-Dev] Pre-pre PEP for 'super' keyword In-Reply-To: References: <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com> <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <002401c78778$75fb7eb0$0201a8c0@ryoko> <00b601c78a9f$38ec9390$0201a8c0@ryoko> Message-ID: On 4/29/07, Jim Jewett wrote: > So it is a "keyword" in the sense that None is a keyword; not in the > stronger sense that "if" is a keyword? Um, how do you see those two differ? Is 'if' a keyword in the same sense as 'or', or in a different sense? I realize that in Python 2.5, None is not a full-fledged keyword but cannot be used as an assignment target. But that's only transitional. In 3.0 I imagine it becoming a keyword in the grammar (whose only appearance would be as one of the alternatives for 'atom'). And we're talking 3.0 here. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From talin at acm.org Mon Apr 30 02:11:44 2007 From: talin at acm.org (Talin) Date: Sun, 29 Apr 2007 17:11:44 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com> Message-ID: <46353440.5060402@acm.org> Guido van Rossum wrote: > Maybe we should stop trying to capture radically different > mathematical number systems using classes or types, and limit > ourselves to capturing the systems one learns in high school: C, R, Q, > Z, and (perhaps) N (really N0). The concrete types would be complex <: > C, float<:R, Decimal<:R, int<:Z. NumPy would have many more. One could > argue that float and Decimal are <:Q, but I'm not sure if that makes > things better pragmatically; I guess I'm coming from the old Algol > school where float was actually called real (and in retrospect I wish > I'd called it that in Python). I'd rather reserve membership of Q for > an infinite precision rational type (which many people have > independently implemented). I haven't really been following this discussion, given my lack of understanding of the issues involved, but I want to make one observation about the discussion. Normally, when someone suggests an idea for a major addition to Python of this scope, the standard response is that they should go develop it as a 3rd-party package and see if becomes popular, and if it does it can be considered for inclusion in the standard library. Unfortunately, we are somewhat constrained in this case, because we're talking about altering the behavior of some fairly fundamental built-in types - which means that it's going to be hard to implement it as an add-on library. And yet, it seems to me that this particular set of features really does need a long gestation period compared to some others that we've discussed. Most of the 3000-series PEPs are really about a fairly small set of base decisions. Even the long PEPs are more about descriptions of the logical consequences of those decisions than about the decisions themselves. The ABC PEPs are different, in that they are standardizing a whole slew of things all at once. Moreover, I think there is a real danger here of a kind of ivory-tower decision making, isolated from day-to-day writing of applications to keep them grounded. The question of whether to limit to the "high school" number classes, or to go with the more mathematically abstract set of things is a decision which, it seems to me, ought to be made in the context of actual use cases, rather than abstract theorizing about use cases. (I'm also generally supportive about copying what other languages have done, on the theory that they too have been tested by real-world use.) If it were technically possible, I would recommend that this PEP have to run the same gauntlet that any other large library addition would, which is to go through a long period of community feedback and criticism, during which a large number of people actually attempt to use the feature for real work. I also think, in this case, that the special power of "core python developer fiat" should not be invoked unless it has to be, because I don't think that there is a firm basis for making such a judgment yet. I would also suggest that some thought be given to ways to allow for experimentation with different variations of this feature. If it is not possible to make these numeric classes definable in Python at runtime, then perhaps it is possible instead to allow for custom builds of the Python 3000 executable with different arrangements and configurations of these built-in classes. -- Talin From steven.bethard at gmail.com Mon Apr 30 02:19:14 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 29 Apr 2007 18:19:14 -0600 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <46353440.5060402@acm.org> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com> <46353440.5060402@acm.org> Message-ID: On 4/29/07, Talin wrote: > If it were technically possible, I would recommend that this PEP have to > run the same gauntlet that any other large library addition would, which > is to go through a long period of community feedback and criticism, > during which a large number of people actually attempt to use the > feature for real work. This sounds like a pretty good reason to add __isinstance__() and __issubclass__(). Then the various ABCs can be distributed as third-party modules but can still make sure that things like isinstance(42, Real) and issubclass(int, Complex) are True (or whatever other assertions people want to make). STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From greg.ewing at canterbury.ac.nz Mon Apr 30 02:53:39 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 30 Apr 2007 12:53:39 +1200 Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD In-Reply-To: <46346DD9.1020101@v.loewis.de> References: <46327CC9.4040901@v.loewis.de> <4632DD83.2030301@v.loewis.de> <4633F918.1010205@canterbury.ac.nz> <46346DD9.1020101@v.loewis.de> Message-ID: <46353E13.5080501@canterbury.ac.nz> Martin v. L?wis wrote: > That doesn't work. For variable-sized objects, it would need > to be ob_base.ob_base.ob_type. I was wondering about that. It seems that each level of inheritance is going to add another level of required member access. Or is it legal to short- circuit all that and just cast directly to the ultimate base object? -- Greg From ironfroggy at gmail.com Mon Apr 30 03:01:42 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 29 Apr 2007 21:01:42 -0400 Subject: [Python-3000] [Python-Dev] Pre-pre PEP for 'super' keyword In-Reply-To: References: <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com> <002401c78778$75fb7eb0$0201a8c0@ryoko> <00b601c78a9f$38ec9390$0201a8c0@ryoko> Message-ID: <76fd5acf0704291801o47733e29u634ffa317d32a0a7@mail.gmail.com> On 4/29/07, Guido van Rossum wrote: > On 4/29/07, Jim Jewett wrote: > > So it is a "keyword" in the sense that None is a keyword; not in the > > stronger sense that "if" is a keyword? > > Um, how do you see those two differ? Is 'if' a keyword in the same > sense as 'or', or in a different sense? > > I realize that in Python 2.5, None is not a full-fledged keyword but > cannot be used as an assignment target. But that's only transitional. > In 3.0 I imagine it becoming a keyword in the grammar (whose only > appearance would be as one of the alternatives for 'atom'). And we're > talking 3.0 here. I think any concerns about it not being fit as a keyword would fall under two catagories or varying validity: 1) Too many keywords is a valid concern, because it complicates the language. 2) It just doesn't "feel" like a keyword. Less valid, unless it _really_ doesn't feel like a keyword. It doesn't feel like a keyword. But it doesn't feel too much not like a keyword. Anyway, I tried to address the concerns laid out, and I'm more than happy to alter the PEP to actually say "Lets implement this as a keyword", and I actually meant to keep more agnostic on that point in the proposal itself. I was more interested in covering the interface, at least to begin, than the actual implementation. Although, being able to have a solid, working reference implementation based on the frame lookups and such is nice, so we can see how it will actually work in real code, and even use it to backport code using the new super to just about any recent Python version. I also checked and PyPy does implement a sys._getframe() and a IronPython currently doesn't, but seems to plan on it (there is a placeholder, at present). I am not sure if notes on this belongs in the PEP or not. Draft Three follows for all. I think I'm turning off e-mail for the rest of this evening, so I'll catch up tomorrow. ------------------------------------------------------- PEP: XXX Title: New Super Version: $Revision$ Last-Modified: $Date$ Author: Calvin Spealman Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 28-Apr-2007 Python-Version: 2.6 Post-History: 28-Apr-2007, 29-Apr-2007 (1), 29-Apr-2007 (2) Abstract ======== The PEP defines the proposal to enhance the super builtin to work implicitly upon the class within which it is used and upon the instance the current function was called on. The premise of the new super usage suggested is as follows: super.foo(1, 2) to replace the old: super(Foo, self).foo(1, 2) Rationale ========= The current usage of super requires an explicit passing of both the class and instance it must operate from, requiring a breaking of the DRY (Don't Repeat Yourself) rule. This hinders any change in class name, and is often considered a wart by many. Specification ============= Within the specification section, some special terminology will be used to distinguish similar and closely related concepts. "Super type" will refer to the actual builtin type named "super". "Next Class/Type in the MRO" will refer to the class where attribute lookups will be performed by super, for example, in the following, A is the "Next class in the MRO" for the use of super. :: class A(object): def f(self): return 'A' class B(A): def f(self): super(B, self).f() # Here, A would be out "Next class in the # MRO", of course. A "super object" is simply an instance of the super type, which is associated with a class and possibly with an instance of that class. Finally, "new super" refers to the new super type, which will replace the original. Replacing the old usage of super, calls to the next class in the MRO (method resolution order) will be made without an explicit super object creation, by simply accessing an attribute on the super type directly, which will automatically apply the class and instance to perform the proper lookup. The following example demonstrates the use of this. :: class A(object): def f(self): return 'A' class B(A): def f(self): return 'B' + super.f() class C(A): def f(self): return 'C' + super.f() class D(B, C): def f(self): return 'D' + super.f() assert D().f() == 'DBCA' The proposal adds a dynamic attribute lookup to the super type, which will automatically determine the proper class and instance parameters. Each super attribute lookup identifies these parameters and performs the super lookup on the instance, as the current super implementation does with the explicit invokation of a super object upon a class and instance. The enhancements to the super type will define a new __getattr__ classmethod of the super type, which must look backwards to the previous frame and locate the instance object. This can be naively determined by located the local named by the first argument to the function. Using super outside of a function where this is a valid lookup for the instance can be considered undocumented in its behavior. This special method will actually be invoked on attribute lookups to the super type itself, as opposed to super objects, as the current implementation works. This may pose open issues, which are detailed below. "Every class will gain a new special attribute, __super__, which refers to an instance of the associated super object for that class" In this capacity, the new super also acts as its own descriptor, create an instance-specific super upon lookup. Much of this was discussed in the thread of the python-dev list, "Fixing super anyone?" [1]_. Open Issues ----------- __call__ methods '''''''''''''''' Backward compatability of the super type API raises some issues. Names, the lookup of the __call__ of the super type itself, which means a conflict with doing an actual super lookup of the __call__ attribute. Namely, the following is ambiguous in the current proposal: :: super.__call__(arg) Which means the backward compatible API, which involves instansiating the super type, will either not be possible, because it will actually do a super lookup on the __call__ attribute, or there will be no way to perform a super lookup on the __call__ attribute. Both seem unacceptable, so any suggestions are welcome. Actually keeping the old super around in 2.x and creating a completely new super type seperately may be the best option. A future import or even a simple import in 2.x of the new super type from some builtin module may offer a way to choose which each module uses, even mixing uses by binding to different names. Such a builtin module might be called 'newsuper'. This module is also the reference implementation, which I will present below. super type's new getattr '''''''''''''''''''''''' To give the behavior needed, the super type either needs a way to do dynamic lookup of attributes on the super type object itself or define a metaclass for the builtin type. This author is unsure which, if either, is possible with C- defined types. When should we create __super__ attributes? ''''''''''''''''''''''''''''''''''''''''''' They either need to be created on class creation or on __super__ attribute lookup. For the second, they could be cached, of course, which seems like it may be the best idea, if implicit creation of a super object for every class is considered too much overhead. How does it work in inner functions? '''''''''''''''''''''''''''''''''''' If a method defines a function and super is used inside of it, how does this work? The frame looking and instance detection breaks here. However, if there can be some unambiguous way to use both the new super form and still be able to explicitly name the type and instance, I think its an acceptable tradeoff to simply be explicit in these cases, rather than add weird super-specific lookup rules in these cases. An example of such a problematic bit of code is: :: class B(A): def f(self): def g(): return super.f() return g() Should super actually become a keyword? ''''''''''''''''''''''''''''''''''''''' This would solve many of the problems and allow more direct implementation of super into the language proper. However, some are against the actual keyword- ization of super. The simplest solution is often the correct solution and the simplest solution may well not be adding additional keywords to the language when they are not needed. Still, it may solve many of the other open issues. Can we also allow super()? '''''''''''''''''''''''''' There is strong sentiment for and against this, but implementation and style concerns are obvious. Particularly, that its "magical" and that super() would differ from super.__call__(), being very unpythonic. Reference Implementation ======================== This implementation was a cooperative contribution in the original thread [1]_. :: #!/usr/bin/env python # # newsuper.py import sys class SuperMetaclass(type): def __getattr__(cls, attr): calling_frame = sys._getframe().f_back instance_name = calling_frame.f_code.co_varnames[0] instance = calling_frame.f_locals[instance_name] return getattr(instance.__super__, attr) class Super(object): __metaclass__ = SuperMetaclass def __init__(self, type, obj=None): if isinstance(obj, Super): obj = obj.__obj__ self.__type__ = type self.__obj__ = obj def __get__(self, obj, cls=None): if obj is None: raise Exception('only supports instances') else: return Super(self.__type__, obj) def __getattr__(self, attr): mro = iter(self.__obj__.__class__.__mro__) for cls in mro: if cls is self.__type__: break for cls in mro: if attr in cls.__dict__: x = cls.__dict__[attr] if hasattr(x, '__get__'): x = x.__get__(self, cls) return x raise AttributeError, attr class autosuper(type): def __init__(cls, name, bases, clsdict): cls.__super__ = Super(cls) if __name__ == '__main__': class A(object): __metaclass__ = autosuper def f(self): return 'A' class B(A): def f(self): return 'B' + Super.f() class C(A): def f(self): return 'C' + Super.f() class D(B, C): def f(self, arg=None): var = None return 'D' + Super.f() assert D().f() == 'DBCA' Alternative Proposals ===================== No Changes ---------- Although its always attractive to just keep things how they are, people have sought a change in the usage of super calling for some time, and for good reason, all mentioned previously. * Decoupling from the class name (which might not even be bound to the right class anymore! [2]_) * Simpler looking, cleaner super calls would be better super(__this_class__, self) --------------------------- This is nearly an anti-proposal, as it basically relies on the acceptance of the __this_class__ PEP, which proposes a special name that would always be bound to the class within which it is used. If that is accepted, __this_class__ could simply be used instead of the class' name explicitly, solving the name binding issues [2]_. self.__super__.foo(*args) ------------------------- The __super__ attribute is mentioned in this PEP in several places, and could be a candidate for the completel solution, actually using it explicitly instead of any super usage directly. However, double-underscore names are usually an internal detail, and attempted to be kept out of everyday code. super(self, *args) or __super__(self, *args) -------------------------------------------- This solution only solves the problem of the type indication, does not handle differently named super methods, and is explicit about the name of the instance. It is less flexable without being able to enacted on other method names, in cases where that is needed. One use case this fails is where a base- class has a factory classmethod and a subclass has two factory classmethods, both of which needing to properly make super calls to the one in the base- class. super.foo(self, *args) ---------------------- This variation actually eliminates the problems with locating the proper instance, and if any of the alternatives were pushed into the spotlight, I would want it to be this one. super or super() ---------------- This proposal leaves no room for different names, signatures, or application to other classes, or instances. A way to allow some similar use alongside the normal proposal would be favorable, encouraging good design of multiple inheritence trees and compatible methods. History ======= 29-Apr-2007 - Changed title from "Super As A Keyword" to "New Super" - Updated much of the language and added a terminology section for clarification in confusing places. - Added reference implementation and history sections. References ========== .. [1] Fixing super anyone? (http://mail.python.org/pipermail/python-3000/2007-April/006667.html) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From guido at python.org Mon Apr 30 03:26:45 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 29 Apr 2007 18:26:45 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <46353440.5060402@acm.org> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com> <46353440.5060402@acm.org> Message-ID: On 4/29/07, Talin wrote: > Guido van Rossum wrote: > > Maybe we should stop trying to capture radically different > > mathematical number systems using classes or types, and limit > > ourselves to capturing the systems one learns in high school: C, R, Q, > > Z, and (perhaps) N (really N0). The concrete types would be complex <: > > C, float<:R, Decimal<:R, int<:Z. NumPy would have many more. One could > > argue that float and Decimal are <:Q, but I'm not sure if that makes > > things better pragmatically; I guess I'm coming from the old Algol > > school where float was actually called real (and in retrospect I wish > > I'd called it that in Python). I'd rather reserve membership of Q for > > an infinite precision rational type (which many people have > > independently implemented). > > I haven't really been following this discussion, given my lack of > understanding of the issues involved, but I want to make one observation > about the discussion. > > Normally, when someone suggests an idea for a major addition to Python > of this scope, the standard response is that they should go develop it > as a 3rd-party package and see if becomes popular, and if it does it can > be considered for inclusion in the standard library. > > Unfortunately, we are somewhat constrained in this case, because we're > talking about altering the behavior of some fairly fundamental built-in > types - which means that it's going to be hard to implement it as an > add-on library. Not entirely true in the latest incarnation -- the proposed overloading of isinstance() and issubclass() will make it possible to add Ring-ness to float and int as an afterthough from a user-defined class. > And yet, it seems to me that this particular set of features really does > need a long gestation period compared to some others that we've > discussed. Most of the 3000-series PEPs are really about a fairly small > set of base decisions. Even the long PEPs are more about descriptions of > the logical consequences of those decisions than about the decisions > themselves. The ABC PEPs are different, in that they are standardizing a > whole slew of things all at once. Moreover, I think there is a real > danger here of a kind of ivory-tower decision making, isolated from > day-to-day writing of applications to keep them grounded. > > The question of whether to limit to the "high school" number classes, or > to go with the more mathematically abstract set of things is a decision > which, it seems to me, ought to be made in the context of actual use > cases, rather than abstract theorizing about use cases. (I'm also > generally supportive about copying what other languages have done, on > the theory that they too have been tested by real-world use.) If that's the criterion (and I agree) it should be fairly obvious by now that Ring and MonoidUnderPlus and everything in between should be cast out, and we should stick with high school math. (Just note that Travis Oliphant, Mr. NumPy Himself, pretty much confessed being confused by the algebra stuff). > If it were technically possible, I would recommend that this PEP have to > run the same gauntlet that any other large library addition would, which > is to go through a long period of community feedback and criticism, > during which a large number of people actually attempt to use the > feature for real work. I also think, in this case, that the special > power of "core python developer fiat" should not be invoked unless it > has to be, because I don't think that there is a firm basis for making > such a judgment yet. > > I would also suggest that some thought be given to ways to allow for > experimentation with different variations of this feature. If it is not > possible to make these numeric classes definable in Python at runtime, > then perhaps it is possible instead to allow for custom builds of the > Python 3000 executable with different arrangements and configurations of > these built-in classes. So how about we reduce the scope of our (!) PEP (or perhaps of a new one) to two items: (a) add @abstractmethod, and (b) overload isinstance() and issubclass()? Library authors can do everything they want with those, and we can always add a specific set of ABCs for containers and/or numbers later in the 3.0 development cycle. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From collinw at gmail.com Mon Apr 30 03:40:41 2007 From: collinw at gmail.com (Collin Winter) Date: Sun, 29 Apr 2007 18:40:41 -0700 Subject: [Python-3000] Traits/roles instead of ABCs Message-ID: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> [cc:-numpy] On 4/29/07, Steven Bethard wrote: > On 4/29/07, Talin wrote: > > If it were technically possible, I would recommend that this PEP have to > > run the same gauntlet that any other large library addition would, which > > is to go through a long period of community feedback and criticism, > > during which a large number of people actually attempt to use the > > feature for real work. > > This sounds like a pretty good reason to add __isinstance__() and > __issubclass__(). Then the various ABCs can be distributed as > third-party modules but can still make sure that things like > isinstance(42, Real) and issubclass(int, Complex) are True (or > whatever other assertions people want to make). To me, having to mess with something as fundamental as isinstance() and issubclass() sounds like a pretty good reason to prefer a different solution over ABCs. The mechanism I'm most familiar with for solving this problem (which, unless I've missed something, is, "how do I make sure this object does what I expect?") is Perl 6's roles system; if you know about Squeak Smalltalk's "traits" system, you're on the same page. (I discussed this with Jeffery Yasskin at lunch the other day, and he seemed interested, so here follows a medium-length explanation of roles/traits.) The idea is that we leave the traditional class system to manage an object's implementation and use a more-or-less orthogonal system to manage that same object's behavior. Quoting from an article on Perl 6's roles, """ A role is a named collection of behavior ? a set of methods identified by a unique name. This resembles a class or a type, in that referring to the role refers to the combined set of behaviors, but it is more general than a class and more concrete than a type. Put another way, a role is an assertion about a set of capabilities. """ [http://www.oreillynet.com/onlamp/blog/2006/08/roles_composable_units_of_obje.html] The key part of traits/roles is that, because the system is separate from classes, you can do runtime role composition without a) mucking with __bases__, or b) making isinstance() and issubclass() squishy and ill-defined. By "runtime role composition", I mean it would be possible to do something like this at runtime: py> int.implements(Ring) or (depending on your spelling preferences) py> Ring.implemented_by(int) That is, it would be possible to distribute Jeffery's numeric kinds as a third-party library and still have them affect the built-in numeric types. Now, how do roles/traits differ from interfaces? The primary distinction is that interfaces only define behavior, whereas roles can provide a dummy implementation. It would be possible for an Equality role to be defined like this (strawman syntax): class Equality(Role): def __eq__(self, other): return not self != other def __ne__(self, other): return not self == other How would all this work in Python? Here's a collection of strawman proposals: 1. For declaring roles statically, either class decorators or class arguments could be used: @does(Complex) class A: ... class A(does=Complex): ... 2. I gave a strawman syntax proposal for runtime role composition above (modulo the method name): "Ring.done_by(int)" or "int.does(Ring)". (Perl 6 speaks in terms of "an object/class *does* a role".) Perl 6 also allows individual instances to do different roles than their class does, but I'm not sure we want to go that far. 3. For querying role performance, something like this could work: "isdoneby(x, Ring)". If class decorators and the "role.method(object)" spellings were used, I think this could probably be issued as a third-party package with no need to modify the interpreter at all. For the interested, here's some more reading about roles/traits in roughly descending order of readability: http://www.perlmonks.org/?node_id=384858 http://www.oreillynet.com/onlamp/blog/2006/08/roles_composable_units_of_obje.html http://www.iam.unibe.ch/~scg/Archive/Papers/Scha03aTraits.pdf If there's interest in this, I could probably whip up a PEP before the deadline. Collin Winter From greg.ewing at canterbury.ac.nz Mon Apr 30 03:48:45 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 30 Apr 2007 13:48:45 +1200 Subject: [Python-3000] Could isinstance/issubclass overriding be dangerous? In-Reply-To: References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com> <46353440.5060402@acm.org> Message-ID: <46354AFD.5070702@canterbury.ac.nz> Currently, isinstance(x, C) implies that the C struct layout of x is compatible with that defined by C. I'm worried that changing this could cause difficulties for extension modules that use PyObject_IsInstance do decide what to do with things, e.g. in a binary operator method. -- Greg From jimjjewett at gmail.com Mon Apr 30 03:51:52 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 29 Apr 2007 21:51:52 -0400 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com> Message-ID: On 4/29/07, Guido van Rossum wrote: > Hmm... Maybe the conclusion to draw from this is that we shouldn't > make Ring a class? Maybe it ought to be a metaclass, so we could ask > isinstance(Complex, Ring)? Yes; all the ABCs are assertions about the class. (Zope interfaces do support instance-specific interfaces, which has been brought up as a relative weakness of ABCs.) The only thing two subclasses of an *Abstract* class need to have in common is that they both (independently) meet the requirements of the ABC. If not for complexity of implementation, that would be better described as a common metaclass. Using a metaclass would also solve the "when to gripe" issue; the metaclass would gripe if it couldn't make every method concrete. If this just used the standard metaclass machinery, then it would mean a much deeper metaclass hierarchy than we're used to; MutableSet would a have highly dervived metaclass. > The more I think about it, it sounds like the right thing to do. To > take PartiallyOrdered (let's say PO for brevity) as an example, the > Set class should specify PO as a metaclass. The PO metaclass could > require that the class implement __lt__ and __le__. If it found a > class that didn't implement them, it could make the class abstract by > adding the missing methods to its __abstractmethods__ attribute. Or by making it a sub(meta)class, instead of a (regular instance) class. > if it found that the class implemented one but not the other, it could > inject a default implementation of the other in terms of the one and > __eq__. This also allows greater freedom in specifying which subsets of methods must be defined. > Now, you could argue that Complex should also be a metaclass. While > that may mathematically meaningful (for all I know there are people > doing complex number theory using Complex[Z/n]), for Python's numeric > classes I think it's better to make Complex a regular class > representing all the usual complex numbers (i.e. a pair of Real > numbers). complex already meets that need. Complex would be the metaclass representing the restrictions on the class, so that independing implementations wouldn't have to fake-inherit from complex. > I expect that the complex subclasses used in practice are > all happy under mixed arithmetic using the usual definition of mixed > arithmetic: convert both arguments to a common base class and compute > the operation in that domain. It is reasonable to insist that all Complex classes have a way to tranform their instances into (builtin) complex instances, if only as a final fallback. There is no need for complex to be a base class. -jJ From jimjjewett at gmail.com Mon Apr 30 04:01:57 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 29 Apr 2007 22:01:57 -0400 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com> <46353440.5060402@acm.org> Message-ID: On 4/29/07, Steven Bethard wrote: > On 4/29/07, Talin wrote: > > If it were technically possible, I would recommend that this PEP have to > > run the same gauntlet that any other large library addition would, which > > is to go through a long period of community feedback and criticism, > > during which a large number of people actually attempt to use the > > feature for real work. > This sounds like a pretty good reason to add __isinstance__() and > __issubclass__(). Then the various ABCs can be distributed as > third-party modules but can still make sure that things like > isinstance(42, Real) and issubclass(int, Complex) are True (or > whatever other assertions people want to make). Or isexample, so that we aren't locked into implementing ABCs as base classes. def isexample(val, ABC): return ABC.__example__(val) class ABC(object): def __example__(cls, val): if val in cls.__instance: return True if val in cls.__non_instance: return False for instclass in type(val).__mro__: if instclass in __class: return True if instclass in __non_class: return False return False ... methods to register classes, unregister subclasses, etc -jJ From guido at python.org Mon Apr 30 04:31:53 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 29 Apr 2007 19:31:53 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com> <46353440.5060402@acm.org> Message-ID: On 4/29/07, Jim Jewett wrote: > Or isexample, so that we aren't locked into implementing ABCs as base classes. You don't have to use the feature even if it exists. :-) I think there are good reasons to support overriding isinstance/issubclass beyond ABCs. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Apr 30 05:02:49 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 29 Apr 2007 20:02:49 -0700 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com> Message-ID: On 4/29/07, Jim Jewett wrote: > On 4/29/07, Guido van Rossum wrote: > > Hmm... Maybe the conclusion to draw from this is that we shouldn't > > make Ring a class? Maybe it ought to be a metaclass, so we could ask > > isinstance(Complex, Ring)? > > Yes; all the ABCs are assertions about the class. I don't think so. Many are quite useful for introspection of instances as well, e.g. Hashable/Iterable (the whole "One Trick Ponies" section) as well as the distinction between Sequence and Mapping. It's the binary operations where the class comes into play. > (Zope interfaces do > support instance-specific interfaces, which has been brought up as a > relative weakness of ABCs.) > > The only thing two subclasses of an *Abstract* class need to have in > common is that they both (independently) meet the requirements of the > ABC. If not for complexity of implementation, that would be better > described as a common metaclass. Again, not so fast; it depends. The way the Set section of the PEP is currently written, all sets are comparable (in the subset/superset sense) to all other sets, and for ComposableSet instances the union, intersection and both types of differences are also computable across class boundaries. > Using a metaclass would also solve the "when to gripe" issue; the > metaclass would gripe if it couldn't make every method concrete. If > this just used the standard metaclass machinery, then it would mean a > much deeper metaclass hierarchy than we're used to; MutableSet would a > have highly dervived metaclass. I think you're going way too fast here. > > The more I think about it, it sounds like the right thing to do. To > > take PartiallyOrdered (let's say PO for brevity) as an example, the > > Set class should specify PO as a metaclass. The PO metaclass could > > require that the class implement __lt__ and __le__. If it found a > > class that didn't implement them, it could make the class abstract by > > adding the missing methods to its __abstractmethods__ attribute. > > Or by making it a sub(meta)class, instead of a (regular instance) class. That makes no sense. Deciding on the fly whether something should be a class or a metaclass sounds like a fine recipe for end-user confusion. > > if it found that the class implemented one but not the other, it could > > inject a default implementation of the other in terms of the one and > > __eq__. > > This also allows greater freedom in specifying which subsets of > methods must be defined. > > > Now, you could argue that Complex should also be a metaclass. While > > that may mathematically meaningful (for all I know there are people > > doing complex number theory using Complex[Z/n]), for Python's numeric > > classes I think it's better to make Complex a regular class > > representing all the usual complex numbers (i.e. a pair of Real > > numbers). > > complex already meets that need. Complex would be the metaclass > representing the restrictions on the class, so that independing > implementations wouldn't have to fake-inherit from complex. I was thinking of other representations of complex numbers as found e.g. in numpy. These vary mostly by using fewer (or more?) bits for the real and imag parts. They can't realistically subclass complex, as their implementation is independent; they should subclass Complex, to indicate that they implement the Complex API. I really think you're going too far with the metaclass idea. Now, if we had parameterizable types (for which I've proposed a notation, e.g. list[int] would be a list of integers, and Mapping[String, Real] would be a mapping from strings to real numbers; but I don't expect this to be in py3k, as it needs more experimentation), Complex might be a parameterizable type, and e.g. the current concrete complex type could be equated to Complex[float]; but without that, I think it's fine to see Complex as the Abstract Base Class and complex as one concrete representation. > > I expect that the complex subclasses used in practice are > > all happy under mixed arithmetic using the usual definition of mixed > > arithmetic: convert both arguments to a common base class and compute > > the operation in that domain. > > It is reasonable to insist that all Complex classes have a way to > tranform their instances into (builtin) complex instances, if only as > a final fallback. There is no need for complex to be a base class. I agree complex shouldn't be a base class (apologies if I implied that by using lowercase) but I still think Complex should be a base class. To be honest, I'm not sure what should happen with mixed operations between classes that only have an abstract common base class. The normal approach for binary operations is that each side gets a try. For pairs like int+float this is easy; int.__add__ returns NotImplemented in this case, and then float.__radd__ is called which converts the first argument to float and returns a float. For pairs like numpy's complex 32-bit float and numpy's complex 64-bit float it should also be easy (numpy is aware of both types and hence always gets to choose); and for numpy's complex combined with the built-in complex it's easy enough too (again, numpy always gets to choose, this time because the built-in complex doesn't know about numpy). But what if I wrote my own complex type based on decimal.Decimal, and I encountered a numpy complex? numpy doesn't know about my type and hence passes the ball to me; but perhaps I don't know about numpy either, and then a TypeError will be raised. So, to be a good citizen, I could check if the other arg was a Complex of unknown provenance, and then I could convert to the built-in complex and pass the ball to that type. But if all good citizens lived by that rule (and it *appears* to be a reasonable rule), then numpy, being the ultimate good citizen, would also convert *my* type to the built-in complex. But that would mean that if my type *did* know about numpy (but numpy didn't know about mine), I wouldn't get to choose what to do in half the cases (if the numpy instance was on the left, it would get and take the first opportunity). Perhaps we need to extend the built-in operation processing so that if both sides return NotImplemented, before raising TypeError, we look for some common base type implementing the same operation. The abstract Complex type could provide abstract implementations of the binary operators that would convert their arguments to the concrete complex type and compute the result that way. (Hah! Another use case for abstract methods with a useful implementation! :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jimjjewett at gmail.com Mon Apr 30 05:29:25 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 29 Apr 2007 23:29:25 -0400 Subject: [Python-3000] PEP 30XZ: Simplified Parsing Message-ID: PEP: 30xz Title: Simplified Parsing Version: $Revision$ Last-Modified: $Date$ Author: Jim J. Jewett Status: Draft Type: Standards Track Content-Type: text/plain Created: 29-Apr-2007 Post-History: 29-Apr-2007 Abstract Python initially inherited its parsing from C. While this has been generally useful, there are some remnants which have been less useful for python, and should be eliminated. + Implicit String concatenation + Line continuation with "\" + 034 as an octal number (== decimal 28). Note that this is listed only for completeness; the decision to raise an Exception for leading zeros has already been made in the context of PEP XXX, about adding a binary literal. Rationale for Removing Implicit String Concatenation Implicit String concatentation can lead to confusing, or even silent, errors. [1] def f(arg1, arg2=None): pass f("abc" "def") # forgot the comma, no warning ... # silently becomes f("abcdef", None) or, using the scons build framework, sourceFiles = [ 'foo.c', 'bar.c', #...many lines omitted... 'q1000x.c'] It's a common mistake to leave off a comma, and then scons complains that it can't find 'foo.cbar.c'. This is pretty bewildering behavior even if you *are* a Python programmer, and not everyone here is. Note that in C, the implicit concatenation is more justified; there is no other way to join strings without (at least) a function call. In Python, strings are objects which support the __add__ operator; it is possible to write: "abc" + "def" Because these are literals, this addition can still be optimized away by the compiler. Guido indicated [2] that this change should be handled by PEP, because there were a few edge cases with other string operators, such as the %. The resolution is to treat them the same as today. ("abc %s def" + "ghi" % var) # fails like today. # raises TypeError because of # precedence. (% before +) ("abc" + "def %s ghi" % var) # works like today; precedence makes # the optimization more difficult to # recognize, but does not change the # semantics. ("abc %s def" + "ghi") % var # works like today, because of # precedence: () before % # CPython compiler can already # add the literals at compile-time. Rationale for Removing Explicit Line Continuation A terminal "\" indicates that the logical line is continued on the following physical line (after whitespace). Note that a non-terminal "\" does not have this meaning, even if the only additional characters are invisible whitespace. (Python depends heavily on *visible* whitespace at the beginning of a line; it does not otherwise depend on *invisible* terminal whitespace.) Adding whitespace after a "\" will typically cause a syntax error rather than a silent bug, but it still isn't desirable. The reason to keep "\" is that occasionally code looks better with a "\" than with a () pair. assert True, ( "This Paren is goofy") But realistically, that paren is no worse than a "\". The only advantage of "\" is that it is slightly more familiar to users of C-based languages. These same languages all also support line continuation with (), so reading code will not be a problem, and there will be one less rule to learn for people entirely new to programming. Rationale for Removing Implicit Octal Literals This decision should be covered by PEP ???, on numeric literals. It is mentioned here only for completeness. C treats integers beginning with "0" as octal, rather than decimal. Historically, Python has inherited this usage. This has caused quite a few annoying bugs for people who forgot the rule, and tried to line up their constants. a = 123 b = 024 # really only 20, because octal c = 245 In Python 3.0, the second line will instead raise a SyntaxError, because of the ambiguity. Instead, the line should be written as in one of the following ways: b = 24 # PEP 8 b = 24 # columns line up, for quick scanning b = 0t24 # really did want an Octal! References [1] Implicit String Concatenation, Jewett, Orendorff http://mail.python.org/pipermail/python-ideas/2007-April/000397.html [2] PEP 12, Sample reStructuredText PEP Template, Goodger, Warsaw http://www.python.org/peps/pep-0012 [3] http://www.opencontent.org/openpub/ Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From jimjjewett at gmail.com Mon Apr 30 05:52:32 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 29 Apr 2007 23:52:32 -0400 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com> Message-ID: On 4/29/07, Guido van Rossum wrote: > On 4/29/07, Jim Jewett wrote: > > On 4/29/07, Guido van Rossum wrote: > > > Hmm... Maybe the conclusion to draw from this is that we shouldn't > > > make Ring a class? Maybe it ought to be a metaclass, so we could ask > > > isinstance(Complex, Ring)? > > Yes; all the ABCs are assertions about the class. > I don't think so. Many are quite useful for introspection of instances > as well, e.g. Hashable/Iterable (the whole "One Trick Ponies" section) > as well as the distinction between Sequence and Mapping. It's the > binary operations where the class comes into play. I think those are one of the reasons it seemed like we should use inheritance. Treating them as assertions about each instance makes sense -- but it makes just as much sense to treat them as assertions about the class. (Is it meaningful to ask about the size of this? Well, for objects of this class, it is..) > > The only thing two subclasses of an *Abstract* class need to have in > > common is that they both (independently) meet the requirements of the > > ABC. If not for complexity of implementation, that would be better > > described as a common metaclass. > Again, not so fast; it depends. The way the Set section of the PEP is > currently written, all sets are comparable (in the subset/superset > sense) to all other sets, and for ComposableSet instances the union, > intersection and both types of differences are also computable across > class boundaries. That is an additional constraint which the Set metaclass imposes -- and it does this by effectively coercing instances of both Set classes to (buitin) frozenset instances to create the return value. (It doesn't actually create the intermediate frozenset instance, but mySet() | mySet() will return a frozenset rather than a MySet. > > Using a metaclass would also solve the "when to gripe" issue; the > > metaclass would gripe if it couldn't make every method concrete. If > > this just used the standard metaclass machinery, then it would mean a > > much deeper metaclass hierarchy than we're used to; MutableSet would a > > have highly dervived metaclass. > I think you're going way too fast here. To be more precise, classes implementing MutabSet would have MutableSet as a metaclass, which would mean (through inheritance) than they would also have ComposableSet, Set, Sized, Iterable, and PartiallyOrdered as metaclasses. This is legal today, but *I* haven't seen code in the wild with a metaclass hierarchy that deep. > > > The more I think about it, it sounds like the right thing to do. To > > > take PartiallyOrdered (let's say PO for brevity) as an example, the > > > Set class should specify PO as a metaclass. The PO metaclass could > > > require that the class implement __lt__ and __le__. If it found a > > > class that didn't implement them, it could make the class abstract by > > > adding the missing methods to its __abstractmethods__ attribute. > > Or by making it a sub(meta)class, instead of a (regular instance) class. > That makes no sense. Deciding on the fly whether something should be a > class or a metaclass sounds like a fine recipe for end-user confusion. Perhaps. But how is that different from deciding on the fly whether the class will be instantiable? Either way, the user does need to keep track of whether it is abstract; the difference is that when inheriting, they would have to say (metaclass=ABC) # I know it is an abstract class instead of (ABC) # Might be a regular class, but I can never instantiate it directly > I was thinking of other representations of complex numbers as found > e.g. in numpy. These vary mostly by using fewer (or more?) bits for > the real and imag parts. They can't realistically subclass complex, as > their implementation is independent; they should subclass Complex, to > indicate that they implement the Complex API. I really think you're > going too far with the metaclass idea. Or they could have Compex as a metaclass, which would serve the same introspective needs. > > > I expect that the complex subclasses used in practice are > > > all happy under mixed arithmetic using the usual definition of mixed > > > arithmetic: convert both arguments to a common base class and compute > > > the operation in that domain. > > > > It is reasonable to insist that all Complex classes have a way to > > tranform their instances into (builtin) complex instances, if only as > > a final fallback. There is no need for complex to be a base class. > I agree complex shouldn't be a base class (apologies if I implied that > by using lowercase) but I still think Complex should be a base class. Why? What do you get by inheriting from it, that you couldn't get by letting it inject any missing methods? > take the first opportunity). Perhaps we need to extend the built-in > operation processing so that if both sides return NotImplemented, > before raising TypeError, we look for some common base type > implementing the same operation. The abstract Complex type could > provide abstract implementations of the binary operators that would > convert their arguments to the concrete complex type and compute the > result that way. (Hah! Another use case for abstract methods with a > useful implementation! :-) That seems sensible -- but you could also do just do it in the _r* method, since the regular method already returned NotImplemented. So Complex could inject a __radd__ method that tried self.__add__(complex(other)) # since addition is commutative for Complex If the method weren't in either class, I would expect it to be a more general fallback, like "if we don't have __lt__, try __cmp__" -jJ From guido at python.org Mon Apr 30 05:58:28 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 29 Apr 2007 20:58:28 -0700 Subject: [Python-3000] [Python-Dev] Pre-pre PEP for 'super' keyword In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5FF1ED71@au3010avexu1.global.avaya.com> References: <2773CAC687FD5F4689F526998C7E4E5FF1ED71@au3010avexu1.global.avaya.com> Message-ID: On 4/29/07, Delaney, Timothy (Tim) wrote: > I think the current PEP draft is way too complicated - I don't think > there's any need for descriptors, etc. I think we can make things work > in the following way: > > 1. When a method is defined, the class is bound to it via an attribute > (which in my version is called func_class). In Py3k all the func_XXX attrs are renamed __XXX__, so this would be __class__; but that's a name reserved for something else, so it would need to be something else. E.g. __containing_class__. Also, this would have to be done when the class is defined; when the method is being defined the class doesn't exist yet. > 2. Every non-static method has an implicit cell variable called 'super'. I think you're using 'cell' in a different sense than it is normally used in Python's implementation. What you are looking for is called a local variable (I deduce this from your initialization of the "cell" with something computed from the first argument). A 'cell' (as I introduced in my original back-of-the-envelop proposal; apparently you totally missed this reference) is a special kind of object containing a reference to another object and used to implement nonlocal variable references (the Python 2 read-only nonlocals as well as the Python 3 writable nonlocals). The cell is initialized in the surrounding scope and attached to the function object (as func_closure in 2.x, or __closure__ in 3.0). When the function is called, a reference to the cell is stored in the call frame. If anything in your proposal resembles a cell (in the sense that I and other Pythonistas mean it), it's the func_class attribute. In fact, in this sense our proposals are equivalent modulo slight implementation details; I proposed using a cell because there's an existing mechanism to get these mapped into a local variable, and because cells are cheaper than function attributes -- the first function attribute requires creating a dict, which is one of the larger built-in objects. > This would preferably not be able to be rebound. I also think it would > be beneficial if the first parameter to the method couldn't be rebound > (point 7 in my original email in this thread). The latter part sounds irrelevant to me. > 3. When a method is entered, the 'super' cell variable is populated by a > call equivalent to: > > super = __builtin__.super(func_class, first_parameter) > > This would result in 'super' being a constant object, within the scope > of the currently-executing method. 'keyword' was perhaps too strong - I > was thinking this would only need to be done if 'super' were actually > used, which would be easier to determine if 'super' actually were a > keyword. This could still be done by not emitting the above call unless > the 'super' cell variable were ever actually used. This sounds about right to me, with the strong preference that super *should* be made a keyword, and the built-in of the same name renamed to __super__, as it has a somewhat similar relationship to the super keyword as the built-in __import__ has to the import keyword. > I've done bytecode-hacky stuff to do the equivalent of the above (as > much as I've been able to), but a real implementation would just emit > the correct bytecode (or java bytecode, or whatever) in the compiled > code object. Shouldn't be too hard for someone who's hacked Python/compile.c before. > The issue of super() vs. super.__call__() ambiguity - I'll need to look > at that when I get home. Sounds irrelevant -- if super is equivalent to __builtin__.__super__(, ) then super never gets called; the only thing ever done to it (in the normal course of things) is to request an attribute of it. > I'm a strong -1 against super() automatically passing the parameters > that were passed to the currently-executing method. Same here. There seems to be the remaining controversy that there are three forms of super calls currently in use: super(ThisClass, self).method(...) # in a regular method super(ThisClass, cls).method(...) # in a class method super(ThisClass).method(...) # ??? The first two are handled by your "use the first argument" approach. The third exists so that you can create an "unbound" super instance which is useful for the oft-misunderstood autosuper example in my "descrintro" essay: http://www.python.org/download/releases/2.2.3/descrintro/#metaclass_examples . But since the latter is the hack that we're trying to replace with a proper implementation here, I suspect we can get away with only supporting the first two forms (and the third form is still supported using __builtin__.__super__). IOW I think you're on to something. Keep up the good word. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Apr 30 06:05:23 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 29 Apr 2007 21:05:23 -0700 Subject: [Python-3000] Could isinstance/issubclass overriding be dangerous? In-Reply-To: <46354AFD.5070702@canterbury.ac.nz> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com> <46353440.5060402@acm.org> <46354AFD.5070702@canterbury.ac.nz> Message-ID: On 4/29/07, Greg Ewing wrote: > Currently, isinstance(x, C) implies that the > C struct layout of x is compatible with that > defined by C. I'm worried that changing this > could cause difficulties for extension modules > that use PyObject_IsInstance do decide what > to do with things, e.g. in a binary operator > method. Unless they were looking at classic classes, why wouldn't they be using the specific Py_Check() macros? If you want this argument to be taken serious, use Google code search (code.google.com) or a similar source code search engine to find examples that would break. Again, the only envisioned overloading is where the 2nd argument is an ABC. Also, note that PyObject_IsInstance already returns abstract results when the first argument is a proxy instance. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Mon Apr 30 06:05:28 2007 From: python at rcn.com (Raymond Hettinger) Date: Sun, 29 Apr 2007 21:05:28 -0700 Subject: [Python-3000] Traits/roles instead of ABCs References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> Message-ID: <014201c78adc$ca70d960$f101a8c0@RaymondLaptop1> [Collin Winter] > Put another way, a role is an assertion about a set of capabilities. . . . > If there's interest in this, I could probably whip up a PEP before the deadline. +100 I'm very interested in seeing a lighter weight alternative to abc.py that: 1) is dynamic 2) doesn't require inheritance to work 3) doesn't require mucking with isinstance or other existing mechansims 4) makes a limited, useful set of assertions rather than broadly covering a whole API. 5) that leaves the notion of duck-typing as the rule rather than the exception 6) that doesn't freeze all of the key APIs in concrete I'm concerned that the current ABC proposal will quickly evolve from optional to required and create somewhat somewhat java-esque landscape where inheritance and full-specification are the order of the day. IMHO, the ABC approach is using a cannon to shoot a mosquito. My day-to-day problems are much smaller are could be solved by a metadata attribute or a role/trait solution: * knowing whether a __getitem__ method implements a mapping or a sequence * knowing whether an object can have more that one iterator (i.e a file has one but a list can have many) * knowing whether a sequence, file, cursor, etc is writable or just readonly. Raymond From guido at python.org Mon Apr 30 06:08:20 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 29 Apr 2007 21:08:20 -0700 Subject: [Python-3000] PEP 30XZ: Simplified Parsing In-Reply-To: References: Message-ID: [-python-dev] I think these should be two separate proposals, with more specific names (e.g. "remove implicit string concatenation" and "remove backslash continuation"). There's no need to mention the octal thing if it's already a separate PEP. On 4/29/07, Jim Jewett wrote: > PEP: 30xz > Title: Simplified Parsing > Version: $Revision$ > Last-Modified: $Date$ > Author: Jim J. Jewett > Status: Draft > Type: Standards Track > Content-Type: text/plain > Created: 29-Apr-2007 > Post-History: 29-Apr-2007 > > > Abstract > > Python initially inherited its parsing from C. While this has > been generally useful, there are some remnants which have been > less useful for python, and should be eliminated. > > + Implicit String concatenation > > + Line continuation with "\" > > + 034 as an octal number (== decimal 28). Note that this is > listed only for completeness; the decision to raise an > Exception for leading zeros has already been made in the > context of PEP XXX, about adding a binary literal. > > > Rationale for Removing Implicit String Concatenation > > Implicit String concatentation can lead to confusing, or even > silent, errors. [1] > > def f(arg1, arg2=None): pass > > f("abc" "def") # forgot the comma, no warning ... > # silently becomes f("abcdef", None) > > or, using the scons build framework, > > sourceFiles = [ > 'foo.c', > 'bar.c', > #...many lines omitted... > 'q1000x.c'] > > It's a common mistake to leave off a comma, and then scons complains > that it can't find 'foo.cbar.c'. This is pretty bewildering behavior > even if you *are* a Python programmer, and not everyone here is. > > Note that in C, the implicit concatenation is more justified; there > is no other way to join strings without (at least) a function call. > > In Python, strings are objects which support the __add__ operator; > it is possible to write: > > "abc" + "def" > > Because these are literals, this addition can still be optimized > away by the compiler. > > Guido indicated [2] that this change should be handled by PEP, because > there were a few edge cases with other string operators, such as the %. > The resolution is to treat them the same as today. > > ("abc %s def" + "ghi" % var) # fails like today. > # raises TypeError because of > # precedence. (% before +) > > ("abc" + "def %s ghi" % var) # works like today; precedence makes > # the optimization more difficult to > # recognize, but does not change the > # semantics. > > ("abc %s def" + "ghi") % var # works like today, because of > # precedence: () before % > # CPython compiler can already > # add the literals at compile-time. > > > Rationale for Removing Explicit Line Continuation > > A terminal "\" indicates that the logical line is continued on the > following physical line (after whitespace). > > Note that a non-terminal "\" does not have this meaning, even if the > only additional characters are invisible whitespace. (Python depends > heavily on *visible* whitespace at the beginning of a line; it does > not otherwise depend on *invisible* terminal whitespace.) Adding > whitespace after a "\" will typically cause a syntax error rather > than a silent bug, but it still isn't desirable. > > The reason to keep "\" is that occasionally code looks better with > a "\" than with a () pair. > > assert True, ( > "This Paren is goofy") > > But realistically, that paren is no worse than a "\". The only > advantage of "\" is that it is slightly more familiar to users of > C-based languages. These same languages all also support line > continuation with (), so reading code will not be a problem, and > there will be one less rule to learn for people entirely new to > programming. > > > Rationale for Removing Implicit Octal Literals > > This decision should be covered by PEP ???, on numeric literals. > It is mentioned here only for completeness. > > C treats integers beginning with "0" as octal, rather than decimal. > Historically, Python has inherited this usage. This has caused > quite a few annoying bugs for people who forgot the rule, and > tried to line up their constants. > > a = 123 > b = 024 # really only 20, because octal > c = 245 > > In Python 3.0, the second line will instead raise a SyntaxError, > because of the ambiguity. Instead, the line should be written > as in one of the following ways: > > b = 24 # PEP 8 > b = 24 # columns line up, for quick scanning > b = 0t24 # really did want an Octal! > > > References > > [1] Implicit String Concatenation, Jewett, Orendorff > http://mail.python.org/pipermail/python-ideas/2007-April/000397.html > > [2] PEP 12, Sample reStructuredText PEP Template, Goodger, Warsaw > http://www.python.org/peps/pep-0012 > > [3] http://www.opencontent.org/openpub/ > > > > Copyright > > This document has been placed in the public domain. > > > > Local Variables: > mode: indented-text > indent-tabs-mode: nil > sentence-end-double-space: t > fill-column: 70 > coding: utf-8 > End: > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Mon Apr 30 07:12:28 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 30 Apr 2007 07:12:28 +0200 Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD In-Reply-To: <46353E13.5080501@canterbury.ac.nz> References: <46327CC9.4040901@v.loewis.de> <4632DD83.2030301@v.loewis.de> <4633F918.1010205@canterbury.ac.nz> <46346DD9.1020101@v.loewis.de> <46353E13.5080501@canterbury.ac.nz> Message-ID: <46357ABC.10305@v.loewis.de> >> That doesn't work. For variable-sized objects, it would need >> to be ob_base.ob_base.ob_type. > > I was wondering about that. It seems that each level > of inheritance is going to add another level of > required member access. Or is it legal to short- > circuit all that and just cast directly to the > ultimate base object? As the PEP explains: you can cast a struct pointer to pointer to the first field. That works recursively: if the first field is a struct again, you can also cast to a pointer to that struct's first field. Regards, Martin From greg.ewing at canterbury.ac.nz Mon Apr 30 08:27:23 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 30 Apr 2007 18:27:23 +1200 Subject: [Python-3000] Could isinstance/issubclass overriding be dangerous? In-Reply-To: References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com> <46353440.5060402@acm.org> <46354AFD.5070702@canterbury.ac.nz> Message-ID: <46358C4B.2020803@canterbury.ac.nz> Guido van Rossum wrote: > Unless they were looking at classic classes, why wouldn't they be > using the specific Py_Check() macros? I'm thinking of Pyrex code. One of the goals of Pyrex is that you should be able to write it without needing to know about the Python/C API. One of the places that's not possible at the moment is in binary operator methods, where you need to write things like cdef class C: def __add(x, y): if PyObject_TypeCheck(x, C): # we're the left operand elif PyObject_TypeCheck(y, C): # we're the right operand I'd like to be able to provide optimised access to isinstance() so that you can think in Python instead of C and write def __add(x, y): if isinstance(x, C): # we're the left operand elif isinstance(y, C): # we're the right operand But it seems like isinstance() is already the wrong thing to use for this, and there is currently *no* Python-level function that does what is needed here. So can we please have another couple of functions that just do a simple, reliable concrete type test? -- Greg From tdelaney at avaya.com Mon Apr 30 04:48:26 2007 From: tdelaney at avaya.com (Delaney, Timothy (Tim)) Date: Mon, 30 Apr 2007 12:48:26 +1000 Subject: [Python-3000] [Python-Dev] Pre-pre PEP for 'super' keyword Message-ID: <2773CAC687FD5F4689F526998C7E4E5FF1ED71@au3010avexu1.global.avaya.com> Jim Jewett wrote: > On 4/29/07, Tim Delaney wrote: >> I've been intending to write up a PEP for fixing super, but I >> haven't had time to get to it. > > Calvin Spealman has the most recent draft. I hope he will incorporate > this into his draft. Sorry about this - wasn't receiving python-dev at home, so didn't realise Calvin had released the PEP. I think the current PEP draft is way too complicated - I don't think there's any need for descriptors, etc. I think we can make things work in the following way: 1. When a method is defined, the class is bound to it via an attribute (which in my version is called func_class). 2. Every non-static method has an implicit cell variable called 'super'. This would preferably not be able to be rebound. I also think it would be beneficial if the first parameter to the method couldn't be rebound (point 7 in my original email in this thread). 3. When a method is entered, the 'super' cell variable is populated by a call equivalent to: super = __builtin__.super(func_class, first_parameter) This would result in 'super' being a constant object, within the scope of the currently-executing method. 'keyword' was perhaps too strong - I was thinking this would only need to be done if 'super' were actually used, which would be easier to determine if 'super' actually were a keyword. This could still be done by not emitting the above call unless the 'super' cell variable were ever actually used. I've done bytecode-hacky stuff to do the equivalent of the above (as much as I've been able to), but a real implementation would just emit the correct bytecode (or java bytecode, or whatever) in the compiled code object. The issue of super() vs. super.__call__() ambiguity - I'll need to look at that when I get home. I'm a strong -1 against super() automatically passing the parameters that were passed to the currently-executing method. Tim Delaney From tdelaney at avaya.com Mon Apr 30 06:56:20 2007 From: tdelaney at avaya.com (Delaney, Timothy (Tim)) Date: Mon, 30 Apr 2007 14:56:20 +1000 Subject: [Python-3000] [Python-Dev] Pre-pre PEP for 'super' keyword Message-ID: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> Guido van Rossum wrote: >> 1. When a method is defined, the class is bound to it via an attribute >> (which in my version is called func_class). > In Py3k all the func_XXX attrs are renamed __XXX__, so this would be > __class__; but that's a name reserved for something else, so it would > need to be something else. E.g. __containing_class__. Yep - I've just used a placeholder name. > Also, this would have to be done when the class is defined; when the > method is being defined the class doesn't exist yet. Good point. Change that to "at the first opportunity" ;) >> 2. Every non-static method has an implicit cell variable called >> 'super'. > > I think you're using 'cell' in a different sense than it is normally > used in Python's implementation. What you are looking for is called a > local variable (I deduce this from your initialization of the "cell" > with something computed from the first argument). Actually, I think I'm using the terminology correctly - I'm talking about an entry in co_cellvars. Given the following class: class A(object): def f(self): super = super_factory() def inner(): return 'A' + super.f() print inner() the use of 'super' in both A.f and A.f.inner will produce an entry in A.f.func_code.co_cellvars and A.f.inner.func_code.co_freevars. What I'm proposing is that the `super = super_factory()` line be implicit in this case, resulting in the following code behaving identically: class A(object): def f(self): def inner(): return 'A' + super.f() print inner() >> The issue of super() vs. super.__call__() ambiguity - I'll need to >> look at that when I get home. > > Sounds irrelevant -- if super is equivalent to > __builtin__.__super__(, ) then super never gets > called; the only thing ever done to it (in the normal course of > things) is to request an attribute of it. Sorry - this is related to my proposal that the following two bits of code behave the same: class A(object): def f(self, *p, **kw): super.f(*p, **kw) class A(object): def f(self, *p, **kw): super(*p, **kw) But as has been pointed out, this creates an ambiguity with: class A(object): def f(self, *p, **kw): super.__call__(*p, **kw) so I want to see if I can resolve it. > super(ThisClass).method(...) # ??? > > The third exists so that you can create an "unbound" super instance > which is useful for the oft-misunderstood autosuper example in my > "descrintro" essay: > http://www.python.org/download/releases/2.2.3/descrintro/#metaclass_exam ples > . > > But since the latter is the hack that we're trying to replace with a > proper implementation here, I suspect we can get away with only > supporting the first two forms (and the third form is still supported > using __builtin__.__super__). Yep - that's my thought as well. I think it would be very rare to need super(ThisClass), although it makes some sense that that would be what super means in a static method ... Tim Delaney From tcdelaney at optusnet.com.au Mon Apr 30 13:38:30 2007 From: tcdelaney at optusnet.com.au (Tim Delaney) Date: Mon, 30 Apr 2007 21:38:30 +1000 Subject: [Python-3000] [Python-Dev] Pre-pre PEP for 'super' keyword References: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> Message-ID: <00d701c78b1c$143b3c40$0201a8c0@ryoko> From: "Delaney, Timothy (Tim)" > Sorry - this is related to my proposal that the following two bits of > code behave the same: > > class A(object): > def f(self, *p, **kw): > super.f(*p, **kw) > > class A(object): > def f(self, *p, **kw): > super(*p, **kw) > > But as has been pointed out, this creates an ambiguity with: > > class A(object): > def f(self, *p, **kw): > super.__call__(*p, **kw) > > so I want to see if I can resolve it. A 'super' instance would be callable, without being able to access it's __call__ method (because super.__call__ would refer to the base class method of that name). But I find I really don't care. The only place where that would really matter IMO is if you want to find out if a 'super' instance is callable. Calling a base class __call__ method would not be ambiguous - the following two classes would work the same: class A(object): def __call__(self, *p, **kw): return super.__call__(*p, **kw) class A(object): def __call__(self, *p, **kw): return super(*p, **kw) So, I guess my question is whether the most common case of calling the base class method with the same name is worth having some further syntactic sugar to avoid repetition? I think it is, but that would be your call Guido. Cheers, Tim Delaney From ironfroggy at gmail.com Mon Apr 30 14:29:22 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Mon, 30 Apr 2007 08:29:22 -0400 Subject: [Python-3000] [Python-Dev] Pre-pre PEP for 'super' keyword In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> References: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> Message-ID: <76fd5acf0704300529r1801bc6at9a2f3d5329d1ea0d@mail.gmail.com> On 4/30/07, Delaney, Timothy (Tim) wrote: > Guido van Rossum wrote: > > >> 1. When a method is defined, the class is bound to it via an > attribute > >> (which in my version is called func_class). > > > In Py3k all the func_XXX attrs are renamed __XXX__, so this would be > > __class__; but that's a name reserved for something else, so it would > > need to be something else. E.g. __containing_class__. > > Yep - I've just used a placeholder name. > > > Also, this would have to be done when the class is defined; when the > > method is being defined the class doesn't exist yet. > > Good point. Change that to "at the first opportunity" ;) > > >> 2. Every non-static method has an implicit cell variable called > >> 'super'. > > > > I think you're using 'cell' in a different sense than it is normally > > used in Python's implementation. What you are looking for is called a > > local variable (I deduce this from your initialization of the "cell" > > with something computed from the first argument). > > Actually, I think I'm using the terminology correctly - I'm talking > about an entry in co_cellvars. Given the following class: > > class A(object): > def f(self): > super = super_factory() > > def inner(): > return 'A' + super.f() > > print inner() > > the use of 'super' in both A.f and A.f.inner will produce an entry in > A.f.func_code.co_cellvars and A.f.inner.func_code.co_freevars. What I'm > proposing is that the `super = super_factory()` line be implicit in this > case, resulting in the following code behaving identically: > > class A(object): > def f(self): > def inner(): > return 'A' + super.f() > > print inner() I believe the direction my PEP took with all this is a good bit primitive compared to this approach, although I still find value in it because at least a prototype came out of it that can be used to test the waters, regardless of if a more direct-in-the-language approach would be superior. However, I seem to think that if the __this_class__ PEP goes through, your version can be simplified as well. No tricky stuffy things in cells would be needed, but we can just expand the super 'keyword' to __super__(__this_class__, self), which has been suggested at least once. It seems this would be much simpler to implement, and it also brings up a second point. Also, I like that the super object is created at the beginning of the function, which my proposal couldn't even do. It is more efficient if you have multiple super calls, and gets around a problem I completely missed: what happens if the instance name were rebound before the implicit lookup of the instance object at the time of the super call? > >> The issue of super() vs. super.__call__() ambiguity - I'll need to > >> look at that when I get home. > > > > Sounds irrelevant -- if super is equivalent to > > __builtin__.__super__(, ) then super never gets > > called; the only thing ever done to it (in the normal course of > > things) is to request an attribute of it. > > Sorry - this is related to my proposal that the following two bits of > code behave the same: > > class A(object): > def f(self, *p, **kw): > super.f(*p, **kw) > > class A(object): > def f(self, *p, **kw): > super(*p, **kw) > > But as has been pointed out, this creates an ambiguity with: > > class A(object): > def f(self, *p, **kw): > super.__call__(*p, **kw) > > so I want to see if I can resolve it. Turns out, it doesn't. A lot of people expect it, but it turns out to be simple. super(*p, **kw) is not equivalent to super.__call__(*p, **kw) but to type(super).__call__.__get__(super, type(super))(*p, **kw), due to operator methods being looked up on the type directly and bound by the descriptor, without any lookup on the super object itself. Thus, no attribute is ever done on the super object itself, type(super).__getattribute__ is never invoked, and there is no problem at all. This also means we can still invoke super the old, explicit way. I've already got my copy of the PEP updated to reflect this. > > super(ThisClass).method(...) # ??? > > > > The third exists so that you can create an "unbound" super instance > > which is useful for the oft-misunderstood autosuper example in my > > "descrintro" essay: > > > http://www.python.org/download/releases/2.2.3/descrintro/#metaclass_exam > ples > > . > > > > But since the latter is the hack that we're trying to replace with a > > proper implementation here, I suspect we can get away with only > > supporting the first two forms (and the third form is still supported > > using __builtin__.__super__). > > Yep - that's my thought as well. I think it would be very rare to need > super(ThisClass), although it makes some sense that that would be what > super means in a static method ... Does super mean anything in a static method today? -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From rrr at ronadam.com Mon Apr 30 13:58:58 2007 From: rrr at ronadam.com (Ron Adam) Date: Mon, 30 Apr 2007 06:58:58 -0500 Subject: [Python-3000] [Python-Dev] Pre-pre PEP for 'super' keyword In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> References: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> Message-ID: <4635DA02.2060802@ronadam.com> Delaney, Timothy (Tim) wrote: > What I'm proposing is that the `super = super_factory()` line be > implicit in this case, resulting in the following code behaving > identically: > > class A(object): > def f(self): > def inner(): > return 'A' + super.f() > > print inner() As Guido pointed out it has some resemblance to how import works, but I also think there is resemblance to the context of how global is used. So if it is made into a keyword, could it work like the global keyword? class A(object): def f(self): def inner(): super f return 'A' + f() print inner() Cheers, Ron From rasky at develer.com Mon Apr 30 14:32:22 2007 From: rasky at develer.com (Giovanni Bajo) Date: Mon, 30 Apr 2007 14:32:22 +0200 Subject: [Python-3000] PEP 30XZ: Simplified Parsing In-Reply-To: References: Message-ID: (-CC python-dev) On 30/04/2007 5.29, Jim Jewett wrote: > Rationale for Removing Explicit Line Continuation > > A terminal "\" indicates that the logical line is continued on the > following physical line (after whitespace). > > Note that a non-terminal "\" does not have this meaning, even if the > only additional characters are invisible whitespace. (Python depends > heavily on *visible* whitespace at the beginning of a line; it does > not otherwise depend on *invisible* terminal whitespace.) Adding > whitespace after a "\" will typically cause a syntax error rather > than a silent bug, but it still isn't desirable. > > The reason to keep "\" is that occasionally code looks better with > a "\" than with a () pair. > > assert True, ( > "This Paren is goofy") > > But realistically, that paren is no worse than a "\". The only > advantage of "\" is that it is slightly more familiar to users of > C-based languages. These same languages all also support line > continuation with (), so reading code will not be a problem, and > there will be one less rule to learn for people entirely new to > programming. I was in favor of one of the alternatives that were proposed here: line continuation through indentation: a = 123 * (12 + 4) / 8 assert True, "No goofy paren" + ", dude" ... and don't tell me that you need a character there because using just indentation is hard to read :) Even if you don't want to champion this solution in your PEP, you should at least list it among the alternatives. -- Giovanni Bajo Develer S.r.l. http://www.develer.com From rasky at develer.com Mon Apr 30 14:43:26 2007 From: rasky at develer.com (Giovanni Bajo) Date: Mon, 30 Apr 2007 14:43:26 +0200 Subject: [Python-3000] Traits/roles instead of ABCs In-Reply-To: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> Message-ID: On 30/04/2007 3.40, Collin Winter wrote: > The mechanism I'm most familiar with for solving this problem (which, > unless I've missed something, is, "how do I make sure this object does > what I expect?") is Perl 6's roles system; if you know about Squeak > Smalltalk's "traits" system, you're on the same page. +1. I like those, *specifically* the fact that they can be dynamic (you can "add" a role to a class or an instance of a class and "remove" it later). This is basically mixins they way they should be done. > The idea is that we leave the traditional class system to manage an > object's implementation and use a more-or-less orthogonal system to > manage that same object's behavior. Yes. In other words, traditional inheritance is very good for *reusing* an implementation, but it's not very flexible for defining behaviour. It works for basic cases of course, but gets limited fast when your object network gets more complex. > 2. I gave a strawman syntax proposal for runtime role composition > above (modulo the method name): "Ring.done_by(int)" or > "int.does(Ring)". (Perl 6 speaks in terms of "an object/class *does* a > role".) Perl 6 also allows individual instances to do different roles > than their class does, but I'm not sure we want to go that far. Well, I do! I know there have been times I have needed it badly, and when you're there there's little you can do. If you want to champion a "role" proposal for Python, please don't forget about this. -- Giovanni Bajo From tcdelaney at optusnet.com.au Mon Apr 30 14:47:41 2007 From: tcdelaney at optusnet.com.au (Tim Delaney) Date: Mon, 30 Apr 2007 22:47:41 +1000 Subject: [Python-3000] [Python-Dev] Pre-pre PEP for 'super' keyword References: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> <76fd5acf0704300529r1801bc6at9a2f3d5329d1ea0d@mail.gmail.com> Message-ID: <00e701c78b25$be456c20$0201a8c0@ryoko> From: "Calvin Spealman" > I believe the direction my PEP took with all this is a good bit > primitive compared to this approach, although I still find value in it > because at least a prototype came out of it that can be used to test > the waters, regardless of if a more direct-in-the-language approach > would be superior. I've been working on improved super syntax for quite a while now - my original approach was 'self.super' which used _getframe() and mro crawling too. I hit on using bytecode hacking to instantiate a super object at the start of the method to gain performance, which required storing the class in co_consts, etc. It turns out that using a metaclass then makes this a lot cleaner. > However, I seem to think that if the __this_class__ PEP goes through, > your version can be simplified as well. No tricky stuffy things in > cells would be needed, but we can just expand the super 'keyword' to > __super__(__this_class__, self), which has been suggested at least > once. It seems this would be much simpler to implement, and it also > brings up a second point. > > Also, I like that the super object is created at the beginning of the > function, which my proposal couldn't even do. It is more efficient if > you have multiple super calls, and gets around a problem I completely > missed: what happens if the instance name were rebound before the > implicit lookup of the instance object at the time of the super call? You could expand it inline, but I think your second point is a strong argument against it. Also, sticking the super instance into a cell means that inner classes get access to it for free. Otherwise each inner class would *also* need to instantiate a super instance, and __this_class__ (or whatever it's called) would need to be in a cell for them to get access to it instead. BTW, one of my test cases involves multiple super calls in the same method - there is a *very* large performance improvement by instantiating it once. >> I think it would be very rare to need >> super(ThisClass), although it makes some sense that that would be what >> super means in a static method ... > > Does super mean anything in a static method today? Well, since all super instantiations are explicit currently, it can mean whatever you want it to. class A(object): @staticmethod def f(): print super(A) print super(A, A) Cheers, Tim Delaney From eric+python-dev at trueblade.com Mon Apr 30 14:39:51 2007 From: eric+python-dev at trueblade.com (Eric V. Smith) Date: Mon, 30 Apr 2007 08:39:51 -0400 Subject: [Python-3000] PEP 30XZ: Simplified Parsing In-Reply-To: References: Message-ID: <4635E397.9000403@trueblade.com> Jim Jewett wrote: > Rationale for Removing Implicit String Concatenation > > Implicit String concatentation can lead to confusing, or even > silent, errors. [1] > > def f(arg1, arg2=None): pass > > f("abc" "def") # forgot the comma, no warning ... > # silently becomes f("abcdef", None) > > or, using the scons build framework, > > sourceFiles = [ > 'foo.c', > 'bar.c', > #...many lines omitted... > 'q1000x.c'] > Since your first example omits the comma, I think this one should, too. sourceFiles = [ 'foo.c' 'bar.c', #...many lines omitted... 'q1000x.c'] That is, either both examples should show an error, or both examples should work, but point out how easy it is to make an error. Eric. From ironfroggy at gmail.com Mon Apr 30 15:09:35 2007 From: ironfroggy at gmail.com (Calvin Spealman) Date: Mon, 30 Apr 2007 09:09:35 -0400 Subject: [Python-3000] [Python-Dev] Pre-pre PEP for 'super' keyword In-Reply-To: <00e701c78b25$be456c20$0201a8c0@ryoko> References: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> <76fd5acf0704300529r1801bc6at9a2f3d5329d1ea0d@mail.gmail.com> <00e701c78b25$be456c20$0201a8c0@ryoko> Message-ID: <76fd5acf0704300609o4285b9d6p4895a891420aa88d@mail.gmail.com> On 4/30/07, Tim Delaney wrote: > I've been working on improved super syntax for quite a while now - my > original approach was 'self.super' which used _getframe() and mro crawling > too. I hit on using bytecode hacking to instantiate a super object at the > start of the method to gain performance, which required storing the class in > co_consts, etc. It turns out that using a metaclass then makes this a lot > cleaner. > > > However, I seem to think that if the __this_class__ PEP goes through, > > your version can be simplified as well. No tricky stuffy things in > > cells would be needed, but we can just expand the super 'keyword' to > > __super__(__this_class__, self), which has been suggested at least > > once. It seems this would be much simpler to implement, and it also > > brings up a second point. > > > > Also, I like that the super object is created at the beginning of the > > function, which my proposal couldn't even do. It is more efficient if > > you have multiple super calls, and gets around a problem I completely > > missed: what happens if the instance name were rebound before the > > implicit lookup of the instance object at the time of the super call? > > You could expand it inline, but I think your second point is a strong > argument against it. Also, sticking the super instance into a cell means > that inner classes get access to it for free. Otherwise each inner class > would *also* need to instantiate a super instance, and __this_class__ (or > whatever it's called) would need to be in a cell for them to get access to > it instead. > > BTW, one of my test cases involves multiple super calls in the same method - > there is a *very* large performance improvement by instantiating it once. Note that I would now advocate the creation of the super object at the beginning of function, and when I talk about expanding super to super(__this_class__, self) I do mean doing it at the beginning of the function, just like you are saying, so we're in agreement here. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From jimjjewett at gmail.com Mon Apr 30 16:49:06 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 30 Apr 2007 10:49:06 -0400 Subject: [Python-3000] octal literals PEP Message-ID: On 4/30/07, Guido van Rossum wrote: > I think these should be two separate proposals, with more specific > names (e.g. "remove implicit string concatenation" and "remove > backslash continuation"). There's no need to mention the octal thing > if it's already a separate PEP. Patrick Guido had set an Apr 30 deadline for Py3000 PEPs that can't be implemented in pure python. Are you still working on the "Integer literal syntax and radices ", which included the octal literal? I would much prefer to leave octal literals with the rest of that PEP, (and to let you do it :D), but I will submit a much-simplified "023 raises SyntaxError" if you have abandoned the rest. -jJ From pje at telecommunity.com Mon Apr 30 17:16:34 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 30 Apr 2007 11:16:34 -0400 Subject: [Python-3000] Could isinstance/issubclass overriding be dangerous? In-Reply-To: <46354AFD.5070702@canterbury.ac.nz> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com> <46353440.5060402@acm.org> Message-ID: <5.1.1.6.0.20070430111601.02e5e6c0@sparrow.telecommunity.com> At 01:48 PM 4/30/2007 +1200, Greg Ewing wrote: >Currently, isinstance(x, C) implies that the >C struct layout of x is compatible with that >defined by C. No, it doesn't, and hasn't since Python 2.3. > I'm worried that changing this >could cause difficulties for extension modules >that use PyObject_IsInstance do decide what >to do with things, e.g. in a binary operator >method. > >-- >Greg >_______________________________________________ >Python-3000 mailing list >Python-3000 at python.org >http://mail.python.org/mailman/listinfo/python-3000 >Unsubscribe: >http://mail.python.org/mailman/options/python-3000/pje%40telecommunity.com From alan.mcintyre at gmail.com Mon Apr 30 17:16:06 2007 From: alan.mcintyre at gmail.com (Alan McIntyre) Date: Mon, 30 Apr 2007 11:16:06 -0400 Subject: [Python-3000] Traits/roles instead of ABCs In-Reply-To: <014201c78adc$ca70d960$f101a8c0@RaymondLaptop1> References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> <014201c78adc$ca70d960$f101a8c0@RaymondLaptop1> Message-ID: <1d36917a0704300816ma3bf9c2o4dd674cfcefa9172@mail.gmail.com> On 4/30/07, Raymond Hettinger wrote: > I'm concerned that the current ABC proposal will quickly evolve from optional > to required and create somewhat somewhat java-esque landscape where > inheritance and full-specification are the order of the day. +1 for preferring simple solutions to complex ones +inf for not turning Python into Java or C++ <0.5 wink> From pje at telecommunity.com Mon Apr 30 17:30:17 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 30 Apr 2007 11:30:17 -0400 Subject: [Python-3000] Could isinstance/issubclass overriding be dangerous? In-Reply-To: <46358C4B.2020803@canterbury.ac.nz> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com> <46353440.5060402@acm.org> <46354AFD.5070702@canterbury.ac.nz> Message-ID: <5.1.1.6.0.20070430112626.04ef35d8@sparrow.telecommunity.com> At 06:27 PM 4/30/2007 +1200, Greg Ewing wrote: >So can we please have another couple of functions that just >do a simple, reliable concrete type test? I believe the operation you're looking for (i.e., C-level layout compatibility) is: C in type.__mro__.__get__(type(x)) or in Pyrex: C in (((PyTypeObject *) type(x)).tp_mro) From janssen at parc.com Mon Apr 30 18:01:09 2007 From: janssen at parc.com (Bill Janssen) Date: Mon, 30 Apr 2007 09:01:09 PDT Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com> <46353440.5060402@acm.org> Message-ID: <07Apr30.090110pdt."57996"@synergy1.parc.xerox.com> GvR wrote: > So how about we reduce the scope of our (!) PEP (or perhaps of a new > one) to two items: (a) add @abstractmethod, and (b) overload > isinstance() and issubclass()? Library authors can do everything they > want with those, and we can always add a specific set of ABCs for > containers and/or numbers later in the 3.0 development cycle. -1. Adding mechanism without content seems less than ideal, despite Talin's misgivings. I'd recommend adding the base classes in, and see how they work in the earliest 3.0 releases, then modify them as necessary in subsequent releases. Bill From janssen at parc.com Mon Apr 30 18:08:40 2007 From: janssen at parc.com (Bill Janssen) Date: Mon, 30 Apr 2007 09:08:40 PDT Subject: [Python-3000] Traits/roles instead of ABCs In-Reply-To: <014201c78adc$ca70d960$f101a8c0@RaymondLaptop1> References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> <014201c78adc$ca70d960$f101a8c0@RaymondLaptop1> Message-ID: <07Apr30.090845pdt."57996"@synergy1.parc.xerox.com> > [Collin Winter] > +100 I'm very interested in seeing a lighter weight alternative to abc.py that: > > 1) is dynamic There's no reason ABC's can't be dynamic. > 2) doesn't require inheritance to work > 3) doesn't require mucking with isinstance or other existing mechansims Using existing mechanisms that work for the issue is always preferable to adding new ones that *also* work for the issue. > 4) makes a limited, useful set of assertions rather than broadly covering a whole API. That's what an API is, isn't it? > 5) that leaves the notion of duck-typing as the rule rather than the exception Here we disagree. > 6) that doesn't freeze all of the key APIs in concrete After 15 years of experience with the key APIs, we could perhaps freeze some of them? > IMHO, the ABC approach is using a cannon to shoot a mosquito. My day-to-day > problems are much smaller are could be solved by a metadata attribute or a > role/trait solution: And then you cite a few symptoms of not having an API-based system: > * knowing whether a __getitem__ method implements a mapping or a sequence In other words, what APIs does this value support? > * knowing whether an object can have more that one iterator (i.e a file has one > but a list can have many) In other words, what APIs does this value support? > * knowing whether a sequence, file, cursor, etc is writable or just readonly. In other words, what APIs does this value support? If we simply had a small standard extensible set of APIs, and a way to indicate that a value does or doesn't support one or more of them, we'd be done. We could either add a new mechanism to do this, or simply actually use the one we already have, which should work perfectly well for this purpose. Bill From janssen at parc.com Mon Apr 30 18:11:24 2007 From: janssen at parc.com (Bill Janssen) Date: Mon, 30 Apr 2007 09:11:24 PDT Subject: [Python-3000] Traits/roles instead of ABCs In-Reply-To: <1d36917a0704300816ma3bf9c2o4dd674cfcefa9172@mail.gmail.com> References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> <014201c78adc$ca70d960$f101a8c0@RaymondLaptop1> <1d36917a0704300816ma3bf9c2o4dd674cfcefa9172@mail.gmail.com> Message-ID: <07Apr30.091130pdt."57996"@synergy1.parc.xerox.com> > On 4/30/07, Raymond Hettinger wrote: > > I'm concerned that the current ABC proposal will quickly evolve from optional > > to required and create somewhat somewhat java-esque landscape where > > inheritance and full-specification are the order of the day. > > +1 for preferring simple solutions to complex ones Me, too. But which is the simple solution? I tend to think ABCs are. > +inf for not turning Python into Java or C++ <0.5 wink> Not into the bad parts of either, to be sure. Bill From talin at acm.org Mon Apr 30 18:27:29 2007 From: talin at acm.org (Talin) Date: Mon, 30 Apr 2007 09:27:29 -0700 Subject: [Python-3000] Traits/roles instead of ABCs In-Reply-To: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> Message-ID: <463618F1.6070309@acm.org> Collin Winter wrote: > The key part of traits/roles is that, because the system is separate > from classes, you can do runtime role composition without a) mucking > with __bases__, or b) making isinstance() and issubclass() squishy and > ill-defined. By "runtime role composition", I mean it would be > possible to do something like this at runtime: > > py> int.implements(Ring) > > or (depending on your spelling preferences) > > py> Ring.implemented_by(int) > > That is, it would be possible to distribute Jeffery's numeric kinds as > a third-party library and still have them affect the built-in numeric > types. The question I have about all this runtime composition stuff is, what happens if different modules have different ideas about how things should be composed? How are conflicts resolved? With ABCs, we have a limitation that is also a strength: The mapping between ABC and its bases is either built-in to the ABC or built-in to the class, meaning that these decisions have to all be made up front - which forces the ABC implementer to pre-decide the outcome of all such conflicts before they happen. -- Talin From collinw at gmail.com Mon Apr 30 18:50:09 2007 From: collinw at gmail.com (Collin Winter) Date: Mon, 30 Apr 2007 09:50:09 -0700 Subject: [Python-3000] Traits/roles instead of ABCs In-Reply-To: <-3604285971386182607@unknownmsgid> References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> <014201c78adc$ca70d960$f101a8c0@RaymondLaptop1> <-3604285971386182607@unknownmsgid> Message-ID: <43aa6ff70704300950t4b8179beob3b8df7001a2819b@mail.gmail.com> On 4/30/07, Bill Janssen wrote: > > [Collin Winter] > > +100 I'm very interested in seeing a lighter weight alternative to abc.py that: > > > > 1) is dynamic > > There's no reason ABC's can't be dynamic. > > > 2) doesn't require inheritance to work > > 3) doesn't require mucking with isinstance or other existing mechansims > > Using existing mechanisms that work for the issue is always preferable > to adding new ones that *also* work for the issue. No existing mechanism would allow PEP 3141 to be distributed as a third-party module and still impact the built-in types; that's what this whole business of __isinstance__ and __issubclass__ is about. In order to achieve dynamic composition, we're having to monkey with fundamental, bedrock properties of inheritance-based object orientation. The definition of "subclass" and "instance" is no longer objective; the answer to "is X an instance of Y?" becomes, "I dunno, ask Y". Collin Winter From python3now at gmail.com Mon Apr 30 18:53:16 2007 From: python3now at gmail.com (James Thiele) Date: Mon, 30 Apr 2007 09:53:16 -0700 Subject: [Python-3000] Addition to PEP 3101 Message-ID: <8f01efd00704300953t6154d7e1j7ef18cead1acb344@mail.gmail.com> For PEP: 3101 - Advanced String Formatting - I would like to suggest the following additions. First, that an additional pair of forms be added to format strings so that callable objects could be used as parameters to the format string method: {0()} for positional arguments and {keyword()} for keyword parameters. The proposal is that if the format string had this form that the format string method would treat the parameter as a callable and use the value of the return for the formatting. The second addition would be the forms {0.method()} and {keyword.method()} where "method" is a method of the object. This would allow things such as: "I want to shout '{0.upper()}'".format("hello world") which would yield: "I want to shout 'HELLO WORLD'" From p.f.moore at gmail.com Mon Apr 30 19:01:19 2007 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 30 Apr 2007 18:01:19 +0100 Subject: [Python-3000] Traits/roles instead of ABCs In-Reply-To: <438708814690534630@unknownmsgid> References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> <014201c78adc$ca70d960$f101a8c0@RaymondLaptop1> <438708814690534630@unknownmsgid> Message-ID: <79990c6b0704301001ga0d2429sdaded9ac75fa15c5@mail.gmail.com> On 30/04/07, Bill Janssen wrote: > > 6) that doesn't freeze all of the key APIs in concrete > > After 15 years of experience with the key APIs, we could perhaps freeze some of > them? After 15 years not being able to clearly state what "file-like" or "mapping-like" means to different people, perhaps we should accept that there is no clear-cut answer...? > In other words, what APIs does this value support? > In other words, what APIs does this value support? > In other words, what APIs does this value support? Yes, but the ABC proposal is about establishing a vocabulary for answering those questions, not about asking them. It's not at all clear (to me, at least) that there is agreement that the vocabulary proposed is helpful. > If we simply had a small standard extensible set of APIs, and a way to > indicate that a value does or doesn't support one or more of them, > we'd be done. We could either add a new mechanism to do this, or simply > actually use the one we already have, which should work perfectly well for > this purpose. And which has been around for many years, but has never actually been used in practice (unlike, say, interfaces). Does this not say something? Paul. From collinw at gmail.com Mon Apr 30 19:14:34 2007 From: collinw at gmail.com (Collin Winter) Date: Mon, 30 Apr 2007 10:14:34 -0700 Subject: [Python-3000] Traits/roles instead of ABCs In-Reply-To: <463618F1.6070309@acm.org> References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> <463618F1.6070309@acm.org> Message-ID: <43aa6ff70704301014y3dfbac77yaa42a07a8dd57499@mail.gmail.com> On 4/30/07, Talin wrote: > Collin Winter wrote: > > The key part of traits/roles is that, because the system is separate > > from classes, you can do runtime role composition without a) mucking > > with __bases__, or b) making isinstance() and issubclass() squishy and > > ill-defined. By "runtime role composition", I mean it would be > > possible to do something like this at runtime: > > > > py> int.implements(Ring) > > > > or (depending on your spelling preferences) > > > > py> Ring.implemented_by(int) > > > > That is, it would be possible to distribute Jeffery's numeric kinds as > > a third-party library and still have them affect the built-in numeric > > types. > > The question I have about all this runtime composition stuff is, what > happens if different modules have different ideas about how things > should be composed? How are conflicts resolved? > > With ABCs, we have a limitation that is also a strength: The mapping > between ABC and its bases is either built-in to the ABC or built-in to > the class, meaning that these decisions have to all be made up front - > which forces the ABC implementer to pre-decide the outcome of all such > conflicts before they happen. What do you mean by "different ideas about how things should be composed"? Do you mean the case where two modules say the same class should do different roles? Roles are purely additive, so if module X says that class Z does one role, and module Y says class Z does another role, there's no conflict: the class does both roles simultaneously. Things get interesting when you try to compose roles that both offer default implementations of the same method. The standard response (and the one chosen by Perl 6) is to just throw up your hands and shriek wildly and let the programmer sort it out. http://www.perlmonks.org/?node_id=384909 has a good example that addresses this exact question. Collin Winter From guido at python.org Mon Apr 30 19:19:52 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 30 Apr 2007 10:19:52 -0700 Subject: [Python-3000] octal literals PEP In-Reply-To: References: Message-ID: On 4/30/07, Jim Jewett wrote: > On 4/30/07, Guido van Rossum wrote: > > I think these should be two separate proposals, with more specific > > names (e.g. "remove implicit string concatenation" and "remove > > backslash continuation"). There's no need to mention the octal thing > > if it's already a separate PEP. > > Patrick > > Guido had set an Apr 30 deadline for Py3000 PEPs that can't be > implemented in pure python. I don't recall limiting the deadline in that way. Is this a general understanding? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Apr 30 19:30:28 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 30 Apr 2007 10:30:28 -0700 Subject: [Python-3000] PEP 30XZ: Simplified Parsing In-Reply-To: References: Message-ID: On 4/30/07, Giovanni Bajo wrote: > I was in favor of one of the alternatives that were proposed here: Was? Are you no longer in favor of it? (Then I should shut up. :-) > line continuation through indentation: > > a = 123 * > (12 + 4) / 8 > > assert True, > "No goofy paren" + > ", dude" > > ... and don't tell me that you need a character there because using just > indentation is hard to read :) > > Even if you don't want to champion this solution in your PEP, you should at > least list it among the alternatives. Then please add my -1 to it: I don't think this is feasible. It would require allowing matching INDENT and DEDENT tokens *anywhere* in the grammar; *or* you would have to limit where in the grammar you allow the indent to occur (e.g. after '*' but not before it; after the ',' but not before it). It seems a really big departure from the current Python parsing methodology. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Mon Apr 30 19:38:14 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 30 Apr 2007 11:38:14 -0600 Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities) In-Reply-To: <-5382129169035830252@unknownmsgid> References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com> <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com> <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com> <46353440.5060402@acm.org> <-5382129169035830252@unknownmsgid> Message-ID: On 4/30/07, Bill Janssen wrote: > GvR wrote: > > So how about we reduce the scope of our (!) PEP (or perhaps of a new > > one) to two items: (a) add @abstractmethod, and (b) overload > > isinstance() and issubclass()? Library authors can do everything they > > want with those, and we can always add a specific set of ABCs for > > containers and/or numbers later in the 3.0 development cycle. > > -1. Adding mechanism without content seems less than ideal, despite > Talin's misgivings. I'd recommend adding the base classes in, and see > how they work in the earliest 3.0 releases, then modify them as > necessary in subsequent releases. +1 for Guido's simplified PEP. There are clearly use cases for @abstractmethod, __isinstance__() and __issubclass__(), so I see no reason why the simplified PEP shouldn't be acceptable on its own. Complaining about "adding mechanism without content" seems like saying, for example, that we shouldn't have introduced "functools.partial" without *using* it somewhere in the standard library. Whether the use cases are in the standard library or somewhere else should not determine whether a PEP is acceptable or not. I personally think the simplified PEP is a great compromise -- proponents of ABCs will now be able to introduce them (even for existing types like 'int' and 'list') and folks that don't want ABCs can simply never install the ABC module. Plus, it's much easier to *add* a module to the standard library than it is to *remove* one (even given Python 3.0's somewhat laxer first release plans). STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From guido at python.org Mon Apr 30 19:43:18 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 30 Apr 2007 10:43:18 -0700 Subject: [Python-3000] [Python-Dev] Pre-pre PEP for 'super' keyword In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> References: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com> Message-ID: On 4/29/07, Delaney, Timothy (Tim) wrote: > Guido van Rossum wrote: > >> 2. Every non-static method has an implicit cell variable called > >> 'super'. > > > > I think you're using 'cell' in a different sense than it is normally > > used in Python's implementation. What you are looking for is called a > > local variable (I deduce this from your initialization of the "cell" > > with something computed from the first argument). > > Actually, I think I'm using the terminology correctly - I'm talking > about an entry in co_cellvars. Given the following class: > > class A(object): > def f(self): > super = super_factory() > > def inner(): > return 'A' + super.f() > > print inner() > > the use of 'super' in both A.f and A.f.inner will produce an entry in > A.f.func_code.co_cellvars and A.f.inner.func_code.co_freevars. What I'm > proposing is that the `super = super_factory()` line be implicit in this > case, resulting in the following code behaving identically: > > class A(object): > def f(self): > def inner(): > return 'A' + super.f() > > print inner() OK, I see now. I thought you meant for the cell-ness to be related to the basic implementation; but you meant it so that it woks correctly inside nested functions. That's fine of course. You might clarify this somewhere. > >> The issue of super() vs. super.__call__() ambiguity - I'll need to > >> look at that when I get home. > > > > Sounds irrelevant -- if super is equivalent to > > __builtin__.__super__(, ) then super never gets > > called; the only thing ever done to it (in the normal course of > > things) is to request an attribute of it. > > Sorry - this is related to my proposal that the following two bits of > code behave the same: > > class A(object): > def f(self, *p, **kw): > super.f(*p, **kw) > > class A(object): > def f(self, *p, **kw): > super(*p, **kw) > > But as has been pointed out, this creates an ambiguity with: > > class A(object): > def f(self, *p, **kw): > super.__call__(*p, **kw) > > so I want to see if I can resolve it. I think the shortcut is more confusing than helpful; I propose to drop it for the sake of focusing on the essential. > > super(ThisClass).method(...) # ??? > > > > The third exists so that you can create an "unbound" super instance > > which is useful for the oft-misunderstood autosuper example in my > > "descrintro" essay: > > > > http://www.python.org/download/releases/2.2.3/descrintro/#metaclass_exampples > > . > > > > But since the latter is the hack that we're trying to replace with a > > proper implementation here, I suspect we can get away with only > > supporting the first two forms (and the third form is still supported > > using __builtin__.__super__). > > Yep - that's my thought as well. I think it would be very rare to need > super(ThisClass), although it makes some sense that that would be what > super means in a static method ... But that doesn't work with the current (2.x) super, and hasn't been requested AFAIK, so I'd suggest dropping the use case -- KISS again. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Mon Apr 30 19:46:31 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 30 Apr 2007 11:46:31 -0600 Subject: [Python-3000] Traits/roles instead of ABCs In-Reply-To: <43aa6ff70704300950t4b8179beob3b8df7001a2819b@mail.gmail.com> References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> <014201c78adc$ca70d960$f101a8c0@RaymondLaptop1> <-3604285971386182607@unknownmsgid> <43aa6ff70704300950t4b8179beob3b8df7001a2819b@mail.gmail.com> Message-ID: On 4/30/07, Collin Winter wrote: > No existing mechanism would allow PEP 3141 to be distributed as a > third-party module and still impact the built-in types; that's what > this whole business of __isinstance__ and __issubclass__ is about. In > order to achieve dynamic composition, we're having to monkey with > fundamental, bedrock properties of inheritance-based object > orientation. The definition of "subclass" and "instance" is no longer > objective; the answer to "is X an instance of Y?" becomes, "I dunno, > ask Y". I guess I don't understand why this bothers you more than how you can currently mess with __class__:: >>> class X(object): ... __class__ = property(lambda self: Y) ... >>> class Y(object): ... pass ... >>> isinstance(X(), Y) True The X class does *not* inherit from the Y class, yet it tells us that it does. To me, that certainly looks like "monkey[ing] with fundamental, bedrock properties of inheritance-based object orientation". Maybe you could give a couple realistic examples of where you think people using __isinstance__ and __issubclass__ are going to break things unintentionally? STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From steven.bethard at gmail.com Mon Apr 30 19:49:34 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 30 Apr 2007 11:49:34 -0600 Subject: [Python-3000] octal literals PEP In-Reply-To: References: Message-ID: On 4/30/07, Guido van Rossum wrote: > On 4/30/07, Jim Jewett wrote: > > Guido had set an Apr 30 deadline for Py3000 PEPs that can't be > > implemented in pure python. > > I don't recall limiting the deadline in that way. Is this a general > understanding? FWIW, my understanding was only standard library changes would be allowed after Apr 30. I don't recall hearing anything about "pure python" vs. otherwise, but I guess "pure python" corresponds to "standard library"...? STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From guido at python.org Mon Apr 30 19:52:49 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 30 Apr 2007 10:52:49 -0700 Subject: [Python-3000] octal literals PEP In-Reply-To: References: Message-ID: On 4/30/07, Steven Bethard wrote: > On 4/30/07, Guido van Rossum wrote: > > On 4/30/07, Jim Jewett wrote: > > > Guido had set an Apr 30 deadline for Py3000 PEPs that can't be > > > implemented in pure python. > > > > I don't recall limiting the deadline in that way. Is this a general > > understanding? > > FWIW, my understanding was only standard library changes would be > allowed after Apr 30. I don't recall hearing anything about "pure > python" vs. otherwise, but I guess "pure python" corresponds to > "standard library"...? Not at all. There's a whole slew of C extensions that are considered part of the standard library. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From collinw at gmail.com Mon Apr 30 20:01:58 2007 From: collinw at gmail.com (Collin Winter) Date: Mon, 30 Apr 2007 11:01:58 -0700 Subject: [Python-3000] Traits/roles instead of ABCs In-Reply-To: References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> <014201c78adc$ca70d960$f101a8c0@RaymondLaptop1> <-3604285971386182607@unknownmsgid> <43aa6ff70704300950t4b8179beob3b8df7001a2819b@mail.gmail.com> Message-ID: <43aa6ff70704301101w7707fd8m735442cda9d14eb9@mail.gmail.com> On 4/30/07, Steven Bethard wrote: > On 4/30/07, Collin Winter wrote: > > No existing mechanism would allow PEP 3141 to be distributed as a > > third-party module and still impact the built-in types; that's what > > this whole business of __isinstance__ and __issubclass__ is about. In > > order to achieve dynamic composition, we're having to monkey with > > fundamental, bedrock properties of inheritance-based object > > orientation. The definition of "subclass" and "instance" is no longer > > objective; the answer to "is X an instance of Y?" becomes, "I dunno, > > ask Y". > > I guess I don't understand why this bothers you more than how you can > currently mess with __class__:: [snip] > Maybe you could give a couple realistic examples of where you think > people using __isinstance__ and __issubclass__ are going to break > things unintentionally? You assume that being able to change __class__ doesn't bother me: it does. The thing that mitigates my unease is that it comes with a gigantic red flag that boils down to, "Don't even think about this unless you're implementing proxy objects". By pushing capabilities tests into the core object system and making isinstance() and issubclass() the primary mechanisms for determining an object's behaviour, you're making it standard practice to monkey around with inheritance. Do I have specific examples of what I'm afraid of? No. All I can offer is a deep distrust of a system where metaclasses and a redefined concept of inheritance are pulled out so early in the debate in order to solve issues that other mechanisms (like roles) make trivial. Collin Winter From steven.bethard at gmail.com Mon Apr 30 20:21:24 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 30 Apr 2007 12:21:24 -0600 Subject: [Python-3000] Traits/roles instead of ABCs In-Reply-To: <43aa6ff70704301101w7707fd8m735442cda9d14eb9@mail.gmail.com> References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> <014201c78adc$ca70d960$f101a8c0@RaymondLaptop1> <-3604285971386182607@unknownmsgid> <43aa6ff70704300950t4b8179beob3b8df7001a2819b@mail.gmail.com> <43aa6ff70704301101w7707fd8m735442cda9d14eb9@mail.gmail.com> Message-ID: On 4/30/07, Collin Winter wrote: > On 4/30/07, Steven Bethard wrote: > > On 4/30/07, Collin Winter wrote: > > > No existing mechanism would allow PEP 3141 to be distributed as a > > > third-party module and still impact the built-in types; that's what > > > this whole business of __isinstance__ and __issubclass__ is about. In > > > order to achieve dynamic composition, we're having to monkey with > > > fundamental, bedrock properties of inheritance-based object > > > orientation. The definition of "subclass" and "instance" is no longer > > > objective; the answer to "is X an instance of Y?" becomes, "I dunno, > > > ask Y". > > > > I guess I don't understand why this bothers you more than how you can > > currently mess with __class__:: > [snip] > > Maybe you could give a couple realistic examples of where you think > > people using __isinstance__ and __issubclass__ are going to break > > things unintentionally? > > You assume that being able to change __class__ doesn't bother me: it > does. The thing that mitigates my unease is that it comes with a > gigantic red flag that boils down to, "Don't even think about this > unless you're implementing proxy objects". Ahh. I guess I see __isinstance__ and __issubclass__ as having gigantic red flags saying "Don't even think about these unless you're implementing ABCs". ;-) STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From pje at telecommunity.com Mon Apr 30 20:46:47 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 30 Apr 2007 14:46:47 -0400 Subject: [Python-3000] super(), class decorators, and PEP 3115 Message-ID: <5.1.1.6.0.20070430142844.03c96240@sparrow.telecommunity.com> While working on the generic function PEP, I came across an unexpected (to me, anyway) consequence of PEP 3115: it breaks current methods (circa Python 2.2+) of implementing class decorators. However, there does not appear to be a class decorator PEP outstanding. In current Python versions, the only way to implement class decorators is to call a "magic" function in the body of the class, that inserts a special (temporary) metaclass to handle the decoration. This is implemented in the DecoratorTools package as well as in zope.interface (where its most common use case is to supply interface information about a class). Of course, in Py 3.x, some of these features could be handled using a metaclass with keyword arguments, due to PEP 3115. However, the other use cases cannot. Notably, any code that relies on in-class decorators or conditional class decorators (such as method decorators that add class decorators), will not be able to work. However, I have an idea for a small modification to PEP 3115 that would re-enable support for both static and dynamic class decorators, and make it possible to implement an improved super() as well. Suppose that we reserved a special name, '__decorators__', that would be contained in the mapping object returned by '__prepare__', and initialized to contain any static decorators defined outside the class, e.g., this code: @implements(IFoo) @otherdecorator class X: print __decorators__ would print a list equal to [implements(IFoo), otherdecorator]. The idea here is that following the execution of metaclass(name, bases, classdict, **kw), the __decorators__ list would be repeatedly popped to yield decorators to be invoked on the "finished" class. In this way, code that needs dynamic access to the finished class can simply add objects to __decorators__. As a result, we could then implement a super() enhancement of the form "super(__class__, self).methodname(args)", where "__class__" would be a compiler-defined variable (ala "__debug__"). If any function inside a class body uses "__class__", a preamble is added to the class body that defines an anonymous callback function that sets a closure variable to point to the class, and returns the class. This anonymous callback function is then appended to __decorators__. So, this approach could be used to enhance super() directly (using __class__ instead of a specific class), or as a behind-the-scenes part of the implementation of a more "sugary" syntax. It would also continue to enable dynamic class decorators for other uses. Thoughts? From guido at python.org Mon Apr 30 21:17:53 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 30 Apr 2007 12:17:53 -0700 Subject: [Python-3000] super(), class decorators, and PEP 3115 In-Reply-To: <5.1.1.6.0.20070430142844.03c96240@sparrow.telecommunity.com> References: <5.1.1.6.0.20070430142844.03c96240@sparrow.telecommunity.com> Message-ID: Hm... Where *is* the class decorators PEP? I was expecting it. Assuming class decorators are added, can't you do all of this using a custom metaclass? I'm not sure that your proposal for implementing an improved super has anything over the currently most-favored proposal by Timothy Delaney. --Guido On 4/30/07, Phillip J. Eby wrote: > While working on the generic function PEP, I came across an unexpected (to > me, anyway) consequence of PEP 3115: it breaks current methods (circa > Python 2.2+) of implementing class decorators. However, there does not > appear to be a class decorator PEP outstanding. > > In current Python versions, the only way to implement class decorators is > to call a "magic" function in the body of the class, that inserts a special > (temporary) metaclass to handle the decoration. This is implemented in the > DecoratorTools package as well as in zope.interface (where its most common > use case is to supply interface information about a class). > > Of course, in Py 3.x, some of these features could be handled using a > metaclass with keyword arguments, due to PEP 3115. However, the other use > cases cannot. Notably, any code that relies on in-class decorators or > conditional class decorators (such as method decorators that add class > decorators), will not be able to work. > > However, I have an idea for a small modification to PEP 3115 that would > re-enable support for both static and dynamic class decorators, and make it > possible to implement an improved super() as well. > > Suppose that we reserved a special name, '__decorators__', that would be > contained in the mapping object returned by '__prepare__', and initialized > to contain any static decorators defined outside the class, e.g., this code: > > @implements(IFoo) > @otherdecorator > class X: > print __decorators__ > > would print a list equal to [implements(IFoo), otherdecorator]. The idea > here is that following the execution of metaclass(name, bases, classdict, > **kw), the __decorators__ list would be repeatedly popped to yield > decorators to be invoked on the "finished" class. In this way, code that > needs dynamic access to the finished class can simply add objects to > __decorators__. > > As a result, we could then implement a super() enhancement of the form > "super(__class__, self).methodname(args)", where "__class__" would be a > compiler-defined variable (ala "__debug__"). If any function inside a > class body uses "__class__", a preamble is added to the class body that > defines an anonymous callback function that sets a closure variable to > point to the class, and returns the class. This anonymous callback > function is then appended to __decorators__. > > So, this approach could be used to enhance super() directly (using > __class__ instead of a specific class), or as a behind-the-scenes part of > the implementation of a more "sugary" syntax. It would also continue to > enable dynamic class decorators for other uses. > > Thoughts? > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Mon Apr 30 21:25:39 2007 From: brett at python.org (Brett Cannon) Date: Mon, 30 Apr 2007 12:25:39 -0700 Subject: [Python-3000] octal literals PEP In-Reply-To: References: Message-ID: On 4/30/07, Steven Bethard wrote: > On 4/30/07, Guido van Rossum wrote: > > On 4/30/07, Jim Jewett wrote: > > > Guido had set an Apr 30 deadline for Py3000 PEPs that can't be > > > implemented in pure python. > > > > I don't recall limiting the deadline in that way. Is this a general > > understanding? > > FWIW, my understanding was only standard library changes would be > allowed after Apr 30. Ditto from me. I never thought there was any connection to how the solution would eventually be implemented. -Brett From pje at telecommunity.com Mon Apr 30 21:35:36 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 30 Apr 2007 15:35:36 -0400 Subject: [Python-3000] super(), class decorators, and PEP 3115 In-Reply-To: References: <5.1.1.6.0.20070430142844.03c96240@sparrow.telecommunity.com> <5.1.1.6.0.20070430142844.03c96240@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070430152320.02d31868@sparrow.telecommunity.com> At 12:17 PM 4/30/2007 -0700, Guido van Rossum wrote: >Hm... Where *is* the class decorators PEP? I was expecting it. > >Assuming class decorators are added, can't you do all of this using a >custom metaclass? The only thing I need for the GF PEP is a way for a method decorator to get a callback after the class is created, so that overloading will work correctly in cases where overloaded methods are defined in a subclass. In essence, when you define an overloaded method inside a class body, you would like to be able to treat it as if it were defined with "self:__class__", where __class__ is the enclosing class. In practice, this means that the actual overloading has to wait until the class definition is finished. In Python 2.x, RuleDispatch implements this by temporary tinkering with __metaclass__, but if I understand correctly this would not be possible with PEP 3115. I didn't make this connection until I was fleshing out my PEP's explanation of how precedence works when you are overloading instance methods (as opposed to standalone functions). If PEP 3115 were changed to restore support for __metaclass__, I could continue to use that approach. Otherwise, some other sort of hook is required. The class decorator thing isn't an issue for the GF PEP as such; it doesn't use them directly, only via the __metaclass__ hack. I just brought it up because I was looking for the class decorator PEP when I realized that the old way of doing them wouldn't be possible any more. >I'm not sure that your proposal for implementing an improved super has >anything over the currently most-favored proposal by Timothy Delaney. It's merely another use for the hook, that would save on having another special-purpose mechanism strictly for super(); I figured that having other uses for it (besides mine) would be a plus. From brett at python.org Mon Apr 30 21:34:55 2007 From: brett at python.org (Brett Cannon) Date: Mon, 30 Apr 2007 12:34:55 -0700 Subject: [Python-3000] super(), class decorators, and PEP 3115 In-Reply-To: References: <5.1.1.6.0.20070430142844.03c96240@sparrow.telecommunity.com> Message-ID: On 4/30/07, Guido van Rossum wrote: > Hm... Where *is* the class decorators PEP? I was expecting it. > Jack Diedrich was working on it. Jack, you going to get this in to meet the April 30th deadline for Py3K PEPs (which is obviously today)? -Brett From pmaupin at gmail.com Mon Apr 30 21:51:24 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Mon, 30 Apr 2007 14:51:24 -0500 Subject: [Python-3000] octal literals PEP In-Reply-To: References: Message-ID: I sent an email with an initial PEP to the PEP editors a few weeks ago. Never got a reply. I noticed some traffic about this recently but was too busy to follow it really carefully. Pat On 4/30/07, Jim Jewett wrote: > On 4/30/07, Guido van Rossum wrote: > > I think these should be two separate proposals, with more specific > > names (e.g. "remove implicit string concatenation" and "remove > > backslash continuation"). There's no need to mention the octal thing > > if it's already a separate PEP. > > Patrick > > Guido had set an Apr 30 deadline for Py3000 PEPs that can't be > implemented in pure python. > > Are you still working on the "Integer literal syntax and radices ", > which included the octal literal? I would much prefer to leave octal > literals with the rest of that PEP, (and to let you do it :D), but I > will submit a much-simplified "023 raises SyntaxError" if you have > abandoned the rest. > > -jJ > From pmaupin at gmail.com Mon Apr 30 21:56:12 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Mon, 30 Apr 2007 14:56:12 -0500 Subject: [Python-3000] Addition to PEP 3101 In-Reply-To: <8f01efd00704300953t6154d7e1j7ef18cead1acb344@mail.gmail.com> References: <8f01efd00704300953t6154d7e1j7ef18cead1acb344@mail.gmail.com> Message-ID: On 4/30/07, James Thiele wrote: > For PEP: 3101 - Advanced String Formatting - I would like to suggest > the following additions. > > First, that an additional pair of forms be added to format strings so > that callable objects could be used as parameters to the format string > method: > {0()} for positional arguments and > {keyword()} for keyword parameters. > The proposal is that if the format string had this form that the > format string method would treat the parameter as a callable and use > the value of the return for the formatting. > > The second addition would be the forms > {0.method()} and > {keyword.method()} > where "method" is a method of the object. > > This would allow things such as: > "I want to shout '{0.upper()}'".format("hello world") > which would yield: > "I want to shout 'HELLO WORLD'" Method calls are deliberately disallowed by the PEP, so that the implementation has some hope of being securable. Regards, Pat From tcdelaney at optusnet.com.au Mon Apr 30 22:59:29 2007 From: tcdelaney at optusnet.com.au (Tim Delaney) Date: Tue, 1 May 2007 06:59:29 +1000 Subject: [Python-3000] super(), class decorators, and PEP 3115 References: <5.1.1.6.0.20070430142844.03c96240@sparrow.telecommunity.com> Message-ID: <011b01c78b6a$72098810$0201a8c0@ryoko> From: "Guido van Rossum" > I'm not sure that your proposal for implementing an improved super has > anything over the currently most-favored proposal by Timothy Delaney. Call me Tim ;) Unfortunately, I can't change the format that work sends my name out as - Delaney, Timothy. I'm not going to be able to submit a spearate PEP for my proposal before COB April 30th (it's already the 1st over here ...) and I have to head to work. Would you prefer me to work with Calvin to get his existing PEP to match my proposal, or would you prefer a competing PEP? Tim Delaney From collinw at gmail.com Mon Apr 30 23:03:39 2007 From: collinw at gmail.com (Collin Winter) Date: Mon, 30 Apr 2007 14:03:39 -0700 Subject: [Python-3000] super(), class decorators, and PEP 3115 In-Reply-To: <011b01c78b6a$72098810$0201a8c0@ryoko> References: <5.1.1.6.0.20070430142844.03c96240@sparrow.telecommunity.com> <011b01c78b6a$72098810$0201a8c0@ryoko> Message-ID: <43aa6ff70704301403q5bf557c2wf43148f7a339353d@mail.gmail.com> On 4/30/07, Tim Delaney wrote: > Would you prefer me to work with Calvin to get his existing PEP to match my > proposal, or would you prefer a competing PEP? Please work together with Calvin. One PEP is enough. Thanks, Collin Winter From janssen at parc.com Mon Apr 30 23:15:02 2007 From: janssen at parc.com (Bill Janssen) Date: Mon, 30 Apr 2007 14:15:02 PDT Subject: [Python-3000] Traits/roles instead of ABCs In-Reply-To: <79990c6b0704301001ga0d2429sdaded9ac75fa15c5@mail.gmail.com> References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> <014201c78adc$ca70d960$f101a8c0@RaymondLaptop1> <438708814690534630@unknownmsgid> <79990c6b0704301001ga0d2429sdaded9ac75fa15c5@mail.gmail.com> Message-ID: <07Apr30.141511pdt."57996"@synergy1.parc.xerox.com> > > If we simply had a small standard extensible set of APIs, and a way to > > indicate that a value does or doesn't support one or more of them, > > we'd be done. We could either add a new mechanism to do this, or simply > > actually use the one we already have, which should work perfectly well for > > this purpose. > > And which has been around for many years, but has never actually been > used in practice (unlike, say, interfaces). Does this not say > something? Paul, Actually, it hasn't been around in workable form for years. I'm not sure of the actual chronology, but it wasn't really workable till 2.3, with reasonable unification of built-in and user-defined types, and the ability to subclass basic types like "dict". So there are lots of existing kludges that try to get around older limitations (like interfaces). The existence of these kludges is a symptom, not a fix. Bill From janssen at parc.com Mon Apr 30 23:19:15 2007 From: janssen at parc.com (Bill Janssen) Date: Mon, 30 Apr 2007 14:19:15 PDT Subject: [Python-3000] Traits/roles instead of ABCs In-Reply-To: <79990c6b0704301001ga0d2429sdaded9ac75fa15c5@mail.gmail.com> References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> <014201c78adc$ca70d960$f101a8c0@RaymondLaptop1> <438708814690534630@unknownmsgid> <79990c6b0704301001ga0d2429sdaded9ac75fa15c5@mail.gmail.com> Message-ID: <07Apr30.141916pdt."57996"@synergy1.parc.xerox.com> > On 30/04/07, Bill Janssen wrote: > > > 6) that doesn't freeze all of the key APIs in concrete > > > > After 15 years of experience with the key APIs, we could perhaps freeze some of > > them? > > After 15 years not being able to clearly state what "file-like" or > "mapping-like" means to different people, perhaps we should accept > that there is no clear-cut answer...? And that's a problem -- people are confused. Instead of throwing up our hands, I think we should define what "file-like" means. > > In other words, what APIs does this value support? > > In other words, what APIs does this value support? > > In other words, what APIs does this value support? > > Yes, but the ABC proposal is about establishing a vocabulary for > answering those questions, not about asking them. That's what "isinstance" does. The mechanism for asking is already there; we're just trying to standardize a few answers. And, by the way, I'm not sure adding the ability to override "isinstance" is a good idea. Bill From janssen at parc.com Mon Apr 30 23:22:36 2007 From: janssen at parc.com (Bill Janssen) Date: Mon, 30 Apr 2007 14:22:36 PDT Subject: [Python-3000] Traits/roles instead of ABCs In-Reply-To: <43aa6ff70704300950t4b8179beob3b8df7001a2819b@mail.gmail.com> References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> <014201c78adc$ca70d960$f101a8c0@RaymondLaptop1> <-3604285971386182607@unknownmsgid> <43aa6ff70704300950t4b8179beob3b8df7001a2819b@mail.gmail.com> Message-ID: <07Apr30.142241pdt."57996"@synergy1.parc.xerox.com> > No existing mechanism would allow PEP 3141 to be distributed as a > third-party module and still impact the built-in types; that's what > this whole business of __isinstance__ and __issubclass__ is about. Perhaps 3143 should not be distributed as a third-party module. Perhaps it should be built into the base Python. I think this "end-of-April" deadline, and Guido's ABC PEP, has spurred some useful discussion. I don't see why it has to cripple Python 3000 by ending that discussion prematurely. Bill From tcdelaney at optusnet.com.au Mon Apr 30 23:30:49 2007 From: tcdelaney at optusnet.com.au (Tim Delaney) Date: Tue, 1 May 2007 07:30:49 +1000 Subject: [Python-3000] super() PEP References: <5.1.1.6.0.20070430142844.03c96240@sparrow.telecommunity.com> <011b01c78b6a$72098810$0201a8c0@ryoko> <43aa6ff70704301403q5bf557c2wf43148f7a339353d@mail.gmail.com> Message-ID: <014901c78b6e$d2d66d80$0201a8c0@ryoko> From: "Collin Winter" > On 4/30/07, Tim Delaney wrote: >> Would you prefer me to work with Calvin to get his existing PEP to match >> my >> proposal, or would you prefer a competing PEP? > > Please work together with Calvin. One PEP is enough. Fine with me. Calvin - want to send me your latest draft, and I'll do some modifications? I think we've got to the point now where we can take this off-list. Cheers, Tim Delaney From pje at telecommunity.com Mon Apr 30 23:42:52 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 30 Apr 2007 17:42:52 -0400 Subject: [Python-3000] Traits/roles instead of ABCs In-Reply-To: <07Apr30.141511pdt."57996"@synergy1.parc.xerox.com> References: <79990c6b0704301001ga0d2429sdaded9ac75fa15c5@mail.gmail.com> <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com> <014201c78adc$ca70d960$f101a8c0@RaymondLaptop1> <438708814690534630@unknownmsgid> <79990c6b0704301001ga0d2429sdaded9ac75fa15c5@mail.gmail.com> Message-ID: <5.1.1.6.0.20070430173220.045a1648@sparrow.telecommunity.com> At 02:15 PM 4/30/2007 -0700, Bill Janssen wrote: >Actually, it hasn't been around in workable form for years. Correction: Zope's Interface package has been around since 2002; PEAK's originated in 2003. I'd say those count as both "workable form" and "for years". :) >I'm not sure of the actual chronology, but it wasn't really workable till 2.3, You probably mean 2.2, but I'm fairly certain that the earliest Interface packages implemented by Jim Fulton actually predated Python 2.2, using a different metaclass hook mechanism.