From steven.bethard at gmail.com Sat Jul 1 04:05:17 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Fri, 30 Jun 2006 20:05:17 -0600 Subject: [Python-3000] replace globals() and global statement with global builtin object Message-ID: BJ?rn Lindqvist wrote: > I have often wanted something similar to that for global > variables, instead of the global declaration: > > cache = None > def init(): > if not global.cache: > global.cache = init_cache() Redirected since this seemed like a Python 3000 kind of request. I like the idea, particularly because it coincides well with my usual uses for global/globals(). Seems like it might require some changes in things like eval and exec that take locals and globals dicts, but I don't know how much of a drawback that is. Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From guido at python.org Sat Jul 1 04:53:08 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 30 Jun 2006 19:53:08 -0700 Subject: [Python-3000] replace globals() and global statement with global builtin object In-Reply-To: References: Message-ID: On 6/30/06, Steven Bethard wrote: > BJ?rn Lindqvist wrote: > > I have often wanted something similar to that for global > > variables, instead of the global declaration: > > > > cache = None > > def init(): > > if not global.cache: > > global.cache = init_cache() > > Redirected since this seemed like a Python 3000 kind of request. I > like the idea, particularly because it coincides well with my usual > uses for global/globals(). Seems like it might require some changes > in things like eval and exec that take locals and globals dicts, but I > don't know how much of a drawback that is. You realize that *reading* a global doesn't need the "global." prefix, do you? So you could have written "if not cache: global.cache = init_cache()" in the function body. I'm not sure I like this asymmetry much. (I'm trying to help you understand that this idea just might not work.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Sat Jul 1 06:03:04 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Fri, 30 Jun 2006 22:03:04 -0600 Subject: [Python-3000] replace globals() and global statement with global builtin object In-Reply-To: References: Message-ID: On 6/30/06, Guido van Rossum wrote: > On 6/30/06, Steven Bethard wrote: > > BJ?rn Lindqvist wrote: > > > I have often wanted something similar to that for global > > > variables, instead of the global declaration: > > > > > > cache = None > > > def init(): > > > if not global.cache: > > > global.cache = init_cache() > > > > Redirected since this seemed like a Python 3000 kind of request. I > > like the idea, particularly because it coincides well with my usual > > uses for global/globals(). Seems like it might require some changes > > in things like eval and exec that take locals and globals dicts, but I > > don't know how much of a drawback that is. > > You realize that *reading* a global doesn't need the "global." prefix, > do you? So you could have written "if not cache: global.cache = > init_cache()" in the function body. That's also true for the global statement, right? That is, the ``if not cache`` part would not need a global statement, whlie the ``cache = init_cache()`` part would. That said, I agree it does nothing to solve the asymmetry. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From guido at python.org Sat Jul 1 06:47:54 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 30 Jun 2006 21:47:54 -0700 Subject: [Python-3000] replace globals() and global statement with global builtin object In-Reply-To: References: Message-ID: On 6/30/06, Steven Bethard wrote: > On 6/30/06, Guido van Rossum wrote: > > On 6/30/06, Steven Bethard wrote: > > > BJ?rn Lindqvist wrote: > > > > I have often wanted something similar to that for global > > > > variables, instead of the global declaration: > > > > > > > > cache = None > > > > def init(): > > > > if not global.cache: > > > > global.cache = init_cache() > > > > > > Redirected since this seemed like a Python 3000 kind of request. I > > > like the idea, particularly because it coincides well with my usual > > > uses for global/globals(). Seems like it might require some changes > > > in things like eval and exec that take locals and globals dicts, but I > > > don't know how much of a drawback that is. > > > > You realize that *reading* a global doesn't need the "global." prefix, > > do you? So you could have written "if not cache: global.cache = > > init_cache()" in the function body. > > That's also true for the global statement, right? That is, the ``if > not cache`` part would not need a global statement, whlie the ``cache > = init_cache()`` part would. > > That said, I agree it does nothing to solve the asymmetry. At least with the global statement, the notation for the variable reference remains the same. E.g. you can write x = f(x) and everybody will understand that the x on the left and right refer to the same variable. You'd lose that with globals.x = f(x) IMO that's enough of a drawback to give it a solid -1. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Sat Jul 1 19:19:15 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Sat, 1 Jul 2006 11:19:15 -0600 Subject: [Python-3000] replace globals() and global statement with global builtin object In-Reply-To: References: Message-ID: On 6/30/06, Guido van Rossum wrote: > On 6/30/06, Steven Bethard wrote: > > On 6/30/06, Guido van Rossum wrote: > > > On 6/30/06, Steven Bethard wrote: > > > > BJ?rn Lindqvist wrote: > > > > > I have often wanted something similar to that for global > > > > > variables, instead of the global declaration: > > > > > > > > > > cache = None > > > > > def init(): > > > > > if not global.cache: > > > > > global.cache = init_cache() > > > > > > > > Redirected since this seemed like a Python 3000 kind of request. I > > > > like the idea, particularly because it coincides well with my usual > > > > uses for global/globals(). Seems like it might require some changes > > > > in things like eval and exec that take locals and globals dicts, but I > > > > don't know how much of a drawback that is. > > > > > > You realize that *reading* a global doesn't need the "global." prefix, > > > do you? So you could have written "if not cache: global.cache = > > > init_cache()" in the function body. > > > > That's also true for the global statement, right? That is, the ``if > > not cache`` part would not need a global statement, whlie the ``cache > > = init_cache()`` part would. > > > > That said, I agree it does nothing to solve the asymmetry. > > At least with the global statement, the notation for the variable > reference remains the same. E.g. you can write > > x = f(x) > > and everybody will understand that the x on the left and right refer > to the same variable. > > You'd lose that with > > globals.x = f(x) > > IMO that's enough of a drawback to give it a solid -1. Sounds reasonable. Should I update PEP 3099? Something like this comes up every few months. Oh, also, is the -1 only to replacing the global keyword? Or is it also to the idea of replacing globals() with a builtin pseudo-namespace object like the one above? STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From guido at python.org Sun Jul 2 19:44:52 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 2 Jul 2006 19:44:52 +0200 Subject: [Python-3000] replace globals() and global statement with global builtin object In-Reply-To: References: Message-ID: On 7/1/06, Steven Bethard wrote: > On 6/30/06, Guido van Rossum wrote: > > On 6/30/06, Steven Bethard wrote: > > > On 6/30/06, Guido van Rossum wrote: > > > > On 6/30/06, Steven Bethard wrote: > > > > > BJ?rn Lindqvist wrote: > > > > > > I have often wanted something similar to that for global > > > > > > variables, instead of the global declaration: > > > > > > > > > > > > cache = None > > > > > > def init(): > > > > > > if not global.cache: > > > > > > global.cache = init_cache() > > > > > > > > > > Redirected since this seemed like a Python 3000 kind of request. I > > > > > like the idea, particularly because it coincides well with my usual > > > > > uses for global/globals(). Seems like it might require some changes > > > > > in things like eval and exec that take locals and globals dicts, but I > > > > > don't know how much of a drawback that is. > > > > > > > > You realize that *reading* a global doesn't need the "global." prefix, > > > > do you? So you could have written "if not cache: global.cache = > > > > init_cache()" in the function body. > > > > > > That's also true for the global statement, right? That is, the ``if > > > not cache`` part would not need a global statement, whlie the ``cache > > > = init_cache()`` part would. > > > > > > That said, I agree it does nothing to solve the asymmetry. > > > > At least with the global statement, the notation for the variable > > reference remains the same. E.g. you can write > > > > x = f(x) > > > > and everybody will understand that the x on the left and right refer > > to the same variable. > > > > You'd lose that with > > > > globals.x = f(x) > > > > IMO that's enough of a drawback to give it a solid -1. > > Sounds reasonable. Should I update PEP 3099? Something like this > comes up every few months. Yes, with reference to this thread. > Oh, also, is the -1 only to replacing the global keyword? Or is it > also to the idea of replacing globals() with a builtin > pseudo-namespace object like the one above? The -1 is specifically for the idea that globals.foo would be a way to reference the global variable foo. I personally don't consider the global statement a mistake, so I don't really want to have to think about changing it in py3k (there's enough that I *do* want to change). But I'm not quite feeling strongly about not messing with global as I feel about the particular proposal that started this thread, so I don't yet want to add "improve global" as a no-no to PEP 3099. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Mon Jul 3 03:49:43 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 03 Jul 2006 13:49:43 +1200 Subject: [Python-3000] replace globals() and global statement with global builtin object In-Reply-To: References: Message-ID: <44A877B7.7050008@canterbury.ac.nz> Guido van Rossum wrote: > I don't yet want to add "improve global" as a > no-no to PEP 3099. What about simply redefining it to mean "not local"? No syntax change at all, and just a very small semantic change. -- Greg From guido at python.org Mon Jul 3 07:06:58 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 3 Jul 2006 07:06:58 +0200 Subject: [Python-3000] replace globals() and global statement with global builtin object In-Reply-To: <44A877B7.7050008@canterbury.ac.nz> References: <44A877B7.7050008@canterbury.ac.nz> Message-ID: On 7/3/06, Greg Ewing wrote: > Guido van Rossum wrote: > > I don't yet want to add "improve global" as a > > no-no to PEP 3099. > > What about simply redefining it to mean "not local"? > No syntax change at all, and just a very small > semantic change. That can be debated. What's your goal? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From collinw at gmail.com Mon Jul 3 15:05:44 2006 From: collinw at gmail.com (Collin Winter) Date: Mon, 3 Jul 2006 15:05:44 +0200 Subject: [Python-3000] Removing functions from the operator module Message-ID: <43aa6ff70607030605p2ca6af88i3c716f2e4b7d192@mail.gmail.com> PEP 3100 mentions that isCallable() and sequenceIncludes() are to be removed from the operator module in Python 3000 because there are better, more obvious ways of spelling these things. So, on that note, should operator.truth() and operator.abs() be added to the to-be-removed list, in favour of the builtins bool() and abs()? Collin Winter From bjourne at gmail.com Mon Jul 3 15:51:11 2006 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Mon, 3 Jul 2006 15:51:11 +0200 Subject: [Python-3000] replace globals() and global statement with global builtin object In-Reply-To: References: Message-ID: <740c3aec0607030651j69e09ca0ra2153964724b583e@mail.gmail.com> On 7/1/06, Guido van Rossum wrote: > On 6/30/06, Steven Bethard wrote: > > BJ?rn Lindqvist wrote: > > > I have often wanted something similar to that for global > > > variables, instead of the global declaration: > > > > > > cache = None > > > def init(): > > > if not global.cache: > > > global.cache = init_cache() > > > > Redirected since this seemed like a Python 3000 kind of request. I > > like the idea, particularly because it coincides well with my usual > > uses for global/globals(). Seems like it might require some changes > > in things like eval and exec that take locals and globals dicts, but I > > don't know how much of a drawback that is. > > You realize that *reading* a global doesn't need the "global." prefix, > do you? So you could have written "if not cache: global.cache = > init_cache()" in the function body. > > I'm not sure I like this asymmetry much. I think the fix for that is to remove the "scope inheritance." I.e: cache = None def init(): if not cache: pass Throws a NameError because cache is not declared in function init's scope. So you would be forced to write: cache = None def init(): if not global.cache: global.cache = "foobar" I like the symmetry with self in classes. YMMV -- mvh Bj?rn From guido at python.org Mon Jul 3 15:57:12 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 3 Jul 2006 15:57:12 +0200 Subject: [Python-3000] replace globals() and global statement with global builtin object In-Reply-To: <740c3aec0607030651j69e09ca0ra2153964724b583e@mail.gmail.com> References: <740c3aec0607030651j69e09ca0ra2153964724b583e@mail.gmail.com> Message-ID: On 7/3/06, BJ?rn Lindqvist wrote: > On 7/1/06, Guido van Rossum wrote: > > On 6/30/06, Steven Bethard wrote: > > > BJ?rn Lindqvist wrote: > > > > I have often wanted something similar to that for global > > > > variables, instead of the global declaration: > > > > > > > > cache = None > > > > def init(): > > > > if not global.cache: > > > > global.cache = init_cache() > > > > > > Redirected since this seemed like a Python 3000 kind of request. I > > > like the idea, particularly because it coincides well with my usual > > > uses for global/globals(). Seems like it might require some changes > > > in things like eval and exec that take locals and globals dicts, but I > > > don't know how much of a drawback that is. > > > > You realize that *reading* a global doesn't need the "global." prefix, > > do you? So you could have written "if not cache: global.cache = > > init_cache()" in the function body. > > > > I'm not sure I like this asymmetry much. > > I think the fix for that is to remove the "scope inheritance." I.e: > > cache = None > def init(): > if not cache: > pass > > Throws a NameError because cache is not declared in function init's > scope. So you would be forced to write: > > cache = None > def init(): > if not global.cache: > global.cache = "foobar" > > I like the symmetry with self in classes. YMMV That can't work because imported names are also globals, as are classes and functions defined in the same module. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Jul 3 15:58:31 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 3 Jul 2006 15:58:31 +0200 Subject: [Python-3000] Removing functions from the operator module In-Reply-To: <43aa6ff70607030605p2ca6af88i3c716f2e4b7d192@mail.gmail.com> References: <43aa6ff70607030605p2ca6af88i3c716f2e4b7d192@mail.gmail.com> Message-ID: On 7/3/06, Collin Winter wrote: > PEP 3100 mentions that isCallable() and sequenceIncludes() are to be > removed from the operator module in Python 3000 because there are > better, more obvious ways of spelling these things. > > So, on that note, should operator.truth() and operator.abs() be added > to the to-be-removed list, in favour of the builtins bool() and abs()? I don't think so. Or else you'd have to remove operator.add as well. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From collinw at gmail.com Mon Jul 3 16:11:22 2006 From: collinw at gmail.com (Collin Winter) Date: Mon, 3 Jul 2006 16:11:22 +0200 Subject: [Python-3000] Removing functions from the operator module In-Reply-To: References: <43aa6ff70607030605p2ca6af88i3c716f2e4b7d192@mail.gmail.com> Message-ID: <43aa6ff70607030711y32c8072am4197d98eff2ae6b6@mail.gmail.com> On 7/3/06, Guido van Rossum wrote: > On 7/3/06, Collin Winter wrote: > > PEP 3100 mentions that isCallable() and sequenceIncludes() are to be > > removed from the operator module in Python 3000 because there are > > better, more obvious ways of spelling these things. > > > > So, on that note, should operator.truth() and operator.abs() be added > > to the to-be-removed list, in favour of the builtins bool() and abs()? > > I don't think so. Or else you'd have to remove operator.add as well. Stupid question: why? What's the better spelling of operator.add? Collin Winter From bjourne at gmail.com Mon Jul 3 16:14:20 2006 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Mon, 3 Jul 2006 16:14:20 +0200 Subject: [Python-3000] replace globals() and global statement with global builtin object In-Reply-To: References: <740c3aec0607030651j69e09ca0ra2153964724b583e@mail.gmail.com> Message-ID: <740c3aec0607030714t3ccdf139g93efd95636001201@mail.gmail.com> > > cache = None > > def init(): > > if not cache: > > pass > > > > Throws a NameError because cache is not declared in function init's > > scope. So you would be forced to write: > > > > cache = None > > def init(): > > if not global.cache: > > global.cache = "foobar" > > > > I like the symmetry with self in classes. YMMV > > That can't work because imported names are also globals, as are > classes and functions defined in the same module. I... didn't think about that. Thanks for crushing my idea. :) -- mvh Bj?rn From guido at python.org Mon Jul 3 16:40:19 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 3 Jul 2006 16:40:19 +0200 Subject: [Python-3000] Removing functions from the operator module In-Reply-To: <43aa6ff70607030711y32c8072am4197d98eff2ae6b6@mail.gmail.com> References: <43aa6ff70607030605p2ca6af88i3c716f2e4b7d192@mail.gmail.com> <43aa6ff70607030711y32c8072am4197d98eff2ae6b6@mail.gmail.com> Message-ID: + On 7/3/06, Collin Winter wrote: > On 7/3/06, Guido van Rossum wrote: > > On 7/3/06, Collin Winter wrote: > > > PEP 3100 mentions that isCallable() and sequenceIncludes() are to be > > > removed from the operator module in Python 3000 because there are > > > better, more obvious ways of spelling these things. > > > > > > So, on that note, should operator.truth() and operator.abs() be added > > > to the to-be-removed list, in favour of the builtins bool() and abs()? > > > > I don't think so. Or else you'd have to remove operator.add as well. > > Stupid question: why? What's the better spelling of operator.add? > > Collin Winter > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From murman at gmail.com Mon Jul 3 16:52:40 2006 From: murman at gmail.com (Michael Urman) Date: Mon, 3 Jul 2006 09:52:40 -0500 Subject: [Python-3000] Removing functions from the operator module In-Reply-To: References: <43aa6ff70607030605p2ca6af88i3c716f2e4b7d192@mail.gmail.com> <43aa6ff70607030711y32c8072am4197d98eff2ae6b6@mail.gmail.com> Message-ID: On 7/3/06, Guido van Rossum wrote: > On 7/3/06, Collin Winter wrote: > > Stupid question: why? What's the better spelling of operator.add? > > + I reject your slippery slope argument that we'd have to remove operator.add. Not that we'd be able to tell, but + is not callable. Or would that change as well? (Actually the more I briefly consider making in place operators the callables that perform the represented operation, the more attractive it seems, if it doesn't blow the parser out of the water.) SyntaxError: invalid syntax >>> reduce(+, [1,2,3,4]) File "", line 1 reduce(+, [1,2,3,4]) ^ SyntaxError: invalid syntax Michael -- Michael Urman http://www.tortall.net/mu/blog From collinw at gmail.com Mon Jul 3 16:55:38 2006 From: collinw at gmail.com (Collin Winter) Date: Mon, 3 Jul 2006 16:55:38 +0200 Subject: [Python-3000] Removing functions from the operator module In-Reply-To: References: <43aa6ff70607030605p2ca6af88i3c716f2e4b7d192@mail.gmail.com> <43aa6ff70607030711y32c8072am4197d98eff2ae6b6@mail.gmail.com> Message-ID: <43aa6ff70607030755p32c4e32drc0c66620913afb4d@mail.gmail.com> On 7/3/06, Guido van Rossum wrote: > + But you can't pass a plus sign to higher-order functions. > On 7/3/06, Collin Winter wrote: > > On 7/3/06, Guido van Rossum wrote: > > > On 7/3/06, Collin Winter wrote: > > > > PEP 3100 mentions that isCallable() and sequenceIncludes() are to be > > > > removed from the operator module in Python 3000 because there are > > > > better, more obvious ways of spelling these things. > > > > > > > > So, on that note, should operator.truth() and operator.abs() be added > > > > to the to-be-removed list, in favour of the builtins bool() and abs()? > > > > > > I don't think so. Or else you'd have to remove operator.add as well. > > > > Stupid question: why? What's the better spelling of operator.add? > > > > Collin Winter > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > From guido at python.org Mon Jul 3 17:16:01 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 3 Jul 2006 17:16:01 +0200 Subject: [Python-3000] Removing functions from the operator module In-Reply-To: References: <43aa6ff70607030605p2ca6af88i3c716f2e4b7d192@mail.gmail.com> <43aa6ff70607030711y32c8072am4197d98eff2ae6b6@mail.gmail.com> Message-ID: On 7/3/06, Michael Urman wrote: > On 7/3/06, Guido van Rossum wrote: > > On 7/3/06, Collin Winter wrote: > > > Stupid question: why? What's the better spelling of operator.add? > > > > + > > I reject your slippery slope argument that we'd have to remove > operator.add. Not that we'd be able to tell, but + is not callable. It turns out I was misled by Collin's claim that the PEP wants isCallable and sequenceIncludes removed "because there are better, more obvious ways to spell these things." He must have made up the motivation, as the PEP doesn't give any (and the footnote [17] is particularly unhelpful -- who came up with the idea of having 7 different places in the PEP link to 5 different parts of the manual via the same footnote?). I believe that the real reasons are different in each case: - operator.sequenceIncludes() is redundant with operator.contains - operator.isCallable() corresponds to the callable() built-in and the PEP also wants to remove callable(). I don't see how this generalizes to operator.truth() or operator.abs(). The key thing to understand about the operator module is that its methods correpond to the C-level APIs. As the C-level API changes we can expect more changes to the operator module. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From collinw at gmail.com Mon Jul 3 17:40:43 2006 From: collinw at gmail.com (Collin Winter) Date: Mon, 3 Jul 2006 17:40:43 +0200 Subject: [Python-3000] Removing functions from the operator module In-Reply-To: References: <43aa6ff70607030605p2ca6af88i3c716f2e4b7d192@mail.gmail.com> <43aa6ff70607030711y32c8072am4197d98eff2ae6b6@mail.gmail.com> Message-ID: <43aa6ff70607030840l3bf31fbdw45664884578d3f83@mail.gmail.com> On 7/3/06, Guido van Rossum wrote: > On 7/3/06, Michael Urman wrote: > > On 7/3/06, Guido van Rossum wrote: > > > On 7/3/06, Collin Winter wrote: > > > > Stupid question: why? What's the better spelling of operator.add? > > > > > > + > > > > I reject your slippery slope argument that we'd have to remove > > operator.add. Not that we'd be able to tell, but + is not callable. > > It turns out I was misled by Collin's claim that the PEP wants > isCallable and sequenceIncludes removed "because there are better, > more obvious ways to spell these things." He must have made up the > motivation, as the PEP doesn't give any. I was inferring that motivation based on the presence of alternate, shorter spellings: in the case of operator.sequenceIncludes(), the shorter version is operator.contains(); in the case of operator.isCallable(), the shorter version is the callable() builtin. (I thought I had seen some mailing list traffic hinting that callable() would stick around after all; I guess I was wrong.) > I believe that the real reasons are different in each case: > > - operator.sequenceIncludes() is redundant with operator.contains > > - operator.isCallable() corresponds to the callable() built-in and the > PEP also wants to remove callable(). > > I don't see how this generalizes to operator.truth() or operator.abs(). Because these are redundant as well: operator.truth() is redundant with bool() and operator.abs() with the builtin abs(). Collin Winter From greg.ewing at canterbury.ac.nz Tue Jul 4 01:34:17 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 04 Jul 2006 11:34:17 +1200 Subject: [Python-3000] replace globals() and global statement with global builtin object In-Reply-To: <740c3aec0607030651j69e09ca0ra2153964724b583e@mail.gmail.com> References: <740c3aec0607030651j69e09ca0ra2153964724b583e@mail.gmail.com> Message-ID: <44A9A979.5030402@canterbury.ac.nz> BJ?rn Lindqvist wrote: > I think the fix for that is to remove the "scope inheritance." I.e: > > cache = None > def init(): > if not cache: > pass > > Throws a NameError because cache is not declared in function init's > scope. And do you want def f(): print "Eff!" def g(): f() g() to throw a NameError, too? If not, how do you tell the difference? -- Greg From greg.ewing at canterbury.ac.nz Tue Jul 4 01:37:31 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 04 Jul 2006 11:37:31 +1200 Subject: [Python-3000] Removing functions from the operator module In-Reply-To: References: <43aa6ff70607030605p2ca6af88i3c716f2e4b7d192@mail.gmail.com> <43aa6ff70607030711y32c8072am4197d98eff2ae6b6@mail.gmail.com> Message-ID: <44A9AA3B.7050700@canterbury.ac.nz> Guido van Rossum wrote: > > On 7/3/06, Collin Winter wrote: > > > Stupid question: why? What's the better spelling of operator.add? > > + But that's not a function you can pass around, whereas bool and abs are. -- Greg From krstic at fas.harvard.edu Tue Jul 4 01:58:59 2006 From: krstic at fas.harvard.edu (Ivan Krstic) Date: Mon, 03 Jul 2006 19:58:59 -0400 Subject: [Python-3000] xrange vs. int.__getslice__ In-Reply-To: <448E6A74.3010409@renet.ru> References: <448E6A74.3010409@renet.ru> Message-ID: <44A9AF43.3070004@fas.harvard.edu> Vladimir 'Yu' Stepanov wrote: > --------------------------------------------- > for i in xrange(100): pass > vs. > for i in int[:100]: pass > --------------------------------------------- I'm a strong -1 on this; it makes it look like float[:100] should also be legal and reasonable, but I can't see how it could possibly be. -- Ivan Krstic | GPG: 0x147C722D From vys at renet.ru Tue Jul 4 11:37:12 2006 From: vys at renet.ru (Vladimir 'Yu' Stepanov) Date: Tue, 04 Jul 2006 13:37:12 +0400 Subject: [Python-3000] xrange vs. int.__getslice__ In-Reply-To: <44A9AF43.3070004@fas.harvard.edu> References: <448E6A74.3010409@renet.ru> <44A9AF43.3070004@fas.harvard.edu> Message-ID: <44AA36C8.9030409@renet.ru> Ivan Krstic wrote: > Vladimir 'Yu' Stepanov wrote: > >> --------------------------------------------- >> for i in xrange(100): pass >> vs. >> for i in int[:100]: pass >> --------------------------------------------- >> > > I'm a strong -1 on this; it makes it look like float[:100] should also > be legal and reasonable, but I can't see how it could possibly be. In general that I thought to stop only on type int :). But your idea to distribute it on other types seems to me quite good even if it has been resulted as counter-argument. It is how much possible to understand slice in Python language is a reception of a range of values with the certain step. By default this step is equal to 1 for objects of any type. It is possible to assume, that the similar behaviour can be at types float, decimal, long and other. As the method __getslice__ till now is not involved by any image for numbers. Advantage of such approach is that it is not necessary to hold function in space __builtins__. Each numerical object (excepting unless complex) can describe the number of values. It is possible to give an example still. We shall tell, it is necessary to pass on ranges of values with 0 on 1000 numbers with a floating point. Example: ----------------------------- for i in xrange(1000): i = float(i) .. do something .. vs. for i in float[:1000]: .. do something .. ----------------------------- Now for fractional types widely to use this opportunity it will not turn out. C-API as arguments are transferred only ?-`long' integers. To achieve, that a step of less unit it was now impossible, therefore as all arguments will be approximated. From diogoko at gmail.com Tue Jul 4 14:59:39 2006 From: diogoko at gmail.com (Diogo Kollross) Date: Tue, 4 Jul 2006 09:59:39 -0300 Subject: [Python-3000] xrange vs. int.__getslice__ In-Reply-To: <44AA36C8.9030409@renet.ru> References: <448E6A74.3010409@renet.ru> <44A9AF43.3070004@fas.harvard.edu> <44AA36C8.9030409@renet.ru> Message-ID: > It is how much possible to understand slice in Python language is a > reception of a range of values with the certain step. By default this > step is equal to 1 for objects of any type. It is possible to assume, > that the similar behaviour can be at types float, decimal, long and > other. As the method __getslice__ till now is not involved by any > image for numbers. How the following case should work? class even(int): .. "Only the even integers..." for i in even[:100]: .. do something From ncoghlan at iinet.net.au Tue Jul 4 15:14:14 2006 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue, 04 Jul 2006 23:14:14 +1000 Subject: [Python-3000] The main module in Py3k Message-ID: <44AA69A6.9070806@iinet.net.au> PEP 299 recently got added to the list of ideas to be reconsidered for Py3k. The motivation seemed to be that the current idiom is both ugly and unintuitive if you haven't drunk enough of the relevant Koolaid. I can't really argue with that assessment, but I think PEP 299 proposes more radical changes to the idiom than are really necessary. My preferred solution would look like this: 1. Determining if the current module is the main module: The boolean value __main__ would be defined in all modules. It would be set to True in the main module, and False everywhere else. The existing "if __name__ == '__main__' idiom would be replaced by a simple "if __main__:" A boolean flag that's true in the main module and false everywhere else should be straightforward to both teach and remember (even simpler than trying to explain a magic function that gets called only when the module is the main module). This part of the proposal is all that most users would ever care about. 2. Accessing the main module from another module: A new attribute in the sys module "main" would always reference the main module of the application. The main module would also be stored in sys.modules under the name "sys.main", so that "import sys.main" and "from sys import main" would correctly retrieve the main module. Usages of the form "import __main__" would be replaced by "import sys.main", and "__main__." would be replaced by "sys.main.". This part of the proposal is to avoid the name conflict between "import __main__" and the proposed boolean flag. An alternative would be to use a different name for the boolean flag (e.g. invert the sense of it, and call it "__imported__"). 3. Relative imports from the main module: Files that are executed directly would have their __name__ attribute set to "
". Modules executed using -m would have their __name__ attribute set to their real module name as provided on the command line. The main module would actually be referenced from sys.modules twice (once under the name "sys.main" and once under the value of sys.main.__name__) Relative imports from directly executed modules would continue to behave as for the 2.x series (since the import machinery only cares about the number of dots preceding the module name, "
" will behave the same as "__main__"). Relative imports from modules executed with -m, however, would now be able to use the normal relative import mechanism and the __module_name__ workaround proposed for the 2.x series could disappear. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Tue Jul 4 16:22:27 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 4 Jul 2006 16:22:27 +0200 Subject: [Python-3000] [Python-Dev] xrange vs. int.__getslice__ In-Reply-To: <448E6A74.3010409@renet.ru> References: <448E6A74.3010409@renet.ru> Message-ID: Please don't cross-post python-dev and python-3000. This proposal will not be adopted in Py3k. Instead, range() will become more like xrange(). --Guido On 6/13/06, Vladimir 'Yu' Stepanov wrote: > You were bothered yet with function xrange ? :) I suggest to replace it. > > --------------------------------------------- > for i in xrange(100): pass > vs. > for i in int[:100]: pass > --------------------------------------------- > > --------------------------------------------- > for i in xrange(1000, 1020): pass > vs. > for i in int[1000:1020]: pass > --------------------------------------------- > > --------------------------------------------- > for i in xrange(200, 100, -2): pass > vs. > for i in int[200:100:-2]: pass > --------------------------------------------- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Jul 4 16:35:56 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 4 Jul 2006 16:35:56 +0200 Subject: [Python-3000] The main module in Py3k In-Reply-To: <44AA69A6.9070806@iinet.net.au> References: <44AA69A6.9070806@iinet.net.au> Message-ID: On 7/4/06, Nick Coghlan wrote: > PEP 299 recently got added to the list of ideas to be reconsidered for Py3k. > The motivation seemed to be that the current idiom is both ugly and > unintuitive if you haven't drunk enough of the relevant Koolaid. > > I can't really argue with that assessment, but I think PEP 299 proposes more > radical changes to the idiom than are really necessary. > > My preferred solution would look like this: > > 1. Determining if the current module is the main module: > > The boolean value __main__ would be defined in all modules. It would be set to > True in the main module, and False everywhere else. > > The existing "if __name__ == '__main__' idiom would be replaced by a simple > "if __main__:" > > A boolean flag that's true in the main module and false everywhere else should > be straightforward to both teach and remember (even simpler than trying to > explain a magic function that gets called only when the module is the main > module). > > This part of the proposal is all that most users would ever care about. Two comments: - I'm not convinced that it's worth changing this. - *If* we're going to change this, we should reconsider whether it's really necessary to spell this with __underscores__. > 2. Accessing the main module from another module: > > A new attribute in the sys module "main" would always reference the main > module of the application. The main module would also be stored in sys.modules > under the name "sys.main", so that "import sys.main" and "from sys import > main" would correctly retrieve the main module. > > Usages of the form "import __main__" would be replaced by "import sys.main", > and "__main__." would be replaced by "sys.main.". > > This part of the proposal is to avoid the name conflict between "import > __main__" and the proposed boolean flag. An alternative would be to use a > different name for the boolean flag (e.g. invert the sense of it, and call it > "__imported__"). Please consider what the use case is for accessing the main module, and whether we should consider providing a different solution for that use case rather than a different way to access the main module. > 3. Relative imports from the main module: > > Files that are executed directly would have their __name__ attribute set to > "
". Modules executed using -m would have their __name__ attribute set to > their real module name as provided on the command line. > > The main module would actually be referenced from sys.modules twice (once > under the name "sys.main" and once under the value of sys.main.__name__) > > Relative imports from directly executed modules would continue to behave as > for the 2.x series (since the import machinery only cares about the number of > dots preceding the module name, "
" will behave the same as "__main__"). > > Relative imports from modules executed with -m, however, would now be able to > use the normal relative import mechanism and the __module_name__ workaround > proposed for the 2.x series could disappear. Frankly, I am beginning to regret ever having endorsed the -m option. It seems leads to an ever increasing series of discussions about mundane details that aren't worth the intellectual power devoted to them. I still think that relative import from a main module is a solution in search of a problem, and I am beginning to feel the same about the other issues that you are addressing above. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rasky at develer.com Tue Jul 4 18:01:08 2006 From: rasky at develer.com (Giovanni Bajo) Date: Tue, 4 Jul 2006 18:01:08 +0200 Subject: [Python-3000] The main module in Py3k References: <44AA69A6.9070806@iinet.net.au> Message-ID: <041c01c69f83$1036d940$d503030a@trilan> Guido van Rossum wrote: >> Relative imports from modules executed with -m, however, would now >> be able to use the normal relative import mechanism and the >> __module_name__ workaround proposed for the 2.x series could >> disappear. > > Frankly, I am beginning to regret ever having endorsed the -m option. In case you mean the -m pkg.mod spelling, I agree. It's bringing a lot of complexity, including the __module_name__ hack, for exactly what gain? So that one doesn't have to write 2 liner wrapper installed as top-level module in the site-packages? -- Giovanni Bajo From vys at renet.ru Tue Jul 4 19:07:43 2006 From: vys at renet.ru (Vladimir 'Yu' Stepanov) Date: Tue, 04 Jul 2006 21:07:43 +0400 Subject: [Python-3000] range vs. int.__getslice__ In-Reply-To: References: <448E6A74.3010409@renet.ru> <44A9AF43.3070004@fas.harvard.edu> <44AA36C8.9030409@renet.ru> Message-ID: <44AAA05F.7040605@renet.ru> Diogo Kollross wrote: >> It is how much possible to understand slice in Python language is a >> reception of a range of values with the certain step. By default this >> step is equal to 1 for objects of any type. It is possible to assume, >> that the similar behaviour can be at types float, decimal, long and >> other. As the method __getslice__ till now is not involved by any >> image for numbers. >> > > How the following case should work? > > class even(int): > .. "Only the even integers..." > > for i in even[:100]: > .. do something This method should not be accessible through the inherited classes. Ambiguity from the point of view of the user (you have already resulted an example) will be shown differently. It will be probably necessary to add one more METH_* an option. For example METH_NOEXPORT. Besides optimization should be not worse existing range, that is impossible without adaptation of a method under concrete type of data. The method __getslice__ cannot be set for type with option METH_CLASS or METH_STATIC. I expect occurrence of the syntax, satisfying to needs of the given method. Recently it was discussed parametrization of types. In this connection, how it will be realized __getitem__ for work with classes ? The similar design could be used and for alternative realization of syntax `range': http://mail.python.org/pipermail/python-3000/2006-May/002149.html That it would be possible is high-grade to work and with other types of data, except for int and long, it is required that PEP-357 has been completed: http://www.python.org/dev/peps/pep-0357/ Thanks. -- Vladimir From g.brandl at gmx.net Tue Jul 4 21:54:22 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 04 Jul 2006 21:54:22 +0200 Subject: [Python-3000] [Python-Dev] xrange vs. int.__getslice__ In-Reply-To: References: <448E6A74.3010409@renet.ru> Message-ID: Guido van Rossum wrote: > Please don't cross-post python-dev and python-3000. > > This proposal will not be adopted in Py3k. Instead, range() will > become more like xrange(). Added to PEP 3099. Georg From guido at python.org Tue Jul 4 23:58:53 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 4 Jul 2006 23:58:53 +0200 Subject: [Python-3000] The main module in Py3k In-Reply-To: <041c01c69f83$1036d940$d503030a@trilan> References: <44AA69A6.9070806@iinet.net.au> <041c01c69f83$1036d940$d503030a@trilan> Message-ID: \On 7/4/06, Giovanni Bajo wrote: > Guido van Rossum wrote: > > >> Relative imports from modules executed with -m, however, would now > >> be able to use the normal relative import mechanism and the > >> __module_name__ workaround proposed for the 2.x series could > >> disappear. > > > > Frankly, I am beginning to regret ever having endorsed the -m option. > > In case you mean the -m pkg.mod spelling, I agree. It's bringing a lot of > complexity, including the __module_name__ hack, for exactly what gain? So > that one doesn't have to write 2 liner wrapper installed as top-level module > in the site-packages? And by extension even the -m mod spelling, since its introduction led to the argument that -m pkg.mod should not be left behind... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Wed Jul 5 03:19:38 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 05 Jul 2006 13:19:38 +1200 Subject: [Python-3000] The main module in Py3k In-Reply-To: <44AA69A6.9070806@iinet.net.au> References: <44AA69A6.9070806@iinet.net.au> Message-ID: <44AB13AA.5070600@canterbury.ac.nz> Nick Coghlan wrote: > The existing "if __name__ == '__main__' idiom would be replaced by a simple > "if __main__:" That sounds good. > A new attribute in the sys module "main" would always reference the main > module of the application. The main module would also be stored in sys.modules > under the name "sys.main", Why not just __main__ as the alias name? Is there any need to invent a new name for this? > Files that are executed directly would have their __name__ attribute set to > "
". Again, why not leave it as "__main__" in this case? And if it must change, does it really have to be something that's not a valid identifier and/or doesn't correspond to anything in sys.modules? -- Greg From ncoghlan at gmail.com Wed Jul 5 12:03:58 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 05 Jul 2006 20:03:58 +1000 Subject: [Python-3000] The main module in Py3k In-Reply-To: References: <44AA69A6.9070806@iinet.net.au> <041c01c69f83$1036d940$d503030a@trilan> Message-ID: <44AB8E8E.20702@gmail.com> Guido van Rossum wrote: > \On 7/4/06, Giovanni Bajo wrote: >> Guido van Rossum wrote: >> >>>> Relative imports from modules executed with -m, however, would now >>>> be able to use the normal relative import mechanism and the >>>> __module_name__ workaround proposed for the 2.x series could >>>> disappear. >>> Frankly, I am beginning to regret ever having endorsed the -m option. >> In case you mean the -m pkg.mod spelling, I agree. It's bringing a lot of >> complexity, including the __module_name__ hack, for exactly what gain? So >> that one doesn't have to write 2 liner wrapper installed as top-level module >> in the site-packages? The original -m switch was added so that it was possible to easily run utility modules (like pdb, timeit, profile, etc): 1. without the user needing to care about how the current Python installation is laid out on the filesystem 2. without module developers needing to worry about namespace conflicts with other applications on the system path On any Python 2.4 system, the following runs a script: python script.py And the following profiles it: python -m profile script.py Before 2.4, you had to figure out the path to the utility module before you could run it: C:\>python -c "import profile; print profile.__file__" c:\python24\lib\profile.pyc C:\>python c:/python24/lib/profile.py script.py The second command was not only *platform* dependent, but potentially even *machine* dependent, if the standard library was in an unusual location. PEP 338 expanded the -m switch to packages so that: 1. packages could be used to avoid namespace conflicts for utility modules as well as support modules 2. utility modules could benefit from the file system independence provided by PEP 302 For example, 2.5 makes it trivial to run pychecker over a script: python -m pychecker.checker script.py Before 2.5, you had to figure out the filename as for the top level utility module example above. And if it turned out the answer referred to a module stored in a zip file, or if you only had a .pyc file (without the source file), then you were fresh out of luck - even though the Python interpreter knew exactly where to find the script, how to extract it, and how to run it, it didn't provide any way to run it as the main module (because the main module had to be a real source file). As far as the question of allowing relative imports from a main module goes, they're useful for exactly the same reason that relative imports are useful from *any* module that forms part of a package: it allows the entire package to be relocated to a different part of the module namespace without requiring any changes to the modules inside the package. If you use absolute imports, then changing the name of the package, or moving it inside another package, requires touching every single module in the package in order to update the import statements. > And by extension even the -m mod spelling, since its introduction led > to the argument that -m pkg.mod should not be left behind... The genesis of this really lies in PEPs 273 and 302 and the difficulty of locating the different parts of the Python standard library on the filesystem. The key thing about the implementation of those two PEPs is that they are what originally broke the previously straightforward mapping between the file system directory structure and the package namespace structure. They provided a *lot* of flexibility in packaging Python applications, but were unable to help with packaging of the main module of the application. The -m switch was really just a small extension to allow this idea of filesystem independence to be applied to the main module as well as support modules. If filesystem independence is a bad thing, then PEPs 273 and 302 should never have been accepted. If file system independence is a good thing, then the main module shouldn't be treated as a second class citizen. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Wed Jul 5 12:54:37 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 5 Jul 2006 12:54:37 +0200 Subject: [Python-3000] Switch and static, redux Message-ID: I'm sitting here at EuroPython listening to a speaker who has analyzed interaction patterns on python-dev. (She represents me as a red dot and Raymond H as a yellow star in a diagram. I'm trying not to read too much into that. :-) One of her comments struck home: the open source development process, which is often represented as exclusively taking place on line, is actually augmented in a significant way by direct physical interaction between developers at events like EuroPython (and PyCon, sprints etc.). A good example is the discussion I had over breakfast with Marc-Andre and Samuele today. We rehashed switch and static/once, and I believe the discussion has influenced my position a bit. Short version: I want to press on with switch option 3 (def-time freezing), and I'm more than ever skeptical about static or once. (Incidentally, the semantics for this that most people like best, including myself, are "captured each time 'def' is executed, which makes 'once' a rather misleading name, I have to agree with Samuele. So, for lack of a better word, I'll call it static for now, realizing that that also isn't such a great name given its meaning in C and Java.) When I wrote PEP 3103 I ended with a conclusion that requested more work on static/once ("precomputed values"). Talking about that this morning, we noted that there were really two different uses of 'static' in the discussion. Both are intended to replace the use of parameters with default values: one is the speed-up of a built-in or other constant reference in a short method, while the other is the capture of the current value of a variable in an outer scope (typically a loop variable) when defining an inner function. These cases are very different -- the optimization is just that, while the variable capture is essential for the meaning of the inner function (see another recent thread). Samuele has been working for a long time on compiler tricks that will do the optimization without hints from the user, and I think we can change the rules of the language slightly to support this better (these ideas aren't new). Import * is a bit of a snag but I think we can find a way to make it work. So the only remaining real use case for static is capturing the value of an outer variable. But for this purpose, static expressions are arguably just as clumsy if not more than the default-parameter trick we currently use, especially if the captured value is used more than once in the inner function; we'd have to write 'static x' each time, or we have to write something like "x = static x" (if that's even legal -- it might have to be "x_ = static x" if we disallow static expressions referencing variables that are shadowed by locals). None of this makes static very attractive compared to the default-parameter trick. On the other hand, we (those present at the breakfast table) all agreed that the switch statement can stand on its own and does not need static to make it useful. Rather, freezing the cases at def time, while different from anything Python does today, produces no surprises when used idiomatically; and when used non-idiomatically, the rule is simple enough to be able to predict exactly what will happen, no matter how obfuscated the code. So, my proposal is to give up on static, accept PEP 3103 with the following options: - Syntax alternative 2+B (unindented cases, 'case in ...' for multiple cases). - Semantics option 3 (def-time freezing) Do we need any more discussion about the PEP before I pronounce? (I'm not super confident about the syntax alternatives yet.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas at python.org Wed Jul 5 13:00:18 2006 From: thomas at python.org (Thomas Wouters) Date: Wed, 5 Jul 2006 13:00:18 +0200 Subject: [Python-3000] Switch and static, redux In-Reply-To: References: Message-ID: <9e804ac0607050400m5a98dccam6cdc1bfec0ee7a2@mail.gmail.com> On 7/5/06, Guido van Rossum wrote: > One of her comments struck home: the open source development process, > which is often represented as exclusively taking place on line, is > actually augmented in a significant way by direct physical interaction > between developers at events like EuroPython (and PyCon, sprints > etc.). Speaking of that, is anyone else coming to the python-dev sprint at Google in August? Py3K is one of the topics :) http://wiki.python.org/moin/GoogleSprint -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20060705/dbdd29ce/attachment.html From and-dev at doxdesk.com Wed Jul 5 16:10:22 2006 From: and-dev at doxdesk.com (Andrew Clover) Date: Wed, 05 Jul 2006 23:10:22 +0900 Subject: [Python-3000] Switch and static, redux In-Reply-To: References: Message-ID: <44ABC84E.5060408@doxdesk.com> Guido van Rossum wrote: > (I'm not super confident about the syntax alternatives yet.) The switch:-without-indent business still naggingly makes me feel uncomfortable. Is this a valid no-case-switch? switch cake: print 'hello' Here's another syntax off the top of my head - put the first case in the switch statement? switch biscuit.type if 'digestive': ... elif in 'jammy_dodger', 'garibaldi': ... else: ... -- And Clover mailto:and at doxdesk.com http://www.doxdesk.com/ From aahz at pythoncraft.com Wed Jul 5 16:35:09 2006 From: aahz at pythoncraft.com (Aahz) Date: Wed, 5 Jul 2006 07:35:09 -0700 Subject: [Python-3000] August Google sprint In-Reply-To: <9e804ac0607050400m5a98dccam6cdc1bfec0ee7a2@mail.gmail.com> References: <9e804ac0607050400m5a98dccam6cdc1bfec0ee7a2@mail.gmail.com> Message-ID: <20060705143509.GB687@panix.com> On Wed, Jul 05, 2006, Thomas Wouters wrote: > > Speaking of that, is anyone else coming to the python-dev sprint at Google > in August? Py3K is one of the topics :) > http://wiki.python.org/moin/GoogleSprint I'd love to because it's local, but I'm actually going to be at Worldcon instead -- dunno how many other people are in that boat. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "I saw `cout' being shifted "Hello world" times to the left and stopped right there." --Steve Gonedes From g.brandl at gmx.net Wed Jul 5 16:39:46 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 05 Jul 2006 16:39:46 +0200 Subject: [Python-3000] Switch and static, redux In-Reply-To: <44ABC84E.5060408@doxdesk.com> References: <44ABC84E.5060408@doxdesk.com> Message-ID: Andrew Clover wrote: > Guido van Rossum wrote: > >> (I'm not super confident about the syntax alternatives yet.) > > The switch:-without-indent business still naggingly makes me feel > uncomfortable. > > Is this a valid no-case-switch? > > switch cake: > print 'hello' There's no "no-case-switch". As I understand it, every switch must have at least one case. I'm not very comfortable about the no-additional-indentation-after-colon syntax too, but I also realize that requiring two levels of indentation for a single statement is too much. > Here's another syntax off the top of my head - put the first case in the > switch statement? > switch biscuit.type if 'digestive': > ... > elif in 'jammy_dodger', 'garibaldi': > ... > else: > ... For a large switch, it's too easy to confuse this with an if-else in case the initial switch is off-screen. Georg From guido at python.org Wed Jul 5 16:56:19 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 5 Jul 2006 16:56:19 +0200 Subject: [Python-3000] Switch and static, redux In-Reply-To: <44ABC84E.5060408@doxdesk.com> References: <44ABC84E.5060408@doxdesk.com> Message-ID: On 7/5/06, Andrew Clover wrote: > Guido van Rossum wrote: > > > (I'm not super confident about the syntax alternatives yet.) > > The switch:-without-indent business still naggingly makes me feel > uncomfortable. Me too, but only on even-numbered days... > Is this a valid no-case-switch? > > switch cake: > print 'hello' No. A switch must have one or more case suites and then optionally an else suite. > Here's another syntax off the top of my head - put the first case in the > switch statement? > > switch biscuit.type if 'digestive': > ... > elif in 'jammy_dodger', 'garibaldi': > ... > else: > ... Ah please. If you *have* to make up new syntax from a random jumble of keywords, at least make it a bit fun, like Barry's "as if" proposal in another thread. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Wed Jul 5 19:50:56 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 5 Jul 2006 13:50:56 -0400 Subject: [Python-3000] Switch and static, redux In-Reply-To: References: Message-ID: <9D1EBB55-246C-475A-B949-A2D7080ADDE9@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 5, 2006, at 6:54 AM, Guido van Rossum wrote: > So, my proposal is to give up on static, accept PEP 3103 with the > following options: > - Syntax alternative 2+B (unindented cases, 'case in ...' for > multiple cases). > - Semantics option 3 (def-time freezing) > > Do we need any more discussion about the PEP before I pronounce? (I'm > not super confident about the syntax alternatives yet.) I've been away for a couple of weeks and unsub'd from this list, so I haven't followed all the discussions lately. I have to say that PEP 3103 is an excellent summary of all the issues, and in reading it, several of my preferences for a Python switch statement have, er, switched after reading it. Thanks Guido! While conceptually I agree with your conclusions on semantics, I think my gut still wants school 1a because to me that is the most natural way of thinking about how a switch statement should work. I think that if that were adopted, there would still be value in a switch statement even without the optimization because it would be so familiar to people coming from other languages. One of the questions I still get from time-to-time from Python newbies is why there is no switch statement. Even after explaining if-chains, I still get a "yeah, but...". My main concern with semantic option 3 is that the rules and restrictions it imposes will be confusing or obscure to the casual Python programmer. Maybe in practice those restrictions won't ordinarily be visible, but I'm not sure we'll know until the feature is implemented and available for use. I can suspend judgement for now because your arguments for semantic option 3 are so compelling (or rather, your arguments against the alternatives are so compelling). I also don't have a good feel for whether those restrictions would be common enough that you'd end up falling back to if-chains most of the time anyway. If that were the case, then switch wouldn't be worth the added language complexity. As for the syntax question, the no-indent-for-cases make me queasy too, but maybe my opinion is clouded by my python-mode hat. OTOH, I'm used to excessive indentation with try/except/finally so it doesn't bother me all that much (okay, okay, so we're trading one for the other :). I strongly prefer 'case in' for multiple cases than any of the other alternatives. I'd also be mildly in favor of default fall-thru a la C, but with 'case in' I don't care too much. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (Darwin) iQCVAwUBRKv8AHEjvBPtnXfVAQJmSQP+JAHMZ/F4MZiudju73oDKl2p3L8g0NKE4 yAuda7scht86uNC/ocHEcd+jMMugXNQBSyGWuuEXTj+7FpNTo9KHTWPqX5EJdWKf 5p3ly6tS7nSelYYPwA65cYm6kEz5byZL0bFmUfJGaE6DBWs1asZUZCs2KC6m2YYn KVo309EL09c= =UIRI -----END PGP SIGNATURE----- From greg.ewing at canterbury.ac.nz Thu Jul 6 02:24:19 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Jul 2006 12:24:19 +1200 Subject: [Python-3000] Switch and static, redux In-Reply-To: References: Message-ID: <44AC5833.5010608@canterbury.ac.nz> Guido van Rossum wrote: > So, my proposal is to give up on static, accept PEP 3103 with the > following options: > - Syntax alternative 2+B (unindented cases, 'case in ...' for multiple cases). > - Semantics option 3 (def-time freezing) I agree with giving up on static, or at least treating it as a separate issue from switch. But there's one more thing I'd like to see discussed concerning def-time freezing: apple = 1 banana = 2 def f(): ... def g(fruit): ... switch fruit: case apple: ... case banana: ... With def-time freezing as currently proposed, this will evaluate the cases inside g() every time f() is called. But that would be wasteful, because they only depend on names defined at the module level. So I'd like to propose that "def time" be modified slightly to mean "def time of the outermost function that is still inside the scopes containing all the names that the case expressions depend on". Not sure about the wording of that, but I hope you get the idea. In the above example, that would be the def time of f(), rather than g(). I'd further suggest that a switch be allowed to reference names in the current scope, in which case no optimisation is done. That would take care of switches at the module level, without making the module level a special case with respect to the evaluation time rules. -- Greg From greg.ewing at canterbury.ac.nz Thu Jul 6 02:34:56 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Jul 2006 12:34:56 +1200 Subject: [Python-3000] Switch and static, redux In-Reply-To: <44ABC84E.5060408@doxdesk.com> References: <44ABC84E.5060408@doxdesk.com> Message-ID: <44AC5AB0.3060203@canterbury.ac.nz> Andrew Clover wrote: > Here's another syntax off the top of my head - put the first case in the > switch statement? > > switch biscuit.type if 'digestive': > ... > elif in 'jammy_dodger', 'garibaldi': > ... > else: > ... Ugly. -- Greg From greg.ewing at canterbury.ac.nz Thu Jul 6 02:39:50 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Jul 2006 12:39:50 +1200 Subject: [Python-3000] Switch and static, redux In-Reply-To: References: <44ABC84E.5060408@doxdesk.com> Message-ID: <44AC5BD6.4070605@canterbury.ac.nz> Georg Brandl wrote: > I'm not very comfortable about the no-additional-indentation-after-colon > syntax too, but I also realize that requiring two levels of indentation > for a single statement is too much. That's a matter of opinion. I don't have any trouble with it, but then I don't tend to set my indentation levels very high. On the other hand, I am bothered by *not* indenting the cases. They're all subordinate to the switch, so they should be indented, I feel. Maybe a compromise could be reached by having the style guide recommend a two-space indent for the cases, and a further two-space indent for the case bodies. Then the overall indentation of the bodies would still be only four spaces. -- Greg From mcherm at mcherm.com Thu Jul 6 17:47:03 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Thu, 06 Jul 2006 08:47:03 -0700 Subject: [Python-3000] Switch and static, redux Message-ID: <20060706084703.ufken3t3yi8sgo4s@login.werra.lunarpages.com> Greg Ewing writes: > Maybe a compromise could be reached by having the > style guide recommend a two-space indent for the > cases, and a further two-space indent for the > case bodies. Then the overall indentation of the > bodies would still be only four spaces. +1: I tried it and it looks good on my screen. -- Michael Chermside From ironfroggy at gmail.com Thu Jul 6 19:51:57 2006 From: ironfroggy at gmail.com (Calvin Spealman) Date: Thu, 6 Jul 2006 13:51:57 -0400 Subject: [Python-3000] Type Coersion Syntax Message-ID: <76fd5acf0607061051q26e32299gb4148f129317f195@mail.gmail.com> I would like to propose a syntax that would explicitly be read as "Create an object of Foo converted from object Bar". Being explicit, as such, could bring about some interesting optimizations in the future or immediately, and reduce redundant signatures (such as the type(foo) syntax). I understand this is likely to be given some negative comments, but I am interested in everyone's thoughts on the idea. # Coerce an object into its type type from foo # Create a list from some iterable list from bar Obviously you can see I propose the use of the 'from' keyword to mean "create this from that". This would not translate directly to type(foo) and list(bar), however, but would look for a new special method, named '__coerce__', and would pass the original object to this method to request a coerced instance of the desired type (or interface). This special method, which is explicitly for coercing from other types, can do more explicit things when we know exactly what we are coercing, rather than just creating a new instance. # Gives a mutable string interface to a file-like object string from some_file # Would create a new string, iterating over the file-like object string(some_file) # Where 'string' is some generic mutable string type. The same syntax could be used to implement other things, such as key-from-value operations: # Get the first index of a value in a list idx = 'b' from ['a', 'b', 'c'] assert idx == 'b' In this example, 'b' would not have a __coerce__ method of its own as an instance, and the list instance's __rcoerce__ would be called to acquire the index. From tomerfiliba at gmail.com Thu Jul 6 20:19:05 2006 From: tomerfiliba at gmail.com (tomer filiba) Date: Thu, 6 Jul 2006 20:19:05 +0200 Subject: [Python-3000] gettype Message-ID: <1d85506f0607061119w1c3cab60o6f762a8e3849e45c@mail.gmail.com> as you may remember, i suggest some time ago to distinguish between the *function* type that returns the type of an object, and *metaclass* type that creates new types. the main issue was type() behaving both as a function and as a factory, which is an overloaded behavior. if i recall correctly, the general spirit was pro-disambiguation, and the argument turned around the semantics of the change. i suggested calling the function "typeof", and keeping the metaclass in tact, but GvR didn't like the "%sof()" notation, as it's quite unprecidented in the language. so why not choose the "get%s()" notation? we already have getattr, and other __get%s__ special methods, so gettype() would perfectly fit into this convention. it also feels more natural imo: >>> type("blah", (), {}) >>> gettype(5) link to the old discussion (including the full details for the suggested change): http://mail.python.org/pipermail/python-3000/2006-May/002224.html any more comments? a pronouncement maybe? -tomer From krstic at fas.harvard.edu Thu Jul 6 20:26:46 2006 From: krstic at fas.harvard.edu (Ivan Krstic) Date: Thu, 06 Jul 2006 14:26:46 -0400 Subject: [Python-3000] Type Coersion Syntax In-Reply-To: <76fd5acf0607061051q26e32299gb4148f129317f195@mail.gmail.com> References: <76fd5acf0607061051q26e32299gb4148f129317f195@mail.gmail.com> Message-ID: <44AD55E6.1030206@fas.harvard.edu> Calvin Spealman wrote: > # Get the first index of a value in a list > idx = 'b' from ['a', 'b', 'c'] > assert idx == 'b' 'assert idx == 1'? -- Ivan Krstic | GPG: 0x147C722D From brett at python.org Thu Jul 6 22:29:31 2006 From: brett at python.org (Brett Cannon) Date: Thu, 6 Jul 2006 13:29:31 -0700 Subject: [Python-3000] Type Coersion Syntax In-Reply-To: <76fd5acf0607061051q26e32299gb4148f129317f195@mail.gmail.com> References: <76fd5acf0607061051q26e32299gb4148f129317f195@mail.gmail.com> Message-ID: On 7/6/06, Calvin Spealman wrote: > > I would like to propose a syntax that would explicitly be read as > "Create an object of Foo converted from object Bar". Being explicit, > as such, could bring about some interesting optimizations in the > future or immediately, and reduce redundant signatures (such as the > type(foo) syntax). I understand this is likely to be given some > negative comments, but I am interested in everyone's thoughts on the > idea. > > # Coerce an object into its type > type from foo > > # Create a list from some iterable > list from bar > > Obviously you can see I propose the use of the 'from' keyword to mean > "create this from that". This would not translate directly to > type(foo) and list(bar), however, but would look for a new special > method, named '__coerce__', and would pass the original object to this > method to request a coerced instance of the desired type (or > interface). This special method, which is explicitly for coercing from > other types, can do more explicit things when we know exactly what we > are coercing, rather than just creating a new instance. > > # Gives a mutable string interface to a file-like object > string from some_file > # Would create a new string, iterating over the file-like object > string(some_file) > # Where 'string' is some generic mutable string type. > > The same syntax could be used to implement other things, such as > key-from-value operations: > > # Get the first index of a value in a list > idx = 'b' from ['a', 'b', 'c'] > assert idx == 'b' > > In this example, 'b' would not have a __coerce__ method of its own as > an instance, and the list instance's __rcoerce__ would be called to > acquire the index. We just removed __coerce__ from Py3K. =) Anyway, I don't see how this will allow for any special optimization since it will still be calling a method on the object (constructor or otherwise) and that cannot be optimized away easily. Plus, what is wrong with passing on object to a type/class' constructor: ``str(42)``? Not redundant and it's still clean, obvious, and does not require new syntax. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20060706/58549271/attachment.html From guido at python.org Fri Jul 7 01:53:02 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 7 Jul 2006 01:53:02 +0200 Subject: [Python-3000] Switch and static, redux In-Reply-To: <44AC5833.5010608@canterbury.ac.nz> References: <44AC5833.5010608@canterbury.ac.nz> Message-ID: On 7/6/06, Greg Ewing wrote: > Guido van Rossum wrote: > > > So, my proposal is to give up on static, accept PEP 3103 with the > > following options: > > - Syntax alternative 2+B (unindented cases, 'case in ...' for multiple cases). > > - Semantics option 3 (def-time freezing) > > I agree with giving up on static, or at least treating > it as a separate issue from switch. OK, this much is a deal then. > But there's one more thing I'd like to see discussed > concerning def-time freezing: > > apple = 1 > banana = 2 > > def f(): > ... > def g(fruit): > ... > switch fruit: > case apple: > ... > case banana: > ... > > With def-time freezing as currently proposed, this will > evaluate the cases inside g() every time f() is called. > But that would be wasteful, because they only depend on > names defined at the module level. > > So I'd like to propose that "def time" be modified > slightly to mean "def time of the outermost function > that is still inside the scopes containing all the > names that the case expressions depend on". Not > sure about the wording of that, but I hope you get the > idea. In the above example, that would be the def time > of f(), rather than g(). I get the idea, and I don't like it. I thought of this case before and decided that it really isn't worth the complexification (and I'm using that word intentionally :-) of the rules. We don't have any rules like that ATM, and it's not the kind of rule that is even close to anything we do have (I claim that the def-time rule *is* close to something we already have, i.e. parameter default values -- close, not the same, mind you). Somehow a switch doesn't really strike me as a typical idiom for functional programming, so I doubt that the issue will matter in practice; in order for this to make a difference you'd have to have an inner function containing a switch *and* the inner function must only be called once (or a small number of times) for each time it is defined *and* the user must give a damn about optimization. The combination of the first and second suggests a functional programming mindset which somehow makes the third less likely. Your suggestion also makes it hard to find a good place to store the dispatch dict between the time the outer function is defined and the time the the inner function is defined. We can't store it in the outer function's local namespace (since it's not being called yet) so the only place to store it would be the outer function object. But that is not readily accessibly by the time the inner function object is being defined (the outer function's name is not a safe way to reference it). > I'd further suggest that a switch be allowed to reference > names in the current scope, in which case no optimisation > is done. That would take care of switches at the module > level, without making the module level a special case > with respect to the evaluation time rules. More complexification that I don't like. The only idiom I'm interesting in supporting is the one where the cases are constants (even though the compiler may not know it if they are named constants). Having a rule that prescribes optimization depending on the scopes of variables involved in a set of expressions sounds like a recipe for hard to debug bugs (since these rules are so subtle). I might as well repeat a quote I heard recently (and had heard before) from my good friend Brian Kernighan: "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." The changes to the rules that you are proposing make me think that we are making the rules too "clever", which will making it that much harder to debug issues around this. The much simpler rule that I propose is simpler to debug because you don't have to know the scope of the variables involved to know when the switch is frozen, which means that you can't be misled by a variable whose scope you are under- or over-estimating. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From chris at seberino.org Fri Jul 7 03:51:01 2006 From: chris at seberino.org (chris at seberino.org) Date: Thu, 6 Jul 2006 18:51:01 -0700 Subject: [Python-3000] Suggestion to replace `global' keyword with `modular' ... Message-ID: <20060707015101.GA1467@seberino.org> I would like to suggest replacing the 'global' keyword with a new keyword 'modular'. Why? In my limited understanding, 'global' means "in all namespaces" or "visible everywhere". Because globals are accessed like this: mymodule.myvariable they are just as "global" as this: mymodule.myclass.myclassattribute In other words, Python does not *really* have true globals (which is a good thing). It really has attributes visible at the module level. Hence, perhaps they should be called "modular". (If you look up "modular" at dictionary.com you will see the first definition is exactly what we are trying to say.. 'Of, relating to, or based on a module...") Chris From krstic at solarsail.hcs.harvard.edu Fri Jul 7 04:28:55 2006 From: krstic at solarsail.hcs.harvard.edu (Ivan Krstic) Date: Thu, 06 Jul 2006 22:28:55 -0400 Subject: [Python-3000] Suggestion to replace `global' keyword with `modular' ... In-Reply-To: <20060707015101.GA1467@seberino.org> References: <20060707015101.GA1467@seberino.org> Message-ID: <44ADC6E7.4080709@solarsail.hcs.harvard.edu> chris at seberino.org wrote: > I would like to suggest replacing the 'global' keyword > with a new keyword 'modular'. What you're calling 'true globals,' other languages (e.g. PHP) call superglobals. Python does have these, sort of: you can emulate a superglobal by sticking things into __builtins__, which isn't particularly clean, but gets the job done. I'm a complete -1 on replacing the name 'global' for no good reason, and a further -1 on the word 'modular,' which to me implies that the variable itself is composed of, or can be broken into, modules. -- Ivan Krstic | GPG: 0x147C722D From chris at seberino.org Fri Jul 7 05:45:04 2006 From: chris at seberino.org (chris at seberino.org) Date: Thu, 6 Jul 2006 20:45:04 -0700 Subject: [Python-3000] Suggestion to replace `global' keyword with `modular' ... In-Reply-To: <44ADC6E7.4080709@solarsail.hcs.harvard.edu> References: <20060707015101.GA1467@seberino.org> <44ADC6E7.4080709@solarsail.hcs.harvard.edu> Message-ID: <20060707034504.GA1589@seberino.org> > What you're calling 'true globals,' other languages (e.g. PHP) call > superglobals. Python does have these, sort of: you can emulate a > superglobal by sticking things into __builtins__, which isn't > particularly clean, but gets the job done. Thanks for reply. It appears from your comments above that you are agreeing with me that what Python currently calls "globals" are not really "true globals" the way many other languages & people think about it. > I'm a complete -1 on replacing the name 'global' for no good reason Didn't you just supply the 'good reason' in the sense that the current usage is ambiguous in some sense that could be made more precise with a different word? > a further -1 on the word 'modular,' which to me implies that the > variable itself is composed of, or can be broken into, modules. Well a Python can be broken into modules yes. And a 'modular' variable would be at the 'module level'. Still seems like 'modular' would avoid the ambiguity of 'global'. chris From thomas at python.org Fri Jul 7 09:40:02 2006 From: thomas at python.org (Thomas Wouters) Date: Fri, 7 Jul 2006 09:40:02 +0200 Subject: [Python-3000] Type Coersion Syntax In-Reply-To: References: <76fd5acf0607061051q26e32299gb4148f129317f195@mail.gmail.com> Message-ID: <9e804ac0607070040h7f0dbeb3oe460f54ba905b277@mail.gmail.com> On 7/6/06, Brett Cannon wrote: > > > Plus, what is wrong with passing on object to a type/class' constructor: > ``str(42)``? Not redundant and it's still clean, obvious, and does not > require new syntax. > And you can already have multiple 'constructor-from-data' methods that way, like dict already has: dict.fromkeys(). -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20060707/62585887/attachment.htm From talin at acm.org Fri Jul 7 09:47:22 2006 From: talin at acm.org (Talin) Date: Fri, 07 Jul 2006 00:47:22 -0700 Subject: [Python-3000] Switch and static, redux In-Reply-To: <44AC5AB0.3060203@canterbury.ac.nz> References: <44ABC84E.5060408@doxdesk.com> <44AC5AB0.3060203@canterbury.ac.nz> Message-ID: <44AE118A.4070102@acm.org> Greg Ewing wrote: > Andrew Clover wrote: > > >>Here's another syntax off the top of my head - put the first case in the >>switch statement? >> >> switch biscuit.type if 'digestive': >> ... >> elif in 'jammy_dodger', 'garibaldi': >> ... >> else: >> ... > > > Ugly. switch x: pass case 1: ... case 2: ... (OK you can shoot me now.) -- Talin From greg.ewing at canterbury.ac.nz Fri Jul 7 10:30:42 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 07 Jul 2006 20:30:42 +1200 Subject: [Python-3000] Switch and static, redux In-Reply-To: References: <44AC5833.5010608@canterbury.ac.nz> Message-ID: <44AE1BB2.1030609@canterbury.ac.nz> Guido van Rossum wrote: > The > combination of the first and second suggests a functional programming > mindset which somehow makes the third less likely. You seem to be conflating "nested functions" and "functional programming", which doesn't make sense to me. > Your suggestion also makes it hard to find a good place to store the > dispatch dict between the time the outer function is defined and the > time the the inner function is defined. Not at all -- you store it in a cell in the environment of f(). In other words, you treat it like _g_cases = {...} # build the case dict def f(): ... def g(): ... # switch using _g_cases > Having a rule that prescribes optimization depending on > the scopes of variables involved in a set of expressions sounds like a > recipe for hard to debug bugs (since these rules are so subtle). I don't mean that optimisation would be forbidden, just that it wouldn't be guaranteed if you use local names. I don't see that as being any worse than the situation now where, e.g. a loop written at the module level tends to be slower because it's using globals for its variables. I don't think there's anything particularly subtle about this, either. If the cases are based on the values of local variables, it seems pretty obvious that they're going to be recomputed every time you enter the function. > The changes to the rules that you are proposing make me think that we > are making the rules too "clever", which will making it that much > harder to debug issues around this. As you've said yourself, the use cases for all this are ones where the case values are effectively *constant*. Given that, it doesn't matter a damn how early or late they're evaluated, except insofar as it affects execution speed. What I'm proposing is a rule that says, "We don't guarantee to make it fast, but we'll try, and the more global your case values are, the faster we're likely to be able to make it". > The much simpler rule that I > propose is simpler to debug because you don't have to know the scope > of the variables involved to know when the switch is frozen, But you shouldn't *have* to know exactly when it's frozen if your case values are constant, as they should be. -- Greg From greg.ewing at canterbury.ac.nz Fri Jul 7 10:36:43 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 07 Jul 2006 20:36:43 +1200 Subject: [Python-3000] Suggestion to replace `global' keyword with `modular' ... In-Reply-To: <20060707015101.GA1467@seberino.org> References: <20060707015101.GA1467@seberino.org> Message-ID: <44AE1D1B.8040103@canterbury.ac.nz> chris at seberino.org wrote: > (If you look up "modular" at dictionary.com you will > see the first definition is exactly what we are trying to say.. > 'Of, relating to, or based on a module...") I'd say that dictionary is wrong. I've only ever seen "modular" used to mean "organised in the form of modules". -- Greg From talin at acm.org Fri Jul 7 10:47:19 2006 From: talin at acm.org (Talin) Date: Fri, 07 Jul 2006 01:47:19 -0700 Subject: [Python-3000] Switch and static, redux In-Reply-To: <44AE1BB2.1030609@canterbury.ac.nz> References: <44AC5833.5010608@canterbury.ac.nz> <44AE1BB2.1030609@canterbury.ac.nz> Message-ID: <44AE1F97.9000803@acm.org> Greg Ewing wrote: > Guido van Rossum wrote: > >>The >>combination of the first and second suggests a functional programming >>mindset which somehow makes the third less likely. > > You seem to be conflating "nested functions" and > "functional programming", which doesn't make sense > to me. > >>Your suggestion also makes it hard to find a good place to store the >>dispatch dict between the time the outer function is defined and the >>time the the inner function is defined. > > > Not at all -- you store it in a cell in the environment > of f(). In other words, you treat it like > > _g_cases = {...} # build the case dict > > def f(): > ... > def g(): > ... > # switch using _g_cases > > >>Having a rule that prescribes optimization depending on >>the scopes of variables involved in a set of expressions sounds like a >>recipe for hard to debug bugs (since these rules are so subtle). > > > I don't mean that optimisation would be forbidden, just > that it wouldn't be guaranteed if you use local names. > I don't see that as being any worse than the situation > now where, e.g. a loop written at the module level tends > to be slower because it's using globals for its variables. > > I don't think there's anything particularly subtle about > this, either. If the cases are based on the values of > local variables, it seems pretty obvious that they're > going to be recomputed every time you enter the function. > > >>The changes to the rules that you are proposing make me think that we >>are making the rules too "clever", which will making it that much >>harder to debug issues around this. > > > As you've said yourself, the use cases for all this are > ones where the case values are effectively *constant*. > Given that, it doesn't matter a damn how early or late > they're evaluated, except insofar as it affects execution > speed. What I'm proposing is a rule that says, "We don't > guarantee to make it fast, but we'll try, and the more global > your case values are, the faster we're likely to be able to > make it". > > >>The much simpler rule that I >>propose is simpler to debug because you don't have to know the scope >>of the variables involved to know when the switch is frozen, > > > But you shouldn't *have* to know exactly when it's frozen > if your case values are constant, as they should be. I'm in agreement with Greg here - his version sounds easier to understand, with less hidden gotchas. I would rather have "it always works with the same semantics, but some cases are slower than others", than "it only works in some cases". (And to turn Guido's argument on its head, if you 'give a damn' about optimization, then you probably care enough to track down the underlying causes. But for everyone else, it should 'just work', and no detective work required as to deciding what kinds of case values are legal or not.) And I agree that inner functions != functional programming. As Ping pointed out, closures pretty much required in Javascript programming (especially if you are doing AJAXy stuff), but I doubt most people would consider that "functional". (Strictly speaking, one of the defining characteristics of functional programming is that there be no side effects - which is exactly the opposite of the way most Python programmers use inner functions.) -- Talin From python at zesty.ca Fri Jul 7 10:04:53 2006 From: python at zesty.ca (Ka-Ping Yee) Date: Fri, 7 Jul 2006 03:04:53 -0500 (CDT) Subject: [Python-3000] Switch and static, redux In-Reply-To: References: Message-ID: On Wed, 5 Jul 2006, Guido van Rossum wrote: > So, my proposal is to give up on static, accept PEP 3103 with the > following options: > - Syntax alternative 2+B (unindented cases, 'case in ...' for > multiple cases). > - Semantics option 3 (def-time freezing) > > Do we need any more discussion about the PEP before I pronounce? (I'm > not super confident about the syntax alternatives yet.) I think i'd argue for indented cases, mainly because it's what everyone expects. "Everyone" includes - people who have seen 'switch' in other languages - people who expect ':' to introduce an indented block - editors that autoindent new lines - editors that fold indented blocks - editor hotkeys/macros for manipulating indented blocks - tools that highlight Python code - interactive Python consoles Unindented cases are not exactly the Spanish inquisition :) but in a room full of programmers who have ever used C, C++, C#, Java, or JavaScript, it's pretty close. -- ?!ng From g.brandl at gmx.net Fri Jul 7 10:55:18 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 07 Jul 2006 10:55:18 +0200 Subject: [Python-3000] Switch and static, redux In-Reply-To: References: Message-ID: Ka-Ping Yee wrote: > On Wed, 5 Jul 2006, Guido van Rossum wrote: >> So, my proposal is to give up on static, accept PEP 3103 with the >> following options: >> - Syntax alternative 2+B (unindented cases, 'case in ...' for >> multiple cases). >> - Semantics option 3 (def-time freezing) >> >> Do we need any more discussion about the PEP before I pronounce? (I'm >> not super confident about the syntax alternatives yet.) > > I think i'd argue for indented cases, mainly because it's what > everyone expects. "Everyone" includes > > - people who have seen 'switch' in other languages > - people who expect ':' to introduce an indented block > - editors that autoindent new lines > - editors that fold indented blocks > - editor hotkeys/macros for manipulating indented blocks > - tools that highlight Python code > - interactive Python consoles I agree with Ping here. Indented cases are more consistent, unless we come up with something to fill the "empty switch suite", such as switch expr: case 1: ... vs switch: expr case 1: ... Georg From guido at python.org Fri Jul 7 22:09:51 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 7 Jul 2006 22:09:51 +0200 Subject: [Python-3000] gettype In-Reply-To: <1d85506f0607061119w1c3cab60o6f762a8e3849e45c@mail.gmail.com> References: <1d85506f0607061119w1c3cab60o6f762a8e3849e45c@mail.gmail.com> Message-ID: Hm... I'd rather not break my head over this right now. I've got a feeling that we have bigger fish to fry, and I'd like to focus on important things first. I'm also not convinced this is all that broken. --Guido On 7/6/06, tomer filiba wrote: > as you may remember, i suggest some time ago to distinguish > between the *function* type that returns the type of an object, > and *metaclass* type that creates new types. the main issue > was type() behaving both as a function and as a factory, which > is an overloaded behavior. > > if i recall correctly, the general spirit was pro-disambiguation, and > the argument turned around the semantics of the change. > i suggested calling the function "typeof", and keeping the > metaclass in tact, but GvR didn't like the "%sof()" notation, as > it's quite unprecidented in the language. > > so why not choose the "get%s()" notation? we already have > getattr, and other __get%s__ special methods, so gettype() > would perfectly fit into this convention. it also feels more > natural imo: > > >>> type("blah", (), {}) > > > >>> gettype(5) > > > link to the old discussion (including the full details for the > suggested change): > http://mail.python.org/pipermail/python-3000/2006-May/002224.html > > any more comments? a pronouncement maybe? > > > -tomer > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jul 7 22:17:49 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 7 Jul 2006 22:17:49 +0200 Subject: [Python-3000] Switch and static, redux In-Reply-To: References: Message-ID: On 7/7/06, Georg Brandl wrote: > Ka-Ping Yee wrote: > > On Wed, 5 Jul 2006, Guido van Rossum wrote: > >> So, my proposal is to give up on static, accept PEP 3103 with the > >> following options: > >> - Syntax alternative 2+B (unindented cases, 'case in ...' for > >> multiple cases). > >> - Semantics option 3 (def-time freezing) > >> > >> Do we need any more discussion about the PEP before I pronounce? (I'm > >> not super confident about the syntax alternatives yet.) > > > > I think i'd argue for indented cases, mainly because it's what > > everyone expects. "Everyone" includes > > > > - people who have seen 'switch' in other languages > > - people who expect ':' to introduce an indented block > > - editors that autoindent new lines > > - editors that fold indented blocks > > - editor hotkeys/macros for manipulating indented blocks > > - tools that highlight Python code > > - interactive Python consoles > > I agree with Ping here. Indented cases are more consistent, I think I've been convinced. I'll update the PEP. > unless > we come up with something to fill the "empty switch suite", such as > > switch expr: > case 1: > ... > > vs > > switch: > expr > case 1: > ... I don't think anything like this is likely to displace syntax alternatives 1-4 in PEP 3103l, so I recommend that people not bother to try. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tomerfiliba at gmail.com Fri Jul 7 22:19:52 2006 From: tomerfiliba at gmail.com (tomer filiba) Date: Fri, 7 Jul 2006 22:19:52 +0200 Subject: [Python-3000] set literals Message-ID: <1d85506f0607071319u6dc1d954s54f4becb6c83b4cc@mail.gmail.com> i had this idea -- since quoted literals can be prefixed by a letter that alters their meaning, why not use the same semantics with curly braces? "hello" -- byte string literal r"hello" -- unescaped byte string literal u"hello" -- unicode string literal {1:2, 3:4, 5:6} -- dict literal s{1, 2, 3} -- set literal {} -- empty dict literal s{} - empty set literal this is more explicit, as {1:2} as {1,2} are too similar imo, and also solves the no-literal-for-empty-set issue that has been discussed at length. this should only be a slight change to the parser, and is a syntax error today, so there shouldn't be any backward-compat issues. it may look strange at first (at least it me), but it's just a matter of getting used to -- it's not any stranger than u"hello" -tomer From diogoko at gmail.com Fri Jul 7 22:45:15 2006 From: diogoko at gmail.com (Diogo Kollross) Date: Fri, 7 Jul 2006 17:45:15 -0300 Subject: [Python-3000] set literals In-Reply-To: <1d85506f0607071319u6dc1d954s54f4becb6c83b4cc@mail.gmail.com> References: <1d85506f0607071319u6dc1d954s54f4becb6c83b4cc@mail.gmail.com> Message-ID: > i had this idea -- since quoted literals can be prefixed by a letter > that alters their meaning, why not use the same semantics > with curly braces? String literals are always strings, no matter what kind of string as defined by the prefix. On the other hand, dictionaries and sets seems different enough for me to not see the second as a special case of the first. Sets literals are not top on my priority list, anyways... =] -- diogoko From tomerfiliba at gmail.com Fri Jul 7 23:34:49 2006 From: tomerfiliba at gmail.com (tomer filiba) Date: Fri, 7 Jul 2006 23:34:49 +0200 Subject: [Python-3000] set literals Message-ID: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> str and unicode are *distinct* types. they may be related, but not in the sense that type("hello") and type(r"hello") both return str. moreover, you can say a set is a "kind of" a keys-only dict. in fact, the first implementation of set used a dict, where the keys where the elements of the set, and their value was always True. -tomer --------------- > From: Diogo Kollross gmail.com> > Subject: Re: set literals > Newsgroups: gmane.comp.python.python-3000.devel > Date: 2006-07-07 20:45:15 GMT (41 minutes ago) > > i had this idea -- since quoted literals can be prefixed by a letter >> that alters their meaning, why not use the same semantics >> with curly braces? > > String literals are always strings, no matter what kind of string as > defined by the prefix. On the other hand, dictionaries and sets seems > different enough for me to not see the second as a special case of the > first. > > Sets literals are not top on my priority list, anyways... =] > > -- > diogoko From guido at python.org Sat Jul 8 05:03:18 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 8 Jul 2006 05:03:18 +0200 Subject: [Python-3000] set literals In-Reply-To: <1d85506f0607071319u6dc1d954s54f4becb6c83b4cc@mail.gmail.com> References: <1d85506f0607071319u6dc1d954s54f4becb6c83b4cc@mail.gmail.com> Message-ID: -1. On 7/7/06, tomer filiba wrote: > i had this idea -- since quoted literals can be prefixed by a letter > that alters their meaning, why not use the same semantics > with curly braces? > > "hello" -- byte string literal > r"hello" -- unescaped byte string literal > u"hello" -- unicode string literal > > {1:2, 3:4, 5:6} -- dict literal > s{1, 2, 3} -- set literal > > {} -- empty dict literal > s{} - empty set literal > > this is more explicit, as {1:2} as {1,2} are too similar imo, and also > solves the no-literal-for-empty-set issue that has been discussed > at length. > > this should only be a slight change to the parser, and is a syntax > error today, so there shouldn't be any backward-compat issues. > > it may look strange at first (at least it me), but it's just a matter of > getting used to -- it's not any stranger than u"hello" > > > -tomer > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Jul 8 05:34:19 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 8 Jul 2006 05:34:19 +0200 Subject: [Python-3000] Switch and static, redux In-Reply-To: <44AE1BB2.1030609@canterbury.ac.nz> References: <44AC5833.5010608@canterbury.ac.nz> <44AE1BB2.1030609@canterbury.ac.nz> Message-ID: On 7/7/06, Greg Ewing wrote: > Guido van Rossum wrote: > > > The > > combination of the first and second suggests a functional programming > > mindset which somehow makes the third less likely. > > You seem to be conflating "nested functions" and > "functional programming", which doesn't make sense > to me. Not even a smidgen? Small nested functions (or lambdas) are pretty common in a functional style. UI callbacks are more typically implemented as methods in my experience. We just talked in another thread about I/O callbacks a la JavaScript but I hope we have better ways to do that too. Nested functions used strictly as a code structuring device are not so great IMO; they tend to make the outer function really large, and the possibility of accidentally sharing variables in the outer scope worries me. Code that *returns* a function is often also indicative of a functional style (unless it's a decorator :-). > > Your suggestion also makes it hard to find a good place to store the > > dispatch dict between the time the outer function is defined and the > > time the the inner function is defined. > > Not at all -- you store it in a cell in the environment > of f(). In other words, you treat it like > > _g_cases = {...} # build the case dict > > def f(): > ... > def g(): > ... > # switch using _g_cases (I knew that argument was going to be considered a puzzle to solve. :-) I don't like storing it as a global; and that's probably incorrect too, e.g. if multiple definitions of f are being made in a loop. The only safe place I can think of is on the function object created by 'def f' but this still begs the question of where the definition of g finds it. > > Having a rule that prescribes optimization depending on > > the scopes of variables involved in a set of expressions sounds like a > > recipe for hard to debug bugs (since these rules are so subtle). > > I don't mean that optimisation would be forbidden, just > that it wouldn't be guaranteed if you use local names. The kind of optimization that's within reach for the CPython compiler is pretty much limited to compile-time constant expressions, and I don't expect that to change (it would require way too much static analysis of multiple modules). So in practice your rule would imply that. And that's what's causing the debugging problem for me. Questions like "why did my code suddenly change from O(log N) to O(N) performance" are really hard to answer if it's the compiler (not) taking a shortcut in one case. > I don't see that as being any worse than the situation > now where, e.g. a loop written at the module level tends > to be slower because it's using globals for its variables. No, that's different, because there is a very obvious syntactic clue: the presence of a 'def' around the for loop. In the case considered here, we can have two nearly identical cases involving an outer and an inner def, the latter containing a switch; yet one is fast and the other is slow, and the clue is that the slow one uses a local variable of the outer def in the switch. > I don't think there's anything particularly subtle about > this, either. If the cases are based on the values of > local variables, it seems pretty obvious that they're > going to be recomputed every time you enter the function. Understood. Once you see that you've solved the debugging problem. The subtlety that I fear is that, when reading the switch code casually (as you tend to do when first debugging something :-), you'll miss the fact that one of the cases references an outer local variable. And remember, the hypothetical case we're debugging is "why did this suddenly become slower" -- the switch isn't yet the focus of the investigation. > > The changes to the rules that you are proposing make me think that we > > are making the rules too "clever", which will making it that much > > harder to debug issues around this. > > As you've said yourself, the use cases for all this are > ones where the case values are effectively *constant*. > Given that, it doesn't matter a damn how early or late > they're evaluated, except insofar as it affects execution > speed. What I'm proposing is a rule that says, "We don't > guarantee to make it fast, but we'll try, and the more global > your case values are, the faster we're likely to be able to > make it". All I'm arguing is that that's a much more subtle rule to have to remember when debugging the speed thing than "switch cases are frozen when the closest outer def is executed". > > The much simpler rule that I > > propose is simpler to debug because you don't have to know the scope > > of the variables involved to know when the switch is frozen, > > But you shouldn't *have* to know exactly when it's frozen > if your case values are constant, as they should be. Of course. But I'm talking about debugging the case where you *accidentally* violated that rule. I guess the problem (with this continued argument) is that I'm constantly switching between two points of view, and need both to decide on an optimal solution. One POV is that of the casual user of the switch statement. They should stick to the "use only things that are (conceptually) constants". The other POV is that of the expert user trying to understand (accidentally) obfuscated code. I still like there to be simple rules to rememver for the latter. The difference is that the rules don't have to be obvious unless you're somewhat familiar with the implementation (in this case, storing the dispatch table on the nearest function object). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Jul 8 05:49:48 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 8 Jul 2006 05:49:48 +0200 Subject: [Python-3000] Switch and static, redux In-Reply-To: <44AE1F97.9000803@acm.org> References: <44AC5833.5010608@canterbury.ac.nz> <44AE1BB2.1030609@canterbury.ac.nz> <44AE1F97.9000803@acm.org> Message-ID: On 7/7/06, Talin wrote: > I'm in agreement with Greg here - his version sounds easier to > understand, with less hidden gotchas. I would rather have "it always > works with the same semantics, but some cases are slower than others", > than "it only works in some cases". I think if my proposal as the former too; except for switches using locals, which I don't care about. I find the "unoptimization" if a local is involved rule very subtle -- it could even be used to force reevaluation of all cases (say, when there's some non-constant global involve) by inserting a case that's a local variable with an impossible value. Since sometimes things don't work as expected, and then you have to debug them using the full set of rules, I find simple a "implementation" (even if it's also formal semantics) important. > (And to turn Guido's argument on its > head, if you 'give a damn' about optimization, then you probably care > enough to track down the underlying causes. But for everyone else, it > should 'just work', and no detective work required as to deciding what > kinds of case values are legal or not.) What's so hard to remember about the rule "cases should be constant"? > And I agree that inner functions != functional programming. As Ping > pointed out, closures pretty much required in Javascript programming > (especially if you are doing AJAXy stuff), but I doubt most people would > consider that "functional". Nor Pythonic. > (Strictly speaking, one of the defining characteristics of functional > programming is that there be no side effects - which is exactly the > opposite of the way most Python programmers use inner functions.) I guess I used "functional" where I might have said "Scheme-ish" -- I meant excessive use of HOFs. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ark-mlist at att.net Sat Jul 8 15:38:12 2006 From: ark-mlist at att.net (Andrew Koenig) Date: Sat, 8 Jul 2006 09:38:12 -0400 Subject: [Python-3000] set literals In-Reply-To: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> Message-ID: <001b01c6a293$c255c150$6402a8c0@arkdesktop> > moreover, you can say a set is a "kind of" a keys-only dict. in fact, > the first implementation of set used a dict, where the keys where the > elements of the set, and their value was always True. Or you could adopt the approach used by SETL: A dict is equivalent to a set of 2-tuples. In other words, {1:2, 3:4} could be defined as being equivalent to {(1,2), (3,4)}, with the run-time system being responsible for maintaining the information needed for efficient associative access. Then there's no question about the type of {}, because there's really only one type. No, I'm not serious; I think it would be too big a change. But you have to admit it's a cool idea :-) From diogoko at gmail.com Sat Jul 8 16:28:47 2006 From: diogoko at gmail.com (Diogo Kollross) Date: Sat, 8 Jul 2006 11:28:47 -0300 Subject: [Python-3000] set literals In-Reply-To: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> Message-ID: > str and unicode are *distinct* types. they may be related, but not in > the sense that type("hello") and type(r"hello") both return str. They're distinct types, but not that distinct: Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> len(set(dir(set)).symmetric_difference(set(dir(dict)))) 38 >>> len(set(dir(frozenset)).symmetric_difference(set(dir(dict)))) 31 >>> len(set(dir(str)).symmetric_difference(set(dir(unicode)))) 9 -- diogoko From ironfroggy at gmail.com Sat Jul 8 17:12:26 2006 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 8 Jul 2006 11:12:26 -0400 Subject: [Python-3000] Type Coersion Syntax In-Reply-To: References: <76fd5acf0607061051q26e32299gb4148f129317f195@mail.gmail.com> Message-ID: <76fd5acf0607080812g48704477we048a9954490d7c@mail.gmail.com> Well one thing this fixes is removal of unneeded overloading, which is an admitted problem with builtin types doubling as factories. The special method doesn't have to be __coerce__, and as a matter of fact something better named would probably be more appropriate. On 7/6/06, Brett Cannon wrote: > > > > On 7/6/06, Calvin Spealman wrote: > > I would like to propose a syntax that would explicitly be read as > > "Create an object of Foo converted from object Bar". Being explicit, > > as such, could bring about some interesting optimizations in the > > future or immediately, and reduce redundant signatures (such as the > > type(foo) syntax). I understand this is likely to be given some > > negative comments, but I am interested in everyone's thoughts on the > > idea. > > > > # Coerce an object into its type > > type from foo > > > > # Create a list from some iterable > > list from bar > > > > Obviously you can see I propose the use of the 'from' keyword to mean > > "create this from that". This would not translate directly to > > type(foo) and list(bar), however, but would look for a new special > > method, named '__coerce__', and would pass the original object to this > > method to request a coerced instance of the desired type (or > > interface). This special method, which is explicitly for coercing from > > other types, can do more explicit things when we know exactly what we > > are coercing, rather than just creating a new instance. > > > > # Gives a mutable string interface to a file-like object > > string from some_file > > # Would create a new string, iterating over the file-like object > > string(some_file) > > # Where 'string' is some generic mutable string type. > > > > The same syntax could be used to implement other things, such as > > key-from-value operations: > > > > # Get the first index of a value in a list > > idx = 'b' from ['a', 'b', 'c'] > > assert idx == 'b' > > > > In this example, 'b' would not have a __coerce__ method of its own as > > an instance, and the list instance's __rcoerce__ would be called to > > acquire the index. > > > We just removed __coerce__ from Py3K. =) > > Anyway, I don't see how this will allow for any special optimization since > it will still be calling a method on the object (constructor or otherwise) > and that cannot be optimized away easily. Plus, what is wrong with passing > on object to a type/class' constructor: ``str(42)``? Not redundant and it's > still clean, obvious, and does not require new syntax. > > > -Brett From ironfroggy at gmail.com Sat Jul 8 17:18:32 2006 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 8 Jul 2006 11:18:32 -0400 Subject: [Python-3000] No Container Literals Message-ID: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> Just throwing this out there, but I would love to see a complete dropping of container literals from Python. That is why I proposed the coercion syntax (ex: list from something) because it would allow things like list(1, 2, 3) and we can already do dict(ten=10, eleven=11), so what is the real need for literals as they are? With some proper compiler optimization we can deduce if list, dict, and such are in fact bound to the builtins we know, and build literals from these expressions just the same, but I feel they seem much more readable, and allow better addition of more literal compilations (set literals are fixed then, for example). I know no one will like this, but I have to make the idea known anyway. From diogoko at gmail.com Sat Jul 8 18:08:30 2006 From: diogoko at gmail.com (Diogo Kollross) Date: Sat, 8 Jul 2006 13:08:30 -0300 Subject: [Python-3000] No Container Literals In-Reply-To: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> References: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> Message-ID: > ...things like list(1, 2, 3) and we can already do dict(ten=10, > eleven=11), so what is the real need for literals as they are?... I like list(1, 2, 3). From steven.bethard at gmail.com Sat Jul 8 18:51:38 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Sat, 8 Jul 2006 10:51:38 -0600 Subject: [Python-3000] No Container Literals In-Reply-To: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> References: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> Message-ID: On 7/8/06, Calvin Spealman wrote: > Just throwing this out there, but I would love to see a complete > dropping of container literals from Python. That is why I proposed the > coercion syntax (ex: list from something) because it would allow > things like list(1, 2, 3) I don't see how your proposal solves the problem of distinguishing ``["abc"]`` from ``list("abc")`` where the former is a list of a single three-character element and the later is a list of three one-character elements. And no, throwing away the () constructor is not going to work -- it's far too useful. Remember, Python 3000 is not a new language. It's Python 2.X with some of the warts removed. Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From g.brandl at gmx.net Sat Jul 8 19:28:56 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 08 Jul 2006 19:28:56 +0200 Subject: [Python-3000] No Container Literals In-Reply-To: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> References: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> Message-ID: Calvin Spealman wrote: > Just throwing this out there, but I would love to see a complete > dropping of container literals from Python. That is why I proposed the > coercion syntax (ex: list from something) because it would allow > things like list(1, 2, 3) and we can already do dict(ten=10, > eleven=11), so what is the real need for literals as they are? With > some proper compiler optimization we can deduce if list, dict, and > such are in fact bound to the builtins we know, and build literals > from these expressions just the same, but I feel they seem much more > readable, and allow better addition of more literal compilations (set > literals are fixed then, for example). I know no one will like this, > but I have to make the idea known anyway. -1. List and dict displays are a great feature of the language. Why throw them out just because of purism? How would you spell {1: 2} with your syntax? Georg From aahz at pythoncraft.com Sat Jul 8 20:08:58 2006 From: aahz at pythoncraft.com (Aahz) Date: Sat, 8 Jul 2006 11:08:58 -0700 Subject: [Python-3000] No Container Literals In-Reply-To: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> References: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> Message-ID: <20060708180858.GA23759@panix.com> On Sat, Jul 08, 2006, Calvin Spealman wrote: > > Just throwing this out there, but I would love to see a complete > dropping of container literals from Python. That is why I proposed the > coercion syntax (ex: list from something) because it would allow > things like list(1, 2, 3) and we can already do dict(ten=10, > eleven=11), so what is the real need for literals as they are? With > some proper compiler optimization we can deduce if list, dict, and > such are in fact bound to the builtins we know, and build literals > from these expressions just the same, but I feel they seem much more > readable, and allow better addition of more literal compilations (set > literals are fixed then, for example). I know no one will like this, > but I have to make the idea known anyway. Could someone add this to the rejected proposals PEP? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "I saw `cout' being shifted "Hello world" times to the left and stopped right there." --Steve Gonedes From ironfroggy at gmail.com Sat Jul 8 20:40:31 2006 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 8 Jul 2006 14:40:31 -0400 Subject: [Python-3000] No Container Literals In-Reply-To: References: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> Message-ID: <76fd5acf0607081140g12674487q9689350256831e4b@mail.gmail.com> On 7/8/06, Steven Bethard wrote: > On 7/8/06, Calvin Spealman wrote: > > Just throwing this out there, but I would love to see a complete > > dropping of container literals from Python. That is why I proposed the > > coercion syntax (ex: list from something) because it would allow > > things like list(1, 2, 3) > > I don't see how your proposal solves the problem of distinguishing > ``["abc"]`` from ``list("abc")`` where the former is a list of a > single three-character element and the later is a list of three > one-character elements. And no, throwing away the () > constructor is not going to work -- it's far too useful. With my proposals, list("abc") would the same as ["abc"] is today. Creating a list of the characters in a string would be done with 'list from "abc"', which would coerce a string into a list explicitly. Thus, we still have an easy, and in my opinion more readable, method of coercing any iterable (or any type) to any other type. > Remember, Python 3000 is not a new language. It's Python 2.X with > some of the warts removed. > > Steve > -- > I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a > tiny blip on the distant coast of sanity. > --- Bucky Katt, Get Fuzzy > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/ironfroggy%40gmail.com > From ironfroggy at gmail.com Sat Jul 8 20:42:05 2006 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 8 Jul 2006 14:42:05 -0400 Subject: [Python-3000] No Container Literals In-Reply-To: References: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> Message-ID: <76fd5acf0607081142w3c838e7al7d290595f1f9aa83@mail.gmail.com> On 7/8/06, Georg Brandl wrote: > Calvin Spealman wrote: > > Just throwing this out there, but I would love to see a complete > > dropping of container literals from Python. That is why I proposed the > > coercion syntax (ex: list from something) because it would allow > > things like list(1, 2, 3) and we can already do dict(ten=10, > > eleven=11), so what is the real need for literals as they are? With > > some proper compiler optimization we can deduce if list, dict, and > > such are in fact bound to the builtins we know, and build literals > > from these expressions just the same, but I feel they seem much more > > readable, and allow better addition of more literal compilations (set > > literals are fixed then, for example). I know no one will like this, > > but I have to make the idea known anyway. > > -1. List and dict displays are a great feature of the language. Why throw > them out just because of purism? > > How would you spell {1: 2} with your syntax? dict(1=2) could be allowed, with additional syntax rules. > Georg > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/ironfroggy%40gmail.com > From g.brandl at gmx.net Sat Jul 8 20:54:08 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 08 Jul 2006 20:54:08 +0200 Subject: [Python-3000] No Container Literals In-Reply-To: <76fd5acf0607081142w3c838e7al7d290595f1f9aa83@mail.gmail.com> References: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> <76fd5acf0607081142w3c838e7al7d290595f1f9aa83@mail.gmail.com> Message-ID: Calvin Spealman wrote: > On 7/8/06, Georg Brandl wrote: >> Calvin Spealman wrote: >> > Just throwing this out there, but I would love to see a complete >> > dropping of container literals from Python. That is why I proposed the >> > coercion syntax (ex: list from something) because it would allow >> > things like list(1, 2, 3) and we can already do dict(ten=10, >> > eleven=11), so what is the real need for literals as they are? With >> > some proper compiler optimization we can deduce if list, dict, and >> > such are in fact bound to the builtins we know, and build literals >> > from these expressions just the same, but I feel they seem much more >> > readable, and allow better addition of more literal compilations (set >> > literals are fixed then, for example). I know no one will like this, >> > but I have to make the idea known anyway. >> >> -1. List and dict displays are a great feature of the language. Why throw >> them out just because of purism? >> >> How would you spell {1: 2} with your syntax? > > dict(1=2) could be allowed, with additional syntax rules. So dict() would be special-cased. What's the difference to {}, then? Or do you propose to allow arbitrary objects as keyword argument *names* for *every function*? If so, how would one distinguish dict(a=1) as in {"a": 1} and dict(a=1) as in {a: 1} ? Georg From ironfroggy at gmail.com Sat Jul 8 20:59:39 2006 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 8 Jul 2006 14:59:39 -0400 Subject: [Python-3000] No Container Literals In-Reply-To: References: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> <76fd5acf0607081142w3c838e7al7d290595f1f9aa83@mail.gmail.com> Message-ID: <76fd5acf0607081159q569de2e4m5a5f2b220e0f32ee@mail.gmail.com> On 7/8/06, Georg Brandl wrote: > Calvin Spealman wrote: > > On 7/8/06, Georg Brandl wrote: > >> Calvin Spealman wrote: > >> > Just throwing this out there, but I would love to see a complete > >> > dropping of container literals from Python. That is why I proposed the > >> > coercion syntax (ex: list from something) because it would allow > >> > things like list(1, 2, 3) and we can already do dict(ten=10, > >> > eleven=11), so what is the real need for literals as they are? With > >> > some proper compiler optimization we can deduce if list, dict, and > >> > such are in fact bound to the builtins we know, and build literals > >> > from these expressions just the same, but I feel they seem much more > >> > readable, and allow better addition of more literal compilations (set > >> > literals are fixed then, for example). I know no one will like this, > >> > but I have to make the idea known anyway. > >> > >> -1. List and dict displays are a great feature of the language. Why throw > >> them out just because of purism? > >> > >> How would you spell {1: 2} with your syntax? > > > > dict(1=2) could be allowed, with additional syntax rules. > > So dict() would be special-cased. What's the difference to {}, then? > > Or do you propose to allow arbitrary objects as keyword argument *names* > for *every function*? > > If so, how would one distinguish > dict(a=1) as in {"a": 1} > and > dict(a=1) as in {a: 1} > ? > > Georg Not objects, names. 'foo(bar=baz)' can parse where bar is any valid identifier (alphanumerical) and would not be ambigous. Objects _could_ be allowed with some syntax like 'dict((a)=1)' or 'dict(*a=1)' where the baz(*foo=bar) syntax would parse *foo as the object named foo, rather than the name foo itself. Of course, that isn't needed because you can just do dict((foo, bar), (a, 1)) Maybe it isn't a perfect solution, but I think its worthy for consideration is all. From krstic at solarsail.hcs.harvard.edu Sat Jul 8 23:03:21 2006 From: krstic at solarsail.hcs.harvard.edu (Ivan Krstic) Date: Sat, 08 Jul 2006 17:03:21 -0400 Subject: [Python-3000] No Container Literals In-Reply-To: <76fd5acf0607081142w3c838e7al7d290595f1f9aa83@mail.gmail.com> References: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> <76fd5acf0607081142w3c838e7al7d290595f1f9aa83@mail.gmail.com> Message-ID: <44B01D99.7040305@solarsail.hcs.harvard.edu> Calvin Spealman wrote: > dict(1=2) That's really ugly. Equating two different integers hurts my (mathematician's) eyes. Remember that we're not making a new language, and even if we were, this kind of decision would be hard to justify. -- Ivan Krstic | GPG: 0x147C722D From fredrik.johansson at gmail.com Sun Jul 9 00:11:14 2006 From: fredrik.johansson at gmail.com (Fredrik Johansson) Date: Sun, 9 Jul 2006 00:11:14 +0200 Subject: [Python-3000] No Container Literals In-Reply-To: References: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> Message-ID: <3d0cebfb0607081511l41073019s944d070e051c39a2@mail.gmail.com> I'm -0 on the proposal; it might make the syntax more streamlined, but I don't see a practical benefit. The current distinction between different types of brackets works very well. I absolutely don't see how it'd be worth to throw away dict literals just to buy set literals (the natural spelling would be set(1,2,3) with this change, anyway). On 7/8/06, Georg Brandl wrote: > How would you spell {1: 2} with your syntax? I'd just create dicts from lists of tuples. Unfortunately dict((1,'a'), (2,'b'), (3,'c')) is an extremely clumsy syntax, but an alternative might be to change the role of ; to become a sequence separator with lower precedence than a comma. Then you could write dict(1,'a'; 2,'b'; 3,'c'). (And use the more Matlab-like syntax [1,2,3 ; 4,5,6 ; 7,8,9] to create a matrix.) I'm not advocating this change to this language, just providing a possible answer to the question. Fredrik From talin at acm.org Sun Jul 9 00:30:18 2006 From: talin at acm.org (Talin) Date: Sat, 08 Jul 2006 15:30:18 -0700 Subject: [Python-3000] Lexical Scoping and Javascript Message-ID: <44B031FA.4070204@acm.org> I would like to refer everyone to a very interesting URL: http://developer.mozilla.org/en/docs/New_in_JavaScript_1.7 As you can see, there are a lot of features that are being considered for inclusion in Javascript that have been pretty much copied verbatim from Python. However, what's even more interesting is the new features that *haven't* been taken from Python. Some of those proposals address many of the same issues that we've been discussing here recently, and I'm pretty impressed with some of the solutions they have come up with. I'd like to encourage those who are interested to read the Javascript document carefully (in particular, all of the sections describing 'let', and the 'dictionary assignment' feature) and think about how this line of thinking might affect the discussion here. Who knows, perhaps there might be some cross-borrowing :) -- Talin From ncoghlan at gmail.com Sun Jul 9 04:03:15 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 09 Jul 2006 12:03:15 +1000 Subject: [Python-3000] Lexical Scoping and Javascript In-Reply-To: <44B031FA.4070204@acm.org> References: <44B031FA.4070204@acm.org> Message-ID: <44B063E3.2070000@gmail.com> Talin wrote: > I'd like to encourage those who are interested to read the Javascript > document carefully (in particular, all of the sections describing 'let', > and the 'dictionary assignment' feature) and think about how this line > of thinking might affect the discussion here. > > Who knows, perhaps there might be some cross-borrowing :) Interesting, very interesting. The 'let' statement/expression does seem like a very reasonable approach to the early binding/late binding question, since the order of writing matches the order of execution. Try this for an adaptation to Python: let NAME=EXPR in EXPR2 # Early binding expression let NAME=EXPR in: # Early binding statement SUITE Instead of the single NAME=EXPR part, you could also use a sequence of assignments (similar to keyword arguments). The let statement would create a new local scope, similar to def. Unlike def, the new local scope is executed immediately, rather than being kept around for later execution. Just like a nested function definition, assignment statements in the body of a 'let' statement would *not* be able to write to the outer scope in the absence of some form of 'nonlocal' declaration. The expression form would be essentially equivalent to: (lamda NAME=EXPR: EXPR)() And the statement form would be equivalent to: def _hidden1(NAME=EXPR): SUITE _hidden1() del _hidden1 To make the following use early binding instead of late binding: def f(): funcs = [] for i in range(10): funcs.append(lamba: i) return funcs >>> [x() for x in f()] [9, 9, 9, 9, 9, 9, 9, 9, 9, 9] You could do either: def f(): funcs = [] for i in range(10): let i=i in funcs.append(lamba: i) return funcs Or: def f(): funcs = [] for i in range(10): let i=i in: def inner(): return i funcs.append(inner) # 'inner' is NOT defined anymore! return funcs Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sun Jul 9 04:22:13 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 09 Jul 2006 12:22:13 +1000 Subject: [Python-3000] No Container Literals In-Reply-To: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> References: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> Message-ID: <44B06855.8090200@gmail.com> Calvin Spealman wrote: > Just throwing this out there, but I would love to see a complete > dropping of container literals from Python. That is why I proposed the > coercion syntax (ex: list from something) because it would allow > things like list(1, 2, 3) and we can already do dict(ten=10, > eleven=11), so what is the real need for literals as they are? With > some proper compiler optimization we can deduce if list, dict, and > such are in fact bound to the builtins we know, and build literals > from these expressions just the same, but I feel they seem much more > readable, and allow better addition of more literal compilations (set > literals are fixed then, for example). I know no one will like this, > but I have to make the idea known anyway. To turn the idea around a bit, could that *syntax* be a way to get rid of the plethora of parentheses in set literals? That is, NAME from TUPLE Would be equivalent to: NAME((TUPLE)) To avoid the ambiguity problems suffered string % formatting, the item on the right would have to be a tuple (with a single value without a trailing comma being either implicitly converted to a singleton tuple or else raising a syntax error). So instead of 'set((1, 2, 3))' you could write 'set from 1, 2, 3'. This would be usable for any function that took an iterable as its sole required argument. Instead of writing: if any((a, b, c)): # Do something You could write: if any from a, b, c: # Do something I don't personally think there's sufficient payoff in readability to justify the duplication of functionality, but figured I'd put the idea out there anyway. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ironfroggy at gmail.com Sun Jul 9 04:26:00 2006 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 8 Jul 2006 22:26:00 -0400 Subject: [Python-3000] No Container Literals In-Reply-To: <44B06855.8090200@gmail.com> References: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> <44B06855.8090200@gmail.com> Message-ID: <76fd5acf0607081926n368b028bt820c7d8a5e16c99a@mail.gmail.com> On 7/8/06, Nick Coghlan wrote: > Calvin Spealman wrote: > > Just throwing this out there, but I would love to see a complete > > dropping of container literals from Python. That is why I proposed the > > coercion syntax (ex: list from something) because it would allow > > things like list(1, 2, 3) and we can already do dict(ten=10, > > eleven=11), so what is the real need for literals as they are? With > > some proper compiler optimization we can deduce if list, dict, and > > such are in fact bound to the builtins we know, and build literals > > from these expressions just the same, but I feel they seem much more > > readable, and allow better addition of more literal compilations (set > > literals are fixed then, for example). I know no one will like this, > > but I have to make the idea known anyway. > > To turn the idea around a bit, could that *syntax* be a way to get rid of the > plethora of parentheses in set literals? > > That is, > > NAME from TUPLE > > Would be equivalent to: > > NAME((TUPLE)) > > To avoid the ambiguity problems suffered string % formatting, the item on the > right would have to be a tuple (with a single value without a trailing comma > being either implicitly converted to a singleton tuple or else raising a > syntax error). > > So instead of 'set((1, 2, 3))' you could write 'set from 1, 2, 3'. > > This would be usable for any function that took an iterable as its sole > required argument. > > Instead of writing: > > if any((a, b, c)): > # Do something > > You could write: > > if any from a, b, c: > # Do something > > I don't personally think there's sufficient payoff in readability to justify > the duplication of functionality, but figured I'd put the idea out there anyway. +1 From python at zesty.ca Sun Jul 9 05:50:32 2006 From: python at zesty.ca (Ka-Ping Yee) Date: Sat, 8 Jul 2006 22:50:32 -0500 (CDT) Subject: [Python-3000] No Container Literals In-Reply-To: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> References: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> Message-ID: On Sat, 8 Jul 2006, Calvin Spealman wrote: > Just throwing this out there, but I would love to see a complete > dropping of container literals from Python. That is why I proposed the > coercion syntax (ex: list from something) because it would allow > things like list(1, 2, 3) and we can already do dict(ten=10, > eleven=11), so what is the real need for literals as they are? This to me makes about as much sense as forcing programmers to write str('h', 'e', 'l', 'l', 'o') instead of 'hello'. -- ?!ng From ironfroggy at gmail.com Sun Jul 9 05:55:35 2006 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 8 Jul 2006 23:55:35 -0400 Subject: [Python-3000] No Container Literals In-Reply-To: References: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> Message-ID: <76fd5acf0607082055h40fecc99m899939f4476a35c1@mail.gmail.com> On 7/8/06, Ka-Ping Yee wrote: > On Sat, 8 Jul 2006, Calvin Spealman wrote: > > Just throwing this out there, but I would love to see a complete > > dropping of container literals from Python. That is why I proposed the > > coercion syntax (ex: list from something) because it would allow > > things like list(1, 2, 3) and we can already do dict(ten=10, > > eleven=11), so what is the real need for literals as they are? > > This to me makes about as much sense as forcing programmers to write > str('h', 'e', 'l', 'l', 'o') instead of 'hello'. I don't see a correlation between those. I don't see strings as a container type, either. This also changes, drastically, how we would write every element within the container, which nothing I've proposed does. From guido at python.org Sun Jul 9 22:04:58 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 9 Jul 2006 22:04:58 +0200 Subject: [Python-3000] set literals In-Reply-To: <001b01c6a293$c255c150$6402a8c0@arkdesktop> References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> <001b01c6a293$c255c150$6402a8c0@arkdesktop> Message-ID: I've also sometimes thought of unifying dicts and sets by implementing set operations on the keys of dicts only. When the values aren't the same, we could make an arbitrary decision e.g. the left operand wins. You get quite far. E.g. a = {1: "a", 2: "b"} b = {1: "c", 3: "d"} # These already work: assert 1 in a assert 1 in b assert 3 not in a # etc. # These would be new (the equivalent augmented assignments would work too): assert a|b == {1: "a", 2: "b", 3: "c"} assert a&b == {1: "a"} assert a^b == {2: "b", 3: "d"} assert a-b == {2: "b"} assert b-a == {3: "d"} # We could use these equivalencies: assert {1} == {1: None} == set({1: "a"}) # I.e. canonical sets have all None values # And of course it would solve the empty set notation problem nicely: assert dict() == {} == set() Unfortunately we couldn't redefine <=, <, >=, > to mean various subset/superset tests without backwards incompatibilities (but does anyone care?), and == and != would of course take the values into account which would occasionally sting. Also, sets would grow some operations that don't make a lot of sense (e.g. __getitem__, get, setdefault) but that's minor once you know the same implementation is used. I still expect there's a fatal flaw in the scheme, but I can't think of it right now... --Guido On 7/8/06, Andrew Koenig wrote: > > moreover, you can say a set is a "kind of" a keys-only dict. in fact, > > the first implementation of set used a dict, where the keys where the > > elements of the set, and their value was always True. > > Or you could adopt the approach used by SETL: A dict is equivalent to a set > of 2-tuples. In other words, {1:2, 3:4} could be defined as being > equivalent to {(1,2), (3,4)}, with the run-time system being responsible for > maintaining the information needed for efficient associative access. Then > there's no question about the type of {}, because there's really only one > type. > > No, I'm not serious; I think it would be too big a change. But you have to > admit it's a cool idea :-) > > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From krstic at solarsail.hcs.harvard.edu Sun Jul 9 22:15:04 2006 From: krstic at solarsail.hcs.harvard.edu (Ivan Krstic) Date: Sun, 09 Jul 2006 16:15:04 -0400 Subject: [Python-3000] Lexical Scoping and Javascript In-Reply-To: <44B031FA.4070204@acm.org> References: <44B031FA.4070204@acm.org> Message-ID: <44B163C8.3010408@solarsail.hcs.harvard.edu> Talin wrote: > I would like to refer everyone to a very interesting URL: > > http://developer.mozilla.org/en/docs/New_in_JavaScript_1.7 Let me follow up with Brendan Eich's XTech 2006 presentation on 'JavaScript 2 And the Future of The Web': http://developer.mozilla.org/presentations/xtech2006/javascript/ The short version is that JS2 is actively being reshaped to look a lot more like Python. -- Ivan Krstic | GPG: 0x147C722D From guido at python.org Sun Jul 9 23:30:55 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 9 Jul 2006 23:30:55 +0200 Subject: [Python-3000] No Container Literals In-Reply-To: <20060708180858.GA23759@panix.com> References: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> <20060708180858.GA23759@panix.com> Message-ID: On 7/8/06, Aahz wrote: > On Sat, Jul 08, 2006, Calvin Spealman wrote: > > Just throwing this out there, but I would love to see a complete > > dropping of container literals from Python. That is why I proposed the > > coercion syntax (ex: list from something) because it would allow > > things like list(1, 2, 3) and we can already do dict(ten=10, > > eleven=11), so what is the real need for literals as they are? With > > some proper compiler optimization we can deduce if list, dict, and > > such are in fact bound to the builtins we know, and build literals > > from these expressions just the same, but I feel they seem much more > > readable, and allow better addition of more literal compilations (set > > literals are fixed then, for example). I know no one will like this, > > but I have to make the idea known anyway. > > Could someone add this to the rejected proposals PEP? Done. Has Calvin explained *why* he wants this yet? (Not that that will make me change my mind. I'm just curious. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Mon Jul 10 01:45:03 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 10 Jul 2006 11:45:03 +1200 Subject: [Python-3000] Switch and static, redux In-Reply-To: References: <44AC5833.5010608@canterbury.ac.nz> <44AE1BB2.1030609@canterbury.ac.nz> Message-ID: <44B194FF.20503@canterbury.ac.nz> Guido van Rossum wrote: > On 7/7/06, Greg Ewing wrote: > >> You seem to be conflating "nested functions" and >> "functional programming", which doesn't make sense >> to me. > > Not even a smidgen? Small nested functions (or lambdas) are pretty > common in a functional style. Yes, but the implication doesn't go the other way -- there are uses for nested functions outside of the functional paradigm. The defining characteristic of the functional style is absence of side effects, not presence of nested functions. And it's certainly not absence of case statements -- case-like pattern matching is used all the time in functional languages. > I don't like storing it as a global; and that's probably incorrect > too, e.g. if multiple definitions of f are being made in a loop. I'm not so sure it would be wrong -- the cases are meant to be constant, remember? So they shouldn't be changing between definitions of f()! But if you really want to allow for this, it's still easy -- store in a cell in the f_closure of the function object created by def f(). Functions nested within f find it the same way they find any other cell, i.e. it gets passed down to them through the scoping system. This is exactly the same mechanism you would use to attach it to g(), except that you're hauling it up more than one level. > The kind of optimization that's within reach for the CPython compiler > is pretty much limited to compile-time constant expressions, By "optimisation" I just mean avoiding the recomputation of case values as much as reasonably practical. And it seems quite reasonably practical to me to only compute the case values of g() once in the example I gave -- for reasons which I think are obvious to a human reader of the program. > Questions like "why did my code suddenly change from O(log N) to O(N) > performance" are really hard to answer if it's the compiler (not) > taking a shortcut in one case. The number of times the cases are evaluated would only make a constant-factor difference to the running time, not an O() difference. > we can have two nearly identical cases involving an outer and an > inner def, the latter containing a switch; yet one is fast and the > other is slow, and the clue is that the slow one uses a local variable > of the outer def in the switch. That's true, but we seem to differ on how much of a problem that will be in practice. If the case values really are constant (we do agree that they *should* be, don't we?) then this situation is almost never going to arise, because there's no reason you would want to use anything other than a module-level variable in a case. So I'm calling FUD on that one. With your scheme, the nested case would *always* be slow, with no straightforward means available to make it fast. With mine, it would *almost always* be fast, unless you're trying to do something screwy like use non-constant cases, in which case you're asking for trouble anyway. And if you *mistakenly* use something non-constant as a case, then that's a bug which is likely to bite you in other ways as well, so you've got a debugging problem anyway. > The other POV is that of the expert > user trying to understand (accidentally) obfuscated code. I still like > there to be simple rules to rememver for the latter. For an expert user, I don't think "outermost possible function object" is substantially harder to grasp than "immediately enclosing function object". -- Greg From greg.ewing at canterbury.ac.nz Mon Jul 10 02:01:04 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 10 Jul 2006 12:01:04 +1200 Subject: [Python-3000] set literals In-Reply-To: <001b01c6a293$c255c150$6402a8c0@arkdesktop> References: <001b01c6a293$c255c150$6402a8c0@arkdesktop> Message-ID: <44B198C0.6000500@canterbury.ac.nz> Andrew Koenig wrote: > Or you could adopt the approach used by SETL: A dict is equivalent to a set > of 2-tuples. > > No, I'm not serious; I think it would be too big a change. But you have to > admit it's a cool idea :-) One fairly disastrous consequence would be that the *values* would have to be hashable as well as the keys... -- Greg From greg.ewing at canterbury.ac.nz Mon Jul 10 02:03:53 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 10 Jul 2006 12:03:53 +1200 Subject: [Python-3000] No Container Literals In-Reply-To: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> References: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> Message-ID: <44B19969.8020904@canterbury.ac.nz> Calvin Spealman wrote: > Just throwing this out there, but I would love to see a complete > dropping of container literals from Python. ... it would allow > things like list(1, 2, 3) ... I feel they seem much more > readable, That's highly debatable. There's semiotic value in not having everything built out of function calls. -- Greg From greg.ewing at canterbury.ac.nz Mon Jul 10 02:13:12 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 10 Jul 2006 12:13:12 +1200 Subject: [Python-3000] No Container Literals In-Reply-To: <76fd5acf0607081142w3c838e7al7d290595f1f9aa83@mail.gmail.com> References: <76fd5acf0607080818rb2f82ddjdae55dafafdbd214@mail.gmail.com> <76fd5acf0607081142w3c838e7al7d290595f1f9aa83@mail.gmail.com> Message-ID: <44B19B98.2000109@canterbury.ac.nz> Calvin Spealman wrote: > dict(1=2) could be allowed, with additional syntax rules. No, it couldn't, unless you were willing to give up on dict(a=2) being equivalent to {'a':2} rather than {a:2}. -- Greg From greg.ewing at canterbury.ac.nz Mon Jul 10 02:19:45 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 10 Jul 2006 12:19:45 +1200 Subject: [Python-3000] Lexical Scoping and Javascript In-Reply-To: <44B031FA.4070204@acm.org> References: <44B031FA.4070204@acm.org> Message-ID: <44B19D21.8030506@canterbury.ac.nz> Talin wrote: > As you can see, there are a lot of features that are being considered > for inclusion in Javascript that have been pretty much copied verbatim > from Python. If Javascript and Python continue to converge like this, there mightn't need to be a browser-embeddable Python implementation -- the Javascript people will end up doing it for us! -- Greg From guido at python.org Mon Jul 10 05:27:02 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 9 Jul 2006 20:27:02 -0700 Subject: [Python-3000] Switch and static, redux In-Reply-To: <44B194FF.20503@canterbury.ac.nz> References: <44AC5833.5010608@canterbury.ac.nz> <44AE1BB2.1030609@canterbury.ac.nz> <44B194FF.20503@canterbury.ac.nz> Message-ID: On 7/9/06, Greg Ewing wrote: > For an expert user, I don't think "outermost possible > function object" is substantially harder to grasp than > "immediately enclosing function object". I'm not getting through here. I think it is harder to debug because now you have to take into account the scopes of all cases, which may mean inspecting a lot of code. I'm just going to make an executive decision here to end the discussion. IMO the number of times that your proposal produces better results than mine is so small that it's not worth the complexity. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Mon Jul 10 06:30:51 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 10 Jul 2006 00:30:51 -0400 Subject: [Python-3000] Lexical Scoping and Javascript References: <44B031FA.4070204@acm.org> <44B19D21.8030506@canterbury.ac.nz> Message-ID: "Greg Ewing" wrote in message news:44B19D21.8030506 at canterbury.ac.nz... > Talin wrote: > >> As you can see, there are a lot of features that are being considered >> for inclusion in Javascript that have been pretty much copied verbatim >> from Python. > > If Javascript and Python continue to converge like this, > there mightn't need to be a browser-embeddable Python > implementation -- the Javascript people will end up > doing it for us! If only we could persuade them to drop the braces... though maybe embedded in html/xml is one place they are useful. tjr From ncoghlan at gmail.com Mon Jul 10 12:19:59 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 10 Jul 2006 20:19:59 +1000 Subject: [Python-3000] set literals In-Reply-To: References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> <001b01c6a293$c255c150$6402a8c0@arkdesktop> Message-ID: <44B229CF.7040200@gmail.com> Guido van Rossum wrote: > I've also sometimes thought of unifying dicts and sets by implementing > set operations on the keys of dicts only. When the values aren't the > same, we could make an arbitrary decision e.g. the left operand wins. If the right operand always won then a |= b would be equivalent to a.update(b). > # And of course it would solve the empty set notation problem nicely: > assert dict() == {} == set() So dicts would always start life using the set implementation, and switch to the full dict implementation only when a non-None value was added? > Unfortunately we couldn't redefine <=, <, >=, > to mean various > subset/superset tests without backwards incompatibilities (but does > anyone care? You mean those are defined *now*? I'm trying to figure out what the heck they could even mean. . . > I still expect there's a fatal flaw in the scheme, but I can't think > of it right now... If you've got an idea of how to handle the promotion from the set implementation to the dict implementation when a non-None value is entered, then I can't see any fatal flaws, either. I'm sure Raymond will be able to come up with something, though :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From alexander.belopolsky at gmail.com Mon Jul 10 18:39:03 2006 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 10 Jul 2006 12:39:03 -0400 Subject: [Python-3000] set literals Message-ID: Guido van Rossum: > # We could use these equivalencies: > assert {1} == {1: None} == set({1: "a"}) # I.e. canonical sets have > all None values An alternative canonical representation of a set as a dict could be a dict d for which d[k] is k for k in d.keys(). Guido van Rossum: > Also, sets would grow some operations that don't make a lot of sense > (e.g. __getitem__, get, setdefault) but that's minor once you know the > same implementation is used. These operations will see some use if sets behave as identity dictionaries, for example, setdefault will become an interning operation. From guido at python.org Mon Jul 10 19:26:53 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 10 Jul 2006 10:26:53 -0700 Subject: [Python-3000] set literals In-Reply-To: <44B229CF.7040200@gmail.com> References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> <001b01c6a293$c255c150$6402a8c0@arkdesktop> <44B229CF.7040200@gmail.com> Message-ID: On 7/10/06, Nick Coghlan wrote: > Guido van Rossum wrote: > > I've also sometimes thought of unifying dicts and sets by implementing > > set operations on the keys of dicts only. When the values aren't the > > same, we could make an arbitrary decision e.g. the left operand wins. > > If the right operand always won then a |= b would be equivalent to a.update(b). But somehow it seems more reasonable to let the left operand win. Finally we'd have a way to spell both choices: a |= b and a.update(b). > > # And of course it would solve the empty set notation problem nicely: > > assert dict() == {} == set() > > So dicts would always start life using the set implementation, and switch to > the full dict implementation only when a non-None value was added? No, there would only be one implementation. > > Unfortunately we couldn't redefine <=, <, >=, > to mean various > > subset/superset tests without backwards incompatibilities (but does > > anyone care? > > You mean those are defined *now*? I'm trying to figure out what the heck they > could even mean. . . They're intended to provide stable though arbitrary ordering for dicts. Admittedly not very useful (and a pain to implement!). In py3k we won't require everybody to support <= etc.; while so far I've been saying this about different types, it makes sense that even within one type one might not want to define them. > > I still expect there's a fatal flaw in the scheme, but I can't think > > of it right now... > > If you've got an idea of how to handle the promotion from the set > implementation to the dict implementation when a non-None value is entered, > then I can't see any fatal flaws, either. No promotion is necessary. set() would simply be a different factory function than dict(). I guess set(x) would equal dict.fromkeys(x). (I have to admit that I like the set() spelling better. :-) But type(set()) is dict. > I'm sure Raymond will be able to come up with something, though :) Haven't heard from him yet. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Mon Jul 10 22:28:17 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 10 Jul 2006 16:28:17 -0400 Subject: [Python-3000] set literals References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com><001b01c6a293$c255c150$6402a8c0@arkdesktop><44B229CF.7040200@gmail.com> Message-ID: "Guido van Rossum" wrote in message news:ca471dc20607101026j4c3d7de4h1c587e13ae7510cd at mail.gmail.com... >> > Unfortunately we couldn't redefine <=, <, >=, > to mean various >> > subset/superset tests without backwards incompatibilities (but does >> > anyone care? >> >> You mean those are defined *now*? I'm trying to figure out what the heck >> they >> could even mean. . . > > They're intended to provide stable though arbitrary ordering for > dicts. Admittedly not very useful (and a pain to implement!). In py3k > we won't require everybody to support <= etc.; while so far I've been > saying this about different types, it makes sense that even within one > type one might not want to define them. If you don't let complex numbers be ordered, even lexicographically, then I see even less need to arbitrarily order dicts. If anyone really does use the current feature, and really needs it, an ordict with the old behavior for <, etc, could be added to collections. I like the idea of at least partially unifying the set and dict interfaces. Terry Jan Reedy From rhettinger at ewtllc.com Mon Jul 10 22:54:58 2006 From: rhettinger at ewtllc.com (Raymond Hettinger) Date: Mon, 10 Jul 2006 13:54:58 -0700 Subject: [Python-3000] set literals In-Reply-To: References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> <001b01c6a293$c255c150$6402a8c0@arkdesktop> Message-ID: <44B2BEA2.3010207@ewtllc.com> Guido van Rossum wrote: >I've also sometimes thought of unifying dicts and sets by implementing >set operations on the keys of dicts only. When the values aren't the >same, we could make an arbitrary decision e.g. the left operand wins. >You get quite far. E.g. > >a = {1: "a", 2: "b"} >b = {1: "c", 3: "d"} > ># These already work: >assert 1 in a >assert 1 in b >assert 3 not in a ># etc. > ># These would be new (the equivalent augmented assignments would work too): >assert a|b == {1: "a", 2: "b", 3: "c"} >assert a&b == {1: "a"} >assert a^b == {2: "b", 3: "d"} >assert a-b == {2: "b"} >assert b-a == {3: "d"} > ># We could use these equivalencies: >assert {1} == {1: None} == set({1: "a"}) # I.e. canonical sets have >all None values > ># And of course it would solve the empty set notation problem nicely: >assert dict() == {} == set() > >Unfortunately we couldn't redefine <=, <, >=, > to mean various >subset/superset tests without backwards incompatibilities (but does >anyone care?), and == and != would of course take the values into >account which would occasionally sting. Also, sets would grow some >operations that don't make a lot of sense (e.g. __getitem__, get, >setdefault) but that's minor once you know the same implementation is >used. > >I still expect there's a fatal flaw in the scheme, but I can't think >of it right now... > > > I don't think there is a fatal flaw in terms of implementation -- a little modification of sets.py would demonstrate that with certainly. There would be some implementation consequences in terms of speed and memory usage (we would lose the memory savings and some of the speed-ups gained in the Py2.5 implementation of sets). The storage size would grow 50%. All set building operations would incur the overhead of copying in values as well as keys. The biggest issues are on the conceptual side -- do we think about set operations and mapping operations in the same way or in different ways? Guido listed the basic conflicts 1) arbitrary decisions as to which values to keep, 2) unexpected stings from != and == taking values into account, 3) operations that don't make sense (__setitem__, setdefault, etc). The deeper problem is that decisions on which values to keep will inevitability break set invariants (see a long list of these in test.test_set.py): assert a|b == b|a, 'commutative property' assert (a-b) | (a&b) | (b-a) == a|b, 'whole is the sum of the parts' The notion of sets carries so much mathematical significance that the loss of expected invariants would be a recurring surprise. On the positive side, I've occasionally wanted some set style operations for dictionaries (i.e. give me all the entries in d1 that aren't in d2, etc.) The LUA programming language demonstrated that even unifying lists and dicts is possible and that people can get used to just about anything. The question boils down to what is best for the user in terms of learnability, clarity, reliability, and performance. Raymond From guido at python.org Tue Jul 11 01:21:45 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 10 Jul 2006 16:21:45 -0700 Subject: [Python-3000] set literals In-Reply-To: <44B2BEA2.3010207@ewtllc.com> References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> <001b01c6a293$c255c150$6402a8c0@arkdesktop> <44B2BEA2.3010207@ewtllc.com> Message-ID: On 7/10/06, Raymond Hettinger wrote: > Guido van Rossum wrote: > > >I've also sometimes thought of unifying dicts and sets by implementing > >set operations on the keys of dicts only. When the values aren't the > >same, we could make an arbitrary decision e.g. the left operand wins. > >You get quite far. E.g. > > > >a = {1: "a", 2: "b"} > >b = {1: "c", 3: "d"} > > > ># These already work: > >assert 1 in a > >assert 1 in b > >assert 3 not in a > ># etc. > > > ># These would be new (the equivalent augmented assignments would work too): > >assert a|b == {1: "a", 2: "b", 3: "c"} (That should be 3: "d" BTW.) > >assert a&b == {1: "a"} > >assert a^b == {2: "b", 3: "d"} > >assert a-b == {2: "b"} > >assert b-a == {3: "d"} > > > ># We could use these equivalencies: > >assert {1} == {1: None} == set({1: "a"}) # I.e. canonical sets have > >all None values > > > ># And of course it would solve the empty set notation problem nicely: > >assert dict() == {} == set() > > > >Unfortunately we couldn't redefine <=, <, >=, > to mean various > >subset/superset tests without backwards incompatibilities (but does > >anyone care?), and == and != would of course take the values into > >account which would occasionally sting. Also, sets would grow some > >operations that don't make a lot of sense (e.g. __getitem__, get, > >setdefault) but that's minor once you know the same implementation is > >used. > > > >I still expect there's a fatal flaw in the scheme, but I can't think > >of it right now... > > I don't think there is a fatal flaw in terms of implementation -- a > little modification of sets.py would demonstrate that with certainly. > > There would be some implementation consequences in terms of speed and > memory usage (we would lose the memory savings and some of the speed-ups > gained in the Py2.5 implementation of sets). The storage size would > grow 50%. All set building operations would incur the overhead of > copying in values as well as keys. All of those could probably be eliminated through sheer willpower. E.g. a flag "are any values non-None" would go a long way. Of course, the 50% growth is only in terms of memory for the dict itself; if we take the space for the keys into account the percentage would be much smaller. > The biggest issues are on the conceptual side -- do we think about set > operations and mapping operations in the same way or in different ways? > Guido listed the basic conflicts 1) arbitrary decisions as to which > values to keep, 2) unexpected stings from != and == taking values into > account, 3) operations that don't make sense (__setitem__, setdefault, > etc). > > The deeper problem is that decisions on which values to keep will > inevitability break set invariants (see a long list of these in > test.test_set.py): > > assert a|b == b|a, 'commutative property' > assert (a-b) | (a&b) | (b-a) == a|b, 'whole is the sum of the parts' Yeah, I think when I was cosidering this earlier (it's been a shower thought for quite a while on and off) I considered those fatal flaws. Today, I think the invariants would simply change to assert set(a|b) == set(b|a) == set(a)|set(b) == set(b)|set(a) with set() being an alias for dict.fromkeys(). (I wonder if we need a similar thing that doesn't make a copy if the input is already a set, i.e. has all-None values.) > The notion of sets carries so much mathematical significance that the > loss of expected invariants would be a recurring surprise. But only when mixing sets with non-sets. > On the positive side, I've occasionally wanted some set style operations > for dictionaries (i.e. give me all the entries in d1 that aren't in d2, > etc.) Right. Plus of course there's a long history of pre-2.3 code that (ab)uses dicts as sets. (We could get rid of the "ab". :-) > The LUA programming language demonstrated that even unifying lists and > dicts is possible and that people can get used to just about anything. And ABC has lists and tables that were quite different from Python's lists and dicts... > The question boils down to what is best for the user in terms of > learnability, clarity, reliability, and performance. Where the unified data type doesn't necessarily score poorly -- one less type means less to learn, less code, and fewer choices to make. I'm still very much undecided but I don't want to rule this out for py3k. Perhaps I'll write up a PEP and see how it flies. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Tue Jul 11 01:36:10 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 11 Jul 2006 11:36:10 +1200 Subject: [Python-3000] Lexical Scoping and Javascript In-Reply-To: References: <44B031FA.4070204@acm.org> <44B19D21.8030506@canterbury.ac.nz> Message-ID: <44B2E46A.5060901@canterbury.ac.nz> Terry Reedy wrote: > If only we could persuade them to drop the braces... > though maybe embedded in html/xml is one place they are useful. The braces wouldn't be a problem if the semantics were close enough to permit automatic Python-->Javascript conversion... -- Greg From diogoko at gmail.com Tue Jul 11 01:48:16 2006 From: diogoko at gmail.com (Diogo Kollross) Date: Mon, 10 Jul 2006 20:48:16 -0300 Subject: [Python-3000] set literals In-Reply-To: References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> <001b01c6a293$c255c150$6402a8c0@arkdesktop> <44B2BEA2.3010207@ewtllc.com> Message-ID: While we're at it... what happens to frozenset? From rhettinger at ewtllc.com Tue Jul 11 02:03:58 2006 From: rhettinger at ewtllc.com (Raymond Hettinger) Date: Mon, 10 Jul 2006 17:03:58 -0700 Subject: [Python-3000] set literals In-Reply-To: References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> <001b01c6a293$c255c150$6402a8c0@arkdesktop> <44B2BEA2.3010207@ewtllc.com> Message-ID: <44B2EAEE.8060807@ewtllc.com> Guido van Rossum wrote: >> >> The deeper problem is that decisions on which values to keep will >> inevitability break set invariants (see a long list of these in >> test.test_set.py): >> >> assert a|b == b|a, 'commutative property' >> assert (a-b) | (a&b) | (b-a) == a|b, 'whole is the sum of the parts' > > > Yeah, I think when I was cosidering this earlier (it's been a shower > thought for quite a while on and off) I considered those fatal flaws. > Today, I think the invariants would simply change to > > assert set(a|b) == set(b|a) == set(a)|set(b) == set(b)|set(a) > > with set() being an alias for dict.fromkeys(). The outermost set coercion is not especially attractive or efficient. Since equality/inequality is an important set operation, we would likely need to add a method for equality testing that ignores dict values: d1.compareKeys(d2) # equivalent to: set(d1)==set(d2) The frozenset type also needs to be addressed. They don't come-up much in practice but when they do, they are vital to the solution, so some kind of frozenset type should be kept around (perhaps in the collections module). However, I have long argued against having frozendicts and I want to kill THAT discussion before it rears its ugly head again. I'm curious as to whether people will find one-type-with-two-purposes easier to learn that what we have now. My experience so far is that sets have a near zero learning curve as they are currently implemented. Also, I wonder whether it is wise to place any further burden on either the mapping API or its implementation (do we want weakref dictionaries and dbm's supporting set ops?). Raymond From greg.ewing at canterbury.ac.nz Tue Jul 11 02:06:39 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 11 Jul 2006 12:06:39 +1200 Subject: [Python-3000] set literals In-Reply-To: <44B2BEA2.3010207@ewtllc.com> References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> <001b01c6a293$c255c150$6402a8c0@arkdesktop> <44B2BEA2.3010207@ewtllc.com> Message-ID: <44B2EB8F.60409@canterbury.ac.nz> Raymond Hettinger wrote: > There would be some implementation consequences in terms of speed and > memory usage (we would lose the memory savings and some of the speed-ups > gained in the Py2.5 implementation of sets). Is that really necessary? Couldn't the keys and values be kept in separate arrays, and only allocate the values array the first time a non-canonical value is inserted? > Guido listed the basic conflicts 1) arbitrary decisions as to which > values to keep, 2) unexpected stings from != and == taking values into > account, 3) operations that don't make sense (__setitem__, setdefault, > etc). > ... > assert a|b == b|a, 'commutative property' > assert (a-b) | (a&b) | (b-a) == a|b, 'whole is the sum of the parts' These could be resolved by refusing to carry out set operations on a dict which contains non-canonical values. On the other hand, this mightn't really be much of a problem. These invariants will still be true of set-dicts which are being used as sets, i.e. contain only canonical values. And as you say, extending them to non-set dicts could have uses. Another way of thinking about this is that there are really two conceptual types, sets and dicts, which just happen to be implemented by the same concrete type. I don't think that's any worse than e.g. using a list to implement a stack or queue. As long as you use it as per the conceptual type you're thinking of, all the relevant invariants hold. -- Greg From greg.ewing at canterbury.ac.nz Tue Jul 11 02:20:20 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 11 Jul 2006 12:20:20 +1200 Subject: [Python-3000] set literals In-Reply-To: References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> <001b01c6a293$c255c150$6402a8c0@arkdesktop> <44B2BEA2.3010207@ewtllc.com> Message-ID: <44B2EEC4.8070000@canterbury.ac.nz> Diogo Kollross wrote: > While we're at it... what happens to frozenset? Probably it becomes an immutable subsclass of dict. Then, since the word 'set' is now free, we can rename the somewhat awkward-sounding 'frozenset' to 'set'. There would then be a nice symmetry between list/tuple and dict/set: +-----------------------+ | Immutable Mutable | +-----------+-----------------------+ | Sequence | tuple list | | Mapping | set dict | +-----------+-----------------------+ -- Greg From greg.ewing at canterbury.ac.nz Tue Jul 11 02:23:39 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 11 Jul 2006 12:23:39 +1200 Subject: [Python-3000] set literals In-Reply-To: <44B2EAEE.8060807@ewtllc.com> References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> <001b01c6a293$c255c150$6402a8c0@arkdesktop> <44B2BEA2.3010207@ewtllc.com> <44B2EAEE.8060807@ewtllc.com> Message-ID: <44B2EF8B.8010508@canterbury.ac.nz> Raymond Hettinger wrote: > The outermost set coercion is not especially attractive or efficient. > Since equality/inequality is an important set operation, we would likely > need to add a method for equality testing that ignores dict values: Or coerction to a set could return a "set view" that only exposes keys. -- Greg From guido at python.org Tue Jul 11 02:37:41 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 10 Jul 2006 17:37:41 -0700 Subject: [Python-3000] set literals In-Reply-To: References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> <001b01c6a293$c255c150$6402a8c0@arkdesktop> <44B2BEA2.3010207@ewtllc.com> Message-ID: On 7/10/06, Diogo Kollross wrote: > While we're at it... what happens to frozenset? I say nuke it. Who needs it? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Tue Jul 11 02:45:15 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 11 Jul 2006 12:45:15 +1200 Subject: [Python-3000] set literals In-Reply-To: References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> <001b01c6a293$c255c150$6402a8c0@arkdesktop> <44B2BEA2.3010207@ewtllc.com> Message-ID: <44B2F49B.3060101@canterbury.ac.nz> Guido van Rossum wrote: > On 7/10/06, Diogo Kollross wrote: > >>While we're at it... what happens to frozenset? > > I say nuke it. Who needs it? There are algorithms which effectively involve a mapping keyed by a set. E.g. converting an NFA to a DFA requires building up sets of states from the NFA and associating each one with a state in the DFA. You can kludge around it, but it would be nice to be able to express this kind of thing directly using the builtin support for sets. -- Greg From diogoko at gmail.com Tue Jul 11 03:15:01 2006 From: diogoko at gmail.com (Diogo Kollross) Date: Mon, 10 Jul 2006 22:15:01 -0300 Subject: [Python-3000] set literals In-Reply-To: <44B2EAEE.8060807@ewtllc.com> References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> <001b01c6a293$c255c150$6402a8c0@arkdesktop> <44B2BEA2.3010207@ewtllc.com> <44B2EAEE.8060807@ewtllc.com> Message-ID: > I'm curious as to whether people will find one-type-with-two-purposes > easier to learn that what we have now. I see no problem with that. We already have lists-that-can-work-as-stacks-or-queues. From rhettinger at ewtllc.com Tue Jul 11 03:59:01 2006 From: rhettinger at ewtllc.com (Raymond Hettinger) Date: Mon, 10 Jul 2006 18:59:01 -0700 Subject: [Python-3000] set literals In-Reply-To: <44B2EB8F.60409@canterbury.ac.nz> References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> <001b01c6a293$c255c150$6402a8c0@arkdesktop> <44B2BEA2.3010207@ewtllc.com> <44B2EB8F.60409@canterbury.ac.nz> Message-ID: <44B305E5.7050705@ewtllc.com> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20060710/dfea5e9a/attachment.htm From guido at python.org Tue Jul 11 06:43:04 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 10 Jul 2006 21:43:04 -0700 Subject: [Python-3000] Detecting conflicts in dict displays Message-ID: Currently, this is valid: a = {'a': 1, 'a': 2} print a # {'a': 2} I wonder if we shouldn't make this a run-time error instead. If people agree, what should we do with a = {'a': 1, 'a': 1} ??? In ABC, that's legal (and the first one is indeed an error; I took a shortcut when I decided not to test for that in Python). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Tue Jul 11 07:11:40 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 10 Jul 2006 23:11:40 -0600 Subject: [Python-3000] Detecting conflicts in dict displays In-Reply-To: References: Message-ID: On 7/10/06, Guido van Rossum wrote: > Currently, this is valid: > > a = {'a': 1, 'a': 2} > print a # {'a': 2} > > I wonder if we shouldn't make this a run-time error instead. If it's possible, definitely. That's gotta be a typo if it appears in real code. > If people agree, what should we do with > > a = {'a': 1, 'a': 1} > > ??? It should also be an error. I can't imagine that not being a typo. And there isn't really any reason to write that since we're talking dict literals here. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From tjreedy at udel.edu Tue Jul 11 09:48:10 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 11 Jul 2006 03:48:10 -0400 Subject: [Python-3000] Detecting conflicts in dict displays References: Message-ID: "Steven Bethard" wrote in message news:d11dcfba0607102211x1be4033sdb42a175bf27e730 at mail.gmail.com... > On 7/10/06, Guido van Rossum wrote: >> Currently, this is valid: >> >> a = {'a': 1, 'a': 2} >> print a # {'a': 2} >> >> I wonder if we shouldn't make this a run-time error instead. > > If it's possible, definitely. That's gotta be a typo if it appears in > real code. I think it would overall be helpful to tell people "duplicate key 'a'" >> If people agree, what should we do with >> >> a = {'a': 1, 'a': 1} >> >> ??? > > It should also be an error. I can't imagine that not being a typo. > And there isn't really any reason to write that since we're talking > dict literals here. Something effectively the same could also be an act of ignorance: >>> d={1:'one', 1.0:'one'} >>> d {1: 'one'} I think this should be caught also. The exception message is a bit trickier here. Terry Jan Reedy From thomas at python.org Tue Jul 11 16:27:59 2006 From: thomas at python.org (Thomas Wouters) Date: Tue, 11 Jul 2006 16:27:59 +0200 Subject: [Python-3000] Detecting conflicts in dict displays In-Reply-To: References: Message-ID: <9e804ac0607110727r303b2781s51fef76c08ee6817@mail.gmail.com> On 7/11/06, Guido van Rossum wrote: > > Currently, this is valid: > > a = {'a': 1, 'a': 2} > print a # {'a': 2} > > I wonder if we shouldn't make this a run-time error instead. Maybe. Like that, sure. But what about: a = {a: 1, b: 2} where 'a' and 'b' happen to be equal (or equal enough for dicts)? I'd rather leave this kind of checking up to pycheker or pylint (which already check for quite a lot more than I'd want the Python runtime to check :) That doesn't mean I think pychecker/pylint shouldn't be incorporated in the core, maybe even run by default, but that's another discussion ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20060711/ad8605c2/attachment.htm From Ben.Young at risk.sungard.com Tue Jul 11 16:54:34 2006 From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com) Date: Tue, 11 Jul 2006 15:54:34 +0100 Subject: [Python-3000] [Python-Dev] Explicit Lexical Scoping (pre-PEP?) In-Reply-To: Message-ID: python-dev-bounces+python=theyoungfamily.co.uk at python.org wrote on 11/07/2006 15:46:44: > outbound x = 1 > x = 2 > > evaluating using Jeremy Hilton's' list: > > 1. is a real word > 2. For me - in python - it would mean: Is found in 'outer' scope and > is already bound. > And the literal meaning of 'outbound 'headed away' [1] is pretty > darn close to what I mean when I spell the usual mutables kluge. > > 3 statement is positive form > 4. I like it > > could not find a use of outbound in python source (2.4.3) > (Shifting to python-3000) On a similar note, how about: prebound x, or predefined x Both clearly state that the name has already been used in an outer scope (I think) Cheers, Ben > [1] http://dictionary.reference.com/search?q=outbound > > > Robin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python- > dev/python%40theyoungfamily.co.uk > From rhettinger at ewtllc.com Tue Jul 11 18:06:35 2006 From: rhettinger at ewtllc.com (Raymond Hettinger) Date: Tue, 11 Jul 2006 09:06:35 -0700 Subject: [Python-3000] Detecting conflicts in dict displays In-Reply-To: <9e804ac0607110727r303b2781s51fef76c08ee6817@mail.gmail.com> References: <9e804ac0607110727r303b2781s51fef76c08ee6817@mail.gmail.com> Message-ID: <44B3CC8B.7080703@ewtllc.com> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20060711/0808ef39/attachment.html From jimjjewett at gmail.com Wed Jul 12 21:32:36 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 12 Jul 2006 15:32:36 -0400 Subject: [Python-3000] Switch and static, redux In-Reply-To: References: <44AC5833.5010608@canterbury.ac.nz> <44AE1BB2.1030609@canterbury.ac.nz> Message-ID: On 7/7/06, Guido van Rossum wrote: > I don't like storing it as a global; and that's probably incorrect > too, e.g. if multiple definitions of f are being made in a loop. Should the case expressions be different for each loop? If not, then it doesn't matter, and a per-module storage is fine, and offers the best optimization. (It may not be the easiest initial implementation, but that is no reason to rule it out forever.) If the case expressions can differ on each iteration, then you're supporting non-constant expressions. How important is this use case? Important enough to slow down all nested switches and complicate the (detailed) semantics? > I guess the problem (with this continued argument) is that I'm > constantly switching between two points of view, and need both to > decide on an optimal solution. One POV is that of the casual user of > the switch statement. They should stick to the "use only things that > are (conceptually) constants". Aside from AttributeError/NameError, this case doesn't care when the freeze happens. > The other POV is that of the expert > user trying to understand (accidentally) obfuscated code. I still like > there to be simple rules to rememver for the latter. The difference is > that the rules don't have to be obvious unless you're somewhat > familiar with the implementation (in this case, storing the dispatch > table on the nearest function object). If you're going to require familiarity with the implementation anyhow, then you can use almost arbitrary rules. Why not store the switch in a module-level cellvar that gets set at most once? -jJ From bingham at cenix-bioscience.com Thu Jul 13 09:56:40 2006 From: bingham at cenix-bioscience.com (Aaron Bingham) Date: Thu, 13 Jul 2006 09:56:40 +0200 Subject: [Python-3000] Detecting conflicts in dict displays In-Reply-To: <44B3CC8B.7080703@ewtllc.com> References: <9e804ac0607110727r303b2781s51fef76c08ee6817@mail.gmail.com> <44B3CC8B.7080703@ewtllc.com> Message-ID: <44B5FCB8.7070702@cenix-bioscience.com> Raymond Hettinger wrote: > Thomas Wouters wrote: > >> >> >> On 7/11/06, *Guido van Rossum* > > wrote: >> >> Currently, this is valid: >> >> a = {'a': 1, 'a': 2} >> print a # {'a': 2} >> >> I wonder if we shouldn't make this a run-time error instead. >> >> >> Maybe. Like that, sure. But what about: >> >> a = {a: 1, b: 2} >> >> where 'a' and 'b' happen to be equal (or equal enough for dicts)? I'd >> rather leave this kind of checking up to pycheker or pylint (which >> already check for quite a lot more than I'd want the Python runtime >> to check :) > > +1 for leaving this to PyLint or PyChecker. Wouldn't the behavior of the above be undefined for the case where a == b? I'd much rather get an exception than undefined behavior. Cheers, -- -------------------------------------------------------------------- Aaron Bingham Senior Software Engineer Cenix BioScience GmbH -------------------------------------------------------------------- From fredrik at pythonware.com Thu Jul 13 12:29:17 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 13 Jul 2006 12:29:17 +0200 Subject: [Python-3000] set literals References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> <001b01c6a293$c255c150$6402a8c0@arkdesktop> <44B2BEA2.3010207@ewtllc.com> <44B2EAEE.8060807@ewtllc.com> Message-ID: Raymond Hettinger wrote: > I'm curious as to whether people will find one-type-with-two-purposes > easier to learn that what we have now. we haven't had that for long, and I cannot recall anyone being puzzled by the fact that you can use a dictionary of (item, None) pairs to efficiently represent a set. From fredrik at pythonware.com Thu Jul 13 12:31:09 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 13 Jul 2006 12:31:09 +0200 Subject: [Python-3000] set literals References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com><001b01c6a293$c255c150$6402a8c0@arkdesktop><44B2BEA2.3010207@ewtllc.com> <44B2F49B.3060101@canterbury.ac.nz> Message-ID: Greg Ewing wrote: >> I say nuke it. Who needs it? > > There are algorithms which effectively involve a > mapping keyed by a set. E.g. converting an NFA > to a DFA requires building up sets of states from > the NFA and associating each one with a state in > the DFA. > > You can kludge around it, but it would be nice > to be able to express this kind of thing directly > using the builtin support for sets. but wouldn't the typical kludge (use a serial based on len(dict) to keep track of things) be a lot more efficient than the non-kludge solution (use a pointer and a full recursive compare) ? is it even a kludge ? From fredrik at pythonware.com Thu Jul 13 12:44:43 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 13 Jul 2006 12:44:43 +0200 Subject: [Python-3000] set literals References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> <001b01c6a293$c255c150$6402a8c0@arkdesktop> <44B2BEA2.3010207@ewtllc.com><44B2EAEE.8060807@ewtllc.com> Message-ID: > we haven't had that for long, and I cannot recall anyone being puzzled by the > fact that you can use a dictionary of (item, None) pairs to efficiently represent > a set. message += " before set was added to the language" From guido at python.org Thu Jul 13 16:29:04 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 13 Jul 2006 07:29:04 -0700 Subject: [Python-3000] Detecting conflicts in dict displays In-Reply-To: <44B5FCB8.7070702@cenix-bioscience.com> References: <9e804ac0607110727r303b2781s51fef76c08ee6817@mail.gmail.com> <44B3CC8B.7080703@ewtllc.com> <44B5FCB8.7070702@cenix-bioscience.com> Message-ID: On 7/13/06, Aaron Bingham wrote: [Quoting someone else] > >> Maybe. Like that, sure. But what about: > >> > >> a = {a: 1, b: 2} > >> > >> where 'a' and 'b' happen to be equal (or equal enough for dicts)? I'd > >> rather leave this kind of checking up to pycheker or pylint (which > >> already check for quite a lot more than I'd want the Python runtime > >> to check :) > > > > +1 for leaving this to PyLint or PyChecker. > > Wouldn't the behavior of the above be undefined for the case where a == > b? I'd much rather get an exception than undefined behavior. There's nothing undefined in the language currently about {a: 1, b: 2}. It creates an empty dict d, and then does d[a] = 1; d[b] = 2. If a and b are equal, you end up with {a: 2}. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik at pythonware.com Thu Jul 13 16:40:23 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 13 Jul 2006 16:40:23 +0200 Subject: [Python-3000] Detecting conflicts in dict displays References: <9e804ac0607110727r303b2781s51fef76c08ee6817@mail.gmail.com><44B3CC8B.7080703@ewtllc.com> <44B5FCB8.7070702@cenix-bioscience.com> Message-ID: Guido van Rossum wrote: >> Wouldn't the behavior of the above be undefined for the case where a == >> b? I'd much rather get an exception than undefined behavior. > > There's nothing undefined in the language currently about {a: 1, b: > 2}. It creates an empty dict d, and then does d[a] = 1; d[b] = 2. If a > and b are equal, you end up with {a: 2}. for the formal definition, see "dictionary displays" in the language reference: The key/datum pairs are evaluated from left to right to define the entries of the dictionary: each key object is used as a key into the dictionary to store the corresponding datum. /.../ Clashes between duplicate keys are not detected; the last datum (textually rightmost in the display) stored for a given key value pre- vails. ( http://pyref.infogami.com/dictionary-displays ) From collinw at gmail.com Thu Jul 13 22:12:10 2006 From: collinw at gmail.com (Collin Winter) Date: Thu, 13 Jul 2006 16:12:10 -0400 Subject: [Python-3000] Removing functions from the operator module In-Reply-To: <43aa6ff70607030840l3bf31fbdw45664884578d3f83@mail.gmail.com> References: <43aa6ff70607030605p2ca6af88i3c716f2e4b7d192@mail.gmail.com> <43aa6ff70607030711y32c8072am4197d98eff2ae6b6@mail.gmail.com> <43aa6ff70607030840l3bf31fbdw45664884578d3f83@mail.gmail.com> Message-ID: <43aa6ff70607131312q1d323a07hce351a1d648e1d71@mail.gmail.com> On 7/3/06, Collin Winter wrote: > On 7/3/06, Guido van Rossum wrote: > > It turns out I was misled by Collin's claim that the PEP wants > > isCallable and sequenceIncludes removed "because there are better, > > more obvious ways to spell these things." He must have made up the > > motivation, as the PEP doesn't give any. > > I was inferring that motivation So that others don't have to infer in the future, I've posted patch #1522038, which adds an explanation as to why these items are being removed. Also, in case you (Guido) have changed your mind about removing operator.truth and operator.abs, I've also posted patch #1522059 which will remove them from the stdlib. Thanks, Collin Winter From bingham at cenix-bioscience.com Fri Jul 14 16:03:28 2006 From: bingham at cenix-bioscience.com (Aaron Bingham) Date: Fri, 14 Jul 2006 16:03:28 +0200 Subject: [Python-3000] Detecting conflicts in dict displays In-Reply-To: References: <9e804ac0607110727r303b2781s51fef76c08ee6817@mail.gmail.com><44B3CC8B.7080703@ewtllc.com> <44B5FCB8.7070702@cenix-bioscience.com> Message-ID: <44B7A430.9080900@cenix-bioscience.com> Fredrik Lundh wrote: >Guido van Rossum wrote: > > > >>>Wouldn't the behavior of the above be undefined for the case where a == >>>b? I'd much rather get an exception than undefined behavior. >>> >>> >>There's nothing undefined in the language currently about {a: 1, b: >>2}. It creates an empty dict d, and then does d[a] = 1; d[b] = 2. If a >>and b are equal, you end up with {a: 2}. >> >> > >for the formal definition, see "dictionary displays" in the language reference: > > The key/datum pairs are evaluated from left to right to define the > entries of the dictionary: each key object is used as a key into the > dictionary to store the corresponding datum. /.../ > > Clashes between duplicate keys are not detected; the last datum > (textually rightmost in the display) stored for a given key value pre- > vails. > > ( http://pyref.infogami.com/dictionary-displays ) > > Thanks for the clarification. That is reassuring. Of course, this may not do what was intended under certain circumstances, but at least the behavior is well defined. Regards, -- -------------------------------------------------------------------- Aaron Bingham Senior Software Engineer Cenix BioScience GmbH -------------------------------------------------------------------- From qrczak at knm.org.pl Sat Jul 15 20:38:34 2006 From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: Sat, 15 Jul 2006 20:38:34 +0200 Subject: [Python-3000] Switch and static, redux In-Reply-To: <44AC5833.5010608@canterbury.ac.nz> (Greg Ewing's message of "Thu, 06 Jul 2006 12:24:19 +1200") References: <44AC5833.5010608@canterbury.ac.nz> Message-ID: <87zmfayaqd.fsf@qrnik.zagroda> Greg Ewing writes: > So I'd like to propose that "def time" be modified > slightly to mean "def time of the outermost function > that is still inside the scopes containing all the > names that the case expressions depend on". Not > sure about the wording of that, but I hope you get the > idea. In the above example, that would be the def time > of f(), rather than g(). If it was possible to define constants (as opposed to variables), including recognizing imported named constants, then the natural semantics of switch (evaluate cases at the time of the switch) would be easy to optimize. -- __("< Marcin Kowalczyk \__/ qrczak at knm.org.pl ^^ http://qrnik.knm.org.pl/~qrczak/ From jcarlson at uci.edu Sat Jul 15 22:14:19 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 15 Jul 2006 13:14:19 -0700 Subject: [Python-3000] Switch and static, redux In-Reply-To: <87zmfayaqd.fsf@qrnik.zagroda> References: <44AC5833.5010608@canterbury.ac.nz> <87zmfayaqd.fsf@qrnik.zagroda> Message-ID: <20060715131306.0C9C.JCARLSON@uci.edu> "Marcin 'Qrczak' Kowalczyk" wrote: > > Greg Ewing writes: > > > So I'd like to propose that "def time" be modified > > slightly to mean "def time of the outermost function > > that is still inside the scopes containing all the > > names that the case expressions depend on". Not > > sure about the wording of that, but I hope you get the > > idea. In the above example, that would be the def time > > of f(), rather than g(). > > If it was possible to define constants (as opposed to variables), > including recognizing imported named constants, then the natural > semantics of switch (evaluate cases at the time of the switch) > would be easy to optimize. You aren't saying anything that hasn't been said in this thread at least a dozen times. You are at least 3 weeks behind in this conversation. Start with the python-dev archives and catch up, or read the PEP. - Josiah From ncoghlan at gmail.com Sun Jul 16 06:43:35 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 16 Jul 2006 14:43:35 +1000 Subject: [Python-3000] Import system questions to be considered for Py3k Message-ID: <44B9C3F7.8090806@gmail.com> Taking the "import system" to be the overall interaction between the Python module namespace and the file system of the underlying computer, I thought I'd start compiling a list of the questions we'll want to consider for Py3k. The answers to some of them may be "the status quo is fine" but we should still ask the questions. I'll eventually capture the discussion in a Py3k PEP (although I believe many of the questions could actually be addressed for 2.6). The list I've got so far (including some thoughts about possible solutions): Change to hybrid implementation ------------------------------- This idea would try to reduce the amount of code in import.c, pushing more of the logic into Python code. An advantage of this is that much of the PEP 302 structure for the standard import mechanisms already exists in pkgutil (since PJE consolidated the various emulations that had been added to the standard library). Additionally, import logic written in Python would automatically benefit from the full Unicode filename support of the builtin open() function. The various string manipulation operations involved would also be significantly easier to handle. There would be some bootstrapping issues, but I think it would be better to try to solve them, rather than continuing to maintain the partial file system access API reimplementation that import.c currently uses (that, for example, doesn't provide full Unicode filename support on Windows). Even if most of the logic stays in C code, it would be good to find a way to use the full filesystem API, rather than the current import-only subset. Use smarter data structures --------------------------- Currently, the individual handlers to load a fully identified module are exposed to Python code in a way that reflects the C-style data structures used in the current implementation. Simply switching to more powerful data structures for the file type handlers (i.e. use a PyTuple for filedescr values, a PyList for _PyImport_FileTab, and a PyDict instead of a switch statement to go from filedescr values to module loading/initialisation functions) and manipulating them all as normal Python objects could make the code in import.c much easier to follow. Extensible file type handling ----------------------------- If the file type handlers are stored in normal Python data structures as described above, it becomes feasible to make the import system extensible to different file types as well as to different file locations. This could be handled on a per-package basis, e.g. via a __file_types__ special attribute in packages. Locating support files ---------------------- Currently, locating support files is difficult because __loader__ isn't defined for standard modules, and __file__ may not be defined properly if the module isn't being executed via load_module(). This needs to be changed so that there is an obvious way to locate support files located in the same directory as the current module. Determining the value of __file__ --------------------------------- In PEP 302, the logic to determine the value of __file__ is internal to the load_module() method. Should this be exposed so that, e.g., runpy.run_module can use it? Handling sys.argv[0] -------------------- Should new attributes be added to sys to separate out argv[0] from the command line arguments? For example, sys.mainfile (== sys.argv[0]) and sys.args (== sys.argv[1:]). This has compatibility implications for code that _sets_ sys.argv, and expects other code to see the changes. Determining the value of sys.path[0] ------------------------------------ sys.path[0] is set by the interpreter, depending on how the interpreter was started. If the main module is executed by filename, then sys.path[0] is set to the directory containing that file. If the main module is inside a package, all of the modules in that package have an aliasing problem (reachable as both top-level modules and by their full name). All other means of invocation leave sys.path[0] set to '', to indicate "current working directory". Should this actually read the name of the current working directory from the OS when the interpreter starts, or should it continue to reflect changes to the working directory over the course of execution? Should there be a command line switch to set sys.path[0] directly (or avoid having it set at all)? This would make it possible to avoid the aliasing problem with running modules from inside package directories, as well as allowing -m execution to be used for a module or package that is not in the current directory, but isn't on PYTHONPATH or in site-packages, either. (The latter would be a convenience for testing purposes, rather than something an installed Python application should be reliant on) Handling relative imports ------------------------- Currently the import system has to look at __name__, and then check if __path__ is present, in order to decide how to handle a relative import - the handling is different depending on whether the current module is a package or not. Defining a new special variable __pkg_name__ would allow the import system to use consistent logic for both packages and normal modules. This would also mean that relative imports would work correctly even when __name__ is set to something like "__main__". Revisiting PEP 299 ------------------ The general consensus recently has been that the "if __name__ == '__main__':" idiom for modules that can be both support modules and main modules is both ugly and unintuitive. PEP 299 (__main__ functions) was rejected for the 2.x series due to backward compatibility concerns (in particular, with modules that include the line "import __main__"). Py3k provides an opportunity to revisit that decision. If it is taken as a given that the idiom needs to change, then the question is whether the major change proposed by PEP 299 is a better option than a simpler change such as a new builtin boolean variable that can be tested via something like "if is_main:". Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From brett at python.org Sun Jul 16 07:53:47 2006 From: brett at python.org (Brett Cannon) Date: Sat, 15 Jul 2006 22:53:47 -0700 Subject: [Python-3000] Import system questions to be considered for Py3k In-Reply-To: <44B9C3F7.8090806@gmail.com> References: <44B9C3F7.8090806@gmail.com> Message-ID: On 7/15/06, Nick Coghlan wrote: > > Taking the "import system" to be the overall interaction between the > Python > module namespace and the file system of the underlying computer, I thought > I'd > start compiling a list of the questions we'll want to consider for Py3k. > The > answers to some of them may be "the status quo is fine" but we should > still > ask the questions. > > I'll eventually capture the discussion in a Py3k PEP (although I believe > many > of the questions could actually be addressed for 2.6). > > The list I've got so far (including some thoughts about possible > solutions): > > Change to hybrid implementation > ------------------------------- > This idea would try to reduce the amount of code in import.c, pushing more > of > the logic into Python code. An advantage of this is that much of the PEP > 302 > structure for the standard import mechanisms already exists in pkgutil > (since > PJE consolidated the various emulations that had been added to the > standard > library). Additionally, import logic written in Python would automatically > benefit from the full Unicode filename support of the builtin open() > function. > > The various string manipulation operations involved would also be > significantly easier to handle. > > There would be some bootstrapping issues, but I think it would be better > to > try to solve them, rather than continuing to maintain the partial file > system > access API reimplementation that import.c currently uses (that, for > example, > doesn't provide full Unicode filename support on Windows). > > Even if most of the logic stays in C code, it would be good to find a way > to > use the full filesystem API, rather than the current import-only subset. There is a strong possibility I will be rewriting the import machinery in Python in order to make my sandboxing life easier. Plus I realize it just needs to be done. And the bootstrapping issue is not a problem. I think I have a solution for that one. As long as I write in C the part of importing that handles modules that are compiled into the interpreter (so that you can get access to posix & friends along with sys without having to actually import any other code), then most of it can be written in Python. -Brett Use smarter data structures > --------------------------- > Currently, the individual handlers to load a fully identified module are > exposed to Python code in a way that reflects the C-style data structures > used > in the current implementation. > > Simply switching to more powerful data structures for the file type > handlers > (i.e. use a PyTuple for filedescr values, a PyList for _PyImport_FileTab, > and > a PyDict instead of a switch statement to go from filedescr values to > module > loading/initialisation functions) and manipulating them all as normal > Python > objects could make the code in import.c much easier to follow. > > Extensible file type handling > ----------------------------- > If the file type handlers are stored in normal Python data structures as > described above, it becomes feasible to make the import system extensible > to > different file types as well as to different file locations. > > This could be handled on a per-package basis, e.g. via a __file_types__ > special attribute in packages. > > Locating support files > ---------------------- > Currently, locating support files is difficult because __loader__ isn't > defined for standard modules, and __file__ may not be defined properly if > the > module isn't being executed via load_module(). This needs to be changed so > that there is an obvious way to locate support files located in the same > directory as the current module. > > Determining the value of __file__ > --------------------------------- > In PEP 302, the logic to determine the value of __file__ is internal to > the > load_module() method. Should this be exposed so that, e.g., > runpy.run_module > can use it? > > Handling sys.argv[0] > -------------------- > Should new attributes be added to sys to separate out argv[0] from the > command > line arguments? For example, sys.mainfile (== sys.argv[0]) and sys.args(== > sys.argv[1:]). > > This has compatibility implications for code that _sets_ sys.argv, and > expects > other code to see the changes. > > > Determining the value of sys.path[0] > ------------------------------------ > sys.path[0] is set by the interpreter, depending on how the interpreter > was > started. > > If the main module is executed by filename, then sys.path[0] is set to the > directory containing that file. If the main module is inside a package, > all of > the modules in that package have an aliasing problem (reachable as both > top-level modules and by their full name). > > All other means of invocation leave sys.path[0] set to '', to indicate > "current working directory". Should this actually read the name of the > current > working directory from the OS when the interpreter starts, or should it > continue to reflect changes to the working directory over the course of > execution? > > Should there be a command line switch to set sys.path[0] directly (or > avoid > having it set at all)? This would make it possible to avoid the aliasing > problem with running modules from inside package directories, as well as > allowing -m execution to be used for a module or package that is not in > the > current directory, but isn't on PYTHONPATH or in site-packages, either. > (The > latter would be a convenience for testing purposes, rather than something > an > installed Python application should be reliant on) > > Handling relative imports > ------------------------- > Currently the import system has to look at __name__, and then check if > __path__ is present, in order to decide how to handle a relative import - > the > handling is different depending on whether the current module is a package > or not. > > Defining a new special variable __pkg_name__ would allow the import system > to > use consistent logic for both packages and normal modules. This would also > mean that relative imports would work correctly even when __name__ is set > to > something like "__main__". > > Revisiting PEP 299 > ------------------ > The general consensus recently has been that the "if __name__ == > '__main__':" > idiom for modules that can be both support modules and main modules is > both > ugly and unintuitive. > > PEP 299 (__main__ functions) was rejected for the 2.x series due to > backward > compatibility concerns (in particular, with modules that include the line > "import __main__"). Py3k provides an opportunity to revisit that decision. > > If it is taken as a given that the idiom needs to change, then the > question is > whether the major change proposed by PEP 299 is a better option than a > simpler > change such as a new builtin boolean variable that can be tested via > something > like "if is_main:". > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > http://www.boredomandlaziness.org > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20060715/7910a272/attachment-0001.html From python at zesty.ca Sun Jul 16 12:37:08 2006 From: python at zesty.ca (Ka-Ping Yee) Date: Sun, 16 Jul 2006 05:37:08 -0500 (CDT) Subject: [Python-3000] Import system questions to be considered for Py3k In-Reply-To: <44B9C3F7.8090806@gmail.com> References: <44B9C3F7.8090806@gmail.com> Message-ID: On Sun, 16 Jul 2006, Nick Coghlan wrote: > Taking the "import system" to be the overall interaction between the > Python module namespace and the file system of the underlying computer, > I thought I'd start compiling a list of the questions we'll want to > consider for Py3k. I'd like to add "reloading" to your list. Updating a Python interpreter such that it reflects changes in module files is currently quite difficult. Calling reload() works somewhat for a single module if it has no dependencies, but beyond that the problem gets very tricky. It would be nice if we had a design that made it possible to achieve such an update (even if it were expensive to do -- it would be nice just to be able to identify the steps one would have to take to be confident that everything was up to date). If that's too much to ask, i think we should at least be able to say under what conditions we expect reloading to work (e.g. is it possible to write programs in a style that makes reloading feasible?) and/or what guarantees a realod will make or not make. A coherent reloading model is one of the huge advantages that PHP currently has over Python that contributes to the hurdles blocking Python's use for lightweight web scripting. When the context is rapid prototyping, it is perhaps *the* killer advantage, outweighing all the other flaws in PHP's language design. -- ?!ng From jcarlson at uci.edu Sun Jul 16 18:12:23 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Sun, 16 Jul 2006 09:12:23 -0700 Subject: [Python-3000] Import system questions to be considered for Py3k In-Reply-To: References: <44B9C3F7.8090806@gmail.com> Message-ID: <20060716085034.0C9F.JCARLSON@uci.edu> Ka-Ping Yee wrote: > > On Sun, 16 Jul 2006, Nick Coghlan wrote: > > Taking the "import system" to be the overall interaction between the > > Python module namespace and the file system of the underlying computer, > > I thought I'd start compiling a list of the questions we'll want to > > consider for Py3k. > > I'd like to add "reloading" to your list. Updating a Python interpreter > such that it reflects changes in module files is currently quite > difficult. Calling reload() works somewhat for a single module if > it has no dependencies, but beyond that the problem gets very tricky. > > It would be nice if we had a design that made it possible to achieve > such an update (even if it were expensive to do -- it would be nice > just to be able to identify the steps one would have to take to be > confident that everything was up to date). If that's too much to ask, > i think we should at least be able to say under what conditions we > expect reloading to work (e.g. is it possible to write programs in > a style that makes reloading feasible?) and/or what guarantees a > realod will make or not make. > > A coherent reloading model is one of the huge advantages that PHP > currently has over Python that contributes to the hurdles blocking > Python's use for lightweight web scripting. When the context is > rapid prototyping, it is perhaps *the* killer advantage, outweighing > all the other flaws in PHP's language design. I'd first start out by claiming that reload() works. That is, you can reload a specified module, and that should really be the only change in modules. It sounds to me like you want a recursive reload() such that when you are reloading moduleX that imports non-system moduleY, that Python also reload moduleY. While I agree that such a thing would be convenient, especially in the realm of web frameworks, etc., it wouldn't necessarily solve the 'stale code' problem, because it doesn't necessarily update all references to that module or its contents due to from imports. What may or may not be more effective is to only ever use references to modules in local scopes, which are generated from another object that verifies the "most recent version" nature that a web framework would necessitate. That is, one could access a reloader via attributes, which will dynamically reload modules as necessary... DR.moduleX.moduleY.classZ DR would check the 'currentness' of moduleX on access, check the 'currentness' of moduleY, and would note that classZ was not a module in the most recent moduleY, so would return it. For gaining non-dynamic references, one could even offer a moduleY = getmodule(DR.moduleX.moduleY) . From this semantic, it is a (somewhat tedious) matter of writing the implementation of DR, and replacing imports with the above references. While it still wouldn't be optimal, it's significantly better than the current alternatives (even a recursive reload). - Josiah From ncoghlan at gmail.com Mon Jul 17 11:22:53 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 17 Jul 2006 19:22:53 +1000 Subject: [Python-3000] Import system questions to be considered for Py3k In-Reply-To: <44B9C3F7.8090806@gmail.com> References: <44B9C3F7.8090806@gmail.com> Message-ID: <44BB56ED.5030804@gmail.com> A couple I accidentally left off my original list: Rationalise *.pkg and *.pth files --------------------------------- site.py processes *.pth files in the site-packages directory (the exact location being platform dependent) and adds things to sys.path as per the documentation of the site module. pkgutil.extend_path processes *.pkg files as per the documentation of the pkgutil module. *.pkg files are generally easier to handle than *.pth files - are the additional features of *.pth files really needed? What advantages do *.pth files offer over a normal Python module in site-packages (or anywhere else on the system path) that uses pkgutil.extend_path() to set the module's __path__ attribute? Use an interpreter configuration file? -------------------------------------- PYTHONPATH and the other interpreter controlling environment variables are shared between Python interpreters. This may cause problems if multiple versions of the interpreter are installed on a single machine. On Windows, some additional configuration data is squirreled away in the Windows registry by the installer (although this data is at least version specific). Jython deals with configuration aspects like PYTHONPATH in a cross-platform fashion by consulting a hierarchy of configuration data (system defaults, installation configuration file, user configuration file, command line argument overrides) [1]. In addition to being more consistent across platforms, the generic configuration file and -D property definition mechanism used by Jython is also more extensible than CPython's current approach of having to define a new command line switch or environment variable for each configurable option. Obviously, restraint would still be needed in the addition of configuration options in order to prevent a blowout in the number of scenarios requiring testing. [1] http://www.jython.org/docs/registry.html -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From fwierzbicki at gmail.com Mon Jul 17 16:31:45 2006 From: fwierzbicki at gmail.com (Frank Wierzbicki) Date: Mon, 17 Jul 2006 10:31:45 -0400 Subject: [Python-3000] Import system questions to be considered for Py3k In-Reply-To: <44BB56ED.5030804@gmail.com> References: <44B9C3F7.8090806@gmail.com> <44BB56ED.5030804@gmail.com> Message-ID: <4dab5f760607170731m4ea115bek883823962735d06a@mail.gmail.com> > [1] http://www.jython.org/docs/registry.html > On the new Jython website the same content is here: http://www.jython.org/Project/userguide.html#the-jython-registry Though I don't think there are any changes to this content. -Frank From ark at acm.org Mon Jul 17 18:19:44 2006 From: ark at acm.org (Andrew Koenig) Date: Mon, 17 Jul 2006 12:19:44 -0400 Subject: [Python-3000] callable() Message-ID: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> I note in PEP 3000 the proposal to remove callable(), with the comment "just call the object and catch the exception." I think that's a bad idea, because it takes away the ability to separate the callability test from the first call. As a simple example, suppose you're writing a function that you expect to be given a function as one of its arguments: def foo(bar, fun): assert callable(fun) # ... It might be that foo doesn't actually call fun until much later. Nevertheless, from a testing viewpoint, it would be better to detect the error immediately of passing something that can't be called. If you didn't have callable, how would you write this example? From ironfroggy at gmail.com Mon Jul 17 18:49:33 2006 From: ironfroggy at gmail.com (Calvin Spealman) Date: Mon, 17 Jul 2006 12:49:33 -0400 Subject: [Python-3000] callable() In-Reply-To: <76fd5acf0607170949g5b40b908qa808bad7b7be93cc@mail.gmail.com> References: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> <76fd5acf0607170949g5b40b908qa808bad7b7be93cc@mail.gmail.com> Message-ID: <76fd5acf0607170949m67206534u96e1f209f51071e2@mail.gmail.com> I second that removal of callable() would be a bad idea. There are too many cases where it is not feasable to "just call the object and catch the exception". In the cases where the call would not happen until later than the test for callability, it would require then that any state changes caused before the call would be reversed if calling the object raised any error, and sometimes that is not even possible. Further, it ignores all cases where we are simply introspecting and do not want any side effects from said introspection. On 7/17/06, Andrew Koenig wrote: > > I note in PEP 3000 the proposal to remove callable(), with the comment > "just > call the object and catch the exception." > > I think that's a bad idea, because it takes away the ability to separate > the > callability test from the first call. As a simple example, suppose you're > > writing a function that you expect to be given a function as one of its > arguments: > > def foo(bar, fun): > assert callable(fun) > # ... > > It might be that foo doesn't actually call fun until much later. > Nevertheless, from a testing viewpoint, it would be better to detect the > error immediately of passing something that can't be called. > > If you didn't have callable, how would you write this example? > > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/ironfroggy%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20060717/36034d33/attachment.htm From fperez.net at gmail.com Mon Jul 17 19:08:27 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 17 Jul 2006 11:08:27 -0600 Subject: [Python-3000] callable() References: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> <76fd5acf0607170949g5b40b908qa808bad7b7be93cc@mail.gmail.com> <76fd5acf0607170949m67206534u96e1f209f51071e2@mail.gmail.com> Message-ID: Calvin Spealman wrote: > I second that removal of callable() would be a bad idea. There are too > many cases where it is not feasable to "just call the object and catch the > exception". In the cases where the call would not happen until later than > the test for callability, it would require then that any state changes > caused before the call would be reversed if calling the object raised any > error, and sometimes that is not even possible. Further, it ignores all > cases where we are simply introspecting and do not want any side effects > from said introspection. +1 on not removing callable(). Given how calling something has potentially side effects, this is a good example of a situation where 'look before you leap' is a safer approach, IMHO. IPython makes extensive use of callable() to provide some of its command-line conveniences, and I don't see a clean way to replace it (short of just adding back a private implementation of what callable does today by copying out of a 2.x tree into ipython's). Cheers, f From aleaxit at gmail.com Mon Jul 17 19:30:25 2006 From: aleaxit at gmail.com (Alex Martelli) Date: Mon, 17 Jul 2006 10:30:25 -0700 Subject: [Python-3000] callable() In-Reply-To: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> References: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> Message-ID: On 7/17/06, Andrew Koenig wrote: > I note in PEP 3000 the proposal to remove callable(), with the comment "just > call the object and catch the exception." > > I think that's a bad idea, because it takes away the ability to separate the > callability test from the first call. As a simple example, suppose you're > writing a function that you expect to be given a function as one of its > arguments: > > def foo(bar, fun): > assert callable(fun) > # ... > > It might be that foo doesn't actually call fun until much later. > Nevertheless, from a testing viewpoint, it would be better to detect the > error immediately of passing something that can't be called. > > If you didn't have callable, how would you write this example? I might equivalently hasattr(type(fun), '__call__') [[net of oldstyle classes, which won't be around in py3k anyway:-)]] and it would be just as feeble -- essentially because when the time comes to CALL fun, it will be called with specific signature[s], not "in a void". IOW, callable as it stands is feeble because it tries to check for some too-wide category of types -- all types whose instances may be called in SOME way or other, while in fact the call[s] when they come will use *specific* signatures. This general kind of problem is of course familiar to users of compile-time-checked languages, who may be able e.g. to specify "x is an int" when what they really need is to assert "x is an even positive int" and the like -- but being a familiar evil does not make it any more sensible to me. If 'callable' is to stay, then in order to pull its weight it needs to grow to provide a way to check "callable with the following signature" -- at the very least in terms of numbers and names of arguments, though (if py3k does gain some syntax for specifying argument types) it might do more. E.g., just "callable(fun)" should mean "fun appears to be callable without arguments, e.g. I believe that doing 'fun()' will not result in a TypeError from the call operation itself", while "callable(fun, int, zap=list[int])" should mean "fun appears to be callable with one positional argument that's an int and an argument named zap that is a list of ints", i.e. the apparently-acceptable call is of some form such as "fun(23, [4, 5])" [[presumably using object as a placeholder for unchecked-types]] -- so that for example a def fun(zip, zap): would be OK (the name of the 2nd arg is correct, and fun doesn't check argument types so passing int and list of int looks like it should be OK), as would, say, def fun(zip: float, **k): assuming int->float is intrinsically accepted, since **k accepts arbitrary named arguments, but for example def fun(zip, zop): would not pass the check, since the misnamed 2nd argument would produce a type error at calltime; etc, etc. The def statement (or __init__/__new__ for callability of classes, or __call__ for callability of instances) should record enough information about acceptable signatures to make this enriched "callable" conceptually feasible, as long as we accept some false positives (we probably should be ready to accept them, since it's unfeasible to say exactly what further constraints fun's code body could place on acceptable arguments, sigh). Perhaps the best syntax for the functionality in question might be different: e.g., callable() could be some kind of type or interface, so that one would check with isinstance(fun, callable()) to check if fun is callable without arguments, etc, etc; this should have the advantage of playing well with functions specifying constraints on their arguments, so that e.g. def foo(bar, fun: callable()): would be the vastly-preferred way to express the code snippet you give (assuming that you want fun to be "callable _without arguments_" specifically, of course -- but then, I've already argued that being "callable in SOME ways that nobody can guess" is not a sensible constraint to check for, hm?-). Putting typechecks in foo's signature records them for posterity and eases somebody else's task of checking if *foo* itself is callable in the way they'd like to call it... Alex From collinw at gmail.com Mon Jul 17 19:45:37 2006 From: collinw at gmail.com (Collin Winter) Date: Mon, 17 Jul 2006 13:45:37 -0400 Subject: [Python-3000] callable() In-Reply-To: References: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> Message-ID: <43aa6ff70607171045g1b9d3951t57d46528127c583d@mail.gmail.com> On 7/17/06, Alex Martelli wrote: > [proposal for a type annotations-based version of callable()] To weigh in from the function/type annotations side of things, the consensus has been that Python 3000's function annotations are only a way to map parameter names to some arbitrary expression, i.e., they have no semantic value of their own. If I write 'def foo(a: int)', associating the int type with the 'a' parameter, core Python won't actually check that values provided for 'a' are in fact integers. Interpreting these annotations, giving them meaning, will be left completely up to third-party libraries. This reliance on third-party libraries, then, means that any annotations-based version of callable() would also have to be provided by these same third-party libraries. Put simply, the proposed implementation of callable() wouldn't be able to ship in __builtin__. Collin Winter From jcarlson at uci.edu Mon Jul 17 19:58:31 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 17 Jul 2006 10:58:31 -0700 Subject: [Python-3000] callable() In-Reply-To: References: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> Message-ID: <20060717105337.0CB6.JCARLSON@uci.edu> "Alex Martelli" wrote: > > On 7/17/06, Andrew Koenig wrote: > > I note in PEP 3000 the proposal to remove callable(), with the comment "just > > call the object and catch the exception." > > > > I think that's a bad idea, because it takes away the ability to separate the > > callability test from the first call. As a simple example, suppose you're > > writing a function that you expect to be given a function as one of its > > arguments: > > > > def foo(bar, fun): > > assert callable(fun) > > # ... > > > > It might be that foo doesn't actually call fun until much later. > > Nevertheless, from a testing viewpoint, it would be better to detect the > > error immediately of passing something that can't be called. > > > > If you didn't have callable, how would you write this example? > > I might equivalently hasattr(type(fun), '__call__') [[net of oldstyle > classes, which won't be around in py3k anyway:-)]] and it would be > just as feeble -- essentially because when the time comes to CALL fun, > it will be called with specific signature[s], not "in a void". I believe that callable as it stands is useful for what it was intended to do; tell you if something is callable. Whether or not a function is callable with a specific set of arguments as you suggest, would be useful, but not necessary in the cases that Andrew and others have been talking about. Also note that specifying and/or verifying call signatures is its own PEP. I personally believe that callable should stay as-is, and if signature verification is going to be done, it should probably have its own builtin (as code that was using callable(foo) before could break with changed semantics, and such breakage, I believe, would be gratuitous). - Josiah From baptiste13 at altern.org Mon Jul 17 20:07:39 2006 From: baptiste13 at altern.org (Baptiste Carvello) Date: Mon, 17 Jul 2006 20:07:39 +0200 Subject: [Python-3000] callable() In-Reply-To: References: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> Message-ID: > If 'callable' is to stay, then in order to pull its weight it needs to > grow to provide a way to check "callable with the following signature" > -- at the very least in terms of numbers and names of arguments, > though (if py3k does gain some syntax for specifying argument types) > it might do more. E.g., just "callable(fun)" should mean "fun appears > to be callable without arguments, e.g. I believe that doing 'fun()' > will not result in a TypeError from the call operation itself", while > "callable(fun, int, zap=list[int])" should mean "fun appears to be > callable with one positional argument that's an int and an argument > named zap that is a list of ints", i.e. the apparently-acceptable call > is of some form such as "fun(23, [4, 5])" [[presumably using object as > a placeholder for unchecked-types]] -- so that for example a > +1 on the concept. If it doesn't go in __builtin__, maybe it can go somewhere in the stdlib so that people don't have to reimplement it badly. BC From ironfroggy at gmail.com Mon Jul 17 20:18:18 2006 From: ironfroggy at gmail.com (Calvin Spealman) Date: Mon, 17 Jul 2006 14:18:18 -0400 Subject: [Python-3000] callable() In-Reply-To: <20060717105337.0CB6.JCARLSON@uci.edu> References: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> <20060717105337.0CB6.JCARLSON@uci.edu> Message-ID: <76fd5acf0607171118oa14328cwb91606f85439c89e@mail.gmail.com> On 7/17/06, Josiah Carlson wrote: > > > "Alex Martelli" wrote: > > > > On 7/17/06, Andrew Koenig wrote: > > > I note in PEP 3000 the proposal to remove callable(), with the comment > "just > > > call the object and catch the exception." > > > > > > I think that's a bad idea, because it takes away the ability to > separate the > > > callability test from the first call. As a simple example, suppose > you're > > > writing a function that you expect to be given a function as one of > its > > > arguments: > > > > > > def foo(bar, fun): > > > assert callable(fun) > > > # ... > > > > > > It might be that foo doesn't actually call fun until much later. > > > Nevertheless, from a testing viewpoint, it would be better to detect > the > > > error immediately of passing something that can't be called. > > > > > > If you didn't have callable, how would you write this example? > > > > I might equivalently hasattr(type(fun), '__call__') [[net of oldstyle > > classes, which won't be around in py3k anyway:-)]] and it would be > > just as feeble -- essentially because when the time comes to CALL fun, > > it will be called with specific signature[s], not "in a void". > > I believe that callable as it stands is useful for what it was intended > to do; tell you if something is callable. Whether or not a function is > callable with a specific set of arguments as you suggest, would be > useful, but not necessary in the cases that Andrew and others have been > talking about. > > Also note that specifying and/or verifying call signatures is its own > PEP. I personally believe that callable should stay as-is, and if > signature verification is going to be done, it should probably have its > own builtin (as code that was using callable(foo) before could break > with changed semantics, and such breakage, I believe, would be > gratuitous). > > - Josiah I like the idea of being able to ask callable about specific signitures. I don't think it needs to be a seperate function, but you would need a way to specify "is this callable with an unknown number of arguments?" and the signiture of callable itself in this context would grow ambiguous without well thought out arguments. Still, it would seem clunky to have seperate functions to deal with it. Then again, this could easily get out of hand and become interface discussions all over again. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20060717/08c834c6/attachment.html From mcherm at mcherm.com Mon Jul 17 23:32:39 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Mon, 17 Jul 2006 14:32:39 -0700 Subject: [Python-3000] set literals Message-ID: <20060717143239.1aoze81i7z0gko80@login.werra.lunarpages.com> Guido writes: > I've also sometimes thought of unifying dicts and sets by implementing > set operations on the keys of dicts only. [... much discussion ...] > I'm still very much undecided but I don't want to rule this out for > py3k. Perhaps I'll write up a PEP and see how it flies. Playing with it, and PEPing it both sound fine, but I think DOING it seems like a bad idea. I see two advantages. One is public: it solves the issue of a set literal. The other is private: it allows us to reuse the implementation. Fixing the set literal just isn't sufficient justification, IMHO. And as for the implementation, we care VERY much about perfectly tuning the performance of the dict type, because its performance is so key to the implementation of namespaces throughout Python. So I would not want to accept any unnecessary restrictions on the implementation that might constrain future optimizations of dict performance. Besides, how difficult is it to maintain the existing C implementation of set and frozenset (now that they're written and have been through the wringer of being in a production release). It's not zero cost, but it's also probably not THAT big. Of course, that's the idea behind trying it out and even writing a PEP -- then we'll see whether my guess (or yours!) is correct. -- Michael Chermside From guido at python.org Mon Jul 17 23:54:14 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 17 Jul 2006 14:54:14 -0700 Subject: [Python-3000] set literals In-Reply-To: <20060717143239.1aoze81i7z0gko80@login.werra.lunarpages.com> References: <20060717143239.1aoze81i7z0gko80@login.werra.lunarpages.com> Message-ID: Hm. Your objections seem to be purely from a performance tuning POV. I think that if we agree that API-wise this is an improvement (fewer things to learn, set literals problem solved, and dicts grow some useful new methods) we should make a decision to do it and damn the tuning (I trust Raymond will find a way :-). --Guido On 7/17/06, Michael Chermside wrote: > Guido writes: > > I've also sometimes thought of unifying dicts and sets by implementing > > set operations on the keys of dicts only. > [... much discussion ...] > > I'm still very much undecided but I don't want to rule this out for > > py3k. Perhaps I'll write up a PEP and see how it flies. > > Playing with it, and PEPing it both sound fine, but I think DOING it > seems like a bad idea. > > I see two advantages. One is public: it solves the issue of a set > literal. The other is private: it allows us to reuse the implementation. > > Fixing the set literal just isn't sufficient justification, IMHO. And > as for the implementation, we care VERY much about perfectly tuning > the performance of the dict type, because its performance is so key to > the implementation of namespaces throughout Python. So I would not want > to accept any unnecessary restrictions on the implementation that might > constrain future optimizations of dict performance. > > Besides, how difficult is it to maintain the existing C implementation > of set and frozenset (now that they're written and have been through > the wringer of being in a production release). It's not zero cost, but > it's also probably not THAT big. > > Of course, that's the idea behind trying it out and even writing a PEP > -- then we'll see whether my guess (or yours!) is correct. > > -- Michael Chermside > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From monpublic at gmail.com Tue Jul 18 00:14:16 2006 From: monpublic at gmail.com (CM) Date: Mon, 17 Jul 2006 15:14:16 -0700 Subject: [Python-3000] set literals In-Reply-To: References: <20060717143239.1aoze81i7z0gko80@login.werra.lunarpages.com> Message-ID: On 7/17/06, Guido van Rossum wrote: > > Hm. Your objections seem to be purely from a performance tuning POV. I > think that if we agree that API-wise this is an improvement (fewer > things to learn, set literals problem solved, and dicts grow some > useful new methods) we should make a decision to do it and damn the > tuning (I trust Raymond will find a way :-). +1 Raymond always finds a way. - C --Guido > > On 7/17/06, Michael Chermside wrote: > > Guido writes: > > > I've also sometimes thought of unifying dicts and sets by implementing > > > set operations on the keys of dicts only. > > [... much discussion ...] > > > I'm still very much undecided but I don't want to rule this out for > > > py3k. Perhaps I'll write up a PEP and see how it flies. > > > > Playing with it, and PEPing it both sound fine, but I think DOING it > > seems like a bad idea. > > > > I see two advantages. One is public: it solves the issue of a set > > literal. The other is private: it allows us to reuse the implementation. > > > > Fixing the set literal just isn't sufficient justification, IMHO. And > > as for the implementation, we care VERY much about perfectly tuning > > the performance of the dict type, because its performance is so key to > > the implementation of namespaces throughout Python. So I would not want > > to accept any unnecessary restrictions on the implementation that might > > constrain future optimizations of dict performance. > > > > Besides, how difficult is it to maintain the existing C implementation > > of set and frozenset (now that they're written and have been through > > the wringer of being in a production release). It's not zero cost, but > > it's also probably not THAT big. > > > > Of course, that's the idea behind trying it out and even writing a PEP > > -- then we'll see whether my guess (or yours!) is correct. > > > > -- Michael Chermside > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/monpublic%40gmail.com > -- C -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20060717/63434980/attachment.htm From jack at psynchronous.com Tue Jul 18 05:28:58 2006 From: jack at psynchronous.com (Jack Diederich) Date: Mon, 17 Jul 2006 23:28:58 -0400 Subject: [Python-3000] set literals In-Reply-To: References: <20060717143239.1aoze81i7z0gko80@login.werra.lunarpages.com> Message-ID: <20060718032858.GD27124@performancedrivers.com> On Mon, Jul 17, 2006 at 02:54:14PM -0700, Guido van Rossum wrote: > Hm. Your objections seem to be purely from a performance tuning POV. I > think that if we agree that API-wise this is an improvement (fewer > things to learn, set literals problem solved, and dicts grow some > useful new methods) we should make a decision to do it and damn the > tuning (I trust Raymond will find a way :-). Sets are dicts like lists are tuples. Similar but different. Heck no one has proposed ditching tuples just because lists are a superset (err, not recently). Back in the bad old days of 2.2 when I used a dict as a set I would defensively initialize it as uniques = dict(zip(somelist, somelist)) so the values could never be abused a a side effect ten lines down. Python has a great tradition of supporting a few basic operations across different kinds of containers - the 'in' and 'iter(x)' and sometimes 'foo[x]' Commonalities accross types exist right up until the uses are different. Let's not mess it up by going towards the PHP route of unifying more than we need to (I'm thinking of the unification of hashes and vectors). Not even Raymond can make dicts as cheap as sets. He did after all scoop out portions of the dict machinery to make sets as fast as they could be. He was kind enough to write Object/dictnotes.txt to record the different ways dictionaires are used and again to break out sets as a distinct type. For what it is worth I'm biased towards the specialization of basic types as long as there is a conceptual split. I spent a week before the Need For Speed sprint and a day during separating dictionaries used for symbol lookup from general hash tables. The result worked but was unusable a week before 2.5-beta1 as it was far too invasive. I still like the idea of namespaces being dict work-alikes but not necessarily dict()s proper. -jack nb, the suggestion in this thread that 'dictA |= dictB' have the opposite meaning of 'dictA.update(dictB)' with respect to values has a code smell all its own. > On 7/17/06, Michael Chermside wrote: > > Guido writes: > > > I've also sometimes thought of unifying dicts and sets by implementing > > > set operations on the keys of dicts only. > > [... much discussion ...] > > > I'm still very much undecided but I don't want to rule this out for > > > py3k. Perhaps I'll write up a PEP and see how it flies. > > > > Playing with it, and PEPing it both sound fine, but I think DOING it > > seems like a bad idea. > > > > I see two advantages. One is public: it solves the issue of a set > > literal. The other is private: it allows us to reuse the implementation. > > > > Fixing the set literal just isn't sufficient justification, IMHO. And > > as for the implementation, we care VERY much about perfectly tuning > > the performance of the dict type, because its performance is so key to > > the implementation of namespaces throughout Python. So I would not want > > to accept any unnecessary restrictions on the implementation that might > > constrain future optimizations of dict performance. > > > > Besides, how difficult is it to maintain the existing C implementation > > of set and frozenset (now that they're written and have been through > > the wringer of being in a production release). It's not zero cost, but > > it's also probably not THAT big. > > > > Of course, that's the idea behind trying it out and even writing a PEP > > -- then we'll see whether my guess (or yours!) is correct. > > > > -- Michael Chermside > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/jack%40performancedrivers.com > From greg.ewing at canterbury.ac.nz Tue Jul 18 09:04:29 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 18 Jul 2006 19:04:29 +1200 Subject: [Python-3000] callable() In-Reply-To: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> References: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> Message-ID: <44BC87FD.8000808@canterbury.ac.nz> Andrew Koenig wrote: > I note in PEP 3000 the proposal to remove callable(), with the comment "just > call the object and catch the exception." I think that should be amended to "just use hasattr(obj. '__call__') instead". That's what callable() will become equivalent to once old_style classes are gone. Then it becomes a matter of whether it's worth having callable() around as an alternative spelling. Those arguing in favour of it would have to explain whether we should also have addable(), subtractable(), mutiplyable(), indexable(), etc. etc. etc... -- Greg From behnel_ml at gkec.informatik.tu-darmstadt.de Tue Jul 18 10:27:03 2006 From: behnel_ml at gkec.informatik.tu-darmstadt.de (Stefan Behnel) Date: Tue, 18 Jul 2006 10:27:03 +0200 Subject: [Python-3000] callable() In-Reply-To: <44BC87FD.8000808@canterbury.ac.nz> References: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> <44BC87FD.8000808@canterbury.ac.nz> Message-ID: <44BC9B57.5020302@gkec.informatik.tu-darmstadt.de> Hi, Greg Ewing wrote: > Andrew Koenig wrote: >> I note in PEP 3000 the proposal to remove callable(), with the comment "just >> call the object and catch the exception." > > I think that should be amended to "just use hasattr(obj. '__call__') > instead". That's what callable() will become equivalent to once > old_style classes are gone. > > Then it becomes a matter of whether it's worth having callable() > around as an alternative spelling. Those arguing in favour of > it would have to explain whether we should also have addable(), > subtractable(), mutiplyable(), indexable(), etc. etc. etc... What about moving callable() into the "types" module? Stefan From murman at gmail.com Tue Jul 18 15:45:27 2006 From: murman at gmail.com (Michael Urman) Date: Tue, 18 Jul 2006 08:45:27 -0500 Subject: [Python-3000] callable() In-Reply-To: <44BC87FD.8000808@canterbury.ac.nz> References: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> <44BC87FD.8000808@canterbury.ac.nz> Message-ID: On 7/18/06, Greg Ewing wrote: > I think that should be amended to "just use hasattr(obj. '__call__') > instead". That's what callable() will become equivalent to once > old_style classes are gone. > > Then it becomes a matter of whether it's worth having callable() > around as an alternative spelling. Those arguing in favour of > it would have to explain whether we should also have addable(), > subtractable(), mutiplyable(), indexable(), etc. etc. etc... I think if the proponents of removing callable had made it more obvious that hasatr(obj, '__call__') was a relatively equivalent option which served to make it obvious what its limitations were, there would have been fewer objections. Then I personally find the intent indication of callable(obj) vs hasattr(obj, '__call__') to be a good trade. As for callable() vs addable(), etc., the reason is obvious. Function calls very commonly have large side effects and are only rarely idempotent. The operations you mention are expected to have no side effects on the object. This makes a try/except much more palatable for addition, subscripting, etc. than for calling a function. Michael -- Michael Urman http://www.tortall.net/mu/blog From guido at python.org Tue Jul 18 19:16:41 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 18 Jul 2006 10:16:41 -0700 Subject: [Python-3000] callable() In-Reply-To: References: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> <44BC87FD.8000808@canterbury.ac.nz> Message-ID: On 7/18/06, Michael Urman wrote: > On 7/18/06, Greg Ewing wrote: > > I think that should be amended to "just use hasattr(obj. '__call__') > > instead". That's what callable() will become equivalent to once > > old_style classes are gone. > > > > Then it becomes a matter of whether it's worth having callable() > > around as an alternative spelling. Those arguing in favour of > > it would have to explain whether we should also have addable(), > > subtractable(), mutiplyable(), indexable(), etc. etc. etc... > > I think if the proponents of removing callable had made it more > obvious that hasatr(obj, '__call__') was a relatively equivalent > option which served to make it obvious what its limitations were, > there would have been fewer objections. Then I personally find the > intent indication of callable(obj) vs hasattr(obj, '__call__') to be a > good trade. But I'm not proposing to use hasattr(obj, '__call__'). I'm basically saying that the callable test has very little to do with whether the later call will succeed, because we have no way to test the signature. > As for callable() vs addable(), etc., the reason is obvious. Function > calls very commonly have large side effects and are only rarely > idempotent. The operations you mention are expected to have no side > effects on the object. This makes a try/except much more palatable for > addition, subscripting, etc. than for calling a function. That's not at all obvious to me, and certainly I'm *not* proposing surrounding everything with a try/except. IMO the right solution lies in something like pychecker, not in adding look-before-you-leap type checks, and certainly not in adding code like try: do_something() except: print "An error occurred!" -- --Guido van Rossum (home page: http://www.python.org/~guido/) From murman at gmail.com Tue Jul 18 19:51:26 2006 From: murman at gmail.com (Michael Urman) Date: Tue, 18 Jul 2006 12:51:26 -0500 Subject: [Python-3000] callable() In-Reply-To: References: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> <44BC87FD.8000808@canterbury.ac.nz> Message-ID: On 7/18/06, Guido van Rossum wrote: > But I'm not proposing to use hasattr(obj, '__call__'). I'm basically > saying that the callable test has very little to do with whether the > later call will succeed, because we have no way to test the signature. Agreed. I think the people who want to use this as a test for whether a client passed them a usable object are barking up the wrong tree. What I do see it as useful for is making an api that accepts a foo-like-object, or a callable object that returns a foo-like-object. For properly defined scenarios this is perfectly well defined. One such case may be for passing strings or string-yielders into a templating system. If calculating the string is expensive, and it may only be used conditionally, one may wrap the calculation as a deferred callable. On the other hand, that scenario is just as amenable to the try/except handling, because if it can be called you do want to call it right then. The client-callback being passed as the wrong parameter scenario is one that benefits from early checking. If you wait until you need to call it, and it's not callable, the error message is more disorienting than could be arranged by an early check. If it has the wrong signature, passing the callable() check, but fails at call, the error message is much more useful. If it raises another error inside the function, there's a useful trace pointing to the raising code. > > As for callable() vs addable(), etc., the reason is obvious. Function > > calls very commonly have large side effects and are only rarely > > idempotent. The operations you mention are expected to have no side > > effects on the object. This makes a try/except much more palatable for > > addition, subscripting, etc. than for calling a function. > > That's not at all obvious to me, and certainly I'm *not* proposing > surrounding everything with a try/except. Absolutely not. For both I think the error should generally just be thrown. But when the error needs to be caught locally, it would be an unusual case that the try/except would have side effects for addition. > IMO the right solution lies in something like pychecker, not in adding > look-before-you-leap type checks I think this varies strongly on what scenario you are addressing. It's hard to verify your own API against future users with a static tool like pychecker. Michael -- Michael Urman http://www.tortall.net/mu/blog From ark-mlist at att.net Tue Jul 18 20:24:21 2006 From: ark-mlist at att.net (Andrew Koenig) Date: Tue, 18 Jul 2006 14:24:21 -0400 Subject: [Python-3000] callable() In-Reply-To: Message-ID: <000601c6aa97$643ada40$6402a8c0@arkdesktop> > Agreed. I think the people who want to use this as a test for whether > a client passed them a usable object are barking up the wrong tree. > What I do see it as useful for is making an api that accepts a > foo-like-object, or a callable object that returns a foo-like-object. Yes. What really got me started down this particular line of reasoning was Snobol4's notion (from 1968!) of an "unevaluated expression," (indicated by unary *) which was like a function with no arguments. A quick example: A = *(B + C) B = 4 C = 5 OUTPUT = EVAL(A) This code fragment prints 9. The Python analogy: a = (lambda: b + c) b = 4 c = 5 print a() Now, there are some contexts (particularly in pattern matching) that accept either an unevaluated expression or an ordinary value, and if they get an unevaluated expression, they evaluate it at the last possible instant during pattern matching. Imagine a program that is trying to build a data structure out of values, some of which might be unevaluated expressions. There are various optimizations you can do if you know that you are dealing with an actual value at the moment; so you would like to be able to determine whether something is an unevaluated expression so you can decide whether to attempt the optimizations. So you need a way to tell whether something is an unevaluated expression even though it's not yet time to evaluate it. If we use Python functions to represent unevaluated expressions, this desire leads naturally to a way to tell whether a value is a function. Of course we could solve the problem by using an instance of a specific class instead, but I feel that's just adding complexity without correspondingly adding utility. From ark-mlist at att.net Tue Jul 18 20:25:58 2006 From: ark-mlist at att.net (Andrew Koenig) Date: Tue, 18 Jul 2006 14:25:58 -0400 Subject: [Python-3000] callable() In-Reply-To: <44BC87FD.8000808@canterbury.ac.nz> Message-ID: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> > Andrew Koenig wrote: > > I note in PEP 3000 the proposal to remove callable(), with the comment > "just call the object and catch the exception." > I think that should be amended to "just use hasattr(obj. '__call__') > instead". That's what callable() will become equivalent to once > old_style classes are gone. I am uncomfortable about exposing the implementation this way, if only because it would require fixing the equivalence between callable() and hasattr(obj, '__call__') for all time. I much prefer the extra abstraction layer that callable() provides. From ark at acm.org Tue Jul 18 20:28:49 2006 From: ark at acm.org (Andrew Koenig) Date: Tue, 18 Jul 2006 14:28:49 -0400 Subject: [Python-3000] callable() In-Reply-To: <44BC87FD.8000808@canterbury.ac.nz> Message-ID: <000801c6aa98$0685a2d0$6402a8c0@arkdesktop> > Then it becomes a matter of whether it's worth having callable() > around as an alternative spelling. Those arguing in favour of > it would have to explain whether we should also have addable(), > subtractable(), mutiplyable(), indexable(), etc. etc. etc... I'd love to be able to determine whether a+b can be evaluated, and, if so, what function it would call. I don't think such a determination fits neatly into a notion that could be spelled "addable". Note that when you evaluate a+b, it might call a.__add__ or b.__radd__, or perhaps even something else entirely in the future. So it might be nice to have a function to which you can give two values and a binary operator (or one value and a unary operator), and which would tell you what function, if any, that operator would call. You might even treat "()" as an operator and unify it with the rest of them. Just one more form of introspection. But I don't think a separate function for each operator would be the best way of achieving that introspection. From rasky at develer.com Tue Jul 18 22:45:01 2006 From: rasky at develer.com (Giovanni Bajo) Date: Tue, 18 Jul 2006 22:45:01 +0200 Subject: [Python-3000] Import system questions to be considered for Py3k References: <44B9C3F7.8090806@gmail.com> <44BB56ED.5030804@gmail.com> Message-ID: <067701c6aaab$0a7fe040$d503030a@trilan> Nick Coghlan wrote: > Rationalise *.pkg and *.pth files > --------------------------------- > [...] > > Use an interpreter configuration file? > -------------------------------------- When dealing with these issues, please consider also startup time. Python is already "pretty slow" at startup, and I hope Py3k doesn't get any worse (it should get much better, if anything). I'm sure you all can find out a way to rationalite these startup import stuff so that they are faster. I'm afraid I don't have any constructive suggestion since I'm not knowledgable enough. $ strace python -c "print ''" 2>&1 | wc -l 855 $ strace perl -e "print ''" 2>&1 | wc -l 327 Stuff like: stat64("/usr/lib/python24.zip/posixpath", 0xbfd8f934) = -1 ENOENT (No such file or directory) open("/usr/lib/python24.zip/posixpath.so", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory) open("/usr/lib/python24.zip/posixpathmodule.so", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory) open("/usr/lib/python24.zip/posixpath.py", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory) open("/usr/lib/python24.zip/posixpath.pyc", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory) is at least questionable, given that /usr/lib/python24.zip does not even exist. -- Giovanni Bajo From greg.ewing at canterbury.ac.nz Wed Jul 19 03:12:36 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 19 Jul 2006 13:12:36 +1200 Subject: [Python-3000] callable() In-Reply-To: References: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> <44BC87FD.8000808@canterbury.ac.nz> Message-ID: <44BD8704.4020309@canterbury.ac.nz> Guido van Rossum wrote: > But I'm not proposing to use hasattr(obj, '__call__'). I'm basically > saying that the callable test has very little to do with whether the > later call will succeed, because we have no way to test the signature. I don't think that's needed for the sort of things people want callable() for. I think they want it for the purpose of implementing type-dependent APIs -- similar in spirit to doing different things depending on whether you're passed a sequence or not, etc. For that purpose, hasattr(obj, '__call__') is sufficient, I think. And if it's not, the API you're trying to implement is ill-conceived in the first place, just like testing for sequenceness is not a well-defined operation. -- Greg From greg.ewing at canterbury.ac.nz Wed Jul 19 03:21:06 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 19 Jul 2006 13:21:06 +1200 Subject: [Python-3000] callable() In-Reply-To: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> Message-ID: <44BD8902.7030900@canterbury.ac.nz> Andrew Koenig wrote: > I am uncomfortable about exposing the implementation this way, if only > because it would require fixing the equivalence between callable() and > hasattr(obj, '__call__') for all time. I don't see anything bad about fixing that equivalence. I regard the fact that it *wasn't* fixed before as a language design bug that Py3k will hopefully fix. -- Greg From fperez.net at gmail.com Wed Jul 19 08:47:12 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 19 Jul 2006 00:47:12 -0600 Subject: [Python-3000] callable() References: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> <44BC87FD.8000808@canterbury.ac.nz> <44BD8704.4020309@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > For that purpose, hasattr(obj, '__call__') is sufficient, > I think. And if it's not, the API you're trying to > implement is ill-conceived in the first place, just like > testing for sequenceness is not a well-defined operation. I should add that for ipython's purposes, it seems I could just implement def callable(o): return hasattr(o,'__call__') and it would be OK. I hadn't realized that Python reports true to the hasattr() call form for lambdas, simple plain functions, functions in extension modules, extension types, and most things I could quickly think of and check from code I have. I wasn't aware of this fact, I thought it would only be true for instances of classes which explicitly implemented a __call__ method, and that callable() did some other magic. Having understood this, I then don't really care whether callable() goes away or not, it's a one-liner as an interactive ipython convenience. Sorry for not checking more carefully, I trusted a misconception I had. Cheers, f From talin at acm.org Wed Jul 19 09:42:38 2006 From: talin at acm.org (Talin) Date: Wed, 19 Jul 2006 00:42:38 -0700 Subject: [Python-3000] callable() In-Reply-To: <44BD8704.4020309@canterbury.ac.nz> References: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> <44BC87FD.8000808@canterbury.ac.nz> <44BD8704.4020309@canterbury.ac.nz> Message-ID: <44BDE26E.5030504@acm.org> Greg Ewing wrote: > Guido van Rossum wrote: > > >>But I'm not proposing to use hasattr(obj, '__call__'). I'm basically >>saying that the callable test has very little to do with whether the >>later call will succeed, because we have no way to test the signature. > > > I don't think that's needed for the sort of things people > want callable() for. I think they want it for the purpose > of implementing type-dependent APIs -- similar in spirit > to doing different things depending on whether you're > passed a sequence or not, etc. I think this is exactly correct. Those who want to retain callable() [including myself] aren't looking for an ironclad guarantee that calling a given object will succeed - they merely want to check to see if the object is "of callable type", in other words, is it the roughly sort of object that one would expect to be able to call. I think part of the reason that this debate has gone on so long is that it's really a debate about the nature of Python as a scripting language. Every so-called 'scripting' language maintains a balance between strictness and forgiveness. On the one extreme, there are languages such as Lua, where accessing an undefined variable is silently ignored, or Javascript, where exceptions that would otherwise be fatal errors are merely logged as warnings. On the other hand, there is a long tail of increasingly bondage-and-dominance-oriented languages with ever more strict checking of the rules. As I am sure that all of us have discovered, the 'looser' language rules can greatly increase the speed of rapid prototyping. My own experience is that I can code Java about twice as fast as C++, and Python about twice as fast as Java, for equivalent functionality. However, these looser rules have their own costs, which I won't go into detail about. Certainly the strict languages create lots more opportunities for compile-time optimization. As to whether the strict languages are more reliable overall, that's an open question - while its likely that the the programs written in such languages are more provably correct, the very strictness of the language's own self-checking can itself become a source of, um, programs that don't work when they should. There's a common architectural pattern seen in both the Python standard library as well as many user-written applications, which I will call the "rough protocol" pattern (you might also think of it as "duck protocols"). The pattern is simple: Objects which conform to a given interface need not conform to it in theoretical exactitude, as would be required in a language such as C# or C++. Instead, objects only need conform to an interface so far as it is practical and convenient to do so. As an example, I can create a "file-like" object, without having to support all of the various little methods and subtle behaviors of "file". The reason these rough protocols are of such benefit is that you save a great deal of time - you can avoid having to track down and implement all the little subtle fiddly bits of the given interface that would otherwise be needed to make the compiler happy, but which you don't actually plan to use. One of the types of rough protocols that we often see is one in which an object is passed as an argument to a function, and the function attempts to classify the object according to some simple criteria into one of several recognized types, and then take action based on that categorization. You see this all the time with things like "if X is a sequence, then do this with it, otherwise do that". Now, this violates all kinds of theoretical considerations such as "well, what if X only vaguely resembles a sequence', or 'what if X looks like a sequence, but really isn't'. One can take the formal definition of 'sequence' and generate an endless series of mutations and permutations of that definition which may or may not work for a given function (or even make sense for that matter). The practical answer to all of this is that such contrivances hardly ever occur in real code. As programmers, we have common-sense expectations of what is a sequence, what is a map, what is a function, and so on. Those are the primitive concepts with which we assemble our mental designs before putting them down on disk. As soon as a class starts to stray too far outside the envelope of those expectations, we tend to call it something besides 'sequence' or 'map' or whatever. Rough protocols don't need to know exactly the type of object or what its exact method signatures are. It only needs to know enough about the object to decide what common-sense category to place it in. Rough protocols *aren't* guaranteed to be able to inspect the object thoroughly enough to know whether the category test is valid - it assumes that the user is smart enough to only pass in objects for which the classification test makes sense. If you happen to pass in an object that doesn't fit in any category, or is categorized wrongly, then that's a bug in your code and you should go fix it. So for example, when the documentation for 'sort' says that the 'key' argument can be a function that returns the key value, that doesn't mean that you can pass in an arbitrary function such as 'file.open' and expect it to magically work. So, the point of all this is that while callable() isn't at all useful for the kind of formal proof-that-it-will-not-blow-up-horribly kind of inspection, it is very useful as a rough category test. In fact that's pretty much all I have ever used it for, and I am sure that many others would agree. In fact, I would go so far as to say that the kind of "exact parameter signature" test being advocated would be *less* useful, simply because it would take more time to code to get the same effect. -- Talin From behnel_ml at gkec.informatik.tu-darmstadt.de Wed Jul 19 11:11:41 2006 From: behnel_ml at gkec.informatik.tu-darmstadt.de (Stefan Behnel) Date: Wed, 19 Jul 2006 11:11:41 +0200 Subject: [Python-3000] callable() In-Reply-To: <44BD8704.4020309@canterbury.ac.nz> References: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> <44BC87FD.8000808@canterbury.ac.nz> <44BD8704.4020309@canterbury.ac.nz> Message-ID: <44BDF74D.7040302@gkec.informatik.tu-darmstadt.de> Greg Ewing wrote: > Guido van Rossum wrote: > >> But I'm not proposing to use hasattr(obj, '__call__'). I'm basically >> saying that the callable test has very little to do with whether the >> later call will succeed, because we have no way to test the signature. > > I don't think that's needed for the sort of things people > want callable() for. I think they want it for the purpose > of implementing type-dependent APIs -- similar in spirit > to doing different things depending on whether you're > passed a sequence or not, etc. Exactly. > For that purpose, hasattr(obj, '__call__') is sufficient, But it's not the same: it's concerned with implementation details and unless you are used to reading it, it's not obvious that both are equivalent. It doesn't matter if it's the same. It just feels different to use it. As I said, if it's not wanted as a builtin, there's always the way to move it into a module. I proposed "types" for this, but I guess "inspect" is a better place: inspect.iscallable(obj) Then it's a matter of importing this function as "callable" to provide backwards compatibility (both in terms of code and readability). The same applies to "__iter__", BTW. What about adding these two also: inspect.isiterable(obj) -> hasattr(obj, "__iter__") inspect.isiterator(obj) -> isiterable(obj) and hasattr(obj, "next") Same reasoning: getting rid of implementation details by using abstracted, well-named interface tests. Stefan From guido at python.org Wed Jul 19 18:31:22 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 19 Jul 2006 09:31:22 -0700 Subject: [Python-3000] callable() In-Reply-To: <44BD8902.7030900@canterbury.ac.nz> References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> Message-ID: On 7/18/06, Greg Ewing wrote: > Andrew Koenig wrote: > > > I am uncomfortable about exposing the implementation this way, if only > > because it would require fixing the equivalence between callable() and > > hasattr(obj, '__call__') for all time. > > I don't see anything bad about fixing that equivalence. > I regard the fact that it *wasn't* fixed before as a > language design bug that Py3k will hopefully fix. I seem to recall fixing it. Are there still callable objects without a __call__ attribute? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ronaldoussoren at mac.com Wed Jul 19 19:13:48 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 19 Jul 2006 19:13:48 +0200 Subject: [Python-3000] callable() In-Reply-To: References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> Message-ID: <67055B69-D714-4C7C-8675-3C98C085B755@mac.com> On Jul 19, 2006, at 6:31 PM, Guido van Rossum wrote: > On 7/18/06, Greg Ewing wrote: >> Andrew Koenig wrote: >> >>> I am uncomfortable about exposing the implementation this way, if >>> only >>> because it would require fixing the equivalence between callable >>> () and >>> hasattr(obj, '__call__') for all time. >> >> I don't see anything bad about fixing that equivalence. >> I regard the fact that it *wasn't* fixed before as a >> language design bug that Py3k will hopefully fix. > > I seem to recall fixing it. Are there still callable objects without a > __call__ attribute? Classic classes? Python 2.5b2 (r25b2:50570, Jul 11 2006, 09:46:24) [GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> class O: pass ... >>> O.__call__ Traceback (most recent call last): File "", line 1, in AttributeError: class O has no attribute '__call__' >>> Ronald > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/ > ronaldoussoren%40mac.com From greg.ewing at canterbury.ac.nz Thu Jul 20 03:26:14 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 20 Jul 2006 13:26:14 +1200 Subject: [Python-3000] callable() In-Reply-To: <67055B69-D714-4C7C-8675-3C98C085B755@mac.com> References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <67055B69-D714-4C7C-8675-3C98C085B755@mac.com> Message-ID: <44BEDBB6.5020600@canterbury.ac.nz> Ronald Oussoren wrote: > Classic classes? I just checked, and it seems they've been fixed too: callable() and hasattr(obj, '__call_') give the same result -- true if and only if a __call__ method has been defined. Anyway, the point I was making is that this equivalence is a *good* thing -- I can't think of any reason one would want it to be different. -- Greg From ronaldoussoren at mac.com Thu Jul 20 07:28:31 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu, 20 Jul 2006 07:28:31 +0200 Subject: [Python-3000] callable() In-Reply-To: <44BEDBB6.5020600@canterbury.ac.nz> References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <67055B69-D714-4C7C-8675-3C98C085B755@mac.com> <44BEDBB6.5020600@canterbury.ac.nz> Message-ID: On Jul 20, 2006, at 3:26 AM, Greg Ewing wrote: > Ronald Oussoren wrote: > >> Classic classes? > > I just checked, and it seems they've been fixed too: > callable() and hasattr(obj, '__call_') give the same > result -- true if and only if a __call__ method has > been defined. But classic classes theirself are callable yet don't have a __call__ attribute. New-style classes do have a call method. To expand on my previous interpreter session: Python 2.5b2 (r25b2:50570, Jul 11 2006, 09:46:24) [GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> class O: pass # Old-style class ... >>> O.__call__ # No __call__ Traceback (most recent call last): File "", line 1, in AttributeError: class O has no attribute '__call__' >>> class N (object): pass # New-style class ... >>> N.__call__ # Has __call__ >>> >>> # Instantition by calling works for both: >>> O() <__main__.O instance at 0x701c0> >>> N() <__main__.N object at 0x6e410> >>> Ronald From talin at acm.org Thu Jul 20 10:08:26 2006 From: talin at acm.org (Talin) Date: Thu, 20 Jul 2006 01:08:26 -0700 Subject: [Python-3000] Parameter types and multiple method dispatch Message-ID: <44BF39FA.1070705@acm.org> I've been thinking about the proposed "parameter types" feature of Python 3000. And there are two aspects of the current proposal that I find somewhat troubling. Let me first review what the proposal is. The idea is that a new syntax would be added to allow specification of a "function signature" - essentially a way to annotate individual parameters and have those annotations be accessible via the function object. In addition, one of the key design points is that these annotations would have no fixed meaning, but would instead simply be objects that are interpreted in various contexts, including function decorators for which the annotations might have special meaning. I'm going to go out on a limb here and say that both of these concepts are less useful than one would think, and are likely to create a host of problems. I suspect that the proposed design will lead to difficulty of interpretation, and conflicts between different but overlapping goals within the design of a program. The first difficulty has to do with the way in which the arguments are represented, specifically the notion that each annotation is associated with a specific formal parameter. A python function declaration is a kind of mapping between input arguments and formal parameters. As we know from reading the Python language reference, this mapping is in many instances a non-trivial transformation. For example, if I have a function: def X( a=2, b=3, *c, **d ): ... and I call it with: X( b=100, f=9 ) We end with with values of 'a=2, b=100, c=[], and d={f:9}. However, those values can only be accessed from within the function body - there is no way for a decorator or any other external code to know what those values are. Even if the decorator 'wraps' the function, it only knows what actual parameters were passed - it can't *directly* know the value of a, b, c, and d, although it can attempt to *emulate* the mapping by reimplementing the parameter assignment algorithm. Now, normally decorators don't care about any of this - typically one writes a wrapper that simply has the same function signature as the method it wraps, and the code that calls the inner function is written by the programmer to reverse the transformation, so that the wrapped function can recieve the same arguments. That technique is only useful, however, if the decorator is custom-written for functions that have a specific signature. Truly generic wrappers typically don't examine the arguments at all, they simply pass *args and **kwargs without looking at them too closely. However, in order to use type annotations, one needs to associate between the actual arguments and the formal parameters - because the annotations are keyed by the formal parameter they are attached to. One way to do this is to simulate the mapping of arguments to formal parameters within the decorator. Once you know which formal parameter a given value is assigned to, you can then retrieve the annotation for that parameter and apply it to the value. However, there are a number of drawbacks to doing this - first, the mapping algorithm isn't really that simple in the general case. Worse, the assignment of arguments to formal parameters is a prime suspect in the relative 'slowness' of Python function calls compared to other language features - and in order for the decorator to figure out which argument goes with which formal parameter, the decorator is now forced to perform that same mapping *again*. Worse still if there is more than one decorator on the function, say adding multiple constraints, then each one has to perform this mapping step. A different possibility is to attempt to do the mapping in reverse - that is, for a given formal parameter (and its associated type annotation), attempt to determine which input argument corresponds to that parameter. The problem with this notion is that the parameter mapping algorithm is an iterative one - it would be difficult to come up with an algorithm that does Python parameter assignment *in reverse*, that was guaranteed to be correct for all possible edge cases. All of this is fairly trivial for the simple cases (i.e. a function with only positional arguments), but that assumes that the decorator has some foreknowledge of the kinds of functions that it will decorate. The problem is, you don't *need* type annotations for that kind of decorator - because if the decorator already has _a priori_ knowledge of the function, then usually it will know enough to know what the types of the arguments are supposed to be! Type annotations are more likely to be useful for truly generic decorators which will take action based on an inspection of the annotations. But the problem is that these are exactly the kind of decorators for which the parameter mapping problem is most acute. My second difficulty is with the notion that type annotations have no fixed meaning. The problem here is that, unlike decorators which are essentially functions, type annotations are more like data. A function interface with no assigned meaning (such as a plug-in) can be quite useful because functions carry their own 'meaning' with them, buried inside the function. Data, on the other hand, requires some external agency to interpret them in order to convey meaning. By saying that annotations have no fixed meaning, what we're really saying is that there's no "standard interpretation" of these annotations, and that's where IMHO the trouble lies. If there's no standard interpretation, then everyone is going to interpret them differently. A function with more than one independently-written decorator is going to have problems, each decorator trying to pick out the parts of the annotations that's meant for them and not the others. It seems to me that a better solution is to allow the *decorators* to have function signatures. For example: @overload(int,int) def add(a,b): ... The advantage of this is twofold: First, it means that each decorator can have its own separate signature, independent of any other. That means there's no longer any conflict between different decorators attempting to appply different interpretations to the same data. Second, it means that the mapping problem hasn't entirely gone away, but has somewhat been reduced, because now the type annotations are referring to the *wrapper function's* arguments, not the arguments of the wrapped function. This means that the wrapper is now on the 'inside', and can see all of the variables in their mapped configuration. Of course, there is still the problem of passing the arguments to the inner function - but that's no worse than the problem that decorators have today. The only drawback here is that the syntactical options for expressing type signatures as arguments to the decorators are somewhat limited. For example, '@overload(int,int,a=int)' doesn't express the notion that a is a keyword argument that should be constrained to an int value. This is more due to a general characteristic of Python, which is that syntactical and semantic phases of interpretation of the language are kept as discretely separate as possible. Specifically, Python lacks the means to pass syntactical constructs as arguments to functions. -- Talin From talin at acm.org Thu Jul 20 10:58:22 2006 From: talin at acm.org (Talin) Date: Thu, 20 Jul 2006 01:58:22 -0700 Subject: [Python-3000] Proposal: Metasyntax operator Message-ID: <44BF45AE.3060307@acm.org> A number of dynamic languages, such as Lisp, support the notion of an 'unevaluated' expression. In Lisp, a macro is simply a function that can operate on the *syntax* of the expression before it is actually compiled and interpreted. A number of Python libraries attempt to use operator overloading to achieve similar results. A good example is SQLObject's 'sqlbuilder' module, which allows the user to construct SQL statements using regular Python operators. Typically these libraries work by creating a set of standard constants (i.e. function names, symbols, etc.) and overloading all of the operators to produce an expression tree rather than actually evaluating the result. Thus, 'a + b', instead of producing the sum of a and b, will instead produce something along the lines of '('+', a, b)'. One weak area, however, is in the treatment of variables. This is because you can't recover the name of a variable at runtime - the variable name only exists in the compiler's imagination, and is generally long gone by the time the program is actually executed. There have been a number of attempts to get around this limitation. One approach is to define a limited set of "standard" variables (X, Y, Z, etc.) Unfortunately, this is quite limiting - having to pre-declare variables is hardly 'Pythonic'. The other is to wrap the variables in a class (such as "Variable('x')", which is cumbersome. Various tricks with __getattr__ can also be done, such as Variable.x and so on. All of these are examples of "quoted" variables - which is another way of saying that the variables are unevaluated, and that their syntactical rather than semantic attributes are available to the program. I'd like to propose a standard way to represent one of these syntactical variables. The syntax I would like to see is '?x' - i.e. a question mark followed by the variable name. The reason for choosing the question mark is that this is exactly how many languages - including expert systems and inference engines - represent a substitution variable. The other reason, which was only of minor consideration, is that the '?' is one of the few symbols in Python whose meaning is not already assigned. The actual meaning of ? would is very simple: '?x' is equivalent to '__quote__("x")' Where __quote__ is a user-defined symbol accessible from the current scope. There will also be a standard, importable implementation of __quote__ which overloads all operators to create a simple AST. It is likely that most code that does syntactic manipulation (such as sqlbuilder) could be modified to use the standard AST created by the built-in quote class. The reason for this is because the AST itself has no inherent meaning, its only meaningful to the function that you pass it so. So the SQL 'select()' function could walk through the AST transforming it into SQL syntax. The quote operator has one other effect, which is that unlike regular variables it can overload assignment: ?x = 3 Normally this would be an error, since ?x isn't an L-value. However the interpretr notes the presence of the ? and allows the __assign__ method to be called instead. A possible use for this would be to create syntactical descriptions of keyword arguments that can be inspected at runtime: @overload(int,int,?x=int) Now, one potential objection is what to do about conflicting definitions of __quote__. My response is that it is up to the user to make sure that __quote__ is defined to the correct class at the correct places in the code, even if this means re-assigning it from time to time: __quote__ = SQLVar expr = Select( User.name, Where( ?name = 'Fred' ) ) __quote__ = MathVar formula = ?a + ?b (I should note that I don't expect this proposal to get very far. However, its something I've been thinking about for a long time.) -- Talin From ncoghlan at gmail.com Thu Jul 20 12:52:59 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 20 Jul 2006 20:52:59 +1000 Subject: [Python-3000] callable() In-Reply-To: References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <67055B69-D714-4C7C-8675-3C98C085B755@mac.com> <44BEDBB6.5020600@canterbury.ac.nz> Message-ID: <44BF608B.6050800@gmail.com> Ronald Oussoren wrote: > On Jul 20, 2006, at 3:26 AM, Greg Ewing wrote: > >> Ronald Oussoren wrote: >> >>> Classic classes? >> I just checked, and it seems they've been fixed too: >> callable() and hasattr(obj, '__call_') give the same >> result -- true if and only if a __call__ method has >> been defined. > > But classic classes theirself are callable yet don't have a __call__ > attribute. Which really won't be a problem in Py3k :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From walter at livinglogic.de Thu Jul 20 13:32:12 2006 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu, 20 Jul 2006 13:32:12 +0200 Subject: [Python-3000] callable() In-Reply-To: References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> Message-ID: <44BF69BC.9010801@livinglogic.de> Guido van Rossum wrote: > On 7/18/06, Greg Ewing wrote: >> Andrew Koenig wrote: >> >>> I am uncomfortable about exposing the implementation this way, if only >>> because it would require fixing the equivalence between callable() and >>> hasattr(obj, '__call__') for all time. >> I don't see anything bad about fixing that equivalence. >> I regard the fact that it *wasn't* fixed before as a >> language design bug that Py3k will hopefully fix. > > I seem to recall fixing it. Are there still callable objects without a > __call__ attribute? I don't know about __call__, but str and unicode don't have __iter__, list, tuple and dict do: Python 2.5b2 (r25b2:50512, Jul 20 2006, 13:27:27) [GCC 3.3.5 (Debian 1:3.3.5-13)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> list.__iter__ >>> tuple.__iter__ >>> dict.__iter__ >>> str.__iter__ Traceback (most recent call last): File "", line 1, in AttributeError: type object 'str' has no attribute '__iter__' >>> unicode.__iter__ Traceback (most recent call last): File "", line 1, in AttributeError: type object 'unicode' has no attribute '__iter__' Should that be fixed too? Servus, Walter From guido at python.org Thu Jul 20 16:24:03 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 20 Jul 2006 07:24:03 -0700 Subject: [Python-3000] callable() In-Reply-To: <44BF69BC.9010801@livinglogic.de> References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <44BF69BC.9010801@livinglogic.de> Message-ID: On 7/20/06, Walter D?rwald wrote: > str and unicode don't have __iter__, list, tuple and dict do: [...] > Should that be fixed too? Yes, please. (In Python 2.6 if you can.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Thu Jul 20 18:10:07 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 20 Jul 2006 12:10:07 -0400 Subject: [Python-3000] Proposal: Metasyntax operator In-Reply-To: Message-ID: <5.1.1.6.0.20060720115715.039ac008@sparrow.telecommunity.com> At 01:58 AM 7/20/2006 -0700, Talin wrote: >I'd like to propose a standard way to represent one of these syntactical >variables. The syntax I would like to see is '?x' - i.e. a question mark >followed by the variable name. > >The reason for choosing the question mark is that this is exactly how >many languages - including expert systems and inference engines - >represent a substitution variable. The other reason, which was only of >minor consideration, is that the '?' is one of the few symbols in Python >whose meaning is not already assigned. > >The actual meaning of ? would is very simple: > > '?x' is equivalent to '__quote__("x")' > >Where __quote__ is a user-defined symbol accessible from the current scope. I'd like to just point out that having a syntax for AST literals, i.e., simply having something like `foo+bar*23` result in an AST, gives you a better result in several ways: 1. No ugly symbols next to every name 2. "and", "or", "if", "for", "yield", and even lambda expressions can be used without further chicanery 3. The AST can be constructed at compilation time If we were going to have ?x mean something special, I'd want it to instead be a *reverse* syntactic operation. For example, if backquotes denoted an AST literal (and I know Guido's not going to allow that, but it's convenient for an example), then ? would be an escape to reference a *non-AST* expression to be inserted. E.g.:: `x + ?y` Would mean "take the runtime value of Y and insert it as a child node of the AST at the position shown". Of course, for this to work you'd have to come up with a better AST literal syntax than backquotes. About the only thing that comes to mind at the moment is using the backslash, followed by an atom. So "\x" would be an AST literal for the symbol "x", and "\(x+y)" would be an AST literal for x+y. I seem to recall that at one time Guido suggested that this type of thing should just use strings, however, and I don't know if that's still his position. From pje at telecommunity.com Thu Jul 20 17:52:15 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 20 Jul 2006 11:52:15 -0400 Subject: [Python-3000] Parameter types and multiple method dispatch In-Reply-To: Message-ID: <5.1.1.6.0.20060720114401.02622318@sparrow.telecommunity.com> At 01:08 AM 7/20/2006 -0700, Talin wrote: >In addition, one of the key design points is that these annotations >would have no fixed meaning, but would instead simply be objects that >are interpreted in various contexts, including function decorators for >which the annotations might have special meaning. Actually, you've left out an important bit: the __typecheck__ function. Guido's original proposal called for __typecheck__(arg, type) to be called on each argument that has a type declaration, with the result replacing the argument value. The builtin __typecheck__ function would be a no-op, but you could change its meaning within a module. Given this definition for the semantics, none of the issues you raised are relevant. >I'm going to go out on a limb here and say that both of these concepts >are less useful than one would think, and are likely to create a host of >problems. I suspect that the proposed design will lead to difficulty of >interpretation, and conflicts between different but overlapping goals >within the design of a program. I think you're worrying too much. I've written decorators that create signature-specific wrapper functions by exec'ing a string to create the wrapping function. It's a bit tricky, but you might be able to create a helper library to make it easier. There are some annoyances, too, like not being able to have the line numbers make sense in the decorator itself. But, if Guido's __typecheck__ idea is in effect, none of that would be necessary in order to support type checking. Also, for overloaded methods based on argument types, this isn't really a problem either. To do fast dispatching you need to generate code anyway -- which as it turns out is *why* I've written decorators that create signature-specific wrappers. :) That is, the need to generate code exists even without having argument type declarations. Ergo, argument type declarations don't create any new problems; I was already generating custom wrappers to deal with other esoteric aspects of function signatures (like nested argument tuples, *args, and **kw). From rhettinger at ewtllc.com Thu Jul 20 18:38:06 2006 From: rhettinger at ewtllc.com (Raymond Hettinger) Date: Thu, 20 Jul 2006 09:38:06 -0700 Subject: [Python-3000] callable() In-Reply-To: References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <44BF69BC.9010801@livinglogic.de> Message-ID: <44BFB16E.8080303@ewtllc.com> Guido van Rossum wrote: >On 7/20/06, Walter D?rwald wrote: > > >>str and unicode don't have __iter__, list, tuple and dict do: >> >> >[...] > > >>Should that be fixed too? >> >> > >Yes, please. (In Python 2.6 if you can.) > > > Why is the a defect? Have we abandoned the notion of SeqIter automatically wrapping any object with __getitem__()? IIRC, list and tuple have __iter__ simply as optimizations, it did not change their functionality. Raymond ----------------- >>> class X: def __getitem__(self, i): if i < 5: return i*5 raise IndexError >>> for i in X(): print i 0 5 10 15 20 From collinw at gmail.com Thu Jul 20 20:22:51 2006 From: collinw at gmail.com (Collin Winter) Date: Thu, 20 Jul 2006 14:22:51 -0400 Subject: [Python-3000] Parameter types and multiple method dispatch In-Reply-To: <44BF39FA.1070705@acm.org> References: <44BF39FA.1070705@acm.org> Message-ID: <43aa6ff70607201122m719795d2k2e8c082e4780d251@mail.gmail.com> On 7/20/06, Talin wrote: > However, in order to use type annotations, one needs to associate > between the actual arguments and the formal parameters - because the > annotations are keyed by the formal parameter they are attached to. One > way to do this is to simulate the mapping of arguments to formal > parameters within the decorator. Once you know which formal parameter a > given value is assigned to, you can then retrieve the annotation for > that parameter and apply it to the value. > > However, there are a number of drawbacks to doing this - first, the > mapping algorithm isn't really that simple in the general case. Worse, > the assignment of arguments to formal parameters is a prime suspect in > the relative 'slowness' of Python function calls compared to other > language features - and in order for the decorator to figure out which > argument goes with which formal parameter, the decorator is now forced > to perform that same mapping *again*. [snip] > A different possibility is to attempt to do the mapping in reverse - > that is, for a given formal parameter (and its associated type > annotation), attempt to determine which input argument corresponds to > that parameter. > > The problem with this notion is that the parameter mapping algorithm is > an iterative one - it would be difficult to come up with an algorithm > that does Python parameter assignment *in reverse*, that was guaranteed > to be correct for all possible edge cases. I've implemented both solutions before, and neither proved to be all that difficult (after actually reading the manual, that is ; ) Also, I'm not sure what you're considering to be 'the general case', but I remember from my own adventures in this area that the most frequent function types were "def foo(a, b, c)"-style functions, ie, functions with only positional arguments. In this case, the mapping becomes dead simple. > By saying that annotations have no fixed meaning, what we're really > saying is that there's no "standard interpretation" of these > annotations, and that's where IMHO the trouble lies. If there's no > standard interpretation, then everyone is going to interpret them > differently. A function with more than one independently-written > decorator is going to have problems, each decorator trying to pick out > the parts of the annotations that's meant for them and not the others. Guido and I went back and forth on this for a while, whether or not to include a "standard interpretation" with Python itself. We're counting on the shear complexity of implementing an annotation-interpreting library to keep the number of these libraries low and in the hands of people who know what they're doing. > It seems to me that a better solution is to allow the *decorators* to > have function signatures. For example: > > @overload(int,int) > def add(a,b): > ... > > The advantage of this is twofold: [snip] > Second, it means that the mapping problem hasn't entirely gone away, but > has somewhat been reduced, because now the type annotations are > referring to the *wrapper function's* arguments, not the arguments of > the wrapped function. This means that the wrapper is now on the > 'inside', and can see all of the variables in their mapped configuration. Huh? Unless you have a raft of decorators for every possible permutation of function parameters, you've probably written that overload() decorator to have a signature of "def overload(*vargs)" or "def overload(*vargs, **kwargs)", in which case you still have to do the argument -> formal parameter mapping. > Of course, there is still the problem of passing the arguments to the > inner function - but that's no worse than the problem that decorators > have today. I don't understand; what kind of decorators do you write that have this problem? > For example, '@overload(int,int,a=int)' doesn't express the notion that a is > a keyword argument that should be constrained to an int value. It certainly can express that if you want it to. Just use **kwargs in the decorator to catch all keyword arguments and then treat them differently. Collin Winter From guido at python.org Thu Jul 20 21:58:57 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 20 Jul 2006 12:58:57 -0700 Subject: [Python-3000] callable() In-Reply-To: <44BFB16E.8080303@ewtllc.com> References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <44BF69BC.9010801@livinglogic.de> <44BFB16E.8080303@ewtllc.com> Message-ID: On 7/20/06, Raymond Hettinger wrote: > Guido van Rossum wrote: > >On 7/20/06, Walter D?rwald wrote: > >>str and unicode don't have __iter__, list, tuple and dict do: > >>Should that be fixed too? > >Yes, please. (In Python 2.6 if you can.) > Why is the a defect? Have we abandoned the notion of SeqIter > automatically wrapping any object with __getitem__()? Actually, the autowrapping was intended a backwards compatibility measure. We should make a conscious decision whether we should make it a permanent feature or not. (I also think that the two-argument form iter(function, sentinel) is not very successful or useful and might be dropped, but that's a separate issue.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rhettinger at ewtllc.com Thu Jul 20 22:53:58 2006 From: rhettinger at ewtllc.com (Raymond Hettinger) Date: Thu, 20 Jul 2006 13:53:58 -0700 Subject: [Python-3000] callable() In-Reply-To: References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <44BF69BC.9010801@livinglogic.de> <44BFB16E.8080303@ewtllc.com> Message-ID: <44BFED66.3050401@ewtllc.com> >> Why is this a defect? Have we abandoned the notion of SeqIter >> automatically wrapping any object with __getitem__()? > > > Actually, the autowrapping was intended a backwards compatibility > measure. > > We should make a conscious decision whether we should make it a > permanent feature or not. It sure simplified writing extensions. And it provided a convenient guarantee that all sequences are iterable. I don't see a downside to keeping it. > (I also think that the two-argument form > iter(function, sentinel) is not very successful or useful and might be > dropped, but that's a separate issue.) This functionality should be moved to itertools. That will clear-up the odd function signature for iter(). As it stands now, the function/sentinel form suffers from invisibility. Raymond From rrr at ronadam.com Fri Jul 21 00:07:19 2006 From: rrr at ronadam.com (Ron Adam) Date: Thu, 20 Jul 2006 17:07:19 -0500 Subject: [Python-3000] set literals In-Reply-To: <44B2EAEE.8060807@ewtllc.com> References: <1d85506f0607071434o5ade0972m7cbd3c17aa65de36@mail.gmail.com> <001b01c6a293$c255c150$6402a8c0@arkdesktop> <44B2BEA2.3010207@ewtllc.com> <44B2EAEE.8060807@ewtllc.com> Message-ID: Raymond Hettinger wrote: > I'm curious as to whether people will find one-type-with-two-purposes > easier to learn that what we have now. My experience so far is that > sets have a near zero learning curve as they are currently implemented. And it's not just about ease of learning to use, but also about ease of understanding expressed code once it's written. Having sets as a separate data type makes reading code easier because I don't have to first determine the context to determine how it's being used. Ron From guido at python.org Fri Jul 21 02:32:20 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 20 Jul 2006 17:32:20 -0700 Subject: [Python-3000] callable() In-Reply-To: <44BFED66.3050401@ewtllc.com> References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <44BF69BC.9010801@livinglogic.de> <44BFB16E.8080303@ewtllc.com> <44BFED66.3050401@ewtllc.com> Message-ID: On 7/20/06, Raymond Hettinger wrote: > > >> Why is this a defect? Have we abandoned the notion of SeqIter > >> automatically wrapping any object with __getitem__()? > > > > > > Actually, the autowrapping was intended a backwards compatibility > > measure. > > > > We should make a conscious decision whether we should make it a > > permanent feature or not. > > It sure simplified writing extensions. > And it provided a convenient guarantee that all sequences are iterable. > I don't see a downside to keeping it. Well, it will also make mappings pseudo-iterable; in Py3k I plan to completely get rid of the distinction between x[...] for mappings and for sequences, which exist in C but not at the Python level. I'd like to hear from others whether the default iter fallback ought to stay or go. > > (I also think that the two-argument form > > iter(function, sentinel) is not very successful or useful and might be > > dropped, but that's a separate issue.) > > This functionality should be moved to itertools. > That will clear-up the odd function signature for iter(). > As it stands now, the function/sentinel form suffers from invisibility. That doesn't matter much since there are very few uses for it. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jul 21 02:34:06 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 20 Jul 2006 17:34:06 -0700 Subject: [Python-3000] Proposal: Metasyntax operator In-Reply-To: <44BF45AE.3060307@acm.org> References: <44BF45AE.3060307@acm.org> Message-ID: I'd like to see this fall under the blanket "Python will not have programmable syntax" rule in PEO 3099. On 7/20/06, Talin wrote: > A number of dynamic languages, such as Lisp, support the notion of an > 'unevaluated' expression. In Lisp, a macro is simply a function that can > operate on the *syntax* of the expression before it is actually compiled > and interpreted. > > A number of Python libraries attempt to use operator overloading to > achieve similar results. A good example is SQLObject's 'sqlbuilder' > module, which allows the user to construct SQL statements using regular > Python operators. > > Typically these libraries work by creating a set of standard constants > (i.e. function names, symbols, etc.) and overloading all of the > operators to produce an expression tree rather than actually evaluating > the result. Thus, 'a + b', instead of producing the sum of a and b, will > instead produce something along the lines of '('+', a, b)'. > > One weak area, however, is in the treatment of variables. This is > because you can't recover the name of a variable at runtime - the > variable name only exists in the compiler's imagination, and is > generally long gone by the time the program is actually executed. > > There have been a number of attempts to get around this limitation. One > approach is to define a limited set of "standard" variables (X, Y, Z, > etc.) Unfortunately, this is quite limiting - having to pre-declare > variables is hardly 'Pythonic'. > > The other is to wrap the variables in a class (such as "Variable('x')", > which is cumbersome. Various tricks with __getattr__ can also be done, > such as Variable.x and so on. > > All of these are examples of "quoted" variables - which is another way > of saying that the variables are unevaluated, and that their syntactical > rather than semantic attributes are available to the program. > > I'd like to propose a standard way to represent one of these syntactical > variables. The syntax I would like to see is '?x' - i.e. a question mark > followed by the variable name. > > The reason for choosing the question mark is that this is exactly how > many languages - including expert systems and inference engines - > represent a substitution variable. The other reason, which was only of > minor consideration, is that the '?' is one of the few symbols in Python > whose meaning is not already assigned. > > The actual meaning of ? would is very simple: > > '?x' is equivalent to '__quote__("x")' > > Where __quote__ is a user-defined symbol accessible from the current scope. > > There will also be a standard, importable implementation of __quote__ > which overloads all operators to create a simple AST. It is likely that > most code that does syntactic manipulation (such as sqlbuilder) could be > modified to use the standard AST created by the built-in quote class. > The reason for this is because the AST itself has no inherent meaning, > its only meaningful to the function that you pass it so. So the SQL > 'select()' function could walk through the AST transforming it into SQL > syntax. > > The quote operator has one other effect, which is that unlike regular > variables it can overload assignment: > > ?x = 3 > > Normally this would be an error, since ?x isn't an L-value. However the > interpretr notes the presence of the ? and allows the __assign__ method > to be called instead. A possible use for this would be to create > syntactical descriptions of keyword arguments that can be inspected at > runtime: > > @overload(int,int,?x=int) > > Now, one potential objection is what to do about conflicting definitions > of __quote__. My response is that it is up to the user to make sure that > __quote__ is defined to the correct class at the correct places in the > code, even if this means re-assigning it from time to time: > > __quote__ = SQLVar > expr = Select( User.name, Where( ?name = 'Fred' ) ) > __quote__ = MathVar > formula = ?a + ?b > > (I should note that I don't expect this proposal to get very far. > However, its something I've been thinking about for a long time.) > > -- Talin > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jcarlson at uci.edu Fri Jul 21 03:02:59 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu, 20 Jul 2006 18:02:59 -0700 Subject: [Python-3000] Proposal: Metasyntax operator In-Reply-To: References: <44BF45AE.3060307@acm.org> Message-ID: <20060720180015.4CCA.JCARLSON@uci.edu> "Guido van Rossum" wrote: > I'd like to see this fall under the blanket "Python will not have > programmable syntax" rule in PEO 3099. Would this also mean that you don't want people to be manipulating AST objects a'la Boo macros? - Josiah From jcarlson at uci.edu Fri Jul 21 03:06:52 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu, 20 Jul 2006 18:06:52 -0700 Subject: [Python-3000] Proposal: Metasyntax operator In-Reply-To: <20060720180015.4CCA.JCARLSON@uci.edu> References: <20060720180015.4CCA.JCARLSON@uci.edu> Message-ID: <20060720180622.4CD0.JCARLSON@uci.edu> Josiah Carlson wrote: > "Guido van Rossum" wrote: > > I'd like to see this fall under the blanket "Python will not have > > programmable syntax" rule in PEO 3099. > > Would this also mean that you don't want people to be manipulating AST > objects a'la Boo macros? Nevermind, obvious no in reading your post. Move along people, nothing to see here. - Josiah From greg.ewing at canterbury.ac.nz Fri Jul 21 03:22:42 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 21 Jul 2006 13:22:42 +1200 Subject: [Python-3000] callable() In-Reply-To: <44BF69BC.9010801@livinglogic.de> References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <44BF69BC.9010801@livinglogic.de> Message-ID: <44C02C62.4020609@canterbury.ac.nz> Walter D?rwald wrote: > I don't know about __call__, but str and unicode don't have __iter__, > list, tuple and dict do: That's probably because str and unicode don't do their own iteration, but rely on the fallback implementation. In which case it's perfectly correct for them not to have an __iter__ method. If you want to test whether something is iterable, it's not enough to test for __iter__ -- you also need to test for __len__ and __getitem__ as an alternative. Which means it might make sense to have an iterable() function which does that. -- Greg From brett at python.org Fri Jul 21 03:31:14 2006 From: brett at python.org (Brett Cannon) Date: Thu, 20 Jul 2006 18:31:14 -0700 Subject: [Python-3000] callable() In-Reply-To: <44C02C62.4020609@canterbury.ac.nz> References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <44BF69BC.9010801@livinglogic.de> <44C02C62.4020609@canterbury.ac.nz> Message-ID: On 7/20/06, Greg Ewing wrote: > > Walter D?rwald wrote: > > > I don't know about __call__, but str and unicode don't have __iter__, > > list, tuple and dict do: > > That's probably because str and unicode don't do their > own iteration, but rely on the fallback implementation. > In which case it's perfectly correct for them not to > have an __iter__ method. > > If you want to test whether something is iterable, it's > not enough to test for __iter__ -- you also need to > test for __len__ and __getitem__ as an alternative. > Which means it might make sense to have an iterable() > function which does that. > > -- > Greg > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20060720/f33858ac/attachment.html From greg.ewing at canterbury.ac.nz Fri Jul 21 03:32:50 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 21 Jul 2006 13:32:50 +1200 Subject: [Python-3000] Parameter types and multiple method dispatch In-Reply-To: <5.1.1.6.0.20060720114401.02622318@sparrow.telecommunity.com> References: <5.1.1.6.0.20060720114401.02622318@sparrow.telecommunity.com> Message-ID: <44C02EC2.40304@canterbury.ac.nz> Phillip J. Eby wrote: > Guido's original proposal called for __typecheck__(arg, type) to > be called on each argument that has a type declaration, with the result > replacing the argument value. > > Given this definition for the semantics, none of the issues you raised are > relevant. I think some of it is still relevant, such as the concern about different parts of the same program fighting over the interpretation of the type annotations. I'm rather leery about the whole thing myself because of that. The idea of adding something in such a fundamental and intrusive way to the language when we don't even have a clear idea of what we're going to use it for smells very bad to me. I get the feeling that if anyone other than Guido suggested doing any such thing, he would tell them in no uncertain terms to go away and come back when they had a complete and concrete proposal... -- Greg From brett at python.org Fri Jul 21 03:33:03 2006 From: brett at python.org (Brett Cannon) Date: Thu, 20 Jul 2006 18:33:03 -0700 Subject: [Python-3000] callable() In-Reply-To: References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <44BF69BC.9010801@livinglogic.de> <44BFB16E.8080303@ewtllc.com> <44BFED66.3050401@ewtllc.com> Message-ID: On 7/20/06, Guido van Rossum wrote: > > On 7/20/06, Raymond Hettinger wrote: > > > > >> Why is this a defect? Have we abandoned the notion of SeqIter > > >> automatically wrapping any object with __getitem__()? > > > > > > > > > Actually, the autowrapping was intended a backwards compatibility > > > measure. > > > > > > We should make a conscious decision whether we should make it a > > > permanent feature or not. > > > > It sure simplified writing extensions. > > And it provided a convenient guarantee that all sequences are iterable. > > I don't see a downside to keeping it. > > Well, it will also make mappings pseudo-iterable; in Py3k I plan to > completely get rid of the distinction between x[...] for mappings and > for sequences, which exist in C but not at the Python level. Woohoo! I'd like to hear from others whether the default iter fallback ought > to stay or go. Eh, I have never had any direct need for it. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20060720/923378bb/attachment.htm From guido at python.org Fri Jul 21 03:48:16 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 20 Jul 2006 18:48:16 -0700 Subject: [Python-3000] callable() In-Reply-To: <44C02C62.4020609@canterbury.ac.nz> References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <44BF69BC.9010801@livinglogic.de> <44C02C62.4020609@canterbury.ac.nz> Message-ID: On 7/20/06, Greg Ewing wrote: > Walter D?rwald wrote: > > > I don't know about __call__, but str and unicode don't have __iter__, > > list, tuple and dict do: > > That's probably because str and unicode don't do their > own iteration, but rely on the fallback implementation. > In which case it's perfectly correct for them not to > have an __iter__ method. > > If you want to test whether something is iterable, it's > not enough to test for __iter__ -- you also need to > test for __len__ and __getitem__ as an alternative. > Which means it might make sense to have an iterable() > function which does that. And that's a problem, because if uses those to implement a mapping, it isn't reasonably iterable with just those. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Fri Jul 21 03:48:31 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 21 Jul 2006 13:48:31 +1200 Subject: [Python-3000] callable() In-Reply-To: References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <44BF69BC.9010801@livinglogic.de> <44BFB16E.8080303@ewtllc.com> Message-ID: <44C0326F.1070504@canterbury.ac.nz> Guido van Rossum wrote: > Actually, the autowrapping was intended a backwards compatibility measure. But it seems like a perfectly good and useful feature in its own right to me. Why force every sequence to implement its own __iter__ if there is a default one that does the same as what your custom one would have done anyway? An alternative would be to give object an __iter__ method containing the default implementation. But then you've lost all ability to test for potential iterability, since everything has an __iter__ whether it works or not. So +1 from me on keeping the status quo wrt __iter__ in Py3k. -- Greg From ncoghlan at gmail.com Fri Jul 21 11:33:22 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 21 Jul 2006 19:33:22 +1000 Subject: [Python-3000] callable() In-Reply-To: <44C0326F.1070504@canterbury.ac.nz> References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <44BF69BC.9010801@livinglogic.de> <44BFB16E.8080303@ewtllc.com> <44C0326F.1070504@canterbury.ac.nz> Message-ID: <44C09F62.2030103@gmail.com> Greg Ewing wrote: > Guido van Rossum wrote: > >> Actually, the autowrapping was intended a backwards compatibility measure. > > But it seems like a perfectly good and useful feature > in its own right to me. Why force every sequence to > implement its own __iter__ if there is a default one > that does the same as what your custom one would have > done anyway? The standard sequence iterator should still be somewhere in the standard library (e.g. as itertools.seqiter). This could then be used by assigning it as the __iter__ method in a class definition, or to a tp_iter slot through the C API. The fallback shouldn't be in iter() itself because falling back on the sequence iterator is a bug when __getitem__ and __len__ are used to implement a mapping. Alternatively, if a mechanism is introduced to allow a class to explicitly flag itself as "I'm a sequence", then iter() could be changed to only use the sequence iterator if that marker was set. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From walter at livinglogic.de Fri Jul 21 11:40:31 2006 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Fri, 21 Jul 2006 11:40:31 +0200 Subject: [Python-3000] callable() In-Reply-To: References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <44BF69BC.9010801@livinglogic.de> Message-ID: <44C0A10F.30201@livinglogic.de> Guido van Rossum wrote: > On 7/20/06, Walter D?rwald wrote: >> str and unicode don't have __iter__, list, tuple and dict do: > [...] >> Should that be fixed too? > > Yes, please. (In Python 2.6 if you can.) Here's the patch: bugs.python.org/1526367 Servus, Walter From thomas at python.org Fri Jul 21 14:04:33 2006 From: thomas at python.org (Thomas Wouters) Date: Fri, 21 Jul 2006 14:04:33 +0200 Subject: [Python-3000] callable() In-Reply-To: References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <44BF69BC.9010801@livinglogic.de> <44BFB16E.8080303@ewtllc.com> <44BFED66.3050401@ewtllc.com> Message-ID: <9e804ac0607210504o3299ed3dna11b233ae520e34e@mail.gmail.com> On 7/21/06, Guido van Rossum wrote: > I'd like to hear from others whether the default iter fallback ought > to stay or go. It should go (in py3k, obviously not in 2.6.) Maybe there should be a convenient way to spell 'iter me like a sequence', but it should definately be explicit, not implicit. > > (I also think that the two-argument form > > > iter(function, sentinel) is not very successful or useful and might be > > > dropped, but that's a separate issue.) > > > > This functionality should be moved to itertools. > > That will clear-up the odd function signature for iter(). > > As it stands now, the function/sentinel form suffers from invisibility. > > That doesn't matter much since there are very few uses for it. I disagree, I use it reasonably often. Much more often than I'd use, say, the 'with' statement or the ifelse operator, even more often than I use classmethods (which I like, and use wherever appropriate.) I agree that the spelling is somewhat unfortunate, much like the different uses of type(), but I do consider it less of a wart than type(). I'd be -0 on moving it to itertools in py3k. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20060721/7f277cc0/attachment.htm From rhettinger at ewtllc.com Fri Jul 21 18:06:36 2006 From: rhettinger at ewtllc.com (Raymond Hettinger) Date: Fri, 21 Jul 2006 09:06:36 -0700 Subject: [Python-3000] callable() In-Reply-To: References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <44BF69BC.9010801@livinglogic.de> <44BFB16E.8080303@ewtllc.com> <44BFED66.3050401@ewtllc.com> Message-ID: <44C0FB8C.4050205@ewtllc.com> >> >> > We should make a conscious decision whether we should make it a >> > permanent feature or not. >> >> It sure simplified writing extensions. >> And it provided a convenient guarantee that all sequences are iterable. >> I don't see a downside to keeping it. > > > Well, it will also make mappings pseudo-iterable; in Py3k I plan to > completely get rid of the distinction between x[...] for mappings and > for sequences, which exist in C but not at the Python level. > Since builtin dicts and dict subclasses will already be iterable, the mappings in question would be from user-defined classes with lookup behavior and but not providing an iterator: class C: '''This could be either a mapping or sequence so it would be foolish to wrap it with a SeqIter''' def __getitem__(self, k): return f(k) def __len__(self): return L So, by default, iter(C()) will now raise a TypeError and provide earlier detection of an error rather than waiting for a potential KeyError during iteration. And even if C was intended to be a sequence, list(C()) would raise a TypeError rather than risk a KeyError if C turned-out to be a mapping. Since all of the builtin sequences will already define __iter__, they are unaffected by this change. What is left are user-defined sequence classes or extensions with __getitem__ but not __iter__. To make those iterable, the user would need to either modify the class (if possible) or provide a SeqIte- like wrapper: def SeqIter(s): try: for i in itertools.count(): yield s[i] except IndexError: return for elem in SeqIter(s): ... # be explicit about looping over a generic sequence L = list(SeqIter(s)) # explicit wrapper to avoid a TypeError for generic sequences Does that correctly recap your proposal, its motivitations, and its implications? Raymond From rasky at develer.com Fri Jul 21 19:07:55 2006 From: rasky at develer.com (Giovanni Bajo) Date: Fri, 21 Jul 2006 19:07:55 +0200 Subject: [Python-3000] Parameter types and multiple method dispatch References: <5.1.1.6.0.20060720114401.02622318@sparrow.telecommunity.com> <44C02EC2.40304@canterbury.ac.nz> Message-ID: <06af01c6ace8$35aae460$d503030a@trilan> Greg Ewing wrote: >> Guido's original proposal called for __typecheck__(arg, type) to >> be called on each argument that has a type declaration, with the >> result replacing the argument value. >> >> Given this definition for the semantics, none of the issues you >> raised are relevant. > > I think some of it is still relevant, such as the concern > about different parts of the same program fighting over > the interpretation of the type annotations. > > I'm rather leery about the whole thing myself because of > that. The idea of adding something in such a fundamental > and intrusive way to the language when we don't even > have a clear idea of what we're going to use it for > smells very bad to me. Another thing I totally don't understand is how type annotations without any semantic assigned to them can be used to help speeding up Python and its compiler/interpreter. I thought (and hoped) that was one of the main scopes of type annotations. -- Giovanni Bajo From rhettinger at ewtllc.com Fri Jul 21 19:10:15 2006 From: rhettinger at ewtllc.com (Raymond Hettinger) Date: Fri, 21 Jul 2006 10:10:15 -0700 Subject: [Python-3000] callable() In-Reply-To: <44C0A10F.30201@livinglogic.de> References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <44BF69BC.9010801@livinglogic.de> <44C0A10F.30201@livinglogic.de> Message-ID: <44C10A77.4030906@ewtllc.com> >>>str and unicode don't have __iter__, list, tuple and dict do: >>> >>> >>[...] >> >> >>>Should that be fixed too? >>> >>> >>Yes, please. (In Python 2.6 if you can.) >> >> If implicit SeqIter wrapping won't potentially go away until Py3.0, why would this go into Py2.6? What is the benefit? Also, IIRC, there was some move afoot to make strings not iterable at all and instead add a .chars() method (though it would be a bit weird for an object to be slicable and indexable but not iterable). Also, I'm unclear on the goal here. Are you trying to make it possible to write an isiterable() that returns True whenever __iter__ is present? That would be at odds with the approach taken with list.__hash__ which has to be called in order to find-out it is not hashable. Raymond From guido at python.org Fri Jul 21 19:28:29 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 21 Jul 2006 10:28:29 -0700 Subject: [Python-3000] callable() In-Reply-To: <44C0FB8C.4050205@ewtllc.com> References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <44BF69BC.9010801@livinglogic.de> <44BFB16E.8080303@ewtllc.com> <44BFED66.3050401@ewtllc.com> <44C0FB8C.4050205@ewtllc.com> Message-ID: On 7/21/06, Raymond Hettinger wrote: > Since builtin dicts and dict subclasses will already be iterable, the > mappings in question would be from user-defined classes with lookup > behavior and but not providing an iterator: > > class C: > '''This could be either a mapping or sequence > so it would be foolish to wrap it with a SeqIter''' > def __getitem__(self, k): > return f(k) > def __len__(self): > return L > > So, by default, iter(C()) will now raise a TypeError and provide earlier > detection of an error rather than waiting for a potential KeyError > during iteration. And even if C was intended to be a sequence, > list(C()) would raise a TypeError rather than risk a KeyError if C > turned-out to be a mapping. > > Since all of the builtin sequences will already define __iter__, they > are unaffected by this change. What is left are user-defined sequence > classes or extensions with __getitem__ but not __iter__. To make those > iterable, the user would need to either modify the class (if possible) > or provide a SeqIte- like wrapper: > > def SeqIter(s): > try: > for i in itertools.count(): > yield s[i] > except IndexError: > return > > for elem in SeqIter(s): ... # be explicit about looping over a > generic sequence > > L = list(SeqIter(s)) # explicit wrapper to avoid a > TypeError for generic sequences > > > Does that correctly recap your proposal, its motivitations, and its > implications? Yes. Though I would strongly recommend that all user-defined sequences should provide an explicit __iter__(). It's only a few lines (and of course often can be done faster using knowledge of the implementation): def __iter__(self): for i in itertools.count(): yield self[i] -- --Guido van Rossum (home page: http://www.python.org/~guido/) From behnel_ml at gkec.informatik.tu-darmstadt.de Fri Jul 21 22:17:10 2006 From: behnel_ml at gkec.informatik.tu-darmstadt.de (Stefan Behnel) Date: Fri, 21 Jul 2006 22:17:10 +0200 Subject: [Python-3000] callable() In-Reply-To: References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <44BF69BC.9010801@livinglogic.de> <44BFB16E.8080303@ewtllc.com> <44BFED66.3050401@ewtllc.com> <44C0FB8C.4050205@ewtllc.com> Message-ID: <44C13646.8030607@gkec.informatik.tu-darmstadt.de> Guido van Rossum wrote: > def __iter__(self): > for i in itertools.count(): > yield self[i] Note that this likely raises IndexError, not StopIteration. Stefan From mcherm at mcherm.com Fri Jul 21 23:40:58 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Fri, 21 Jul 2006 14:40:58 -0700 Subject: [Python-3000] callable() Message-ID: <20060721144058.ltxjqxr9yekgck8o@login.werra.lunarpages.com> Guido writes: > I'd like to hear from others whether the default iter fallback ought > to stay or go. Go, I think. It's not that hard to supply __iter__(), and the simplicity and explicitness feel Pythonic. -- Michael Chermside From greg.ewing at canterbury.ac.nz Sat Jul 22 10:32:34 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 22 Jul 2006 20:32:34 +1200 Subject: [Python-3000] callable() In-Reply-To: <44C10A77.4030906@ewtllc.com> References: <000701c6aa97$9fb43990$6402a8c0@arkdesktop> <44BD8902.7030900@canterbury.ac.nz> <44BF69BC.9010801@livinglogic.de> <44C0A10F.30201@livinglogic.de> <44C10A77.4030906@ewtllc.com> Message-ID: <44C1E2A2.9030603@canterbury.ac.nz> Raymond Hettinger wrote: > That would be at odds with the approach taken with > list.__hash__ which has to be called in order to find-out it is not > hashable. That feature of __hash__ is just an unfortunate necessity. It arises because hashability is sometimes a "deep" property (i.e. it depends on the hashability of a container's contents as well as the nature of the container itself). No such thing applies to __iter__. -- Greg From ark-mlist at att.net Sat Jul 22 16:12:28 2006 From: ark-mlist at att.net (Andrew Koenig) Date: Sat, 22 Jul 2006 10:12:28 -0400 Subject: [Python-3000] callable() In-Reply-To: <44C1E2A2.9030603@canterbury.ac.nz> Message-ID: <001301c6ad98$dd83e280$6402a8c0@arkdesktop> > > That would be at odds with the approach taken with > > list.__hash__ which has to be called in order to find-out it is not > > hashable. > That feature of __hash__ is just an unfortunate necessity. > It arises because hashability is sometimes a "deep" > property (i.e. it depends on the hashability of a > container's contents as well as the nature of the > container itself). No such thing applies to __iter__. This example illustrates an important point: Some object properties don't correspond directly to the presence of a particular attribute, and can't easily be made to do so. In other words: Is it callable? No problem, just check for __call__ Is it iterable? Well, you can't quite check for __iter__ because some iterable types don't have them. Well, we can fix that problem: Change those types so that they have __iter__ to signal that they are iterable. Is it hashable? That's a tough question to answer, because you have to inspect recursively all of the object's components. So you can't just test for __hash__; you have to call it. To my way of thinking, callable, iterable, and hashable are the same kind of concept, and I wish Python would provide a uniform way of finding out whether such concepts apply to an object. That uniform way doesn't have to be in __builtins__, but it would be nice for it to exist. From guido at python.org Sat Jul 22 16:58:09 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 22 Jul 2006 07:58:09 -0700 Subject: [Python-3000] callable() In-Reply-To: <001301c6ad98$dd83e280$6402a8c0@arkdesktop> References: <44C1E2A2.9030603@canterbury.ac.nz> <001301c6ad98$dd83e280$6402a8c0@arkdesktop> Message-ID: On 7/22/06, Andrew Koenig wrote: > This example illustrates an important point: Some object properties don't > correspond directly to the presence of a particular attribute, and can't > easily be made to do so. > > In other words: > > Is it callable? No problem, just check for __call__ > > Is it iterable? Well, you can't quite check for __iter__ because > some iterable types don't have them. Well, we can fix that problem: > Change those types so that they have __iter__ to signal that they > are iterable. > > Is it hashable? That's a tough question to answer, because you > have to inspect recursively all of the object's components. So > you can't just test for __hash__; you have to call it. > > To my way of thinking, callable, iterable, and hashable are the same kind of > concept, and I wish Python would provide a uniform way of finding out > whether such concepts apply to an object. That uniform way doesn't have to > be in __builtins__, but it would be nice for it to exist. Hm... Is a uniform API really a good idea for things that have totally different performance characteristics? Computing hashability is about as expensive as computing the hash... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Sun Jul 23 03:30:55 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 23 Jul 2006 13:30:55 +1200 Subject: [Python-3000] callable() In-Reply-To: <001301c6ad98$dd83e280$6402a8c0@arkdesktop> References: <001301c6ad98$dd83e280$6402a8c0@arkdesktop> Message-ID: <44C2D14F.5020906@canterbury.ac.nz> Andrew Koenig wrote: > To my way of thinking, callable, iterable, and hashable are the same kind of > concept, But you've just pointed out that they're *not* the same kind of concept, no matter how much you might wish that there were. The only way to make hashability testable at less cost than attempting to do it would be to have a separate __is_hashable__ method for that purpose, which would recursively test contents when necessary. I'm far from convinced that testing for hashability is necessary enough to be worth going to that much bother. To my mind it's another case where you shouldn't be designing APIs based on that. -- Greg From ark-mlist at att.net Sun Jul 23 15:11:58 2006 From: ark-mlist at att.net (Andrew Koenig) Date: Sun, 23 Jul 2006 09:11:58 -0400 Subject: [Python-3000] callable() In-Reply-To: <44C2D14F.5020906@canterbury.ac.nz> Message-ID: <001201c6ae59$945d1b50$6402a8c0@arkdesktop> > But you've just pointed out that they're *not* > the same kind of concept, no matter how much > you might wish that there were. > The only way to make hashability testable at > less cost than attempting to do it would be > to have a separate __is_hashable__ method for > that purpose, which would recursively test > contents when necessary. This issue started out as a question of side effects; from an architectural viewpoint, I consider performance to be less important because side effects are likely to affect correctness. So I don't particularly care whether testing for hashability is less expensive than trying to do the hash. What I do care about is being able to determine whether an object has a particular property without having to worry about whether I might change the state of the system in whatever ways are necessary to compute that property. This question is probably sharpest for callability, because it is clear that evaluating foo() might do anything at all, and sometimes I want to control when that anything happens. Nevertheless, I don't agree that testing hashability has to be as expensive as computing the hash. As a simple example, one can see instantly that even a very long string is hashable, even though computing the value of the hash might take a long time if the string is large. From guido at python.org Sun Jul 23 23:23:00 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 23 Jul 2006 14:23:00 -0700 Subject: [Python-3000] callable() In-Reply-To: <001201c6ae59$945d1b50$6402a8c0@arkdesktop> References: <44C2D14F.5020906@canterbury.ac.nz> <001201c6ae59$945d1b50$6402a8c0@arkdesktop> Message-ID: On 7/23/06, Andrew Koenig wrote: > > But you've just pointed out that they're *not* > > the same kind of concept, no matter how much > > you might wish that there were. > > > The only way to make hashability testable at > > less cost than attempting to do it would be > > to have a separate __is_hashable__ method for > > that purpose, which would recursively test > > contents when necessary. > > This issue started out as a question of side effects; from an architectural > viewpoint, I consider performance to be less important because side effects > are likely to affect correctness. > > So I don't particularly care whether testing for hashability is less > expensive than trying to do the hash. What I do care about is being able to > determine whether an object has a particular property without having to > worry about whether I might change the state of the system in whatever ways > are necessary to compute that property. > > This question is probably sharpest for callability, because it is clear that > evaluating foo() might do anything at all, and sometimes I want to control > when that anything happens. > > Nevertheless, I don't agree that testing hashability has to be as expensive > as computing the hash. As a simple example, one can see instantly that even > a very long string is hashable, even though computing the value of the hash > might take a long time if the string is large. I propose to take the same approach as for callable: if it has __hash__ we consider it hashable even though the hash may fail (e.g. if it's a tuple containing an unhashable object). This is roughly equivalent to the usefulness of callable: the call may still fail if the signature doesn't match. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Mon Jul 24 03:45:48 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 24 Jul 2006 13:45:48 +1200 Subject: [Python-3000] callable() In-Reply-To: References: <44C2D14F.5020906@canterbury.ac.nz> <001201c6ae59$945d1b50$6402a8c0@arkdesktop> Message-ID: <44C4264C.9030000@canterbury.ac.nz> Guido van Rossum wrote: > I propose to take the same approach as for callable: if it has > __hash__ we consider it hashable even though the hash may fail Fair enough, although since object has __hash__ we end up concluding that everything is hashable except when it isn't. :-) -- Greg From mcherm at mcherm.com Mon Jul 24 15:03:31 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Mon, 24 Jul 2006 06:03:31 -0700 Subject: [Python-3000] callable() Message-ID: <20060724060331.kgficemrhs84s08w@login.werra.lunarpages.com> Andrew Koenig writes: > This issue started out as a question of side effects; from an architectural > viewpoint, I consider performance to be less important because side effects > are likely to affect correctness. Indeed... side effects are the primary concern. A __hash__ method with side effects is not formally prohibited in the documentation but nevertheless, a __hash__ which is not idempotent is an abomination.[1] Thus, there is no need for a test of whether __hash__ will succeed: just try it. Both calling and iterating are well-known to have irreversable side effects in many commonly-used callables/iterables. Thus it would be nice to have a test other than invocation to see whether the object is "intended to work". Obviously, an object "intended to work" may still fail: the file system being iterated over could generate hardware errors, or the function supplied might raise a TypeError when invoked with the particular signature you use. But at least it's willing to try... you can identify those cases where an integer was accidentally provided instead of the intended callable/iterable. Personally, I don't care whether the test is spelled "callable(x)" or "hasattr(x, '__call__')" (and similarly for iterable), so long as it is documented. -- Michael Chermside [1] I subtlely switched language from "side effect free" to "idempotent" here because caching *IS* acceptable in __hash__ mehtods. If you're not paying close attention I probably got away with it. Then I went and spoiled it with a footnote. From ark-mlist at att.net Mon Jul 24 16:15:16 2006 From: ark-mlist at att.net (Andrew Koenig) Date: Mon, 24 Jul 2006 10:15:16 -0400 Subject: [Python-3000] callable() In-Reply-To: <20060724060331.kgficemrhs84s08w@login.werra.lunarpages.com> Message-ID: <002d01c6af2b$9651b6f0$6402a8c0@arkdesktop> > A __hash__ method with side effects is not formally prohibited in the > documentation but nevertheless, a __hash__ which is not idempotent is > an abomination.[1] Thus, there is no need for a test of whether __hash__ > will succeed: just try it. > [1] I subtlely switched language from "side effect free" to "idempotent" > here because caching *IS* acceptable in __hash__ mehtods. If you're > not paying close attention I probably got away with it. Then I went > and spoiled it with a footnote. I picked it up before seeing the footnote :-) I can think of two troublesome cases; there may be others I haven't thought of. 1) An object x with a component y that might be hashable. x is hashable apart from x.y, but hashing x for the first time has side effects, so the value is cached. The hash of x depends partly on the hash of x.y. In other words: def __init__(self): self.hashmem = None def __hash__(self): if not self.hashmem: self.hashmem = self.y.partialhash(self.partialhash()) return self.hashmem If we call x.__hash__, it will try to hash x.y; if that fails, x.__hash__ will raise an exception but the side effect of computing x.partialhash() will already have happened. In other words, even though hashing is idempotent when it succeeds, you may still wish to avoid the side effects of trying to evaluate it if it is going to fail. 2) An object for which computing the hash for the first time requires human action. For example, we might have an object that represents a CD in a CD library. Computing the hash for such an object might require someone to load the CD into a drive so that it can be read. From mcherm at mcherm.com Mon Jul 24 22:42:42 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Mon, 24 Jul 2006 13:42:42 -0700 Subject: [Python-3000] callable() Message-ID: <20060724134242.ik22b5ih6z4808wk@login.werra.lunarpages.com> I wrote: > [1] I subtlely switched language from "side effect free" to "idempotent" > here because caching *IS* acceptable in __hash__ mehtods. If you're > not paying close attention I probably got away with it. Andrew Koenig responds: > I picked it up before seeing the footnote :-) Yeah... I have to keep in mind that the readers of python-dev and python-3000 are a lot sharper than most folks I correspond with. Andrew again: > I can think of two troublesome cases; there may be others I haven't thought > of. > > 1) An object x with a component y that might be hashable. x is hashable > apart from x.y, but hashing x for the first time has side effects, so the > value is cached. The hash of x depends partly on the hash of x.y. [...] > 2) An object for which computing the hash for the first time requires human > action. In both cases, __hash__ is not idempotent, and is thus an abomination. Case 1 is a perverse programmer -- well known to be capable of abominations. Case 2 sounds like a made-up example... can you think of anything real which behaves this way? (For instance: the file type doesn't even though it might, and I presume this is intentional.) -- Michael Chermside From ark-mlist at att.net Mon Jul 24 23:27:30 2006 From: ark-mlist at att.net (Andrew Koenig) Date: Mon, 24 Jul 2006 17:27:30 -0400 Subject: [Python-3000] callable() In-Reply-To: <20060724134242.ik22b5ih6z4808wk@login.werra.lunarpages.com> Message-ID: <007e01c6af67$f85af280$6402a8c0@arkdesktop> > In both cases, __hash__ is not idempotent, and is thus an abomination. Why do you say it's not idempotent? The first time you call it, either it works or it doesn't. If it doesn't work, then you shouldn't have called it in the first place. If it does work, all subsequent calls will return the same result. > Case > 1 is a perverse programmer -- well known to be capable of abominations. What is perverse about case 1? I'm not being disingenuous here; I really don't know. I am assuming, of course, that the object in question never changes the value of its component once constructed. > Case 2 sounds like a made-up example... can you think of anything real > which behaves this way? (For instance: the file type doesn't even though > it might, and I presume this is intentional.) I'll admit it's a little more far-fetched; that's why it's second on the list. But hypothetical or not, I think it serves to underscore the original point, which is that it is entirely plausible for objects to have properties that trigger side effects when used, and for which it may be useful to have a way of asking about their presence without triggering the side effects. From mcherm at mcherm.com Tue Jul 25 00:27:28 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Mon, 24 Jul 2006 15:27:28 -0700 Subject: [Python-3000] callable() Message-ID: <20060724152728.16om0i26isw84s0c@login.werra.lunarpages.com> Earlier in the thread: > two troublesome cases: [...] > 1) An object x with a component y that might be hashable. x is hashable > apart from x.y, but hashing x for the first time has side effects, so the > value is cached. The hash of x depends partly on the hash of x.y. [...] > 2) An object for which computing the hash for the first time requires human > action. Andrew Koenig writes: > > In both cases, __hash__ is not idempotent, and is thus an abomination. > > Why do you say it's not idempotent? The first time you call it, either it > works or it doesn't. If it doesn't work, then you shouldn't have called it > in the first place. If it does work, all subsequent calls will return the > same result. Which case are you considering? In case 1, __hash__ of the parent object is not idempotent because it depends on whether __hash__ of the child object has a cached value or not. In case 2, __hash__ either returns a value or raises an exception (or perhaps blocks forever if poorly written) depending on the behavior of the human being. Neither is idempotent. Andrew some more: > What is perverse about case 1? I'm not being disingenuous here; I really > don't know. I am assuming, of course, that the object in question never > changes the value of its component once constructed. Hash values should only be cached if they will never change. Furthermore, hash values are formally required to be consistant (not change). The "value" of the component may never change but it's __hash__ may. Consider the following series of calls: x1 = hash(x) y1 = hash(y) x2 = hash(x) y2 = hash(y) After this series of calls, x1 != x2 and y1 != y2. That breaks the contract for __hash__ which I would call perverse. Andrew concludes: > ...the original point, which is that it is entirely plausible for > objects to have properties > that trigger side effects when used, and for which it may be useful to have > a way of asking about their presence without triggering the side effects. I don't doubt that some properties like the behavior of __call__ or __iter__ may trigger side effects, and it would nice to be able to test for the property without triggering the side effects. Your argument is bolstered if you add __hash__ to the list because hashability is something that *must* be tested for (by dict at least!). But I object... __hash__ cannot trigger side effects (well, it must be idempotent anyhow) thus you are limited to __call__ and __iter__ as examples, of which the best we can say is that it would *nice* to be able to test for them. This is all moot anyway, because I think everyone agrees to make hasattr(x, '__call__') and hasattr(x, '__iter__') be the tests and to ensure in Python3000 that (unlike older CPythons) those work for all objects. But-moot-means-debatable-so-I-guess-that's-what-we're-doing-lly, yours Michael Chermside From ark-mlist at att.net Tue Jul 25 05:19:23 2006 From: ark-mlist at att.net (Andrew Koenig) Date: Mon, 24 Jul 2006 23:19:23 -0400 Subject: [Python-3000] callable() In-Reply-To: <20060724152728.16om0i26isw84s0c@login.werra.lunarpages.com> Message-ID: <00a901c6af99$207d0880$6402a8c0@arkdesktop> > Which case are you considering? In case 1, __hash__ of the parent object > is not idempotent because it depends on whether __hash__ of the child > object has a cached value or not. I don't see why you should say that; it's certainly not my intent. What did I say that makes you think that that dependency exists? > In case 2, __hash__ either returns > a value or raises an exception (or perhaps blocks forever if poorly > written) depending on the behavior of the human being. Neither is > idempotent. Perhaps we are using "idempotent" to mean different things. I think of a function f as being idempotent if f(x) == x for all x in dom(f). If __hash__ fails to return a value, you've made a mistake by calling it on an object outside its domain. But if it does return a value, I intend for it always to return that same value on subsequent calls. > Hash values should only be cached if they will never change. Agreed. > Furthermore, hash values are formally required to be consistant > (not change). The "value" of the component may never change but| > it's __hash__ may. I should certainly hope not. > Consider the following series of calls: > x1 = hash(x) > y1 = hash(y) > x2 = hash(x) > y2 = hash(y) > After this series of calls, x1 != x2 and y1 != y2. That breaks the > contract for __hash__ which I would call perverse. If you think my examples make that possible, then I've stated them poorly, and would appreciate knowing where the error lies. That was not my intent. > I don't doubt that some properties like the behavior of __call__ or > __iter__ may trigger side effects, and it would nice to be able to test > for the property without triggering the side effects. Your argument is > bolstered if you add __hash__ to the list because hashability is something > that *must* be tested for (by dict at least!). But I object... __hash__ > cannot trigger side effects (well, it must be idempotent anyhow) thus you > are limited to __call__ and __iter__ as examples, of which the best we > can say is that it would *nice* to be able to test for them. I don't know why you think that __call__ and __iter__ are the only similar properties that any object might ever have. > This is all moot anyway, because I think everyone agrees to make > hasattr(x, '__call__') and hasattr(x, '__iter__') be the tests and to > ensure in Python3000 that (unlike older CPythons) those work for all > objects. It's OK with me, but I still think it strikes me as strange, because it makes a commitment about the implementation that I think would be better concealed behind an abstraction layer. I freely admit that I'm not as steeped in Python culture as many on this list--which may or may not be a good thing as far as this discussion goes. I guess I've just seen enough cases where people have exposed implementations and then regretted it later that little alarm bells go off in my head when I see designs such as these. Sometimes, of course, they are false alarms. From ncoghlan at gmail.com Tue Jul 25 14:53:45 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 25 Jul 2006 22:53:45 +1000 Subject: [Python-3000] callable() In-Reply-To: <007e01c6af67$f85af280$6402a8c0@arkdesktop> References: <007e01c6af67$f85af280$6402a8c0@arkdesktop> Message-ID: <44C61459.40603@gmail.com> Andrew Koenig wrote: >> In both cases, __hash__ is not idempotent, and is thus an abomination. > > Why do you say it's not idempotent? The first time you call it, either it > works or it doesn't. If it doesn't work, then you shouldn't have called it > in the first place. If it does work, all subsequent calls will return the > same result. > >> Case >> 1 is a perverse programmer -- well known to be capable of abominations. > > What is perverse about case 1? I'm not being disingenuous here; I really > don't know. I am assuming, of course, that the object in question never > changes the value of its component once constructed. I wouldn't call case 1 perverse, but I would call it buggy if x.partialhash() wasn't idempotent, or if it used the same hash cache as the full hash function. E.g. there's no state consistency problems with the following: def __init__(self): self._fullhash = None self._partialhash = None def partialhash(self, init_hash=None): if self._partialhash is None: #work it out and set it return self._partialhash def __hash__(self): if self._fullhash is None: selfpart = self.partialhash() self._fullhash = self.y.partialhash(selfpart) return self._fullhash Alternatively, the __hash__ function could be written in a transactional style, backing out the call to the partial hash if the hash of the subcomponent failed: def __init__(self): self._fullhash = None def __hash__(self): if self._fullhash is None: selfpart = self.partialhash() try: self._fullhash = self.y.partialhash(selfpart) except: self._clearpartialhash() raise return self._fullhash Either way, if the __hash__ function can fail in a way that can leave the object in an inconsistent state, then that's a bug in the implementation of the __hash__ function. For case 2, the problem is the idea of using the hash of the entire CD as the __hash__ of the object that represents that CD in memory, and then making the retrieval of that data a side effect of attempting to hash the object. Touching an IO device or the network to compute the hash of an in memory data structure sounds like an incredibly bad idea. If that information is an important enough part of the object's identity to be included in its hash, it needs to be retrieved before the object can be considered fully created, and it should NOT be done as a side effect of trying to hash the object. Instead, if the attribute is not set, the hash operation should simply fail with something like RuntimeError("necessary attribute not set"). Or you can be stricter, and make the attribute mandatory at object creation time. All of which is a long-winded way of saying "calculation of an object hash should be both cheap and idempotent" :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ark-mlist at att.net Tue Jul 25 15:10:18 2006 From: ark-mlist at att.net (Andrew Koenig) Date: Tue, 25 Jul 2006 09:10:18 -0400 Subject: [Python-3000] callable() In-Reply-To: <44C61459.40603@gmail.com> Message-ID: <000601c6afeb$ad3b2e30$6402a8c0@arkdesktop> > All of which is a long-winded way of saying "calculation of an object hash > should be both cheap and idempotent" :) Actually, I disagree -- I don't see why there's anything wrong with a hash being expensive to calculate the first time you do it. For example, consider a string type in which the hash algorithm examines every character of the string. Those characters had to get there in the first place, so the total time spent computing the hash is no more than a constant multiple of the time spent creating the string. Nevertheless, it seems reasonable to me to defer the effort of computing the hash until you know that it's needed -- that is, until the first time you are asked to compute the hash. If you're going to say that computing the hash isn't expensive compared with dealing with the string itself, then I'll reply that computing the hash of a CD isn't expensive either, if you compare it with dealing with the CD itself :-) From ncoghlan at gmail.com Tue Jul 25 15:53:37 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 25 Jul 2006 23:53:37 +1000 Subject: [Python-3000] callable() In-Reply-To: <000601c6afeb$ad3b2e30$6402a8c0@arkdesktop> References: <000601c6afeb$ad3b2e30$6402a8c0@arkdesktop> Message-ID: <44C62261.2090409@gmail.com> Andrew Koenig wrote: >> All of which is a long-winded way of saying "calculation of an object hash >> should be both cheap and idempotent" :) > > Actually, I disagree -- I don't see why there's anything wrong with a hash > being expensive to calculate the first time you do it. True, but if you cache the result, the amortized cost may still work out to be cheap. > For example, consider a string type in which the hash algorithm examines > every character of the string. Those characters had to get there in the > first place, so the total time spent computing the hash is no more than a > constant multiple of the time spent creating the string. Nevertheless, it > seems reasonable to me to defer the effort of computing the hash until you > know that it's needed -- that is, until the first time you are asked to > compute the hash. > > If you're going to say that computing the hash isn't expensive compared with > dealing with the string itself, then I'll reply that computing the hash of a > CD isn't expensive either, if you compare it with dealing with the CD itself > :-) The difference between the two cases is that when you create the string object, you have all the information you need to calculate the hash when you need it, so there's no problem with deferring the actual calculation. With your CD example, you need an external resource (the CD itself) in order to calculate the hash - in that case, you can't safely defer the hash calculation until the first time you know you need it, since you don't know whether or not you'll have access to the physical CD at that point. In such a case, an application would need to either hash the CD at object creation time, or else use the default hash() for the objects representing the CDs in memory, and auxiliary data structures to map the physical CD hashes when they are available. Having __hash__ depend on attributes which are not defined at object creation time is just asking for trouble. This aligns with the concept that the hash should be based solely on immutable aspects of the object, which _have_ to be defined at object creation time :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From krstic at solarsail.hcs.harvard.edu Tue Jul 25 16:15:52 2006 From: krstic at solarsail.hcs.harvard.edu (Ivan Krstic) Date: Tue, 25 Jul 2006 10:15:52 -0400 Subject: [Python-3000] Proposal: Metasyntax operator In-Reply-To: References: <44BF45AE.3060307@acm.org> Message-ID: <44C62798.207@solarsail.hcs.harvard.edu> Guido van Rossum wrote: > I'd like to see this fall under the blanket "Python will not have > programmable syntax" rule in PEO 3099. Why? I understand your general position, but categorically rejecting anything smelling of metasyntax seems ill-conceived to me. Python already has just enough programmable bits in place to invite abuse which emulates programmable syntax; look at SQLObject or SQLalchemy for examples. And the former is a very high-profile module. This is akin to Zope's incredibly messy 'implements' hack (walk up the stack, inject a factory metaclass which does base class resolution on the fly), which is completely obviated by the introduction of class decorators. Class decorators are not a feature that many of us consider obviously-needed or beautiful, but the code that's out there demonstrates a need for it. The same is the case with (explicit!) AST literals of some kind. -- Ivan Krstic | GPG: 0x147C722D From ark-mlist at att.net Tue Jul 25 16:19:54 2006 From: ark-mlist at att.net (Andrew Koenig) Date: Tue, 25 Jul 2006 10:19:54 -0400 Subject: [Python-3000] callable() In-Reply-To: <44C62261.2090409@gmail.com> Message-ID: <001901c6aff5$66975580$6402a8c0@arkdesktop> > With your CD example, you need an external resource (the CD itself) in > order to calculate the hash - in that case, you can't safely defer > the hash calculation until the first time you know you need it, > since you don't know whether or not you'll have access to the > physical CD at that point. OK. Then instead of a CD, use a (read-only) disk file. The example is still valid in that case, and avoids the extraneous matter of operator intervention. It's still the case that evaluating a hash might be expensive the first time you do it, so you may want to be able to test whether it can be done and actually do it only later. From rasky at develer.com Tue Jul 25 23:43:41 2006 From: rasky at develer.com (Giovanni Bajo) Date: Tue, 25 Jul 2006 23:43:41 +0200 Subject: [Python-3000] callable() References: <44C62261.2090409@gmail.com> <001901c6aff5$66975580$6402a8c0@arkdesktop> Message-ID: <0d1c01c6b033$653f3660$d503030a@trilan> Andrew Koenig wrote: > It's still the case that evaluating a hash might be expensive the > first time you do it, so you may want to be able to test whether it > can be done and actually do it only later. I think you're doing a lot of guess-work here. In real world situations, computing the hash is very very fast by design (at worse O(n) as you noticed before for the string case, but a very fast algorithm with a very low constant multiplier), it is done only once, and checking if the whole object is (recursively) hashable is *very* near to the actual cost of computing the hash. Simple immutable objects have simple, fast hash functions; large mutable objects use id(). An object with a very long string can be an exception, but I don't think this situation is common enough to warrant a special-case and a complication of the language. I still need to hear one good reason to have a (performance critical!) check of the *hashability* of an object, in the first place. I never ever had to do it, let alone in a critical path of my code. -- Giovanni Bajo From collinw at gmail.com Wed Jul 26 00:15:47 2006 From: collinw at gmail.com (Collin Winter) Date: Tue, 25 Jul 2006 18:15:47 -0400 Subject: [Python-3000] callable() In-Reply-To: <001901c6aff5$66975580$6402a8c0@arkdesktop> References: <44C62261.2090409@gmail.com> <001901c6aff5$66975580$6402a8c0@arkdesktop> Message-ID: <43aa6ff70607251515l3ef35191qfb2cb9a5d60724fc@mail.gmail.com> On 7/25/06, Andrew Koenig wrote: > > With your CD example, you need an external resource (the CD itself) in > > order to calculate the hash - in that case, you can't safely defer > > the hash calculation until the first time you know you need it, > > since you don't know whether or not you'll have access to the > > physical CD at that point. > > OK. Then instead of a CD, use a (read-only) disk file. The example is > still valid in that case, and avoids the extraneous matter of operator > intervention. Oh, please. No-one's arguing that hashes MUST be fast to compute; it's just a good rule of thumb. There will of course be exceptional cases, like the contrived examples you've come up with, but they're just that: exceptional. If you find yourself writing classes that must hash a large on-disk file or CD or whatever else, you can add custom tests for hashability yourself. There's no need to clutter the language to cater to every odd corner case someone digs up. Collin Winter PS: A read-only disk file has the same problem that Nick pointed out with your CD example, namely that that file might not exist when you try to compute the hash. Just because it's read-only to you doesn't mean the superuser can't remove it. From guido at python.org Wed Jul 26 05:23:54 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 25 Jul 2006 20:23:54 -0700 Subject: [Python-3000] callable() In-Reply-To: <44C4264C.9030000@canterbury.ac.nz> References: <44C2D14F.5020906@canterbury.ac.nz> <001201c6ae59$945d1b50$6402a8c0@arkdesktop> <44C4264C.9030000@canterbury.ac.nz> Message-ID: Ouch, that's a problem indeed. There's not really a clean way out unfortunately: either we remove object.__hash__ and let hash() do the default implementation, and then presence of __hash__ is no longer an indicator of hashability; or we keep it, and override it to raise an exception in unhashable types (like list.__hash__ does); or we remove the hashability of objects by default, which lets us solve the above problems at the cost of having a convenient hash by default. Personally, I'm not sure this problem needs solving; I don't recall ever needing to know whether something is hashable. So perhaps it's of purely theoretical importance? That would suit me fine given the above dilemma... --Guido On 7/23/06, Greg Ewing wrote: > Guido van Rossum wrote: > > I propose to take the same approach as for callable: if it has > > __hash__ we consider it hashable even though the hash may fail > > Fair enough, although since object has __hash__ > we end up concluding that everything is > hashable except when it isn't. :-) > > -- > Greg > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From behnel_ml at gkec.informatik.tu-darmstadt.de Wed Jul 26 05:47:47 2006 From: behnel_ml at gkec.informatik.tu-darmstadt.de (Stefan Behnel) Date: Wed, 26 Jul 2006 05:47:47 +0200 Subject: [Python-3000] callable() In-Reply-To: <43aa6ff70607251515l3ef35191qfb2cb9a5d60724fc@mail.gmail.com> References: <44C62261.2090409@gmail.com> <001901c6aff5$66975580$6402a8c0@arkdesktop> <43aa6ff70607251515l3ef35191qfb2cb9a5d60724fc@mail.gmail.com> Message-ID: <44C6E5E3.5080708@gkec.informatik.tu-darmstadt.de> Collin Winter wrote: > On 7/25/06, Andrew Koenig wrote: >>> With your CD example, you need an external resource (the CD itself) in >>> order to calculate the hash - in that case, you can't safely defer >>> the hash calculation until the first time you know you need it, >>> since you don't know whether or not you'll have access to the >>> physical CD at that point. >> OK. Then instead of a CD, use a (read-only) disk file. The example is >> still valid in that case, and avoids the extraneous matter of operator >> intervention. > > Oh, please. No-one's arguing that hashes MUST be fast to compute; it's > just a good rule of thumb. There will of course be exceptional cases, > like the contrived examples you've come up with, but they're just > that: exceptional. If you find yourself writing classes that must hash > a large on-disk file or CD or whatever else, you can add custom tests > for hashability yourself. There's no need to clutter the language to > cater to every odd corner case someone digs up. Very well said and I couldn't agree more. +1 for not adding hashable() to the language. Stefan From greg.ewing at canterbury.ac.nz Wed Jul 26 08:20:37 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 26 Jul 2006 18:20:37 +1200 Subject: [Python-3000] callable() In-Reply-To: <00a901c6af99$207d0880$6402a8c0@arkdesktop> References: <00a901c6af99$207d0880$6402a8c0@arkdesktop> Message-ID: <44C709B5.5080507@canterbury.ac.nz> Andrew Koenig wrote: > I think of a > function f as being idempotent if f(x) == x for all x in dom(f). Um, that's an identity function, isn't it? I think you mean something more like f(f(x)) == f(x). But even that's not quite what we're talking about here, because we're not hashing the result of the hash function. If idempotency is involved it's about side effects rather than the return value. From a functional point of view, the hash value of a given object should simply be constant. -- Greg From greg.ewing at canterbury.ac.nz Wed Jul 26 08:43:21 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 26 Jul 2006 18:43:21 +1200 Subject: [Python-3000] Proposal: Metasyntax operator In-Reply-To: <44C62798.207@solarsail.hcs.harvard.edu> References: <44BF45AE.3060307@acm.org> <44C62798.207@solarsail.hcs.harvard.edu> Message-ID: <44C70F09.1030706@canterbury.ac.nz> Ivan Krstic wrote: > Python already has > just enough programmable bits in place to invite abuse which emulates > programmable syntax; look at SQLObject or SQLalchemy for examples. Also, the "just use a string" workaround means that people are effectively doing this anyway, but in an uglier and less efficient way. -- Greg From greg.ewing at canterbury.ac.nz Wed Jul 26 09:09:18 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 26 Jul 2006 19:09:18 +1200 Subject: [Python-3000] callable() In-Reply-To: References: <44C2D14F.5020906@canterbury.ac.nz> <001201c6ae59$945d1b50$6402a8c0@arkdesktop> <44C4264C.9030000@canterbury.ac.nz> Message-ID: <44C7151E.6000209@canterbury.ac.nz> Guido van Rossum wrote: > Personally, I'm not sure this problem needs solving; I don't recall > ever needing to know whether something is hashable. I for one have gotten along quite happily without ever wanting to test for hashability. -- Greg From ncoghlan at gmail.com Wed Jul 26 12:25:41 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 26 Jul 2006 20:25:41 +1000 Subject: [Python-3000] callable() In-Reply-To: References: <44C2D14F.5020906@canterbury.ac.nz> <001201c6ae59$945d1b50$6402a8c0@arkdesktop> <44C4264C.9030000@canterbury.ac.nz> Message-ID: <44C74325.5030504@gmail.com> Guido van Rossum wrote: > Ouch, that's a problem indeed. There's not really a clean way out > unfortunately: either we remove object.__hash__ and let hash() do the > default implementation, and then presence of __hash__ is no longer an > indicator of hashability; or we keep it, and override it to raise an > exception in unhashable types (like list.__hash__ does); or we remove > the hashability of objects by default, which lets us solve the above > problems at the cost of having a convenient hash by default. > > Personally, I'm not sure this problem needs solving; I don't recall > ever needing to know whether something is hashable. So perhaps it's of > purely theoretical importance? That would suit me fine given the above > dilemma... The only thing I've occasionally wanted in this area is an easy way to tell __getattribute__ to raise an AttributeError for attribute 'x' even if a base class defines it. The use case is being able to block the inheritance of special methods that object provides default implementations for (like '__hash__'), such that a hasattr() check (or a check for a type slot being 0) for those special methods will actually fail. It would be nice to be able to spell it: class Unhashable(object): undef __hash__ Although that would probably require a signature change for metaclass calling to accept an optional list of undefined names along with the dictionary of assigned names. Alternatively, another special attribute like __slots__ could do it: class Unhashable(object): __undef__ = ["__hash__"] Or a special value recognised by the default metaclass: class Unhashable(object): __hash__ = Undefined When the attributes being undefined represent type slots, the corresponding slot would be set to 0. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From greg.ewing at canterbury.ac.nz Wed Jul 26 12:32:11 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 26 Jul 2006 22:32:11 +1200 Subject: [Python-3000] callable() In-Reply-To: <44C74325.5030504@gmail.com> References: <44C2D14F.5020906@canterbury.ac.nz> <001201c6ae59$945d1b50$6402a8c0@arkdesktop> <44C4264C.9030000@canterbury.ac.nz> <44C74325.5030504@gmail.com> Message-ID: <44C744AB.4010907@canterbury.ac.nz> Nick Coghlan wrote: > The use case is being able to block the inheritance of special methods > that object provides default implementations for (like '__hash__'), such > that a hasattr() check (or a check for a type slot being 0) for those > special methods will actually fail. Maybe descriptors could be given a __has__ slot that got called by hasattr() if present. Then a descriptor could be created that would have this effect, and your > class Unhashable(object): > __hash__ = Undefined spelling could be used. -- Greg From ncoghlan at gmail.com Wed Jul 26 13:33:55 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 26 Jul 2006 21:33:55 +1000 Subject: [Python-3000] callable() In-Reply-To: <44C744AB.4010907@canterbury.ac.nz> References: <44C2D14F.5020906@canterbury.ac.nz> <001201c6ae59$945d1b50$6402a8c0@arkdesktop> <44C4264C.9030000@canterbury.ac.nz> <44C74325.5030504@gmail.com> <44C744AB.4010907@canterbury.ac.nz> Message-ID: <44C75323.6020903@gmail.com> Greg Ewing wrote: > Nick Coghlan wrote: > >> The use case is being able to block the inheritance of special methods >> that object provides default implementations for (like '__hash__'), >> such that a hasattr() check (or a check for a type slot being 0) for >> those special methods will actually fail. > > Maybe descriptors could be given a __has__ slot that got > called by hasattr() if present. Then a descriptor could > be created that would have this effect, and your > >> class Unhashable(object): >> __hash__ = Undefined > > spelling could be used. I just realised that a __get__ method that unconditionally raised AttributeError would do the trick in terms of getting hasattr() to give the right answer. This wouldn't help C code that checked for whether or not the relevant tp_* slot was NULL, though. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From rasky at develer.com Wed Jul 26 14:24:33 2006 From: rasky at develer.com (Giovanni Bajo) Date: Wed, 26 Jul 2006 14:24:33 +0200 Subject: [Python-3000] callable() References: <44C2D14F.5020906@canterbury.ac.nz><001201c6ae59$945d1b50$6402a8c0@arkdesktop><44C4264C.9030000@canterbury.ac.nz> <44C7151E.6000209@canterbury.ac.nz> Message-ID: <038001c6b0ae$734f1b00$d503030a@trilan> Greg Ewing wrote: > Guido van Rossum wrote: > >> Personally, I'm not sure this problem needs solving; I don't recall >> ever needing to know whether something is hashable. > > I for one have gotten along quite happily without > ever wanting to test for hashability. Me too, as said elsewhere. This whole thread seems like a purely theoretical debate. -- Giovanni Bajo From guido at python.org Wed Jul 26 17:37:20 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 26 Jul 2006 08:37:20 -0700 Subject: [Python-3000] callable() In-Reply-To: <44C74325.5030504@gmail.com> References: <44C2D14F.5020906@canterbury.ac.nz> <001201c6ae59$945d1b50$6402a8c0@arkdesktop> <44C4264C.9030000@canterbury.ac.nz> <44C74325.5030504@gmail.com> Message-ID: On 7/26/06, Nick Coghlan wrote: > class Unhashable(object): > __hash__ = Undefined Simpler: set it to None. That could be a convention. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From gmccaughan at synaptics-uk.com Wed Jul 26 16:47:41 2006 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Wed, 26 Jul 2006 15:47:41 +0100 Subject: [Python-3000] callable() In-Reply-To: <44C709B5.5080507@canterbury.ac.nz> References: <00a901c6af99$207d0880$6402a8c0@arkdesktop> <44C709B5.5080507@canterbury.ac.nz> Message-ID: <200607261547.42986.gmccaughan@synaptics-uk.com> On Wednesday 2006-07-26 07:20, Greg Ewing wrote: > Andrew Koenig wrote: > > I think of a > > function f as being idempotent if f(x) == x for all x in dom(f). > > Um, that's an identity function, isn't it? I think > you mean something more like f(f(x)) == f(x). > > But even that's not quite what we're talking about > here, because we're not hashing the result of the > hash function. If idempotency is involved it's > about side effects rather than the return value. > From a functional point of view, the hash value > of a given object should simply be constant. What needs to be idempotent -- i.e., to satisfy f(f(x))==f(x) -- is the function (from states of the world to states of the world, suitably abstracted) that describes the total effect of calling the hash function. -- g From ark-mlist at att.net Wed Jul 26 18:35:41 2006 From: ark-mlist at att.net (Andrew Koenig) Date: Wed, 26 Jul 2006 12:35:41 -0400 Subject: [Python-3000] callable() In-Reply-To: <038001c6b0ae$734f1b00$d503030a@trilan> Message-ID: <001501c6b0d1$89973780$6402a8c0@arkdesktop> > > I for one have gotten along quite happily without > > ever wanting to test for hashability. > Me too, as said elsewhere. This whole thread seems like a purely > theoretical debate. That may be because it has departed from its original intent. I started the thread because I wanted to call attention to an issue: Objects sometimes have properties that have side effects when exploited, and it can be useful to be able to test for the presence of such properties without evoking the side effects. I used callable as an example, and several people responded that there was no problem: An object should be considered callable if and only if it has a __call__ attribute. My response was that I felt uncomfortable about exposing the implementation that way, and some others said it wasn't really an issue because there weren't enough such properties to worry about. Someone suggested hashability as another such property, to which the response was that that also wasn't a problem: Just try doing it and see if it works. This suggestion led to a branch about whether hashing could ever cause side effects, and whether it might be expensive to compute the hash as a way of testing whether it could be computed. Most recently, someone observed that class object has __hash__ defined, which means that to test for hashability, you have to try it; you can't just check for the presence of __hash__. The response to that is that we could have a convention: if __hash__ is None, then the object isn't hashable. So now we have the corresponding question for __call__. Suppose you want to inherit from a callable class but you don't want your own class to be callable. Can you set __call__ to None? If that convention is adopted, it would break the equivalence between callability and the existence of __call__. In other words, the current notion appears to be: An object is callable iff it has a __call__ attribute. An object is hashable iff its __hash__ attribute is not None. This example only strengthens my original intuition that if it is desirable to check whether an object has a given property, the way of doing so should be uniform across different properties. In other words, if the notion of checking for a property is useful, there should be a single abstraction for doing that check. From guido at python.org Wed Jul 26 18:52:28 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 26 Jul 2006 09:52:28 -0700 Subject: [Python-3000] callable() In-Reply-To: <001501c6b0d1$89973780$6402a8c0@arkdesktop> References: <038001c6b0ae$734f1b00$d503030a@trilan> <001501c6b0d1$89973780$6402a8c0@arkdesktop> Message-ID: On 7/26/06, Andrew Koenig wrote: > In other words, the current notion appears to be: > > An object is callable iff it has a __call__ attribute. > > An object is hashable iff its __hash__ attribute is not None. > > This example only strengthens my original intuition that if it is desirable > to check whether an object has a given property, the way of doing so should > be uniform across different properties. In other words, if the notion of > checking for a property is useful, there should be a single abstraction for > doing that check. I'd be happy to extend the convention to all such attributes -- setting it to None to mean that the subclass doesn't want to provide it. That's clean, can't possibly be interpreted to mean anything else, and doesn't require you to actually call the attribute. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Jul 26 18:54:58 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 26 Jul 2006 09:54:58 -0700 Subject: [Python-3000] Proposal: Metasyntax operator In-Reply-To: <44C70F09.1030706@canterbury.ac.nz> References: <44BF45AE.3060307@acm.org> <44C62798.207@solarsail.hcs.harvard.edu> <44C70F09.1030706@canterbury.ac.nz> Message-ID: On 7/25/06, Greg Ewing wrote: > Ivan Krstic wrote: > > Python already has > > just enough programmable bits in place to invite abuse which emulates > > programmable syntax; look at SQLObject or SQLalchemy for examples. > > Also, the "just use a string" workaround means that > people are effectively doing this anyway, but in an > uglier and less efficient way. That depends on what exactly they're trying to do. The problem with exposing the AST a la Boo is that it requires all Python implementations to support the same AST. That's fine for Boo which chose to limit itself to the .NET platform. I don't like this for Python though. I'm not against having a library module that exposes this functionality in a CPython-specific way but I don't want to integrate it with the language. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From mcherm at mcherm.com Wed Jul 26 22:43:29 2006 From: mcherm at mcherm.com (Michael Chermside) Date: Wed, 26 Jul 2006 13:43:29 -0700 Subject: [Python-3000] callable() Message-ID: <20060726134329.er27og60l0w0wcww@login.werra.lunarpages.com> Andrew Koenig writes: > I started the thread because I wanted to call attention to an issue: Objects > sometimes have properties that have side effects when exploited, and it can > be useful to be able to test for the presence of such properties without > evoking the side effects. Good point. I agree. I cannot list every possible property, but so far three have been mentioned. For callability and iterability, I think that hasattr(x, '__call__') and hasattr(x, '__iter__') are perfectly good spellings for the test. You say: > I felt uncomfortable about exposing the implementation that way ...but double-underscore methods are part of the language definition, not part of the implementation. If we were saying to check whether certain tp* fields in a C struct were populated, then I would object, but checking for double-underscore methods doesn't expose implementation. (It might, however, require the C implementation to set some extra fields just to simulate having a __iter__ when it really relies on tpIter. No big deal.) Now, as Greg has pointed out, hashability cannot be tested in this manner: > since object has __hash__ we end up concluding that everything is > hashable Fortunately, I (and apparently others) believe this limitation is OK because there is no need to test for hashability without evoking the side effects. In general, __hash__ should be "quick" (which may be a relative term) and return the same value each time it is called, so simply calling __hash__ to find whether it works ought to do nicely. --- Much earlier, Andrew had also asked me this: > Perhaps we are using "idempotent" to mean different things. I think of a > function f as being idempotent if f(x) == x for all x in dom(f). Apparently so. In mathematics, the term means a function where f(f(x)) = f(x) for all x. But in programming I have always understood it to mean a function which will return the same result no matter how many times it is invoked. In a quick glance at internet dictionaries I found surprisingly little support for this definition, but I still believe it is a fairly widely-used definition as I have heard it used by co-workers from more than one job. The one source I DID find was "http://en.wiktionary.org/wiki/idempotent". -- Michael Chermside From ark-mlist at att.net Thu Jul 27 00:02:28 2006 From: ark-mlist at att.net (Andrew Koenig) Date: Wed, 26 Jul 2006 18:02:28 -0400 Subject: [Python-3000] callable() In-Reply-To: <20060726134329.er27og60l0w0wcww@login.werra.lunarpages.com> Message-ID: <004501c6b0ff$360c4280$6402a8c0@arkdesktop> > > I felt uncomfortable about exposing the implementation that way > ...but double-underscore methods are part of the language definition, > not part of the implementation. Yes and no. For example, it appears to be part of the language definition that foo() is equivalent to foo.__call__(), but apparently not every object that has a __hash__ attribute is hashable. > > since object has __hash__ we end up concluding that everything is > > hashable > Fortunately, I (and apparently others) believe this limitation is > OK because there is no need to test for hashability without evoking > the side effects. In general, __hash__ should be "quick" (which may > be a relative term) and return the same value each time it is > called, so simply calling __hash__ to find whether it works ought > to do nicely. Indeed -- but I think this statement is still pulling us away from my original observation, which I might state this way: Some objects support particular capabilities. That is, you can do particular things with those objects and not others. I think it would be useful if there were a way to find out whether an object has a given capability that is * Uniform -- the general form of the inquiry does not depend on the particular capability being inquired about * Free of side effects My general experience has been that such inquiries should not be coupling too closely to the details of the language definition, because such coupling makes it more difficult to maintain uniformity over time. One of my concerns about hasattr(foo, '__call__') is that it is not uniform: You can't test hashability by calling hasattr(foo, '__hash__'). Another is that it is too closely coupled. For example, if foo.__call__ exists but is None, is foo callable? Apparently this is an issue that doesn't concern most of the people on this list, in which case I'll just chalk it up to different people having different backgrounds. Nevertheless, I feel that removing callable() from Python-3000 is a step backward. From greg.ewing at canterbury.ac.nz Thu Jul 27 03:04:02 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 27 Jul 2006 13:04:02 +1200 Subject: [Python-3000] callable() In-Reply-To: References: <038001c6b0ae$734f1b00$d503030a@trilan> <001501c6b0d1$89973780$6402a8c0@arkdesktop> Message-ID: <44C81102.6080604@canterbury.ac.nz> Guido van Rossum wrote: > I'd be happy to extend the convention to all such attributes -- > setting it to None to mean that the subclass doesn't want to provide > it. That's clean, can't possibly be interpreted to mean anything else, > and doesn't require you to actually call the attribute. Although unless there's some special casing for it in the interpreter, attempting to use such an attribute will give a somewhat confusing error message -- something like "Object of type NoneType is not callable" instead of "Object of type has no __xxx__ attribute". From g.brandl at gmx.net Thu Jul 27 10:41:07 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 27 Jul 2006 10:41:07 +0200 Subject: [Python-3000] callable() In-Reply-To: <44C81102.6080604@canterbury.ac.nz> References: <038001c6b0ae$734f1b00$d503030a@trilan> <001501c6b0d1$89973780$6402a8c0@arkdesktop> <44C81102.6080604@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Guido van Rossum wrote: > >> I'd be happy to extend the convention to all such attributes -- >> setting it to None to mean that the subclass doesn't want to provide >> it. That's clean, can't possibly be interpreted to mean anything else, >> and doesn't require you to actually call the attribute. > > Although unless there's some special casing for it in the > interpreter, attempting to use such an attribute will > give a somewhat confusing error message -- something > like "Object of type NoneType is not callable" instead > of "Object of type has no __xxx__ attribute". Only if the slot isn't automatically set to NULL iff the corresponding attribute is set to None. (The "no __xxx__ attribute" error message doesn't make sense then either, but I think the real messages are more in the style of "xxx is not hashable"). Georg From ncoghlan at gmail.com Thu Jul 27 12:36:36 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 27 Jul 2006 20:36:36 +1000 Subject: [Python-3000] callable() In-Reply-To: References: <44C2D14F.5020906@canterbury.ac.nz> <001201c6ae59$945d1b50$6402a8c0@arkdesktop> <44C4264C.9030000@canterbury.ac.nz> <44C74325.5030504@gmail.com> Message-ID: <44C89734.9000204@gmail.com> Guido van Rossum wrote: > On 7/26/06, Nick Coghlan wrote: >> class Unhashable(object): >> __hash__ = Undefined > > Simpler: set it to None. That could be a convention. > And someone else pointed out that this could still be a trigger for NULL'ing out slots at the C level, since None is a compiler-enforced singleton. A direct check for callability/hashability/whateverability gets more verbose though: getattr(obj, "__hash__", None) is not None The upside is that it would be simple to embody the convention in a couple of functions: def getslotattr(obj, slotname): attr = getattr(obj, slotname) if attr is None: raise AttributeError("meaningful error message") return attr def hasslotattr(obj, slotname): return getattr(obj, slotname, None) is not None Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From murman at gmail.com Thu Jul 27 16:06:38 2006 From: murman at gmail.com (Michael Urman) Date: Thu, 27 Jul 2006 09:06:38 -0500 Subject: [Python-3000] callable() In-Reply-To: <004501c6b0ff$360c4280$6402a8c0@arkdesktop> References: <20060726134329.er27og60l0w0wcww@login.werra.lunarpages.com> <004501c6b0ff$360c4280$6402a8c0@arkdesktop> Message-ID: On 7/26/06, Andrew Koenig wrote: > * Uniform -- the general form of the inquiry does not > depend on the particular capability being inquired about While this makes sense from the perspective you mention, paraphrased as "different objects have different capabilities, and I want to query what capabilities this object has," I'm not convinced that any but the most uncommon uses involve enumerating the capabilities of an object. And those users are likely to be experts of the implementation. I would expect that code is more often concerned with checking no more than one or two capabilities, letting the implicit check of a raised exception do the rest. Since the times you would use __call__ and the times you would use __hash__ are so different, it seems reasonable to me that the corresponding capability check might be different. Case in point, I almost never use hash() directly; I delegate that to the dict I'm using. Whether the check were spelled "type(obj).__hash__ is not None" or "hasattr(type(obj).__hash__, "__call__")" or "try: ... except ...: ..." I wouldn't care; I've never needed to write his myself, and don't expect to. Is there code that needs to check the hashability? Sure. Does this code also need to check callability? Maybe. How about addability? Subtractibility? I doubt it. > * Free of side effects This is an ideal I agree with for some uses, although again I prefer the software version of idempotence (calling it once or many times has the same result) to side-effect-free. For many capabilities, the underlying implementation is expected to be idempotent, so I see no reason to require a separate idempotent check. We're adults here; if an implementation doesn't follow convention, it does so for a reason, and accepts the risk that someone may accidentally use it the conventional way. Michael -- Michael Urman http://www.tortall.net/mu/blog From ark-mlist at att.net Thu Jul 27 17:07:19 2006 From: ark-mlist at att.net (Andrew Koenig) Date: Thu, 27 Jul 2006 11:07:19 -0400 Subject: [Python-3000] callable() In-Reply-To: Message-ID: <002a01c6b18e$5b0c6850$6402a8c0@arkdesktop> > While this makes sense from the perspective you mention, paraphrased > as "different objects have different capabilities, and I want to query > what capabilities this object has," I'm not convinced that any but the > most uncommon uses involve enumerating the capabilities of an object. > And those users are likely to be experts of the implementation. Doesn't matter. The point is not whether I know how to test for a capability, but whether the test will continue to work in future implementations. Having an explicitly defined abstraction for such a test is useful for that reason (among others). > I would expect that code is more often concerned with checking no more > than one or two capabilities, letting the implicit check of a raised > exception do the rest. Since the times you would use __call__ and the > times you would use __hash__ are so different, it seems reasonable to > me that the corresponding capability check might be different. Most code is concerned with checking zero capabilities. > Case in point, I almost never use hash() directly; I delegate that to > the dict I'm using. Whether the check were spelled "type(obj).__hash__ > is not None" or "hasattr(type(obj).__hash__, "__call__")" or "try: ... > except ...: ..." I wouldn't care; I've never needed to write his > myself, and don't expect to. I think you've just proved my point. Do the expressions you've written actually check for hashability? Are you sure? Try it with obj=[42] > Is there code that needs to check the hashability? Sure. Does this > code also need to check callability? Maybe. How about addability? > Subtractibility? I doubt it. On the other hand, once we have the notion of capability, it is not hard to think of other capabilities that objects might have that are not easily testable at present. One example that comes to mind is "< defines a total ordering over objects of this type." That's something that's not automatically testable, but I could imagine it being provided by convention as part of a library. > > * Free of side effects > This is an ideal I agree with for some uses, although again I prefer > the software version of idempotence (calling it once or many times has > the same result) to side-effect-free. For many capabilities, the > underlying implementation is expected to be idempotent, so I see no > reason to require a separate idempotent check. We're adults here; if > an implementation doesn't follow convention, it does so for a reason, > and accepts the risk that someone may accidentally use it the > conventional way. Whenever there are side effects, it is nice to have a way of deferring those side effects until it is absolutely impossible to avoid them. From steven.bethard at gmail.com Thu Jul 27 17:49:49 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Thu, 27 Jul 2006 09:49:49 -0600 Subject: [Python-3000] object.__hash__() Message-ID: [Guido van Rossum] > I propose to take the same approach as for callable: if it has > __hash__ we consider it hashable even though the hash may fail [Greg Ewing] > Fair enough, although since object has __hash__ > we end up concluding that everything is > hashable except when it isn't. :-) [Guido van Rossum] > Ouch, that's a problem indeed. There's not really a clean way out > unfortunately: either we remove object.__hash__ and let hash() do the > default implementation, and then presence of __hash__ is no longer an > indicator of hashability; or we keep it, and override it to raise an > exception in unhashable types (like list.__hash__ does); or we remove > the hashability of objects by default, which lets us solve the above > problems at the cost of having a convenient hash by default. If I understand the problem right, the reason for not wanting to remove __hash__ from object is that it's often convenient to have a dict or set where objects are compared by id() instead of by value. One alternative that I didn't see mentioned is to drop object.__hash__() and introduce something like collections.iddict() and collections.idset() for these purposes. Basically, we'd be moving the "default implementation" of hash() into the specific classes that needed it. I guess you might then need to add id* variants of some user-defined collection classes. How big of an issue this is depends on how common it is to create a user-defined collection class that (1) requires hashing and (2) needs to be able to compare objects either by value or id(). STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From guido at python.org Thu Jul 27 18:14:47 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 27 Jul 2006 09:14:47 -0700 Subject: [Python-3000] object.__hash__() In-Reply-To: References: Message-ID: On 7/27/06, Steven Bethard wrote: > If I understand the problem right, the reason for not wanting to > remove __hash__ from object is that it's often convenient to have a > dict or set where objects are compared by id() instead of by value. No, not at all. This is for objects whose __eq__ is also the default (which happens to compare by id). Such objects aren't considered to have a "value" in the sense that a list or dict or int has a value, because whatever attributes they have aren't considered for comparison. Each instance of such classes is a unique value. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Thu Jul 27 18:23:38 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Thu, 27 Jul 2006 10:23:38 -0600 Subject: [Python-3000] object.__hash__() In-Reply-To: References: Message-ID: On 7/27/06, Guido van Rossum wrote: > On 7/27/06, Steven Bethard wrote: > > If I understand the problem right, the reason for not wanting to > > remove __hash__ from object is that it's often convenient to have a > > dict or set where objects are compared by id() instead of by value. > > No, not at all. This is for objects whose __eq__ is also the default [snip] So what's the problem with losing the default hash then? I thought the default hash was basically just hash(id(obj)):: >>> class C(object): ... pass ... >>> hash(C) 13565672 >>> hash(id(C)) 13565672 >>> c = C() >>> hash(c) 15130928 >>> hash(id(c)) 15130928 If object.__hash__() disappeared, the hash(obj) versions would fail, but the same results seem to be available as hash(id(obj)). Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From guido at python.org Thu Jul 27 18:27:56 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 27 Jul 2006 09:27:56 -0700 Subject: [Python-3000] callable() In-Reply-To: <002a01c6b18e$5b0c6850$6402a8c0@arkdesktop> References: <002a01c6b18e$5b0c6850$6402a8c0@arkdesktop> Message-ID: So how about we change callable() and add hashable(), iterable() and whatever else makes sense so that these all become like this: def callable(x): return getattr(x, "__call__", None) is not None This way classes can explicitly declare that they aren't callable, hashable, iterable etc. by setting the corresponding attribute to None. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Jul 27 18:27:56 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 27 Jul 2006 09:27:56 -0700 Subject: [Python-3000] callable() In-Reply-To: <002a01c6b18e$5b0c6850$6402a8c0@arkdesktop> References: <002a01c6b18e$5b0c6850$6402a8c0@arkdesktop> Message-ID: So how about we change callable() and add hashable(), iterable() and whatever else makes sense so that these all become like this: def callable(x): return getattr(x, "__call__", None) is not None This way classes can explicitly declare that they aren't callable, hashable, iterable etc. by setting the corresponding attribute to None. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Jul 27 19:56:06 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 27 Jul 2006 10:56:06 -0700 Subject: [Python-3000] callable() In-Reply-To: <44C81102.6080604@canterbury.ac.nz> References: <038001c6b0ae$734f1b00$d503030a@trilan> <001501c6b0d1$89973780$6402a8c0@arkdesktop> <44C81102.6080604@canterbury.ac.nz> Message-ID: On 7/26/06, Greg Ewing wrote: > Guido van Rossum wrote: > > I'd be happy to extend the convention to all such attributes -- > > setting it to None to mean that the subclass doesn't want to provide > > it. That's clean, can't possibly be interpreted to mean anything else, > > and doesn't require you to actually call the attribute. > > Although unless there's some special casing for it in the > interpreter, attempting to use such an attribute will > give a somewhat confusing error message -- something > like "Object of type NoneType is not callable" instead > of "Object of type has no __xxx__ attribute". Yes, of course something like that should be done. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Thu Jul 27 20:27:12 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Thu, 27 Jul 2006 12:27:12 -0600 Subject: [Python-3000] callable() In-Reply-To: References: <002a01c6b18e$5b0c6850$6402a8c0@arkdesktop> Message-ID: On 7/27/06, Guido van Rossum wrote: > So how about we change callable() and add hashable(), iterable() and > whatever else makes sense so that these all become like this: > > def callable(x): > return getattr(x, "__call__", None) is not None > > This way classes can explicitly declare that they aren't callable, > hashable, iterable etc. by setting the corresponding attribute to > None. FWLIW, I'm still about -0.5 on this. If you're defining a new class, this gets you nothing as far as __iter__() and __call__() go -- just don't define the method if you don't want it [1]. And how often do you really want to define a subclass of an iterable or callable class that is not iterable or callable? Maybe someone could post some real-world use cases? I don't see what the added complexity is buying us. I also worry that it might set a bad precedent. Should we now also add getitemable() and getattrable()? Maybe there's a good rule of thumb for which *able() functions are needed, but I couldn't figure one out. STeVe [1] It might get you something for __hash__() if object.__hash__() is not removed, but that debate still seems to be in progress and I'm not sure it's worth the added complexity to handle one special case. -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From guido at python.org Fri Jul 28 01:33:32 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 27 Jul 2006 16:33:32 -0700 Subject: [Python-3000] callable() In-Reply-To: References: <002a01c6b18e$5b0c6850$6402a8c0@arkdesktop> Message-ID: On 7/27/06, Steven Bethard wrote: > [1] It might get you something for __hash__() if object.__hash__() is > not removed, but that debate still seems to be in progress and I'm not > sure it's worth the added complexity to handle one special case. FWIW, I don't believe your "solution" to remove __hash__ and supply an id-dict will work. Unless someone else supports it, it's off the table as far as I'm concerned. Sorry, I can't exactly explain why, but my gut tells me that. Maybe someone else can explain it to you. Or maybe someone else can provide better support for your idea. By default, I consider it dead. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Fri Jul 28 02:54:12 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 28 Jul 2006 12:54:12 +1200 Subject: [Python-3000] object.__hash__() In-Reply-To: References: Message-ID: <44C96034.40204@canterbury.ac.nz> Steven Bethard wrote: > One alternative that I didn't see mentioned is to drop > object.__hash__() and introduce something like collections.iddict() > and collections.idset() for these purposes. -1. Too big a change for something we're fairly sure isn't a real problem in the first place. -- Greg From greg.ewing at canterbury.ac.nz Fri Jul 28 03:00:58 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 28 Jul 2006 13:00:58 +1200 Subject: [Python-3000] callable() In-Reply-To: References: <002a01c6b18e$5b0c6850$6402a8c0@arkdesktop> Message-ID: <44C961CA.3040402@canterbury.ac.nz> Guido van Rossum wrote: > So how about we change callable() and add hashable(), iterable() and > whatever else makes sense so that these all become like this: > > def callable(x): > return getattr(x, "__call__", None) is not None This *still* doesn't fully solve the problem in the case of __hash__, since a container may be unhashable due to its contents, even if it has a __hash__ method itself. It might be good enough for whatever these hypothetical hashability-testing use cases require though. We'd have to catch one of those elusive beasts to find out... -- Greg From guido at python.org Fri Jul 28 03:05:18 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 27 Jul 2006 18:05:18 -0700 Subject: [Python-3000] callable() In-Reply-To: <44C961CA.3040402@canterbury.ac.nz> References: <002a01c6b18e$5b0c6850$6402a8c0@arkdesktop> <44C961CA.3040402@canterbury.ac.nz> Message-ID: On 7/27/06, Greg Ewing wrote: > Guido van Rossum wrote: > > So how about we change callable() and add hashable(), iterable() and > > whatever else makes sense so that these all become like this: > > > > def callable(x): > > return getattr(x, "__call__", None) is not None > > This *still* doesn't fully solve the problem in the case > of __hash__, since a container may be unhashable due to > its contents, even if it has a __hash__ method itself. > > It might be good enough for whatever these hypothetical > hashability-testing use cases require though. We'd have > to catch one of those elusive beasts to find out... I already proposed to deal with that the same way we handle signature mismatches with __call__ -- you'll find out when you actually try to do it. That should be good enough. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Fri Jul 28 03:53:29 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Thu, 27 Jul 2006 19:53:29 -0600 Subject: [Python-3000] callable() In-Reply-To: <44C961CA.3040402@canterbury.ac.nz> References: <002a01c6b18e$5b0c6850$6402a8c0@arkdesktop> <44C961CA.3040402@canterbury.ac.nz> Message-ID: On 7/27/06, Greg Ewing wrote: > Guido van Rossum wrote: > > So how about we change callable() and add hashable(), iterable() and > > whatever else makes sense so that these all become like this: > > > > def callable(x): > > return getattr(x, "__call__", None) is not None > > This *still* doesn't fully solve the problem in the case > of __hash__, since a container may be unhashable due to > its contents, even if it has a __hash__ method itself. For that matter, it doesn't solve the problem for __call__ or __iter__ either since they could also raise TypeErrors. Any of these method-checking approaches can only tell you if an object *doesn't* support operation X. Their return values should really be read as: False -- definitely doesn't support operation X True -- could possibly support operation X Maybe we should reverse the polarity and rename them to isdefinitelynotcallable(), isdefinitelynotiterable(), etc.? ;-) STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From behnel_ml at gkec.informatik.tu-darmstadt.de Fri Jul 28 07:04:42 2006 From: behnel_ml at gkec.informatik.tu-darmstadt.de (Stefan Behnel) Date: Fri, 28 Jul 2006 07:04:42 +0200 Subject: [Python-3000] callable() In-Reply-To: References: <002a01c6b18e$5b0c6850$6402a8c0@arkdesktop> <44C961CA.3040402@canterbury.ac.nz> Message-ID: <44C99AEA.8020500@gkec.informatik.tu-darmstadt.de> Steven Bethard wrote: > On 7/27/06, Greg Ewing wrote: >> Guido van Rossum wrote: >>> So how about we change callable() and add hashable(), iterable() and >>> whatever else makes sense so that these all become like this: >>> >>> def callable(x): >>> return getattr(x, "__call__", None) is not None >> This *still* doesn't fully solve the problem in the case >> of __hash__, since a container may be unhashable due to >> its contents, even if it has a __hash__ method itself. > > For that matter, it doesn't solve the problem for __call__ or __iter__ > either since they could also raise TypeErrors. Any of these > method-checking approaches can only tell you if an object *doesn't* > support operation X. What they tell you is that the object supports the /protocol/. It does not mean that everything from the protocol will succeed when you try. It does not tell you about the /state/ of the object or its environment that might let the operation fail (IOError etc.). In the case of iterable(), it does not tell you that there is something to iterate over or that things like file-iteration will not fail. In the case of callable(), it does not tell you what the correct signature is or if the operation triggered by the call will succeed. In the case of hashable(), it does not tell you that the entire thing (including subtrees or whatever) is hashable in its current state at the time of hashing. It just says: I support the protocol, so it makes sense to apply the protocol operations to me if you need to. I think that's perfectly reasonable semantics for the three functions. So, with the small enhancement above that allows setting the special methods to None to deliberately /disable/ support for the protocol in an object (or class), I think the semantics of the *able() functions are very clear and clean. Stefan From qrczak at knm.org.pl Fri Jul 28 09:19:42 2006 From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: Fri, 28 Jul 2006 09:19:42 +0200 Subject: [Python-3000] callable() In-Reply-To: <44C99AEA.8020500@gkec.informatik.tu-darmstadt.de> (Stefan Behnel's message of "Fri, 28 Jul 2006 07:04:42 +0200") References: <002a01c6b18e$5b0c6850$6402a8c0@arkdesktop> <44C961CA.3040402@canterbury.ac.nz> <44C99AEA.8020500@gkec.informatik.tu-darmstadt.de> Message-ID: <87ejw6qjqp.fsf@qrnik.zagroda> Stefan Behnel writes: > It just says: I support the protocol, so it makes sense to apply the > protocol operations to me if you need to. > > I think that's perfectly reasonable semantics for the three functions. I see no essential difference between an object which doesn't support a protocol at all, and an object for which using the given protocol always fails. What is hashable() useful for, especially if so loosely defined? -- __("< Marcin Kowalczyk \__/ qrczak at knm.org.pl ^^ http://qrnik.knm.org.pl/~qrczak/ From behnel_ml at gkec.informatik.tu-darmstadt.de Fri Jul 28 10:56:01 2006 From: behnel_ml at gkec.informatik.tu-darmstadt.de (Stefan Behnel) Date: Fri, 28 Jul 2006 10:56:01 +0200 Subject: [Python-3000] callable() In-Reply-To: <87ejw6qjqp.fsf@qrnik.zagroda> References: <002a01c6b18e$5b0c6850$6402a8c0@arkdesktop> <44C961CA.3040402@canterbury.ac.nz> <44C99AEA.8020500@gkec.informatik.tu-darmstadt.de> <87ejw6qjqp.fsf@qrnik.zagroda> Message-ID: <44C9D121.10104@gkec.informatik.tu-darmstadt.de> Marcin 'Qrczak' Kowalczyk wrote: > Stefan Behnel writes: > >> It just says: I support the protocol, so it makes sense to apply the >> protocol operations to me if you need to. >> >> I think that's perfectly reasonable semantics for the three functions. > > I see no essential difference between an object which doesn't support > a protocol at all, and an object for which using the given protocol > always fails. > > What is hashable() useful for, especially if so loosely defined? Well, if you /know/ it always fails, you should best disable it by setting it to None, right? So, if hashable() returns True, it means that a) the designer didn't care about switching it off or b) there are cases where it works, so it makes sense to try calling it. The first case (i.e. the one you described) is bad design and therefore ignorable, the second fits the above semantics. Stefan From qrczak at knm.org.pl Fri Jul 28 13:45:13 2006 From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: Fri, 28 Jul 2006 13:45:13 +0200 Subject: [Python-3000] callable() In-Reply-To: <44C9D121.10104@gkec.informatik.tu-darmstadt.de> (Stefan Behnel's message of "Fri, 28 Jul 2006 10:56:01 +0200") References: <002a01c6b18e$5b0c6850$6402a8c0@arkdesktop> <44C961CA.3040402@canterbury.ac.nz> <44C99AEA.8020500@gkec.informatik.tu-darmstadt.de> <87ejw6qjqp.fsf@qrnik.zagroda> <44C9D121.10104@gkec.informatik.tu-darmstadt.de> Message-ID: <87d5bq54xi.fsf@qrnik.zagroda> Stefan Behnel writes: > So, if hashable() returns True, it means that a) the designer didn't > care about switching it off or b) there are cases where it works, so > it makes sense to try calling it. If the code can successfully work with either hashable or non-hashable objects, it can just always try to call it (and deal with potential failure), rather than test first and then check for failure anyway. Let me repeat the question: what is hashable() useful for? Please describe how would you use it, and I will reply with how to do the same thing simpler, without hashable(), or why it was a bad idea to begin with. It's not much better with callable() and iterable(). The only use I can see is overloading (a parameter plays different roles depending on its type); the nature of these predicates makes this usage fragile (adding a seemingly unimportant __call__ definition to a class breaks code unnecessarily), and the roles should better be specified more explicitly. -- __("< Marcin Kowalczyk \__/ qrczak at knm.org.pl ^^ http://qrnik.knm.org.pl/~qrczak/ From ark-mlist at att.net Fri Jul 28 16:20:13 2006 From: ark-mlist at att.net (Andrew Koenig) Date: Fri, 28 Jul 2006 10:20:13 -0400 Subject: [Python-3000] callable() In-Reply-To: <44C961CA.3040402@canterbury.ac.nz> Message-ID: <002a01c6b250$f15eaf80$6402a8c0@arkdesktop> > Guido van Rossum wrote: > > So how about we change callable() and add hashable(), iterable() and > > whatever else makes sense so that these all become like this: > > > > def callable(x): > > return getattr(x, "__call__", None) is not None > This *still* doesn't fully solve the problem in the case > of __hash__, since a container may be unhashable due to > its contents, even if it has a __hash__ method itself. Methinks that the suggestion includes defining hashable(x) in such a way as to give the right answer, irrespective of how x.__hash__ is defined. From ncoghlan at gmail.com Fri Jul 28 16:23:45 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 29 Jul 2006 00:23:45 +1000 Subject: [Python-3000] callable() In-Reply-To: <87d5bq54xi.fsf@qrnik.zagroda> References: <002a01c6b18e$5b0c6850$6402a8c0@arkdesktop> <44C961CA.3040402@canterbury.ac.nz> <44C99AEA.8020500@gkec.informatik.tu-darmstadt.de> <87ejw6qjqp.fsf@qrnik.zagroda> <44C9D121.10104@gkec.informatik.tu-darmstadt.de> <87d5bq54xi.fsf@qrnik.zagroda> Message-ID: <44CA1DF1.7040204@gmail.com> Marcin 'Qrczak' Kowalczyk wrote: > Stefan Behnel writes: > >> So, if hashable() returns True, it means that a) the designer didn't >> care about switching it off or b) there are cases where it works, so >> it makes sense to try calling it. > > If the code can successfully work with either hashable or non-hashable > objects, it can just always try to call it (and deal with potential > failure), rather than test first and then check for failure anyway. The use cases are where the check and the subsequent usage are in different parts of the code - by doing the check early, you can flag some errors near the point where they are introduced (i.e. the object passed in definitely doesn't support the needed protocol), rather than doing it later where the traceback will actually lead you on a wild goose chase. This case is strongest for callable() - calling an object early generally won't be an option, especially if it is a callback that should only be called after a particular event has occurred. Note that in the case of a bug in the callback function itself, the traceback will start from inside the function, but if the callback isn't actually callable, you have to somehow figure out which code was responsible for registering the callback in the first place. Flagging that something that definitely isn't callable has been passed as a callback parameter at the time it is passed in, rather than when the callback is invoked, will make such errors far easier to resolve. Use cases for a "may be hashable" pretest are much weaker (and typically hypothetical), but there are cases where it makes a certain amount of sense. For example, if you have a set-based fast path if all the objects being handled are hashable, and a list-based slow path if one or more aren't hashable, and the hashes themselves may be expensive to calculate (e.g. some of the objects may be large strings) then it may make sense to perform a precheck to ensure that all of the objects are at least *potentially* hashable before you try to put any of them into a set. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Fri Jul 28 16:35:46 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 29 Jul 2006 00:35:46 +1000 Subject: [Python-3000] callable() In-Reply-To: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> References: <001c01c6a9bc$d09be020$6402a8c0@arkdesktop> Message-ID: <44CA20C2.1080305@gmail.com> Andrew Koenig wrote: > I note in PEP 3000 the proposal to remove callable(), with the comment "just > call the object and catch the exception." > > I think that's a bad idea, because it takes away the ability to separate the > callability test from the first call. As a simple example, suppose you're > writing a function that you expect to be given a function as one of its > arguments: > > def foo(bar, fun): > assert callable(fun) > # ... > > It might be that foo doesn't actually call fun until much later. > Nevertheless, from a testing viewpoint, it would be better to detect the > error immediately of passing something that can't be called. > > If you didn't have callable, how would you write this example? How about changing callable's semantics to something like the following?: def callable(obj, attr="__call__"): if attr != "__call__": obj = getattr(obj, attr, None) return getattr(obj, "__call__", None) is not None Test for callability: assert callable(obj) Test for hashability: assert callable(obj, "__hash__") Test for iterability: assert callable(obj, "__iter__") Test for a context manager: assert callable(obj, "__enter__") and callable(obj, "__exit__") Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Fri Jul 28 16:38:52 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 29 Jul 2006 00:38:52 +1000 Subject: [Python-3000] object.__hash__() In-Reply-To: References: Message-ID: <44CA217C.9030407@gmail.com> Steven Bethard wrote: > If object.__hash__() disappeared, the hash(obj) versions would fail, > but the same results seem to be available as hash(id(obj)). Not in Jython - Jython's id() is expensive so the default hash() doesn't use it. Last time the idea of getting rid of object.__hash__() came up, Guido eventually killed the idea because nobody could come up with an acceptable alternate spelling for the default hash. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From jcarlson at uci.edu Fri Jul 28 18:01:48 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Fri, 28 Jul 2006 09:01:48 -0700 Subject: [Python-3000] callable() In-Reply-To: <44CA1DF1.7040204@gmail.com> References: <87d5bq54xi.fsf@qrnik.zagroda> <44CA1DF1.7040204@gmail.com> Message-ID: <20060728084725.E607.JCARLSON@uci.edu> Nick Coghlan wrote: > Use cases for a "may be hashable" pretest are much weaker (and typically > hypothetical), but there are cases where it makes a certain amount of sense. > For example, if you have a set-based fast path if all the objects being > handled are hashable, and a list-based slow path if one or more aren't > hashable, and the hashes themselves may be expensive to calculate (e.g. some > of the objects may be large strings) then it may make sense to perform a > precheck to ensure that all of the objects are at least *potentially* hashable > before you try to put any of them into a set. Why doesn't it make sense to assume the "fast path" and on failure switch to the "slow path"? (assuming the "fast path" wins on big-O or even a nontrivial constant factor) Such a method can only, in the worst case, be twice as slow as the 'full knowledge zero overhead are all items hashable?' check. But since the evaluation went to the "slow path", the "fast path" overhead may be inconsequential. However, hashability checks on the "fast path" may not be inconsequential in relation to the normal running time of the "fast path". In that sense, hashable() is a waste of time, as it slows down the "fast path" by up to a constant, without improving the "slow path" by anything more than a constant. But those constants are different, relatively large in the case of the "fast path", relatively small in the case of the "slow path". Hashable is getting a big, fat, -1 from me. On the other hand, I'm a full supporter of callable(), and don't much care about iterable(), but leaning towards -1. - Josiah From guido at python.org Fri Jul 28 19:09:25 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 28 Jul 2006 10:09:25 -0700 Subject: [Python-3000] callable() In-Reply-To: <002a01c6b250$f15eaf80$6402a8c0@arkdesktop> References: <44C961CA.3040402@canterbury.ac.nz> <002a01c6b250$f15eaf80$6402a8c0@arkdesktop> Message-ID: On 7/28/06, Andrew Koenig wrote: > Methinks that the suggestion includes defining hashable(x) in such a way as > to give the right answer, irrespective of how x.__hash__ is defined. I'm getting tired of this discussion, so I'm just going to stop reading this thread. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Fri Jul 28 22:32:13 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Fri, 28 Jul 2006 14:32:13 -0600 Subject: [Python-3000] guessing a rule of thumb for methods on object Message-ID: On 7/27/06, Guido van Rossum wrote: > On 7/27/06, Steven Bethard wrote: > > [1] It might get you something for __hash__() if object.__hash__() is > > not removed, but that debate still seems to be in progress and I'm not > > sure it's worth the added complexity to handle one special case. > > FWIW, I don't believe your "solution" to remove __hash__ and supply an > id-dict will work. Unless someone else supports it, it's off the table > as far as I'm concerned. Sorry, I can't exactly explain why, but my > gut tells me that. I thought about this for a while, and I think maybe a good rule of thumb (and something like what Python seems to follow) is: "If it's there's an obvious default implementation of a protocol that would be generally useful, it should go on all objects." So, for example, the __hash__ method has a default implementation that's generally useful if you want to map unique objects to other values. So it makes sense to have the object.__hash__ method. And for the same reason, it also makes sense to have something like an object.__eq__ method[1]. OTOH, something like __call__ or __iter__ doesn't have an obvious default implementation -- e.g. what would it mean to call an integer? -- so those don't go on object. And dropping object.__cmp__[2] for Python 3000 makes sense because there isn't really an obvious default implementation -- there's no real reason why ``['a'] < 'a' < ('a',)`` ought to be True. STeVe [1] I know that since in Python 2.X some things aren't children of object, this is actually implemented in the C comparisons, but it's useful to think of it this way. Are there any plans for Python 3000 to move this behavior into object.__eq__? [2] See note 1. Yes, I know object.__cmp__ doesn't actually exist. -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From stian at soiland.no Fri Jul 28 23:57:11 2006 From: stian at soiland.no (Stian Soiland) Date: Fri, 28 Jul 2006 23:57:11 +0200 Subject: [Python-3000] Detecting conflicts in dict displays In-Reply-To: References: Message-ID: On 7/11/06, Terry Reedy wrote: > >>> d={1:'one', 1.0:'one'} > >>> d > {1: 'one'} This highlights the problem. Imagine that you don't type out the actual objects, but just receive them: def make_dict(x, y, obj): return {x: obj, y: obj} x1 = x2 = "fish" d = make_dict(x1, x2, 1337) Here I don't really care if a and b are the same or not, I just care that I can do d[x1] and d[x2]. (Of course this is a dangerous path to go down too deep, what about del d[x1] ? And who would make such code, anyway?) Maybe only for the literal values and direct variable names..? Is the following behaviour as expected, or is it just The Lock? import threading count = 0 def counter(): global count while True: count += 1 threading.Thread(target=counter).start() >>> {count-1+1: 1, count: 2, count-1+1: 3, count: 4, count: 5, count: 6, count: 7, count: 8, count-1+1: 9, count: 10, count: 11} {897203081: 11} -- Stian S?iland Any society that would give up a little Manchester, UK liberty to gain a little security will http://soiland.no/ deserve neither and lose both. [Franklin] =/\= From jcarlson at uci.edu Sat Jul 29 00:15:38 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Fri, 28 Jul 2006 15:15:38 -0700 Subject: [Python-3000] Detecting conflicts in dict displays In-Reply-To: References: Message-ID: <20060728151056.E61A.JCARLSON@uci.edu> "Stian Soiland" wrote: > > On 7/11/06, Terry Reedy wrote: > > > >>> d={1:'one', 1.0:'one'} > > >>> d > > {1: 'one'} > > This highlights the problem. Imagine that you don't type out the > actual objects, but just receive them: > > def make_dict(x, y, obj): > return {x: obj, y: obj} > > x1 = x2 = "fish" > d = make_dict(x1, x2, 1337) > > > Here I don't really care if a and b are the same or not, I just care > that I can do d[x1] and d[x2]. > > (Of course this is a dangerous path to go down too deep, what about > del d[x1] ? And who would make such code, anyway?) > > > Maybe only for the literal values and direct variable names..? > > Is the following behaviour as expected, or is it just The Lock? It's not related to the GIL at all, it just happens that dict creation occurred within an entire bytecode 'time slice' (sys.get/setcheckinterval()). >>> def foo(): ... x = {count-1+1: 1, count: 2, count-1+1: 3, count: 4, count: 5, ... count:6, count: 7, count: 8, count-1+1: 9, count: 10, count: 11} ... while len(x) == 1: ... x = {count-1+1: 1, count: 2, count-1+1: 3, count: 4, ... count: 5, count: 6, count: 7, count: 8, count-1+1: 9, count: 10, ... count: 11} ... return x ... >>> foo() {149703626: 11, 149703605: 1} - Josiah From greg.ewing at canterbury.ac.nz Sat Jul 29 05:28:15 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 29 Jul 2006 15:28:15 +1200 Subject: [Python-3000] guessing a rule of thumb for methods on object In-Reply-To: References: Message-ID: <44CAD5CF.9090706@canterbury.ac.nz> Steven Bethard wrote: > I thought about this for a while, and I think maybe a good rule of > thumb (and something like what Python seems to follow) is: "If it's > there's an obvious default implementation of a protocol that would be > generally useful, it should go on all objects." Please, no! One of the things I like about Python is that 'object' is very nearly a blank slate. I'd hate to see it cluttered up with a pile of methods that someone thought might be useful to someone sometime. -- Greg From steven.bethard at gmail.com Sat Jul 29 19:33:39 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Sat, 29 Jul 2006 11:33:39 -0600 Subject: [Python-3000] guessing a rule of thumb for methods on object In-Reply-To: <44CAD5CF.9090706@canterbury.ac.nz> References: <44CAD5CF.9090706@canterbury.ac.nz> Message-ID: On 7/28/06, Greg Ewing wrote: > Steven Bethard wrote: > > > I thought about this for a while, and I think maybe a good rule of > > thumb (and something like what Python seems to follow) is: "If it's > > there's an obvious default implementation of a protocol that would be > > generally useful, it should go on all objects." > > Please, no! > > One of the things I like about Python is that 'object' > is very nearly a blank slate. I'd hate to see it cluttered > up with a pile of methods that someone thought might be > useful to someone sometime. Of course not. That's the whole point. The only things that belong there are *protocols* (as in language-defined __XXX__ methods) that have a sensible default implementation for all objects. AFAICT, the only things this really applies to are __hash__ and __eq__ and the other things that are already there (__class__, __{get|set|del}attr__, etc.). STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From martin at v.loewis.de Sun Jul 30 10:13:50 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 30 Jul 2006 10:13:50 +0200 Subject: [Python-3000] gettype In-Reply-To: <1d85506f0607061119w1c3cab60o6f762a8e3849e45c@mail.gmail.com> References: <1d85506f0607061119w1c3cab60o6f762a8e3849e45c@mail.gmail.com> Message-ID: <44CC6A3E.8000003@v.loewis.de> tomer filiba schrieb: > so why not choose the "get%s()" notation? Why not o.__class__? Regards, Martin