From tjreedy at udel.edu Fri Jul 1 01:38:11 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 30 Jun 2011 19:38:11 -0400 Subject: [Python-ideas] dir with a glob? In-Reply-To: References: <4E0C601F.5010001@molden.no> <87liwjlcy2.fsf@benfinney.id.au> Message-ID: On 6/30/2011 8:41 AM, Joao S. O. Bueno wrote: > I believe a second parameter to dir, being a glob filter string, would be > fine. Being able to type: > > dir (gtk.Window, "set*") > > , for example > instead of > > [attrib for attrib in dir(gtk.Window) if attrib.startswith("set")] > > > would be indeed a good thing to have. Ihad much the same thought. I am still trying to see what is wrong with it. It might be worth a feature request. From ncoghlan at gmail.com Fri Jul 1 02:12:37 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 1 Jul 2011 10:12:37 +1000 Subject: [Python-ideas] dir with a glob? In-Reply-To: References: <4E0C601F.5010001@molden.no> <87liwjlcy2.fsf@benfinney.id.au> <4E0C79B4.3030209@molden.no> <87hb77l9me.fsf@benfinney.id.au> <4E0CA4EA.2010705@pearwood.info> Message-ID: On Fri, Jul 1, 2011 at 3:22 AM, Georg Brandl wrote: > On 30.06.2011 18:31, Steven D'Aprano wrote: >> dir is a convenience function, designed for interactive use. The docs >> make it explicit: >> >> [quote] >> Because dir() is supplied primarily as a convenience for use at an >> interactive prompt, it tries to supply an interesting set of names more >> than it tries to supply a rigorously or consistently defined set of names... >> >> http://docs.python.org/library/functions.html#dir >> >> >> Given that, I see no downside to making dir more convenient for >> interactive use. That's what it's for. > > I agree. ?I often am looking for a specific member that I know exists, but > don't recall the exact name (and in particular, not what the name starts > with: at least dir() output is sorted). ?Searching through one screenful > of members isn't pretty. > > So, +1 for the second argument. Yep, I found Steven's reply quite persuasive, so consider my objection withdrawn. And for interactive prompt usage, simple glob-style matching is a better choice than the more powerful re. However, this now needs a tracker issue and a patch - as a builtin that may legitimately be used before the import machinery is fully initialised, it isn't really acceptable for dir() to depend on the fnmatch module just for this feature, so implementing this isn't going to be quite as easy as it might otherwise be. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ben+python at benfinney.id.au Fri Jul 1 02:17:59 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 01 Jul 2011 10:17:59 +1000 Subject: [Python-ideas] dir with a glob? References: <4E0C601F.5010001@molden.no> <87liwjlcy2.fsf@benfinney.id.au> <4E0C79B4.3030209@molden.no> <4E0C9C35.4080405@pearwood.info> Message-ID: <87d3huluq0.fsf@benfinney.id.au> Steven D'Aprano writes: > Nick Coghlan wrote: > > On Thu, Jun 30, 2011 at 11:27 PM, Sturla Molden wrote: > >> What I am asking is if the need to filter the output from dir is so > >> common that it could warrant a change to Python? > > > > No, if an object is complicated enough that pprint(dir(obj)) and > > help(obj) aren't adequate to explore it, then it is time to go read > > the documentation (or the source, if the documentation is lacking). > > Do you think that *reading the source* is to be preferred over a > simple tool like running a filter over the output of dir()? I'm not > exactly sure what point you're trying to make, but I don't think it's > a good one. I understood Nick's meaning to entail that in such a circumstance where usage for a complex object is difficult to discover, the solution isn't to alter Python but to address the poor documentation and/or poor design of those complex objects. But that might be because that's *my* position. -- \ ?[F]reedom of speech does not entail freedom to have your ideas | `\ accepted by governments and incorporated into law and policy.? | _o__) ?Russell Blackford, 2010-03-06 | Ben Finney From ncoghlan at gmail.com Fri Jul 1 02:23:27 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 1 Jul 2011 10:23:27 +1000 Subject: [Python-ideas] dir with a glob? In-Reply-To: <87d3huluq0.fsf@benfinney.id.au> References: <4E0C601F.5010001@molden.no> <87liwjlcy2.fsf@benfinney.id.au> <4E0C79B4.3030209@molden.no> <4E0C9C35.4080405@pearwood.info> <87d3huluq0.fsf@benfinney.id.au> Message-ID: On Fri, Jul 1, 2011 at 10:17 AM, Ben Finney wrote: > Steven D'Aprano writes: > >> Nick Coghlan wrote: >> > On Thu, Jun 30, 2011 at 11:27 PM, Sturla Molden wrote: >> >> What I am asking is if the need to filter the output from dir is so >> >> common that it could warrant a change to Python? >> > >> > No, if an object is complicated enough that pprint(dir(obj)) and >> > help(obj) aren't adequate to explore it, then it is time to go read >> > the documentation (or the source, if the documentation is lacking). >> >> Do you think that *reading the source* is to be preferred over a >> simple tool like running a filter over the output of dir()? I'm not >> exactly sure what point you're trying to make, but I don't think it's >> a good one. > > I understood Nick's meaning to entail that in such a circumstance where > usage for a complex object is difficult to discover, the solution isn't > to alter Python but to address the poor documentation and/or poor design > of those complex objects. Yeah, that's about it - dir-based exploration hits a point of diminishing returns where it becomes better to go find a source of information that doesn't require you to explicitly request each piece of additional data. However, I found Steven and Georg's point about "how do I spell that again?" for objects with a broad API much more compelling, since I *have* been in that situation many, many times. It's not exploration in that case, it's a dir-based equivalent of code completion support. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ben+python at benfinney.id.au Fri Jul 1 02:27:52 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 01 Jul 2011 10:27:52 +1000 Subject: [Python-ideas] dir with a glob? References: <4E0C601F.5010001@molden.no> <4E0C759A.4000904@gmx.net> Message-ID: <877h82lu9j.fsf@benfinney.id.au> Georg Brandl writes: > On 30.06.2011 15:09, Mathias Panzenb?ck wrote: > > Then one could use it in a filter expression: > > > > filter(re.compile("^i"), dir(sp.fft)) > > What's wrong with > > filter(re.compile("^i").search, dir(sp.fft)) (I'm confused over this example, since there are no attributes of ?scipy.fft? that start with ?i?. I'll switch to an example where we're looking for its attributes that start with ?f?.) The globs people have shown are AFAICT easily done by the existing string methods, and don't need regex. So why not: >>> import scipy >>> [a for a in dir(scipy.fft) if a.startswith("f")] ['func_closure', 'func_code', 'func_defaults', 'func_dict', 'func_doc', 'func_globals', 'func_name'] -- \ ?I don't accept the currently fashionable assertion that any | `\ view is automatically as worthy of respect as any equal and | _o__) opposite view.? ?Douglas Adams | Ben Finney From ncoghlan at gmail.com Fri Jul 1 04:15:08 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 1 Jul 2011 12:15:08 +1000 Subject: [Python-ideas] dir with a glob? In-Reply-To: <877h82lu9j.fsf@benfinney.id.au> References: <4E0C601F.5010001@molden.no> <4E0C759A.4000904@gmx.net> <877h82lu9j.fsf@benfinney.id.au> Message-ID: On Fri, Jul 1, 2011 at 10:27 AM, Ben Finney wrote: >>>> import scipy >>>> [a for a in dir(scipy.fft) if a.startswith("f")] > ['func_closure', 'func_code', 'func_defaults', 'func_dict', 'func_doc', 'func_globals', 'func_name'] See Steven's quote about the reason for dir's very existence (i.e. as a convenience function). There's no real comparison from a usability point of view between the above and "dir(scipy.fft, 'f*')" fnmatch.filter(dir(scipy.fft), "f*") is even simpler than the list comprehension, but still no match for the builtin version. However, as I mentioned in my other email, the dependency problem means this is likely to be harder to implement than it first looks, so the next step if for someone that is sufficiently interested to create a patch. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From steve at pearwood.info Fri Jul 1 06:17:14 2011 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 01 Jul 2011 14:17:14 +1000 Subject: [Python-ideas] dir with a glob? In-Reply-To: <4E0C601F.5010001@molden.no> References: <4E0C601F.5010001@molden.no> Message-ID: <4E0D4A4A.6060903@pearwood.info> Sturla Molden wrote: > Often when exploring an object with the 'dir' function, particularly in > large packages like SciPy, I find that I need to filter the outout. > Since a dir reminds me of a dos 'dir' or linux 'ls', a glob feels like > the most natural to use. I have added a recipe for this here: http://code.activestate.com/recipes/577774-enhancing-dir-with-globs/ Comments and improvements welcome. -- Steven From steve at pearwood.info Fri Jul 1 06:43:37 2011 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 01 Jul 2011 14:43:37 +1000 Subject: [Python-ideas] dir with a glob? In-Reply-To: References: <4E0C601F.5010001@molden.no> <87liwjlcy2.fsf@benfinney.id.au> <4E0C79B4.3030209@molden.no> <87hb77l9me.fsf@benfinney.id.au> <4E0CA4EA.2010705@pearwood.info> Message-ID: <4E0D5079.20406@pearwood.info> Nick Coghlan wrote: > However, this now needs a tracker issue and a patch - as a builtin > that may legitimately be used before the import machinery is fully > initialised, it isn't really acceptable for dir() to depend on the > fnmatch module just for this feature, so implementing this isn't going > to be quite as easy as it might otherwise be. When does site.py run? If it is guaranteed to run after the import machinery is working, then perhaps the simplest solution is to patch built-ins in site.py, similar to how help() and quit() are added. -- Steven From stefan_ml at behnel.de Fri Jul 1 08:15:13 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 01 Jul 2011 08:15:13 +0200 Subject: [Python-ideas] dir with a glob? In-Reply-To: References: <4E0C601F.5010001@molden.no> <87liwjlcy2.fsf@benfinney.id.au> Message-ID: Terry Reedy, 01.07.2011 01:38: > On 6/30/2011 8:41 AM, Joao S. O. Bueno wrote: > >> I believe a second parameter to dir, being a glob filter string, would be >> fine. Being able to type: >> >> dir (gtk.Window, "set*") >> >> , for example >> instead of >> >> [attrib for attrib in dir(gtk.Window) if attrib.startswith("set")] >> >> >> would be indeed a good thing to have. > > Ihad much the same thought. I am still trying to see what is wrong with it. > It might be worth a feature request. Same from here, seems to be the most obvious way to do it. Stefan From g.brandl at gmx.net Fri Jul 1 08:26:08 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 01 Jul 2011 08:26:08 +0200 Subject: [Python-ideas] dir with a glob? In-Reply-To: References: <4E0C601F.5010001@molden.no> <87liwjlcy2.fsf@benfinney.id.au> <4E0C79B4.3030209@molden.no> <87hb77l9me.fsf@benfinney.id.au> <4E0CA4EA.2010705@pearwood.info> Message-ID: On 01.07.2011 02:12, Nick Coghlan wrote: > On Fri, Jul 1, 2011 at 3:22 AM, Georg Brandl wrote: >> On 30.06.2011 18:31, Steven D'Aprano wrote: >>> dir is a convenience function, designed for interactive use. The docs >>> make it explicit: >>> >>> [quote] >>> Because dir() is supplied primarily as a convenience for use at an >>> interactive prompt, it tries to supply an interesting set of names more >>> than it tries to supply a rigorously or consistently defined set of names... >>> >>> http://docs.python.org/library/functions.html#dir >>> >>> >>> Given that, I see no downside to making dir more convenient for >>> interactive use. That's what it's for. >> >> I agree. I often am looking for a specific member that I know exists, but >> don't recall the exact name (and in particular, not what the name starts >> with: at least dir() output is sorted). Searching through one screenful >> of members isn't pretty. >> >> So, +1 for the second argument. > > Yep, I found Steven's reply quite persuasive, so consider my objection > withdrawn. > > And for interactive prompt usage, simple glob-style matching is a > better choice than the more powerful re. > > However, this now needs a tracker issue and a patch - as a builtin > that may legitimately be used before the import machinery is fully > initialised, it isn't really acceptable for dir() to depend on the > fnmatch module just for this feature, so implementing this isn't going > to be quite as easy as it might otherwise be. Well, apparently someone called "Guido van Rossum" wrote a C implementation of fnmatch once, that is now included in all sorts of software (and in PHP): http://www.google.de/search?q=fnmatch+"guido+van+rossum" Georg From fuzzyman at gmail.com Mon Jul 4 00:47:01 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Sun, 3 Jul 2011 23:47:01 +0100 Subject: [Python-ideas] Allow isinstance second argument to be a set of types Message-ID: Hello all, Occasionally I build up tuples of types for passing as the second argument to isinstance. I've noticed on occasions having duplicates, for example the following code is Python 2 & 3 compatible but results in a duplicate on one of them: FunctionTypes = ( # python function type(create_autospec), # instance method type(ANY.__eq__), # unbound method type(_ANY.__eq__), ) It would feel cleaner to me if I could make FunctionTypes a set and use that as the second argument to isinstance. :-) Michael -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From robertc at robertcollins.net Mon Jul 4 01:04:09 2011 From: robertc at robertcollins.net (Robert Collins) Date: Mon, 4 Jul 2011 11:04:09 +1200 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: Message-ID: On Mon, Jul 4, 2011 at 10:47 AM, Michael Foord wrote: > Hello all, .. > It would feel cleaner to me if I could make FunctionTypes a set and use that > as the second argument to isinstance. :-) > Michael FWIW +1. From steve at pearwood.info Mon Jul 4 02:48:13 2011 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 04 Jul 2011 10:48:13 +1000 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: Message-ID: <4E110DCD.4020807@pearwood.info> Michael Foord wrote: > Hello all, > > Occasionally I build up tuples of types for passing as the second argument > to isinstance. I've noticed on occasions having duplicates, for example the > following code is Python 2 & 3 compatible but results in a duplicate on one > of them: [...] > It would feel cleaner to me if I could make FunctionTypes a set and use that > as the second argument to isinstance. :-) Does it matter if there is a duplicate entry? -- Steven From python at mrabarnett.plus.com Mon Jul 4 02:51:02 2011 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 04 Jul 2011 01:51:02 +0100 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: Message-ID: <4E110E76.6050803@mrabarnett.plus.com> On 03/07/2011 23:47, Michael Foord wrote: > Hello all, > > Occasionally I build up tuples of types for passing as the second > argument to isinstance. I've noticed on occasions having duplicates, for > example the following code is Python 2 & 3 compatible but results in a > duplicate on one of them: > > FunctionTypes = ( > # python function > type(create_autospec), > # instance method > type(ANY.__eq__), > # unbound method > type(_ANY.__eq__), > ) > > It would feel cleaner to me if I could make FunctionTypes a set and use > that as the second argument to isinstance. :-) > Why not just do FunctionTypes = tuple(set(FunctionTypes))? Or instead of just a tuple or set, an iterable. From greg.ewing at canterbury.ac.nz Mon Jul 4 09:02:03 2011 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 04 Jul 2011 19:02:03 +1200 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: Message-ID: <4E11656B.4050908@canterbury.ac.nz> Michael Foord wrote: > It would feel cleaner to me if I could make FunctionTypes a set and use > that as the second argument to isinstance. :-) Or more generally, any iterable object? -- Greg From vigneshbitmantra at gmail.com Mon Jul 4 14:31:17 2011 From: vigneshbitmantra at gmail.com (VIGNESH BALA) Date: Mon, 4 Jul 2011 18:01:17 +0530 Subject: [Python-ideas] python Message-ID: ideas abt python.. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gokcen at pardus.org.tr Mon Jul 4 17:53:52 2011 From: gokcen at pardus.org.tr (=?utf-8?q?G=C3=B6k=C3=A7en_Eraslan?=) Date: Mon, 4 Jul 2011 15:53:52 +0000 Subject: [Python-ideas] python In-Reply-To: References: Message-ID: <201107041553.55828.gokcen@pardus.org.tr> Pazartesi 04 Temmuz 2011 g?n? (saat 12:31:17) VIGNESH BALA ?unlar? yazm??t?: > ideas abt python.. FWIW +1. -- G?k?en Eraslan -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: From fuzzyman at gmail.com Mon Jul 4 16:35:55 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 4 Jul 2011 15:35:55 +0100 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: <4E11656B.4050908@canterbury.ac.nz> References: <4E11656B.4050908@canterbury.ac.nz> Message-ID: On 4 July 2011 08:02, Greg Ewing wrote: > Michael Foord wrote: > > It would feel cleaner to me if I could make FunctionTypes a set and use >> that as the second argument to isinstance. :-) >> > > Or more generally, any iterable object? > I don't care about any iterable. If a set would work then I'd be happy. :-) Michael > > -- > Greg > > ______________________________**_________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/**mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Mon Jul 4 17:28:43 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 4 Jul 2011 17:28:43 +0200 Subject: [Python-ideas] Allow isinstance second argument to be a set of types References: <4E11656B.4050908@canterbury.ac.nz> Message-ID: <20110704172843.50c5b445@pitrou.net> On Mon, 4 Jul 2011 15:35:55 +0100 Michael Foord wrote: > On 4 July 2011 08:02, Greg Ewing wrote: > > > Michael Foord wrote: > > > > It would feel cleaner to me if I could make FunctionTypes a set and use > >> that as the second argument to isinstance. :-) > >> > > > > Or more generally, any iterable object? > > I don't care about any iterable. If a set would work then I'd be happy. :-) Sure, but making it work for tuples and sets, but not other iterables, would be a pretty lousy API. Regards Antoine. From greg at krypto.org Mon Jul 4 19:51:01 2011 From: greg at krypto.org (Gregory P. Smith) Date: Mon, 4 Jul 2011 10:51:01 -0700 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: <20110704172843.50c5b445@pitrou.net> References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> Message-ID: On Mon, Jul 4, 2011 at 8:28 AM, Antoine Pitrou wrote: > On Mon, 4 Jul 2011 15:35:55 +0100 > Michael Foord wrote: > > On 4 July 2011 08:02, Greg Ewing wrote: > > > > > Michael Foord wrote: > > > > > > It would feel cleaner to me if I could make FunctionTypes a set and > use > > >> that as the second argument to isinstance. :-) > > >> > > > > > > Or more generally, any iterable object? > > > > I don't care about any iterable. If a set would work then I'd be happy. > :-) > > Sure, but making it work for tuples and sets, but not other iterables, > would be a pretty lousy API. > A set lookup should be faster than iteration for anything over ~3 types. I support the change even if it doesn't include all iterables. The use case for arbitrary iterables of types to isinstance seems low. > > Regards > > Antoine. > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg at krypto.org Mon Jul 4 19:52:37 2011 From: greg at krypto.org (Gregory P. Smith) Date: Mon, 4 Jul 2011 10:52:37 -0700 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> Message-ID: On Mon, Jul 4, 2011 at 10:51 AM, Gregory P. Smith wrote: > > > On Mon, Jul 4, 2011 at 8:28 AM, Antoine Pitrou wrote: > >> On Mon, 4 Jul 2011 15:35:55 +0100 >> Michael Foord wrote: >> > On 4 July 2011 08:02, Greg Ewing wrote: >> > >> > > Michael Foord wrote: >> > > >> > > It would feel cleaner to me if I could make FunctionTypes a set and >> use >> > >> that as the second argument to isinstance. :-) >> > >> >> > > >> > > Or more generally, any iterable object? >> > >> > I don't care about any iterable. If a set would work then I'd be happy. >> :-) >> >> Sure, but making it work for tuples and sets, but not other iterables, >> would be a pretty lousy API. >> > > A set lookup should be faster than iteration for anything over ~3 types. I > support the change even if it doesn't include all iterables. The use case > for arbitrary iterables of types to isinstance seems low. > note that a fast lookup implies exact type and not subclass making my point silly... at which point you're back to iterating so I suspect supporting arbitrary iterables is actually how this will be implemented regardless. > > >> >> Regards >> >> Antoine. >> >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Mon Jul 4 19:55:25 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 04 Jul 2011 19:55:25 +0200 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> Message-ID: <1309802125.3688.54.camel@localhost.localdomain> Le lundi 04 juillet 2011 ? 10:52 -0700, Gregory P. Smith a ?crit : > note that a fast lookup implies exact type and not subclass making my > point silly... at which point you're back to iterating so I suspect > supporting arbitrary iterables is actually how this will be > implemented regardless. Indeed. Note that the tuple case should remain fast, and therefore special-cased (or the general list/tuple case, since the difference in C is rather small). Regards Antoine. From amauryfa at gmail.com Mon Jul 4 21:41:33 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Mon, 4 Jul 2011 21:41:33 +0200 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: <1309802125.3688.54.camel@localhost.localdomain> References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> <1309802125.3688.54.camel@localhost.localdomain> Message-ID: Hi, 2011/7/4 Antoine Pitrou : > Le lundi 04 juillet 2011 ? 10:52 -0700, Gregory P. Smith a ?crit : >> note that a fast lookup implies exact type and not subclass making my >> point silly... at which point you're back to iterating so I suspect >> supporting arbitrary iterables is actually how this will be >> implemented regardless. > > Indeed. Note that the tuple case should remain fast, and therefore > special-cased (or the general list/tuple case, since the difference in C > is rather small). Arbitrary iterables, arbitrarily nested... beware of objects which are also their first element, like str('a')... -- Amaury Forgeot d'Arc From fuzzyman at gmail.com Mon Jul 4 22:09:11 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 4 Jul 2011 21:09:11 +0100 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: <20110704172843.50c5b445@pitrou.net> References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> Message-ID: On 4 July 2011 16:28, Antoine Pitrou wrote: > On Mon, 4 Jul 2011 15:35:55 +0100 > Michael Foord wrote: > > On 4 July 2011 08:02, Greg Ewing wrote: > > > > > Michael Foord wrote: > > > > > > It would feel cleaner to me if I could make FunctionTypes a set and > use > > >> that as the second argument to isinstance. :-) > > >> > > > > > > Or more generally, any iterable object? > > > > I don't care about any iterable. If a set would work then I'd be happy. > :-) > > Sure, but making it work for tuples and sets, but not other iterables, > would be a pretty lousy API. > I have no *objection* to it working for arbitrary iterables, I just don't care about it. So does this mean no-one has any objections to my suggested change? I'll create an issue. Michael > > Regards > > Antoine. > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Mon Jul 4 22:26:39 2011 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 4 Jul 2011 20:26:39 +0000 (UTC) Subject: [Python-ideas] =?utf-8?q?Allow_isinstance_second_argument_to_be_a?= =?utf-8?q?_set_of=09types?= References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> Message-ID: Michael Foord writes: > I have no *objection* to it working for arbitrary iterables, I just don't care about it. > > So does this mean no-one has any objections to my suggested change? I'll create an issue. Can you explain why tuple(x) is not sufficient? It seems like useless bloat to me. From fuzzyman at gmail.com Mon Jul 4 22:31:08 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 4 Jul 2011 21:31:08 +0100 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> Message-ID: On 4 July 2011 21:26, Benjamin Peterson wrote: > Michael Foord writes: > > I have no *objection* to it working for arbitrary iterables, I just don't > care > about it. > > > > So does this mean no-one has any objections to my suggested change? I'll > create an issue. > > > Can you explain why tuple(x) is not sufficient? > What is x in that case? Do you mean in the isinstance call? That adds the runtime overhead of an extra tuple call every time - plus the mental overhead of reading it every time someone looks at the code. Small maybe, but it seems like you're suggesting adding bloat... Seriously though. Conceptually the second argument to isinstance is a set of types you want to check the object against. It would be nice if that set of types were permitted to actually be a set. Michael > > It seems like useless bloat to me. > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Mon Jul 4 23:08:21 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 4 Jul 2011 23:08:21 +0200 Subject: [Python-ideas] Allow isinstance second argument to be a set of types References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> <1309802125.3688.54.camel@localhost.localdomain> Message-ID: <20110704230821.4b3ad626@pitrou.net> On Mon, 4 Jul 2011 21:41:33 +0200 "Amaury Forgeot d'Arc" wrote: > Hi, > > 2011/7/4 Antoine Pitrou : > > Le lundi 04 juillet 2011 ? 10:52 -0700, Gregory P. Smith a ?crit : > >> note that a fast lookup implies exact type and not subclass making my > >> point silly... at which point you're back to iterating so I suspect > >> supporting arbitrary iterables is actually how this will be > >> implemented regardless. > > > > Indeed. Note that the tuple case should remain fast, and therefore > > special-cased (or the general list/tuple case, since the difference in C > > is rather small). > > Arbitrary iterables, arbitrarily nested... > beware of objects which are also their first element, like str('a')... Ouch. I guess that shoots the proposal dead in the water, then. Regards Antoine. From jeanpierreda at gmail.com Mon Jul 4 23:46:36 2011 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Mon, 4 Jul 2011 17:46:36 -0400 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: <20110704230821.4b3ad626@pitrou.net> References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> <1309802125.3688.54.camel@localhost.localdomain> <20110704230821.4b3ad626@pitrou.net> Message-ID: On Mon, Jul 4, 2011 at 5:08 PM, Antoine Pitrou wrote: >> Arbitrary iterables, arbitrarily nested... >> beware of objects which are also their first element, like str('a')... > > Ouch. I guess that shoots the proposal dead in the water, then. Wasn't the suggestion only for flat iterables, like {int, float, complex}, rather than nested iterables? I'm ambivalent, as long as the implementation is careful to make sure one can still use isinstance on classes like these: class WeirdMetaclass(type): def __iter__(self): yield 1 class MyObject(metaclass=WeirdMetaclass): pass It still feels a little dirty though, to have to pick between options that are not mutually exclusive. Devin From fuzzyman at gmail.com Mon Jul 4 23:48:55 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 4 Jul 2011 22:48:55 +0100 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: <20110704230821.4b3ad626@pitrou.net> References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> <1309802125.3688.54.camel@localhost.localdomain> <20110704230821.4b3ad626@pitrou.net> Message-ID: On 4 July 2011 22:08, Antoine Pitrou wrote: > On Mon, 4 Jul 2011 21:41:33 +0200 > "Amaury Forgeot d'Arc" > wrote: > > Hi, > > > > 2011/7/4 Antoine Pitrou : > > > Le lundi 04 juillet 2011 ? 10:52 -0700, Gregory P. Smith a ?crit : > > >> note that a fast lookup implies exact type and not subclass making my > > >> point silly... at which point you're back to iterating so I suspect > > >> supporting arbitrary iterables is actually how this will be > > >> implemented regardless. > > > > > > Indeed. Note that the tuple case should remain fast, and therefore > > > special-cased (or the general list/tuple case, since the difference in > C > > > is rather small). > > > > Arbitrary iterables, arbitrarily nested... > > beware of objects which are also their first element, like str('a')... > > Ouch. I guess that shoots the proposal dead in the water, then. > > Why? Iteration doesn't imply recursing into contained iterators. Michael > Regards > > Antoine. > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at gmail.com Mon Jul 4 23:50:44 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 4 Jul 2011 22:50:44 +0100 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> <1309802125.3688.54.camel@localhost.localdomain> <20110704230821.4b3ad626@pitrou.net> Message-ID: On 4 July 2011 22:46, Devin Jeanpierre wrote: > On Mon, Jul 4, 2011 at 5:08 PM, Antoine Pitrou > wrote: > >> Arbitrary iterables, arbitrarily nested... > >> beware of objects which are also their first element, like str('a')... > > > > Ouch. I guess that shoots the proposal dead in the water, then. > > Wasn't the suggestion only for flat iterables, like {int, float, > complex}, rather than nested iterables? > > I'm ambivalent, as long as the implementation is careful to make sure > one can still use isinstance on classes like these: > > class WeirdMetaclass(type): > def __iter__(self): > yield 1 > > class MyObject(metaclass=WeirdMetaclass): > pass > > It still feels a little dirty though, to have to pick between options > that are not mutually exclusive. > That's a more reasonable objection for not accepting arbitrary iterables (although iterable types are a little "specialised"). Michael > > Devin > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From amauryfa at gmail.com Mon Jul 4 23:55:43 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Mon, 4 Jul 2011 23:55:43 +0200 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> <1309802125.3688.54.camel@localhost.localdomain> <20110704230821.4b3ad626@pitrou.net> Message-ID: 2011/7/4 Michael Foord : >> > Arbitrary iterables, arbitrarily nested... >> > beware of objects which are also their first element, like str('a')... >> >> Ouch. I guess that shoots the proposal dead in the water, then. >> > > Why? Iteration doesn't imply recursing into contained iterators. isinstance already accept nested tuples: assert isinstance(1, (str, (int, float))) -- Amaury Forgeot d'Arc From pyideas at rebertia.com Tue Jul 5 00:18:03 2011 From: pyideas at rebertia.com (Chris Rebert) Date: Mon, 4 Jul 2011 15:18:03 -0700 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> <1309802125.3688.54.camel@localhost.localdomain> <20110704230821.4b3ad626@pitrou.net> Message-ID: On Mon, Jul 4, 2011 at 2:55 PM, Amaury Forgeot d'Arc wrote: > 2011/7/4 Michael Foord : >>> > Arbitrary iterables, arbitrarily nested... >>> > beware of objects which are also their first element, like str('a')... >>> >>> Ouch. I guess that shoots the proposal dead in the water, then. >> >> Why? Iteration doesn't imply recursing into contained iterators. > > isinstance already accept nested tuples: > > assert isinstance(1, (str, (int, float))) That doesn't mean it has to necessarily accept other nested containers; tuples can be left as a special case for backward compatibility. Why does it accept nested tuples in the first place for that matter? Cheers, Chris From jeanpierreda at gmail.com Tue Jul 5 00:31:35 2011 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Mon, 4 Jul 2011 18:31:35 -0400 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> <1309802125.3688.54.camel@localhost.localdomain> <20110704230821.4b3ad626@pitrou.net> Message-ID: On Mon, Jul 4, 2011 at 6:18 PM, Chris Rebert wrote: > Why does it accept nested tuples in the first place for that matter? Whoops, that was news to me too. Apparently it's so that if you have code that does isinstance(foo, bar) you can do isinstance(foo, (bar, baz)) no matter what bar is. It's the only way to do this generically given that isinstance must accept both types and collections of types. Devin From amauryfa at gmail.com Tue Jul 5 00:50:51 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Tue, 5 Jul 2011 00:50:51 +0200 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> <1309802125.3688.54.camel@localhost.localdomain> <20110704230821.4b3ad626@pitrou.net> Message-ID: 2011/7/5 Chris Rebert : > On Mon, Jul 4, 2011 at 2:55 PM, Amaury Forgeot d'Arc wrote: >> 2011/7/4 Michael Foord : >>>> > Arbitrary iterables, arbitrarily nested... >>>> > beware of objects which are also their first element, like str('a')... >>>> >>>> Ouch. I guess that shoots the proposal dead in the water, then. >>> >>> Why? Iteration doesn't imply recursing into contained iterators. >> >> isinstance already accept nested tuples: >> >> assert isinstance(1, (str, (int, float))) > > That doesn't mean it has to necessarily accept other nested > containers; tuples can be left as a special case for backward > compatibility. > Why does it accept nested tuples in the first place for that matter? All this has already been discussed: http://mail.python.org/pipermail/python-list/2009-January/1188226.html -- Amaury Forgeot d'Arc From greg.ewing at canterbury.ac.nz Tue Jul 5 03:07:26 2011 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 05 Jul 2011 13:07:26 +1200 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> Message-ID: <4E1263CE.7060106@canterbury.ac.nz> Gregory P. Smith wrote: > A set lookup should be faster than iteration for anything over ~3 types. But you can't use a set lookup. You have to do a subclass test against all the members of the set, so the fact that it's a set rather than some other iterable doesn't help you. -- Greg From greg.ewing at canterbury.ac.nz Tue Jul 5 03:26:03 2011 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 05 Jul 2011 13:26:03 +1200 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> <1309802125.3688.54.camel@localhost.localdomain> <20110704230821.4b3ad626@pitrou.net> Message-ID: <4E12682B.2060808@canterbury.ac.nz> Chris Rebert wrote: > Why does it accept nested tuples in the first place for that matter? Probably so that tuples could be used to emulate a "set of types" before we had real sets. Now that we do have real sets, it doesn't seem necessary to continue with this and allow sets of sets, etc. So +1 on leaving tuple as a special case for this and treating all other iterables as flat. -- Greg From raymond.hettinger at gmail.com Tue Jul 5 03:55:28 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Mon, 4 Jul 2011 18:55:28 -0700 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: <4E12682B.2060808@canterbury.ac.nz> References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> <1309802125.3688.54.camel@localhost.localdomain> <20110704230821.4b3ad626@pitrou.net> <4E12682B.2060808@canterbury.ac.nz> Message-ID: <4FBB58CE-9080-46C4-9570-74FB31C034EE@gmail.com> On Jul 4, 2011, at 6:26 PM, Greg Ewing wrote: > Chris Rebert wrote: > >> Why does it accept nested tuples in the first place for that matter? > > Probably so that tuples could be used to emulate a "set > of types" before we had real sets. > > Now that we do have real sets, it doesn't seem necessary > to continue with this and allow sets of sets, etc. So > +1 on leaving tuple as a special case for this and > treating all other iterables as flat. -1 on further garbaging this API by introducing yet another signature variant -- even worse, a signature variant with a special case for the tuple type (didn't we learn our lesson with old-style string formatting). I have to agree with Benjamin that this proposal would be utterly unnecessary software bloat. Raymond From stefan_ml at behnel.de Tue Jul 5 06:32:56 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 05 Jul 2011 06:32:56 +0200 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: <4FBB58CE-9080-46C4-9570-74FB31C034EE@gmail.com> References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> <1309802125.3688.54.camel@localhost.localdomain> <20110704230821.4b3ad626@pitrou.net> <4E12682B.2060808@canterbury.ac.nz> <4FBB58CE-9080-46C4-9570-74FB31C034EE@gmail.com> Message-ID: Raymond Hettinger, 05.07.2011 03:55: > > On Jul 4, 2011, at 6:26 PM, Greg Ewing wrote: > >> Chris Rebert wrote: >> >>> Why does it accept nested tuples in the first place for that matter? >> >> Probably so that tuples could be used to emulate a "set >> of types" before we had real sets. >> >> Now that we do have real sets, it doesn't seem necessary >> to continue with this and allow sets of sets, etc. So >> +1 on leaving tuple as a special case for this and >> treating all other iterables as flat. > > -1 on further garbaging this API by introducing yet another signature variant -- even worse, a signature variant with a special case for the tuple type (didn't we learn our lesson with old-style string formatting). > > I have to agree with Benjamin that this proposal would be utterly unnecessary software bloat. FWIW, another -1 from me. The current API is simple: you pass in either a type or a tuple of types. Distinguishing the two cases is easy and safe, and even the somewhat bloatish case of passing in a tuple of tuples of types is safe, although somewhat complicated already. This proposal will complicate the implementation for no obvious reason, and will introduce slightly weird corner cases, like the handling of iterable type objects. After all, testing for a type multiple times, just because it ended up in the type tuple multiple times, will still work correctly. And even if repeated testing is not desired for some reason, using tuple(set(x)) instead of set(x) won't make that a big difference. Stefan From masklinn at masklinn.net Tue Jul 5 08:23:22 2011 From: masklinn at masklinn.net (Masklinn) Date: Tue, 5 Jul 2011 08:23:22 +0200 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> <1309802125.3688.54.camel@localhost.localdomain> <20110704230821.4b3ad626@pitrou.net> <4E12682B.2060808@canterbury.ac.nz> <4FBB58CE-9080-46C4-9570-74FB31C034EE@gmail.com> Message-ID: <30F86B1E-BC7A-4EBD-918B-6DFCC8FDC4F9@masklinn.net> On 2011-07-05, at 06:32 , Stefan Behnel wrote: > Raymond Hettinger, 05.07.2011 03:55: >> >> On Jul 4, 2011, at 6:26 PM, Greg Ewing wrote: >> >>> Chris Rebert wrote: >>> >>>> Why does it accept nested tuples in the first place for that matter? >>> >>> Probably so that tuples could be used to emulate a "set >>> of types" before we had real sets. >>> >>> Now that we do have real sets, it doesn't seem necessary >>> to continue with this and allow sets of sets, etc. So >>> +1 on leaving tuple as a special case for this and >>> treating all other iterables as flat. >> >> -1 on further garbaging this API by introducing yet another signature variant -- even worse, a signature variant with a special case for the tuple type (didn't we learn our lesson with old-style string formatting). >> >> I have to agree with Benjamin that this proposal would be utterly unnecessary software bloat. > > FWIW, another -1 from me. The current API is simple: you pass in either a type or a tuple of types. Or a tuple of tuples of types. Or a tuple of tuple of tuple of types. The actual API is something like this: TypeSet = Type | [TypeSet] isinstance :: (object, TypeSet) -> bool From stefan_ml at behnel.de Tue Jul 5 10:14:25 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 05 Jul 2011 10:14:25 +0200 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: <30F86B1E-BC7A-4EBD-918B-6DFCC8FDC4F9@masklinn.net> References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> <1309802125.3688.54.camel@localhost.localdomain> <20110704230821.4b3ad626@pitrou.net> <4E12682B.2060808@canterbury.ac.nz> <4FBB58CE-9080-46C4-9570-74FB31C034EE@gmail.com> <30F86B1E-BC7A-4EBD-918B-6DFCC8FDC4F9@masklinn.net> Message-ID: Masklinn, 05.07.2011 08:23: > > On 2011-07-05, at 06:32 , Stefan Behnel wrote: > >> Raymond Hettinger, 05.07.2011 03:55: >>> >>> On Jul 4, 2011, at 6:26 PM, Greg Ewing wrote: >>> >>>> Chris Rebert wrote: >>>> >>>>> Why does it accept nested tuples in the first place for that matter? >>>> >>>> Probably so that tuples could be used to emulate a "set >>>> of types" before we had real sets. >>>> >>>> Now that we do have real sets, it doesn't seem necessary >>>> to continue with this and allow sets of sets, etc. So >>>> +1 on leaving tuple as a special case for this and >>>> treating all other iterables as flat. >>> >>> -1 on further garbaging this API by introducing yet another signature variant -- even worse, a signature variant with a special case for the tuple type (didn't we learn our lesson with old-style string formatting). >>> >>> I have to agree with Benjamin that this proposal would be utterly unnecessary software bloat. >> >> FWIW, another -1 from me. The current API is simple: you pass in either a type or a tuple of types. > Or a tuple of tuples of types. Or a tuple of tuple of tuple of types. Yes, that's more or less what I wrote in the next sentence, right the first one that you discarded. ;) Stefan From masklinn at masklinn.net Tue Jul 5 10:33:45 2011 From: masklinn at masklinn.net (Masklinn) Date: Tue, 5 Jul 2011 10:33:45 +0200 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> <1309802125.3688.54.camel@localhost.localdomain> <20110704230821.4b3ad626@pitrou.net> <4E12682B.2060808@canterbury.ac.nz> <4FBB58CE-9080-46C4-9570-74FB31C034EE@gmail.com> <30F86B1E-BC7A-4EBD-918B-6DFCC8FDC4F9@masklinn.net> Message-ID: <35E9967C-4D2B-4587-8499-2D31267C80AC@masklinn.net> On 2011-07-05, at 10:14 , Stefan Behnel wrote: > > Yes, that's more or less what I wrote in the next sentence, right the first one that you discarded. ;) Not quite. Your next sentence talks about tuples of tuples, but the nesting is entirely arbitrary. Talking about tuples of tuples is way simplifying what's truly accepted by `isinstance`. From g.brandl at gmx.net Tue Jul 5 22:29:21 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 05 Jul 2011 22:29:21 +0200 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: <35E9967C-4D2B-4587-8499-2D31267C80AC@masklinn.net> References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> <1309802125.3688.54.camel@localhost.localdomain> <20110704230821.4b3ad626@pitrou.net> <4E12682B.2060808@canterbury.ac.nz> <4FBB58CE-9080-46C4-9570-74FB31C034EE@gma il.com> <30F86B1E-BC7A-4EBD-918B-6DFCC8FDC4F9@masklinn.net> <35E9967C-4D2B-4587-8499-2D31267C80AC@masklinn.net> Message-ID: Am 05.07.2011 10:33, schrieb Masklinn: > On 2011-07-05, at 10:14 , Stefan Behnel wrote: >> >> Yes, that's more or less what I wrote in the next sentence, right the first >> one that you discarded. ;) > Not quite. Your next sentence talks about tuples of tuples, but the nesting > is entirely arbitrary. Talking about tuples of tuples is way simplifying > what's truly accepted by `isinstance`. Where can I vote for renaming this list to python-nitpicking? (If you really read Stefan's post you'll see that he knows very well how this works.) Georg From guido at python.org Wed Jul 6 00:36:30 2011 From: guido at python.org (Guido van Rossum) Date: Tue, 5 Jul 2011 15:36:30 -0700 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: Message-ID: FWIW, the restriction to tuples is to avoid burdening the recursive C code with checks for cycles. Please keep that property. On Jul 3, 2011 3:47 PM, "Michael Foord" wrote: > Hello all, > > Occasionally I build up tuples of types for passing as the second argument > to isinstance. I've noticed on occasions having duplicates, for example the > following code is Python 2 & 3 compatible but results in a duplicate on one > of them: > > FunctionTypes = ( > # python function > type(create_autospec), > # instance method > type(ANY.__eq__), > # unbound method > type(_ANY.__eq__), > ) > > It would feel cleaner to me if I could make FunctionTypes a set and use that > as the second argument to isinstance. :-) > > Michael > > -- > > http://www.voidspace.org.uk/ > > May you do good and not evil > May you find forgiveness for yourself and forgive others > May you share freely, never taking more than you give. > -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Wed Jul 6 18:55:41 2011 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 6 Jul 2011 16:55:41 +0000 (UTC) Subject: [Python-ideas] Allow isinstance second argument to be a set of types References: Message-ID: Michael Foord writes: > It would feel cleaner to me if I could make FunctionTypes a set and use that as the second argument to isinstance. I just had a another thought about abritrary collections. It was explode loudly if you made an class. From alexander.belopolsky at gmail.com Wed Jul 6 20:01:28 2011 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 6 Jul 2011 14:01:28 -0400 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: Message-ID: On Tue, Jul 5, 2011 at 6:36 PM, Guido van Rossum wrote: > FWIW, the restriction to tuples is to avoid burdening the recursive C code > with checks for cycles. Please keep that property. Keeping this property does not seem that hard: preserve the current behavior when classinfo is a type or a tuple and return any(_isinstance(object, type) for type in classinfo) for any other type of classinfo. (In the last expression, "_isinstance" accepts only types for the second argument.) It looks like in this discussion a useful feature is held hostage to an exotic corner case. I understand that it may sometimes be useful to write isinstance(x, (MyString, StringTypes)), but isinstance(x, (MyString,)+StringTypes) is just as readable. Furthermore, with str/unicode unification and addition of ABCs in py3k, I would expect nested classinfo to become even less useful. Note that the current error message from isinstance() does not mention a possibility of nested tuples: >>> isinstance(1, [int]) Traceback (most recent call last): File "", line 1, in TypeError: isinstance() arg 2 must be a type or tuple of types Since no one has ever complained about this, I conclude that people rightfully consider nested tuple classinfo to be too exotic to mention in an error message. I don't see any problem with restricting recursive behavior to tuples. It is often recommended that lists be used for homogeneous collections and tuples for mixed-type collections. This rule will naturally lead users to choose tuples when they need a nested collection. On the other hand, some users may find isinstance(x, [int, float]) or isinstance(x, {int, float}) more readable than isinstance(x, (int, float)) either simply because [] or {} stand out better in the argument list or because they are used to seeing homogeneous collections displayed as lists. From alexander.belopolsky at gmail.com Wed Jul 6 20:09:48 2011 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 6 Jul 2011 14:09:48 -0400 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: <4E11656B.4050908@canterbury.ac.nz> <20110704172843.50c5b445@pitrou.net> <1309802125.3688.54.camel@localhost.localdomain> <20110704230821.4b3ad626@pitrou.net> Message-ID: On Mon, Jul 4, 2011 at 5:50 PM, Michael Foord wrote: .. >> I'm ambivalent, as long as the implementation is careful to make sure >> one can still use isinstance on classes like these: >> >> ? ?class WeirdMetaclass(type): >> ? ? ? ?def __iter__(self): >> ? ? ? ? ? ?yield 1 >> >> ? ?class MyObject(metaclass=WeirdMetaclass): >> ? ? ? ?pass >> >> It still feels a little dirty though, to have to pick between options >> that are not mutually exclusive. > > That's a more reasonable objection for not accepting arbitrary iterables I don't see a problem here. If classinfo is a type, isinstance should not attempt to iterate over it. If one really wants to check for membership in the list that MyObject yields, it is always possible to do it explicitly: isinstance(x, list(MyObject)). From fuzzyman at gmail.com Thu Jul 7 01:06:45 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Thu, 7 Jul 2011 00:06:45 +0100 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: Message-ID: On 6 July 2011 17:55, Benjamin Peterson wrote: > Michael Foord writes: > > > It would feel cleaner to me if I could make FunctionTypes a set and use > that > as the second argument to isinstance. > > I just had a another thought about abritrary collections. It was explode > loudly > if you made an class. > Could you rephrase please? I have no idea what this means. :-) All the best, Michael Foord > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Thu Jul 7 01:35:34 2011 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 6 Jul 2011 23:35:34 +0000 (UTC) Subject: [Python-ideas] =?utf-8?q?Allow_isinstance_second_argument_to_be_a?= =?utf-8?q?_set_of=09types?= References: Message-ID: Michael Foord writes: > > > On 6 July 2011 17:55, Benjamin Peterson python.org> wrote: > Michael Foord ...> writes: > > It would feel cleaner to me if I could make FunctionTypes a set and use that > as the second argument to isinstance. > I just had a another thought about abritrary collections. It was explode loudly > if you made an class. > > > Could you rephrase please? I have no idea what this means. All the best,Michael Foord? Mm. I don't blame you. I'll speak code: class Evil(type): def __iter__(self): return range(42) class Accomplice(metaclass=Evil): pass isinstance(12, Accomplice()) # boom if arbitrary iterables are allowed From alexander.belopolsky at gmail.com Thu Jul 7 02:02:22 2011 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 6 Jul 2011 20:02:22 -0400 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: Message-ID: On Wed, Jul 6, 2011 at 7:35 PM, Benjamin Peterson wrote: .. > Mm. I don't blame you. I'll speak code: > > class Evil(type): > ? ?def __iter__(self): > ? ? ? ?return range(42) > > > class Accomplice(metaclass=Evil): > ? ?pass > > isinstance(12, Accomplice()) # boom if arbitrary iterables are allowed What is "boom"? If "boom" is a TypeError - it is to be expected. For example, with def better_isinstance(object, classinfo): if isinstance(classinfo, (type, tuple)): return isinstance(object, classinfo) else: return any(isinstance(object, cls) for cls in classinfo) I get: >>> better_isinstance(12, Accomplice) False and >>> better_isinstance(12, Accomplice()) Traceback (most recent call last): .. TypeError: 'Accomplice' object is not iterable From fuzzyman at gmail.com Thu Jul 7 19:14:51 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Thu, 7 Jul 2011 18:14:51 +0100 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: Message-ID: On 7 July 2011 01:02, Alexander Belopolsky wrote: > On Wed, Jul 6, 2011 at 7:35 PM, Benjamin Peterson > wrote: > .. > > Mm. I don't blame you. I'll speak code: > > > > class Evil(type): > > def __iter__(self): > > return range(42) > > > > > > class Accomplice(metaclass=Evil): > > pass > > > > isinstance(12, Accomplice()) # boom if arbitrary iterables are allowed > > What is "boom"? If "boom" is a TypeError - it is to be expected. For > example, with > I think the instantiation of Accompalice there is a typo. The point is that as Accompalice (the type) is an instance of Evil and is therefore an iterable type. So if isintance naively iterated over *anything iterable* it would attempt to iterate over Accomplice. > > def better_isinstance(object, classinfo): > if isinstance(classinfo, (type, tuple)): > return isinstance(object, classinfo) > else: > return any(isinstance(object, cls) for cls in classinfo) > > You avoid this problem by checking explicitly for type. Michael > I get: > > >>> better_isinstance(12, Accomplice) > False > > and > > >>> better_isinstance(12, Accomplice()) > Traceback (most recent call last): > .. > TypeError: 'Accomplice' object is not iterable > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From jimjjewett at gmail.com Thu Jul 7 19:22:33 2011 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 7 Jul 2011 13:22:33 -0400 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: Message-ID: On Wed, Jul 6, 2011 at 7:35 PM, Benjamin Peterson wrote: > class Evil(type): > ? ?def __iter__(self): > ? ? ? ?return range(42) > class Accomplice(metaclass=Evil): > ? ?pass ... leading to isinstance(x, 42) rather than isinstance(x, Evil) if the iterable check is done too soon. So, at a minimum, this class should be in the test suite to guard against future ill-fated optimizations. -jJ From solipsis at pitrou.net Thu Jul 7 19:43:21 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 7 Jul 2011 19:43:21 +0200 Subject: [Python-ideas] Allow isinstance second argument to be a set of types References: Message-ID: <20110707194321.158a8e4f@pitrou.net> On Thu, 7 Jul 2011 13:22:33 -0400 Jim Jewett wrote: > On Wed, Jul 6, 2011 at 7:35 PM, Benjamin Peterson wrote: > > > class Evil(type): > > ? ?def __iter__(self): > > ? ? ? ?return range(42) > > > class Accomplice(metaclass=Evil): > > ? ?pass > > ... leading to isinstance(x, 42) rather than isinstance(x, Evil) if > the iterable check is done too soon. In what way is that a problem? That you can trigger exceptions by writing malicious code doesn't sound like a particularly novel event ;) Regards Antoine. From mikegraham at gmail.com Thu Jul 7 19:54:00 2011 From: mikegraham at gmail.com (Mike Graham) Date: Thu, 7 Jul 2011 13:54:00 -0400 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: Message-ID: Any of the changes I see would make isinstance work. This basic sort of API -- take a thing or a tuple-of-things (or often a list-of-things or precariously an iterable-of-things) is a poor, inflexible one I avoid in application code as I think do most Python programmers when they can. The builtins and stdlib have this scattered about, but for the most part it makes the relevant APIs less clear and simple. I wonder if it isn't time to rethink isinstance at a more basic level. The API SomeClass.isinstance(some_potential_instance) makes more sense to me than isinstance(some_potential_instance, SomeClass), since it is really an issue to do with the class. Indeed, recent Pythons are implemented this way. It seems attractive with metaclasses to be able to duck type this operation where your own metaclass which does not inherit type can look just like an instance of type. If you wanted to use an iterable of classes, you could use any(t.isinstance(o) for t in types) or we could introduce a utility ABC factory ComposeTypes such that we could do ComposeTypes(types).isinstance(o). I do hesitate to think this change is good for a few reasons. It introduces change and compatibility issues by introducing a new method for all classes and deprecating a builtin. Additionally, it could seem to encourage typechecking by improving it. Mike From mikegraham at gmail.com Thu Jul 7 21:08:54 2011 From: mikegraham at gmail.com (Mike Graham) Date: Thu, 7 Jul 2011 15:08:54 -0400 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: <20110707194321.158a8e4f@pitrou.net> References: <20110707194321.158a8e4f@pitrou.net> Message-ID: On Thu, Jul 7, 2011 at 1:43 PM, Antoine Pitrou wrote: > On Thu, 7 Jul 2011 13:22:33 -0400 > Jim Jewett wrote: >> On Wed, Jul 6, 2011 at 7:35 PM, Benjamin Peterson wrote: >> >> > class Evil(type): >> > ? ?def __iter__(self): >> > ? ? ? ?return range(42) >> >> > class Accomplice(metaclass=Evil): >> > ? ?pass >> >> ... leading to isinstance(x, 42) rather than isinstance(x, Evil) if >> the iterable check is done too soon. > > In what way is that a problem? > That you can trigger exceptions by writing malicious code doesn't sound > like a particularly novel event ;) I think people are assuming that an iterable metaclass isn't fundamentally wrong. Mike From solipsis at pitrou.net Thu Jul 7 21:12:17 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 7 Jul 2011 21:12:17 +0200 Subject: [Python-ideas] Allow isinstance second argument to be a set of types References: <20110707194321.158a8e4f@pitrou.net> Message-ID: <20110707211217.4b802c7a@pitrou.net> On Thu, 7 Jul 2011 15:08:54 -0400 Mike Graham wrote: > On Thu, Jul 7, 2011 at 1:43 PM, Antoine Pitrou wrote: > > On Thu, 7 Jul 2011 13:22:33 -0400 > > Jim Jewett wrote: > >> On Wed, Jul 6, 2011 at 7:35 PM, Benjamin Peterson wrote: > >> > >> > class Evil(type): > >> > ? ?def __iter__(self): > >> > ? ? ? ?return range(42) > >> > >> > class Accomplice(metaclass=Evil): > >> > ? ?pass > >> > >> ... leading to isinstance(x, 42) rather than isinstance(x, Evil) if > >> the iterable check is done too soon. > > > > In what way is that a problem? > > That you can trigger exceptions by writing malicious code doesn't sound > > like a particularly novel event ;) > > I think people are assuming that an iterable metaclass isn't > fundamentally wrong. Well, it does sound fundamentally useless! Regards Antoine. From fuzzyman at gmail.com Fri Jul 8 00:12:48 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Thu, 7 Jul 2011 23:12:48 +0100 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: <20110707211217.4b802c7a@pitrou.net> References: <20110707194321.158a8e4f@pitrou.net> <20110707211217.4b802c7a@pitrou.net> Message-ID: On 7 July 2011 20:12, Antoine Pitrou wrote: > On Thu, 7 Jul 2011 15:08:54 -0400 > Mike Graham wrote: > > On Thu, Jul 7, 2011 at 1:43 PM, Antoine Pitrou < > solipsis-xNDA5Wrcr86sTnJN9+BGXg at public.gmane.org> wrote: > > > On Thu, 7 Jul 2011 13:22:33 -0400 > > > Jim Jewett wrote: > > >> On Wed, Jul 6, 2011 at 7:35 PM, Benjamin Peterson < > benjamin-+ZN9ApsXKcEdnm+yROfE0A at public.gmane.org> wrote: > > >> > > >> > class Evil(type): > > >> > def __iter__(self): > > >> > return range(42) > > >> > > >> > class Accomplice(metaclass=Evil): > > >> > pass > > >> > > >> ... leading to isinstance(x, 42) rather than isinstance(x, Evil) if > > >> the iterable check is done too soon. > > > > > > In what way is that a problem? > > > That you can trigger exceptions by writing malicious code doesn't sound > > > like a particularly novel event ;) > > > > I think people are assuming that an iterable metaclass isn't > > fundamentally wrong. > > Well, it does sound fundamentally useless! > I've used indexable types before. I've never created an iterable one, but it isn't unthinkable. :-) Michael > > Regards > > Antoine. > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From ryan at rfk.id.au Fri Jul 8 00:28:11 2011 From: ryan at rfk.id.au (Ryan Kelly) Date: Fri, 08 Jul 2011 08:28:11 +1000 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: <20110707194321.158a8e4f@pitrou.net> <20110707211217.4b802c7a@pitrou.net> Message-ID: <1310077691.2106.3.camel@durian> On Thu, 2011-07-07 at 23:12 +0100, Michael Foord wrote: > > > On 7 July 2011 20:12, Antoine Pitrou wrote: > On Thu, 7 Jul 2011 15:08:54 -0400 > Mike Graham wrote: > > On Thu, Jul 7, 2011 at 1:43 PM, Antoine Pitrou > wrote: > > > On Thu, 7 Jul 2011 13:22:33 -0400 > > > Jim Jewett wrote: > > >> On Wed, Jul 6, 2011 at 7:35 PM, Benjamin Peterson > wrote: > > >> > > >> > class Evil(type): > > >> > def __iter__(self): > > >> > return range(42) > > >> > > >> > class Accomplice(metaclass=Evil): > > >> > pass > > >> > > >> ... leading to isinstance(x, 42) rather than > isinstance(x, Evil) if > > >> the iterable check is done too soon. > > > > > > In what way is that a problem? > > > That you can trigger exceptions by writing malicious code > doesn't sound > > > like a particularly novel event ;) > > > > I think people are assuming that an iterable metaclass isn't > > fundamentally wrong. > > > Well, it does sound fundamentally useless! > > I've used indexable types before. I've never created an iterable one, > but it isn't unthinkable. :-) I've created one, once. It was a metaclass for defining enumerations like this: class COLOURS(Enum): RED = 1 BLUE = 2 GREEN = 3 If you wanted to iterate over all the members of the enum, you could do: for colour in COLOURS: blah(colour) Of course, in this case it would be totally useless to pass COLOURS into isintance(). But I don't see what doing so should be capable of breaking things. Ryan -- Ryan Kelly http://www.rfk.id.au | This message is digitally signed. Please visit ryan at rfk.id.au | http://www.rfk.id.au/ramblings/gpg/ for details -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From g.rodola at gmail.com Fri Jul 8 01:08:25 2011 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Fri, 8 Jul 2011 01:08:25 +0200 Subject: [Python-ideas] os.path.isparent and os.path.ischild Message-ID: >>> isparent('/a', '/a/b') True >>> isparent('/a/b', '/a') False >>> ischild('/a/b', '/a') True >>> ischild('/a', '/a/b') False Thoughts? Regards, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ From jxo6948 at rit.edu Fri Jul 8 01:24:07 2011 From: jxo6948 at rit.edu (John O'Connor) Date: Thu, 7 Jul 2011 16:24:07 -0700 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: References: Message-ID: On Thu, Jul 7, 2011 at 4:08 PM, Giampaolo Rodol? wrote: > >>> isparent('/a', '/a/b') > True > >>> isparent('/a/b', '/a') > False > >>> ischild('/a/b', '/a') > True > >>> ischild('/a', '/a/b') > False > What about is{ancestor, descendant}() ? >>> isancestor('/a', '/a/b/c') True >>> isparent('/a', '/a/b/c') False -------------- next part -------------- An HTML attachment was scrubbed... URL: From jxo6948 at rit.edu Fri Jul 8 01:54:35 2011 From: jxo6948 at rit.edu (John O'Connor) Date: Thu, 7 Jul 2011 16:54:35 -0700 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: <5E25C96030E66B44B9CFAA95D3DE59193522A94AA2@EX-MB08.ohsu.edu> References: <5E25C96030E66B44B9CFAA95D3DE59193522A94AA2@EX-MB08.ohsu.edu> Message-ID: On Thu, Jul 7, 2011 at 4:45 PM, Brendan Moloney wrote: > Why provide two functions instead of just something like innested and allow > the user to switch the arguments? > > The former would likely be more readable. One implementation would be expressed in terms of the other. -------------- next part -------------- An HTML attachment was scrubbed... URL: From moloney at ohsu.edu Fri Jul 8 01:45:01 2011 From: moloney at ohsu.edu (Brendan Moloney) Date: Thu, 7 Jul 2011 16:45:01 -0700 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: References: , Message-ID: <5E25C96030E66B44B9CFAA95D3DE59193522A94AA2@EX-MB08.ohsu.edu> Why provide two functions instead of just something like innested and allow the user to switch the arguments? Brendan Moloney Senior Research Assistant / Programmer Advanced Imaging Research Center Oregon Health Science University ________________________________________ From: python-ideas-bounces+moloney=ohsu.edu at python.org [python-ideas-bounces+moloney=ohsu.edu at python.org] On Behalf Of John O'Connor [jxo6948 at rit.edu] Sent: Thursday, July 07, 2011 4:24 PM To: Giampaolo Rodol? Cc: python-ideas at python.org Subject: Re: [Python-ideas] os.path.isparent and os.path.ischild On Thu, Jul 7, 2011 at 4:08 PM, Giampaolo Rodol? > wrote: >>> isparent('/a', '/a/b') True >>> isparent('/a/b', '/a') False >>> ischild('/a/b', '/a') True >>> ischild('/a', '/a/b') False What about is{ancestor, descendant}() ? >>> isancestor('/a', '/a/b/c') True >>> isparent('/a', '/a/b/c') False From pyideas at rebertia.com Fri Jul 8 02:30:29 2011 From: pyideas at rebertia.com (Chris Rebert) Date: Thu, 7 Jul 2011 17:30:29 -0700 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: <5E25C96030E66B44B9CFAA95D3DE59193522A94AA2@EX-MB08.ohsu.edu> References: <5E25C96030E66B44B9CFAA95D3DE59193522A94AA2@EX-MB08.ohsu.edu> Message-ID: > From: John O'Connor [jxo6948 at rit.edu] > What about is{ancestor, descendant}() ? > >>>> isancestor('/a', '/a/b/c') > True >>>> isparent('/a', '/a/b/c') > False On Thu, Jul 7, 2011 at 4:45 PM, Brendan Moloney wrote: > Why provide two functions instead of just something like innested and allow the user to switch the arguments? Why provide pairs of comparison operators [(>, <), (>=, <=)] instead of just one from each pair? The user can always switch the arguments themself. Cheers, Chris From moloney at ohsu.edu Fri Jul 8 02:54:18 2011 From: moloney at ohsu.edu (Brendan Moloney) Date: Thu, 7 Jul 2011 17:54:18 -0700 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: References: <5E25C96030E66B44B9CFAA95D3DE59193522A94AA2@EX-MB08.ohsu.edu>, Message-ID: <5E25C96030E66B44B9CFAA95D3DE59193522A94AA3@EX-MB08.ohsu.edu> There are a small number of operators which can be used on a large number of types. There is also a long standing tradition of having distinct symbols for reciprocal operations in mathematics. Should we provide reciprocal functions whenever possible? Should there be a 'issuperclass' function to compliment the 'issubclass' function? At some point you have to ask if the increased clutter in the API is worth the improved readability. In this case I am not sure that it does. Brendan ________________________________________ From: chris at rebertia.com [chris at rebertia.com] On Behalf Of Chris Rebert [pyideas at rebertia.com] Sent: Thursday, July 07, 2011 5:30 PM To: Brendan Moloney Cc: John O'Connor; Giampaolo Rodol?; python-ideas at python.org Subject: Re: [Python-ideas] os.path.isparent and os.path.ischild > From: John O'Connor [jxo6948 at rit.edu] > What about is{ancestor, descendant}() ? > >>>> isancestor('/a', '/a/b/c') > True >>>> isparent('/a', '/a/b/c') > False On Thu, Jul 7, 2011 at 4:45 PM, Brendan Moloney wrote: > Why provide two functions instead of just something like innested and allow the user to switch the arguments? Why provide pairs of comparison operators [(>, <), (>=, <=)] instead of just one from each pair? The user can always switch the arguments themself. Cheers, Chris From greg.ewing at canterbury.ac.nz Fri Jul 8 03:18:55 2011 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 08 Jul 2011 13:18:55 +1200 Subject: [Python-ideas] Allow isinstance second argument to be a set of types In-Reply-To: References: Message-ID: <4E165AFF.2000602@canterbury.ac.nz> Michael Foord wrote: > The point is > that as Accompalice (the type) is an instance of Evil and is therefore > an iterable type. So if isintance naively iterated over *anything > iterable* it would attempt to iterate over Accomplice. But if it checks whether something is a type first, it won't get as far as trying to iterate over it. This example illustrates that it's possible to create an object that is ambiguous with respect to the proposed API. But it's a case that is unlikely to arise in practice, so I think resolving it using the rule that type-ness takes precedence over iterable-ness is a reasonable solution. -- Greg From g.rodola at gmail.com Fri Jul 8 09:30:00 2011 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Fri, 8 Jul 2011 09:30:00 +0200 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: <5E25C96030E66B44B9CFAA95D3DE59193522A94AA2@EX-MB08.ohsu.edu> References: <5E25C96030E66B44B9CFAA95D3DE59193522A94AA2@EX-MB08.ohsu.edu> Message-ID: 2011/7/8 Brendan Moloney : > Why provide two functions instead of just something like innested and allow the user to switch the arguments? I'm not sure I'm following you. Could you please provide an example? Regards, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ From steve at pearwood.info Fri Jul 8 10:00:26 2011 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 08 Jul 2011 18:00:26 +1000 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: References: Message-ID: <4E16B91A.204@pearwood.info> Giampaolo Rodol? wrote: >>>> isparent('/a', '/a/b') > True I'm not sure that this is so common and useful that it needs to be included in os.path. You can always just add it to your own library, or even write them in-line: the implementation is just a one-liner. def isparent(path1, path2): "Returns True if path1 is a parent of path2." return os.path.commonprefix([path1, path2]) == path1 def ischild(path1, path2): "Returns True if path1 is a child of path2." return os.path.commonprefix([path1, path2]) == path2 -- Steven From p.f.moore at gmail.com Fri Jul 8 10:51:35 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 8 Jul 2011 09:51:35 +0100 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: References: Message-ID: On 8 July 2011 00:08, Giampaolo Rodol? wrote: >>>> isparent('/a', '/a/b') > True >>>> isparent('/a/b', '/a') > False > >>>> ischild('/a/b', '/a') > True >>>> ischild('/a', '/a/b') > False > > Thoughts? Underspecified. Are these simply string operations? If not, how would symlinks be handled? What about case sensitivity (given that detecting case sensitivity per filesystem is not possible, in practical terms)? What about os.sep and os.altsep (\ and / on Windows)? By the time you've answered all of these questions, you may as well code the operations. Then let's see if people use them before putting them into the standard library. If you're only looking at small utility wrappers round os.path.commonprefix, then I don't see the point (such a wrapper is easy enough to write if you need it in your own code). In reality, os.path is a compromise providing "simplistic but good enough" path handling. If I assume you intend these operations also to be "simplistic but good enough", then I see little benefit that I can't get with os.path.commonprefix (and maybe a couple of lines of wrapper, if I use the idiom a lot). Personally, I don't expect I'd ever use something like this. Also, if isparent(x,y) and ischild(y,x) are intended to be equivalent, then I don't see the value in having both (comments about comparison operators notwithstanding). And if there's a subtle difference I've missed, you haven't explained it well enough :-) Having said all this, I support the idea of having a better set of path handling functions in the stdlib. But it's far from a simple problem to do it right in a sufficiently cross-platform way. And just adding functions to os.path probably isn't the right way to start. Paul. From andrew at bemusement.org Fri Jul 8 10:40:24 2011 From: andrew at bemusement.org (Andrew Bennetts) Date: Fri, 8 Jul 2011 18:40:24 +1000 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: <4E16B91A.204@pearwood.info> References: <4E16B91A.204@pearwood.info> Message-ID: <20110708084024.GB5425@aihal.home.puzzling.org> Steven D'Aprano wrote: > I'm not sure that this is so common and useful that it needs to be > included in os.path. You can always just add it to your own library, > or even write them in-line: the implementation is just a one-liner. > > def isparent(path1, path2): > "Returns True if path1 is a parent of path2." > return os.path.commonprefix([path1, path2]) == path1 > > def ischild(path1, path2): > "Returns True if path1 is a child of path2." > return os.path.commonprefix([path1, path2]) == path2 Those one-liners are simple, obvious, and dangerously wrong: >>> ischild('aa', 'a') True -Andrew. From steve at pearwood.info Fri Jul 8 11:07:05 2011 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 08 Jul 2011 19:07:05 +1000 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: <20110708084024.GB5425@aihal.home.puzzling.org> References: <4E16B91A.204@pearwood.info> <20110708084024.GB5425@aihal.home.puzzling.org> Message-ID: <4E16C8B9.7060603@pearwood.info> Andrew Bennetts wrote: > Steven D'Aprano wrote: >> I'm not sure that this is so common and useful that it needs to be >> included in os.path. You can always just add it to your own library, >> or even write them in-line: the implementation is just a one-liner. >> >> def isparent(path1, path2): >> "Returns True if path1 is a parent of path2." >> return os.path.commonprefix([path1, path2]) == path1 >> >> def ischild(path1, path2): >> "Returns True if path1 is a child of path2." >> return os.path.commonprefix([path1, path2]) == path2 > > Those one-liners are simple, obvious, and dangerously wrong: > >>>> ischild('aa', 'a') > True I would call that a bug in commonprefix. >>> os.path.commonprefix(['/dir/pics/file', '/dir/pictures/file']) '/dir/pic' commonprefix is documented as returning the longest common path component, not leading substring: >>> help(os.path.commonprefix) commonprefix(m) Given a list of pathnames, returns the longest common leading component -- Steven From phd at phdru.name Fri Jul 8 11:25:24 2011 From: phd at phdru.name (Oleg Broytman) Date: Fri, 8 Jul 2011 13:25:24 +0400 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: <4E16B91A.204@pearwood.info> References: <4E16B91A.204@pearwood.info> Message-ID: <20110708092524.GB18857@iskra.aviel.ru> On Fri, Jul 08, 2011 at 06:00:26PM +1000, Steven D'Aprano wrote: > def isparent(path1, path2): > "Returns True if path1 is a parent of path2." > return os.path.commonprefix([path1, path2]) == path1 >>> isparent('C:\\Program Files', 'C:/Program Files/Python') False Oops... Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From steve at pearwood.info Fri Jul 8 11:57:04 2011 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 08 Jul 2011 19:57:04 +1000 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: <20110708092524.GB18857@iskra.aviel.ru> References: <4E16B91A.204@pearwood.info> <20110708092524.GB18857@iskra.aviel.ru> Message-ID: <4E16D470.4030108@pearwood.info> Oleg Broytman wrote: > On Fri, Jul 08, 2011 at 06:00:26PM +1000, Steven D'Aprano wrote: >> def isparent(path1, path2): >> "Returns True if path1 is a parent of path2." >> return os.path.commonprefix([path1, path2]) == path1 > >>>> isparent('C:\\Program Files', 'C:/Program Files/Python') > False > > Oops... Why is that an oops? On my Linux system, I can create a file "C:\\Program Files" which is very different from the path "C:/Program Files/Python" (two directories and a file). Other operating systems may do things differently, but for me, they have no path in common: >>> os.path.split('C:\\Program Files') ('', 'C:\\Program Files') >>> os.path.split('C:/Program Files/Python') ('C:/Program Files', 'Python') If you want to normalise the paths first, normalise them. That's what normpath and normcase are for. The other path manipulation functions, like os.path.split(), don't normalise paths before operating on them. Why should isparent and ischild? -- Steven From facundobatista at gmail.com Fri Jul 8 15:18:15 2011 From: facundobatista at gmail.com (Facundo Batista) Date: Fri, 8 Jul 2011 10:18:15 -0300 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: References: Message-ID: On Thu, Jul 7, 2011 at 8:08 PM, Giampaolo Rodol? wrote: >>>> isparent('/a', '/a/b') > True >>>> isparent('/a/b', '/a') > False > >>>> ischild('/a/b', '/a') > True >>>> ischild('/a', '/a/b') > False >>> def isparent(a, b): ... a += os.path.sep ... return b.startswith(a) ... >>> isparent('/a', '/a/b') True >>> isparent('/a/b', '/a') False >>> isparent('/ab/c', '/a') False See the append of the path separator for the third case later. ischild() would be very similar. Regards, -- .? ? Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From phd at phdru.name Fri Jul 8 15:35:08 2011 From: phd at phdru.name (Oleg Broytman) Date: Fri, 8 Jul 2011 17:35:08 +0400 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: References: Message-ID: <20110708133508.GA23001@iskra.aviel.ru> On Fri, Jul 08, 2011 at 10:18:15AM -0300, Facundo Batista wrote: > >>> def isparent(a, b): > ... a += os.path.sep > ... return b.startswith(a) > ... > >>> isparent('/a', '/a/b') > True > >>> isparent('/a/b', '/a') > False > >>> isparent('/ab/c', '/a') > False > > > See the append of the path separator for the third case later. > > ischild() would be very similar. Look at a slightly better implementation shutil.destinsrc. Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From moloney at ohsu.edu Fri Jul 8 21:38:13 2011 From: moloney at ohsu.edu (Brendan Moloney) Date: Fri, 8 Jul 2011 12:38:13 -0700 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: <4E16C8B9.7060603@pearwood.info> References: <4E16B91A.204@pearwood.info> <20110708084024.GB5425@aihal.home.puzzling.org>, <4E16C8B9.7060603@pearwood.info> Message-ID: <5E25C96030E66B44B9CFAA95D3DE59193522A94AA4@EX-MB08.ohsu.edu> > Steven D'Aprano wrote: > I would call that a bug in commonprefix. The documentation on the website correctly describes this behavior (http://docs.python.org/library/os.path.html) while the docstring does not. Someone tried to make the behavior more sane some time ago (http://bugs.python.org/issue400788) but this was reverted since it was backwards incompatible and the documentation correctly described the behavior. There is currently an open issue (http://bugs.python.org/issue10395) to add a function that does what commonprefix should have done to start with. Once this function is present I see no need for an is{parent|child} function. Brendan From cool-rr at cool-rr.com Sat Jul 9 11:35:13 2011 From: cool-rr at cool-rr.com (cool-RR) Date: Sat, 9 Jul 2011 05:35:13 -0400 Subject: [Python-ideas] Opening a folder in Explorer/Nautilus/Mac-thingie Message-ID: Hello, I have the path of a certain folder. I want to open it using the default folder explorer for that system. For example, if it's a Windows computer, I want to use Explorer, if it's Linux, I want to use Nautilus or whatever is the default there, if it's Mac, I want to use whatever Mac OS's explorer is called. I asked this on StackOverflow: http://stackoverflow.com/questions/6631299/python-opening-a-folder-in-explorer-nautilus-mac-thingie But it seems that Python doesn't offer a cross-platform way to do this. I see this functionality as pretty similar to that of the `webbrowser` module. Do you think that this is something that could be included with the standard library? Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pyideas at rebertia.com Sat Jul 9 12:02:30 2011 From: pyideas at rebertia.com (Chris Rebert) Date: Sat, 9 Jul 2011 03:02:30 -0700 Subject: [Python-ideas] Opening a folder in Explorer/Nautilus/Mac-thingie In-Reply-To: References: Message-ID: On Sat, Jul 9, 2011 at 2:35 AM, cool-RR wrote: > Hello, > I have the path of a certain folder. I want to open it using the default > folder explorer for that system. For example, if it's a Windows computer, I > want to use Explorer, if it's Linux, I want to use Nautilus or whatever is > the default there, if it's Mac, I want to use whatever Mac OS's explorer is > called. > I asked this on StackOverflow: > http://stackoverflow.com/questions/6631299/python-opening-a-folder-in-explorer-nautilus-mac-thingie > But it seems that Python doesn't offer a cross-platform way to do this. Invoking the appropriate platform-specific open-with-default-program command with a folder argument should work: http://stackoverflow.com/questions/434597/open-document-with-default-application-in-python If anything should be added to the std lib, I would say it should be this rather than a more narrow open-with-platform's-file-manager function. Cheers, Chris From cool-rr at cool-rr.com Sat Jul 9 13:55:27 2011 From: cool-rr at cool-rr.com (cool-RR) Date: Sat, 9 Jul 2011 07:55:27 -0400 Subject: [Python-ideas] Opening a folder in Explorer/Nautilus/Mac-thingie In-Reply-To: References: Message-ID: On Sat, Jul 9, 2011 at 6:02 AM, Chris Rebert wrote: > On Sat, Jul 9, 2011 at 2:35 AM, cool-RR wrote: > > Hello, > > I have the path of a certain folder. I want to open it using the default > > folder explorer for that system. For example, if it's a Windows computer, > I > > want to use Explorer, if it's Linux, I want to use Nautilus or whatever > is > > the default there, if it's Mac, I want to use whatever Mac OS's explorer > is > > called. > > I asked this on StackOverflow: > > > http://stackoverflow.com/questions/6631299/python-opening-a-folder-in-explorer-nautilus-mac-thingie > > But it seems that Python doesn't offer a cross-platform way to do this. > > Invoking the appropriate platform-specific open-with-default-program > command with a folder argument should work: > > http://stackoverflow.com/questions/434597/open-document-with-default-application-in-python > > If anything should be added to the std lib, I would say it should be > this rather than a more narrow open-with-platform's-file-manager > function. > > Cheers, > Chris > I agree, it's better to introduce an open-with-default-program function. It seems that `os.startfile` is supposed to do this: http://docs.python.org/dev/library/os.html#os.startfile But it's only available under Windows. Perhaps the solution is to implement it under Linux and Mac as well, so we don't need to introduce a new function? Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ram.rachum at gmail.com Sat Jul 9 13:59:51 2011 From: ram.rachum at gmail.com (cool-RR) Date: Sat, 9 Jul 2011 04:59:51 -0700 (PDT) Subject: [Python-ideas] Opening a folder in Explorer/Nautilus/Mac-thingie In-Reply-To: Message-ID: <20008313.185.1310212791363.JavaMail.geo-discussion-forums@yqcj24> Opened a ticket for this: http://bugs.python.org/issue12522?@ok_message=msg%20140057%20created%3Cbr%3Eissue%2012522%20created&@template=item Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ericsnowcurrently at gmail.com Wed Jul 13 21:26:17 2011 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Wed, 13 Jul 2011 13:26:17 -0600 Subject: [Python-ideas] deferred default arguments Message-ID: I'll get this out of the way now. This is probably a dumb idea. However, I have found myself working around a use case lately and have a thought of a cleaner solution. The Situation --------------- As you know, you can use default arguments when defining a function. Sometimes you want to pass an argument through from a function's arguments to another function that the original ("outer") calls. I run into this during inheritance-related calls and when interfacing between two compatible APIs. Here's an example: class X: def f(self, name="N/A"): ... class Y(X): def f(self, name="N/A"): super().f(name) Y().f() So, the name argument is passed through from Y's f() to X's. In this case, we want to keep the default argument the same. Current Solutions --------------- Keeping the default arguments the same can be accomplished in a few different ways right now, but they are less than ideal: * Manually keep the two sync'ed (like above), a DRY violation. * Use introspection at definition time (using X.f.__defaults__ and X.f.__kwdefaults__), which is fragile (if the X.f attributes are changed later or the function definition's signature changes). * Use a dummy value to indicate that the default should be supplied by X's f(). This requires some clutter in the function and possibly a less obvious default argument. Here's an example of that last one: class DeferredType: pass DEFERRED = DeferredType() class X1: def f(self, name="N/A"): if name is DEFERRED: name = "N/A" # DRY #name = X1.f.__defaults__[0] # fragile ... class Y1(X1): def f(self, name=DEFERRED): super().f(name) Of course, the DEFERRED object could be any other specific object, particularly None. Here's another similar example that reminds me of Jack's example[1] from the recent thread on mutable default arguments. It is the solution I am using currently. class X2: def f(self, name=None): if name is None: name = "N/A" ... class Y2(X2): def f(self, name=None): super().f(name) Of course, using DEFERRED instead of None would allow you to use None as the actual default argument, but you get the idea. Notice that the actual default argument is no longer in the argument list, so it is less obvious and no longer introspect-able. I have also run into this use-case outside of inheritance situations. A New Solution --------------- Provide a builtin version of a Deferred singleton, like None or NotImplemented. When resolving arguments for a call, if an argument is this Deferred object, replace it with the default argument for that parameter on the called function, if there is one. If there isn't, act as though that argument was not passed at all. Here's how it would look: class X: def f(self, name="N/A"): ... class Y(X): def f(self, name=Deferred): super().f(name) [2] This way, Y's f has a sensible default argument for "name"[3] and implies where to find the actual default argument (on super().f)[4]. X's f is not cluttered up and has a clear, introspect-able default argument. Thoughts? -eric p.s. while I usually pursue random flights of fancy (answers looking for questions), this idea was born of an honest-to-goodness practical use-case I use semi-frequently. One thing I've learned about the Python community is that ideas that come from real [and common] use-cases have a much better chance of not dying a quick and painful death (not holding my breath here though ). [1] http://mail.python.org/pipermail/python-ideas/2011-May/010263.html [2] As a bonus, an "outer" function could take advantage of different default arguments when calling more than one function: def f(name="N/A"): ... def g(name="Undefined"): ... def h(name=Deferred): f(name) g(name) [3] "Deferred" indicates what kind of argument it is in a straightforward way. However, I don't want the name to be confused with anything like Twisted Deferred objects, so whatever name is clear would, of course, be appropriate. [4] That is true from a people perspective, but not for introspection; e.g. if you want to introspect the actual default argument for Y.f, you would have to be able to programmatically determine how that parameter is used in the function body... From tjreedy at udel.edu Wed Jul 13 23:36:19 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 13 Jul 2011 17:36:19 -0400 Subject: [Python-ideas] deferred default arguments In-Reply-To: References: Message-ID: On 7/13/2011 3:26 PM, Eric Snow wrote: > class X: > def f(self, name="N/A"): print(name) > > class Y(X): > def f(self, name="N/A"): > super().f(name) I believe class Y(X): def f(self, name=None): super().f(name) f.__defaults__ = X.f.__defaults__ will more or less do what you want. Using 'super()' instead of 'X' does not seem to work. The default replacement might be done with a function or class decorator. > Y().f() prints 'N/A' -- Terry Jan Reedy From paul at colomiets.name Wed Jul 13 23:39:23 2011 From: paul at colomiets.name (Paul Colomiets) Date: Thu, 14 Jul 2011 00:39:23 +0300 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: References: Message-ID: Hi, On Fri, Jul 8, 2011 at 2:08 AM, Giampaolo Rodol? wrote: >>>> isparent('/a', '/a/b') > True >>>> ischild('/a/b', '/a') > True Do you mean: >>> '/a' == dirname('/a/b') True >>> dirname('/a/b') == '/a' True I do not think it's harder to write. -- Paul From ericsnowcurrently at gmail.com Thu Jul 14 00:43:09 2011 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Wed, 13 Jul 2011 16:43:09 -0600 Subject: [Python-ideas] deferred default arguments In-Reply-To: References: Message-ID: On Wed, Jul 13, 2011 at 3:36 PM, Terry Reedy wrote: > On 7/13/2011 3:26 PM, Eric Snow wrote: > >> class X: >> ? ? def f(self, name="N/A"): > > ? ? ? ? print(name) >> >> class Y(X): >> ? ? def f(self, name="N/A"): >> ? ? ? ? super().f(name) > > I believe > > class Y(X): > ? ?def f(self, name=None): > ? ? ? ?super().f(name) > ? ?f.__defaults__ = X.f.__defaults__ > > will more or less do what you want. Using 'super()' instead of 'X' does not > seem to work. The default replacement might be done with a function or class > decorator. Yeah, but if the defaults of X.f get changed at runtime, after the definition of Y, the defaults for Y.f will likely be out of sync. Also, usually I want to be selective about which defaults I assume. Following your recommendation would involve calculating a new __defaults__ and a new __kwdefaults__. PEP 362 would make this easier, but probably won't be done for a while. However, you're right that a decorator could probably handle this. The downside is that it would not be a trivial decorator to get what I am after. Thanks for the feedback. -eric > >> Y().f() > > prints 'N/A' > > -- > Terry Jan Reedy > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From ben+python at benfinney.id.au Thu Jul 14 03:14:04 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 14 Jul 2011 11:14:04 +1000 Subject: [Python-ideas] os.path.isparent and os.path.ischild References: Message-ID: <87ipr5d5pf.fsf@benfinney.id.au> Paul Colomiets writes: > Hi, > > On Fri, Jul 8, 2011 at 2:08 AM, Giampaolo Rodol? wrote: > >>>> isparent('/a', '/a/b') > > True > >>>> ischild('/a/b', '/a') > > True > > Do you mean: > > >>> '/a' == dirname('/a/b') > True > >>> dirname('/a/b') == '/a' > True > > I do not think it's harder to write. I think this behaviour is also part of the expectation:: >>> path = '/a/b/c/d' >>> isparent('/a', path) True >>> isparent('/a/b', path) True >>> isparent('/a/b/z', path) False >>> ischild(path, '/a') True >>> ischild(path, '/a/b') True >>> ischild(path, '/a/b/z') False >>> '/a' == dirname(path) False >>> '/a/b' == dirname(path) False >>> '/a/b/z' == dirname(path) False -- \ ?See, in my line of work you gotta keep repeating things over | `\ and over and over again, for the truth to sink in; to kinda | _o__) catapult the propaganda.? ?George W. Bush, 2005-05 | Ben Finney From steve at pearwood.info Thu Jul 14 03:27:00 2011 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 14 Jul 2011 11:27:00 +1000 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: References: Message-ID: <4E1E45E4.4070408@pearwood.info> Paul Colomiets wrote: > Hi, > > On Fri, Jul 8, 2011 at 2:08 AM, Giampaolo Rodol? wrote: >>>>> isparent('/a', '/a/b') >> True >>>>> ischild('/a/b', '/a') >> True > > Do you mean: > >>>> '/a' == dirname('/a/b') > True >>>> dirname('/a/b') == '/a' > True > > I do not think it's harder to write. > How about this? isparent('/a/b/c', '/a/b/c/d/e/f/g') => should return True -- Steven From steve at pearwood.info Thu Jul 14 03:23:17 2011 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 14 Jul 2011 11:23:17 +1000 Subject: [Python-ideas] deferred default arguments In-Reply-To: References: Message-ID: <4E1E4505.2030201@pearwood.info> Eric Snow wrote: > As you know, you can use default arguments when defining a function. > Sometimes you want to pass an argument through from a function's > arguments to another function that the original ("outer") calls. I > run into this during inheritance-related calls and when interfacing > between two compatible APIs. Here's an example: > > class X: > def f(self, name="N/A"): > ... > > class Y(X): > def f(self, name="N/A"): > super().f(name) The problem is wider than inheritance of classes. I often have a series of functions that share identical signatures, including defaults: def mean(data, missing=False): pass def variance(data, missing=False): pass def pvariance(data, missing=False): pass def stdev(data, missing=False): pass def pstdev(data, missing=False): pass For API consistency, a change to one signature requires corresponding changes to the others. I'm not *entirely* sure that this is a problem that needs solving, although it is a nuisance. (See my final paragraph, below.) What do other languages provide? Wild idea: if Python had a "signature object", you could do something like this: common = signature(data, missing=False) def mean*common: pass where poor old * gets yet another meaning, namely "use this signature as the function parameter list". (I'm not wedded to that syntax, it was just the first idea that came to mind.) > * Manually keep the two sync'ed (like above), a DRY violation. > * Use introspection at definition time (using X.f.__defaults__ and > X.f.__kwdefaults__), which is fragile (if the X.f attributes are > changed later or the function definition's signature changes). This becomes messy if you only want to "inherit" the default value for one parameter rather than all. > * Use a dummy value to indicate that the default should be supplied by > X's f(). This requires some clutter in the function and possibly a > less obvious default argument. I believe that the usual name for this is a sentinel. > A New Solution > --------------- > > Provide a builtin version of a Deferred singleton, like None or > NotImplemented. When resolving arguments for a call, if an argument > is this Deferred object, replace it with the default argument for that > parameter on the called function, if there is one. If there isn't, > act as though that argument was not passed at all. I don't believe that should be a public object. If it's public, people will say "Yes, but what do you do if you want the default to actually be the Deferred singleton?" It's the None-as-sentinel problem all over again... sometimes you want None to stand in for no value, and sometimes you want it to be a first class value. Better(?) to make it syntax, and overload * yet again: def f(arg, another_arg, name=*): pass This has the advantage(?) that inside the body of f, name doesn't get a spurious value Deferred. If the caller doesn't supply a value for name, and f tries to use it (except as below), then you get an unambiguous UnboundLocalError. The exception is, calling another function with it as an argument is permitted. This call in the body of the function: result = g(1, 3, name, keyword="spam") behaves something like this: try: name except NameError: result = g(1, 3, keyword="spam") else: result = g(1, 3, name, keyword="spam") only handled by the compiler. Seems awfully complicated just to solve a few DRY violations. I'm not sure it is worth the added complexity of implementation, and the added cognitive burden of learning about it. That's not to say that DRY violations in function signatures aren't a real problem -- I've run into them myself. But I question how large a problem they are, in practice. Language support solving them seems to me to be a case of using a sledgehammer to crack a peanut. DRY is mostly a problem for implementation, not interface: implementation should remain free to change rapidly, and so repeating yourself makes that harder. But interface (e.g. your library API, including function signatures) should be static for long periods of time, so redundancy in function signatures is less important. -- Steven From greg.ewing at canterbury.ac.nz Thu Jul 14 05:00:18 2011 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 14 Jul 2011 15:00:18 +1200 Subject: [Python-ideas] deferred default arguments In-Reply-To: <4E1E4505.2030201@pearwood.info> References: <4E1E4505.2030201@pearwood.info> Message-ID: <4E1E5BC2.4030809@canterbury.ac.nz> Steven D'Aprano wrote: > def mean(data, missing=False): pass > def variance(data, missing=False): pass > def pvariance(data, missing=False): pass > def stdev(data, missing=False): pass > def pstdev(data, missing=False): pass > > For API consistency, a change to one signature requires corresponding > changes to the others. If you think you might change your mind about the default value of "missing", you can do default_missing = False def mean(data, missing=default_missing): ... def variance(data, missing=default_missing): ... etc. > If the caller doesn't supply a value for name, > and f tries to use it (except as below), then you get an unambiguous > UnboundLocalError. > > The exception is, calling another function with it as an argument is > permitted. That just leads to another version of the "None-as-sentinel" problem, though. What happens if you don't want to *directly* pass it to another function, but via some intermediate name or data structure? And how do debuggers deal with it? -- Greg From ericsnowcurrently at gmail.com Thu Jul 14 05:01:43 2011 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Wed, 13 Jul 2011 21:01:43 -0600 Subject: [Python-ideas] deferred default arguments In-Reply-To: <4E1E4505.2030201@pearwood.info> References: <4E1E4505.2030201@pearwood.info> Message-ID: On Wed, Jul 13, 2011 at 7:23 PM, Steven D'Aprano wrote: > Wild idea: if Python had a "signature object", you could do something like You mean like PEP 362? I would love it. Brett has talked about taking it from PyPI to 3.3, but I'm guessing he is still pretty busy with life. (There, I just poked the tracker issue) >> * Use introspection at definition time (using X.f.__defaults__ and >> X.f.__kwdefaults__), which is fragile (if the X.f attributes are >> changed later or the function definition's signature changes). > > This becomes messy if you only want to "inherit" the default value for one > parameter rather than all. Yeah, I had meant that, and agree it is messy. >> >> Provide a builtin version of a Deferred singleton, like None or >> NotImplemented. ?When resolving arguments for a call, if an argument >> is this Deferred object, replace it with the default argument for that >> parameter on the called function, if there is one. ?If there isn't, >> act as though that argument was not passed at all. > > I don't believe that should be a public object. If it's public, people will > say "Yes, but what do you do if you want the default to actually be the > Deferred singleton?" It's the None-as-sentinel problem all over again... > sometimes you want None to stand in for no value, and sometimes you want it > to be a first class value. The idea is to have an object that has no meaning in normal code. Sure we use None all over, but the sample size for builtin singletons in Python is pretty small to say that would happen with a new one. But maybe None started as an object that didn't have any meaning in normal code (doubt it). Agreed that if it takes meaning it won't be much different than using None as a sentinel. However, it would still not be None, which would still be worth it. > > Better(?) to make it syntax, and overload * yet again: > > def f(arg, another_arg, name=*): pass > > This has the advantage(?) that inside the body of f, name doesn't get a > spurious value Deferred. If the caller doesn't supply a value for name, and > f tries to use it (except as below), then you get an unambiguous > UnboundLocalError. > > The exception is, calling another function with it as an argument is > permitted. This call in the body of the function: > > ? ?result = g(1, 3, name, keyword="spam") > > behaves something like this: > > ? ?try: > ? ? ? ?name > ? ?except NameError: > ? ? ? ?result = g(1, 3, keyword="spam") > ? ?else: > ? ? ? ?result = g(1, 3, name, keyword="spam") > > only handled by the compiler. > > Seems awfully complicated just to solve a few DRY violations. I'm not sure > it is worth the added complexity of implementation, and the added cognitive > burden of learning about it. Yeah, that is neat, but it is a lot more complicated than having a builtin singleton that implicitly triggers a simple(?) behavior when passed as an argument to a function. You can trust me that my way is better! I'm an expert on the inner workings of the CPython implementation! At least, the singleton *seems* conceptually less complicated to me. > > That's not to say that DRY violations in function signatures aren't a real > problem -- I've run into them myself. But I question how large a problem > they are, in practice. Language support solving them seems to me to be a > case of using a sledgehammer to crack a peanut. DRY is mostly a problem for > implementation, not interface: implementation should remain free to change > rapidly, and so repeating yourself makes that harder. But interface (e.g. > your library API, including function signatures) should be static for long > periods of time, so redundancy in function signatures is less important. I gotta say I love how practical Python and its community is. Not only do things have to be useful, but widely useful and substantially more useful than the existing way of doing it. I remember Nick said something along those lines a few months ago. I think that attitude has kept the language awesome. In this situation, you're probably right that the use case isn't large enough in practice to be important for the language. For now I guess I can do something like this, which is along the lines of what I was imagining would happen implicitly: def handle_deferred(f, name, locals_, fname=None): # simpler with a function signature object... if not fname: fname = name if locals_[name] is not DEFERRED: return locals_[name] if name in f.__code__.co_kwonlyargs: try: return f.__kwdefaults__[fname] except KeyError: # TypeError would normally be raised if f were called and a # non-default parameter were not passed an argument so # that's what we'll raise here and below. raise TypeError("Can't convert DEFERRED for {}.format(name)) default_names = f.__code__._co_varnames[ f.__code__.co_argcount - len(f.__defaults__): f.__code__.co_argcount] try: return f.__defaults__[default_names.index(fname)] except ValueError: raise TypeError("Can't convert DEFERRED for {}.format(name)) def f(x=5): ... def g(x=DEFERRED): ... x = handle_deferred(f, "x", locals()) f(x) Thanks for having a look. -eric > > > > -- > Steven > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From greg.ewing at canterbury.ac.nz Thu Jul 14 04:49:56 2011 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 14 Jul 2011 14:49:56 +1200 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: References: Message-ID: <4E1E5954.5040609@canterbury.ac.nz> Paul Colomiets wrote: > Do you mean: > >>>>dirname('/a/b') == '/a' > > True It's not so easy if you want isparent("/a", "/a/b/c") to be true as well. -- Greg From cmjohnson.mailinglist at gmail.com Thu Jul 14 06:52:38 2011 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Wed, 13 Jul 2011 18:52:38 -1000 Subject: [Python-ideas] deferred default arguments In-Reply-To: References: <4E1E4505.2030201@pearwood.info> Message-ID: If you want perfect API compatibility from the sub-class to the super, why not just use *args, **kwargs? It kills the method signature, I guess, but just make your docstring say, "Check my superclass for the exact calling API." From tjreedy at udel.edu Thu Jul 14 06:58:54 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 14 Jul 2011 00:58:54 -0400 Subject: [Python-ideas] deferred default arguments In-Reply-To: References: Message-ID: On 7/13/2011 6:43 PM, Eric Snow wrote: > On Wed, Jul 13, 2011 at 3:36 PM, Terry Reedy wrote: >> On 7/13/2011 3:26 PM, Eric Snow wrote: >> >>> class X: >>> def f(self, name="N/A"): >> >> print(name) >>> >>> class Y(X): >>> def f(self, name="N/A"): >>> super().f(name) >> >> I believe >> >> class Y(X): >> def f(self, name=None): >> super().f(name) >> f.__defaults__ = X.f.__defaults__ >> >> will more or less do what you want. Using 'super()' instead of 'X' does not >> seem to work. The default replacement might be done with a function or class >> decorator. > > Yeah, but if the defaults of X.f get changed at runtime, That would be *extremely* unusual. I was not sure __defaults__ was writable until I tried it. -- Terry Jan Reedy From ericsnowcurrently at gmail.com Thu Jul 14 07:04:58 2011 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Wed, 13 Jul 2011 23:04:58 -0600 Subject: [Python-ideas] deferred default arguments In-Reply-To: References: Message-ID: On Wed, Jul 13, 2011 at 10:58 PM, Terry Reedy wrote: > On 7/13/2011 6:43 PM, Eric Snow wrote: >> Yeah, but if the defaults of X.f get changed at runtime, > > That would be *extremely* unusual. I was not sure __defaults__ was writable > until I tried it. Ah, what's possible vs. what's likely. I do agree with you. -eric > -- > Terry Jan Reedy > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From eric at trueblade.com Thu Jul 14 11:51:49 2011 From: eric at trueblade.com (Eric V. Smith) Date: Thu, 14 Jul 2011 05:51:49 -0400 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: <4E1E5954.5040609@canterbury.ac.nz> References: <4E1E5954.5040609@canterbury.ac.nz> Message-ID: <4E1EBC35.8000102@trueblade.com> On 7/13/2011 10:49 PM, Greg Ewing wrote: > Paul Colomiets wrote: > >> Do you mean: >> >>>>> dirname('/a/b') == '/a' >> >> True > > It's not so easy if you want isparent("/a", "/a/b/c") to > be true as well. Then isparent() seems like a bad name. isancestor(), maybe? Not that I think it needs to be added to the stdlib. From phd at phdru.name Thu Jul 14 12:08:11 2011 From: phd at phdru.name (Oleg Broytman) Date: Thu, 14 Jul 2011 14:08:11 +0400 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: <4E1EBC35.8000102@trueblade.com> References: <4E1E5954.5040609@canterbury.ac.nz> <4E1EBC35.8000102@trueblade.com> Message-ID: <20110714100811.GA23937@iskra.aviel.ru> On Thu, Jul 14, 2011 at 05:51:49AM -0400, Eric V. Smith wrote: > Then isparent() seems like a bad name. isancestor(), maybe? > Not that I think it needs to be added to the stdlib. It's already in the stdlib, called shutil.destinsrc ("destination in source"). Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From ben+python at benfinney.id.au Thu Jul 14 12:45:06 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 14 Jul 2011 20:45:06 +1000 Subject: [Python-ideas] os.path.isparent and os.path.ischild References: <4E1E5954.5040609@canterbury.ac.nz> <4E1EBC35.8000102@trueblade.com> <20110714100811.GA23937@iskra.aviel.ru> Message-ID: <87aachcf9p.fsf@benfinney.id.au> Oleg Broytman writes: > It's already in the stdlib, called shutil.destinsrc ("destination > in source"). Not according to nor ; no mention of a ?destinsrc?. Which documentation should we be looking at to find that? -- \ ?The best is the enemy of the good.? ?Voltaire, _Dictionnaire | `\ Philosophique_ | _o__) | Ben Finney From phd at phdru.name Thu Jul 14 13:11:24 2011 From: phd at phdru.name (Oleg Broytman) Date: Thu, 14 Jul 2011 15:11:24 +0400 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: <87aachcf9p.fsf@benfinney.id.au> References: <4E1E5954.5040609@canterbury.ac.nz> <4E1EBC35.8000102@trueblade.com> <20110714100811.GA23937@iskra.aviel.ru> <87aachcf9p.fsf@benfinney.id.au> Message-ID: <20110714111124.GA25200@iskra.aviel.ru> On Thu, Jul 14, 2011 at 08:45:06PM +1000, Ben Finney wrote: > Oleg Broytman writes: > > It's already in the stdlib, called shutil.destinsrc ("destination > > in source"). > > Not according to nor > ; no mention of a > ???destinsrc???. Which documentation should we be looking at to find that? In py3k it was renamed to _destinsrc. It never was documented. So to "add" it to the stdlib just rename it back and document it. Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From ben+python at benfinney.id.au Thu Jul 14 15:10:09 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 14 Jul 2011 23:10:09 +1000 Subject: [Python-ideas] os.path.isparent and os.path.ischild References: <4E1E5954.5040609@canterbury.ac.nz> <4E1EBC35.8000102@trueblade.com> <20110714100811.GA23937@iskra.aviel.ru> <87aachcf9p.fsf@benfinney.id.au> <20110714111124.GA25200@iskra.aviel.ru> Message-ID: <874o2pc8jy.fsf@benfinney.id.au> Oleg Broytman writes: > In py3k it was renamed to _destinsrc. It never was documented. Both of which strongly indicate the intent that it shouldn't be used as part of the library API. So it is disingenuous to suggest that it's already in the standard library; it's nice that there's an implementation there, but it's clearly being moved further from intended use. Who made that change? Presumably their opinion on this matter is opposite to the proponents of the functionality in this thread. Can someone ask them to comment in this thread as to their rationale? -- \ ?Beware of bugs in the above code; I have only proved it | `\ correct, not tried it.? ?Donald Knuth, 1977-03-29 | _o__) | Ben Finney From arnodel at gmail.com Thu Jul 14 16:34:57 2011 From: arnodel at gmail.com (Arnaud Delobelle) Date: Thu, 14 Jul 2011 15:34:57 +0100 Subject: [Python-ideas] os.path.isparent and os.path.ischild In-Reply-To: <874o2pc8jy.fsf@benfinney.id.au> References: <4E1E5954.5040609@canterbury.ac.nz> <4E1EBC35.8000102@trueblade.com> <20110714100811.GA23937@iskra.aviel.ru> <87aachcf9p.fsf@benfinney.id.au> <20110714111124.GA25200@iskra.aviel.ru> <874o2pc8jy.fsf@benfinney.id.au> Message-ID: On 14 July 2011 14:10, Ben Finney wrote: > Oleg Broytman writes: > >> ? ?In py3k it was renamed to _destinsrc. It never was documented. > > Both of which strongly indicate the intent that it shouldn't be used as > part of the library API. > > So it is disingenuous to suggest that it's already in the standard > library; it's nice that there's an implementation there, but it's > clearly being moved further from intended use. > > Who made that change? Presumably their opinion on this matter is > opposite to the proponents of the functionality in this thread. Can > someone ask them to comment in this thread as to their rationale? It's easy to find out who made the change: http://hg.python.org/cpython/rev/0ab36796dd79 -- Arnaud From ben+python at benfinney.id.au Thu Jul 14 23:34:52 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 15 Jul 2011 07:34:52 +1000 Subject: [Python-ideas] =?utf-8?q?Why_make_=E2=80=98shutil=2Edestinsrc?= =?utf-8?b?4oCZIHByaXZhdGU/ICh3YXM6IG9zLnBhdGguaXNwYXJlbnQgYW5kIG9zLnBh?= =?utf-8?q?th=2Eischild=29?= References: <4E1E5954.5040609@canterbury.ac.nz> <4E1EBC35.8000102@trueblade.com> <20110714100811.GA23937@iskra.aviel.ru> <87aachcf9p.fsf@benfinney.id.au> <20110714111124.GA25200@iskra.aviel.ru> <874o2pc8jy.fsf@benfinney.id.au> Message-ID: <87zkkgbl6r.fsf_-_@benfinney.id.au> Arnaud Delobelle writes: > On 14 July 2011 14:10, Ben Finney wrote: > > Oleg Broytman writes: > > > >> ? ?In py3k it was renamed to _destinsrc. It never was documented. > > > > Both of which strongly indicate the intent that it shouldn't be used > > as part of the library API. [?] > > Who made that change? Presumably their opinion on this matter is > > opposite to the proponents of the functionality in this thread. Can > > someone ask them to comment in this thread as to their rationale? > > It's easy to find out who made the change: I'm glad you found it easy, and thank you for doing so. > http://hg.python.org/cpython/rev/0ab36796dd79 Made by Benjamin Peterson on 2009-02-07, with the commit message ?make destinsrc private?. Benjamin, what was the rationale for making ?shutil.destinsrc? no longer part of the public API? I'm asking in the context of a thread that suggests adding that exact functionality to the standard library, before we were aware of this code being there. -- \ ?I am too firm in my consciousness of the marvelous to be ever | `\ fascinated by the mere supernatural ?? ?Joseph Conrad, _The | _o__) Shadow-Line_ | Ben Finney From benjamin at python.org Thu Jul 14 23:52:15 2011 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 14 Jul 2011 21:52:15 +0000 (UTC) Subject: [Python-ideas] =?utf-8?q?Why_make_=E2=80=98shutil=2Edestinsrc?= =?utf-8?b?4oCZIHByaXZhdGU/ICh3YXM6IG9zLnBhdGguaXNwYXJlbnQgYW5kIG9z?= =?utf-8?q?=2Epath=2Eischild=29?= References: <4E1E5954.5040609@canterbury.ac.nz> <4E1EBC35.8000102@trueblade.com> <20110714100811.GA23937@iskra.aviel.ru> <87aachcf9p.fsf@benfinney.id.au> <20110714111124.GA25200@iskra.aviel.ru> <874o2pc8jy.fsf@benfinney.id.au> <87zkkgbl6r.fsf_-_@benfinney.id.au> Message-ID: Ben Finney writes: > Benjamin, what was the rationale for making ?shutil.destinsrc? no longer > part of the public API? Please see http://bugs.python.org/issue2047 From ben+python at benfinney.id.au Fri Jul 15 00:59:18 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 15 Jul 2011 08:59:18 +1000 Subject: [Python-ideas] =?utf-8?q?Why_make_=E2=80=98shutil=2Edestinsrc?= =?utf-8?b?4oCZICBwcml2YXRlPw==?= References: <4E1E5954.5040609@canterbury.ac.nz> <4E1EBC35.8000102@trueblade.com> <20110714100811.GA23937@iskra.aviel.ru> <87aachcf9p.fsf@benfinney.id.au> <20110714111124.GA25200@iskra.aviel.ru> <874o2pc8jy.fsf@benfinney.id.au> <87zkkgbl6r.fsf_-_@benfinney.id.au> Message-ID: <87tyaobha1.fsf@benfinney.id.au> Benjamin Peterson writes: > Ben Finney writes: > > Benjamin, what was the rationale for making ?shutil.destinsrc? no longer > > part of the public API? > > Please see http://bugs.python.org/issue2047 Thank you. (Has the Python development team considered a policy of noting bug report URLs in the commits that address those bug reports? That would have made the forensic work easier.) So AFAICT the strongest justification for the change has its origin in by Raghuram Devarakonda: destinsrc() is an internal function used only in shutil.move(). But I don't see an argument made against Joseph Armbruster's suggestion : I would have expected to find a function like this in os.path, not shutil. -- \ ?Come on Milhouse, there?s no such thing as a soul! It?s just | `\ something they made up to scare kids, like the Boogie Man or | _o__) Michael Jackson.? ?Bart, _The Simpsons_ | Ben Finney From tjreedy at udel.edu Fri Jul 15 05:04:01 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 14 Jul 2011 23:04:01 -0400 Subject: [Python-ideas] =?utf-8?q?Why_make_=E2=80=98shutil=2Edestinsrc?= =?utf-8?b?4oCZICBwcml2YXRlPw==?= In-Reply-To: <87tyaobha1.fsf@benfinney.id.au> References: <4E1E5954.5040609@canterbury.ac.nz> <4E1EBC35.8000102@trueblade.com> <20110714100811.GA23937@iskra.aviel.ru> <87aachcf9p.fsf@benfinney.id.au> <20110714111124.GA25200@iskra.aviel.ru> <874o2pc8jy.fsf@benfinney.id.au> <87zkkgbl6r.fsf_-_@benfinney.id.au> <87tyaobha1.fsf@benfinney.id.au> Message-ID: On 7/14/2011 6:59 PM, Ben Finney wrote: > Benjamin Peterson > writes: > >> Ben Finney writes: >>> Benjamin, what was the rationale for making ?shutil.destinsrc? no longer >>> part of the public API? >> >> Please see http://bugs.python.org/issue2047 > > Thank you. (Has the Python development team considered a policy of > noting bug report URLs in the commits that address those bug reports? > That would have made the forensic work easier.) This is now standard. If 'issue #xxxx' appears in the commit message, a message reporting the commit is auto-generated and sent to the tracker. Since this saves developer effort, it usually is done. -- Terry Jan Reedy From benjamin at python.org Fri Jul 15 05:45:51 2011 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 15 Jul 2011 03:45:51 +0000 (UTC) Subject: [Python-ideas] =?utf-8?q?Why_make_=E2=80=98shutil=2Edestinsrc?= =?utf-8?b?4oCZIHByaXZhdGU/?= References: <4E1E5954.5040609@canterbury.ac.nz> <4E1EBC35.8000102@trueblade.com> <20110714100811.GA23937@iskra.aviel.ru> <87aachcf9p.fsf@benfinney.id.au> <20110714111124.GA25200@iskra.aviel.ru> <874o2pc8jy.fsf@benfinney.id.au> <87zkkgbl6r.fsf_-_@benfinney.id.au> <87tyaobha1.fsf@benfinney.id.au> Message-ID: Ben Finney writes: > > Benjamin Peterson python.org> > > Please see http://bugs.python.org/issue2047 > > Thank you. (Has the Python development team considered a policy of > noting bug report URLs in the commits that address those bug reports? > That would have made the forensic work easier.) They generally do when they remember. :) > > So AFAICT the strongest justification for the change has its origin in > by Raghuram Devarakonda: > > destinsrc() is an internal function used only in shutil.move(). > > But I don't see an argument made against Joseph Armbruster's suggestion > : > > I would have expected to find a function like this in os.path, not > shutil. Sounds more like an observation than a suggestion. I personally don't care whether/where this functionality is exposed, so by all means file an issue if you want. From ncoghlan at gmail.com Fri Jul 15 06:38:47 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 15 Jul 2011 14:38:47 +1000 Subject: [Python-ideas] deferred default arguments In-Reply-To: <4E1E4505.2030201@pearwood.info> References: <4E1E4505.2030201@pearwood.info> Message-ID: On Thu, Jul 14, 2011 at 11:23 AM, Steven D'Aprano wrote: > That's not to say that DRY violations in function signatures aren't a real > problem -- I've run into them myself. But I question how large a problem > they are, in practice. Language support solving them seems to me to be a > case of using a sledgehammer to crack a peanut. DRY is mostly a problem for > implementation, not interface: implementation should remain free to change > rapidly, and so repeating yourself makes that harder. But interface (e.g. > your library API, including function signatures) should be static for long > periods of time, so redundancy in function signatures is less important. +1 A PEP 362 based answer would likely involve assigning to __signature__ on the wrapping function and making good use of Signature.bind() inside it. There are also much bigger problems that PEP 362 will let us solve (e.g. decent introspection of builtin functions, and functions that use pass-through *args and **kwds). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From mikegraham at gmail.com Fri Jul 15 15:57:35 2011 From: mikegraham at gmail.com (Mike Graham) Date: Fri, 15 Jul 2011 09:57:35 -0400 Subject: [Python-ideas] Deprecate str.find Message-ID: str.find (and bytes.find) is worse than the alternatives in every way. It should be explicitly deprecated in favour of str.__contains__ and str.index. str.find when used to check for substring is inferior to the in operator. "if sub in s:" is shorter, easier-to-read, and more efficient than "if s.find(sub) != -1:" and is not prone to the error "if s.find(sub):" I have occasionally seen. str.index is better for finding indices in that it supports an idiomatic exception-based API rather than a return-code API. Every usage of str.find should look like "index = s.find(sub); if index == -1: (exception code)", which is an antipattern in Python. This problem is compounded by the fact that the returned value is actually a valid value; consider s = 'bar'--s[s.find('x')] is somewhat surprisingly 'r'. Additionally, the existence of str.find violates the there's-one-way-to-do-it principle. Mike From donspauldingii at gmail.com Fri Jul 15 16:05:11 2011 From: donspauldingii at gmail.com (Don Spaulding) Date: Fri, 15 Jul 2011 09:05:11 -0500 Subject: [Python-ideas] None-as-sentinel alternatives? (was: deferred default arguments) Message-ID: On Wed, Jul 13, 2011 at 8:23 PM, Steven D'Aprano wrote: > Eric Snow wrote: > > A New Solution >> --------------- >> >> Provide a builtin version of a Deferred singleton, like None or >> NotImplemented. When resolving arguments for a call, if an argument >> is this Deferred object, replace it with the default argument for that >> parameter on the called function, if there is one. If there isn't, >> act as though that argument was not passed at all. >> > > I don't believe that should be a public object. If it's public, people will > say "Yes, but what do you do if you want the default to actually be the > Deferred singleton?" It's the None-as-sentinel problem all over again... > sometimes you want None to stand in for no value, and sometimes you want it > to be a first class value. When I come across a situation where I want a sentinel that doesn't preclude None from being a valid value, I've always spelled it like this: SENTINEL = object() # Guaranteed to be unique and not None. def f(arg=SENTINEL): if arg is SENTINEL: arg = 'foo' x(arg) It's always worked well, but that little voice in my head tells me there's probably something I'm not thinking of that will come back to bite me in the future. I don't think this helps the OP in any meaningful way, but I was wondering if you bright people might shed some light on the drawbacks of this approach for me. What are the better alternatives? -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikegraham at gmail.com Fri Jul 15 16:18:47 2011 From: mikegraham at gmail.com (Mike Graham) Date: Fri, 15 Jul 2011 10:18:47 -0400 Subject: [Python-ideas] None-as-sentinel alternatives? (was: deferred default arguments) In-Reply-To: References: Message-ID: On Fri, Jul 15, 2011 at 10:05 AM, Don Spaulding wrote: > When I come across a situation where I want a sentinel that doesn't preclude > None from being a valid value, I've always spelled it like this: > ? ? SENTINEL = object() ?# Guaranteed to be unique and not None. > > ? ? def f(arg=SENTINEL): > ? ? ? ? if arg is SENTINEL: > ? ? ? ? ? ? arg = 'foo' > ? ? ? ? x(arg) > It's always worked well, but that little voice in my head tells me there's > probably something I'm not thinking of that will come back to bite me in the > future. ?I don't think this helps the OP in any meaningful way, but I was > wondering if you bright people might shed some light on the drawbacks of > this approach for me. ?What are the better alternatives? No, this approach isn't error-prone. For non-idea questions you might consider comp.lang.python in the future. =) Mike From guido at python.org Fri Jul 15 16:57:24 2011 From: guido at python.org (Guido van Rossum) Date: Fri, 15 Jul 2011 07:57:24 -0700 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: Message-ID: However, in many cases absence of the string is not an error -- you just need to do something else. So in cases where *if* it's found you need the position, and *if* it isn't found you need to do something else, you'd have to use a try/except block to catch the non-error that is absence. All in all I don't see enough reason to start deprecating find. But perhaps popular lint-like programs could flag likely abuses of find? --Guido On Fri, Jul 15, 2011 at 6:57 AM, Mike Graham wrote: > str.find (and bytes.find) is worse than the alternatives in every way. > It should be explicitly deprecated in favour of str.__contains__ and > str.index. > > str.find when used to check for substring is inferior to the in > operator. "if sub in s:" is shorter, easier-to-read, and more > efficient than "if s.find(sub) != -1:" and is not prone to the error > "if s.find(sub):" I have occasionally seen. > > str.index is better for finding indices in that it supports an > idiomatic exception-based API rather than a return-code API. Every > usage of str.find should look like "index = s.find(sub); if index == > -1: (exception code)", which is an antipattern in Python. This problem > is compounded by the fact that the returned value is actually a valid > value; consider s = 'bar'--s[s.find('x')] is somewhat surprisingly > 'r'. > > Additionally, the existence of str.find violates the > there's-one-way-to-do-it principle. > > > Mike > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- --Guido van Rossum (python.org/~guido) From mikegraham at gmail.com Fri Jul 15 18:12:33 2011 From: mikegraham at gmail.com (Mike Graham) Date: Fri, 15 Jul 2011 12:12:33 -0400 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: Message-ID: On Fri, Jul 15, 2011 at 10:57 AM, Guido van Rossum wrote: > However, in many cases absence of the string is not an error -- you > just need to do something else. So in cases where *if* it's found you > need the position, and *if* it isn't found you need to do something > else, you'd have to use a try/except block to catch the non-error that > is absence. All in all I don't see enough reason to start deprecating > find. But perhaps popular lint-like programs could flag likely abuses > of find? > > --Guido It isn't necessarily an error if the substring is not in the string (though it sometimes is), but it is an exceptional case. Python uses exceptions pretty liberally most places -- it isn't necessarily an error if an iterator is exhausted or if float("4.2 bad user input") is called or if BdbQuit was raised. In these cases, an exception can be perfectly expected to indicate that what happened is different from the information used in a return value. Making a Python user write a try/except block when she wants to handle both the cases "substring is in s" and "substring isn't in s" seems perfectly fine to me and, really, preferable to the if statement required to handle these two cases. The base two cases really are about the same: try: i = s.index(sub) except IndexError: do_something() vs. i = s.find(sub) if i == -1: do_something() But what if I forgot to handle the special case? i = s.index(sub) # An exception is raised right here and I can fix my code vs. i = s.find(sub) # No exception is raised In this second case, I get the value of -1. Later I can use it as an index, use it in a slice, or perform arithmetic on it. This can introduce seemingly-unrelated values later on, making this especially hard to track down. If the failure return code was at least None it would behave more sanely, but at present the failure return code is a perfectly valid value for almost any use. If a programmer is sill averse to using try/except, we can still write if sub in s: i = s.index(sub) else: do_something() Now. we can dredge up some examples where -1 is the actual value someone wants to use. These cases are so rare and so subtle as to make their use so clever I don't really see their existence as an advantage. Additionally, it is unfortunate that we currently have two methods to do the same thing (which isn't even a super-common task) with different APIs. Nothing about the names "find" and "index" really makes clear which is which. This violates the "There should be one-- and preferably only one --obvious way to do it." principle and makes the Python user need to memorize an unnecessary, arbitrary distinction. I would also point out that it was not a contrived case that I mentioned where a beginner introduces a bug by trying "if s.find(sub):" instead of "if sub in s:"; I have really seen people try this several times. Obviously we cannot make many decisions based on new Python programmers' mistakes, it is worth recognizing them. I hope this additional discussion might be able to sway your opinion here. The only advantage to using str.find is that you do not have to use try/except blocks, but in fact you don't have to with str.index either. On the other hand, there are numerous disadvantages--practical, pedagogical, stylistic, and design--to having and using str.find. Mike From solipsis at pitrou.net Fri Jul 15 18:31:29 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 15 Jul 2011 18:31:29 +0200 Subject: [Python-ideas] Deprecate str.find References: Message-ID: <20110715183129.27b67946@pitrou.net> On Fri, 15 Jul 2011 12:12:33 -0400 Mike Graham wrote: > [...] > Now. we can dredge up some examples where -1 is the actual value > someone wants to use. These cases are so rare and so subtle as to make > their use so clever I don't really see their existence as an > advantage. > > Additionally, it is unfortunate that we currently have two methods to > do the same thing (which isn't even a super-common task) with > different APIs. Nothing about the names "find" and "index" really > makes clear which is which. This violates the "There should be one-- > and preferably only one --obvious way to do it." principle and makes > the Python user need to memorize an unnecessary, arbitrary > distinction. While this would be a very good argument to make if we were currently designing the str API, I don't think the benefits of suppressing str.find outweight the burden of converting old code to use a different idiom. We could choose to write something about it in the documentation, though. Regards Antoine. From guido at python.org Fri Jul 15 18:38:47 2011 From: guido at python.org (Guido van Rossum) Date: Fri, 15 Jul 2011 09:38:47 -0700 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: Message-ID: On Fri, Jul 15, 2011 at 9:12 AM, Mike Graham wrote: > It isn't necessarily an error if the substring is not in the string > (though it sometimes is), but it is an exceptional case. Python uses > exceptions pretty liberally most places -- it isn't necessarily an > error if an iterator is exhausted or if float("4.2 bad user input") is > called or if BdbQuit was raised. In these cases, an exception can be > perfectly expected to indicate that what happened is different from > the information used in a return value. This smells a bit like uncalled-for religion. Remember that readline() returns an empty string at the end of the file instead of raising an exception, and IMO that makes it the better API. -- --Guido van Rossum (python.org/~guido) From mikegraham at gmail.com Fri Jul 15 18:47:31 2011 From: mikegraham at gmail.com (Mike Graham) Date: Fri, 15 Jul 2011 12:47:31 -0400 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: Message-ID: On Fri, Jul 15, 2011 at 12:38 PM, Guido van Rossum wrote: > On Fri, Jul 15, 2011 at 9:12 AM, Mike Graham wrote: >> It isn't necessarily an error if the substring is not in the string >> (though it sometimes is), but it is an exceptional case. Python uses >> exceptions pretty liberally most places -- it isn't necessarily an >> error if an iterator is exhausted or if float("4.2 bad user input") is >> called or if BdbQuit was raised. In these cases, an exception can be >> perfectly expected to indicate that what happened is different from >> the information used in a return value. > > This smells a bit like uncalled-for religion. Remember that readline() > returns an empty string at the end of the file instead of raising an > exception, and IMO that makes it the better API. > > -- > --Guido van Rossum (python.org/~guido) > Thanks for this prompt, considerate reply (and the other). I really didn't mean to present an "always use exceptions" dogma here, so much as to claim "Having the programmer use a try/except block isn't a problem," and to say that the fact an exception was raised doesn't necessarily indicate that something went wrong. StopIteration is a strong example that exceptions can often be very expected and normal. Mike From mikegraham at gmail.com Fri Jul 15 18:51:40 2011 From: mikegraham at gmail.com (Mike Graham) Date: Fri, 15 Jul 2011 12:51:40 -0400 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <20110715183129.27b67946@pitrou.net> References: <20110715183129.27b67946@pitrou.net> Message-ID: On Fri, Jul 15, 2011 at 12:31 PM, Antoine Pitrou wrote: > While this would be a very good argument to make if we were currently > designing the str API, I don't think the benefits of suppressing > str.find outweight the burden of converting old code to use a different > idiom. > > We could choose to write something about it in the documentation, > though. > > Regards > > Antoine. That could be just as suitable. To be honest, I can't really see too much of a difference between a stronger note about the problems I perceive with str.find in the documentation and a note that uses the word "deprecated". There certainly wouldn't be occasion in the foreseeable future for you actually to remove str.find. Mike From g.brandl at gmx.net Fri Jul 15 18:59:46 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 15 Jul 2011 18:59:46 +0200 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <20110715183129.27b67946@pitrou.net> References: <20110715183129.27b67946@pitrou.net> Message-ID: Am 15.07.2011 18:31, schrieb Antoine Pitrou: > On Fri, 15 Jul 2011 12:12:33 -0400 > Mike Graham wrote: >> [...] >> Now. we can dredge up some examples where -1 is the actual value >> someone wants to use. These cases are so rare and so subtle as to make >> their use so clever I don't really see their existence as an >> advantage. >> >> Additionally, it is unfortunate that we currently have two methods to >> do the same thing (which isn't even a super-common task) with >> different APIs. Nothing about the names "find" and "index" really >> makes clear which is which. This violates the "There should be one-- >> and preferably only one --obvious way to do it." principle and makes >> the Python user need to memorize an unnecessary, arbitrary >> distinction. > > While this would be a very good argument to make if we were currently > designing the str API, I don't think the benefits of suppressing > str.find outweight the burden of converting old code to use a different > idiom. > > We could choose to write something about it in the documentation, > though. Patches are welcome at bugs.python.org ;) Georg From bruce at leapyear.org Fri Jul 15 19:06:27 2011 From: bruce at leapyear.org (Bruce Leban) Date: Fri, 15 Jul 2011 10:06:27 -0700 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: <20110715183129.27b67946@pitrou.net> Message-ID: It seems to me that the solution is that you should never use find and use index instead. You could even modify pylint/etc. to flag any use of find as an error. That doesn't mean it should be deprecated so everyone else has to follow you. Deprecated now => removed in the future. While many can agree that warnings in the documentation about common mistakes in using APIs is a good thing, that doesn't translate into consensus that we should remove the API that might be misused. The documentation already says "The find() method should be used only if you need to know the position of sub. To check if sub is a substring or not, use the in operator." There is *no* general solution to the problem of people misusing APIs. Even with index someone can still write if s.index(sub): which probably doesn't do what they were thinking. Should we remove that too? Note that one reason people might prefer find over index is that exceptions constrain how you write the code: try: i = s.index(sub) do lots of stuff with s and i except ValueError: result = 'not found' In this case the try wraps way too much code so it could catch a ValueError in the middle of 'stuff'. Here's correct code: try: i = s.index(sub) do_more = True except: result = 'not found' do_more = False if do_more: do lots of stuff with s and i Do you think that's better than: i = s.find(sub) if i < 0: result = 'not found' else: do lots of stuff with s and i --- Bruce Follow me: http://www.twitter.com/Vroo http://www.vroospeak.com On Fri, Jul 15, 2011 at 9:51 AM, Mike Graham wrote: > On Fri, Jul 15, 2011 at 12:31 PM, Antoine Pitrou > wrote: > > While this would be a very good argument to make if we were currently > > designing the str API, I don't think the benefits of suppressing > > str.find outweight the burden of converting old code to use a different > > idiom. > > > > We could choose to write something about it in the documentation, > > though. > > > > Regards > > > > Antoine. > > > That could be just as suitable. To be honest, I can't really see too > much of a difference between a stronger note about the problems I > perceive with str.find in the documentation and a note that uses the > word "deprecated". There certainly wouldn't be occasion in the > foreseeable future for you actually to remove str.find. > > Mike > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeanpierreda at gmail.com Fri Jul 15 19:13:06 2011 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Fri, 15 Jul 2011 13:13:06 -0400 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: Message-ID: On Fri, Jul 15, 2011 at 12:38 PM, Guido van Rossum wrote: > This smells a bit like uncalled-for religion. Remember that readline() > returns an empty string at the end of the file instead of raising an > exception, and IMO that makes it the better API. The last time I used file.readline was to pass to tokenize, which is complicated immensely by the logic necessary to support both the iterator protocol and the readline protocol. i.e. it has to check for both StopIteration _and_ the empty string, which makes it harder to use, forces it to contain more redundant information. I'm not swayed by the suggestion that file.readline is beautiful at all: it predates the iterator protocol and should have been subsumed by it. Instead it's another inconsistency in Python that tools have to deal with or ignore completely. Strings don't even bother to be compatible with .find, if you pass in an index that was returned by .find() you might just get the wrong result. It's pretty much only ever useful in the if check, which is in essence identical to the try-except, except it adds the ability to forget to check and produce incorrect behaviour -- this is not a feature. Devin From ethan at stoneleaf.us Fri Jul 15 19:30:50 2011 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 15 Jul 2011 10:30:50 -0700 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: Message-ID: <4E20794A.2010104@stoneleaf.us> Mike Graham wrote: > str.find (and bytes.find) is worse than the alternatives in every way. I disagree. > Additionally, the existence of str.find violates the > there's-one-way-to-do-it principle. The principle is "One *Obvious* Way To Do It" not *Only* One Way. Sometimes which one is 'obvious' is only clear from reading the documentation or learning the language in some other way -- but that doesn't mean it always the _better_ way. I, for one, have zero interest in losing good functionality because somebody else is misusing a feature. Exceptions are not /always/ the best way to do something. ~Ethan~ From ethan at stoneleaf.us Fri Jul 15 19:34:03 2011 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 15 Jul 2011 10:34:03 -0700 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: Message-ID: <4E207A0B.2050506@stoneleaf.us> Devin Jeanpierre wrote: > Strings don't even bother to be compatible with .find, if you pass in > an index that was returned by .find() you might just get the wrong > result. It's pretty much only ever useful in the if check, which is in > essence identical to the try-except, except it adds the ability to > forget to check and produce incorrect behaviour -- this is not a feature. Hind-sight -- what a great thing! I do think None would be a better return value. I suspect the odds of that changing now are pretty slim, though. ~Ethan~ From mikegraham at gmail.com Fri Jul 15 19:18:51 2011 From: mikegraham at gmail.com (Mike Graham) Date: Fri, 15 Jul 2011 13:18:51 -0400 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: <20110715183129.27b67946@pitrou.net> Message-ID: On Fri, Jul 15, 2011 at 1:06 PM, Bruce Leban wrote: > It seems to me that the solution is that you should never use find and use > index instead.?You could even modify pylint/etc. to flag any use of find as > an error. > That doesn't mean it should be deprecated so everyone else has to follow > you. Deprecated now => removed in the future. While many can agree that > warnings in the documentation about common mistakes in using APIs is a good > thing, that doesn't translate into consensus that we should remove the API > that might be misused. The documentation already says "The find() method > should be used only if you need to know the position of sub. To check if sub > is a substring or not, use the in operator." > There is *no* general solution to the problem of people misusing APIs. Even > with index someone can still write > > if s.index(sub): > > which probably doesn't do what they were thinking. Should we remove that > too? The case of "if s.index(sub):" as a misspelling of "if sub in s:" is not nearly as errorprone as the case of "if s.find(sub):". For one, "s.find(sub)" *sounds* like it might be true if you can find sub in s and not true if sub cannot be found. With str.index, the fact the return value is going to be an index is made clear. For two, str.find can fail silently in more cases than the str.index case can fail silently. For three, I've never see a learner write "if s.index(sub)" though I have seen several mistakenly write the "if s.find(sub)" version. > Note that one reason people might prefer find over index is that exceptions > constrain how you write the code: > > try: > ? ? i = s.index(sub) > ? ? do lots of stuff with s and i > except ValueError: > ? ? result = 'not found' > > In this case the try wraps way too much code so it could catch a ValueError > in the middle of 'stuff'. Here's correct code: > > try: > ? ? i = s.index(sub) > ? ? do_more = True > except: > ? ? result = 'not found' > ? ? do_more = False > if do_more: > ? ? do lots of stuff with s and i > > Do you think that's better than: > > i = s.find(sub) > if i < 0: > ? ? result = 'not found' > else: > ? ? do lots of stuff with s and i Are you familiar with the "else" clause of a try/except suite? It already handles writing the "right" version nicely. try: i = s.index(sub) except IndexError: result = 'not found' else: do lots of stuff with s and i > --- Bruce Mike From masklinn at masklinn.net Fri Jul 15 19:18:58 2011 From: masklinn at masklinn.net (Masklinn) Date: Fri, 15 Jul 2011 19:18:58 +0200 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: <20110715183129.27b67946@pitrou.net> Message-ID: <128B85CD-3AB1-497D-90A3-03CD152B8C3C@masklinn.net> On 2011-07-15, at 19:06 , Bruce Leban wrote: > It seems to me that the solution is that you should never use find and use > index instead. You could even modify pylint/etc. to flag any use of find as > an error. > > That doesn't mean it should be deprecated so everyone else has to follow > you. Deprecated now => removed in the future. While many can agree that > warnings in the documentation about common mistakes in using APIs is a good > thing, that doesn't translate into consensus that we should remove the API > that might be misused. The documentation already says "The find() method > should be used only if you need to know the position of sub. To check if sub > is a substring or not, use the in operator." > > There is *no* general solution to the problem of people misusing APIs. Even > with index someone can still write > > if s.index(sub): > > which probably doesn't do what they were thinking. Should we remove that > too? > > Note that one reason people might prefer find over index is that exceptions > constrain how you write the code: > > try: > i = s.index(sub) > do lots of stuff with s and i > except ValueError: > result = 'not found' > > > In this case the try wraps way too much code so it could catch a ValueError > in the middle of 'stuff'. Here's correct code: > > try: > i = s.index(sub) > do_more = True > except: > result = 'not found' > do_more = False > if do_more: > do lots of stuff with s and i > Note that you could replace `do_more` by a use of `i`: i = None try: i = s.index(sub) except ValueError: result = 'not found' if i is not None: do stuff with s and i > > Do you think that's better than: > > i = s.find(sub) > if i < 0: > result = 'not found' > else: > do lots of stuff with s and i There's always the `in` alternative: if sub not in s: result = 'not found' else: do stuff with s and s.index(sub) it's a more fluent interface, and uses less error-prone APIs, though it does search twice through `s`. (an other option would be to build this feature via higher-order functions, but that's not really one of Python's forte) From donspauldingii at gmail.com Fri Jul 15 19:22:03 2011 From: donspauldingii at gmail.com (Don Spaulding) Date: Fri, 15 Jul 2011 12:22:03 -0500 Subject: [Python-ideas] None-as-sentinel alternatives? (was: deferred default arguments) In-Reply-To: References: Message-ID: On Fri, Jul 15, 2011 at 9:18 AM, Mike Graham wrote: > On Fri, Jul 15, 2011 at 10:05 AM, Don Spaulding > wrote: > > When I come across a situation where I want a sentinel that doesn't > preclude > > None from being a valid value, I've always spelled it like this: > > SENTINEL = object() # Guaranteed to be unique and not None. > > > > def f(arg=SENTINEL): > > if arg is SENTINEL: > > arg = 'foo' > > x(arg) > > It's always worked well, but that little voice in my head tells me > there's > > probably something I'm not thinking of that will come back to bite me in > the > > future. I don't think this helps the OP in any meaningful way, but I was > > wondering if you bright people might shed some light on the drawbacks of > > this approach for me. What are the better alternatives? > > No, this approach isn't error-prone. > Glad to hear it. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.brandl at gmx.net Fri Jul 15 20:26:42 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 15 Jul 2011 20:26:42 +0200 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: Message-ID: Am 15.07.2011 19:13, schrieb Devin Jeanpierre: > On Fri, Jul 15, 2011 at 12:38 PM, Guido van Rossum wrote: >> This smells a bit like uncalled-for religion. Remember that readline() >> returns an empty string at the end of the file instead of raising an >> exception, and IMO that makes it the better API. > > The last time I used file.readline was to pass to tokenize, which is > complicated immensely by the logic necessary to support both the > iterator protocol and the readline protocol. i.e. it has to check for > both StopIteration _and_ the empty string, which makes it harder to > use, forces it to contain more redundant information. Sorry, I don't see the immense complication in try: line = readline() except StopIteration: line = '' Of course, if the tokenize API was designed from scratch nowadays, it would probably only accept iterables, and you'd have to make one yourself from a file object using iter(fd.readline, '') cheers, Georg From solipsis at pitrou.net Fri Jul 15 21:28:02 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 15 Jul 2011 21:28:02 +0200 Subject: [Python-ideas] Deprecate str.find References: Message-ID: <20110715212802.2bd03354@pitrou.net> On Fri, 15 Jul 2011 20:26:42 +0200 Georg Brandl wrote: > > Sorry, I don't see the immense complication in > > try: > line = readline() > except StopIteration: > line = '' > > Of course, if the tokenize API was designed from scratch nowadays, > it would probably only accept iterables, and you'd have to make one > yourself from a file object using > > iter(fd.readline, '') Or, since file objects are iterable out of the box, you could simply pass the file object. Regards Antoine. From steve at pearwood.info Fri Jul 15 21:32:41 2011 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 16 Jul 2011 05:32:41 +1000 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: Message-ID: <4E2095D9.5040603@pearwood.info> Mike Graham wrote: > str.find (and bytes.find) is worse than the alternatives in every way. > It should be explicitly deprecated in favour of str.__contains__ and > str.index. I disagree. > str.find when used to check for substring is inferior to the in > operator. "if sub in s:" is shorter, easier-to-read, and more > efficient than "if s.find(sub) != -1:" and is not prone to the error > "if s.find(sub):" I have occasionally seen. Just because some people (allegedly) misuse str.find is not a reason to remove it. People misuse all sorts of things. I don't believe that it is valid to compare str.find to str.__contains__ since they do different things for different purposes. Using str.find instead of "in" is not misuse if you actually need an index. Better to do a single walk of the source string: p = s.find(sub) if p >= 0: # do something else: ... than wastefully do two: if sub in s: p = s.index(sub) # do something else: ... Whatever efficiency you might gain in the "substring not found" case, you lose in the "found case". You should only use "sub in s" when you don't care about *where* the substring is, only whether or not it is there. Strings are not dicts, and searching is not necessarily fast. If I'm searching the string twice, I'm doing it wrong. Since str.__contains__ is not a valid replacement for str.find, the only question is, should str.find be deprecated in favour of str.index? I say no. str.find is just too useful and neat, compared to catching an exception, to throw out. And it can be considerably faster. For long strings, the time taken for an unsuccessful search may be dominated by the time to traverse the string, and consequently the two alternatives are pretty close to the same speed: >>> from timeit import Timer >>> s = "abcdef"*1000000 >>> sub = "xyz" >>> setup = "from __main__ import s, sub" >>> t1 = Timer("s.find(sub) != -1", setup) >>> t2 = Timer("""try: ... s.index(sub) ... except ValueError: ... pass ... """, setup) >>> >>> t1.timeit(number=10000) 109.69042301177979 >>> t2.timeit(number=10000) 116.63023090362549 Catching the exception is only 6% slower than testing for -1. Not much difference, and we probably shouldn't care one way or the other. However, for short strings, the time taken may be dominated by the cost of catching the exception, and so str.find may be significantly faster: >>> s = "abc"*10 >>> sub = "x" >>> t1.timeit() 0.5977561473846436 >>> t2.timeit() 1.698801040649414 s.index here is nearly three times slower than s.find. (And of course, if the substring is present, index and find should be pretty much identical in speed.) > str.index is better for finding indices in that it supports an > idiomatic exception-based API rather than a return-code API. Being idiomatic is not better merely because it is idiomatic. Rather, what's better becomes idiomatic, rather than the other way around, because people re-use code examples that work well. I expect that in practice str.find is used rather more frequently than str.index, which suggests that at least when it comes to string searching, find is the idiomatic API. > Every > usage of str.find should look like "index = s.find(sub); if index == > -1: (exception code)", "Every" usage? I don't think so. Another common and valid usage is this pattern: index = s.find(sub) if index >= 0: # do something Written with exception handling it becomes significantly longer, trickier and less obvious for beginners: try: index = s.index(sub) except ValueError: pass else: # do something Note especially that this takes the least interesting case, the "do nothing if not found", and promotes it ahead of the interesting case "do something if found". Now that's an anti-pattern! (Albeit a mild one.) And of course the try...except example is subject to its own conceptual failures. Both of these are subtly, or not-so-subtly, wrong: try: index = s.index(sub) # do something except ValueError: pass try: index = s.index(sub) except ValueError: pass # do something > which is an antipattern in Python. Why do you think it is an anti-pattern? I don't consider it an anti-pattern. I often wish that lists also had a find method that returned a sentinel instead of raising an exception. (Although I'd probably use None, as the re module does, rather than -1.) > This problem > is compounded by the fact that the returned value is actually a valid > value; consider s = 'bar'--s[s.find('x')] is somewhat surprisingly > 'r'. Yes, that's a good argument against the use of -1 for "not found". None would have been better. > Additionally, the existence of str.find violates the > there's-one-way-to-do-it principle. You may be confusing Python with some other language, because there is no such principle in Python. Perhaps you are mistaking it for the Zen, There should be one-- and preferably only one --obvious way to do it. which is a statement requiring the existence of an obvious way, not a prohibition against there being multiple non-obvious ways. In any case, it's far from clear to me that str.index is that obvious way. But then again, I'm not Dutch *wink* -- Steven From g.brandl at gmx.net Fri Jul 15 21:49:34 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 15 Jul 2011 21:49:34 +0200 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <20110715212802.2bd03354@pitrou.net> References: <20110715212802.2bd03354@pitrou.net> Message-ID: Am 15.07.2011 21:28, schrieb Antoine Pitrou: > On Fri, 15 Jul 2011 20:26:42 +0200 > Georg Brandl wrote: >> >> Sorry, I don't see the immense complication in >> >> try: >> line = readline() >> except StopIteration: >> line = '' >> >> Of course, if the tokenize API was designed from scratch nowadays, >> it would probably only accept iterables, and you'd have to make one >> yourself from a file object using >> >> iter(fd.readline, '') > > Or, since file objects are iterable out of the box, you could simply > pass the file object. Yeah, that was a bad example :) What I wanted to show is that it's a one-liner to make an iterator out of a readline-type function, even if you have only that. Georg From cs at zip.com.au Sat Jul 16 00:50:50 2011 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 16 Jul 2011 08:50:50 +1000 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: Message-ID: <20110715225050.GA20485@cskk.homeip.net> On 15Jul2011 12:12, Mike Graham wrote: | On Fri, Jul 15, 2011 at 10:57 AM, Guido van Rossum wrote: | > However, in many cases absence of the string is not an error -- you | > just need to do something else. [...] | | It isn't necessarily an error if the substring is not in the string | (though it sometimes is), but it is an exceptional case. No it isn't, IMO. It's simply the _other_ case. | Python uses | exceptions pretty liberally most places -- it isn't necessarily an | error if an iterator is exhausted or if float("4.2 bad user input") is | called or if BdbQuit was raised. In these cases, an exception can be | perfectly expected to indicate that what happened is different from | the information used in a return value. In all the cases you cite the exception indicates failure of the operation: .next() has nothing to "next" to, float is being handed garbage etc. str.find does not have a failure mode, it has string found and string not found. | Making a Python user write a try/except block when she wants to handle | both the cases "substring is in s" and "substring isn't in s" seems | perfectly fine to me and, really, preferable to the if statement | required to handle these two cases. You don't find try/except wordy and opaque? I find "if" more idiomatic most of the time. Not to mention vague: it can often be quite hard to be sure the raised exception came from just the operation you imagine it came from. With str.find there's little scope for vagueness I agree (unless you aren't really using a str, but a duck-type). But plenty of: try: x = foofunc(y) except IndexError, e: ... is subject to uncaught IndexError arbitrarily deep in foofunc's call stack. | The base two cases really are about the same: [... try ... excpt ...] | vs. | | i = s.find(sub) | if i == -1: | do_something() | | But what if I forgot to handle the special case? [...] | In this second case, I get the value of -1. Later I can use it as an | index, use it in a slice, or perform arithmetic on it. This can | introduce seemingly-unrelated values later on, making this especially | hard to track down. I agree it may be a pity that str.find doesn't return None on string not found, which would generally raise an exception on an attempt to use it as a number. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Any company large enough to have a research lab is large enough not to listen to it. - Alan Kay From mikegraham at gmail.com Sat Jul 16 05:43:27 2011 From: mikegraham at gmail.com (Mike Graham) Date: Fri, 15 Jul 2011 23:43:27 -0400 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <20110715225050.GA20485@cskk.homeip.net> References: <20110715225050.GA20485@cskk.homeip.net> Message-ID: On Fri, Jul 15, 2011 at 6:50 PM, Cameron Simpson wrote: > On 15Jul2011 12:12, Mike Graham wrote: > | It isn't necessarily an error if the substring is not in the string > | (though it sometimes is), but it is an exceptional case. > > No it isn't, IMO. It's simply the _other_ case. There are many cases that mean exactly the same thing (0 means the substring was found starting at 0, 1 means the substring was found starting at 1, 2 means the substring was found starting at 2,...), so the other case (substring not contained) we can label special or exceptional. We can recognize such cases by their requiring separate logic. > | Python uses > | exceptions pretty liberally most places -- it isn't necessarily an > | error if an iterator is exhausted or if float("4.2 bad user input") is > | called or if BdbQuit was raised. In these cases, an exception can be > | perfectly expected to indicate that what happened is different from > | the information used in a return value. > > In all the cases you cite the exception indicates failure of the > operation: .next() has nothing to "next" to, float is being handed > garbage etc. > > str.find does not have a failure mode, it has string found and string > not found. I think you are way off here. "Error" is in the eye of the beholder. I could say that an iterator has two cases: one where it gives me a value, and the other, that it's exhausted. Nothing exceptional there. (Indeed, the choice of StopIteration not to have "Error" in the name was made for the precise reason this isn't regarded as an error.) Similarly, when I pass user input to float, there are just two normal cases, no failure mode: user entered a number, user didn't enter a number. The distinction between an error and another type of special case is subtle at best. > | Making a Python user write a try/except block when she wants to handle > | both the cases "substring is in s" and "substring isn't in s" seems > | perfectly fine to me and, really, preferable to the if statement > | required to handle these two cases. > > You don't find try/except wordy and opaque? I find "if" more idiomatic most of the time. Not really. If we don't like exceptionhandling, we're using the wrong language. > Not to mention vague: it can often be quite hard to be sure the raised > exception came from just the operation you imagine it came from. With > str.find there's little scope for vagueness I agree (unless you aren't > really using a str, but a duck-type). But plenty of: > > ?try: > ? ?x = foofunc(y) > ?except IndexError, e: > ? ?... > > is subject to uncaught IndexError arbitrarily deep in foofunc's call stack. This is the strength and the flaw with exceptions period. It is a much broader question than the one we are facing here. If you do not like exceptions period or Python's use of relatively few exception types for many occasions, I really don't think we can start the discussion at the level of str.find. If I did manage to have an IndexError propagate through to my SomeDuckType.index method when it shouldn't have the meaning I ascribe it, then this is a bug in my implementation of SomeDuckType. This bug would be very unfortunate because when a user tries to use my code right--catching the IndexError--they will completely squash the offending exception and the source of the bug will be unclear. Unfortunately, str.find is highly prone to such bugs as I've discussed since -1 is a valid index for the string. > | The base two cases really are about the same: > [... try ... excpt ...] > | vs. > | > | i = s.find(sub) > | if i == -1: > | ? ? do_something() > | > | But what if I forgot to handle the special case? [...] > | In this second case, I get the value of -1. Later I can use it as an > | index, use it in a slice, or perform arithmetic on it. This can > | introduce seemingly-unrelated values later on, making this especially > | hard to track down. > > I agree it may be a pity that str.find doesn't return None on string not > found, which would generally raise an exception on an attempt to use it > as a number. > > Cheers, > -- > Cameron Simpson Mike From cmjohnson.mailinglist at gmail.com Sat Jul 16 06:50:24 2011 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Fri, 15 Jul 2011 18:50:24 -1000 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: <20110715225050.GA20485@cskk.homeip.net> Message-ID: This is an absolute flight of fancy, and I'm sure it's already been rejected in the past (a quick search says Aug. 2009, http://mail.python.org/pipermail/python-ideas/2009-August/thread.html#5576 ), but what about some kind of try/except expression? i = s.index(substr) except ValueError is None I guess my main problem with that is the color of the bikeshed: it's hard to get a good idiomatic way of spelling the except expression. You could use a colon, as proposed in that thread, but it seems to me a colon indicates a new line follows. I would be for making this as simple as possible. No "as", no non-implicit "else", and no nesting of exception types. If you want something fancy, use a statement. From ncoghlan at gmail.com Sat Jul 16 10:26:29 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 16 Jul 2011 18:26:29 +1000 Subject: [Python-ideas] None-as-sentinel alternatives? (was: deferred default arguments) In-Reply-To: References: Message-ID: On Sat, Jul 16, 2011 at 12:05 AM, Don Spaulding wrote: > When I come across a situation where I want a sentinel that doesn't preclude > None from being a valid value, I've always spelled it like this: > ? ? SENTINEL = object() ?# Guaranteed to be unique and not None. > > ? ? def f(arg=SENTINEL): > ? ? ? ? if arg is SENTINEL: > ? ? ? ? ? ? arg = 'foo' > ? ? ? ? x(arg) > It's always worked well, but that little voice in my head tells me there's > probably something I'm not thinking of that will come back to bite me in the > future. ?I don't think this helps the OP in any meaningful way, but I was > wondering if you bright people might shed some light on the drawbacks of > this approach for me. ?What are the better alternatives? Explicit sentinel objects *are* the better alternative :) As they rely on object identity, they typically don't play nicely with serialisation, though. You need a value based sentinel for that use case. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From g.brandl at gmx.net Sat Jul 16 10:47:26 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 16 Jul 2011 10:47:26 +0200 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: <20110715225050.GA20485@cskk.homeip.net> Message-ID: Am 16.07.2011 05:43, schrieb Mike Graham: >> Not to mention vague: it can often be quite hard to be sure the raised >> exception came from just the operation you imagine it came from. With >> str.find there's little scope for vagueness I agree (unless you aren't >> really using a str, but a duck-type). But plenty of: >> >> try: >> x = foofunc(y) >> except IndexError, e: >> ... >> >> is subject to uncaught IndexError arbitrarily deep in foofunc's call stack. > > This is the strength and the flaw with exceptions period. It is a much > broader question than the one we are facing here. If you do not like > exceptions period or Python's use of relatively few exception types > for many occasions, I really don't think we can start the discussion > at the level of str.find. > > If I did manage to have an IndexError propagate through to my > SomeDuckType.index method when it shouldn't have the meaning I ascribe > it, then this is a bug in my implementation of SomeDuckType. This bug > would be very unfortunate because when a user tries to use my code > right--catching the IndexError--they will completely squash the > offending exception and the source of the bug will be unclear. > Unfortunately, str.find is highly prone to such bugs as I've discussed > since -1 is a valid index for the string. By the way: str.index() raises ValueError, not IndexError, if the substring isn't found. That's one argument against "the index() API is foolproof, while the find() API isn't". Georg From ncoghlan at gmail.com Sat Jul 16 10:50:12 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 16 Jul 2011 18:50:12 +1000 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: Message-ID: On Fri, Jul 15, 2011 at 11:57 PM, Mike Graham wrote: > str.find (and bytes.find) is worse than the alternatives in every way. > It should be explicitly deprecated in favour of str.__contains__ and > str.index. As others have noted, the typical usage pattern of: idx = s.find(sub) if idx >= 0: # do something that needs idx is nice and clean and significantly faster than searching the string twice. Universally discouraging find() in favour of index() is questionable, as the above is significantly cleaner than the index based alternative. So, I have a different suggestion (that actually paves the way for the eventual deprecation of find()): Update str.index() to accept an optional sentinel value. If the sentinel argument is not supplied, then a missing substring raises ValueError as it does now. If it is supplied, then a missing substring returns the sentinel value instead of throwing an exception. The above idiom could then be expressed cleanly as: idx = s.index(sub, missing=None) if idx is not None: # do something that needs idx However, this seemingly simple suggestion is complicated by the fact that string methods do not currently accept keyword arguments and index() already accepts two optional positional arguments (for substring searching). Perhaps the more general solution of try/except/value expressions is worth reconsidering. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sat Jul 16 10:51:12 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 16 Jul 2011 18:51:12 +1000 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: <20110715225050.GA20485@cskk.homeip.net> Message-ID: On Sat, Jul 16, 2011 at 6:47 PM, Georg Brandl wrote: > By the way: str.index() raises ValueError, not IndexError, if the substring > isn't found. ?That's one argument against "the index() API is foolproof, > while the find() API isn't". Yeah, that bugs me too. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From mal at egenix.com Sat Jul 16 11:11:10 2011 From: mal at egenix.com (M.-A. Lemburg) Date: Sat, 16 Jul 2011 11:11:10 +0200 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: <20110715225050.GA20485@cskk.homeip.net> Message-ID: <4E2155AE.70305@egenix.com> Georg Brandl wrote: > By the way: str.index() raises ValueError, not IndexError, if the substring > isn't found. That's one argument against "the index() API is foolproof, > while the find() API isn't". That's because the *value* passed as argument to .index() isn't found. Which error class to use often depends on your view point, so in some cases it may seem natural to you, in others you have a different POV, and it feels wrong :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 16 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From ncoghlan at gmail.com Sat Jul 16 11:46:59 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 16 Jul 2011 19:46:59 +1000 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <4E2155AE.70305@egenix.com> References: <20110715225050.GA20485@cskk.homeip.net> <4E2155AE.70305@egenix.com> Message-ID: On Sat, Jul 16, 2011 at 7:11 PM, M.-A. Lemburg wrote: > Georg Brandl wrote: >> By the way: str.index() raises ValueError, not IndexError, if the substring >> isn't found. ?That's one argument against "the index() API is foolproof, >> while the find() API isn't". > > That's because the *value* passed as argument to .index() isn't > found. Which error class to use often depends on your view point, > so in some cases it may seem natural to you, in others you have > a different POV, and it feels wrong :-) Yeah, but trying to catch IndexError instead of ValueError is a pretty easy (and understandable) mistake to make, given the correspondence in names and logical train of thought "index() tells me the index where a substring can be found, IndexError is used to denote that a given index doesn't exist, so if the requested index doesn't exist, then this function will throw IndexError". Easily detected by testing, but still unintuitive and annoying. If IndexError had instead been called IndexNotFound, then there would never have been the slightest question as to which exception should have been thrown. Too late to change it now, though. We just all have to learn that, from the str.index point of view, failing to find a substring means there's something *wrong* with one (or both) of the passed in strings rather than the less opinionated "the index you have requested doesn't actually exist". Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ethan at stoneleaf.us Sat Jul 16 17:25:03 2011 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 16 Jul 2011 08:25:03 -0700 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: <20110715225050.GA20485@cskk.homeip.net> Message-ID: <4E21AD4F.7060904@stoneleaf.us> Mike Graham wrote: > On Fri, Jul 15, 2011 at 6:50 PM, Cameron Simpson wrote: >> You don't find try/except wordy and opaque? I find "if" more idiomatic most of the time. > > Not really. If we don't like exceptionhandling, we're using the wrong language. I think exception handling is awesome. I also think dictionaries are awesome, but I don't use them when what I need is a set. ~Ethan~ From raymond.hettinger at gmail.com Sat Jul 16 17:26:37 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Sat, 16 Jul 2011 08:26:37 -0700 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: Message-ID: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> On Jul 15, 2011, at 6:57 AM, Mike Graham wrote: > str.find (and bytes.find) is worse than the alternatives in every way. > It should be explicitly deprecated in favour of str.__contains__ and > str.index. Unless an API is flat out broken, deprecation is almost always a bad idea. This API has been around for a very long time, so deprecating it will break lots of people's code for almost zero benefit: http://www.google.com/codesearch#search/&q=%5C.find%5C(%20lang:%5Epython$&type=cs Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Sat Jul 16 17:35:10 2011 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 16 Jul 2011 08:35:10 -0700 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: Message-ID: <4E21AFAE.9030805@stoneleaf.us> Nick Coghlan wrote: > So, I have a different suggestion (that actually paves the way for the > eventual deprecation of find()): > > Update str.index() to accept an optional sentinel value. > > If the sentinel argument is not supplied, then a missing substring > raises ValueError as it does now. If it is supplied, then a missing > substring returns the sentinel value instead of throwing an exception. > The above idiom could then be expressed cleanly as: > > idx = s.index(sub, missing=None) > if idx is not None: > # do something that needs idx +1 Excellent idea. > However, this seemingly simple suggestion is complicated by the fact > that string methods do not currently accept keyword arguments and > index() already accepts two optional positional arguments (for > substring searching). Does this require a PEP or just someone to start working on it? Getting str() to accept keyword arguments and adding missing to str.index() sounds like a nice project. ~Ethan~ From ethan at stoneleaf.us Sat Jul 16 17:46:04 2011 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 16 Jul 2011 08:46:04 -0700 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> Message-ID: <4E21B23C.5080302@stoneleaf.us> Raymond Hettinger wrote: > > On Jul 15, 2011, at 6:57 AM, Mike Graham wrote: > >> str.find (and bytes.find) is worse than the alternatives in every way. >> It should be explicitly deprecated in favour of str.__contains__ and >> str.index. > > Unless an API is flat out broken, deprecation is almost always a bad idea. > This API has been around for a very long time, so deprecating it will > break lots of people's code for almost zero benefit: > > http://www.google.com/codesearch#search/&q=%5C.find%5C(%20lang:%5Epython$&type=cs > How ironic that the fist hit seems to display the problem Mike is concerned with: position = min(position, len(self.contents)) if hasattr(newChild, 'parent') and newChild.parent != None: # We're 'inserting' an element that's already one # of this object's children. if newChild.parent == self: index = self.find(newChild) if index and index < position: # Furthermore we're moving it further down the # list of this object's children. That means that # when we extract this element, our target index # will jump down one. position = position - 1 I haven't read all the surrounding code to know if this will ever fail, but the whole 'index = ... .find(...); if index and ...' certainly doesn't lend confidence. After all, if you *know* newChild is in self, why not use .index()? ~Ethan~ From ncoghlan at gmail.com Sat Jul 16 18:04:26 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 17 Jul 2011 02:04:26 +1000 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <4E21AFAE.9030805@stoneleaf.us> References: <4E21AFAE.9030805@stoneleaf.us> Message-ID: On Sun, Jul 17, 2011 at 1:35 AM, Ethan Furman wrote: > Nick Coghlan wrote: >> However, this seemingly simple suggestion is complicated by the fact >> that string methods do not currently accept keyword arguments and >> index() already accepts two optional positional arguments (for >> substring searching). > > Does this require a PEP or just someone to start working on it? ?Getting > str() to accept keyword arguments and adding missing to str.index() sounds > like a nice project. Shouldn't need a full PEP, but will likely need at least some discussion on python-dev and some pre- and post-patch microbenchmarks to assess the impact on the speed of string operations (since passing and parsing keyword arguments for C functions *is* slower than only using positional arguments). Compared to making strings Unicode by default, though, it's a pretty minor change :) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From mikegraham at gmail.com Sat Jul 16 19:39:34 2011 From: mikegraham at gmail.com (Mike Graham) Date: Sat, 16 Jul 2011 13:39:34 -0400 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> Message-ID: On Sat, Jul 16, 2011 at 11:26 AM, Raymond Hettinger wrote: > Unless an API is flat out broken, deprecation is almost always a bad idea. > This API has been around for a very long time, so deprecating it will > break lots of people's code for almost zero benefit: > http://www.google.com/codesearch#search/&q=%5C.find%5C(%20lang:%5Epython$&type=cs > > Raymond I agree that breaking people's code is a bad thing and have not suggested removing str.find. This removal would require a transition like that from Python 2.x to Python 3.x, a move that is not planned and I personally do not ever expect. I appreciate your linking this search, which does indeed does show that str.find is in wide use. However, looking at the first give pages of results, this use seems largely unfortunate?literally the majority of the times str.find is used, I would have used "substring in s" or "s.startswith(substring)". I also see code like "pos = s.find(" ("); if pos + len(" (...)") > self._maxWidth:" which makes me very uncomfortable and which I would have to read in detail to figure out what's happening confidently if I was the maintaining the code. Thanks for the reply, Mike From mikegraham at gmail.com Sat Jul 16 19:43:37 2011 From: mikegraham at gmail.com (Mike Graham) Date: Sat, 16 Jul 2011 13:43:37 -0400 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <4E21B23C.5080302@stoneleaf.us> References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4E21B23C.5080302@stoneleaf.us> Message-ID: On Sat, Jul 16, 2011 at 11:46 AM, Ethan Furman wrote: > Raymond Hettinger wrote: >> Unless an API is flat out broken, deprecation is almost always a bad idea. >> This API has been around for a very long time, so deprecating it will >> break lots of people's code for almost zero benefit: >> >> >> http://www.google.com/codesearch#search/&q=%5C.find%5C(%20lang:%5Epython$&type=cs >> > > > How ironic that the fist hit seems to display the problem Mike is concerned > with: > > ? ? ? ?position = ?min(position, len(self.contents)) > ? ? ? ?if hasattr(newChild, 'parent') and newChild.parent != None: > ? ? ? ? ? ?# We're 'inserting' an element that's already one > ? ? ? ? ? ?# of this object's children. > ? ? ? ? ? ?if newChild.parent == self: > ? ? ? ? ? ? ? ?index = self.find(newChild) > ? ? ? ? ? ? ? ?if index and index < position: > ? ? ? ? ? ? ? ? ? ?# Furthermore we're moving it further down the > ? ? ? ? ? ? ? ? ? ?# list of this object's children. That means that > ? ? ? ? ? ? ? ? ? ?# when we extract this element, our target index > ? ? ? ? ? ? ? ? ? ?# will jump down one. > ? ? ? ? ? ? ? ? ? ?position = position - 1 > > I haven't read all the surrounding code to know if this will ever fail, but > the whole 'index = ... .find(...); if index and ...' certainly doesn't lend > confidence. ?After all, if you *know* newChild is in self, why not use > .index()? > > ~Ethan~ Note that this particular code is using BeautifulSoup.PageElement.find, not str.find. There are, however, in the first few pages similar snippets which use the result of str.find without a check for the special case. Mike From mikegraham at gmail.com Sat Jul 16 19:45:00 2011 From: mikegraham at gmail.com (Mike Graham) Date: Sat, 16 Jul 2011 13:45:00 -0400 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: <20110715225050.GA20485@cskk.homeip.net> Message-ID: On Sat, Jul 16, 2011 at 4:47 AM, Georg Brandl wrote: > Am 16.07.2011 05:43, schrieb Mike Graham: > By the way: str.index() raises ValueError, not IndexError, if the substring > isn't found. ?That's one argument against "the index() API is foolproof, > while the find() API isn't". > > Georg Embarrassing, though I still think that the str.find API is more error-prone for the reasons I've given. Thanks for pointing this out! Mike From Ronny.Pfannschmidt at gmx.de Sat Jul 16 20:43:21 2011 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Sat, 16 Jul 2011 20:43:21 +0200 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> Message-ID: <1310841801.3692.287.camel@Klappe2> On Sat, 2011-07-16 at 08:26 -0700, Raymond Hettinger wrote: > > On Jul 15, 2011, at 6:57 AM, Mike Graham wrote: > > > str.find (and bytes.find) is worse than the alternatives in every > > way. > > It should be explicitly deprecated in favour of str.__contains__ and > > str.index. > > > > > Unless an API is flat out broken, deprecation is almost always a bad > idea. > This API has been around for a very long time, so deprecating it will > break lots of people's code for almost zero benefit: > > > http://www.google.com/codesearch#search/&q=%5C.find%5C(%20lang:% > 5Epython$&type=cs > Although a quick lock over the found snippets tells, that apparently many uses of find are indeed "ugly" code that could be improved by use of index or __contains__ Since a DeprecationWarning seems rather intrusive, what would be a less intrusive way to cause code-enhancement in such places? -- Ronny > > > > Raymond > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: From tjreedy at udel.edu Sat Jul 16 20:46:57 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 16 Jul 2011 14:46:57 -0400 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <20110715225050.GA20485@cskk.homeip.net> References: <20110715225050.GA20485@cskk.homeip.net> Message-ID: On 7/15/2011 6:50 PM, Cameron Simpson wrote: > On 15Jul2011 12:12, Mike Graham wrote: >age etc. > > str.find does not have a failure mode, it has string found and string > not found. Not finding == failure to find == failure to match an implied re. The question is, how should failure to find be indicated? Out of band with a exception? In band with a special value? (I am here ignoring the design bug that the 'special value' returned by str.find is not really special in Python but is a legal and common index and hence bug-bait.) Python usually chooses one method or the other. The index/find pair is very exceptional in providing both. Why? What is so important or exceptional about this particular function? 'Find the index of a substring in a string' is not the most common operation, at least not for most people. To me, the special-pleading arguments given for find would apply to a hundred other functions. For instance, should we add an alternate constructor for int, say int.nix, that would return None instead of ValueError for 'string not consisting of 1 or more base x digits'. [ 'Nix' here means 'nix, you cannot do that, you get nothing in return'.] As with index/find, the input string either does or does not match a particular re. If we have two ways to indicate 'not match' for one function, why not all others? Why not? Because we do not *need* the near duplication because mis-formed inputs to int are handled with try--except because that is the way it is done. The duplication of index/find is a historical aberration with no particular justification other than history. Leave it at that. If str.find did not exist, a proposal to add it would be less welcome than int.nix. Indeed, choice of failure indicator is good, so lets do it right and have it everywhere. Define nix generically as def nix(self, *args, **kwds): try: return self(*args, **kwds) except: return None Make this a builtin and add it as a class or instance method, as appropriate, to appropriate built-in classes. (I am thinking, for instance, that class method for int and instance method for type(lambda:0) should work. See test below.) By exposing it, users could use it too eiher directly or wrapped with classmethod(). Or call the above _nix and define decorators. Then str.find would eventually be deprecated in favor of str.index.nix. A preliminary test: class C(): def __init__(self, f): self.f = f def __call__(self, *args): return self.f(*args) def nix(self, *args, **kwds): try: return self(*args, **kwds) except: return None myint = C(int) print(myint('1'), myint.nix('1'), myint.nix(''), myint.nix('a')) try: myint('') except: print('E caught') >>> 1 1 None None E caught -- Terry Jan Reedy From raymond.hettinger at gmail.com Sat Jul 16 21:07:19 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Sat, 16 Jul 2011 12:07:19 -0700 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> Message-ID: <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> On Jul 16, 2011, at 10:39 AM, Mike Graham wrote: > I agree that breaking people's code is a bad thing and have not > suggested removing str.find. Deprecation is a step towards removal and it always causes a certain amount of pain. ISTM, the time for any move like this would have been the jump from Python 2 to Python 3 where significant breakage was expected and where transition tools were developed. > > I appreciate your linking this search, which does indeed does show > that str.find is in wide use. However, looking at the first give pages > of results, this use seems largely unfortunate?literally the majority > of the times str.find is used, I would have used "substring in s" or > "s.startswith(substring)". I also see code like "pos = s.find(" ("); > if pos + len(" (...)") > self._maxWidth:" which makes me very > uncomfortable and which I would have to read in detail to figure out > what's happening confidently if I was the maintaining the code. I am largely unsympathetic to arguments that are roughly equivalent to "I don't like the way other people write programs". Something akin to the str.find() API has been present in many, many languages for a very long time. For the most part, people seem to be able to use it reasonably well. I find that beginning Python students never seem to have a problem with it. Also keep it mind that startswith() and endswith() were relatively recent additions to Python, so it is no surprise that lots of code uses find() instead of startswith(). Raymond From mikegraham at gmail.com Sat Jul 16 21:26:56 2011 From: mikegraham at gmail.com (Mike Graham) Date: Sat, 16 Jul 2011 15:26:56 -0400 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> Message-ID: On Sat, Jul 16, 2011 at 3:07 PM, Raymond Hettinger wrote: > Deprecation is a step towards removal and it always causes a certain amount of pain. > > ISTM, the time for any move like this would have been the jump from Python 2 to Python 3 where significant breakage was expected and where transition tools were developed. Because this would break backwards compatibility unnecessarily, I think anyone involved can agree that actual removal could not take place until a special jump akin to the Python 2->Python 3 jump. (It would surprise me if such a jump ever actually took place.) >> I appreciate your linking this search, which does indeed does show >> that str.find is in wide use. However, looking at the first give pages >> of results, this use seems largely unfortunate?literally the majority >> of the times str.find is used, I would have used "substring in s" or >> "s.startswith(substring)". I also see code like "pos = s.find(" ?("); >> if pos + len(" (...)") > self._maxWidth:" which makes me very >> uncomfortable and which I would have to read in detail to figure out >> what's happening confidently if I was the maintaining the code. > > I am largely unsympathetic to arguments that are > roughly equivalent to "I don't like the way other people > write programs". Reviewing five pages of results, over 2/3 of the uses of str.find could be replaced by str.__contains__. Using sub in s instead of s.find(sub) != -1 is *already* the advice in the official Python documentation, so I do not believe I am making especially personal judgments about the style of the code. > Something akin to the str.find() API has been present in many, > many languages for a very long time. ?For the most part, people > seem to be able to use it reasonably well. ?I find that beginning > Python students never seem to have a problem with it. I'm glad your experience with learners here has been more consistent than mine. > Also keep it mind that startswith() and endswith() were relatively > recent additions to Python, so it is no surprise that lots of code > uses find() instead of startswith(). > > > Raymond Mike From tjreedy at udel.edu Sat Jul 16 21:26:46 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 16 Jul 2011 15:26:46 -0400 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <1310841801.3692.287.camel@Klappe2> References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <1310841801.3692.287.camel@Klappe2> Message-ID: On 7/16/2011 2:43 PM, Ronny Pfannschmidt wrote: > > Since a DeprecationWarning seems rather intrusive, > what would be a less intrusive way to cause code-enhancement in such > places? A PendingDeprecationWarning, which I believe would be off by default, and which could be left as such for several versions. -- Terry Jan Reedy From sturla at molden.no Sat Jul 16 21:16:25 2011 From: sturla at molden.no (Sturla Molden) Date: Sat, 16 Jul 2011 21:16:25 +0200 Subject: [Python-ideas] Left division operator Message-ID: <4E21E389.8090406@molden.no> I was wondering if anyone else has felt the need for a leftward division operator in Python. That is, y\x in addition to x/y. Thus the operators: y \ x y \\ x Why the statement "y\x" is different from "x/y": 1. Opposite order of evaluation: This is important if evaluating one of the operands has side-effects. 2. In linear algebra, (Y**-1)*X is in general not equal to X*(Y**-1). Matrix and vector algebra in fundamental to computer graphics and scientific computing. 3. NumPy solves the lack of "\" operator by having a function np.linalg.solve. But using it in matrix expressions has the effect of mixing Pythonic infix operators with a Lisp-like prefix operator. Two other issues: 1. The expressions y /= x y //= x are evaluated in the same order as "y \ x", not "x / y". 2. The expressions y \= x y \\= x should perhaps be illegal due to implied side-effects on RHS. Sturla From masklinn at masklinn.net Sat Jul 16 21:43:10 2011 From: masklinn at masklinn.net (Masklinn) Date: Sat, 16 Jul 2011 21:43:10 +0200 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <1310841801.3692.287.camel@Klappe2> Message-ID: <9F53FB53-CBDB-48E3-914C-FE44DFB7381E@masklinn.net> On 2011-07-16, at 21:26 , Terry Reedy wrote: > On 7/16/2011 2:43 PM, Ronny Pfannschmidt wrote: >> >> Since a DeprecationWarning seems rather intrusive, >> what would be a less intrusive way to cause code-enhancement in such >> places? > > A PendingDeprecationWarning, which I believe would be off by default, and which could be left as such for several versions. DeprecationWarning is already silent (since Python 3.2). PDW is of little use nowadays, and DeprecationWarning can already be left on for several versions. From tjreedy at udel.edu Sat Jul 16 21:52:39 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 16 Jul 2011 15:52:39 -0400 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: <20110715225050.GA20485@cskk.homeip.net> Message-ID: On 7/16/2011 2:46 PM, Terry Reedy wrote: > On 7/15/2011 6:50 PM, Cameron Simpson wrote: >> On 15Jul2011 12:12, Mike >> Graham wrote: >> age etc. >> >> str.find does not have a failure mode, it has string found and string >> not found. > > Not finding == failure to find == failure to match an implied re. Indeed, negative values such as -1 are standard error/failure return codes for functions that normally return nonnegative ints and that written in statically typed languages without catchable exceptions. In C, for instance, EOF is defined as an implementation-defined negative int and I am sure -1 is used by some. -- Terry Jan Reedy From raymond.hettinger at gmail.com Sat Jul 16 22:37:50 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Sat, 16 Jul 2011 13:37:50 -0700 Subject: [Python-ideas] [offlist] Re: Deprecate str.find In-Reply-To: References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> Message-ID: On Jul 16, 2011, at 12:26 PM, Mike Graham wrote: > Reviewing five pages of results, over 2/3 of the uses of str.find > could be replaced by str.__contains__. I think the first hit was from BeautifulSoup which predates the introduction of __contains__ and still runs on both old and new versions of Python. It may be hard to believe, but in the old days (not really so long ago), we didn't have __contains__ or startswith() and yet Python became popular anyway ;-) > Using sub in s instead of > s.find(sub) != -1 is *already* the advice in the official Python > documentation, I know about that advice. I believe I'm the one who added it ;-) (as well as many other tips in the code modernization PEP). Raymond From masklinn at masklinn.net Sat Jul 16 22:21:51 2011 From: masklinn at masklinn.net (Masklinn) Date: Sat, 16 Jul 2011 22:21:51 +0200 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: <20110715225050.GA20485@cskk.homeip.net> Message-ID: <75B5F6EE-03E4-4DFD-9B97-C02E74C687EC@masklinn.net> On 2011-07-16, at 21:52 , Terry Reedy wrote: > On 7/16/2011 2:46 PM, Terry Reedy wrote: >> On 7/15/2011 6:50 PM, Cameron Simpson wrote: >>> On 15Jul2011 12:12, Mike >>> Graham wrote: >>> age etc. >>> >>> str.find does not have a failure mode, it has string found and string >>> not found. >> >> Not finding == failure to find == failure to match an implied re. > Indeed, negative values such as -1 are standard error/failure return codes for functions that normally return nonnegative ints and that written in statically typed languages without catchable exceptions ? or type systems worth using. And that's for those which are 0-indexed of course, especially for finding sub-sequences. And interestingly, the function corresponding to `str.find` in libc returns `NULL` in case of failure, not ?1 (its return value is a pointer to the first occurrence of the needle in the haysack). From raymond.hettinger at gmail.com Sat Jul 16 22:52:29 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Sat, 16 Jul 2011 13:52:29 -0700 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> Message-ID: <2EB37336-9E13-4294-BE10-63574DF45793@gmail.com> I just remembered one other thought on the subject. Usually, when Python introduces a method such as .index() that raises an exception for the not-found case, there are immediate requests for variants that don't raise exceptions: dict.pop(key, default) dict.get(key, defalut) next(iterable, default) getattr(obj, attr, default) re.match() --> None or matchobject People seem to hate wrapping try/except around simple calls. Expect those people to be agitated if you take away str.find(). Raymond From mikegraham at gmail.com Sat Jul 16 23:04:58 2011 From: mikegraham at gmail.com (Mike Graham) Date: Sat, 16 Jul 2011 17:04:58 -0400 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <2EB37336-9E13-4294-BE10-63574DF45793@gmail.com> References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> <2EB37336-9E13-4294-BE10-63574DF45793@gmail.com> Message-ID: On Sat, Jul 16, 2011 at 4:37 PM, Raymond Hettinger wrote: > On Jul 16, 2011, at 12:26 PM, Mike Graham wrote: >> Using sub in s instead of >> s.find(sub) != -1 is *already* the advice in the official Python >> documentation, > > I know about that advice. I believe I'm the one who added it ;-) > (as well as many other tips in the code modernization PEP). Then it seems you're guiltier than I of saying what way other people should write their programs. =) On Sat, Jul 16, 2011 at 4:52 PM, Raymond Hettinger wrote: > I just remembered one other thought on the subject. > > Usually, when Python introduces a method such as .index() > that raises an exception for the not-found case, there > are immediate requests for variants that don't raise exceptions: > > ?dict.pop(key, default) > ?dict.get(key, defalut) > ?next(iterable, default) > ?getattr(obj, attr, default) > ?re.match() --> None or matchobject > > People seem to hate wrapping try/except around simple calls. > Expect those people to be agitated if you take away str.find(). > > > Raymond Did you catch Nick's suggestion to enhance str.index to have the same basic API as dict.get/getattr/etc.? I think this might be useful overall by providing a way to get a useful default, which is very often None (as a value in a slice). Mike From raymond.hettinger at gmail.com Sun Jul 17 00:05:23 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Sat, 16 Jul 2011 15:05:23 -0700 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> <2EB37336-9E13-4294-BE10-63574DF45793@gmail.com> Message-ID: <7FCB2B33-9AFC-4D14-B7CA-2062465A304A@gmail.com> On Jul 16, 2011, at 2:04 PM, Mike Graham wrote: > > Did you catch Nick's suggestion to enhance str.index to have the same > basic API as dict.get/getattr/etc.? Sorry, but I thought idea that was way off base. Things like dict.get and getattr are about returning values, so it is possible to provide a meaningful default. In the case of string.find, there is no meaningful default position in the string. So, a default would simply be a way to turn the -1 value into some other object which you would still need to test. We don't need to make the API worse by expanding it just for the sake of change. The str.find() method isn't broken or useless. If you want people to change the way they write code, it would be better to do it through education (blog posts, pylint, etc) rather than by breaking a venerable API. Raymond From jeanpierreda at gmail.com Sun Jul 17 00:15:51 2011 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Sat, 16 Jul 2011 18:15:51 -0400 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <7FCB2B33-9AFC-4D14-B7CA-2062465A304A@gmail.com> References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> <2EB37336-9E13-4294-BE10-63574DF45793@gmail.com> <7FCB2B33-9AFC-4D14-B7CA-2062465A304A@gmail.com> Message-ID: On Sat, Jul 16, 2011 at 6:05 PM, Raymond Hettinger wrote: > > On Jul 16, 2011, at 2:04 PM, Mike Graham wrote: >> >> Did you catch Nick's suggestion to enhance str.index to have the same >> basic API as dict.get/getattr/etc.? > > Sorry, but I thought idea that was way off base. > Things like dict.get and getattr are about returning values, > so it is possible to provide a meaningful default. > In the case of string.find, there is no meaningful > default position in the string. ?So, a default would simply > be a way to turn the -1 value into some other object > which you would still need to test. Slicing was brought up as a use-case where you can use a default without checking. mystring[:mystring.index('#', None)], for example, could conceivably be used somewhere to strip comments from (some sort of) code, for example. It does have other benefits too. It makes the return value on failure explicit, which would help remind people to check, or be more immediately aware when reading code. And it does have nice parallels to those other methods. Devin From raymond.hettinger at gmail.com Sun Jul 17 00:52:27 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Sat, 16 Jul 2011 15:52:27 -0700 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> <2EB37336-9E13-4294-BE10-63574DF45793@gmail.com> <7FCB2B33-9AFC-4D14-B7CA-2062465A304A@gmail.com> Message-ID: <733331DF-3BB2-4879-A984-707CA2D0D094@gmail.com> On Jul 16, 2011, at 3:15 PM, Devin Jeanpierre wrote: > Slicing was brought up as a use-case where you can use a default > without checking. This thread has lost contact with reality. It started with the reasonable observation that many uses of str.find could be replaced with a simple test using the in-operator. Now, the thread is venturing into the typical python-ideas world of making-up random use cases and APIs. Something like str.find() is in many languages and it is definitely not the norm for them to have found a need to both be able to return -1 or to supply a default value. As a Python teacher, speaker, and consultant, I have the opportunity to see and review the code repositories for many companies. I'm pretty sure that I've never seen a utils module with the likes of: def myfind(fullstring, substring, default=None): i = fullstring.find(substring) if i == -1 and default is not None: return default return i When I start seeing people routinely using a helper function like this, I'll start to believe that a str.find default value isn't silly. Mike's initial post was well grounded in observations about code that could be improved by using "in" or str.index() instead of str.find(). Though I disagreed with the recommendation to deprecate, it would be even worse to exacerbate the usability issues by making the method signature even more complex (with a new optional argument and a new signature variant for the return value). That isn't progress. It's aspiring cruft that makes the language harder to learn and remember. Raymond From cs at zip.com.au Sun Jul 17 01:08:57 2011 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 17 Jul 2011 09:08:57 +1000 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <75B5F6EE-03E4-4DFD-9B97-C02E74C687EC@masklinn.net> References: <75B5F6EE-03E4-4DFD-9B97-C02E74C687EC@masklinn.net> Message-ID: <20110716230857.GA2174@cskk.homeip.net> On 16Jul2011 22:21, Masklinn wrote: | On 2011-07-16, at 21:52 , Terry Reedy wrote: | > On 7/16/2011 2:46 PM, Terry Reedy wrote: | >> On 7/15/2011 6:50 PM, Cameron Simpson wrote: | >>> str.find does not have a failure mode, it has string found and string | >>> not found. | >> | >> Not finding == failure to find == failure to match an implied re. | > Indeed, negative values such as -1 are standard error/failure | > return codes for functions that normally return nonnegative ints and | > that written in statically typed languages without catchable exceptions | | ? or type systems worth using. And that's for those which are | 0-indexed of course, especially for finding sub-sequences. | | And interestingly, the function corresponding to `str.find` in libc | returns `NULL` in case of failure, not ?1 (its return value is a | pointer to the first occurrence of the needle in the haysack). Though if you're thinking NULL is equivalent to None (which is often is conceptually), let's remember that in C NULL is just a pointer value; it is a sentinal, but not a different type. So NULL here is in some ways akin to -1 in Python's find return. You still need to test for it; not all platforms will (for example) segfault if NULL is dereferenced. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ There's no need to worry about death, it will not happen in your lifetime. - Raymond Smullyan From cs at zip.com.au Sun Jul 17 01:17:33 2011 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 17 Jul 2011 09:17:33 +1000 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <733331DF-3BB2-4879-A984-707CA2D0D094@gmail.com> References: <733331DF-3BB2-4879-A984-707CA2D0D094@gmail.com> Message-ID: <20110716231733.GA3954@cskk.homeip.net> On 16Jul2011 15:52, Raymond Hettinger wrote: | On Jul 16, 2011, at 3:15 PM, Devin Jeanpierre wrote: | > Slicing was brought up as a use-case where you can use a default | > without checking. | | This thread has lost contact with reality. It started with the | reasonable observation that many uses of str.find could | be replaced with a simple test using the in-operator. To be fair here, Mike's OP also mentioned that -1 is easy to misuse if not checked because it it still numeric. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Of course, I realize that the rain in the UK is much wetter than the rain we get here... - Eric Gunnerson From ethan at stoneleaf.us Sun Jul 17 03:40:35 2011 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 16 Jul 2011 18:40:35 -0700 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <733331DF-3BB2-4879-A984-707CA2D0D094@gmail.com> References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> <2EB37336-9E13-4294-BE10-63574DF45793@gmail.com> <7FCB2B33-9AFC-4D14-B7CA-2062465A304A@gmail.com> <733331DF-3BB2-4879-A984-707CA2D0D094@gmail.com> Message-ID: <4E223D93.7040608@stoneleaf.us> Raymond Hettinger wrote: > This thread has lost contact with reality. It started with the > reasonable observation that many uses of str.find could > be replaced with a simple test using the in-operator. > > Now, the thread is venturing into the typical python-ideas > world of making-up random use cases and APIs. Something > like str.find() is in many languages and it is definitely not the > norm for them to have found a need to both be able to > return -1 or to supply a default value. Nick's proposal was to enhance str.index(), not string.find(); Having str.index() accept a value to return on failure instead of raising an exception means it could do both jobs, and would also make it much less likely to wrongly use the failure return value of -1 from str.find() which is, unfortunately, a legitimate index value. ~Ethan~ From steve at pearwood.info Sun Jul 17 05:17:42 2011 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 17 Jul 2011 13:17:42 +1000 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <4E21B23C.5080302@stoneleaf.us> References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4E21B23C.5080302@stoneleaf.us> Message-ID: <4E225456.7020605@pearwood.info> Ethan Furman wrote: > I haven't read all the surrounding code to know if this will ever fail, > but the whole 'index = ... .find(...); if index and ...' certainly > doesn't lend confidence. After all, if you *know* newChild is in self, > why not use .index()? Perhaps because the name "find" tells you exactly what the method does, while the name "index" is ambiguous. Does it mean "what is at this index?" or "what index is this at?". I've occasionally seen people mistakingly write mylist.index(i) instead of mylist[i]. Including an experienced Python coder who did know better. In that case, it was just a thinko (like a typo, only in the brain *wink*), but in my opinion, the name "index" is not a good name. Since find and index are equally efficient when the substring is present, there's no speed advantage to choosing one over the other if you know that the substring is present. In my opinion str.find beats str.index for readability so comprehensively that there is no contest -- I would *always* use find if available. In my wishlist for Python 4000 I have: * list.index renamed to list.find * str.find and list.find return None if the argument is not found -- Steven From steve at pearwood.info Sun Jul 17 05:18:05 2011 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 17 Jul 2011 13:18:05 +1000 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> Message-ID: <4E22546D.3030707@pearwood.info> Mike Graham wrote: > On Sat, Jul 16, 2011 at 11:26 AM, Raymond Hettinger > wrote: >> Unless an API is flat out broken, deprecation is almost always a bad idea. >> This API has been around for a very long time, so deprecating it will >> break lots of people's code for almost zero benefit: >> http://www.google.com/codesearch#search/&q=%5C.find%5C(%20lang:%5Epython$&type=cs >> >> Raymond > > I agree that breaking people's code is a bad thing and have not > suggested removing str.find. This removal would require a transition > like that from Python 2.x to Python 3.x, a move that is not planned > and I personally do not ever expect. I think that's a disingenuous thing to say. You started this thread with an explicit call to deprecate str.find -- see your chosen subject line, and your first paragraph in this thread states: "It should be explicitly deprecated ..." What is the point of deprecating something if you don't intend to eventually remove it? > I appreciate your linking this search, which does indeed does show > that str.find is in wide use. However, looking at the first give pages > of results, this use seems largely unfortunate?literally the majority > of the times str.find is used, I would have used "substring in s" or A lot of very old code predates startswith and endswith. They only appeared in 2.0. Surprisingly, as late as 2.2, we were limited to testing for a single character: [steve at sylar src]$ python2.2 Python 2.2.3 (#1, Aug 12 2010, 01:08:27) [GCC 4.1.2 20070925 (Red Hat 4.1.2-27)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> "ab" in "abc" Traceback (most recent call last): File "", line 1, in ? TypeError: 'in ' requires character as left operand Some of the code on that first page supports Python 2.1. > "s.startswith(substring)". I also see code like "pos = s.find(" ("); > if pos + len(" (...)") > self._maxWidth:" which makes me very > uncomfortable and which I would have to read in detail to figure out > what's happening confidently if I was the maintaining the code. Whereas "pos = s.index(" ("); if pos + len(" (...)") > self._maxWidth:" is the height of readability, yes? People can write bad code no matter what tools you give them. -- Steven From cmjohnson.mailinglist at gmail.com Sun Jul 17 06:45:16 2011 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Sat, 16 Jul 2011 18:45:16 -1000 Subject: [Python-ideas] Left division operator In-Reply-To: <4E21E389.8090406@molden.no> References: <4E21E389.8090406@molden.no> Message-ID: In Python, \ has a very consistent meaning as "escape character." I would hate for that consistency to be broken by adding a new meaning to \. For example, right now x = loooooooooooooooooooong + "line" \ "something" is valid. But if \ were also an operator, would it mean something different than "line" \ "something"? As to the order of operations problem, why not just do x = eval_first y = eval_second r = y/x ? From ncoghlan at gmail.com Sun Jul 17 09:15:14 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 17 Jul 2011 17:15:14 +1000 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <4E223D93.7040608@stoneleaf.us> References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> <2EB37336-9E13-4294-BE10-63574DF45793@gmail.com> <7FCB2B33-9AFC-4D14-B7CA-2062465A304A@gmail.com> <733331DF-3BB2-4879-A984-707CA2D0D094@gmail.com> <4E223D93.7040608@stoneleaf.us> Message-ID: On Sun, Jul 17, 2011 at 11:40 AM, Ethan Furman wrote: > Raymond Hettinger wrote: >> Now, the thread is venturing into the typical python-ideas >> world of making-up random use cases and APIs. ? Something like str.find() >> is in many languages and it is definitely not the >> norm for them to have found a need to both be able to >> return -1 or to supply a default value. > > Nick's proposal was to enhance str.index(), not string.find(); ?Having > str.index() accept a value to return on failure instead of raising an > exception means it could do both jobs, and would also make it much less > likely to wrongly use the failure return value of -1 from str.find() which > is, unfortunately, a legitimate index value. Indeed, the problem as I see it is that our general idiom for functions and methods that raise 'Not Found' exceptions is to accept an optional parameter that specifies a value to return in the Not Found case. For historical reasons, we currently break that idiom for index() methods: instead of supplying an extra parameter to str.index, one instead switches to a completely different method (.find()) with no control over the sentinel value returned (it's always -1). For other sequences (e.g. list), there's no find equivalent, so you *have* to write the exception handling out explicitly. My proposal is to update the signature of index() (for all sequences, including the ABC) to follow the standard 'Not Found' idiom by accepting a 'missing' parameter that is returned for those cases where ValueError would otherwise be raised. Code that uses str.find would continue to work, but the recommended alternative would be obj.index(x, missing=None) (or appropriate default value). I would advise against any actual deprecation of str,find (cf. the deliberate lack of optparse deprecation). It's unfortunate that backwards compatibility means we can't use the more descriptive name, but that's life. However, I already have too much on my plate to push this forward for Python 3.3. I'm able to offer advice if someone would like to try their hand at writing a PEP, though. Regards, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sun Jul 17 09:22:29 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 17 Jul 2011 17:22:29 +1000 Subject: [Python-ideas] Left division operator In-Reply-To: <4E21E389.8090406@molden.no> References: <4E21E389.8090406@molden.no> Message-ID: On Sun, Jul 17, 2011 at 5:16 AM, Sturla Molden wrote: > I was wondering if anyone else has felt the need for a leftward division > operator in Python. That is, y\x in addition to x/y. Thus the operators: > > ? y \ x > ? y \\ x > > Why the statement "y\x" is different from "x/y": > > 1. Opposite order of evaluation: This is important if evaluating one of the > operands has side-effects. 1. Python evaluates operands in left to right fashion. Breaking this requires extremely good justification (The ternary operator is the only current exception that doesn't involve a closure, and that's only because the alternatives were all even worse). 2. Backslash is not a viable operator choice. Ever. (cf. Carl's answer) 3. The use case is far too narrow to be baked into the core language. If it's beyond high school mathematics, syntactic support within expressions is highly unlikely to ever be provided. Regards, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From raymond.hettinger at gmail.com Sun Jul 17 10:09:35 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Sun, 17 Jul 2011 01:09:35 -0700 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> <2EB37336-9E13-4294-BE10-63574DF45793@gmail.com> <7FCB2B33-9AFC-4D14-B7CA-2062465A304A@gmail.com> <733331DF-3BB2-4879-A984-707CA2D0D094@gmail.com> <4E223D93.7040608@stoneleaf.us> Message-ID: <8F3D40FE-E54C-4BB2-B3AA-56D86005E5FC@gmail.com> On Jul 17, 2011, at 12:15 AM, Nick Coghlan wrote: > > Indeed, the problem as I see it is that our general idiom for > functions and methods that raise 'Not Found' exceptions is to accept > an optional parameter that specifies a value to return in the Not > Found case. There's a difference between methods that return looked-up values (where a default might make sense) versus a method that returns an index (where it usually makes no sense at all). > > For historical reasons, we currently break that idiom for index() > methods: instead of supplying an extra parameter to str.index, one > instead switches to a completely different method (.find()) with no > control over the sentinel value returned (it's always -1). For other > sequences (e.g. list), there's no find equivalent, so you *have* to > write the exception handling out explicitly. > > My proposal is to update the signature of index() (for all sequences, > including the ABC) to follow the standard 'Not Found' idiom by > accepting a 'missing' parameter that is returned for those cases where > ValueError would otherwise be raised. > > Code that uses str.find would continue to work, but the recommended > alternative would be obj.index(x, missing=None) (or appropriate > default value). I would advise against any actual deprecation of > str,find (cf. the deliberate lack of optparse deprecation). > > It's unfortunate that backwards compatibility means we can't use the > more descriptive name, but that's life. > > However, I already have too much on my plate to push this forward for > Python 3.3. I'm able to offer advice if someone would like to try > their hand at writing a PEP, though. If someone takes this out of python-ideas land and into a serious PEP, they should be prepared to answer a number of tough questions: * Is this actually necessary? Is there something you currently can't code? If not, then it adds API complexity without adding any new capabilities. There is a high threshold for expanding the string API -- this would affect everyone learning python, every book written, every lint tool, every class seeking to be string-like, etc. So, it would need be a substantive improvement to be accepted. * Take a look at what other languages do. Practically every general purpose language has an API for doing substring searches. Since we're not blazing new territory here, there needs to be a good precedent for this change (no shooting from the hip when the problem has already been well solved many times over). * Use Google's code search to identify examples of real world code that would better with the new API. If the only use case is creating a new slicing one-liner, that likely is too rare and arcane to warrant a change. * Consider the effects of adding a second-way-to-do-it. Will it add to the learning curve, cause debates about the best way in a given situation, add more PEP 8 entries and pylint checks? Is it worth introducing version incompatibilities (i.e. runs on 3.3 but not earlier), etc. * What should the default value be? Is there any non-numerical result that ever makes sense; otherwise, you're just making a alias for the -1 currently returned by str.find(). If the default is some value that evaluates to False, will that create a common error where an if-test fails to disambiguate the default value from a substring found at position zero. If the new API is ambiguous or confusing in *any* way, then it will be a step backwards and make Python worse rather than better. * See if you can find examples where people have already found the need to write a helper function such as: def index_default(s, sub, default): try: return s.index(sub) except ValueError: return default If you find code like that in the wild, it may be an indication that people want this. If you don't, it may indicate otherwise. * Good API design requires some thinking about function/method signatures. Would making this a keyword-only argument soive the positional arguments problem? Since str.index() already takes arguments for the "start" and "end" index, is the full signature readable without keywords: mystr.index(possible_substr, 0, -1, default_value) Also look at the signature for the return value. Currently, it always returns a number, but if it can return a number or anything else, then all client code must be prepared to handle the alternatives with clean looking code that is self-evidently correct. * Perhaps talk to some people who write python code for a living to determine if they've ever needed this or whether it would end-up as cruft. (In my case, the answer is that I've not needed or wanted this in a over a decade of heavy Python use). Hopefully, this short and incomplete list will provide a good basis for thinking about whether the proposal is a good idea. Defending a PEP is no fun at all, so put in all your deep thinking up front. Cheers, Raymond From masklinn at masklinn.net Sun Jul 17 11:13:23 2011 From: masklinn at masklinn.net (Masklinn) Date: Sun, 17 Jul 2011 11:13:23 +0200 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <8F3D40FE-E54C-4BB2-B3AA-56D86005E5FC@gmail.com> References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> <2EB37336-9E13-4294-BE10-63574DF45793@gmail.com> <7FCB2B33-9AFC-4D14-B7CA-2062465A304A@gmail.com> <733331DF-3BB2-4879-A984-707CA2D0D094@gmail.com> <4E223D93.7040608@stoneleaf.us> <8F3D40FE-E54C-4BB2-B3AA-56D86005E5FC@gmail.com> Message-ID: <12978AE8-6DE7-435E-B1F2-D1C7F397C022@masklinn.net> On 2011-07-17, at 10:09 , Raymond Hettinger wrote: > On Jul 17, 2011, at 12:15 AM, Nick Coghlan wrote: >> >> Indeed, the problem as I see it is that our general idiom for >> functions and methods that raise 'Not Found' exceptions is to accept >> an optional parameter that specifies a value to return in the Not >> Found case. > > There's a difference between methods that return looked-up values > (where a default might make sense) versus a method that returns > an index (where it usually makes no sense at all). Why not? An index is a looked-up value here (it's just a reverse lookup) and .find is returning a (non-configurable) default value is it not? > * Take a look at what other languages do. Practically every general > purpose language has an API for doing substring searches. Since > we're not blazing new territory here, there needs to be a good precedent > for this change (no shooting from the hip when the problem has already > been well solved many times over). Other languages do everything and their reverse. You have languages returning ?1 or 0 (the latter for 1-indexed languages), languages returning differently-typed sentinels, etc? SML even returns the length of the string. See a listing at http://en.wikipedia.org/wiki/Comparison_of_programming_languages_(string_functions)#Find The most common behavior on the page does seem to be returning a numerical sentinel. On the other hand, I'm not sure how many of these languages return a sentinel value which is also a valid index. From dirkjan at ochtman.nl Sun Jul 17 11:15:39 2011 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Sun, 17 Jul 2011 11:15:39 +0200 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <8F3D40FE-E54C-4BB2-B3AA-56D86005E5FC@gmail.com> References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> <2EB37336-9E13-4294-BE10-63574DF45793@gmail.com> <7FCB2B33-9AFC-4D14-B7CA-2062465A304A@gmail.com> <733331DF-3BB2-4879-A984-707CA2D0D094@gmail.com> <4E223D93.7040608@stoneleaf.us> <8F3D40FE-E54C-4BB2-B3AA-56D86005E5FC@gmail.com> Message-ID: On Sun, Jul 17, 2011 at 10:09, Raymond Hettinger wrote: > If someone takes this out of python-ideas land and into a serious PEP, > they should be prepared to answer a number of tough questions: > > * Is this actually necessary? ?Is there something you currently can't code? > If not, then it adds API complexity without adding any new capabilities. > There is a high threshold for expanding the string API -- this would affect > everyone learning python, every book written, every lint tool, every class > seeking to be string-like, etc. ?So, it would need be a substantive improvement > to be accepted. > > * Take a look at what other languages do. ?Practically every general > purpose language has an API for doing substring searches. ?Since > we're not blazing new territory here, there needs to be a good precedent > for this change (no shooting from the hip when the problem has already > been well solved many times over). > > * Use Google's code search to identify examples of real world code > that would better with the new API. ? If the only use case is creating > a new slicing one-liner, that likely is too rare and arcane to warrant > a change. > > * Consider the effects of adding a second-way-to-do-it. ?Will it add to the > learning curve, cause debates about the best way in a given situation, > add more PEP 8 entries and pylint checks? Is it worth introducing > version incompatibilities (i.e. runs on 3.3 but not earlier), etc. > > * What should the default value be? Is there any non-numerical result > that ever makes sense; otherwise, you're just making a alias for the -1 > currently returned by str.find(). ?If the default is some value that evaluates > to False, will that create a common error where an if-test fails to disambiguate > the default value from a substring found at position zero. ?If the new API > is ambiguous or confusing in *any* way, then it will be a step backwards > and make Python worse rather than better. > > * See if you can find examples where people have already found the > need to write a helper function such as: > > ? ?def index_default(s, sub, default): > ? ? ? ? ?try: > ? ? ? ? ? ? ? ?return s.index(sub) > ? ? ? ? ?except ValueError: > ? ? ? ? ? ? ? ?return default > > If you find code like that in the wild, it may be an indication that people > want this. ?If you don't, it may indicate otherwise. > > * Good API design requires some thinking about function/method > signatures. ?Would making this a keyword-only argument soive the > positional arguments problem? ?Since str.index() already takes arguments > for the "start" and "end" index, is the full signature readable without keywords: > > ? ?mystr.index(possible_substr, 0, -1, default_value) > > Also look at the signature for the return value. ?Currently, it always returns > a number, but if it can return a number or anything else, then all client > code must be prepared to handle the alternatives with clean looking code > that is self-evidently correct. > > * Perhaps talk to some people who write python code for a living > to determine if they've ever needed this or whether it would end-up > as cruft. ?(In my case, the answer is that I've not needed or wanted > this in a over a decade of heavy Python use). > > Hopefully, this short and incomplete list will provide a good basis > for thinking about whether the proposal is a good idea. ?Defending > a PEP is no fun at all, so put in all your deep thinking up front. Someone needs to take this list and put it into a wiki or the devguide. Cheers, Dirkjan From ethan at stoneleaf.us Sun Jul 17 11:30:09 2011 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 17 Jul 2011 02:30:09 -0700 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <8F3D40FE-E54C-4BB2-B3AA-56D86005E5FC@gmail.com> References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> <2EB37336-9E13-4294-BE10-63574DF45793@gmail.com> <7FCB2B33-9AFC-4D14-B7CA-2062465A304A@gmail.com> <733331DF-3BB2-4879-A984-707CA2D0D094@gmail.com> <4E223D93.7040608@stoneleaf.us> <8F3D40FE-E54C-4BB2-B3AA-56D86005E5FC@gmail.com> Message-ID: <4E22ABA1.9050608@stoneleaf.us> Raymond Hettinger wrote: > On Jul 17, 2011, at 12:15 AM, Nick Coghlan wrote: >> Indeed, the problem as I see it is that our general idiom for >> functions and methods that raise 'Not Found' exceptions is to accept >> an optional parameter that specifies a value to return in the Not >> Found case. > > There's a difference between methods that return looked-up values > (where a default might make sense) versus a method that returns > an index (where it usually makes no sense at all). We are not talking about a default value to return -- the default will still be the behavior of raising a ValueError if the substring is not found. Consider the proposed signature: _sentinal = object() class str(): def index(substring, start, end, missing=_sentinal): # looks for string .... # string not found -- now what? if missing is _sentinal: raise ValueError('...') else: return missing The addition is that *if* the caller specifies an object for missing, return that value, *otherwise* raise ValueError just like we do now. >> For historical reasons, we currently break that idiom for index() >> methods: instead of supplying an extra parameter to str.index, one >> instead switches to a completely different method (.find()) with no >> control over the sentinel value returned (it's always -1). For other >> sequences (e.g. list), there's no find equivalent, so you *have* to >> write the exception handling out explicitly. >> >> My proposal is to update the signature of index() (for all sequences, >> including the ABC) to follow the standard 'Not Found' idiom by >> accepting a 'missing' parameter that is returned for those cases where >> ValueError would otherwise be raised. >> >> Code that uses str.find would continue to work, but the recommended >> alternative would be obj.index(x, missing=None) (or appropriate >> default value). Hmmm -- okay, perhaps we are... let me say, then, that I agree having a default return is not the way to go; this would break everything that expects .index() to exception out if the substring is not found -- in other words, everything that uses .index(). My take on the idea is to have the new 'missing' argument be optional, and if not specified then current behavior is unchanged, but if specified then that value is returned instead. > If someone takes this out of python-ideas land and into a serious PEP, > they should be prepared to answer a number of tough questions: > > * Is this actually necessary? Is there something you currently can't code? > If not, then it adds API complexity without adding any new capabilities. > There is a high threshold for expanding the string API -- this would affect > everyone learning python, every book written, every lint tool, every class > seeking to be string-like, etc. Trying to be string-like at the moment is such a PITA I really don't see this tiny extra bit as a serious burden. Consider this nice simple code: class MyStr(str): def find(substr, start=None, end=None): # whatever extra I want to do before passing off to str # now pass off to str return str.find(substr, start, end) Too bad it doesn't work: --> test = MyStr('this is a test') --> test.find('is') Traceback (most recent call last): File "", line 1, in File "", line 5, in find TypeError: slice indices must be integers or None or have an __index__ method (Yes, this was fixed in 2.6, and as soon as I'm willing to drop support for earlier versions I can remove the following boilerplate: start = start or 0 end = end or len(self) and yes, I wouldn't be able to use the new .index(..., missing=_whatever) for a while, but that doesn't mean we should stop improving the language.) > * Take a look at what other languages do. Practically every general > purpose language has an API for doing substring searches. Since > we're not blazing new territory here, there needs to be a good precedent > for this change (no shooting from the hip when the problem has already > been well solved many times over). Why not? 'Well solved' does not mean there is no room for improvement. And going through the whole PEP process does not feel like 'shooting from the hip'. > * Consider the effects of adding a second-way-to-do-it. Will it add to the > learning curve, cause debates about the best way in a given situation, > add more PEP 8 entries and pylint checks? Is it worth introducing > version incompatibilities (i.e. runs on 3.3 but not earlier), etc. You mean like 'runs on 2.6+ but not earlier'? > * What should the default value be? There should be no default value, in my opinion. > Is there any non-numerical result > that ever makes sense; otherwise, you're just making a alias for the -1 > currently returned by str.find(). If the default is some value that evaluates > to False, will that create a common error where an if-test fails to disambiguate > the default value from a substring found at position zero. The most effective argument by far, IMO, both for not having a default value, and for being very careful about what the caller chooses to use for the missing argument. I think a bomb would be appropriate here: class Bomb(): 'singleton object: blows up on any usage' def __bool__(self): raise OopsError('yell at the programmer!") etc then in usage it's a check for object identity, anything else reminds somebody they forgot to do something. > Hopefully, this short and incomplete list will provide a good basis > for thinking about whether the proposal is a good idea. Defending > a PEP is no fun at all, so put in all your deep thinking up front. Many good points -- thank you for taking the time. ~Ethan~ From raymond.hettinger at gmail.com Sun Jul 17 12:03:40 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Sun, 17 Jul 2011 03:03:40 -0700 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <12978AE8-6DE7-435E-B1F2-D1C7F397C022@masklinn.net> References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> <2EB37336-9E13-4294-BE10-63574DF45793@gmail.com> <7FCB2B33-9AFC-4D14-B7CA-2062465A304A@gmail.com> <733331DF-3BB2-4879-A984-707CA2D0D094@gmail.com> <4E223D93.7040608@stoneleaf.us> <8F3D40FE-E54C-4BB2-B3AA-56D86005E5FC@gmail.com> <12978AE8-6DE7-435E-B1F2-D1C7F397C022@masklinn.net> Message-ID: <6F862A5F-49DE-4985-B8E3-38AB8E73806B@gmail.com> On Jul 17, 2011, at 2:13 AM, Masklinn wrote: > On 2011-07-17, at 10:09 , Raymond Hettinger wrote: >> On Jul 17, 2011, at 12:15 AM, Nick Coghlan wrote: >>> >>> Indeed, the problem as I see it is that our general idiom for >>> functions and methods that raise 'Not Found' exceptions is to accept >>> an optional parameter that specifies a value to return in the Not >>> Found case. >> >> There's a difference between methods that return looked-up values >> (where a default might make sense) versus a method that returns >> an index (where it usually makes no sense at all). > Why not? An index is a looked-up value here (it's just a reverse lookup) > and .find is returning a (non-configurable) default value is it not? There is a fundamental difference between content (values in the list) and the list position. There are meaningful search defaults for the former but not the latter. It's important that to grasp this distinction before going further. > >> * Take a look at what other languages do. Practically every general >> purpose language has an API for doing substring searches. Since >> we're not blazing new territory here, there needs to be a good precedent >> for this change (no shooting from the hip when the problem has already >> been well solved many times over). > Other languages do everything and their reverse. You have languages > returning ?1 or 0 (the latter for 1-indexed languages), languages > returning differently-typed sentinels, etc? > > SML even returns the length of the string. > > See a listing at http://en.wikipedia.org/wiki/Comparison_of_programming_languages_(string_functions)#Find My reading is that not a single one of these entries has a signature with a user specifiable default value of arbitrary type. IOW, there is nothing resembling proposal on the table. AFAICT, all of them are equivalent to either our current str.find() or str.index(). Raymond From steve at pearwood.info Sun Jul 17 12:56:15 2011 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 17 Jul 2011 20:56:15 +1000 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <12978AE8-6DE7-435E-B1F2-D1C7F397C022@masklinn.net> References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> <2EB37336-9E13-4294-BE10-63574DF45793@gmail.com> <7FCB2B33-9AFC-4D14-B7CA-2062465A304A@gmail.com> <733331DF-3BB2-4879-A984-707CA2D0D094@gmail.com> <4E223D93.7040608@stoneleaf.us> <8F3D40FE-E54C-4BB2-B3AA-56D86005E5FC@gmail.com> <12978AE8-6DE7-435E-B1F2-D1C7F397C022@masklinn.net> Message-ID: <4E22BFCF.5080407@pearwood.info> Masklinn wrote: > On 2011-07-17, at 10:09 , Raymond Hettinger wrote: >> On Jul 17, 2011, at 12:15 AM, Nick Coghlan wrote: >>> Indeed, the problem as I see it is that our general idiom for >>> functions and methods that raise 'Not Found' exceptions is to accept >>> an optional parameter that specifies a value to return in the Not >>> Found case. >> There's a difference between methods that return looked-up values >> (where a default might make sense) versus a method that returns >> an index (where it usually makes no sense at all). > Why not? An index is a looked-up value here (it's just a reverse lookup) > and .find is returning a (non-configurable) default value is it not? No. In context, Raymond is not talking about values as arbitrary objects. He is talking specifically about values of a collection. E.g. given: mylist = [23, 42, 100] the values Raymond is talking about are 23, 42 and 100, *not* 0, 1, 2 (the indexes of the list) or 3 (the length of the list) or 165 (the sum of the list) or any other arbitrary value. >> * Take a look at what other languages do. Practically every general >> purpose language has an API for doing substring searches. Since >> we're not blazing new territory here, there needs to be a good precedent >> for this change (no shooting from the hip when the problem has already >> been well solved many times over). > Other languages do everything and their reverse. You have languages > returning ?1 or 0 (the latter for 1-indexed languages), languages > returning differently-typed sentinels, etc? > > SML even returns the length of the string. > > See a listing at http://en.wikipedia.org/wiki/Comparison_of_programming_languages_(string_functions)#Find I can't really see that they do "everything and their reverse". There are two basic strategies: return an out-of-bound value, and raise an exception, both of which Python already does. Out-of-bound values are usually one smaller than the lowest valid index (0 or -1) or one higher than the highest valid index (length of the string, or more greater than the length of the string). A couple of languages return False, which is inappropriate for Python on account of False equaling 0. Some return a dedicated "Not Found" special value, but Python doesn't go in for a proliferation of special constants. A couple of languages, including Ruby, return the equivalent of Python's None. Notably missing is anything like the ability for the caller to specify what index to return if the sub-string is missing. Ask yourself, can you imagine needing mydict.get(key, 1) or mydict.get(key, set())? I expect that you can easily think of reasons why this would be useful. The usefulness of being able to set the return value of failed lookups like dicts is obvious. I wish lists also had a similar get method, and I bet that everybody reading this, even if they disagree that it should be built-in, can see the value of it as a utility function. But can you think of a realistic scenario where you might want to call mystring.find(substr, missing=1)? Why would you want "substring not present" and "substring found at index 1" to both return the same thing? How about mystring.find(substr, missing=set())? If you can't imagine a realistic scenario where you would want such a feature, then you probably don't need this proposed feature. > The most common behavior on the page does seem to be returning a > numerical sentinel. On the other hand, I'm not sure how many of these > languages return a sentinel value which is also a valid index. The first table on the page says that four languages accept negative indexes: Python, Ruby, Perl and Lua. Perl and Python return -1 on not found; Ruby and Lua return nil. -- Steven From sturla at molden.no Sun Jul 17 14:37:20 2011 From: sturla at molden.no (Sturla Molden) Date: Sun, 17 Jul 2011 14:37:20 +0200 Subject: [Python-ideas] Left division operator In-Reply-To: References: <4E21E389.8090406@molden.no> Message-ID: <4E22D780.6030607@molden.no> Den 17.07.2011 06:45, skrev Carl Johnson: > In Python, \ has a very consistent meaning as "escape character." I > would hate for that consistency to be broken by adding a new meaning > to \. For example, right now > > x = loooooooooooooooooooong + "line" \ > "something" > > is valid. But if \ were also an operator, would it mean something > different than "line" \ "something"? > Not if it's followed by a newline. This would not be affected: x = loooooooooooooooooooong + "line" \ "something" This would be affected though: x = (loooooooooooooooooooong + "line" \ "something") Sturla From ncoghlan at gmail.com Sun Jul 17 14:38:32 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 17 Jul 2011 22:38:32 +1000 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <4E22BFCF.5080407@pearwood.info> References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> <2EB37336-9E13-4294-BE10-63574DF45793@gmail.com> <7FCB2B33-9AFC-4D14-B7CA-2062465A304A@gmail.com> <733331DF-3BB2-4879-A984-707CA2D0D094@gmail.com> <4E223D93.7040608@stoneleaf.us> <8F3D40FE-E54C-4BB2-B3AA-56D86005E5FC@gmail.com> <12978AE8-6DE7-435E-B1F2-D1C7F397C022@masklinn.net> <4E22BFCF.5080407@pearwood.info> Message-ID: On Sun, Jul 17, 2011 at 8:56 PM, Steven D'Aprano wrote: > I can't really see that they do "everything and their reverse". There are > two basic strategies: return an out-of-bound value, and raise an exception, > both of which Python already does. In a language that accepts negative indices, -1 is not out of bounds, and hence is an objectively bad value to return. If str.find() returned None for missing substrings, we wouldn't be having this discussion. However, backwards compatibility requirements mean that option is not available to us. Is this a language wart where the correct answer is to tell everyone new to the language that complains about the current behaviour to "suck it up and learn to live with it"? Maybe. But claiming that str.find() is a *good* API in a language that accepts negative indices would be flat out wrong. Good *enough*, maybe, but not good in an absolute sense. As I see it, there are a few options. 1. Do nothing. Quite a reasonable option. str.find() is flawed and str.index() can be annoying to use, but fixing this may not be worth the hassle. However, this is not the same as claiming the str.find() behaviour is a good API - it's just acknowledging the wart and deciding not to address it. 2. Add an alternate behaviour to str.index() that allows the exception to optionally be converted into a sentinel value. Inspired by getattr, getitem, dict.get, next, etc. Made messy by the need for the sentinel to go after the existing arguments and the fact that the number of sane sentinel values is extremely limited (aside from None, the only remotely useful possibilities I can think of are 0 and len(seq), and even those only hypothetically). This solution requires changing a builtin method API, as well as adding keyword argument support to CPython string methods. Also raises the question of whether or not the Sequence ABC (and sequences in the stdlib) should be adjusted accordingly. 3. Search for a more general solution that simplifies the following common try/except pattern into a single expression (or simple statement): try: val = f() except RelevantException: val = default # further operations using val This has been attempted before, but never successfully as it isn't *that* common (so ugly solutions are never going to be accepted as better than the status quo) and it's extremely difficult to come up with a syntax that neatly captures the 3 necessary elements and can still be handled easily by the parser. Since I have no reason to believe 3 will get any further than it has in that past, making an explicit decision between 1 and 2 is the reason I'd like to see a PEP. PEPs aren't just about getting new features into the language - the Rejected ones are also about documenting the reasons we have chosen *not* to do certain things (and the Deferred ones point out that some things are just hard to do in a way that provides a net benefit to the language). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From sturla at molden.no Sun Jul 17 14:38:55 2011 From: sturla at molden.no (Sturla Molden) Date: Sun, 17 Jul 2011 14:38:55 +0200 Subject: [Python-ideas] Left division operator In-Reply-To: References: <4E21E389.8090406@molden.no> Message-ID: <4E22D7DF.1010800@molden.no> Den 17.07.2011 06:45, skrev Carl Johnson: > In Python, \ has a very consistent meaning as "escape character." Inside strings, yes, but not in the code. Sturla From mikegraham at gmail.com Sun Jul 17 16:48:51 2011 From: mikegraham at gmail.com (Mike Graham) Date: Sun, 17 Jul 2011 10:48:51 -0400 Subject: [Python-ideas] Left division operator In-Reply-To: <4E21E389.8090406@molden.no> References: <4E21E389.8090406@molden.no> Message-ID: On Sat, Jul 16, 2011 at 3:16 PM, Sturla Molden wrote: > I was wondering if anyone else has felt the need for a leftward division > operator in Python. That is, y\x in addition to x/y. Thus the operators: > > ? y \ x > ? y \\ x No, I haven't felt this need. As a former heavy Matlab user it crossed my mind, but I don't really feel we're missing anything. The function approach is just as clear and is more useful, in that different problems call for different linear algebra solution routines. I am not aware for any other good usage for such an operator. > Why the statement "y\x" is different from "x/y": > > 1. Opposite order of evaluation: This is important if evaluating one of the > operands has side-effects. If we did get an \ operator, why not make it work like the rest of Python? I don't see a compelling reason for this. > 2. In linear algebra, (Y**-1)*X is in general not equal to X*(Y**-1). Matrix > and vector algebra in fundamental to computer graphics and scientific > computing. Note that neither one of these are typically written as / or \ in math. > 3. NumPy solves the lack of "\" operator by having a function > ?np.linalg.solve. But using it in matrix expressions has the effect of > mixing Pythonic infix operators with a Lisp-like prefix operator. This isn't a "Lisp-like prefix operator". Normal functions are incredibly common in Python, in fact moreso than operators. They carry the initial advantage that there can be many of them and that they needn't be binary; this advantage is applicable to the problem at hand, because I can do solve_triangular(x, y) or solve(x, y, sym_pos=True, overwrite_a=True). Mike From stefan_ml at behnel.de Sun Jul 17 16:49:51 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 17 Jul 2011 16:49:51 +0200 Subject: [Python-ideas] Left division operator In-Reply-To: References: <4E21E389.8090406@molden.no> Message-ID: Nick Coghlan, 17.07.2011 09:22: > On Sun, Jul 17, 2011 at 5:16 AM, Sturla Molden wrote: >> I was wondering if anyone else has felt the need for a leftward division >> operator in Python. That is, y\x in addition to x/y. Thus the operators: >> >> y \ x >> y \\ x >> >> Why the statement "y\x" is different from "x/y": >> >> 1. Opposite order of evaluation: This is important if evaluating one of the >> operands has side-effects. > > 1. Python evaluates operands in left to right fashion. Breaking this > requires extremely good justification (The ternary operator is the > only current exception that doesn't involve a closure, and that's only > because the alternatives were all even worse). It would still be evaluated left-to-right, as far as I understand Sturla. Note that he swapped the x and y operands in the two examples for \ and /. So both operators would go left-to-right, only the resulting division would be reversed in the backslash case, thus dividing by the operand that was evaluated *first*. That being said, I agree with the comments below. > 2. Backslash is not a viable operator choice. Ever. (cf. Carl's answer) > > 3. The use case is far too narrow to be baked into the core language. > If it's beyond high school mathematics, syntactic support within > expressions is highly unlikely to ever be provided. Plus, I don't find this syntax very readable, but I guess that just sits on top of your above two arguments. Stefan From mikegraham at gmail.com Sun Jul 17 16:49:59 2011 From: mikegraham at gmail.com (Mike Graham) Date: Sun, 17 Jul 2011 10:49:59 -0400 Subject: [Python-ideas] Left division operator In-Reply-To: <4E22D780.6030607@molden.no> References: <4E21E389.8090406@molden.no> <4E22D780.6030607@molden.no> Message-ID: On Sun, Jul 17, 2011 at 8:37 AM, Sturla Molden wrote: > This would be affected though: > > x = (loooooooooooooooooooong + "line" \ > "something") That breaks current code. Mike From stefan_ml at behnel.de Sun Jul 17 17:19:52 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 17 Jul 2011 17:19:52 +0200 Subject: [Python-ideas] Left division operator In-Reply-To: References: <4E21E389.8090406@molden.no> <4E22D780.6030607@molden.no> Message-ID: Mike Graham, 17.07.2011 16:49: > On Sun, Jul 17, 2011 at 8:37 AM, Sturla Molden wrote: >> This would be affected though: >> >> x = (loooooooooooooooooooong + "line" \ >> "something") > > That breaks current code. No, it doesn't. This code would continue to work as before, but that's exactly the problem here. It would be impossible to use a backslash division operator at the end of a line even in a properly braced expression, as that would turn it into an escape operator. Stefan From mikegraham at gmail.com Sun Jul 17 17:38:48 2011 From: mikegraham at gmail.com (Mike Graham) Date: Sun, 17 Jul 2011 11:38:48 -0400 Subject: [Python-ideas] Left division operator In-Reply-To: References: <4E21E389.8090406@molden.no> <4E22D780.6030607@molden.no> Message-ID: On Sun, Jul 17, 2011 at 11:19 AM, Stefan Behnel wrote: > Mike Graham, 17.07.2011 16:49: >> >> On Sun, Jul 17, 2011 at 8:37 AM, Sturla Molden wrote: >>> >>> This would be affected though: >>> >>> x = (loooooooooooooooooooong + "line" \ >>> "something") >> >> That breaks current code. > > No, it doesn't. This code would continue to work as before, but that's > exactly the problem here. It would be impossible to use a backslash division > operator at the end of a line even in a properly braced expression, as that > would turn it into an escape operator. > > Stefan You are recommending this be handled the opposite way than Sturla, to whom I was responding. Mike From steve at pearwood.info Sun Jul 17 18:27:30 2011 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 18 Jul 2011 02:27:30 +1000 Subject: [Python-ideas] Left division operator In-Reply-To: <4E21E389.8090406@molden.no> References: <4E21E389.8090406@molden.no> Message-ID: <4E230D72.507@pearwood.info> Sturla Molden wrote: > I was wondering if anyone else has felt the need for a leftward division > operator in Python. That is, y\x in addition to x/y. Thus the operators: > y \ x > y \\ x You have flipped the operator AND flipped the order of the operands, which makes it confusing to me. I find it easy to reason about this proposal if I stick to one change at a time. I presume that x\y will be equivalent to y/x, or 1/(x/y) if x and y are numbers. Is there a standard mathematical infix operator for this? Since x/y => x.__truediv__(y), then x\y => x.__rtruediv__(y). Once only I have wanted something close to this. It wasn't terribly compelling: I ended up writing the obvious helper function: def divide_by(x, y): return y/x It did cross my mind at the time that it would be neat if this had \ as the operator, but I've met enough people who get their forward and back slashes mixed up that I wouldn't want to add to their confusion. So I would not support using \ as the operator for this. > Why the statement "y\x" is different from "x/y": > > 1. Opposite order of evaluation: This is important if evaluating one of > the operands has side-effects. This is ambiguous to me. Taken literally, you mean that the right-hand operatand (x) will be evaluated before the left-hand operand (y) (the opposite of normal for Python). Is that what you mean? I don't see any justification for that. Even the exponentiation operator evaluates terms from left-to-right although the operator binds more strongly to the right: >>> def f(x): ... print(x) ... return x ... >>> f(2) ** f(3) ** f(4) 2 3 4 2417851639229258349412352 As far as I know, right-to-left evaluation would be a major change to Python. Or do you mean that the \ operator has the same order of evaluation as other operators, but instead of writing this: tmp = denominator() # Evaluate this first, because it has side-effects. result = numerator()/tmp you can write this: result = denominator()\numerator() I don't think this is important enough to justify a new operator. After all, can't we say the same thing for every other non-commutative operator? y = subtrahend() # Evaluate this first, because it has side-effects. result = minuend() - tmp Do we need to introduce new "reversed" operators for subtraction, exponentiation, left and right binary shift, matrix multiplication, even for addition? (Because a type can define __add__ and __radd__ separately, there is no guarantee that a+b is commutative.) If not, why single out division? > 2. In linear algebra, (Y**-1)*X is in general not equal to X*(Y**-1). > Matrix and vector algebra in fundamental to computer graphics and > scientific computing. > > 3. NumPy solves the lack of "\" operator by having a function > np.linalg.solve. But using it in matrix expressions has the effect of > mixing Pythonic infix operators with a Lisp-like prefix operator. Is this supposed to be a problem that needs fixing? > Two other issues: > > 1. The expressions > y /= x > y //= x > are evaluated in the same order as "y \ x", not "x / y". > > 2. The expressions > y \= x > y \\= x > should perhaps be illegal due to implied side-effects on RHS. Surely y \= x would have the same meaning as all the other augmented assignments? It should be the same as y = y\x That is, if x and y are numbers, y \= x would be equivalent to y = x/y I don't see any reason to make that illegal. -- Steven From guido at python.org Sun Jul 17 18:55:49 2011 From: guido at python.org (Guido van Rossum) Date: Sun, 17 Jul 2011 09:55:49 -0700 Subject: [Python-ideas] Left division operator In-Reply-To: <4E21E389.8090406@molden.no> References: <4E21E389.8090406@molden.no> Message-ID: On Sat, Jul 16, 2011 at 12:16 PM, Sturla Molden wrote: > I was wondering if anyone else has felt the need for a leftward division > operator in Python. That is, y\x in addition to x/y. No. -- --Guido van Rossum (python.org/~guido) From sturla at molden.no Sun Jul 17 19:05:46 2011 From: sturla at molden.no (Sturla Molden) Date: Sun, 17 Jul 2011 19:05:46 +0200 Subject: [Python-ideas] Left division operator In-Reply-To: <4E230D72.507@pearwood.info> References: <4E21E389.8090406@molden.no> <4E230D72.507@pearwood.info> Message-ID: <4E23166A.80209@molden.no> Den 17.07.2011 18:27, skrev Steven D'Aprano: > > tmp = denominator() # Evaluate this first, because it has side-effects. > result = numerator()/tmp The order of the arguments is lost to the division operator. > > you can write this: > > result = denominator()\numerator() Only for real numbers. result = denominator()\numerator() would be equivalent to tmp = 1/denominator() result = tmp * numerator() But please ignore this suggestion, I agree that it is a bad suggestion. There is a better solution for this particular problem, which is to make 1/x return a new type, so that (1/x)*y evalutes differenty from y*(1/x). So I am thinking of suggesting addition of an "inverted matrix" type to NumPy instead. Sturla From python at mrabarnett.plus.com Mon Jul 18 00:39:52 2011 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 17 Jul 2011 23:39:52 +0100 Subject: [Python-ideas] Left division operator In-Reply-To: <4E23166A.80209@molden.no> References: <4E21E389.8090406@molden.no> <4E230D72.507@pearwood.info> <4E23166A.80209@molden.no> Message-ID: <4E2364B8.2050305@mrabarnett.plus.com> On 17/07/2011 18:05, Sturla Molden wrote: > Den 17.07.2011 18:27, skrev Steven D'Aprano: >> >> tmp = denominator() # Evaluate this first, because it has side-effects. >> result = numerator()/tmp > > The order of the arguments is lost to the division operator. > >> >> you can write this: >> >> result = denominator()\numerator() > > Only for real numbers. > > result = denominator()\numerator() > > would be equivalent to > > tmp = 1/denominator() > result = tmp * numerator() > > But please ignore this suggestion, I agree that it is a bad suggestion. > > There is a better solution for this particular problem, which is to make > 1/x return a new type, so that (1/x)*y evalutes differenty from y*(1/x). > So I am thinking of suggesting addition of an "inverted matrix" type to > NumPy instead. > If there's a reverse division operator, then why not a reverse subtraction operator too? Or a reverse modulo operator? From mikegraham at gmail.com Mon Jul 18 03:20:16 2011 From: mikegraham at gmail.com (Mike Graham) Date: Sun, 17 Jul 2011 21:20:16 -0400 Subject: [Python-ideas] Left division operator In-Reply-To: <4E2364B8.2050305@mrabarnett.plus.com> References: <4E21E389.8090406@molden.no> <4E230D72.507@pearwood.info> <4E23166A.80209@molden.no> <4E2364B8.2050305@mrabarnett.plus.com> Message-ID: On Sun, Jul 17, 2011 at 6:39 PM, MRAB wrote: > If there's a reverse division operator, then why not a reverse > subtraction operator too? Or a reverse modulo operator? The inspiration was Matlab's \ operator, which comes in handy when you have a matrix A and a vector b and want to solve "A * x = b". In Matlab this is spelled "x = A \ b" and in Python with numpy it is spelled something like numpy.linalg.solve(A, b). This is a fairly common operation in numerical programs. No similar history and utility is obvious for these other operators. That being said, solve(A, b) is fine as far as I can see. Mike From greg.ewing at canterbury.ac.nz Mon Jul 18 07:46:21 2011 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 18 Jul 2011 17:46:21 +1200 Subject: [Python-ideas] Left division operator In-Reply-To: <4E2364B8.2050305@mrabarnett.plus.com> References: <4E21E389.8090406@molden.no> <4E230D72.507@pearwood.info> <4E23166A.80209@molden.no> <4E2364B8.2050305@mrabarnett.plus.com> Message-ID: <4E23C8AD.7010501@canterbury.ac.nz> MRAB wrote: > If there's a reverse division operator, then why not a reverse > subtraction operator too? I don't think the issue arises much with subtraction, because it's uncommon to work with addition-like operations that are not commutative. As for modulo, it's not really the inverse of anything in the way that / is the inverse of *. If there were a reversed division operator, I would probably have used it in a library I wrote recently for coordinate transformations. In that domain it's conventional to write the transformation on the left and the thing being transformed on the right, and it's natural to want to write an inverse transformation the same way. The solution I ended up with was to write the transformation of V by the inverse of T as T.inv * V where the inv attribute is calculated on demand and cached. This corresponds somewhat to the way a mathematician would write -1 T * V -- Greg From mal at egenix.com Mon Jul 18 10:41:58 2011 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 18 Jul 2011 10:41:58 +0200 Subject: [Python-ideas] Deprecate str.find In-Reply-To: <2EB37336-9E13-4294-BE10-63574DF45793@gmail.com> References: <391B0949-B8E9-4997-8824-03034CB569FA@gmail.com> <4D453E6D-F999-49A1-A586-AA1FC499D4B8@gmail.com> <2EB37336-9E13-4294-BE10-63574DF45793@gmail.com> Message-ID: <4E23F1D6.4010500@egenix.com> Raymond Hettinger wrote: > I just remembered one other thought on the subject. > > Usually, when Python introduces a method such as .index() > that raises an exception for the not-found case, there > are immediate requests for variants that don't raise exceptions: > > dict.pop(key, default) > dict.get(key, defalut) > next(iterable, default) > getattr(obj, attr, default) > re.match() --> None or matchobject > > People seem to hate wrapping try/except around simple calls. Indeed, because for simple things, you don't want to clutter up your code with lots of nested try-excepts. It also has to do with performance: exceptions should only be used for exceptional situations. Not finding a sub-string in some line read from a log file is not an exceptional situation. In fact, it's most likely the common case. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 18 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From bborcic at gmail.com Mon Jul 18 18:13:02 2011 From: bborcic at gmail.com (Boris Borcic) Date: Mon, 18 Jul 2011 18:13:02 +0200 Subject: [Python-ideas] Left division operator In-Reply-To: <4E21E389.8090406@molden.no> References: <4E21E389.8090406@molden.no> Message-ID: Sturla Molden wrote: > I was wondering if anyone else has felt the need for a leftward division > operator in Python. That is, y\x in addition to x/y. Thus the operators: > > y \ x > y \\ x > Well, somebody just created a quaternion dtype for Numpy, so might have a point there. From larry.bugbee at gmail.com Wed Jul 20 08:51:35 2011 From: larry.bugbee at gmail.com (Larry Bugbee) Date: Tue, 19 Jul 2011 23:51:35 -0700 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: Message-ID: > Steven D'Aprano wrote: > In my wishlist for Python 4000 I have: and sooner would be nicer. > * list.index renamed to list.find > * str.find and list.find return None if the argument is not found +1 ...and forget throwing an exception if not found. None is much preferred. From jeanpierreda at gmail.com Wed Jul 20 16:47:53 2011 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Wed, 20 Jul 2011 10:47:53 -0400 Subject: [Python-ideas] Make compile('1\n2\n', '', 'single') raise an exception instead of silently truncating? Message-ID: compile('1\n2\n', '','single') == compile('1\n', '','single'). That is, it ignores the second statement ('2\n'), without offering a way for the caller to detect this. Considering that 'single' is primarily used to emulate the behaviour of the Python interpreter, most of the time, giving it multiple statements is an impossibility, and so that case doesn't matter and could raise an exception without affecting existing code. For example, the code module meets this description, as do debuggers and such. However, in cases where it _is_ possible to give the compiler multiple statements, the user should be warned that his input isn't valid, somehow. For example, the following doctest will mysteriously fail, because it was written incorrectly (doctest uses 'single'): >>> import sys ... sys.stdout.write('foo\n') foo This is because the second statement in the doctest was silently discarded by compile(). It might not always be clear to users how to fix this, and I think this kind of non-obvious error would exist in any use of 'single' that can in theory involve multiple statements, through user error or program bug. So I'd appreciate it if compile() raised an exception in this case. Perhaps SyntaxError or ValueError. Devin From svenrahmann at googlemail.com Wed Jul 20 22:48:23 2011 From: svenrahmann at googlemail.com (Sven Rahmann) Date: Wed, 20 Jul 2011 22:48:23 +0200 Subject: [Python-ideas] Specify number of items to allocate for array.array() constructor Message-ID: At the moment, the array module of the standard library allows to create arrays of different numeric types and to initialize them from an iterable (eg, another array). What's missing is the possiblity to specify the final size of the array (number of items), especially for large arrays. I'm thinking of suffix arrays (a text indexing data structure) for large texts, eg the human genome and its reverse complement (about 6 billion characters from the alphabet ACGT). The suffix array is a long int array of the same size (8 bytes per number, so it occupies about 48 GB memory). At the moment I am extending an array in chunks of several million items at a time at a time, which is slow and not elegant. The function below also initializes each item in the array to a given value (0 by default). Is there a reason why there the array.array constructor does not allow to simply specify the number of items that should be allocated? (I do not really care about the contents.) Would this be a worthwhile addition to / modification of the array module? My suggestions is to modify array generation in such a way that you could pass an iterator (as now) as second argument, but if you pass a single integer value, it should be treated as the number of items to allocate. Here is my current workaround (which is slow): def filled_array(typecode, n, value=0, bsize=(1<<22)): """returns a new array with given typecode (eg, "l" for long int, as in the array module) with n entries, initialized to the given value (default 0) """ a = array.array(typecode, [value]*bsize) x = array.array(typecode) r = n while r >= bsize: x.extend(a) r -= bsize x.extend([value]*r) return x From phd at phdru.name Wed Jul 20 22:58:48 2011 From: phd at phdru.name (Oleg Broytman) Date: Thu, 21 Jul 2011 00:58:48 +0400 Subject: [Python-ideas] Specify number of items to allocate for array.array() constructor In-Reply-To: References: Message-ID: <20110720205848.GA8208@iskra.aviel.ru> On Wed, Jul 20, 2011 at 10:48:23PM +0200, Sven Rahmann wrote: > I'm thinking of suffix arrays (a text indexing data structure) for > large texts, eg the human genome and its reverse complement (about 6 > billion characters from the alphabet ACGT). > The suffix array is a long int array of the same size (8 bytes per > number, so it occupies about 48 GB memory). I doubt array.array was designed to handle data of such size. Why not to use bsddb or such? Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From ron3200 at gmail.com Wed Jul 20 19:55:13 2011 From: ron3200 at gmail.com (ron3200) Date: Wed, 20 Jul 2011 12:55:13 -0500 Subject: [Python-ideas] Deprecate str.find In-Reply-To: References: Message-ID: <1311184513.6805.26.camel@Gutsy> On Tue, 2011-07-19 at 23:51 -0700, Larry Bugbee wrote: > > Steven D'Aprano wrote: > > In my wishlist for Python 4000 I have: > > and sooner would be nicer. > > > * list.index renamed to list.find > > * str.find and list.find return None if the argument is not found > > +1 > > ...and forget throwing an exception if not found. None is much preferred. This is my preference also. I also prefer that the ValueError exception not be overused in the library. A good test might be how to write a function that finds a substring in a substring from a larger string. Cheers, Ron -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Thu Jul 21 00:02:37 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 21 Jul 2011 00:02:37 +0200 Subject: [Python-ideas] Specify number of items to allocate for array.array() constructor References: Message-ID: <20110721000237.1ee25228@pitrou.net> Hello, > Is there a reason why there the array.array constructor does not allow > to simply specify the number of items that should be allocated? (I do > not really care about the contents.) > Would this be a worthwhile addition to / modification of the array module? I think it would be. Rather than an additional constructor parameter, perhaps it could be a separate method (like we already have extend(), fromfile(), etc.). In the meantime, on 3.x you should be able to use the following trick, at least under modern Linux / Unix systems: >>> f = open("/dev/zero", "rb") >>> m = mmap.mmap(f.fileno(), 2*1024*1024*1024, access=mmap.ACCESS_READ) # I'm putting 2GB for the sake of example. Since the mmap is # created over /dev/zero, kernel optimizations should avoid any # physical RAM consumption (virtual memory aka address space # will still be used, of course). >>> a = array.array("i") >>> a.frombytes(m) >>> len(a) 536870912 >>> a[0] 0 >>> a[-1] 0 >>> m.close(); f.close() # This releases any virtual memory used by the mmap object Regards Antoine. From alex.gaynor at gmail.com Thu Jul 21 00:36:14 2011 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Wed, 20 Jul 2011 22:36:14 +0000 (UTC) Subject: [Python-ideas] =?utf-8?q?Specify_number_of_items_to_allocate_for?= =?utf-8?q?=09array=2Earray=28=29_constructor?= References: Message-ID: I suggest array.array(typecode, [0]) * N would be cool if it was a param to the constructor though (generated a 0'd array I suppose), as python's which have GC's that allocate pre-zero'd memory could optimize it nicely ;) Alex From pjenvey at underboss.org Thu Jul 21 02:08:23 2011 From: pjenvey at underboss.org (Philip Jenvey) Date: Wed, 20 Jul 2011 17:08:23 -0700 Subject: [Python-ideas] Specify number of items to allocate for array.array() constructor In-Reply-To: <20110721000237.1ee25228@pitrou.net> References: <20110721000237.1ee25228@pitrou.net> Message-ID: <492B4DBB-F0E7-41C7-BF17-2252D2CF5DE2@underboss.org> On Jul 20, 2011, at 3:02 PM, Antoine Pitrou wrote: > > Hello, > >> Is there a reason why there the array.array constructor does not allow >> to simply specify the number of items that should be allocated? (I do >> not really care about the contents.) >> Would this be a worthwhile addition to / modification of the array module? > > I think it would be. Rather than an additional constructor parameter, > perhaps it could be a separate method (like we already have extend(), > fromfile(), etc.). +1, Jython's had a custom function for this forever, originally from its old jarray module: jarray.zeros(length, typecode). I think its swapped order of args are due to jarray probably predating the array module. -- Philip Jenvey From alex.gaynor at gmail.com Fri Jul 22 20:27:44 2011 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Fri, 22 Jul 2011 18:27:44 +0000 (UTC) Subject: [Python-ideas] =?utf-8?q?Specify_number_of_items_to_allocate_for?= =?utf-8?q?=09array=2Earray=28=29_constructor?= References: <20110721000237.1ee25228@pitrou.net> <492B4DBB-F0E7-41C7-BF17-2252D2CF5DE2@underboss.org> Message-ID: Philip Jenvey writes: > +1, Jython's had a custom function for this forever, originally from its old jarray module: > jarray.zeros(length, typecode). I think its swapped order of args are due to jarray probably predating > the array module. > > -- > Philip Jenvey > array.zeroes(typecode, length) seems like a good API to me. It's also consistant with the name for the numpy method which does the same. I'll go ahead and file a bug for this and try to work up a patch. Alex From g.brandl at gmx.net Fri Jul 22 21:04:29 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 22 Jul 2011 21:04:29 +0200 Subject: [Python-ideas] Specify number of items to allocate for array.array() constructor In-Reply-To: References: <20110721000237.1ee25228@pitrou.net> <492B4DBB-F0E7-41C7-BF17-2252D2CF5DE2@underboss.org> Message-ID: Am 22.07.2011 20:27, schrieb Alex Gaynor: > Philip Jenvey writes: >> +1, Jython's had a custom function for this forever, originally from its old > jarray module: >> jarray.zeros(length, typecode). I think its swapped order of args are due to > jarray probably predating >> the array module. >> >> -- >> Philip Jenvey >> > > > array.zeroes(typecode, length) seems like a good API to me. It's also > consistant with the name for the numpy method which does the same. Would have to be array.zeros() then :) Georg From herman at swebpage.com Sun Jul 24 16:04:53 2011 From: herman at swebpage.com (Herman Sheremetyev) Date: Sun, 24 Jul 2011 23:04:53 +0900 Subject: [Python-ideas] anonymous object support Message-ID: It is currently somewhat difficult/awkward to create arbitrary "anonymous" objects in Python. For example, to make some object that simply has a foo() method you have to declare a class that defines foo() and instantiate it: class Foo: def foo(self, x): return x obj = Foo() It would be a bit easier if Python allowed attributes to be set on instances of builtins the way it does for user-defined classes. This would at least allow something like: obj = object() obj.foo = lambda x: x Alas, that is not possible either as the assignment raises an AttributeError when dealing with instances of builtins. I would like to propose a different solution: alter object.__new__() to support keyword arguments and set the attributes named in the keyword arguments to their requested value. This would get us code like: obj = object(foo=1, bar=lambda x: x) obj.foo >>> 1 obj.bar(2) >>> 2 Python implementation of __new__ would look something like the following, though I suppose this would need to be done at the C level in reality: def __new__(typ, **kwargs): obj = object.__new__(typ) for attr, value in kwargs.items(): setattr(obj, attr, value) return obj So why do we need this? Primarily, this sort of thing is very common in tests where it is often useful to generate anonymous objects with default values. As a result, a number of libraries have risen to fill this need which is a pretty strong indicator that it is a useful pattern, and everyone using one of a dozen different libraries to accomplish the same basic task does not seem very Pythonic. I could also see this being quite useful in quick prototyping scenarios in order to avoid using dicts when an object is more suitable without requiring the overhead of defining a class. I'm curious if people on this list would be interested in seeing support for this added to the language, or if there are some good reasons I have overlooked for why this sort of change might be undesirable. Cheers, -Herman From solipsis at pitrou.net Sun Jul 24 16:30:55 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 24 Jul 2011 16:30:55 +0200 Subject: [Python-ideas] anonymous object support References: Message-ID: <20110724163055.532a951e@pitrou.net> On Sun, 24 Jul 2011 23:04:53 +0900 Herman Sheremetyev wrote: > It is currently somewhat difficult/awkward to create arbitrary > "anonymous" objects in Python. For example, to make some object that > simply has a foo() method you have to declare a class that defines > foo() and instantiate it: > > class Foo: > def foo(self, x): return x > obj = Foo() > [...] > > obj = object() > obj.foo = lambda x: x Well, really, you're saving *one* line of code all the while making things more obscure (if "obj" gets printed out for whatever reason, you won't ever know it's a Foo or has a foo method). > I would like to propose a different solution: alter object.__new__() > to support keyword arguments and set the attributes named in the > keyword arguments to their requested value. This would get us code > like: > > obj = object(foo=1, bar=lambda x: x) > obj.foo > >>> 1 > obj.bar(2) > >>> 2 If that's all you need, you just have to write a convenience function that you put in some utilities module: class AnonymousObject: pass def make(**kwargs): obj = AnonymousObject() for k, v in kwargs.items(): setattr(obj, k, v) return obj and then: >>> from myutils import make >>> obj = make(foo=1, bar=lambda x: x) >>> obj.foo 1 >>> obj.bar(1) 1 Or you can also create a namedtuple class, if there's a well-defined set of attributes your instance(s) will have. Regards Antoine. From steve at pearwood.info Sun Jul 24 16:49:21 2011 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 25 Jul 2011 00:49:21 +1000 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: <4E2C30F1.7060304@pearwood.info> Herman Sheremetyev wrote: > It is currently somewhat difficult/awkward to create arbitrary > "anonymous" objects in Python. For example, to make some object that > simply has a foo() method you have to declare a class that defines > foo() and instantiate it: [...] I'm not sure that "anonymous" is the right word here. Normally when we talk about "anonymous", it is in reference to functions and classes, which have a name given them at creation time. In that sense, early all objects in Python are anonymous: few objects have a __name__ attribute. It seems to me that the name, or lack thereof, of the object is irrelevant to what you propose. Essentially, you want a one-liner for the pattern: class MyObject(object): pass obj = MyObject() obj.spam = "spam spam spam" > So why do we need this? Primarily, this sort of thing is very common > in tests where it is often useful to generate anonymous objects with > default values. As a result, a number of libraries have risen to fill > this need which is a pretty strong indicator that it is a useful > pattern, and everyone using one of a dozen different libraries to > accomplish the same basic task does not seem very Pythonic. I could > also see this being quite useful in quick prototyping scenarios in > order to avoid using dicts when an object is more suitable without > requiring the overhead of defining a class. It might be more convincing if you could give examples of those libraries. > I'm curious if people on this list would be interested in seeing > support for this added to the language, or if there are some good > reasons I have overlooked for why this sort of change might be > undesirable. Against the idea, it's a very simple helper class that you can add to your library or application: class MyObject(object): def __init__(self, **kwargs): self.__dict__.update(kwargs) Three lines, and now you can use it as a one-liner any time you want it: obj = MyObject(flag=True, count=1) In favour of the idea of making it built-in, or at least in the standard library: if this pattern is as common as you say, perhaps there should be a standard solution for it, instead of a plethora of almost, but not quite identical versions. After all, that is one of the main reasons we have bool as a built-in. Also in favour: the implementation given may be slightly *too* simple. It doesn't have a nice repr(). There's no validation of the keyword arguments (although I'm not entirely sure they need validating). The more code needed to make it "professional", the better the argument for doing it once, right, in the standard library. I don't think this needs to be object itself. Doing so probably will cause backwards incompatibility issues -- I'm sure that there will be code out there that expects object(attr="spam") to fail. Better to make a new type. +0.5 on a new type for this, not necessarily a built-in. -1 on changing object to behave this way. -- Steven From guido at python.org Sun Jul 24 17:06:17 2011 From: guido at python.org (Guido van Rossum) Date: Sun, 24 Jul 2011 08:06:17 -0700 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: On Sun, Jul 24, 2011 at 7:04 AM, Herman Sheremetyev wrote: > It is currently somewhat difficult/awkward to create arbitrary > "anonymous" objects in Python. It looks like you are trying to duplicate Java's anonymous classes. Please don't try to emulate Java's work-arounds for Java's deficiencies. -- --Guido van Rossum (python.org/~guido) From herman at swebpage.com Sun Jul 24 17:35:30 2011 From: herman at swebpage.com (Herman Sheremetyev) Date: Mon, 25 Jul 2011 00:35:30 +0900 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: On Mon, Jul 25, 2011 at 12:06 AM, Guido van Rossum wrote: > On Sun, Jul 24, 2011 at 7:04 AM, Herman Sheremetyev wrote: >> It is currently somewhat difficult/awkward to create arbitrary >> "anonymous" objects in Python. > > It looks like you are trying to duplicate Java's anonymous classes. > Please don't try to emulate Java's work-arounds for Java's > deficiencies. Ouch, and I don't even know Java well enough to deserve that ;) I think a more accurate characterization would be that I'm trying to emulate Javascript's object literal notation, but I was trying to avoid parallels with other languages in my proposal. FWIW, I think it would be fine if this was a convenience function in the standard library as suggested above. But it seems so basic that having it supported in in the object() constructor would be the most natural place rather than tucking it away in a library. -Herman From guido at python.org Sun Jul 24 18:03:37 2011 From: guido at python.org (Guido van Rossum) Date: Sun, 24 Jul 2011 09:03:37 -0700 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: On Sun, Jul 24, 2011 at 8:35 AM, Herman Sheremetyev wrote: > On Mon, Jul 25, 2011 at 12:06 AM, Guido van Rossum wrote: >> On Sun, Jul 24, 2011 at 7:04 AM, Herman Sheremetyev wrote: >>> It is currently somewhat difficult/awkward to create arbitrary >>> "anonymous" objects in Python. >> >> It looks like you are trying to duplicate Java's anonymous classes. >> Please don't try to emulate Java's work-arounds for Java's >> deficiencies. > > Ouch, and I don't even know Java well enough to deserve that ;) Ah, sorry. > I think a more accurate characterization would be that I'm trying to > emulate Javascript's object literal notation, but I was trying to > avoid parallels with other languages in my proposal. Still, IMO Javascript's objects suffer from a fatal confusion between being dicts and objects, so I still "object" to the idea. :-) > FWIW, I think it would be fine if this was a convenience function in > the standard library as suggested above. But it seems so basic that > having it supported in in the object() constructor would be the most > natural place rather than tucking it away in a library. Python's object is supposed to have only the minimal functionality. Its instances don't even have a __dict__. Also, every one believes there own favorite feature to be so important that it's worth changing the language, but they don't want to grant that to anybody else's favorite feature... :-) -- --Guido van Rossum (python.org/~guido) From masklinn at masklinn.net Sun Jul 24 18:07:03 2011 From: masklinn at masklinn.net (Masklinn) Date: Sun, 24 Jul 2011 18:07:03 +0200 Subject: [Python-ideas] anonymous object support In-Reply-To: <20110724163055.532a951e@pitrou.net> References: <20110724163055.532a951e@pitrou.net> Message-ID: On 2011-07-24, at 16:30 , Antoine Pitrou wrote: > If that's all you need, you just have to write a convenience function > that you put in some utilities module: > > class AnonymousObject: > pass > > def make(**kwargs): > obj = AnonymousObject() > for k, v in kwargs.items(): > setattr(obj, k, v) > return obj > > > and then: > >>>> from myutils import make >>>> obj = make(foo=1, bar=lambda x: x) >>>> obj.foo > 1 >>>> obj.bar(1) > 1 An other option would be to create an anonymous type on the fly to get methods as well (without having to get a hold on the instancemethod constructor): >>> def make(**kwargs): >>> return type('', (), kwargs)() >>> obj = make(foo=1, bar=lambda self, x: x) >>> obj.foo 1 >>> obj.bar(2) 2 although it makes attributes provided via kwargs into class attributes (which probably does not matter since the class is unique to the object) From raymond.hettinger at gmail.com Sun Jul 24 21:28:51 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Sun, 24 Jul 2011 12:28:51 -0700 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: > Also, every one believes there own favorite feature to be so important > that it's worth changing the language, but they don't want to grant > that to anybody else's favorite feature... :-) > > -- > --Guido van Rossum (python.org/~guido) QOTD :-) From ncoghlan at gmail.com Mon Jul 25 00:47:55 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 25 Jul 2011 08:47:55 +1000 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: On Mon, Jul 25, 2011 at 2:03 AM, Guido van Rossum wrote: > Also, every one believes there own favorite feature to be so important > that it's worth changing the language, but they don't want to grant > that to anybody else's favorite feature... :-) We've actually been down the 'namespace object' road years ago (Steve Bethard even wrote a proto-PEP IIRC) and it suffered the same fate as most enum PEPs: everyone has slightly different ideas on what should be supported, so you end up being faced with one of two options: 1. Ignore some of the use cases (so some users still have to do their own thing) 2. Support all of the use cases by increasing the API complexity (so many users will still do their own thing instead of learning the new API) In this space, collections.namedtuple is a well-executed variant of the first alternative. There are some use cases it doesn't handle, but it covers many of them and does it well. For the OP's use case, I'll simply note that this functionality is already provided by the type builtin: obj = type('Foo', (), dict(foo=(lambda x: x)))() Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From greg.ewing at canterbury.ac.nz Mon Jul 25 01:48:47 2011 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 25 Jul 2011 11:48:47 +1200 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: <4E2CAF5F.9050603@canterbury.ac.nz> You could use a class decorator to help with this. def instance(cls): return cls() @instance class foo(object): x = 42 -- Greg From jeanpierreda at gmail.com Mon Jul 25 02:14:02 2011 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Sun, 24 Jul 2011 20:14:02 -0400 Subject: [Python-ideas] anonymous object support In-Reply-To: <4E2CAF5F.9050603@canterbury.ac.nz> References: <4E2CAF5F.9050603@canterbury.ac.nz> Message-ID: @apply Or is that too evil? Devin On Sun, Jul 24, 2011 at 7:48 PM, Greg Ewing wrote: > You could use a class decorator to help with this. > > def instance(cls): > ? ?return cls() > > @instance > class foo(object): > ?x = 42 > > -- > Greg > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From guido at python.org Mon Jul 25 02:33:46 2011 From: guido at python.org (Guido van Rossum) Date: Sun, 24 Jul 2011 17:33:46 -0700 Subject: [Python-ideas] Enums Message-ID: On Sun, Jul 24, 2011 at 3:47 PM, Nick Coghlan wrote: > We've actually been down the 'namespace object' road years ago (Steve > Bethard even wrote a proto-PEP IIRC) and it suffered the same fate as > most enum PEPs: everyone has slightly different ideas on what should > be supported, so you end up being faced with one of two options: > 1. Ignore some of the use cases (so some users still have to do their own thing) > 2. Support all of the use cases by increasing the API complexity (so > many users will still do their own thing instead of learning the new > API) For enums, I think we should just pick a solution. I'm in favor of Barry Warsaw's version, flufl.enum. -- --Guido van Rossum (python.org/~guido) From pyideas at rebertia.com Mon Jul 25 03:01:18 2011 From: pyideas at rebertia.com (Chris Rebert) Date: Sun, 24 Jul 2011 18:01:18 -0700 Subject: [Python-ideas] anonymous object support In-Reply-To: References: <4E2CAF5F.9050603@canterbury.ac.nz> Message-ID: > On Sun, Jul 24, 2011 at 7:48 PM, Greg Ewing wrote: >> You could use a class decorator to help with this. >> >> def instance(cls): >> ? ?return cls() >> >> @instance >> class foo(object): >> ?x = 42 On Sun, Jul 24, 2011 at 5:14 PM, Devin Jeanpierre wrote: > @apply > > Or is that too evil? Apparently so much so that Python 3 eliminated it. Cheers, Chris From max at mxcrypt.com Mon Jul 25 03:06:02 2011 From: max at mxcrypt.com (Maxim Khitrov) Date: Sun, 24 Jul 2011 21:06:02 -0400 Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library Message-ID: Hello python-ideas, My most recent project lead me down a path that eventually ended up at a new implementation of imaplib based on [RFC-3501]. Although I started the project by gradually adding functionality to the existing IMAP4 library, some of the features that I required simply could not be merged in (without breaking everything). As a result, I wrote my own version of the library, which incorporates all existing functionality of imaplib and includes many of my own improvements. I wrote a PEP-style readme file that describes all the details of why the library was written and how it works, which is available from my mercurial repository: http://hg.mxcrypt.com/python/imaplib2/raw-file/tip/README The same repository also contains the library code and an example script that you can run if you have access to an IMAP4 server: http://hg.mxcrypt.com/python/imaplib2/ Is there any interest in adding my code to a future version of Python 3.x standard library? - Max From jackdied at gmail.com Mon Jul 25 03:19:48 2011 From: jackdied at gmail.com (Jack Diederich) Date: Sun, 24 Jul 2011 21:19:48 -0400 Subject: [Python-ideas] Enums In-Reply-To: References: Message-ID: On Sun, Jul 24, 2011 at 8:33 PM, Guido van Rossum wrote: > For enums, I think we should just pick a solution. I'm in favor of > Barry Warsaw's version, flufl.enum. I don't care for enums but enough other people do that I wouldn't mind including a blessed implemenation. Warsaw's version is better than all the other enum ad-hockery I've seen. +0 -Jack From ben+python at benfinney.id.au Mon Jul 25 04:09:32 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 25 Jul 2011 12:09:32 +1000 Subject: [Python-ideas] Enums References: Message-ID: <87r55fqffn.fsf@benfinney.id.au> Guido van Rossum writes: > For enums, I think we should just pick a solution. I'm in favor of > Barry Warsaw's version, flufl.enum. As the maintainer of the PyPI ?enum? package, I also support the FLUFL's ?flufl.enum? to be the One Obvious Way. -- \ ?Odious ideas are not entitled to hide from criticism behind | `\ the human shield of their believers' feelings.? ?Richard | _o__) Stallman | Ben Finney From herman at swebpage.com Mon Jul 25 05:03:04 2011 From: herman at swebpage.com (Herman Sheremetyev) Date: Mon, 25 Jul 2011 12:03:04 +0900 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: On Mon, Jul 25, 2011 at 1:03 AM, Guido van Rossum wrote: > On Sun, Jul 24, 2011 at 8:35 AM, Herman Sheremetyev wrote: >> On Mon, Jul 25, 2011 at 12:06 AM, Guido van Rossum wrote: >>> On Sun, Jul 24, 2011 at 7:04 AM, Herman Sheremetyev wrote: >>>> It is currently somewhat difficult/awkward to create arbitrary >>>> "anonymous" objects in Python. >>> >>> It looks like you are trying to duplicate Java's anonymous classes. >>> Please don't try to emulate Java's work-arounds for Java's >>> deficiencies. >> >> Ouch, and I don't even know Java well enough to deserve that ;) > > Ah, sorry. > >> I think a more accurate characterization would be that I'm trying to >> emulate Javascript's object literal notation, but I was trying to >> avoid parallels with other languages in my proposal. > > Still, IMO Javascript's objects suffer from a fatal confusion between > being dicts and objects, so I still "object" to the idea. :-) > >> FWIW, I think it would be fine if this was a convenience function in >> the standard library as suggested above. But it seems so basic that >> having it supported in in the object() constructor would be the most >> natural place rather than tucking it away in a library. > > Python's object is supposed to have only the minimal functionality. > Its instances don't even have a __dict__. What if there was a base "dummy" class that would allow this kind of behavior as Steven suggested? My choice of object was simply because it seems like the logical choice among the builtins and to get a discussion going about the idea of creating objects as an expression rather than statement, without resorting to defining a new class just for that purpose. For a short list of libraries that do this sort of thing (not all of them support expression syntax) take a look at the "Simple fake object" section in: http://garybernhardt.github.com/python-mock-comparison/ Disclaimer: I'm the author of the flexmock library listed there. There are also probably another half dozen libraries no longer being maintained that are not listed there but have similar functionality that can be found here: http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy#MockTestingTools As well as a couple of new ones, like Chai, that aren't listed in the testing taxonomy but solve the same problem. Cheers, -Herman From raymond.hettinger at gmail.com Mon Jul 25 06:22:34 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Sun, 24 Jul 2011 21:22:34 -0700 Subject: [Python-ideas] Enums In-Reply-To: References: Message-ID: <23AEE43B-051A-4B96-BF37-733B1806D4F3@gmail.com> On Jul 24, 2011, at 6:19 PM, Jack Diederich wrote: > On Sun, Jul 24, 2011 at 8:33 PM, Guido van Rossum wrote: >> For enums, I think we should just pick a solution. I'm in favor of >> Barry Warsaw's version, flufl.enum. > > I don't care for enums but enough other people do that I wouldn't mind > including a blessed implemenation. I also don't care for enums and recommend against adding them to the language. IMO, it is something that makes good sense in statically compiled languages and is unnecessary for us. Not to mention, we already have several ways to do it (module and class namespaces for example). Also, when this idea came up in the past, it tended to get shot down because the various use cases suggested differing implementations with different features. I also urge caution because Python has already stopped being a small language. Enums are especially problematic because they will pop up everywhere. You won't have the option of ignoring them. Python has achieved an amazing adoption rate without enums. Most people just don't need them. Many, large and clean apps have been built without them. Those that have found did were typically able to implement them easily with the existing toolset. Raymond From guido at python.org Mon Jul 25 07:10:39 2011 From: guido at python.org (Guido van Rossum) Date: Sun, 24 Jul 2011 22:10:39 -0700 Subject: [Python-ideas] Enums In-Reply-To: <23AEE43B-051A-4B96-BF37-733B1806D4F3@gmail.com> References: <23AEE43B-051A-4B96-BF37-733B1806D4F3@gmail.com> Message-ID: On Sun, Jul 24, 2011 at 9:22 PM, Raymond Hettinger wrote: > > On Jul 24, 2011, at 6:19 PM, Jack Diederich wrote: > >> On Sun, Jul 24, 2011 at 8:33 PM, Guido van Rossum wrote: >>> For enums, I think we should just pick a solution. I'm in favor of >>> Barry Warsaw's version, flufl.enum. >> >> I don't care for enums but enough other people do that I wouldn't mind >> including a blessed implemenation. > > I also don't care for enums and recommend against adding them to the language. > IMO, it is something that makes good sense in statically compiled languages > and is unnecessary for us. ?Not to mention, we already have several ways to do it > (module and class namespaces for example). > > Also, when this idea came up in the past, it tended to get shot down because > the various use cases suggested differing implementations with different > features. > > I also urge caution because Python has already stopped being a small language. > Enums are especially problematic because they will pop up everywhere. > You won't have the option of ignoring them. > > Python has achieved an amazing adoption rate without enums. > Most people just don't need them. ?Many, large and clean apps have > been built without them. ?Those that have found did were typically > able to implement them easily with the existing toolset. Don't worry about them being added to the language; it's just a stdlib module I'm proposing here. They do have their uses and fans, or there wouldn't be a PEP and several popular 3rd party libraries. My own reason for sometimes wanting them is that it gets old to use a lookup table from name -> string for printing, and using string values instead of integers isn't always an option. Google's protobuf also has them, as does protorpc, a Pythonic protobuf spinoff by my office mate Rafe Kaplan. -- --Guido van Rossum (python.org/~guido) From stefan_ml at behnel.de Mon Jul 25 08:04:45 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 25 Jul 2011 08:04:45 +0200 Subject: [Python-ideas] Enums In-Reply-To: <87r55fqffn.fsf@benfinney.id.au> References: <87r55fqffn.fsf@benfinney.id.au> Message-ID: Ben Finney, 25.07.2011 04:09: > Guido van Rossum writes: > >> For enums, I think we should just pick a solution. I'm in favor of >> Barry Warsaw's version, flufl.enum. > > As the maintainer of the PyPI ?enum? package, I also support the FLUFL's > ?flufl.enum? to be the One Obvious Way. Never used enums in Python, and likely won't do so anywhere in the near future, but I just read through the usage page at http://packages.python.org/flufl.enum/docs/using.html and *if* such a library gets added to the stdlib, then this is the way it should work. The examples totally make sense to me. Stefan From ethan at stoneleaf.us Mon Jul 25 08:13:40 2011 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 24 Jul 2011 23:13:40 -0700 Subject: [Python-ideas] Enums In-Reply-To: References: <87r55fqffn.fsf@benfinney.id.au> Message-ID: <4E2D0994.5020609@stoneleaf.us> Stefan Behnel wrote: > Ben Finney, 25.07.2011 04:09: >> Guido van Rossum writes: >> >>> For enums, I think we should just pick a solution. I'm in favor of >>> Barry Warsaw's version, flufl.enum. >> >> As the maintainer of the PyPI ?enum? package, I also support the FLUFL's >> ?flufl.enum? to be the One Obvious Way. > > Never used enums in Python, and likely won't do so anywhere in the near > future, but I just read through the usage page at > > http://packages.python.org/flufl.enum/docs/using.html > > and *if* such a library gets added to the stdlib, then this is the way > it should work. The examples totally make sense to me. Agreed. +1 ~Ethan~ From pyideas at rebertia.com Mon Jul 25 08:32:19 2011 From: pyideas at rebertia.com (Chris Rebert) Date: Sun, 24 Jul 2011 23:32:19 -0700 Subject: [Python-ideas] Enums In-Reply-To: References: <87r55fqffn.fsf@benfinney.id.au> Message-ID: On Sun, Jul 24, 2011 at 11:04 PM, Stefan Behnel wrote: > Ben Finney, 25.07.2011 04:09: >> Guido van Rossum writes: >> >>> For enums, I think we should just pick a solution. I'm in favor of >>> Barry Warsaw's version, flufl.enum. >> >> As the maintainer of the PyPI ?enum? package, I also support the FLUFL's >> ?flufl.enum? to be the One Obvious Way. > > Never used enums in Python, and likely won't do so anywhere in the near > future, but I just read through the usage page at > > http://packages.python.org/flufl.enum/docs/using.html > > and *if* such a library gets added to the stdlib, then this is the way it > should work. The examples totally make sense to me. My only nitpicks would be: - Why violate TOOWTDI by having both SomeEnum(...) and SomeEnum[...] that do the same thing? - Why is it .enumname and not simply .name? .enumname could also be confused as getting the name of the overarching enum rather than the name of the particular enum value. Cheers, Chris From herman at swebpage.com Mon Jul 25 11:44:57 2011 From: herman at swebpage.com (Herman Sheremetyev) Date: Mon, 25 Jul 2011 18:44:57 +0900 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: On Mon, Jul 25, 2011 at 7:47 AM, Nick Coghlan wrote: > On Mon, Jul 25, 2011 at 2:03 AM, Guido van Rossum wrote: >> Also, every one believes there own favorite feature to be so important >> that it's worth changing the language, but they don't want to grant >> that to anybody else's favorite feature... :-) > > We've actually been down the 'namespace object' road years ago (Steve > Bethard even wrote a proto-PEP IIRC) and it suffered the same fate as > most enum PEPs: everyone has slightly different ideas on what should > be supported, so you end up being faced with one of two options: > 1. Ignore some of the use cases (so some users still have to do their own thing) > 2. Support all of the use cases by increasing the API complexity (so > many users will still do their own thing instead of learning the new > API) > > In this space, collections.namedtuple is a well-executed variant of > the first alternative. There are some use cases it doesn't handle, but > it covers many of them and does it well. > > For the OP's use case, I'll simply note that this functionality is > already provided by the type builtin: > > obj = type('Foo', (), dict(foo=(lambda x: x)))() This one-liner with type is neat. For the record, it's missing "self" so a working version would be a tad bit longer. Using an empty name and using {} instead of dict() makes it look almost usable: obj = type('', (), {'foo': lambda self, x: x})() I think it would actually *be* usable if the dictionary passed in as the last argument could also be built out of keyword arguments: obj = type('', (), foo=lambda self, x: x)() Giving those first two positional arguments default values (empty string and empty tuple?) would make it even better: obj = type(foo=lambda self, x: x)() That's only one set of parens away from my initial proposal, but I think it's close enough. Would there be any objections to changing the type() API to support this? Cheers, -Herman From herman at swebpage.com Mon Jul 25 11:46:20 2011 From: herman at swebpage.com (Herman Sheremetyev) Date: Mon, 25 Jul 2011 18:46:20 +0900 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: [adding python-ideas back to CC] On Mon, Jul 25, 2011 at 5:40 PM, Paul Moore wrote: > On 25 July 2011 08:41, Herman Sheremetyev wrote: >>> >>> class Dummy: pass >>> >>> It's not really worth including something this short in the stdlib: >>> >>> from somewhere import Dummy >>> >>> is actually longer! >> >> This doesn't actually help with the "create object as an expression" >> requirement. Steven D'Arpano's message describes some implementation >> details, and it would be nice if you didn't have to import it though I >> could live with that as well. > > I'm not quite sure I follow. Are you insisting that you need to create > an instance and then assign an instance attribute/method in a > single expression, and have the resulting instance be the value of > that expression? > > If so, then I don't really see what you gain apart from saving a line > or two in your source code, and confusing your reader. But if you must > do that, then Nick's suggestion of using the type builtin would > probably do what you want. (I make no claims that it's particularly > readable, but I'd say you gave up on that when you insisted on a > one-line solution :-)) > > To compare, we're talking about: > > d = Dummy() > d.foo = lambda x: x > expression_involving(d) > > vs > > expression_involving(Dummy(foo = lambda x:x)) You could do that if it seems reasonable or useful in that particular case, but more often than not I think you'd do something like: variable = Dummy(foo=lambda x: x) expression_involving(variable) Having one line of code that accomplishes a simple task (creating an object and saving it in a variable) instead of having to write N lines for each attribute makes for more readability not less IMO. You can still break lines inside the parens and add each attribute on its own line, but the indentation would at least clue the reader to the fact that it's really a single operation. Having N lines, especially if N is more than 2 or 3, is begging for the whole thing to be moved off into a helper function. And in tests this kind of operation happens all the time. > Even using trivial examples like this, I find the first more readable > (and I'd curse the writer if I had to maintain code like the second). > If expression_involving() was anything more complex than a function > call, or foo was more than the identity function, this would be > utterly unreadable. > > Can you point at any real code that implements a class-making function > such as you describe, and then uses it? I'd be curious to see any > real-world examples to check if my intuition that such code is > difficult to understand holds up. https://github.com/has207/flexmock/blob/master/tests/flexmock_test.py You can take a look at the tests starting at line 56 -- the flexmock() function does exactly what I'm proposing when given keyword args. > PS Feel free to quote any of this mail back on-list. I would have > added the list back into my reply, on the assumption that you simply > forgot to include the list, but didn't want to do so without your > agreement... It wasn't my message that took the list off the CC :) Cheers, -Herman From p.f.moore at gmail.com Mon Jul 25 13:00:41 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 25 Jul 2011 12:00:41 +0100 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: On 25 July 2011 10:46, Herman Sheremetyev wrote: > It wasn't my message that took the list off the CC :) > Oops sorry :-) From lyricconch at gmail.com Mon Jul 25 13:52:45 2011 From: lyricconch at gmail.com (=?UTF-8?B?5rW36Z+1?=) Date: Mon, 25 Jul 2011 19:52:45 +0800 Subject: [Python-ideas] use "as" for block scope support Message-ID: New submission from HaiYun Yan : just like mozilla javascript "let" extension https://developer.mozilla.org/en/new_in_javascript_1.7#Block_scope_with_let_%28Merge_into_let_Statement%29 according function args can have the default value let_stmt is unnecessary for python. ============example 1=========== f = None with open(...) as f: assert hasattr(f, "read") f = 1 assert f == 1 assert f is None ==========example 2============== try: e = None 1/0 except ZeroDivisionError as e: assert isinstance(e, ZeroDivisionError) except ... as e: # the same as except: but has the exception instance store to e assert isinstance(e, BaseException) else: assert e is None finally: assert e is None assert e is None ===========example 3============== while SEQ.next_availd() as tid: assert bool(tid) is True assert "tid" not in dir() ===========example 4============== assert "zero" not in dir() A = (id(None) % 1 as zero) zero + 5 assert A == 5 assert "zero" not in dir() ======example 5================== K = 1 A = (K+1 as K, K+2 as Q) K*Q assert A == 6 # (1+1) * (1+2) assert K==1 ============example 6=============== K = 1 A = (K+1 as K) (K+2 as K) K * K assert A == 16 # ((1+1) + 2) * ((1+1) + 2) assert K == 1 From p.f.moore at gmail.com Mon Jul 25 14:18:20 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 25 Jul 2011 13:18:20 +0100 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: On 25 July 2011 10:46, Herman Sheremetyev wrote: >> Can you point at any real code that implements a class-making function >> such as you describe, and then uses it? I'd be curious to see any >> real-world examples to check if my intuition that such code is >> difficult to understand holds up. > > https://github.com/has207/flexmock/blob/master/tests/flexmock_test.py > > You can take a look at the tests starting at line 56 -- the flexmock() > function does exactly what I'm proposing when given keyword args. Thanks. That code does look nice and clean, although it is once again using artificially simple attributes (although that's not surprising in test code). But I'm still puzzled as to what you're asking for. Clearly (I assume) you're not asking for flexmock to be added to the stdlib. So are you asking for something like flexmock - in which case, what are the use cases for it (that aren't use cases for flexmock itself)? Sorry if I'm being dense here. If others understand the request feel free to drop this conversation as I'm not sure I'm adding much that I haven't already said. >From my point of view, I see 3 separate aspects: 1. Specialised functions like flexmock, which fill a particular niche, and which quite reasonably hide the complexity of creating "anonymous" objects behind a simple API. No problem here, these are clearly useful, but they don't need to go into the stdlib unless their particular focus warrants it. 2. A generic "create object with dynamically assigned attributes" function. I haven't seen any use cases for this which aren't rare enough that a multi-line answer, or Nick's type() workaround, aren't sufficient. 3. Some functionality to make writing flexmock-like functions a bit easier. Looking at the implementation of flexmock (specifically the Mock class) I don't see any obvious place it would make sense to use such a thing. You couldn't use type() to simplify Mock.__init__ as far as I can see, and clearly type() is far too lightweight to replace Mock completely. So again, I see no clear use case here. But if this boils down to "I wouldn't use the proposed function myself", then so be it. I'm happy to leave the people who *would* use it to thrash out the details and the benefits. Paul. From herman at swebpage.com Mon Jul 25 16:13:52 2011 From: herman at swebpage.com (Herman Sheremetyev) Date: Mon, 25 Jul 2011 23:13:52 +0900 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: On Mon, Jul 25, 2011 at 9:18 PM, Paul Moore wrote: > On 25 July 2011 10:46, Herman Sheremetyev wrote: >>> Can you point at any real code that implements a class-making function >>> such as you describe, and then uses it? I'd be curious to see any >>> real-world examples to check if my intuition that such code is >>> difficult to understand holds up. >> >> https://github.com/has207/flexmock/blob/master/tests/flexmock_test.py >> >> You can take a look at the tests starting at line 56 -- the flexmock() >> function does exactly what I'm proposing when given keyword args. > > Thanks. That code does look nice and clean, although it is once again > using artificially simple attributes (although that's not surprising > in test code). > > But I'm still puzzled as to what you're asking for. Clearly (I assume) > you're not asking for flexmock to be added to the stdlib. So are you > asking for something like flexmock - in which case, what are the use > cases for it (that aren't use cases for flexmock itself)? No, I'm not asking for flexmock to be added to stdlib :) I'm writing a different library based on my experiences with porting flexmock that has a much cleaner and more Pythonic API that I *might* propose to be added at some point but it's not really at the stage where it's ready for discussion. To answer your question, the use cases are exactly the same as *some* of the uses for flexmock. This is precisely why I've proposed it as I really don't think we need things like flexmock to do this. Flexmock's primary function is creating partial mocks, checking expectations, and ensuring it cleans up after itself. The fact that it allows for creation of objects on the fly is not really what it's for, though it provides that functionality, like most of the other mocking libraries, simply because it's missing and is such a common pattern in tests that a testing library would not really be complete unless it provided it. > Sorry if I'm being dense here. If others understand the request feel > free to drop this conversation as I'm not sure I'm adding much that I > haven't already said. > > From my point of view, I see 3 separate aspects: > > 1. Specialised functions like flexmock, which fill a particular niche, > and which quite reasonably hide the complexity of creating "anonymous" > objects behind a simple API. No problem here, these are clearly > useful, but they don't need to go into the stdlib unless their > particular focus warrants it. > > 2. A generic "create object with dynamically assigned attributes" > function. I haven't seen any use cases for this which aren't rare > enough that a multi-line answer, or Nick's type() workaround, aren't > sufficient. Test code == code :) It's more code than "actual" code in many cases, so I don't know what defines these use cases as rare. Tests are a pretty obvious place where this is useful but I don't think it's the only place and being able to create objects in this manner would improve readability in a number of places where small, throw-away objects are used. A multi-line solution first requires a user-defined class. And I feel like we really shouldn't have to define a class just to create an object that we throw away two lines later. Using type()'s current totally unwieldy API to do such an extremely simple and common thing is again not really a good answer IMO. I proposed some changes that shouldn't break type()'s current API while making it usable for this particular pattern in my other message. It's not a big change and I'd be happy to implement it and send in a patch if nobody has any strong objections. Cheers, -Herman From eric at trueblade.com Mon Jul 25 16:38:35 2011 From: eric at trueblade.com (Eric V. Smith) Date: Mon, 25 Jul 2011 10:38:35 -0400 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: <4E2D7FEB.6080405@trueblade.com> On 07/25/2011 05:44 AM, Herman Sheremetyev wrote: > Giving those first two positional arguments default values (empty > string and empty tuple?) would make it even better: > > obj = type(foo=lambda self, x: x)() > > That's only one set of parens away from my initial proposal, but I > think it's close enough. Would there be any objections to changing the > type() API to support this? -1 on changing type(). Just add this to your code: >>> def mytype(**kwargs): ... return type('', (), kwargs)() Then you can get rid of the extra parens: >>> obj=mytype(foo=lambda self, x: x) >>> obj.foo(3) 3 Eric. From guido at python.org Mon Jul 25 17:02:28 2011 From: guido at python.org (Guido van Rossum) Date: Mon, 25 Jul 2011 08:02:28 -0700 Subject: [Python-ideas] use "as" for block scope support In-Reply-To: References: Message-ID: I *think* you're proposing that the variable named in the "as" clause should go out of scope when the blog ends. We already do a similar thing with "except ... as ...", but I don't see how we can do this without breaking backwards compatibility. I also don't think there is a particularly good reason to do (we did have a good reason in the case of except). Finally, with has lot of special semantics, you couldn't just use "with 2+2 as four: ...". --Guido On Mon, Jul 25, 2011 at 4:52 AM, ?? wrote: > New submission from HaiYun Yan : > just like mozilla javascript "let" extension > https://developer.mozilla.org/en/new_in_javascript_1.7#Block_scope_with_let_%28Merge_into_let_Statement%29 > according function args can have the default value > let_stmt is unnecessary for python. > ============example 1=========== > f = None > with open(...) as f: > ? ? ?assert hasattr(f, "read") > ? ? ?f = 1 > ? ? ?assert f == 1 > assert f is None > ==========example 2============== > try: > ? ? e = None > ? ? 1/0 > except ZeroDivisionError as e: > ? ? assert isinstance(e, ZeroDivisionError) > except ... as e: > ? ? # the same as except: but has the exception instance store to e > ? ? assert isinstance(e, BaseException) > else: > ? ? assert e is None > finally: > ? ? assert e is None > assert e is None > ===========example 3============== > while SEQ.next_availd() as tid: > ? ? ?assert bool(tid) is True > assert "tid" not in dir() > ===========example 4============== > assert "zero" not in dir() > A = (id(None) % 1 as zero) zero + 5 > assert A == 5 > assert "zero" not in dir() > ======example 5================== > K = 1 > A = (K+1 as K, K+2 as Q) K*Q > assert A == 6 # (1+1) * (1+2) > assert K==1 > ============example 6=============== > K = 1 > A = (K+1 as K) (K+2 as K) K * K > assert A == 16 ?# ((1+1) + 2) * ((1+1) + 2) > assert K == 1 > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- --Guido van Rossum (python.org/~guido) From bruce at leapyear.org Mon Jul 25 17:32:21 2011 From: bruce at leapyear.org (Bruce Leban) Date: Mon, 25 Jul 2011 08:32:21 -0700 Subject: [Python-ideas] use "as" for block scope support In-Reply-To: References: Message-ID: On Mon, Jul 25, 2011 at 4:52 AM, ?? wrote: > ======example 5================== > K = 1 > A = (K+1 as K, K+2 as Q) K*Q > assert A == 6 # (1+1) * (1+2) > assert K==1 > ============example 6=============== > K = 1 > A = (K+1 as K) (K+2 as K) K * K > assert A == 16 # ((1+1) + 2) * ((1+1) + 2) > assert K == 1 > With some headscratching, I can figure out these examples. What I can't figure out is why you would possibly want to write code like this. It certainly doesn't enhance readability, and it wouldn't enhance performance or expressiveness of the language either. If you want to propose improvements to the language it should do at least one of those things. -1 as -1 --- Bruce Follow me: http://www.twitter.com/Vroo http://www.vroospeak.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Mon Jul 25 18:27:23 2011 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 25 Jul 2011 17:27:23 +0100 Subject: [Python-ideas] Enums In-Reply-To: References: <87r55fqffn.fsf@benfinney.id.au> Message-ID: <4E2D996B.2060609@mrabarnett.plus.com> On 25/07/2011 07:32, Chris Rebert wrote: > On Sun, Jul 24, 2011 at 11:04 PM, Stefan Behnel wrote: >> Ben Finney, 25.07.2011 04:09: >>> Guido van Rossum writes: >>> >>>> For enums, I think we should just pick a solution. I'm in favor of >>>> Barry Warsaw's version, flufl.enum. >>> >>> As the maintainer of the PyPI ?enum? package, I also support the FLUFL's >>> ?flufl.enum? to be the One Obvious Way. >> >> Never used enums in Python, and likely won't do so anywhere in the near >> future, but I just read through the usage page at >> >> http://packages.python.org/flufl.enum/docs/using.html >> >> and *if* such a library gets added to the stdlib, then this is the way it >> should work. The examples totally make sense to me. > > My only nitpicks would be: > - Why violate TOOWTDI by having both SomeEnum(...) and SomeEnum[...] > that do the same thing? > - Why is it .enumname and not simply .name? .enumname could also be > confused as getting the name of the overarching enum rather than the > name of the particular enum value. > My nitpick would be that the underlying integer values should start at 0, not 1. From fuzzyman at gmail.com Mon Jul 25 18:46:22 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 25 Jul 2011 17:46:22 +0100 Subject: [Python-ideas] Enums In-Reply-To: References: Message-ID: On 25 July 2011 01:33, Guido van Rossum wrote: > On Sun, Jul 24, 2011 at 3:47 PM, Nick Coghlan wrote: > > We've actually been down the 'namespace object' road years ago (Steve > > Bethard even wrote a proto-PEP IIRC) and it suffered the same fate as > > most enum PEPs: everyone has slightly different ideas on what should > > be supported, so you end up being faced with one of two options: > > 1. Ignore some of the use cases (so some users still have to do their own > thing) > > 2. Support all of the use cases by increasing the API complexity (so > > many users will still do their own thing instead of learning the new > > API) > > For enums, I think we should just pick a solution. I'm in favor of > Barry Warsaw's version, flufl.enum. > I generally like the flufl.enum API. There are two things it doesn't do. For new apis it is *usually* possible to just use strings rather than integers - and have your library do the mapping if an underlying api takes integers. For existing ones, for example many parts of the python standard library, we couldn't replace the integer values with enums without a lot of effort. If the flufl enums subclassed integer then replacing most of the existing constants in the standard library would be trivially easy (so we'd have a built-in use case). e The second use case, where you really do want to use integers rather than strings, is where you have flag constants that you "or" together. For example in the standard library we have these in r, gzip, msilib, some in xml.dom.NodeFilter, plus a bunch of others. It would be nice to add support for or'ing of enums whilst retaining a nice repr. I did collect a whole lot of emails from a thread last year and was hoping to put a pep together. I'd support flufl.enum - but it would be better if it was extended so we could use it in the standard library. All the best, Michael Foord > > -- > --Guido van Rossum (python.org/~guido ) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From janssen at parc.com Mon Jul 25 18:46:28 2011 From: janssen at parc.com (Bill Janssen) Date: Mon, 25 Jul 2011 09:46:28 PDT Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library In-Reply-To: References: Message-ID: <30415.1311612388@parc.com> Maxim Khitrov wrote: > The same repository also contains the library code and an example > script that you can run if you have access to an IMAP4 server: > > http://hg.mxcrypt.com/python/imaplib2/ I took a look at this. Nice work! Impressive list of extensions implemented. The first thing to ask is, does it pass the (minimal) test suite in Lib/test/test_imaplib.py? Or is it too different? The code seems to be not quite in compliance with PEP 8. If you're aiming at the stdlib you should probably fix that. I tend to think one of the larger missing pieces of imap4lib is the lack of a higher-level interface. I've used imap4lib for a couple of projects, and I think that if I hadn't already had the experience of implementing my own IMAP server in Python, I wouldn't have done that; I'd have used a package like getmail instead. There's a lot of experience with IMAP necessary to use imap4lib. Looking at your example.py, that still seems to be the case. One thing you might consider is implementing a subclass of "mailbox.Mailbox" which provides that missing higher-level interface to IMAP4. Bill From lyricconch at gmail.com Mon Jul 25 18:50:07 2011 From: lyricconch at gmail.com (=?UTF-8?B?5rW36Z+1?=) Date: Tue, 26 Jul 2011 00:50:07 +0800 Subject: [Python-ideas] use "as" for block scope support In-Reply-To: References: Message-ID: Dupilcate K is used only for describe the scope rule. Advantage: 1. the same grammer, the same behave "t" of """(x+y as t, x-y as s) t * s + s + t""" is only valid in the expression which require just as "e" of """except ... as e""" only live in the suite which it belongs 2. readability includes laconic a = (x+y as t, x-y as s) t * s + s + t it doesnt break thinking. from left to right, people finish thinking inplace. temp var will not keep in memory, people can forget them since they are not valid outside the expression. everything is clean. t, s = x+y, x-y a = t * s + s + t in this case, thinking is break and temp var keep in memory, people should take some care about it (for example, in nested loop) 3. this enchant the "lambda" syntax list.sort(key = lambda p: (sqrt(p.x**2+p.y**2) as r) r**2+ A*r + B) 2011/7/25 Bruce Leban : > On Mon, Jul 25, 2011 at 4:52 AM, ?? wrote: >> >> ======example 5================== >> K = 1 >> A = (K+1 as K, K+2 as Q) K*Q >> assert A == 6 # (1+1) * (1+2) >> assert K==1 >> ============example 6=============== >> K = 1 >> A = (K+1 as K) (K+2 as K) K * K >> assert A == 16 # ((1+1) + 2) * ((1+1) + 2) >> assert K == 1 > > With some headscratching, I can figure out these examples. What I can't > figure out is why you would possibly want to write code like this. It > certainly doesn't enhance readability, and it wouldn't enhance performance > or expressiveness of the language either. If you want to propose > improvements to the language it should do at least one of those things. > -1 as -1 > --- Bruce > Follow me: http://www.twitter.com/Vroo http://www.vroospeak.com > > From g.brandl at gmx.net Mon Jul 25 18:51:46 2011 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 25 Jul 2011 18:51:46 +0200 Subject: [Python-ideas] anonymous object support In-Reply-To: <4E2D7FEB.6080405@trueblade.com> References: <4E2D7FEB.6080405@trueblade.com> Message-ID: Am 25.07.2011 16:38, schrieb Eric V. Smith: > On 07/25/2011 05:44 AM, Herman Sheremetyev wrote: >> Giving those first two positional arguments default values (empty >> string and empty tuple?) would make it even better: >> >> obj = type(foo=lambda self, x: x)() >> >> That's only one set of parens away from my initial proposal, but I >> think it's close enough. Would there be any objections to changing the >> type() API to support this? > > -1 on changing type(). Just add this to your code: > >>>> def mytype(**kwargs): > ... return type('', (), kwargs)() > > Then you can get rid of the extra parens: > >>>> obj=mytype(foo=lambda self, x: x) >>>> obj.foo(3) > 3 And if you call it anonymous_object() instead of mytype(), you have the advantage of more readable code as well ;) Georg From barry at python.org Mon Jul 25 18:52:26 2011 From: barry at python.org (Barry Warsaw) Date: Mon, 25 Jul 2011 12:52:26 -0400 Subject: [Python-ideas] anonymous object support References: Message-ID: <20110725125226.020bfffc@resist.wooz.org> On Jul 24, 2011, at 09:03 AM, Guido van Rossum wrote: >Also, every one believes there own favorite feature to be so important >that it's worth changing the language, but they don't want to grant >that to anybody else's favorite feature... :-) This reminds me of the poll you conducted at a keynote, many Pycons ago. You first asked "who thinks the language is changing too quickly?" Lots of hands went up. Then you asked (probably paraphrasing) "who has a feature they think needs to go into Python?" Lots of the same hands went up. :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Mon Jul 25 19:06:21 2011 From: barry at python.org (Barry Warsaw) Date: Mon, 25 Jul 2011 13:06:21 -0400 Subject: [Python-ideas] Enums References: Message-ID: <20110725130621.5e1249d8@resist.wooz.org> On Jul 24, 2011, at 05:33 PM, Guido van Rossum wrote: >For enums, I think we should just pick a solution. I'm in favor of >Barry Warsaw's version, flufl.enum. Thanks! I would be quite happy to see the library pulled into the stdlib, and I'd be willing to maintain a backward compatible standalone version for older Pythons. For me, the API has been quite stable. The last thing I added was Michael Foord's make_enum() contribution, and PJE's fix for pickling. I haven't felt a burning need to add anything else really in quite some time. (I'll respond to specific details later in the thread.) I'm sure my contributors agreement would cover this, and I'd be happy to release it under the APLv2.0 for that purpose. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From fuzzyman at gmail.com Mon Jul 25 19:06:56 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 25 Jul 2011 18:06:56 +0100 Subject: [Python-ideas] Enums In-Reply-To: References: Message-ID: On 25 July 2011 17:46, Michael Foord wrote: > > > On 25 July 2011 01:33, Guido van Rossum wrote: > >> On Sun, Jul 24, 2011 at 3:47 PM, Nick Coghlan wrote: >> > We've actually been down the 'namespace object' road years ago (Steve >> > Bethard even wrote a proto-PEP IIRC) and it suffered the same fate as >> > most enum PEPs: everyone has slightly different ideas on what should >> > be supported, so you end up being faced with one of two options: >> > 1. Ignore some of the use cases (so some users still have to do their >> own thing) >> > 2. Support all of the use cases by increasing the API complexity (so >> > many users will still do their own thing instead of learning the new >> > API) >> >> For enums, I think we should just pick a solution. I'm in favor of >> Barry Warsaw's version, flufl.enum. >> > > I generally like the flufl.enum API. There are two things it doesn't do. > > For new apis it is *usually* possible to just use strings rather than > integers - and have your library do the mapping if an underlying api takes > integers. For existing ones, for example many parts of the python standard > library, we couldn't replace the integer values with enums without a lot of > effort. > > >From the thread last year: Python standard library modules currently using integers for constants: * re - has flags (OR'able constants) defined in sre_constants, each flag has two names (e.g. re.IGNORECASE and re.I) * os has SEEK_SET, SEEK_CUR, SEEK_END - **plus** those implemented in posix / nt * doctest has its own flag system, but is really just using integer flags / constants (quite a few of them) * token has a tonne of constants (autogenerated) * socket exports a bunch of constants defined in _socket * gzip has flags: FTEXT, FHCRC, FEXTRA, FNAME, FCOMMENT * errno (builtin module) EALREADY, EINPROGRESS, EWOULDBLOCK, ECONNRESET, EINVAL, ENOTCONN, ESHUTDOWN, EINTR, EISCONN, EBADF, ECONNABORTED * opcode has HAVE_ARGUMENT, EXTENDED_ARG. In fact pretty much the whole of opcode is about defining and exposing named constants * msilib uses flag constants * multiprocessing.pool - RUN, CLOSE, TERMINATE * multiprocessing.util - NOTSET, SUBDEBUG, DEBUG, INFO, SUBWARNING * xml.dom and xml.dom.Node (in __init__.py) have a bunch of constants * xml.dom.NodeFilter.NodeFilter holds a bunch of constants (some of them flags) * xmlrpc.client has a bunch of error constants * calendar uses constants to represent weekdays, plus one for the EPOCH that is best left alone * http.client has a tonne of constants - recognisable as ports / error codes though * dis has flags in COMPILER_FLAG_NAMES, which are then set as locals in inspect * io defines SEEK_SET, SEEK_CUR, SEEK_END (same as os) Where constants are implemented in C but exported via a Python module (the constants exported by os and socket for example) they could be wrapped. Where they are exported directly by a C extension or builtin module (e.g. errno) they are probably best left. Here's an email from Guido from that thread putting his vote in for enums *as* integers: http://mail.python.org/pipermail/python-dev/2010-November/105916.html All the best, Michael Foord > If the flufl enums subclassed integer then replacing most of the existing > constants in the standard library would be trivially easy (so we'd have a > built-in use case). > e > The second use case, where you really do want to use integers rather than > strings, is where you have flag constants that you "or" together. For > example in the standard library we have these in r, gzip, msilib, some in > xml.dom.NodeFilter, plus a bunch of others. > > It would be nice to add support for or'ing of enums whilst retaining a nice > repr. > > I did collect a whole lot of emails from a thread last year and was hoping > to put a pep together. I'd support flufl.enum - but it would be better if it > was extended so we could use it in the standard library. > > All the best, > > Michael Foord > > > >> >> -- >> >> --Guido van Rossum (python.org/~guido ) >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> > > > > -- > > http://www.voidspace.org.uk/ > > May you do good and not evil > May you find forgiveness for yourself and forgive others > > May you share freely, never taking more than you give. > -- the sqlite blessing http://www.sqlite.org/different.html > > > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Mon Jul 25 19:15:53 2011 From: barry at python.org (Barry Warsaw) Date: Mon, 25 Jul 2011 13:15:53 -0400 Subject: [Python-ideas] Enums References: <23AEE43B-051A-4B96-BF37-733B1806D4F3@gmail.com> Message-ID: <20110725131553.7fcb24a5@resist.wooz.org> On Jul 24, 2011, at 09:22 PM, Raymond Hettinger wrote: >I also don't care for enums and recommend against adding them to the >language. IMO, it is something that makes good sense in statically compiled >languages and is unnecessary for us. Not to mention, we already have several >ways to do it (module and class namespaces for example). IMHO, enums improve readability and debugging. For better integration with external systems (e.g. databases), enum values are compatible with integers. And this (Pdb) color is much nicer than (Pdb) color 3 Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From bruce at leapyear.org Mon Jul 25 19:17:08 2011 From: bruce at leapyear.org (Bruce Leban) Date: Mon, 25 Jul 2011 10:17:08 -0700 Subject: [Python-ideas] use "as" for block scope support In-Reply-To: References: Message-ID: On Mon, Jul 25, 2011 at 9:50 AM, ?? wrote: > Dupilcate K is used only for describe the scope rule. > Advantage: > > 1. the same grammer, the same behave > "t" of """(x+y as t, x-y as s) t * s + s + t""" is only valid in the > expression which require > just as > "e" of """except ... as e""" only live in the suite which it belongs > > 2. readability includes laconic > a = (x+y as t, x-y as s) t * s + s + t > it doesnt break thinking. from left to right, people finish thinking > inplace. > temp var will not keep in memory, people can forget them since they > are not valid outside the expression. everything is clean. > t, s = x+y, x-y > a = t * s + s + t > in this case, thinking is break and temp var keep in memory, > people should take some care about it (for example, in nested loop) > (x + y) * (x - y) + 2 * x is IMHO more readable - especially note the absence of y in the third term. > > 3. this enchant the "lambda" syntax > list.sort(key = lambda p: (sqrt(p.x**2+p.y**2) as r) r**2+ A*r + B) > I don't know what "enchant" means here, but you can do this if you want: list.sort(key = lambda p: (lambda r: r**2 + A*r + B) (sqrt(p.x**2+p.y**2))) I'm not recommending that as I think it would be more readable to name that function so that it would be more clear why you're using that function as the sort key. But, to me, this example does not justify new syntax. --- Bruce -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Mon Jul 25 19:23:57 2011 From: barry at python.org (Barry Warsaw) Date: Mon, 25 Jul 2011 13:23:57 -0400 Subject: [Python-ideas] Enums References: <87r55fqffn.fsf@benfinney.id.au> Message-ID: <20110725132357.1168dcc7@resist.wooz.org> On Jul 24, 2011, at 11:32 PM, Chris Rebert wrote: >- Why violate TOOWTDI by having both SomeEnum(...) and SomeEnum[...] >that do the same thing? I don't remember exactly, but there was some historical reasons for including both syntaxes. >- Why is it .enumname and not simply .name? .enumname could also be >confused as getting the name of the overarching enum rather than the >name of the particular enum value. Again, historical accident. Probably .enumname was chosen to mimic .enumclass, since obviously .class couldn't be used. Since these are attributes on enumeration values, .name and .enum might have been better choices. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Mon Jul 25 19:27:03 2011 From: barry at python.org (Barry Warsaw) Date: Mon, 25 Jul 2011 13:27:03 -0400 Subject: [Python-ideas] Enums References: <87r55fqffn.fsf@benfinney.id.au> <4E2D996B.2060609@mrabarnett.plus.com> Message-ID: <20110725132703.04eecb42@resist.wooz.org> On Jul 25, 2011, at 05:27 PM, MRAB wrote: >My nitpick would be that the underlying integer values should start at >0, not 1. That's only relevant for the make_enum() convenience function. For the class syntax, you pick the values. OT1H, make_enum() is *just* a convenience, so if you really want to start from 0, use the class syntax. OTOH, it would probably be trivial to add an `iter` keyword argument to make_enum() from which it could pull the integer sequence to use. That would be better than adding something like range()'s crazy api. :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Mon Jul 25 19:34:32 2011 From: barry at python.org (Barry Warsaw) Date: Mon, 25 Jul 2011 13:34:32 -0400 Subject: [Python-ideas] Enums References: Message-ID: <20110725133432.3f29960c@resist.wooz.org> On Jul 25, 2011, at 05:46 PM, Michael Foord wrote: >For new apis it is *usually* possible to just use strings rather than >integers - and have your library do the mapping if an underlying api takes >integers. For existing ones, for example many parts of the python standard >library, we couldn't replace the integer values with enums without a lot of >effort. > >If the flufl enums subclassed integer then replacing most of the existing >constants in the standard library would be trivially easy (so we'd have a >built-in use case). I'm not sure it would be a good idea to do a mass mechanical substitution in the stdlib, so I'm not concerned that adopting enums would require a few int() calls to be added. I've so far resisted the occasional suggestion to make enum values int subclasses, because I think enum values should not *be* ints, but should be compatible with ints (as they are now). Ordered comparisons of enum values bothers me. ;) >The second use case, where you really do want to use integers rather than >strings, is where you have flag constants that you "or" together. For >example in the standard library we have these in r, gzip, msilib, some in >xml.dom.NodeFilter, plus a bunch of others. > >It would be nice to add support for or'ing of enums whilst retaining a nice >repr. Hmm. Of course ORing the enum values' int representation is trivial; you just have to pick the right values when you create the class. I'd be interested to see some specific examples for something more direct you have in mind. (Perhaps as a merge proposal .) >I did collect a whole lot of emails from a thread last year and was hoping >to put a pep together. I'd support flufl.enum - but it would be better if it >was extended so we could use it in the standard library. I think it would be usable in the stdlib today, but I'd like to see specific examples of how you think the API needs to be changed. Making enum values ints and supporting ordered comparisons is a big difference IMO. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Mon Jul 25 19:36:48 2011 From: barry at python.org (Barry Warsaw) Date: Mon, 25 Jul 2011 13:36:48 -0400 Subject: [Python-ideas] Enums References: Message-ID: <20110725133648.52ca7142@resist.wooz.org> On Jul 25, 2011, at 06:06 PM, Michael Foord wrote: >Here's an email from Guido from that thread putting his vote in for enums >*as* integers: > >http://mail.python.org/pipermail/python-dev/2010-November/105916.html Okay, there is that. :) It might be worth doing an example conversion to see what it looks like both with enum values as ints and not as ints, to see if it really makes much of a practical difference. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From tjreedy at udel.edu Mon Jul 25 19:41:06 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 25 Jul 2011 13:41:06 -0400 Subject: [Python-ideas] use "as" for block scope support In-Reply-To: References: Message-ID: On 7/25/2011 12:50 PM, ?? wrote: > 2. readability includes laconic > a = (x+y as t, x-y as s) t * s + s + t > it doesnt break thinking. from left to right, people finish thinking inplace. > temp var will not keep in memory, people can forget them since they > are not valid outside the expression. everything is clean. > t, s = x+y, x-y > a = t * s + s + t > in this case, thinking is break and temp var keep in memory, > people should take some care about it (for example, in nested loop) I prefer this. Temporary names are almost never a problem. In a nested loop, the names are reused and the objects detached for gc. > 3. this enchant the "lambda" syntax > list.sort(key = lambda p: (sqrt(p.x**2+p.y**2) as r) r**2+ A*r + B) I would rather delete lambda rather than enhance it. lambda expressions that refer to names other than the parameters are a constant source of confusion. To me, Python's mixed expression and statement syntax is a feature, not a wart to be avoided, as this suggestion aims to do. -- Terry Jan Reedy From max at mxcrypt.com Mon Jul 25 20:16:35 2011 From: max at mxcrypt.com (Maxim Khitrov) Date: Mon, 25 Jul 2011 14:16:35 -0400 Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library In-Reply-To: <30415.1311612388@parc.com> References: <30415.1311612388@parc.com> Message-ID: On Mon, Jul 25, 2011 at 12:46 PM, Bill Janssen wrote: > Maxim Khitrov wrote: > >> The same repository also contains the library code and an example >> script that you can run if you have access to an IMAP4 server: >> >> http://hg.mxcrypt.com/python/imaplib2/ > > I took a look at this. ?Nice work! ?Impressive list of extensions > implemented. Thank you :) > The first thing to ask is, does it pass the (minimal) test suite in > Lib/test/test_imaplib.py? ?Or is it too different? Too different. Writing my own test suite is the next step. The library already went through extensive testing on Gmail and MS Exchange servers while I was developing my main project, but of course a good set of unit tests is needed for long-term maintenance. > The code seems to be not quite in compliance with PEP 8. ?If you're > aiming at the stdlib you should probably fix that. Will do. > I tend to think one of the larger missing pieces of imap4lib is the lack > of a higher-level interface. ?I've used imap4lib for a couple of > projects, and I think that if I hadn't already had the experience of > implementing my own IMAP server in Python, I wouldn't have done that; > I'd have used a package like getmail instead. ?There's a lot of > experience with IMAP necessary to use imap4lib. True, but that's the case with all low-level interfaces. Having used imaplib for a while before deciding to write a replacement, I do think that building a higher-level interface using my library is significantly easier. I'm not certain if that interface should be part of the library or a separate module, similar to the relationship between zlib and gzip. I think the latter makes more sense, because any serious IMAP4 client will still need to descend to the protocol level. > Looking at your example.py, that still seems to be the case. One thing > you might consider is implementing a subclass of "mailbox.Mailbox" which > provides that missing higher-level interface to IMAP4. I've never used the mailbox module before. It looks like it is aimed primarily at working with local mail stores, no? I'm not sure if that's the right interface to use over the network, but I understand your point. I'll try to write another class on top of my library with a much simpler API for navigating through the mailbox hierarchy and working with messages. Will upload it to the repository when it's ready. - Max From fuzzyman at gmail.com Mon Jul 25 20:43:04 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 25 Jul 2011 19:43:04 +0100 Subject: [Python-ideas] Enums In-Reply-To: <20110725133432.3f29960c@resist.wooz.org> References: <20110725133432.3f29960c@resist.wooz.org> Message-ID: On 25 July 2011 18:34, Barry Warsaw wrote: > On Jul 25, 2011, at 05:46 PM, Michael Foord wrote: > > >For new apis it is *usually* possible to just use strings rather than > >integers - and have your library do the mapping if an underlying api takes > >integers. For existing ones, for example many parts of the python standard > >library, we couldn't replace the integer values with enums without a lot > of > >effort. > > > >If the flufl enums subclassed integer then replacing most of the existing > >constants in the standard library would be trivially easy (so we'd have a > >built-in use case). > > I'm not sure it would be a good idea to do a mass mechanical substitution > in > the stdlib, so I'm not concerned that adopting enums would require a few > int() > calls to be added. > > Well, if it isn't good enough for *us* then who is it good enough for? ;-) Some apis (for example those exported directly from C) can't work with something that isn't a real int. > I've so far resisted the occasional suggestion to make enum values int > subclasses, because I think enum values should not *be* ints, but should be > compatible with ints (as they are now). Ordered comparisons of enum values > bothers me. ;) > > >The second use case, where you really do want to use integers rather than > >strings, is where you have flag constants that you "or" together. For > >example in the standard library we have these in r, gzip, msilib, some in > >xml.dom.NodeFilter, plus a bunch of others. > > > >It would be nice to add support for or'ing of enums whilst retaining a > nice > >repr. > > Hmm. Of course ORing the enum values' int representation is trivial; you > just > have to pick the right values when you create the class. I'd be interested > to > see some specific examples for something more direct you have in mind. > (Perhaps as a merge proposal .) > Well: >>> from flufl.enum import Enum >>> class Thing(Enum): ... a = 1 ... b = 2 ... c = 4 ... >>> Thing.a | Thing.b Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for |: 'EnumValue' and 'EnumValue' > > >I did collect a whole lot of emails from a thread last year and was hoping > >to put a pep together. I'd support flufl.enum - but it would be better if > it > >was extended so we could use it in the standard library. > > I think it would be usable in the stdlib today, but I'd like to see > specific > examples of how you think the API needs to be changed. Making enum values > ints and supporting ordered comparisons is a big difference IMO. > > Sure, I have no use case for ordering enums... I'm not sure I have a specific reason to *prevent* if that is extra work though. Michael > Cheers, > -Barry > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.hettinger at gmail.com Mon Jul 25 20:47:46 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Mon, 25 Jul 2011 14:47:46 -0400 Subject: [Python-ideas] Enums In-Reply-To: References: Message-ID: <1C6CA719-5727-4945-BC88-0CB773022FA3@gmail.com> On Jul 25, 2011, at 1:06 PM, Michael Foord wrote: > > Python standard library modules currently using integers for constants: > > * re - has flags (OR'able constants) defined in sre_constants, each flag has two names (e.g. re.IGNORECASE and re.I) What is being proposed here? Will there be a new namespace so that we would start writing re.Flags.IGNORECASE instead of re.IGNORECASE? Are module level constants now going to be considered bad-form? If constants switch from re.IGNORECASE to re.flags.IGNORECASE, are there any benefits to compensate for being both wordier and slower? > * os has SEEK_SET, SEEK_CUR, SEEK_END - *plus* those implemented in posix / nt Ditto > * doctest has its own flag system, but is really just using integer flags / constants (quite a few of them) > * token has a tonne of constants (autogenerated) > * socket exports a bunch of constants defined in _socket > * gzip has flags: FTEXT, FHCRC, FEXTRA, FNAME, FCOMMENT > > * errno (builtin module) > > EALREADY, EINPROGRESS, EWOULDBLOCK, ECONNRESET, EINVAL, > ENOTCONN, ESHUTDOWN, EINTR, EISCONN, EBADF, ECONNABORTED It seems to me that an enum module or class doesn't make any of this code better. AFAICT enums are a fat solution to a thin problem. Since python makes it so easy to put constants in class namespaces and module namespaces, there is much less need for enums than in statically compiled languages. I think the perceived need is really more of a carry-over coding habit than an actual need. That being said, this proposal seems to have momentum, so it is going happen anyway. Once introduced, people will think we've endorsed enums as the one right way to code constants, so we will start to see them everywhere. Expect enums to become ubiquitous. Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: From lyricconch at gmail.com Mon Jul 25 20:56:37 2011 From: lyricconch at gmail.com (=?UTF-8?B?5rW36Z+1?=) Date: Tue, 26 Jul 2011 02:56:37 +0800 Subject: [Python-ideas] Fwd: use "as" for block scope support In-Reply-To: References: Message-ID: 1. but what about ( t * s + t / s )? temp var always keep in memory. (should not reduce ( t + s ) to ( 2 * x ) since x, y may not a number. any operator here should be conside as independent function. ) 2. try convert this to the lambda form: (x+y as t, x-y as s)(t * s as u, t/s as v)(u << v as p, u>>v as q) p ** q In runtime: lambda = MAKE_FUNCTION/MAKE_CLOSURE = PyFunction_New lambda call = CALL_FUNCTION = PyFrame_New + PyEval_EvalFrameEx all of them are heavy, (you can check the source code) but: as = PyCell_New as enter = SETUP_FINALLY = PyFrame_BlockSetup as leave = END_FINALLY = PyFrame_BlockPop which is much much much more more more cheaper than above. 2011/7/26 Bruce Leban : > > > On Mon, Jul 25, 2011 at 9:50 AM, ?? wrote: >> >> Dupilcate K is used only for describe the scope rule. >> Advantage: >> >> 1. the same grammer, the same behave >> "t" of """(x+y as t, x-y as s) t * s + s + t""" is only valid in the >> expression which require >> just as >> "e" of """except ... as e""" only live in the suite which it belongs >> >> 2. readability includes laconic >> a = (x+y as t, x-y as s) t * s + s + t >> it doesnt break thinking. from left to right, people finish thinking >> inplace. >> temp var will not keep in memory, people can forget them since they >> are not valid outside the expression. everything is clean. >> t, s = x+y, x-y >> a = t * s + s + t >> in this case, thinking is break and temp var keep in memory, >> people should take some care about it (for example, in nested loop) > > (x + y) * (x - y) + 2 * x > is IMHO more readable - especially note the absence of y in the third term. >> >> 3. this enchant the "lambda" syntax >> list.sort(key = lambda p: (sqrt(p.x**2+p.y**2) as r) r**2+ A*r + B) > > I don't know what "enchant" means here, but you can do this if you want: > list.sort(key = lambda p: (lambda r: r**2 + A*r + B) (sqrt(p.x**2+p.y**2))) > I'm not recommending that as I think it would be more readable to name that > function so that it would be more clear why you're using that function as > the sort key. But, to me, this example does not justify new syntax. > --- Bruce From ethan at stoneleaf.us Mon Jul 25 21:16:50 2011 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 25 Jul 2011 12:16:50 -0700 Subject: [Python-ideas] Enums In-Reply-To: <20110725133432.3f29960c@resist.wooz.org> References: <20110725133432.3f29960c@resist.wooz.org> Message-ID: <4E2DC122.4090103@stoneleaf.us> Barry Warsaw wrote: > On Jul 25, 2011, at 05:46 PM, Michael Foord wrote: >> If the flufl enums subclassed integer then replacing most of the existing >> constants in the standard library would be trivially easy (so we'd have a >> built-in use case). > > I've so far resisted the occasional suggestion to make enum values int > subclasses, because I think enum values should not *be* ints, but should be > compatible with ints (as they are now). Ordered comparisons of enum values > bothers me. ;) If it's going to be in the stdlib, I think they should be ints -- while not necessary most of the time, the few times it is will illicit praise for Python and its simplicity instead of complaints about somebody making decisions for them. ~Ethan~ From janssen at parc.com Mon Jul 25 21:03:25 2011 From: janssen at parc.com (Bill Janssen) Date: Mon, 25 Jul 2011 12:03:25 PDT Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library In-Reply-To: References: <30415.1311612388@parc.com> Message-ID: <33493.1311620605@parc.com> Maxim Khitrov wrote: > > I tend to think one of the larger missing pieces of imap4lib is the lack > > of a higher-level interface. ?I've used imap4lib for a couple of > > projects, and I think that if I hadn't already had the experience of > > implementing my own IMAP server in Python, I wouldn't have done that; > > I'd have used a package like getmail instead. ?There's a lot of > > experience with IMAP necessary to use imap4lib. > > True, but that's the case with all low-level interfaces. Having used > imaplib for a while before deciding to write a replacement, I do think > that building a higher-level interface using my library is > significantly easier. > > I'm not certain if that interface should be part of the library or a > separate module, similar to the relationship between zlib and gzip. I > think the latter makes more sense, because any serious IMAP4 client > will still need to descend to the protocol level. There are lots of folks who just need to connect to an IMAP account, and do something with their mail. Many many fewer need to write a serious IMAP4 client. That's the big problem with the current imap4lib; it just doesn't work for most of the use cases, because it's too low-level. So it doesn't get much use -- more's the pity. > > Looking at your example.py, that still seems to be the case. One thing > > you might consider is implementing a subclass of "mailbox.Mailbox" which > > provides that missing higher-level interface to IMAP4. > > I've never used the mailbox module before. It looks like it is aimed > primarily at working with local mail stores, no? I'm not sure if > that's the right interface to use over the network, but I understand > your point. > > I'll try to write another class on top of my library with a much > simpler API for navigating through the mailbox hierarchy and working > with messages. Will upload it to the repository when it's ready. Cool! Looking forward to it. I'd still suggest making it a subclass of mailbox.Mailbox, btw. PEP 20: "There should be one -- and preferably only one -- obvious way to do it." Bill From lyricconch at gmail.com Mon Jul 25 21:08:27 2011 From: lyricconch at gmail.com (=?UTF-8?B?5rW36Z+1?=) Date: Tue, 26 Jul 2011 03:08:27 +0800 Subject: [Python-ideas] use "as" for block scope support In-Reply-To: References: Message-ID: ? 2011?7?26? ??2:56??? ??? > 1. > but what about ( t * s + t / s )? temp var always keep in memory. > (should not reduce ( t + s ) to ( 2 * x ) since x, y may not a number. > any operator here should be conside as independent function. ) > > 2. > try convert this to the lambda form: > (x+y as t, x-y as s)(t * s as u, t/s as v)(u << v as p, u>>v as q) p ** q sorry, here should be : try convert this to the lambda form: R = lamba x, y: (x+y as t, x-y as s)(t * s as u, t/s as v)(u << v as p, u>>v as q) p ** q using "as" can make lambda expression more graceful. > > In runtime: > lambda = MAKE_FUNCTION/MAKE_CLOSURE = PyFunction_New > lambda call = CALL_FUNCTION = PyFrame_New + PyEval_EvalFrameEx > all of them are heavy, (you can check the source code) > > but: > as = PyCell_New > as enter = SETUP_FINALLY = PyFrame_BlockSetup > as leave = END_FINALLY = PyFrame_BlockPop > which is much much much more more more cheaper than above. > > 2011/7/26 Bruce Leban : >> >> >> On Mon, Jul 25, 2011 at 9:50 AM, ?? wrote: >>> >>> Dupilcate K is used only for describe the scope rule. >>> Advantage: >>> >>> 1. the same grammer, the same behave >>> "t" of """(x+y as t, x-y as s) t * s + s + t""" is only valid in the >>> expression which require >>> just as >>> "e" of """except ... as e""" only live in the suite which it belongs >>> >>> 2. readability includes laconic >>> a = (x+y as t, x-y as s) t * s + s + t >>> it doesnt break thinking. from left to right, people finish thinking >>> inplace. >>> temp var will not keep in memory, people can forget them since they >>> are not valid outside the expression. everything is clean. >>> t, s = x+y, x-y >>> a = t * s + s + t >>> in this case, thinking is break and temp var keep in memory, >>> people should take some care about it (for example, in nested loop) >> >> (x + y) * (x - y) + 2 * x >> is IMHO more readable - especially note the absence of y in the third term. >>> >>> 3. this enchant the "lambda" syntax >>> list.sort(key = lambda p: (sqrt(p.x**2+p.y**2) as r) r**2+ A*r + B) >> >> I don't know what "enchant" means here, but you can do this if you want: >> list.sort(key = lambda p: (lambda r: r**2 + A*r + B) (sqrt(p.x**2+p.y**2))) >> I'm not recommending that as I think it would be more readable to name that >> function so that it would be more clear why you're using that function as >> the sort key. But, to me, this example does not justify new syntax. >> --- Bruce > From fuzzyman at gmail.com Mon Jul 25 21:09:15 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 25 Jul 2011 20:09:15 +0100 Subject: [Python-ideas] Enums In-Reply-To: <1C6CA719-5727-4945-BC88-0CB773022FA3@gmail.com> References: <1C6CA719-5727-4945-BC88-0CB773022FA3@gmail.com> Message-ID: On 25 July 2011 19:47, Raymond Hettinger wrote: > > On Jul 25, 2011, at 1:06 PM, Michael Foord wrote: > > > Python standard library modules currently using integers for constants: > > * re - has flags (OR'able constants) defined in sre_constants, each flag > has two names (e.g. re.IGNORECASE and re.I) > > > What is being proposed here? Will there be a new namespace so that we > would start writing re.Flags.IGNORECASE instead of re.IGNORECASE? Are > module level constants now going to be considered bad-form? > > If constants switch from re.IGNORECASE to re.flags.IGNORECASE, are there > any benefits to compensate for being both wordier and slower? > > Nope. Just something like this at the module level. IGNORECASE = Flags.IGNORECASE What it gains you is a nicer representation when looking at the values when debugging. On the other hand being able to have access to all the constants namespaced on a single object (as well) doesn't seem like such a bad thing. Michael > > * os has SEEK_SET, SEEK_CUR, SEEK_END - **plus** those implemented in > posix / nt > > > Ditto > > * doctest has its own flag system, but is really just using integer flags / > constants (quite a few of them) > * token has a tonne of constants (autogenerated) > * socket exports a bunch of constants defined in _socket > * gzip has flags: FTEXT, FHCRC, FEXTRA, FNAME, FCOMMENT > > * errno (builtin module) > > EALREADY, EINPROGRESS, EWOULDBLOCK, ECONNRESET, EINVAL, > ENOTCONN, ESHUTDOWN, EINTR, EISCONN, EBADF, ECONNABORTED > > > It seems to me that an enum module or class doesn't make any of this code > better. > AFAICT enums are a fat solution to a thin problem. > > Since python makes it so easy to put constants in class namespaces > and module namespaces, there is much less need for enums than in > statically compiled languages. I think the perceived need is really > more of a carry-over coding habit than an actual need. > > That being said, this proposal seems to have momentum, > so it is going happen anyway. Once introduced, people will > think we've endorsed enums as the one right way to code > constants, so we will start to see them everywhere. > Expect enums to become ubiquitous. > > > Raymond > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From lyricconch at gmail.com Mon Jul 25 21:18:32 2011 From: lyricconch at gmail.com (=?UTF-8?B?5rW36Z+1?=) Date: Tue, 26 Jul 2011 03:18:32 +0800 Subject: [Python-ideas] use "as" for block scope support In-Reply-To: References: Message-ID: 1. but what about ( t * s + t / s )? temp var always keep in memory. (should not reduce ( t + s ) to ( 2 * x ) since x, y may not a number. any operator here should be conside as independent function. ) 2. you got an extra lambda, that means: In runtime: lambda = MAKE_FUNCTION/MAKE_CLOSURE = PyFunction_New lambda call = CALL_FUNCTION = PyFrame_New + PyEval_EvalFrameEx all of them are heavy, (you can check the source code) but: as = PyCell_New as enter = SETUP_FINALLY = PyFrame_BlockSetup as leave = END_FINALLY = PyFrame_BlockPop which is much much much more more more cheaper than above. 3. try convert this to the lambda form as mention: R= lambda x, y: (x+y as t, x-y as s)(t * s as u, t/s as v)(u << v as p, u>>v as q) p ** q 2011/7/26 Bruce Leban : > > > On Mon, Jul 25, 2011 at 9:50 AM, ?? wrote: >> >> Dupilcate K is used only for describe the scope rule. >> Advantage: >> >> 1. the same grammer, the same behave >> "t" of """(x+y as t, x-y as s) t * s + s + t""" is only valid in the >> expression which require >> just as >> "e" of """except ... as e""" only live in the suite which it belongs >> >> 2. readability includes laconic >> a = (x+y as t, x-y as s) t * s + s + t >> it doesnt break thinking. from left to right, people finish thinking >> inplace. >> temp var will not keep in memory, people can forget them since they >> are not valid outside the expression. everything is clean. >> t, s = x+y, x-y >> a = t * s + s + t >> in this case, thinking is break and temp var keep in memory, >> people should take some care about it (for example, in nested loop) > > (x + y) * (x - y) + 2 * x > is IMHO more readable - especially note the absence of y in the third term. >> >> 3. this enchant the "lambda" syntax >> list.sort(key = lambda p: (sqrt(p.x**2+p.y**2) as r) r**2+ A*r + B) > > I don't know what "enchant" means here, but you can do this if you want: > list.sort(key = lambda p: (lambda r: r**2 + A*r + B) (sqrt(p.x**2+p.y**2))) > I'm not recommending that as I think it would be more readable to name that > function so that it would be more clear why you're using that function as > the sort key. But, to me, this example does not justify new syntax. > --- Bruce From fuzzyman at gmail.com Mon Jul 25 21:21:03 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 25 Jul 2011 20:21:03 +0100 Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library In-Reply-To: References: Message-ID: On 25 July 2011 02:06, Maxim Khitrov wrote: > Hello python-ideas, > > My most recent project lead me down a path that eventually ended up at > a new implementation of imaplib based on [RFC-3501]. Although I > started the project by gradually adding functionality to the existing > IMAP4 library, some of the features that I required simply could not > be merged in (without breaking everything). As a result, I wrote my > own version of the library, which incorporates all existing > functionality of imaplib and includes many of my own improvements. > > There is an existing, well tested and widely used, replaced for imaplib that I would suggest should be the first for consideration in replacing imaplib: http://imapclient.freshfoo.com/ (Sorry.) All the best, Michael Foord > I wrote a PEP-style readme file that describes all the details of why > the library was written and how it works, which is available from my > mercurial repository: > > http://hg.mxcrypt.com/python/imaplib2/raw-file/tip/README > > The same repository also contains the library code and an example > script that you can run if you have access to an IMAP4 server: > > http://hg.mxcrypt.com/python/imaplib2/ > > Is there any interest in adding my code to a future version of Python > 3.x standard library? > > - Max > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From masklinn at masklinn.net Mon Jul 25 21:25:39 2011 From: masklinn at masklinn.net (Masklinn) Date: Mon, 25 Jul 2011 21:25:39 +0200 Subject: [Python-ideas] Enums In-Reply-To: References: <1C6CA719-5727-4945-BC88-0CB773022FA3@gmail.com> Message-ID: <5439382D-AC3C-4243-8EFA-84A9434B328D@masklinn.net> On 2011-07-25, at 21:09 , Michael Foord wrote: > On 25 July 2011 19:47, Raymond Hettinger wrote: >> On Jul 25, 2011, at 1:06 PM, Michael Foord wrote: >> >> >> Python standard library modules currently using integers for constants: >> >> * re - has flags (OR'able constants) defined in sre_constants, each flag >> has two names (e.g. re.IGNORECASE and re.I) >> >> >> What is being proposed here? Will there be a new namespace so that we >> would start writing re.Flags.IGNORECASE instead of re.IGNORECASE? Are >> module level constants now going to be considered bad-form? >> >> If constants switch from re.IGNORECASE to re.flags.IGNORECASE, are there >> any benefits to compensate for being both wordier and slower? >> >> > Nope. Just something like this at the module level. > > IGNORECASE = Flags.IGNORECASE > > What it gains you is a nicer representation when looking at the values when > debugging. On the other hand being able to have access to all the constants > namespaced on a single object (as well) doesn't seem like such a bad thing. Especially nice with unions/a good syntax for enumset. Because "74" being re.IGNORECASE | re.MULITILINE | re.VERBOSE is not exactly obvious. Whereas an str like could be rather nice (not overly fond of automatically OR-ing ints for unions/sets, though, and thinking that there could be value in "enum values" being singletons rather than values) From max at mxcrypt.com Mon Jul 25 21:38:54 2011 From: max at mxcrypt.com (Maxim Khitrov) Date: Mon, 25 Jul 2011 15:38:54 -0400 Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library In-Reply-To: References: Message-ID: On Mon, Jul 25, 2011 at 3:21 PM, Michael Foord wrote: > > > On 25 July 2011 02:06, Maxim Khitrov wrote: >> >> Hello python-ideas, >> >> My most recent project lead me down a path that eventually ended up at >> a new implementation of imaplib based on [RFC-3501]. Although I >> started the project by gradually adding functionality to the existing >> IMAP4 library, some of the features that I required simply could not >> be merged in (without breaking everything). As a result, I wrote my >> own version of the library, which incorporates all existing >> functionality of imaplib and includes many of my own improvements. >> > > There is an existing, well tested and widely used, replaced for imaplib that > I would suggest should be the first for consideration in replacing imaplib: > > http://imapclient.freshfoo.com/ > > (Sorry.) > > All the best, > > Michael Foord I have it beat at the "Python 3 support is in the works" feature ;) Mine doesn't handle 2.x though. In any case, I would not have been able to use IMAPClient for my project, because the requirements were for no dependencies outside of Python 3.2. Do you know if the developers of IMAPClient considered getting it into the standard library? My goal wasn't just to have another IMAP implementation, but something better available as part of Python. - Max From brian.curtin at gmail.com Mon Jul 25 21:42:39 2011 From: brian.curtin at gmail.com (Brian Curtin) Date: Mon, 25 Jul 2011 14:42:39 -0500 Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library In-Reply-To: References: Message-ID: On Mon, Jul 25, 2011 at 14:38, Maxim Khitrov wrote: > On Mon, Jul 25, 2011 at 3:21 PM, Michael Foord wrote: > > > > > > On 25 July 2011 02:06, Maxim Khitrov wrote: > >> > >> Hello python-ideas, > >> > >> My most recent project lead me down a path that eventually ended up at > >> a new implementation of imaplib based on [RFC-3501]. Although I > >> started the project by gradually adding functionality to the existing > >> IMAP4 library, some of the features that I required simply could not > >> be merged in (without breaking everything). As a result, I wrote my > >> own version of the library, which incorporates all existing > >> functionality of imaplib and includes many of my own improvements. > >> > > > > There is an existing, well tested and widely used, replaced for imaplib > that > > I would suggest should be the first for consideration in replacing > imaplib: > > > > http://imapclient.freshfoo.com/ > > > > (Sorry.) > > > > All the best, > > > > Michael Foord > > I have it beat at the "Python 3 support is in the works" feature ;) > Mine doesn't handle 2.x though. That's not something you'd have to worry about anyway - new features, modules, and packages would only be accepted in the next 3.x release (3.3). -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Mon Jul 25 22:01:34 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 25 Jul 2011 16:01:34 -0400 Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library In-Reply-To: References: Message-ID: On 7/25/2011 3:42 PM, Brian Curtin wrote: > That's not something you'd have to worry about anyway - new features, > modules, and packages would only be accepted in the next 3.x release (3.3). And should be written in Py 3 style. In particular, long sequences should generally be returned as iterables rather than as lists. If I understood the proposal, fetch does that, where as old imaplib waited to create a finished list. -- Terry Jan Reedy From paul at colomiets.name Mon Jul 25 22:05:28 2011 From: paul at colomiets.name (Paul Colomiets) Date: Mon, 25 Jul 2011 23:05:28 +0300 Subject: [Python-ideas] Enums In-Reply-To: <1C6CA719-5727-4945-BC88-0CB773022FA3@gmail.com> References: <1C6CA719-5727-4945-BC88-0CB773022FA3@gmail.com> Message-ID: On Mon, Jul 25, 2011 at 9:47 PM, Raymond Hettinger wrote: > > On Jul 25, 2011, at 1:06 PM, Michael Foord wrote: > > Python standard library modules currently using integers for constants: > > * re - has flags (OR'able constants) defined in sre_constants, each flag has > two names (e.g. re.IGNORECASE and re.I) > > What is being proposed here? ?Will there be a new namespace so that we would > start writing re.Flags.IGNORECASE instead of re.IGNORECASE? ?Are module > level constants now going to be considered bad-form? > If constants switch from re.IGNORECASE to re.flags.IGNORECASE, are there any > benefits to compensate for being both wordier and slower? Well, I'll be happy to write: from re import compile as re, Flags as ref (not sure exact names, but you've got the idea) -- Paul From max at mxcrypt.com Mon Jul 25 22:21:16 2011 From: max at mxcrypt.com (Maxim Khitrov) Date: Mon, 25 Jul 2011 16:21:16 -0400 Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library In-Reply-To: References: Message-ID: On Mon, Jul 25, 2011 at 4:01 PM, Terry Reedy wrote: > On 7/25/2011 3:42 PM, Brian Curtin wrote: > >> That's not something you'd have to worry about anyway - new features, >> modules, and packages would only be accepted in the next 3.x release >> (3.3). > > And should be written in Py 3 style. In particular, long sequences should > generally be returned as iterables rather than as lists. > If I understood the proposal, fetch does that, where as old imaplib waited > to create a finished list. Correct. - Max From fuzzyman at gmail.com Mon Jul 25 22:27:35 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 25 Jul 2011 21:27:35 +0100 Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library In-Reply-To: References: Message-ID: On 25 July 2011 20:38, Maxim Khitrov wrote: > On Mon, Jul 25, 2011 at 3:21 PM, Michael Foord wrote: > > > > > > On 25 July 2011 02:06, Maxim Khitrov wrote: > >> > >> Hello python-ideas, > >> > >> My most recent project lead me down a path that eventually ended up at > >> a new implementation of imaplib based on [RFC-3501]. Although I > >> started the project by gradually adding functionality to the existing > >> IMAP4 library, some of the features that I required simply could not > >> be merged in (without breaking everything). As a result, I wrote my > >> own version of the library, which incorporates all existing > >> functionality of imaplib and includes many of my own improvements. > >> > > > > There is an existing, well tested and widely used, replaced for imaplib > that > > I would suggest should be the first for consideration in replacing > imaplib: > > > > http://imapclient.freshfoo.com/ > > > > (Sorry.) > > > > All the best, > > > > Michael Foord > > I have it beat at the "Python 3 support is in the works" feature ;) > Mine doesn't handle 2.x though. > > In any case, I would not have been able to use IMAPClient for my > project, because the requirements were for no dependencies outside of > Python 3.2. > > Do you know if the developers of IMAPClient considered getting it into > the standard library? My goal wasn't just to have another IMAP > implementation, but something better available as part of Python. > > I don't think Menno Smitts would object to adding Python 3 support or adding IMAPClient to the standard library. His goal was to create something useful to overcome what he saw (and evidently you agree) as irreparable brokenness in parts of imaplib. My point is that if there is an existing widely-used and battle-tested alternative, we would be wise to look at that first. Michael > - Max > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Mon Jul 25 22:33:55 2011 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 25 Jul 2011 15:33:55 -0500 Subject: [Python-ideas] Enums In-Reply-To: References: <1C6CA719-5727-4945-BC88-0CB773022FA3@gmail.com> Message-ID: On 7/25/11 2:09 PM, Michael Foord wrote: > > > On 25 July 2011 19:47, Raymond Hettinger > > wrote: > > > On Jul 25, 2011, at 1:06 PM, Michael Foord wrote: >> >> Python standard library modules currently using integers for constants: >> >> * re - has flags (OR'able constants) defined in sre_constants, each flag >> has two names (e.g. re.IGNORECASE and re.I) > > What is being proposed here? Will there be a new namespace so that we would > start writing re.Flags.IGNORECASE instead of re.IGNORECASE? Are module > level constants now going to be considered bad-form? > > If constants switch from re.IGNORECASE to re.flags.IGNORECASE, are there any > benefits to compensate for being both wordier and slower? > > > Nope. Just something like this at the module level. > > IGNORECASE = Flags.IGNORECASE > > What it gains you is a nicer representation when looking at the values when > debugging. +1. It's not a problem I run into every time I debug something, but it happens often enough that the feeling of frustration is quite familiar each time it crops up. It's like an old college acquaintance that shows up to crash "just a few nights" on your couch every time he's in town. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From robert.kern at gmail.com Mon Jul 25 22:47:24 2011 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 25 Jul 2011 15:47:24 -0500 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> Message-ID: On 7/25/11 1:43 PM, Michael Foord wrote: > Some apis (for example those exported directly from C) can't work with something > that isn't a real int. I think almost all of those will usually accept an object that implements __int__ and __index__, don't they? E.g. [~] |1> import os [~] |2> class A(object): ..> def __init__(self,x): ..> self.x = x ..> def __int__(self): ..> return self.x ..> def __index__(self): ..> return self.x ..> [~] |10> os.open('foo', A(os.O_RDWR)) 21 os.open() just uses PyArg_ParseTuple(), like most extension functions. I'm sure you could write an extension function that would reject A() instances, but it's more work, so most people don't. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From fuzzyman at gmail.com Mon Jul 25 23:39:50 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 25 Jul 2011 22:39:50 +0100 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> Message-ID: On 25 July 2011 21:47, Robert Kern wrote: > On 7/25/11 1:43 PM, Michael Foord wrote: > > Some apis (for example those exported directly from C) can't work with >> something >> that isn't a real int. >> > > I think almost all of those will usually accept an object that implements > __int__ and __index__, don't they? E.g. > > [~] > |1> import os > > [~] > |2> class A(object): > ..> def __init__(self,x): > ..> self.x = x > ..> def __int__(self): > ..> return self.x > ..> def __index__(self): > ..> return self.x > ..> > > [~] > |10> os.open('foo', A(os.O_RDWR)) > 21 > > > os.open() just uses PyArg_ParseTuple(), like most extension functions. I'm > sure you could write an extension function that would reject A() instances, > but it's more work, so most people don't. > > Providing __index__ may indeed be enough. Michael > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though it > had > an underlying truth." > -- Umberto Eco > > ______________________________**_________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/**mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at gmail.com Mon Jul 25 23:40:11 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 25 Jul 2011 22:40:11 +0100 Subject: [Python-ideas] Enums In-Reply-To: References: <1C6CA719-5727-4945-BC88-0CB773022FA3@gmail.com> Message-ID: On 25 July 2011 21:33, Robert Kern wrote: > On 7/25/11 2:09 PM, Michael Foord wrote: > >> >> >> On 25 July 2011 19:47, Raymond Hettinger >> > >> >> wrote: >> >> >> On Jul 25, 2011, at 1:06 PM, Michael Foord wrote: >> >>> >>> Python standard library modules currently using integers for >>> constants: >>> >>> * re - has flags (OR'able constants) defined in sre_constants, each >>> flag >>> has two names (e.g. re.IGNORECASE and re.I) >>> >> >> What is being proposed here? Will there be a new namespace so that we >> would >> start writing re.Flags.IGNORECASE instead of re.IGNORECASE? Are module >> level constants now going to be considered bad-form? >> >> If constants switch from re.IGNORECASE to re.flags.IGNORECASE, are >> there any >> benefits to compensate for being both wordier and slower? >> >> >> Nope. Just something like this at the module level. >> >> IGNORECASE = Flags.IGNORECASE >> >> What it gains you is a nicer representation when looking at the values >> when >> debugging. >> > > +1. It's not a problem I run into every time I debug something, but it > happens often enough that the feeling of frustration is quite familiar each > time it crops up. It's like an old college acquaintance that shows up to > crash "just a few nights" on your couch every time he's in town. > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though it > had > an underlying truth." > -- Umberto Eco > > > ______________________________**_________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/**mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Mon Jul 25 23:44:56 2011 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 25 Jul 2011 16:44:56 -0500 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> Message-ID: On 7/25/11 4:39 PM, Michael Foord wrote: > > > On 25 July 2011 21:47, Robert Kern > wrote: > > On 7/25/11 1:43 PM, Michael Foord wrote: > > Some apis (for example those exported directly from C) can't work with > something > that isn't a real int. > > > I think almost all of those will usually accept an object that implements > __int__ and __index__, don't they? E.g. > > [~] > |1> import os > > [~] > |2> class A(object): > ..> def __init__(self,x): > ..> self.x = x > ..> def __int__(self): > ..> return self.x > ..> def __index__(self): > ..> return self.x > ..> > > [~] > |10> os.open('foo', A(os.O_RDWR)) > 21 > > > os.open() just uses PyArg_ParseTuple(), like most extension functions. I'm > sure you could write an extension function that would reject A() instances, > but it's more work, so most people don't. > > > Providing __index__ may indeed be enough. From experiment with os.open() under 2.7 and 3.1, it is not enough, though perhaps either PyArg_ParseTuple() or PyInt_As*() should be modified to make it so. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From max at mxcrypt.com Tue Jul 26 00:39:17 2011 From: max at mxcrypt.com (Maxim Khitrov) Date: Mon, 25 Jul 2011 18:39:17 -0400 Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library In-Reply-To: References: Message-ID: On Mon, Jul 25, 2011 at 4:27 PM, Michael Foord wrote: > > > On 25 July 2011 20:38, Maxim Khitrov wrote: >> >> On Mon, Jul 25, 2011 at 3:21 PM, Michael Foord wrote: >> > >> > >> > On 25 July 2011 02:06, Maxim Khitrov wrote: >> >> >> >> Hello python-ideas, >> >> >> >> My most recent project lead me down a path that eventually ended up at >> >> a new implementation of imaplib based on [RFC-3501]. Although I >> >> started the project by gradually adding functionality to the existing >> >> IMAP4 library, some of the features that I required simply could not >> >> be merged in (without breaking everything). As a result, I wrote my >> >> own version of the library, which incorporates all existing >> >> functionality of imaplib and includes many of my own improvements. >> >> >> > >> > There is an existing, well tested and widely used, replaced for imaplib >> > that >> > I would suggest should be the first for consideration in replacing >> > imaplib: >> > >> > http://imapclient.freshfoo.com/ >> > >> > (Sorry.) >> > >> > All the best, >> > >> > Michael Foord >> >> I have it beat at the "Python 3 support is in the works" feature ;) >> Mine doesn't handle 2.x though. >> >> In any case, I would not have been able to use IMAPClient for my >> project, because the requirements were for no dependencies outside of >> Python 3.2. >> >> Do you know if the developers of IMAPClient considered getting it into >> the standard library? My goal wasn't just to have another IMAP >> implementation, but something better available as part of Python. >> > > > I don't think Menno Smitts would object to adding Python 3 support or adding > IMAPClient to the standard library. His goal was to create something useful > to overcome what he saw (and evidently you agree) as irreparable brokenness > in parts of imaplib. > > My point is that if there is an existing widely-used and battle-tested > alternative, we would be wise to look at that first. Agreed. However, I just took a look through IMAPClient source code and have to correct you on your original assertion. IMAPClient is not a replacement for imaplib, but a wrapper around it. The author went down the same road that I originally started out on. He added a parser, a UTF-7 codec, and changed the overall interface to be more user-friendly. All good improvements, but at the core, IMAPClient relies entirely on imaplib, and thus inherits most of its design flaws. With the exception of the first and fifth bullet points in my README file (server response parser & UTF-7 mailbox name codec, respectively), all others apply equally to IMAPClient. For that reason, I would recommend against incorporating it into the standard library. My opinion on the matter is certainly biased, so I would welcome a review of both libraries by a neutral party familiar with the IMAP protocol, who could then make a recommendation. - Max From steve at pearwood.info Tue Jul 26 01:10:14 2011 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 26 Jul 2011 09:10:14 +1000 Subject: [Python-ideas] use "as" for block scope support In-Reply-To: References: Message-ID: <4E2DF7D6.5000706@pearwood.info> ?? wrote: > 1. > but what about ( t * s + t / s )? temp var always keep in memory. > (should not reduce ( t + s ) to ( 2 * x ) since x, y may not a number. > any operator here should be conside as independent function. ) What about it? The temporary variables will go out of scope when the function exists, and be garbage collected, the same as every other local variable. > 2. you got an extra lambda, that means: > In runtime: > lambda = MAKE_FUNCTION/MAKE_CLOSURE = PyFunction_New > lambda call = CALL_FUNCTION = PyFrame_New + PyEval_EvalFrameEx > all of them are heavy, (you can check the source code) > but: > as = PyCell_New > as enter = SETUP_FINALLY = PyFrame_BlockSetup > as leave = END_FINALLY = PyFrame_BlockPop > which is much much much more more more cheaper than above. Sounds like premature optimization to me. I would like to see actual, real-world code where the performance bottleneck is setting up functions. > 3. > try convert this to the lambda form as mention: > R= lambda x, y: (x+y as t, x-y as s)(t * s as u, t/s as v)(u << v as > p, u>>v as q) p ** q I wouldn't even *try* to convert that to a lambda. That is ugly, unmaintainable code. To understand it, I had to first write it out in block form: def R(x, y): with x+y as t, x-y as s: with t*s as u, t/s as v: with u << v as p, u >> v as q return p**q which is much simpler to read and maintain when written like this: def R(x, y): t, s = x+y, x-y u, v = t*s, t/s p, q = u << v, u >> v return p**q I consider this proposal to encourage excess and unnecessary encapsulation of variables. Not every expression needs to be it's own block. -- Steven From ncoghlan at gmail.com Tue Jul 26 02:02:33 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 26 Jul 2011 10:02:33 +1000 Subject: [Python-ideas] anonymous object support In-Reply-To: <20110725125226.020bfffc@resist.wooz.org> References: <20110725125226.020bfffc@resist.wooz.org> Message-ID: On Tue, Jul 26, 2011 at 2:52 AM, Barry Warsaw wrote: > On Jul 24, 2011, at 09:03 AM, Guido van Rossum wrote: > >>Also, every one believes there own favorite feature to be so important >>that it's worth changing the language, but they don't want to grant >>that to anybody else's favorite feature... :-) > > This reminds me of the poll you conducted at a keynote, many Pycons ago. ?You > first asked "who thinks the language is changing too quickly?" ?Lots of hands > went up. ?Then you asked (probably paraphrasing) "who has a feature they think > needs to go into Python?" ?Lots of the same hands went up. :) Heh, I used the exact same point as my first section heading when blogging about some of the reasons I think python-dev is the way it is: "Python evolves too slowly! You're changing the language too fast!" (http://www.boredomandlaziness.org/2011/04/musings-on-culture-of-python-dev.html). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Tue Jul 26 02:18:33 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 26 Jul 2011 10:18:33 +1000 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: On Tue, Jul 26, 2011 at 12:13 AM, Herman Sheremetyev wrote: > A multi-line solution first requires a user-defined class. And I feel > like we really shouldn't have to define a class just to create an > object that we throw away two lines later. Defining classes isn't a big deal. It's just fancy syntax for calling the metaclass (as the equivalent code based on a direct call to type shows). If it really bothers you, hide it in a helper function: def anon_object(**class_attrs): class Anon(object): pass Anon.__dict__.update(class_attrs) return Anon() def anon_object(**class_attrs): return type('Anon', (), class_attrs)() Or you could have a bit of mercy on the people that are going to have to maintain this code and give the class a meaningful name that will be displayed in tracebacks. Maybe go crazy and do something radical like give it a docstring. The person lost in the maze of twisty anonymous objects, strangely alike, may be you in 6 months time. > Using type()'s current > totally unwieldy API to do such an extremely simple and common thing > is again not really a good answer IMO. What you propose is not common. You have presented no use cases other than mock objects, and you have stated that your own proposal is inadequate for that use case. Anonymous objects trade a little bit of typing and thought for the code author for the loss of meaningful information that could benefit future maintainers. There is absolutely zero reason to add additional complexity to the language core or the standard library for such a niche (and questionable) use case when a simple wrapper function around type can do the job. Regards, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From lyricconch at gmail.com Tue Jul 26 02:24:45 2011 From: lyricconch at gmail.com (=?UTF-8?B?5rW36Z+1?=) Date: Tue, 26 Jul 2011 08:24:45 +0800 Subject: [Python-ideas] use "as" for block scope support In-Reply-To: <4E2DF7D6.5000706@pearwood.info> References: <4E2DF7D6.5000706@pearwood.info> Message-ID: 2011/7/26 Steven D'Aprano : > ?? wrote: >> >> 1. >> but what about ( t * s + t / s )? temp var always keep in memory. >> (should not reduce ( t + s ) to ( 2 * x ) since x, y may not a number. >> any operator here should be conside as independent function. ) > > What about it? The temporary variables will go out of scope when the > function exists, and be garbage collected, the same as every other local > variable. > > >> 2. you got an extra lambda, that means: >> In runtime: >> lambda = MAKE_FUNCTION/MAKE_CLOSURE = PyFunction_New >> lambda call = CALL_FUNCTION = PyFrame_New + PyEval_EvalFrameEx >> all of them are heavy, (you can check the source code) >> but: >> as = PyCell_New >> as enter = SETUP_FINALLY = PyFrame_BlockSetup >> as leave = END_FINALLY = PyFrame_BlockPop >> which is much much much more more more cheaper than above. > > Sounds like premature optimization to me. I would like to see actual, > real-world code where the performance bottleneck is setting up functions. > > >> 3. >> try convert this to the lambda form as mention: <1> >> R= lambda x, y: (x+y as t, x-y as s)(t * s as u, t/s as v)(u << v as >> p, u>>v as q) p ** q > > I wouldn't even *try* to convert that to a lambda. That is ugly, > unmaintainable code. To understand it, I had to first write it out in block > form: <1> A = (x + y as t, x - y as s)(t * s as u, t / s as v)(u << v as p, u >> v as q) p ** q <2> t, s = x+y, x-y u, v = t*s, t/s p, q = u<>v A =p**q is it really ugly? why i feel pretty = =? Stage: first time we read <1> <2> <1> evaluation from left to right all the time. add -> as -> sub -> as -> enter -> mul -> as -> div -> as -> enter -> shl -> as -> shr -> as -> enter -> pow -> leaveall -> return note that in most case there are no dupilcate NAMEs. so enter can be igorn. just keep moving temporary things cleanup by leaveall.... no need to care about context, no one can read them, and they affect nothing <2> evaluation step by step and direction change every time add -> sub <- assign \/ mul -> div <- assign \/ shl -> shr -> assign \/ pow return should take some mind of context. it's possible that somewhere read the temporary var Stage: not the first time <1> calcing A; that is all, move next; <2> calc t,s; calc u,v; calc p,q; calc A; move next; step1-3 is useless. > > def R(x, y): > with x+y as t, x-y as s: > with t*s as u, t/s as v: > with u << v as p, u >> v as q > return p**q > > > which is much simpler to read and maintain when written like this: > <2> > def R(x, y): > t, s = x+y, x-y > u, v = t*s, t/s > p, q = u << v, u >> v > return p**q > > > I consider this proposal to encourage excess and unnecessary encapsulation > of variables. Not every expression needs to be it's own block. > > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From eric at trueblade.com Mon Jul 25 20:08:33 2011 From: eric at trueblade.com (Eric V. Smith) Date: Mon, 25 Jul 2011 14:08:33 -0400 Subject: [Python-ideas] anonymous object support In-Reply-To: References: <4E2D7FEB.6080405@trueblade.com> Message-ID: <4E2DB121.3040206@trueblade.com> On 07/25/2011 12:51 PM, Georg Brandl wrote: > Am 25.07.2011 16:38, schrieb Eric V. Smith: >> On 07/25/2011 05:44 AM, Herman Sheremetyev wrote: >>> Giving those first two positional arguments default values (empty >>> string and empty tuple?) would make it even better: >>> >>> obj = type(foo=lambda self, x: x)() >>> >>> That's only one set of parens away from my initial proposal, but I >>> think it's close enough. Would there be any objections to changing the >>> type() API to support this? >> >> -1 on changing type(). Just add this to your code: >> >>>>> def mytype(**kwargs): >> ... return type('', (), kwargs)() >> >> Then you can get rid of the extra parens: >> >>>>> obj=mytype(foo=lambda self, x: x) >>>>> obj.foo(3) >> 3 > > And if you call it anonymous_object() instead of mytype(), you have the > advantage of more readable code as well ;) Yes, yes :). In my defense, moving the call into "mytype" was a last minute change. Eric. From herman at swebpage.com Tue Jul 26 04:30:51 2011 From: herman at swebpage.com (Herman Sheremetyev) Date: Tue, 26 Jul 2011 11:30:51 +0900 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: On Tue, Jul 26, 2011 at 9:18 AM, Nick Coghlan wrote: > There is absolutely zero reason to add additional complexity to the > language core or the standard library for such a niche (and > questionable) use case when a simple wrapper function around type can > do the job. As you yourself pointed out, the complexity is already there in the core language hiding in a single line of API description to the type() function. It's already there to be used, but provides very little documentation and an API that is sure to baffle anyone that comes across code that uses it. To reiterate, I am proposing improving the type() API to provide some reasonable defaults while retaining backwards compatibility. In other words, making an *existing* difficult-to-use API into an intuitive one. FWIW, I think the type() function is not really a great choice for making classes on the fly. But if that's what we have to use then let's at least make it a little more user-friendly. -Herman From guido at python.org Tue Jul 26 04:42:46 2011 From: guido at python.org (Guido van Rossum) Date: Mon, 25 Jul 2011 19:42:46 -0700 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: On Mon, Jul 25, 2011 at 7:30 PM, Herman Sheremetyev wrote: > On Tue, Jul 26, 2011 at 9:18 AM, Nick Coghlan wrote: > >> There is absolutely zero reason to add additional complexity to the >> language core or the standard library for such a niche (and >> questionable) use case when a simple wrapper function around type can >> do the job. > > As you yourself pointed out, the complexity is already there in the > core language hiding in a single line of API description to the type() > function. It's already there to be used, but provides very little > documentation and an API that is sure to baffle anyone that comes > across code that uses it. > > To reiterate, I am proposing improving the type() API to provide some > reasonable defaults while retaining backwards compatibility. Can you summarize the proposal for those who didn't follow the thread blow-by-blow? Your first message proposed keyword arguments to object(), which can't fly because object() creates objects without a __dict__. Now you suddenly seem to have switched to adding more complexity to type(). But what exactly? > In other > words, making an *existing* difficult-to-use API into an intuitive > one. > > FWIW, I think the type() function is not really a great choice for > making classes on the fly. But if that's what we have to use then > let's at least make it a little more user-friendly. So you are proposing to make an undesirable API more user-friendly. Isn't that creating an attractive nuisance? -- --Guido van Rossum (python.org/~guido) From herman at swebpage.com Tue Jul 26 04:58:22 2011 From: herman at swebpage.com (Herman Sheremetyev) Date: Tue, 26 Jul 2011 11:58:22 +0900 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: On Tue, Jul 26, 2011 at 11:42 AM, Guido van Rossum wrote: > On Mon, Jul 25, 2011 at 7:30 PM, Herman Sheremetyev wrote: >> On Tue, Jul 26, 2011 at 9:18 AM, Nick Coghlan wrote: >> >>> There is absolutely zero reason to add additional complexity to the >>> language core or the standard library for such a niche (and >>> questionable) use case when a simple wrapper function around type can >>> do the job. >> >> As you yourself pointed out, the complexity is already there in the >> core language hiding in a single line of API description to the type() >> function. It's already there to be used, but provides very little >> documentation and an API that is sure to baffle anyone that comes >> across code that uses it. >> >> To reiterate, I am proposing improving the type() API to provide some >> reasonable defaults while retaining backwards compatibility. > > Can you summarize the proposal for those who didn't follow the thread > blow-by-blow? Your first message proposed keyword arguments to > object(), which can't fly because object() creates objects without a > __dict__. Now you suddenly seem to have switched to adding more > complexity to type(). But what exactly? So the initial solution with type() posted above was: obj = type('Foo', (), dict(foo=(lambda x: x)))() Adding the necessary "self" to the lambda, using an empty name, and replacing dict() with {} makes it slightly easier to parse: obj = type('', (), {'foo': lambda self, x: x})() [here are the API changes I propose] Going an extra step and making it possible for the dictionary to be passed in as keyword arguments would make it even nicer: obj = type('', (), foo=lambda self, x: x)() And going one final step to give those first two positional arguments default values (empty string and empty tuple? or "Anonymous" and empty tuple?) would make it even better: obj = type(foo=lambda self, x: x)() >> In other >> words, making an *existing* difficult-to-use API into an intuitive >> one. >> >> FWIW, I think the type() function is not really a great choice for >> making classes on the fly. But if that's what we have to use then >> let's at least make it a little more user-friendly. > > So you are proposing to make an undesirable API more user-friendly. > Isn't that creating an attractive nuisance? Well, I think it's not very intuitive that type() can be used as a class constructor. But it is what it is at this stage, so if I come across it in code or write it myself I'd rather it provided some reasonable defaults. -Herman From pydanny at gmail.com Tue Jul 26 05:29:17 2011 From: pydanny at gmail.com (Daniel Greenfeld) Date: Mon, 25 Jul 2011 20:29:17 -0700 Subject: [Python-ideas] use "as" for block scope support In-Reply-To: References: <4E2DF7D6.5000706@pearwood.info> Message-ID: As a 'Joe Developer', someone not a computer scientist but your 'average user', I'm going to have to step in and agree that the proposed syntax is *much* harder to read. I could follow Steven's code but the other code comes across as obfuscated. On Mon, Jul 25, 2011 at 5:24 PM, ?? wrote: > 2011/7/26 Steven D'Aprano : >> ?? wrote: >>> >>> 1. >>> but what about ( t * s + t / s )? temp var always keep in memory. >>> (should not reduce ( t + s ) to ( 2 * x ) since x, y may not a number. >>> any operator here should be conside as independent function. ) >> >> What about it? The temporary variables will go out of scope when the >> function exists, and be garbage collected, the same as every other local >> variable. >> >> >>> 2. you got an extra lambda, that means: >>> In runtime: >>> lambda = MAKE_FUNCTION/MAKE_CLOSURE = PyFunction_New >>> lambda call = CALL_FUNCTION = PyFrame_New + PyEval_EvalFrameEx >>> all of them are heavy, (you can check the source code) >>> but: >>> as = PyCell_New >>> as enter = SETUP_FINALLY = PyFrame_BlockSetup >>> as leave = END_FINALLY = PyFrame_BlockPop >>> which is much much much more more more cheaper than above. >> >> Sounds like premature optimization to me. I would like to see actual, >> real-world code where the performance bottleneck is setting up functions. >> >> >>> 3. >>> try convert this to the lambda form as mention: > <1> >>> R= lambda x, y: (x+y as t, x-y as s)(t * s as u, t/s as v)(u << v as >>> p, u>>v as q) p ** q >> >> I wouldn't even *try* to convert that to a lambda. That is ugly, >> unmaintainable code. To understand it, I had to first write it out in block >> form: > <1> > A = (x + y as t, x - y as s)(t * s as u, t / s as v)(u << v as p, u >>> v as q) p ** q > <2> > t, s = x+y, x-y > u, v = t*s, t/s > p, q = u<>v > A =p**q > is it really ugly? why i feel pretty = =? > > Stage: first time we read <1> <2> > <1> evaluation from left to right all the time. > add -> as -> sub -> as -> enter -> mul -> as -> div -> as -> enter -> > shl -> as -> shr -> as -> enter -> pow -> leaveall -> return > note that in most case there are no dupilcate NAMEs. > so enter can be igorn. just keep moving > temporary things cleanup by leaveall.... > no need to care about context, no one can read them, and they affect nothing > <2> evaluation step by step and direction change every time > add -> sub <- assign \/ mul -> div <- assign \/ shl -> shr -> assign > \/ pow return > should take some mind of context. it's possible that somewhere read > the temporary var > > Stage: not the first time > <1> calcing A; that is all, move next; > <2> calc t,s; calc u,v; calc p,q; calc A; move next; step1-3 is useless. > >> >> def R(x, y): >> with x+y as t, x-y as s: >> with t*s as u, t/s as v: >> with u << v as p, u >> v as q >> return p**q >> >> >> which is much simpler to read and maintain when written like this: >> > <2> >> def R(x, y): >> t, s = x+y, x-y >> u, v = t*s, t/s >> p, q = u << v, u >> v >> return p**q >> >> >> I consider this proposal to encourage excess and unnecessary encapsulation >> of variables. Not every expression needs to be it's own block. >> >> >> -- >> Steven >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- 'Knowledge is Power' Daniel Greenfeld http://pydanny.blogspot.com http://cartwheelweb.com From guido at python.org Tue Jul 26 05:30:50 2011 From: guido at python.org (Guido van Rossum) Date: Mon, 25 Jul 2011 20:30:50 -0700 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: On Mon, Jul 25, 2011 at 7:58 PM, Herman Sheremetyev wrote: > On Tue, Jul 26, 2011 at 11:42 AM, Guido van Rossum wrote: >> On Mon, Jul 25, 2011 at 7:30 PM, Herman Sheremetyev wrote: >>> On Tue, Jul 26, 2011 at 9:18 AM, Nick Coghlan wrote: >>> >>>> There is absolutely zero reason to add additional complexity to the >>>> language core or the standard library for such a niche (and >>>> questionable) use case when a simple wrapper function around type can >>>> do the job. >>> >>> As you yourself pointed out, the complexity is already there in the >>> core language hiding in a single line of API description to the type() >>> function. It's already there to be used, but provides very little >>> documentation and an API that is sure to baffle anyone that comes >>> across code that uses it. >>> >>> To reiterate, I am proposing improving the type() API to provide some >>> reasonable defaults while retaining backwards compatibility. >> >> Can you summarize the proposal for those who didn't follow the thread >> blow-by-blow? Your first message proposed keyword arguments to >> object(), which can't fly because object() creates objects without a >> __dict__. Now you suddenly seem to have switched to adding more >> complexity to type(). But what exactly? > > So the initial solution with type() posted above was: > > ? ?obj = type('Foo', (), dict(foo=(lambda x: x)))() > > Adding the necessary "self" to the lambda, using an empty name, and > replacing dict() with {} makes it slightly easier to parse: > > ? ?obj = type('', (), {'foo': lambda self, x: x})() Oh, that is bad. A single class object costs many thousands of bytes in overhead and contains lots of cycles so GC needs to come along to clean it up. And all that to create a single tiny object? > [here are the API changes I propose] > > Going an extra step and making it possible for the dictionary to be passed in as > keyword arguments would make it even nicer: > > ? ?obj = type('', (), foo=lambda self, x: x)() Same problem. > And going one final step to give those first two positional arguments > default values (empty > string and empty tuple? or "Anonymous" and empty tuple?) would make it > even better: > > ? ?obj = type(foo=lambda self, x: x)() I recommend that you create a library module that implements this more efficiently and then see how often you *really* use it. >>> In other >>> words, making an *existing* difficult-to-use API into an intuitive >>> one. >>> >>> FWIW, I think the type() function is not really a great choice for >>> making classes on the fly. But if that's what we have to use then >>> let's at least make it a little more user-friendly. >> >> So you are proposing to make an undesirable API more user-friendly. >> Isn't that creating an attractive nuisance? > > Well, I think it's not very intuitive that type() can be used as a > class constructor. There's a way of looking at that which makes it very logical, and intuition is learned. (This is counter-intuitive, but true nevertheless. :-) > But it is what it is at this stage, so if I come > across it in code or write it myself I'd rather it provided some > reasonable defaults. My expectation is that you'll rarely come across a use case where the keyword arguments are actually useful. Plus, creating new dynamic class objects willy-nilly is hugely inefficient. -- --Guido van Rossum (python.org/~guido) From guido at python.org Tue Jul 26 05:32:25 2011 From: guido at python.org (Guido van Rossum) Date: Mon, 25 Jul 2011 20:32:25 -0700 Subject: [Python-ideas] use "as" for block scope support In-Reply-To: References: <4E2DF7D6.5000706@pearwood.info> Message-ID: Right. I have always felt it important that Python remain usable for 'Joe Developer'. 2011/7/25 Daniel Greenfeld : > As a 'Joe Developer', someone not a computer scientist but your > 'average user', I'm going to have to step in and agree that the > proposed syntax is *much* harder to read. I could follow Steven's code > but the other code comes across as obfuscated. > > On Mon, Jul 25, 2011 at 5:24 PM, ?? wrote: >> 2011/7/26 Steven D'Aprano : >>> ?? wrote: >>>> >>>> 1. >>>> but what about ( t * s + t / s )? temp var always keep in memory. >>>> (should not reduce ( t + s ) to ( 2 * x ) since x, y may not a number. >>>> any operator here should be conside as independent function. ) >>> >>> What about it? The temporary variables will go out of scope when the >>> function exists, and be garbage collected, the same as every other local >>> variable. >>> >>> >>>> 2. you got an extra lambda, that means: >>>> In runtime: >>>> lambda = MAKE_FUNCTION/MAKE_CLOSURE = PyFunction_New >>>> lambda call = CALL_FUNCTION = PyFrame_New + PyEval_EvalFrameEx >>>> all of them are heavy, (you can check the source code) >>>> but: >>>> as = PyCell_New >>>> as enter = SETUP_FINALLY = PyFrame_BlockSetup >>>> as leave = END_FINALLY = PyFrame_BlockPop >>>> which is much much much more more more cheaper than above. >>> >>> Sounds like premature optimization to me. I would like to see actual, >>> real-world code where the performance bottleneck is setting up functions. >>> >>> >>>> 3. >>>> try convert this to the lambda form as mention: >> <1> >>>> R= lambda x, y: (x+y as t, x-y as s)(t * s as u, t/s as v)(u << v as >>>> p, u>>v as q) p ** q >>> >>> I wouldn't even *try* to convert that to a lambda. That is ugly, >>> unmaintainable code. To understand it, I had to first write it out in block >>> form: >> <1> >> A = (x + y as t, x - y as s)(t * s as u, t / s as v)(u << v as p, u >>>> v as q) p ** q >> <2> >> t, s = x+y, x-y >> u, v = t*s, t/s >> p, q = u<>v >> A =p**q >> is it really ugly? why i feel pretty = =? >> >> Stage: first time we read <1> <2> >> <1> evaluation from left to right all the time. >> add -> as -> sub -> as -> enter -> mul -> as -> div -> as -> enter -> >> shl -> as -> shr -> as -> enter -> pow -> leaveall -> return >> note that in most case there are no dupilcate NAMEs. >> so enter can be igorn. just keep moving >> temporary things cleanup by leaveall.... >> no need to care about context, no one can read them, and they affect nothing >> <2> evaluation step by step and direction change every time >> add -> sub <- assign \/ mul -> div <- assign \/ shl -> shr -> assign >> \/ pow return >> should take some mind of context. it's possible that somewhere read >> the temporary var >> >> Stage: not the first time >> <1> calcing A; that is all, move next; >> <2> calc t,s; calc u,v; calc p,q; calc A; move next; step1-3 is useless. >> >>> >>> def R(x, y): >>> with x+y as t, x-y as s: >>> with t*s as u, t/s as v: >>> with u << v as p, u >> v as q >>> return p**q >>> >>> >>> which is much simpler to read and maintain when written like this: >>> >> <2> >>> def R(x, y): >>> t, s = x+y, x-y >>> u, v = t*s, t/s >>> p, q = u << v, u >> v >>> return p**q >>> >>> >>> I consider this proposal to encourage excess and unnecessary encapsulation >>> of variables. Not every expression needs to be it's own block. >>> >>> >>> -- >>> Steven >>> _______________________________________________ >>> Python-ideas mailing list >>> Python-ideas at python.org >>> http://mail.python.org/mailman/listinfo/python-ideas >>> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> > > > > -- > 'Knowledge is Power' > Daniel Greenfeld > http://pydanny.blogspot.com > http://cartwheelweb.com > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- --Guido van Rossum (python.org/~guido) From ncoghlan at gmail.com Tue Jul 26 06:06:00 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 26 Jul 2011 14:06:00 +1000 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: On Tue, Jul 26, 2011 at 12:58 PM, Herman Sheremetyev wrote: > Well, I think it's not very intuitive that type() can be used as a > class constructor. Then you don't really understand metaclasses yet (now, metaclasses themselves certainly don't qualify as intuitive, but understanding them is essential to understanding many otherwise confusing elements of Python's type system). type() works the way it does because it is designed to construct a new class object from a name, a tuple of base classes and a namespace dictionary, just like any other metaclass. The slightly hacky (but very convenient) part is actually the ability to use the single argument form of type() to query the metaclass of an existing object. To get back to the ideas discussed in this thread: 1. Use cases where creating a lot of one-shot classes on the fly is a reasonable idea are rare and highly specialised 2. If you want to do so, the obvious way is to use a class statement and adjust __name__ and __dict__ afterwards 3. Alternatively, if you want this functionality as an expression, either the type builtin or a custom function can do the job 4. It is significantly more efficient to abstract out a common class definition and create multiple instances of that class 5. If you just want a namespace to store some data with consistent attribute names, then collections.namedtuple can be used to also make the instance storage efficient. If you store a function in a namedtuple instance then you can still call it, it just won't be passed 'self' as an implicit first parameter (as the descriptor machinery will not be invoked). 6. If you just want an arbitrary attribute holder, than instances of a trivial empty user-defined class can serve that purpose. As with named tuples, functions stored on the instance can still be called, but retrieving them won't invoke the descriptor machinery. 7. Mock objects do *not* qualify as a common use case. Despite the proliferation of libraries that choose to provide slightly different APIs for them, such objects may be used widely in test suites but are still *implemented* rarely - they fall into the "highly specialised" category I mentioned above. The tools are there so that people that know what they're doing can do what needs to be done to achieve their ends. Simplifying the API for dynamic type creation would be an attractive nuisance, as it will likely lead to people writing ill-advised code without giving the matter sufficient thought. Obviously, that's going to happen no matter what we do, but there's no reason for us to deliberately *encourage* bad ideas by making them too easy to write. Points 5 and 6 go back to the original question about namespace objects, which is independent of the dynamic type creation issue. Various attempts have been made to address that over the years, and the current answers are the original approach (just define a namespace type and use instances of it to store arbitrary data, with poor debugging support due to the relative anonymity and arbitrary content of each instance) and the named tuple approach, which provides excellent debugging information and memory efficiency at the cost of being explicit up front regarding the attributes possessed by each instance. Regards, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From anacrolix at gmail.com Tue Jul 26 06:36:26 2011 From: anacrolix at gmail.com (Matt Joiner) Date: Tue, 26 Jul 2011 14:36:26 +1000 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: Just as a slight aside, is the special case of type() purely for convenience? Aren't the "new-style" ways to do this __class__ and isinstance()? On Tue, Jul 26, 2011 at 2:06 PM, Nick Coghlan wrote: > On Tue, Jul 26, 2011 at 12:58 PM, Herman Sheremetyev > wrote: >> Well, I think it's not very intuitive that type() can be used as a >> class constructor. > > Then you don't really understand metaclasses yet (now, metaclasses > themselves certainly don't qualify as intuitive, but understanding > them is essential to understanding many otherwise confusing elements > of Python's type system). type() works the way it does because it is > designed to construct a new class object from a name, a tuple of base > classes and a namespace dictionary, just like any other metaclass. The > slightly hacky (but very convenient) part is actually the ability to > use the single argument form of type() to query the metaclass of an > existing object. > > To get back to the ideas discussed in this thread: > > 1. Use cases where creating a lot of one-shot classes on the fly is a > reasonable idea are rare and highly specialised > 2. If you want to do so, the obvious way is to use a class statement > and adjust __name__ and __dict__ afterwards > 3. Alternatively, if you want this functionality as an expression, > either the type builtin or a custom function can do the job > 4. It is significantly more efficient to abstract out a common class > definition and create multiple instances of that class > 5. If you just want a namespace to store some data with consistent > attribute names, then collections.namedtuple can be used to also make > the instance storage efficient. If you store a function in a > namedtuple instance then you can still call it, it just won't be > passed 'self' as an implicit first parameter (as the descriptor > machinery will not be invoked). > 6. If you just want an arbitrary attribute holder, than instances of a > trivial empty user-defined class can serve that purpose. As with named > tuples, functions stored on the instance can still be called, but > retrieving them won't invoke the descriptor machinery. > 7. Mock objects do *not* qualify as a common use case. Despite the > proliferation of libraries that choose to provide slightly different > APIs for them, such objects may be used widely in test suites but are > still *implemented* rarely - they fall into the "highly specialised" > category I mentioned above. > > The tools are there so that people that know what they're doing can do > what needs to be done to achieve their ends. Simplifying the API for > dynamic type creation would be an attractive nuisance, as it will > likely lead to people writing ill-advised code without giving the > matter sufficient thought. Obviously, that's going to happen no matter > what we do, but there's no reason for us to deliberately *encourage* > bad ideas by making them too easy to write. > > Points 5 and 6 go back to the original question about namespace > objects, which is independent of the dynamic type creation issue. > Various attempts have been made to address that over the years, and > the current answers are the original approach (just define a namespace > type and use instances of it to store arbitrary data, with poor > debugging support due to the relative anonymity and arbitrary content > of each instance) and the named tuple approach, which provides > excellent debugging information and memory efficiency at the cost of > being explicit up front regarding the attributes possessed by each > instance. > > Regards, > Nick. > > -- > Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From ncoghlan at gmail.com Tue Jul 26 07:50:13 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 26 Jul 2011 15:50:13 +1000 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: On Tue, Jul 26, 2011 at 2:36 PM, Matt Joiner wrote: > Just as a slight aside, is the special case of type() purely for convenience? > Aren't the "new-style" ways to do this __class__ and isinstance()? Not really - all three do different things: 1. type(x) always gives you the true metaclass of x 2. x.__class__ tells you what x claims to be (which may differ from type(x) for things like proxy objects) 3. isinstance(x, cls) doesn't tell you what x actually *is*, just whether or not it passes the isinstance check (it may be an instance of a subclass, proxying for another object, or just registered if cls refers to an ABC) There has to be *some* way to spell number 1 and Guido chose to use type() for it. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From masklinn at masklinn.net Tue Jul 26 09:20:44 2011 From: masklinn at masklinn.net (Masklinn) Date: Tue, 26 Jul 2011 09:20:44 +0200 Subject: [Python-ideas] anonymous object support In-Reply-To: References: Message-ID: <497569F6-6F98-49C9-85DF-C21D67F5CF0D@masklinn.net> On 2011-07-26, at 07:50 , Nick Coghlan wrote: > On Tue, Jul 26, 2011 at 2:36 PM, Matt Joiner wrote: >> Just as a slight aside, is the special case of type() purely for convenience? >> Aren't the "new-style" ways to do this __class__ and isinstance()? > > Not really - all three do different things: > > 1. type(x) always gives you the true metaclass of x The true *class*, I think. Although type(type(x)) should give you x's metaclass. From fuzzyman at gmail.com Tue Jul 26 16:42:10 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Tue, 26 Jul 2011 15:42:10 +0100 Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library In-Reply-To: References: Message-ID: On 25 July 2011 23:39, Maxim Khitrov wrote: > On Mon, Jul 25, 2011 at 4:27 PM, Michael Foord wrote: > > > > > > On 25 July 2011 20:38, Maxim Khitrov wrote: > >> > >> On Mon, Jul 25, 2011 at 3:21 PM, Michael Foord > wrote: > >> > > >> > > >> > On 25 July 2011 02:06, Maxim Khitrov wrote: > >> >> > >> >> Hello python-ideas, > >> >> > >> >> My most recent project lead me down a path that eventually ended up > at > >> >> a new implementation of imaplib based on [RFC-3501]. Although I > >> >> started the project by gradually adding functionality to the existing > >> >> IMAP4 library, some of the features that I required simply could not > >> >> be merged in (without breaking everything). As a result, I wrote my > >> >> own version of the library, which incorporates all existing > >> >> functionality of imaplib and includes many of my own improvements. > >> >> > >> > > >> > There is an existing, well tested and widely used, replaced for > imaplib > >> > that > >> > I would suggest should be the first for consideration in replacing > >> > imaplib: > >> > > >> > http://imapclient.freshfoo.com/ > >> > > >> > (Sorry.) > >> > > >> > All the best, > >> > > >> > Michael Foord > >> > >> I have it beat at the "Python 3 support is in the works" feature ;) > >> Mine doesn't handle 2.x though. > >> > >> In any case, I would not have been able to use IMAPClient for my > >> project, because the requirements were for no dependencies outside of > >> Python 3.2. > >> > >> Do you know if the developers of IMAPClient considered getting it into > >> the standard library? My goal wasn't just to have another IMAP > >> implementation, but something better available as part of Python. > >> > > > > > > I don't think Menno Smitts would object to adding Python 3 support or > adding > > IMAPClient to the standard library. His goal was to create something > useful > > to overcome what he saw (and evidently you agree) as irreparable > brokenness > > in parts of imaplib. > > > > My point is that if there is an existing widely-used and battle-tested > > alternative, we would be wise to look at that first. > > Agreed. However, I just took a look through IMAPClient source code and > have to correct you on your original assertion. IMAPClient is not a > replacement for imaplib, but a wrapper around it. > > The author went down the same road that I originally started out on. > He added a parser, a UTF-7 codec, and changed the overall interface to > be more user-friendly. All good improvements, but at the core, > IMAPClient relies entirely on imaplib, and thus inherits most of its > design flaws. > > Well I'm sure Menno would be interested if you could actually demonstrate those limitations rather than merely assert that. :-) (As he implemented to overcome problems with imaplib and that is why people use it.) Not that you're necessarily wrong, but I'm skeptical. Michael > With the exception of the first and fifth bullet points in my README > file (server response parser & UTF-7 mailbox name codec, > respectively), all others apply equally to IMAPClient. For that > reason, I would recommend against incorporating it into the standard > library. > > My opinion on the matter is certainly biased, so I would welcome a > review of both libraries by a neutral party familiar with the IMAP > protocol, who could then make a recommendation. > > - Max > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From anacrolix at gmail.com Tue Jul 26 17:02:59 2011 From: anacrolix at gmail.com (Matt Joiner) Date: Wed, 27 Jul 2011 01:02:59 +1000 Subject: [Python-ideas] anonymous object support In-Reply-To: <497569F6-6F98-49C9-85DF-C21D67F5CF0D@masklinn.net> References: <497569F6-6F98-49C9-85DF-C21D67F5CF0D@masklinn.net> Message-ID: Thanks both, consider me undoubted. On Jul 26, 2011 5:20 PM, "Masklinn" wrote: > > On 2011-07-26, at 07:50 , Nick Coghlan wrote: > >> On Tue, Jul 26, 2011 at 2:36 PM, Matt Joiner wrote: >>> Just as a slight aside, is the special case of type() purely for convenience? >>> Aren't the "new-style" ways to do this __class__ and isinstance()? >> >> Not really - all three do different things: >> >> 1. type(x) always gives you the true metaclass of x > The true *class*, I think. Although type(type(x)) should give you x's metaclass. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max at mxcrypt.com Tue Jul 26 18:54:01 2011 From: max at mxcrypt.com (Maxim Khitrov) Date: Tue, 26 Jul 2011 12:54:01 -0400 Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library In-Reply-To: References: Message-ID: On Tue, Jul 26, 2011 at 10:42 AM, Michael Foord wrote: > On 25 July 2011 23:39, Maxim Khitrov wrote: >> On Mon, Jul 25, 2011 at 4:27 PM, Michael Foord wrote: >> > On 25 July 2011 20:38, Maxim Khitrov wrote: >> >> On Mon, Jul 25, 2011 at 3:21 PM, Michael Foord >> >> wrote: >> >> > There is an existing, well tested and widely used, replaced for >> >> > imaplib >> >> > that >> >> > I would suggest should be the first for consideration in replacing >> >> > imaplib: >> >> > >> >> > http://imapclient.freshfoo.com/ >> >> > >> >> > (Sorry.) >> >> > >> >> > All the best, >> >> > >> >> > Michael Foord >> >> >> >> I have it beat at the "Python 3 support is in the works" feature ;) >> >> Mine doesn't handle 2.x though. >> >> >> >> In any case, I would not have been able to use IMAPClient for my >> >> project, because the requirements were for no dependencies outside of >> >> Python 3.2. >> >> >> >> Do you know if the developers of IMAPClient considered getting it into >> >> the standard library? My goal wasn't just to have another IMAP >> >> implementation, but something better available as part of Python. >> >> >> > >> > >> > I don't think Menno Smitts would object to adding Python 3 support or >> > adding >> > IMAPClient to the standard library. His goal was to create something >> > useful >> > to overcome what he saw (and evidently you agree) as irreparable >> > brokenness >> > in parts of imaplib. >> > >> > My point is that if there is an existing widely-used and battle-tested >> > alternative, we would be wise to look at that first. >> >> Agreed. However, I just took a look through IMAPClient source code and >> have to correct you on your original assertion. IMAPClient is not a >> replacement for imaplib, but a wrapper around it. >> >> The author went down the same road that I originally started out on. >> He added a parser, a UTF-7 codec, and changed the overall interface to >> be more user-friendly. All good improvements, but at the core, >> IMAPClient relies entirely on imaplib, and thus inherits most of its >> design flaws. >> > > Well I'm sure Menno would be interested if you could actually demonstrate > those limitations rather than merely assert that. :-) > > (As he implemented to overcome problems with imaplib and that is why people > use it.) > > Not that you're necessarily wrong, but I'm skeptical. Most of the limitations are outlined in my README file. I'm not sure what kind of a demonstration would convince you. The problem is that if your applications do not require things like data compression, asynchronous/parallel command execution, or multiple literals in commands, you will not agree that there is anything wrong with the existing tools. It's very much a matter of personal experience. One of the things I'm trying to address with my library is strict adherence to the current version of the IMAP4 protocol. The other is performance; hence the implementation of extensions such as SASL-IR, IDLE, non-synchronizing literals, multiappend, and compression. On the performance side, if you have an application that's trying do some sort of processing of a 6 GB mailbox with 700,000 messages in it, executing a separate FETCH command for each message will take you a week to finish. If you try to be clever and FETCH 1000 messages at a time, for example, you'll quickly run into a few problems: 1. The data for all 1000 messages is buffered in memory before you ever to get to see anything. 2. If the server fails to retrieve one of those messages, which actually happens rather frequently, at least on Gmail servers, you lose the data for the other 999. You'll get a 'NO' response from imaplib or an exception from IMAPClient, and an explanation of what went wrong instead of any data. 3. If you requested data for messages 1:1000 and the server decides to send you a FLAGS update for message 9999, imaplib will return that response as part of the data for the original request. All are interface design problems, which are inherited by IMAPClient. - Max P.S. It is not my intention to discourage the use of IMAPClient in any way. Its existence is a good thing for 99% of the users, because it does address a number of key imaplib issues with just the response parser and a UTF-7 codec. My point is that there are real-world use cases out there that cannot be handled by imaplib or IMAPClient, and for those, I'm offering my library as a more general solution that should satisfy the remaining 1% :) From tjreedy at udel.edu Tue Jul 26 20:18:11 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 26 Jul 2011 14:18:11 -0400 Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library In-Reply-To: References: Message-ID: On 7/26/2011 12:54 PM, Maxim Khitrov wrote: > On Tue, Jul 26, 2011 at 10:42 AM, Michael Foord wrote: >> Well I'm sure Menno would be interested if you could actually demonstrate >> those limitations rather than merely assert that. :-) >> >> (As he implemented to overcome problems with imaplib and that is why people >> use it.) >> >> Not that you're necessarily wrong, but I'm skeptical. > > Most of the limitations are outlined in my README file. I'm not sure > what kind of a demonstration would convince you. The problem is that > if your applications do not require things like data compression, > asynchronous/parallel command execution, or multiple literals in > commands, you will not agree that there is anything wrong with the > existing tools. It's very much a matter of personal experience. > > One of the things I'm trying to address with my library is strict > adherence to the current version of the IMAP4 protocol. The other is > performance; hence the implementation of extensions such as SASL-IR, > IDLE, non-synchronizing literals, multiappend, and compression. > > On the performance side, if you have an application that's trying do > some sort of processing of a 6 GB mailbox with 700,000 messages in it, > executing a separate FETCH command for each message will take you a > week to finish. If you try to be clever and FETCH 1000 messages at a > time, for example, you'll quickly run into a few problems: > > 1. The data for all 1000 messages is buffered in memory before you > ever to get to see anything. > > 2. If the server fails to retrieve one of those messages, which > actually happens rather frequently, at least on Gmail servers, you > lose the data for the other 999. You'll get a 'NO' response from > imaplib or an exception from IMAPClient, and an explanation of what > went wrong instead of any data. > > 3. If you requested data for messages 1:1000 and the server decides to > send you a FLAGS update for message 9999, imaplib will return that > response as part of the data for the original request. > > All are interface design problems, which are inherited by IMAPClient. > > - Max > > P.S. It is not my intention to discourage the use of IMAPClient in any > way. Its existence is a good thing for 99% of the users, because it > does address a number of key imaplib issues with just the response > parser and a UTF-7 codec. My point is that there are real-world use > cases out there that cannot be handled by imaplib or IMAPClient, and > for those, I'm offering my library as a more general solution that > should satisfy the remaining 1% :) I think a proper iterator rather than batch mode fetch interface will benefit more than 1% of people. I also think RFC-based modules should be updated to complete-as-possible-and-sensible implementations of current RFCs that have replaced (in usage) previous RFCs. -- Terry Jan Reedy From brett at python.org Wed Jul 27 04:09:20 2011 From: brett at python.org (Brett Cannon) Date: Tue, 26 Jul 2011 19:09:20 -0700 Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library In-Reply-To: References: Message-ID: On Sun, Jul 24, 2011 at 18:06, Maxim Khitrov wrote: > Hello python-ideas, > > My most recent project lead me down a path that eventually ended up at > a new implementation of imaplib based on [RFC-3501]. Although I > started the project by gradually adding functionality to the existing > IMAP4 library, some of the features that I required simply could not > be merged in (without breaking everything). As a result, I wrote my > own version of the library, which incorporates all existing > functionality of imaplib and includes many of my own improvements. > > I wrote a PEP-style readme file that describes all the details of why > the library was written and how it works, which is available from my > mercurial repository: > > http://hg.mxcrypt.com/python/imaplib2/raw-file/tip/README > > The same repository also contains the library code and an example > script that you can run if you have access to an IMAP4 server: > > http://hg.mxcrypt.com/python/imaplib2/ > > Is there any interest in adding my code to a future version of Python > 3.x standard library? > Since no one has pointed you to it, there is a doc explaining what it takes to get a new module added to the stdlib: http://docs.python.org/devguide/stdlibchanges.html#adding-a-new-module . -------------- next part -------------- An HTML attachment was scrubbed... URL: From max at mxcrypt.com Wed Jul 27 13:48:13 2011 From: max at mxcrypt.com (Maxim Khitrov) Date: Wed, 27 Jul 2011 07:48:13 -0400 Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library In-Reply-To: References: Message-ID: On Tue, Jul 26, 2011 at 10:09 PM, Brett Cannon wrote: > > > On Sun, Jul 24, 2011 at 18:06, Maxim Khitrov wrote: >> >> Hello python-ideas, >> >> My most recent project lead me down a path that eventually ended up at >> a new implementation of imaplib based on [RFC-3501]. Although I >> started the project by gradually adding functionality to the existing >> IMAP4 library, some of the features that I required simply could not >> be merged in (without breaking everything). As a result, I wrote my >> own version of the library, which incorporates all existing >> functionality of imaplib and includes many of my own improvements. >> >> I wrote a PEP-style readme file that describes all the details of why >> the library was written and how it works, which is available from my >> mercurial repository: >> >> http://hg.mxcrypt.com/python/imaplib2/raw-file/tip/README >> >> The same repository also contains the library code and an example >> script that you can run if you have access to an IMAP4 server: >> >> http://hg.mxcrypt.com/python/imaplib2/ >> >> Is there any interest in adding my code to a future version of Python >> 3.x standard library? > > Since no one has pointed you to it, there is a doc explaining what it takes > to get a new module added to the stdlib: > http://docs.python.org/devguide/stdlibchanges.html#adding-a-new-module . Thanks, I actually found and read that page before making my initial announcement. For now, I'm making the module available under the simplified BSD license to anyone who wishes to use it. If it has gained some traction over the next year, I'll be happy to submit it to PSF under Apache License 2.0. Is there anything else I should do in the mean time? How do I go about moving the development into the Python's infrastructure, or does that happen later? Over the next week or so, I'll finish writing the higher-level interface, which actually did end up inheriting from Mailbox, but with some limitations that I'll explain later. After that, I'll put together a test suite and update the documentation. The last remaining bit will be to implement some additional extensions and authentication methods. Don't have anything else planned beyond that. - Max From g.rodola at gmail.com Wed Jul 27 13:57:34 2011 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Wed, 27 Jul 2011 13:57:34 +0200 Subject: [Python-ideas] Enums In-Reply-To: <20110725130621.5e1249d8@resist.wooz.org> References: <20110725130621.5e1249d8@resist.wooz.org> Message-ID: 2011/7/25 Barry Warsaw > On Jul 24, 2011, at 05:33 PM, Guido van Rossum wrote: > > >For enums, I think we should just pick a solution. I'm in favor of > >Barry Warsaw's version, flufl.enum. > > Thanks! I would be quite happy to see the library pulled into the stdlib, > and > I'd be willing to maintain a backward compatible standalone version for > older > Pythons. > > For me, the API has been quite stable. The last thing I added was Michael > Foord's make_enum() contribution, and PJE's fix for pickling. I haven't > felt > a burning need to add anything else really in quite some time. > >>> from flufl.enum import Enum >>> class Colors(Enum): ... red = 1 ... green = 2 ... blue = 3 What I find uncomfortable with this is the extra "Colors" name space. Say we change all the socket module constants to make use of this, what will we have? >>> socket.AF_INET ...and: >>> socket.Constants.AF_INET ...? Is that correct? Also: >>> socket.AF_INET >>> socket.AF_INET == 2 False >>> This shouldn't be underestimated: I've seen code comparing against plain integers rather than constants more than once, and that code would eventually break. Personally I've never felt the need of any kind of "enum" type. On the other hand I would welcome a "named constant" type of some kind but IMO, it should remain a base integer and provide just a minimal extra layer of usability, such as: class constant(int): """A constant type; overrides base int to provide a useful name on str().""" def __new__(cls, value, name, doc=None): inst = super(constant, cls).__new__(cls, value) inst._name = name if doc is not None: inst.__doc__ = doc return inst def __str__(self): return self._name >>> STATUS_RUNNING = constant(0, "running") >>> STATUS_RUNNING 0 >>> str(STATUS_RUNNING) "running" >>> In summary, I'm -1 about including this into the stdlib and change all the stdlib constants in accordarce. My 2 cents. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.rodola at gmail.com Wed Jul 27 14:36:23 2011 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Wed, 27 Jul 2011 14:36:23 +0200 Subject: [Python-ideas] Enums In-Reply-To: References: <1C6CA719-5727-4945-BC88-0CB773022FA3@gmail.com> Message-ID: 2011/7/25 Michael Foord > > > On 25 July 2011 19:47, Raymond Hettinger wrote: > >> >> On Jul 25, 2011, at 1:06 PM, Michael Foord wrote: >> >> >> Python standard library modules currently using integers for constants: >> >> * re - has flags (OR'able constants) defined in sre_constants, each flag >> has two names (e.g. re.IGNORECASE and re.I) >> >> >> What is being proposed here? Will there be a new namespace so that we >> would start writing re.Flags.IGNORECASE instead of re.IGNORECASE? Are >> module level constants now going to be considered bad-form? >> >> If constants switch from re.IGNORECASE to re.flags.IGNORECASE, are there >> any benefits to compensate for being both wordier and slower? >> >> > Nope. Just something like this at the module level. > > IGNORECASE = Flags.IGNORECASE > > What it gains you is a nicer representation when looking at the values when > debugging. On the other hand being able to have access to all the constants > namespaced on a single object (as well) doesn't seem like such a bad thing. > Having aliases all around stdlib module namespaces doesn't sound like a good idea to me at all. "There should be one and preferably one obvious way to do it". That aside, it's not clear what name convention to use that extra "Flags" namespace which wasn't there before. Why re.Flags.* instead of re.Constants.*? If we decide to use re.Flags what should we expect to find into the socket module? socket.Constants? If so, why do we find AF_INET in socket.Constants.AF_INET rather than socket.Family.AF_INET? ...and so on. Also, while before it was clear that module.SOMETHING was referring to a constant, now I that I make a dir() of a module and see "Something" instead of "SOMETHING", the first thing which comes to my mind is that "Something" is a class, not a container of some constants. Regards, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mal at egenix.com Wed Jul 27 14:44:08 2011 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 27 Jul 2011 14:44:08 +0200 Subject: [Python-ideas] Enums In-Reply-To: <23AEE43B-051A-4B96-BF37-733B1806D4F3@gmail.com> References: <23AEE43B-051A-4B96-BF37-733B1806D4F3@gmail.com> Message-ID: <4E300818.10809@egenix.com> Raymond Hettinger wrote: > > On Jul 24, 2011, at 6:19 PM, Jack Diederich wrote: > >> On Sun, Jul 24, 2011 at 8:33 PM, Guido van Rossum wrote: >>> For enums, I think we should just pick a solution. I'm in favor of >>> Barry Warsaw's version, flufl.enum. >> >> I don't care for enums but enough other people do that I wouldn't mind >> including a blessed implemenation. > > I also don't care for enums and recommend against adding them to the language. > IMO, it is something that makes good sense in statically compiled languages > and is unnecessary for us. Not to mention, we already have several ways to do it > (module and class namespaces for example). > > Also, when this idea came up in the past, it tended to get shot down because > the various use cases suggested differing implementations with different > features. > > I also urge caution because Python has already stopped being a small language. > Enums are especially problematic because they will pop up everywhere. > You won't have the option of ignoring them. +1 > Python has achieved an amazing adoption rate without enums. > Most people just don't need them. Many, large and clean apps have > been built without them. Those that have found did were typically > able to implement them easily with the existing toolset. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 27 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From fuzzyman at gmail.com Wed Jul 27 15:17:10 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Wed, 27 Jul 2011 14:17:10 +0100 Subject: [Python-ideas] Enums In-Reply-To: <4E300818.10809@egenix.com> References: <23AEE43B-051A-4B96-BF37-733B1806D4F3@gmail.com> <4E300818.10809@egenix.com> Message-ID: On 27 July 2011 13:44, M.-A. Lemburg wrote: > Raymond Hettinger wrote: > > > > On Jul 24, 2011, at 6:19 PM, Jack Diederich wrote: > > > >> On Sun, Jul 24, 2011 at 8:33 PM, Guido van Rossum > wrote: > >>> For enums, I think we should just pick a solution. I'm in favor of > >>> Barry Warsaw's version, flufl.enum. > >> > >> I don't care for enums but enough other people do that I wouldn't mind > >> including a blessed implemenation. > > > > I also don't care for enums and recommend against adding them to the > language. > > IMO, it is something that makes good sense in statically compiled > languages > > and is unnecessary for us. Not to mention, we already have several ways > to do it > > (module and class namespaces for example). > > > > Also, when this idea came up in the past, it tended to get shot down > because > > the various use cases suggested differing implementations with different > > features. > > > > I also urge caution because Python has already stopped being a small > language. > > Enums are especially problematic because they will pop up everywhere. > > You won't have the option of ignoring them. > > +1 > > Actually, as they should behave exactly like constants you should entirely be able to "ignore" them (to the same extent you ignore the existing constants anyway). Whether they're exposed at the module level, or grouped in a single namespace (the container), or both, is a stylistic issue for module authors. I've seen plenty of Python modules that group constants as class attributes for convenience even without an enum in the standard library. This proposal just makes those more useful and doesn't add any overhead for *using* them. I think a lot of the objections come from perceived negative experience of Enums in other languages (not particularly aimed at you here MAL). When we discussed this on python-dev several developers were much happier with the proposal if we called them NamedConstants and GroupedConstants rather than Enums. Michael > > Python has achieved an amazing adoption rate without enums. > > Most people just don't need them. Many, large and clean apps have > > been built without them. Those that have found did were typically > > able to implement them easily with the existing toolset. > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Jul 27 2011) > >>> Python/Zope Consulting and Support ... http://www.egenix.com/ > >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ > >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > > ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: > > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at gmail.com Wed Jul 27 15:20:13 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Wed, 27 Jul 2011 14:20:13 +0100 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725130621.5e1249d8@resist.wooz.org> Message-ID: On 27 July 2011 12:57, Giampaolo Rodol? wrote: > 2011/7/25 Barry Warsaw > >> On Jul 24, 2011, at 05:33 PM, Guido van Rossum wrote: >> >> >For enums, I think we should just pick a solution. I'm in favor of >> >Barry Warsaw's version, flufl.enum. >> >> Thanks! I would be quite happy to see the library pulled into the stdlib, >> and >> I'd be willing to maintain a backward compatible standalone version for >> older >> Pythons. >> >> For me, the API has been quite stable. The last thing I added was Michael >> Foord's make_enum() contribution, and PJE's fix for pickling. I haven't >> felt >> a burning need to add anything else really in quite some time. >> > > > >>> from flufl.enum import Enum > >>> class Colors(Enum): > ... red = 1 > ... green = 2 > ... blue = 3 > > What I find uncomfortable with this is the extra "Colors" name space. > Say we change all the socket module constants to make use of this, what > will we have? > > >>> socket.AF_INET > > > ...and: > > >>> socket.Constants.AF_INET > > > ...? Is that correct? > > It could be. We can do what what we want. > Also: > > >>> socket.AF_INET > > >>> socket.AF_INET == 2 > False > >>> > > This shouldn't be underestimated: I've seen code comparing against plain > integers rather than constants more than once, and that code would > eventually break. > Immediately break. I agree that this is a problem - new constants should compare equal to their old values if we use them in the standard library. > Personally I've never felt the need of any kind of "enum" type. > On the other hand I would welcome a "named constant" type of some kind but > IMO, it should remain a base integer and provide just a minimal extra layer > of usability, such as: > Sure. Providing grouping for named constants for simpler defining of them is a nice feature though. We should *not* add enums but instead should have named constants and grouped constants. Michael > > class constant(int): > """A constant type; overrides base int to provide a useful name on > str().""" > > def __new__(cls, value, name, doc=None): > inst = super(constant, cls).__new__(cls, value) > inst._name = name > if doc is not None: > inst.__doc__ = doc > return inst > > def __str__(self): > return self._name > > >>> STATUS_RUNNING = constant(0, "running") > >>> STATUS_RUNNING > 0 > >>> str(STATUS_RUNNING) > "running" > >>> > > > In summary, I'm -1 about including this into the stdlib and change all the > stdlib constants in accordarce. > > My 2 cents. > > > --- Giampaolo > http://code.google.com/p/pyftpdlib/ > http://code.google.com/p/psutil/ > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at gmail.com Wed Jul 27 15:13:11 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Wed, 27 Jul 2011 14:13:11 +0100 Subject: [Python-ideas] Enums In-Reply-To: References: <1C6CA719-5727-4945-BC88-0CB773022FA3@gmail.com> Message-ID: On 27 July 2011 13:36, Giampaolo Rodol? wrote: > 2011/7/25 Michael Foord > >> >> >> On 25 July 2011 19:47, Raymond Hettinger wrote: >> >>> >>> On Jul 25, 2011, at 1:06 PM, Michael Foord wrote: >>> >>> >>> Python standard library modules currently using integers for constants: >>> >>> * re - has flags (OR'able constants) defined in sre_constants, each flag >>> has two names (e.g. re.IGNORECASE and re.I) >>> >>> >>> What is being proposed here? Will there be a new namespace so that we >>> would start writing re.Flags.IGNORECASE instead of re.IGNORECASE? Are >>> module level constants now going to be considered bad-form? >>> >>> If constants switch from re.IGNORECASE to re.flags.IGNORECASE, are there >>> any benefits to compensate for being both wordier and slower? >>> >>> >> Nope. Just something like this at the module level. >> >> IGNORECASE = Flags.IGNORECASE >> >> What it gains you is a nicer representation when looking at the values >> when debugging. On the other hand being able to have access to all the >> constants namespaced on a single object (as well) doesn't seem like such a >> bad thing. >> > > Having aliases all around stdlib module namespaces doesn't sound like a > good idea to me at all. > "There should be one and preferably one obvious way to do it". > That aside, it's not clear what name convention to use that extra "Flags" > namespace which wasn't there before. > > Why re.Flags.* instead of re.Constants.*? > I used Flags as an arbitrary name for an example. It could be anything. It could be _Constants if it was decided that the constant group itself wasn't useful. I would expect it to be useful. > If we decide to use re.Flags what should we expect to find into the socket > module? socket.Constants? > If so, why do we find AF_INET in socket.Constants.AF_INET rather > than socket.Family.AF_INET? > > ...and so on. > > We can give it whatever name is most useful. > Also, while before it was clear that module.SOMETHING was referring to a > constant, now I that I make a dir() of a module and see "Something" instead > of "SOMETHING", the first thing which comes to my mind is that "Something" > is a class, not a container of some constants. > Something would be a class. SOMETHING would still exist and should behave as expected. "Something" should in fact have a name that *does* tell you what it does. "Constants" for example gives you a good clue. Michael > > Regards, > > --- Giampaolo > http://code.google.com/p/pyftpdlib/ > http://code.google.com/p/psutil/ > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Wed Jul 27 19:29:10 2011 From: brett at python.org (Brett Cannon) Date: Wed, 27 Jul 2011 10:29:10 -0700 Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library In-Reply-To: References: Message-ID: On Wed, Jul 27, 2011 at 04:48, Maxim Khitrov wrote: > On Tue, Jul 26, 2011 at 10:09 PM, Brett Cannon wrote: > > > > > > On Sun, Jul 24, 2011 at 18:06, Maxim Khitrov wrote: > >> > >> Hello python-ideas, > >> > >> My most recent project lead me down a path that eventually ended up at > >> a new implementation of imaplib based on [RFC-3501]. Although I > >> started the project by gradually adding functionality to the existing > >> IMAP4 library, some of the features that I required simply could not > >> be merged in (without breaking everything). As a result, I wrote my > >> own version of the library, which incorporates all existing > >> functionality of imaplib and includes many of my own improvements. > >> > >> I wrote a PEP-style readme file that describes all the details of why > >> the library was written and how it works, which is available from my > >> mercurial repository: > >> > >> http://hg.mxcrypt.com/python/imaplib2/raw-file/tip/README > >> > >> The same repository also contains the library code and an example > >> script that you can run if you have access to an IMAP4 server: > >> > >> http://hg.mxcrypt.com/python/imaplib2/ > >> > >> Is there any interest in adding my code to a future version of Python > >> 3.x standard library? > > > > Since no one has pointed you to it, there is a doc explaining what it > takes > > to get a new module added to the stdlib: > > http://docs.python.org/devguide/stdlibchanges.html#adding-a-new-module . > > Thanks, I actually found and read that page before making my initial > announcement. For now, I'm making the module available under the > simplified BSD license to anyone who wishes to use it. If it has > gained some traction over the next year, I'll be happy to submit it to > PSF under Apache License 2.0. Is there anything else I should do in > the mean time? How do I go about moving the development into the > Python's infrastructure, or does that happen later? > Later; the module has to first get traction enough for python-dev to even consider adding it. Basically the community needs to have decided as a whole that your module is the best solution for the job and that its API is stable and done evolving. Hope this doesn't sound too negative, but we just have to be very cautious about what goes into Python's stdlib. -Brett > > Over the next week or so, I'll finish writing the higher-level > interface, which actually did end up inheriting from Mailbox, but with > some limitations that I'll explain later. After that, I'll put > together a test suite and update the documentation. The last remaining > bit will be to implement some additional extensions and authentication > methods. Don't have anything else planned beyond that. > > - Max > -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Mon Jul 25 21:18:26 2011 From: barry at python.org (Barry Warsaw) Date: Mon, 25 Jul 2011 15:18:26 -0400 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> Message-ID: <20110725151826.12564b47@resist.wooz.org> On Jul 25, 2011, at 07:43 PM, Michael Foord wrote: >On 25 July 2011 18:34, Barry Warsaw wrote: >> I'm not sure it would be a good idea to do a mass mechanical substitution >> in the stdlib, so I'm not concerned that adopting enums would require a few >> int() calls to be added. >> >Well, if it isn't good enough for *us* then who is it good enough for? ;-) I only meant that we have a tradition of not doing wholesale mechanical conversions when new features are added to the language or modules to the stdlib. This wouldn't be any different. :) >Some apis (for example those exported directly from C) can't work with >something that isn't a real int. That might be an important enough use case to sway me! >>>> from flufl.enum import Enum >>>> class Thing(Enum): >... a = 1 >... b = 2 >... c = 4 >... >>>> Thing.a | Thing.b >Traceback (most recent call last): > File "", line 1, in >TypeError: unsupported operand type(s) for |: 'EnumValue' and 'EnumValue' Right. That's a consequence I think of EnumValues not being ints (and no other support for such operations being added. >Sure, I have no use case for ordering enums... I'm not sure I have a >specific reason to *prevent* if that is extra work though. Possibly so. I think my aversion to it is seeing things like >>> if Colors.red < Colors.blue: or even >>> if color < Colors.blue: Most enums (at least IME) are discrete objects that don't have a natural ordering. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From ben+python at benfinney.id.au Wed Jul 27 23:42:07 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 28 Jul 2011 07:42:07 +1000 Subject: [Python-ideas] Enums References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> Message-ID: <87fwlrpfio.fsf@benfinney.id.au> Barry Warsaw writes: > Most enums (at least IME) are discrete objects that don't have a > natural ordering. +1. This is an important property of an enumerated type, IMO. The fact that ?bool? disagrees (?False < True?) is an artifact of its history, and is not relevant for an argument for orderable EnumValues. -- \ ?A thing moderately good is not so good as it ought to be. | `\ Moderation in temper is always a virtue; but moderation in | _o__) principle is always a vice.? ?Thomas Paine | Ben Finney -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From ethan at stoneleaf.us Thu Jul 28 00:10:23 2011 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 27 Jul 2011 15:10:23 -0700 Subject: [Python-ideas] Enums In-Reply-To: <87fwlrpfio.fsf@benfinney.id.au> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <87fwlrpfio.fsf@benfinney.id.au> Message-ID: <4E308CCF.4060003@stoneleaf.us> Ben Finney wrote: > Barry Warsaw writes: > >> Most enums (at least IME) are discrete objects that don't have a >> natural ordering. > > +1. This is an important property of an enumerated type, IMO. In most cases I agree, but there can be exceptions. Consider: class StoveTemps(Enum): off = 0 low = 15 medium = 50 high = 85 In such a case I see no issues with if temp < StoveTemps.medium: what_ever() ~Ethan~ From ncoghlan at gmail.com Thu Jul 28 00:24:09 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 28 Jul 2011 08:24:09 +1000 Subject: [Python-ideas] Enums In-Reply-To: <20110725151826.12564b47@resist.wooz.org> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> Message-ID: On Tue, Jul 26, 2011 at 5:18 AM, Barry Warsaw wrote: > Most enums (at least IME) are discrete objects that don't have a natural > ordering. For compatibility reasons, though, I believe it is worth following the precedent set by bool: if integer constants are to be replaced by named values, the replacement needs to support everything the previous value did. The easiest way to achieve that is by inheriting directly from the previous value's type. I'm getting deja vu as I formulate this next bit, so I suspect I've said similar things in the past... 1. I find the "enum" concept too limiting. NamedValue (as a mixin class) is a better building block since it separates the naming aspect from the "group of values" aspect implied by enum. It would be nice, for example, if we could do things like "tau = NamedFloat('tau', 2*math.pi)" where NamedFloat is essentially just "class NamedFloat(NamedValue, float): pass" (this could, of course, be hidden behind a factory function that used type(value) and a cache to dynamically create appropriate subclasses) 2. All the stdlib use cases have a relevant namespace already (the module level one). They are better served by a namedtuple style solution that adds naming behaviour at the object level without trying to organise groups of named constants in any particular fashion. 3. The named value approach can improve the ease of debugging *any* significant data structure, not just constants. 4. If named values are available as a building block, they can be used by any enum implementation. In and of themselves they wouldn't inherit from any particular type, so enum implementations could decide which types to support and how comparisons and other operations should behave. As Raymond says, clearly none of this is *necessary*. However, the usefulness of objects that know their official name has proven itself extensively in the form of classes and functions, and collections.namedtuple is a recent addition that has proven similarly beneficial for improving the maintainability of tuples by providing additional static metadata. Extending that kind of benefit to arbitrary instances has clear utility without significantly impacting compatibility (since we would be substituting instances of a class with instances of subclasses that only modify display related behaviour). But the grouping behaviour? I really don't see a lot of value in that - just boilerplate. We don't write bool.True and bool.False, we write True and False. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Thu Jul 28 00:24:09 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 28 Jul 2011 08:24:09 +1000 Subject: [Python-ideas] Enums In-Reply-To: <20110725151826.12564b47@resist.wooz.org> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> Message-ID: On Tue, Jul 26, 2011 at 5:18 AM, Barry Warsaw wrote: > Most enums (at least IME) are discrete objects that don't have a natural > ordering. For compatibility reasons, though, I believe it is worth following the precedent set by bool: if integer constants are to be replaced by named values, the replacement needs to support everything the previous value did. The easiest way to achieve that is by inheriting directly from the previous value's type. I'm getting deja vu as I formulate this next bit, so I suspect I've said similar things in the past... 1. I find the "enum" concept too limiting. NamedValue (as a mixin class) is a better building block since it separates the naming aspect from the "group of values" aspect implied by enum. It would be nice, for example, if we could do things like "tau = NamedFloat('tau', 2*math.pi)" where NamedFloat is essentially just "class NamedFloat(NamedValue, float): pass" (this could, of course, be hidden behind a factory function that used type(value) and a cache to dynamically create appropriate subclasses) 2. All the stdlib use cases have a relevant namespace already (the module level one). They are better served by a namedtuple style solution that adds naming behaviour at the object level without trying to organise groups of named constants in any particular fashion. 3. The named value approach can improve the ease of debugging *any* significant data structure, not just constants. 4. If named values are available as a building block, they can be used by any enum implementation. In and of themselves they wouldn't inherit from any particular type, so enum implementations could decide which types to support and how comparisons and other operations should behave. As Raymond says, clearly none of this is *necessary*. However, the usefulness of objects that know their official name has proven itself extensively in the form of classes and functions, and collections.namedtuple is a recent addition that has proven similarly beneficial for improving the maintainability of tuples by providing additional static metadata. Extending that kind of benefit to arbitrary instances has clear utility without significantly impacting compatibility (since we would be substituting instances of a class with instances of subclasses that only modify display related behaviour). But the grouping behaviour? I really don't see a lot of value in that - just boilerplate. We don't write bool.True and bool.False, we write True and False. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From menno at freshfoo.com Thu Jul 28 00:34:25 2011 From: menno at freshfoo.com (Menno Smits) Date: Wed, 27 Jul 2011 15:34:25 -0700 (PDT) Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library In-Reply-To: References: Message-ID: (I'm replying via Google Groups because I just joined and don't have this thread in my email inbox. It's being a bit flaky so apologies if this comes though twice). On Jul 26, 5:54 pm, Maxim Khitrov wrote: > > One of the things I'm trying to address with my library is strict > adherence to the current version of the IMAP4 protocol. The other is > performance; hence the implementation of extensions such as SASL-IR, > IDLE, non-synchronizing literals, multiappend, and compression. > > On the performance side, if you have an application that's trying do > some sort of processing of a 6 GB mailbox with 700,000 messages in it, > executing a separate FETCH command for each message will take you a > week to finish. If you try to be clever and FETCH 1000 messages at a > time, for example, you'll quickly run into a few problems: > > ... > All are interface design problems, which are inherited by IMAPClient. > > - Max > > P.S. It is not my intention to discourage the use of IMAPClient in any > way. Its existence is a good thing for 99% of the users, because it > does address a number of key imaplib issues with just the response > parser and a UTF-7 codec. My point is that there are real-world use > cases out there that cannot be handled by imaplib or IMAPClient, and > for those, I'm offering my library as a more general solution that > should satisfy the remaining 1% :) As the maintainer of IMAPClient, I thought I'd weigh in. I've had a quick look at imaplib2[1] and I must say it's a solid piece of work. The number of IMAP extensions that it covers is impressive, the iterative streaming of fetch responses is great and the way that concurrent and async commands are handled is quite elegant. Max is correct about the limitations of imaplib (and therefore IMAPClient). As with many Python libraries (eg. poplib, email, smtplib), data is loaded into memory without the option of streaming (say via generators) and imaplib is not designed with asynchronous handling in mind. This simplicity has benefits in terms of the implementation and usage of the API but it can also lead to problems. I have considered restructuring IMAPClient so that it no longer depends on imaplib and so free it from those inherent limitations. This may still happen. imaplib2 makes good use of the features available in modern versions of Python. On the other hand, one of my goals with IMAPClient is to support a wide range of Python versions - many newer Python niceties are not used. Python 3 support for IMAPClient is definitely coming but a new addition to our family at home as slowed that effort down somewhat. I have never considered pushing for IMAPClient to be included as part of the stdlib given that it is fairly easy to install 3rd party packages these days, but I wouldn't be against it. Obviously Python 3 support would have to come first. I think imaplib2 is a very capable IMAP client library and the Python community could only benefit from having something like it in the standard library (on the proviso, as Brett mentions, that the Python community supports the library by using it widely). Here's a few comments about imaplib2 from my own biased perspective: It requires too much effort on behalf of the caller. Your example.py highlights how datetimes are returned as strings that need to be converted to real datetimes and FETCH response keys need to be uppercased to ensure consistency. The need to jump through the same non-trivial hoops each time I used imaplib was one of the frustrations that led to the creation of IMAPClient. Please consider having imaplib2 do a little more work so that every user doesn't have to. Similarly, UID support could be better. IMAPClient has a boolean attribute which lets you select whether you want UIDs to be transparently used for future commands. Having to specify whether you want UID support enabled on each call is a little clumsy. It's unlikely that a user of imaplib2 would want to toggle between using UIDs and not on every call. This has already been mentioned but imaplib2 won't get accepted into the stdlib if you don't conform to PEP 8. Those tabs have to go. How much testing has imaplib2 seen against real IMAP implementations? Throughout IMAPClient's history its users have found many unexpected behaviours in various popular IMAP implementations. Those discoveries have lead to updates to IMAPClient's code and tests (this is the "battle-tested" aspect that Michael refers too). On top of its unit tests, IMAPClient has a fairly extensive live test script that can be run (destructively) against a real IMAP account. I have test accounts with a number of different IMAP implementations which I regularly test IMAPClient against. A set of "live" tests is invaluable for testing new features and avoiding regressions between versions. It would be interesting to see what problems you find if you set up something similar for imaplib2. Anyway, I wish you all the best with your project. Regards, Menno [1] - Are you aware there's already another project with the same name? http://www.janeelix.com/piers/python/imaplib2.html From guido at python.org Thu Jul 28 00:49:08 2011 From: guido at python.org (Guido van Rossum) Date: Wed, 27 Jul 2011 15:49:08 -0700 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> Message-ID: On Wed, Jul 27, 2011 at 3:24 PM, Nick Coghlan wrote: [...] > But the grouping behaviour? I really don't see a lot of value in that > - just boilerplate. We don't write bool.True and bool.False, we write > True and False. I have to agree with that. Enums can live in a module (together with stuff that uses them) or in a class (if all the uses are limited to that class); they don't need an extra namespace of their own. The standard examples using homonyms in unrelated enums are typically missing the fact that Python doesn't really have *global* variables -- only "module-global", which is really quite narrowly scoped. If a single module or package is exporting so many enums that extra scoping is considered useful, it's easy enough to introduce an extra namespace using existing mechanisms (e.g. a class or a submodule). -- --Guido van Rossum (python.org/~guido) From barry at python.org Thu Jul 28 01:00:57 2011 From: barry at python.org (Barry Warsaw) Date: Wed, 27 Jul 2011 19:00:57 -0400 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> Message-ID: <20110727190057.4558d211@resist.wooz.org> On Jul 28, 2011, at 08:24 AM, Nick Coghlan wrote: >For compatibility reasons, though, I believe it is worth following the >precedent set by bool: if integer constants are to be replaced by >named values, the replacement needs to support everything the previous >value did. The easiest way to achieve that is by inheriting directly >from the previous value's type. I mentioned in a separate follow up that I'd be open to that, if someone wants to submit a patch. >1. I find the "enum" concept too limiting. NamedValue (as a mixin >class) is a better building block since it separates the naming aspect >from the "group of values" aspect implied by enum. I think that's all fine, but it's getting far from *my* simple concept of enumerated values. Go for it though. :) >But the grouping behaviour? I really don't see a lot of value in that >- just boilerplate. We don't write bool.True and bool.False, we write >True and False. That's only because True and False are bound in a particular module namespace. There's no reason why specific EnumValues couldn't also be similarly bound. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From ben+python at benfinney.id.au Thu Jul 28 01:52:12 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 28 Jul 2011 09:52:12 +1000 Subject: [Python-ideas] Enums References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <87fwlrpfio.fsf@benfinney.id.au> <4E308CCF.4060003@stoneleaf.us> Message-ID: <8762mnp9hv.fsf@benfinney.id.au> Ethan Furman writes: > In most cases I agree, but there can be exceptions. Consider: > > class StoveTemps(Enum): > off = 0 > low = 15 > medium = 50 > high = 85 That doesn't seem like an appropriate use of an enumerated type, then. For an enumerated type, the integers are arbitrary and the important part is the name. If the integer values actually *mean* something, then let's have them as name bindings to those values. > In such a case I see no issues with > > if temp < StoveTemps.medium: > what_ever() -- \ ?Anger makes dull men witty, but it keeps them poor.? | `\ ?Elizabeth Tudor | _o__) | Ben Finney From ncoghlan at gmail.com Thu Jul 28 02:30:42 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 28 Jul 2011 10:30:42 +1000 Subject: [Python-ideas] Enums In-Reply-To: <20110727190057.4558d211@resist.wooz.org> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> Message-ID: On Thu, Jul 28, 2011 at 9:00 AM, Barry Warsaw wrote: > On Jul 28, 2011, at 08:24 AM, Nick Coghlan wrote: > >>For compatibility reasons, though, I believe it is worth following the >>precedent set by bool: if integer constants are to be replaced by >>named values, the replacement needs to support everything the previous >>value did. The easiest way to achieve that is by inheriting directly >>from the previous value's type. > > I mentioned in a separate follow up that I'd be open to that, if someone wants > to submit a patch. I did see that, but I don't see a straighforward way to do it without locking the enum implementation into being *specifically* for integers. >>1. I find the "enum" concept too limiting. NamedValue (as a mixin >>class) is a better building block since it separates the naming aspect >>from the "group of values" aspect implied by enum. > > I think that's all fine, but it's getting far from *my* simple concept of > enumerated values. ?Go for it though. :) How is something that does *less* more complicated? This recipe is pretty much the full extent of the proposal (perhaps with the automatic type generation I mention in the discussion section): http://code.activestate.com/recipes/577810-named-values/ >>But the grouping behaviour? I really don't see a lot of value in that >>- just boilerplate. We don't write bool.True and bool.False, we write >>True and False. > > That's only because True and False are bound in a particular module namespace. > There's no reason why specific EnumValues couldn't also be similarly bound. But the *necessity* of grouping creates a lot of boilerplate and prevents TOOWTDI. The obvious way to do constants in current Python is as module level constants. The obvious way to do enums, on the other hand, is to have a grouping class that holds the names. Translating between the two requires a boilerplate loop over the enum values to add references into the module namespace. Whereas, with the named value approach, all you *have* to do is change "named_constant = x" to "named_constant = named_value('named_constant', x)". The various enum libraries could then become convenience APIs to make it easier to create groups of named values. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From barry at python.org Thu Jul 28 02:56:42 2011 From: barry at python.org (Barry Warsaw) Date: Wed, 27 Jul 2011 20:56:42 -0400 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> Message-ID: <20110727205642.6317d0b4@resist.wooz.org> On Jul 28, 2011, at 10:30 AM, Nick Coghlan wrote: >I did see that, but I don't see a straighforward way to do it without >locking the enum implementation into being *specifically* for >integers. Which is fine by me, FWIW. >>>1. I find the "enum" concept too limiting. NamedValue (as a mixin >>>class) is a better building block since it separates the naming aspect >>>from the "group of values" aspect implied by enum. >> >> I think that's all fine, but it's getting far from *my* simple concept of >> enumerated values. ?Go for it though. :) > >How is something that does *less* more complicated? Again, looking at how I've used them extensively over the last several years, I would much rather write class Colors(Enum): red = 1 green = 2 blue = 3 than red = NamedValue('red', 1) green = NamedValue('green', 2) blue = NamedValue('blue', 3) To me, the duplication is jarring and error prone. >But the *necessity* of grouping creates a lot of boilerplate and >prevents TOOWTDI. I have no problem if folks want to add named values to the stdlib. I use namedtuple on occasion, and I wouldn't mind having named values in my arsenal, but I'm fairly certain I wouldn't use named values in the places I use Enum currently. The grouping does provide benefits such as disallowing duplicates, providing for an inheritance mechanism, and making them easier to pickle and store in (and load from) a database. I didn't original think of using enums as a replacement for module level constants, so I think we're just talking about different use cases. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From ncoghlan at gmail.com Thu Jul 28 03:12:27 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 28 Jul 2011 11:12:27 +1000 Subject: [Python-ideas] Enums In-Reply-To: <20110727205642.6317d0b4@resist.wooz.org> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> Message-ID: On Thu, Jul 28, 2011 at 10:56 AM, Barry Warsaw wrote: > Again, looking at how I've used them extensively over the last several years, > I would much rather write > > ? ?class Colors(Enum): > ? ? ? ?red = 1 > ? ? ? ?green = 2 > ? ? ? ?blue = 3 > > than > > ? ?red = NamedValue('red', 1) > ? ?green = NamedValue('green', 2) > ? ?blue = NamedValue('blue', 3) > > To me, the duplication is jarring and error prone. Yeah, I'd actually be inclined to define such values programmatically rather than writing them out manually like that: _named_colours = dict( red=0xFF0000, green=0x00FF00, blue=0x0000FF, ) globals().update((k, namedvalue(k, v)) for k, v in _named_colours) (where namedvalue is the value based factory function I mentioned in the recipe post) However, my contention is that the fundamentally interesting operation is associating names with values (as your EnumValue class does). Enums and their ilk are then just syntactic sugar for defining groups of such values without needing to repeat yourself. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From bruce at leapyear.org Thu Jul 28 03:14:12 2011 From: bruce at leapyear.org (Bruce Leban) Date: Wed, 27 Jul 2011 18:14:12 -0700 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> Message-ID: On Wed, Jul 27, 2011 at 5:30 PM, Nick Coghlan wrote: > > This recipe is pretty much the full extent of the proposal (perhaps > with the automatic type generation I mention in the discussion > section): > http://code.activestate.com/recipes/577810-named-values/ > > I like the NamedValue/namedvalue recipe. I prefer module constants like os.SEEK_END to nested names like os.Seek.End and this encourages that. If someone decides to change os module to: SEEK_END = namedvalue('SEEK_END', 2) that won't break anything. For the simple enum case, a decorator could transform @enum class Color: red = 1 green = 2 into: class Color: red = namedvalue('Color.red', 1) green = namedvalue('Color.green', 2) --- Bruce Follow me: http://www.twitter.com/Vroo http://www.vroospeak.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Thu Jul 28 03:17:33 2011 From: barry at python.org (Barry Warsaw) Date: Wed, 27 Jul 2011 21:17:33 -0400 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> Message-ID: <20110727211733.7cf509b6@resist.wooz.org> On Jul 28, 2011, at 11:12 AM, Nick Coghlan wrote: >However, my contention is that the fundamentally interesting operation >is associating names with values (as your EnumValue class does). That's *an* interesting part, but for my uses not the only ones. But now I'm repeating myself. :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From max at mxcrypt.com Thu Jul 28 03:35:24 2011 From: max at mxcrypt.com (Maxim Khitrov) Date: Wed, 27 Jul 2011 21:35:24 -0400 Subject: [Python-ideas] New imaplib implementation for Python 3.2+ standard library In-Reply-To: References: Message-ID: On Wed, Jul 27, 2011 at 6:34 PM, Menno Smits wrote: > (I'm replying via Google Groups because I just joined and don't have > this thread in my email inbox. It's being a bit flaky so apologies if > this comes though twice). > > On Jul 26, 5:54 pm, Maxim Khitrov wrote: >> >> One of the things I'm trying to address with my library is strict >> adherence to the current version of the IMAP4 protocol. The other is >> performance; hence the implementation of extensions such as SASL-IR, >> IDLE, non-synchronizing literals, multiappend, and compression. >> >> On the performance side, if you have an application that's trying do >> some sort of processing of a 6 GB mailbox with 700,000 messages in it, >> executing a separate FETCH command for each message will take you a >> week to finish. If you try to be clever and FETCH 1000 messages at a >> time, for example, you'll quickly run into a few problems: >> >> ... >> All are interface design problems, which are inherited by IMAPClient. >> >> - Max >> >> P.S. It is not my intention to discourage the use of IMAPClient in any >> way. Its existence is a good thing for 99% of the users, because it >> does address a number of key imaplib issues with just the response >> parser and a UTF-7 codec. My point is that there are real-world use >> cases out there that cannot be handled by imaplib or IMAPClient, and >> for those, I'm offering my library as a more general solution that >> should satisfy the remaining 1% :) > > As the maintainer of IMAPClient, I thought I'd weigh in. > > ... > > I think imaplib2 is a very capable IMAP client library and the Python > community could only benefit from having something like it in the > standard library (on the proviso, as Brett mentions, that the Python > community supports the library by using it widely). Thanks for the kind words :) > Here's a few comments about imaplib2 from my own biased perspective: > > It requires too much effort on behalf of the caller. Your example.py > highlights how datetimes are returned as strings that need to be > converted to real datetimes and FETCH response keys need to be > uppercased to ensure consistency. The need to jump through the same > non-trivial hoops each time I used imaplib was one of the frustrations > that led to the creation of IMAPClient. Please consider having > imaplib2 > do a little more work so that every user doesn't have to. Part of this will be addressed by the higher-level interface that I'm currently working on. As for imaplib2, there are two reasons why I decided not to do any sort of automatic normalization of the responses (with the exception of CAPABILITY): 1. Performance. Not all responses (and parts of a response) are useful to the caller. There is no point in having the library perform response-specific normalization just to have the whole thing discarded as soon as it is returned. Originally, I even played with the idea of a lazy parser (i.e. parse the response only if some attribute or data item is accessed), but decided to go for a simpler implementation in the end. 2. Consistency, expectations, and bugs. The normalization processes may not do the Right Thing for every single response. Ultimately, only the caller knows for sure what content to expect from the server, especially if you are trying to implement some server-specific commands or a new extension. The library only knows the general syntax rules. If you start to assume that all returned responses are normalized, you could run into some unwelcome surprises when that normalization fails or even corrupts some data for a response type that wasn't recognized. So basically, I think that in a low-level library such as this, it should be the caller's decision whether an INTERNALDATE value is converted to Unix time (or some other format), or if the FETCH response keys are changed to upper case. I'm happy to provide additional utility functions for such conversions, but trying to handle these things automatically could be a source of many additional bugs. Think about the separation between zlib and gzip, or binascii and base64 modules. My library is the low-level interface and I'm working on something that will be easier to use at the cost of some control. > Similarly, UID support could be better. IMAPClient has a boolean > attribute which lets you select whether you want UIDs to be > transparently used for future commands. Having to specify whether you > want UID support enabled on each call is a little clumsy. It's > unlikely > that a user of imaplib2 would want to toggle between using UIDs and > not > on every call. I have to disagree with you here. The application that I wrote this library for does depend on the ability to run UID and regular FETCH commands in the same connection. I was actually very surprised to see that IMAPClient requires you pick one or the other at creation time. In some applications you may need to discover and use the relationships between SNs and UIDs, or use a command like UID EXPUNGE (from UIDPLUS extension) and a regular EXPUNGE in the same session. I think that you do have to let the user make this decision on a per-command basis. > This has already been mentioned but imaplib2 won't get accepted into > the > stdlib if you don't conform to PEP 8. Those tabs have to go. I know. I'll reformat everything once all the major coding is done. > How much testing has imaplib2 seen against real IMAP implementations? > Throughout IMAPClient's history its users have found many unexpected > behaviours in various popular IMAP implementations. Those discoveries > have lead to updates to IMAPClient's code and tests (this is the > "battle-tested" aspect that Michael refers too). On top of its unit > tests, IMAPClient has a fairly extensive live test script that can be > run (destructively) against a real IMAP account. I have test accounts > with a number of different IMAP implementations which I regularly test > IMAPClient against. A set of "live" tests is invaluable for testing > new > features and avoiding regressions between versions. It would be > interesting to see what problems you find if you set up something > similar for imaplib2. I've tested against Gmail servers, Microsoft Exchange 2007, and ran simulated tests based on example sessions in various RFCs and other sources. I also wrote a "shell" script that connects to an IMAP server and goes into interactive mode, allowing me to run IMAP or Python commands exactly as you would in an interactive Python session. I'll try to upload it to the repository in the next few days. My library does need more testing. Although I tried to follow the robustness principle (be conservative in what you send; be liberal in what you accept) when writing the command generator and response parser, there probably are some bugs remaining, but hopefully not many. Which IMAP servers do you test against and how did you go about getting the test accounts? > [1] - Are you aware there's already another project with the same > name? > http://www.janeelix.com/piers/python/imaplib2.html Hmm... I probably should have tried searching before using that name. I'm happy to go with something else, since my library is not in wide-spread use right now. Would suggesting imaplib3 for stdlib be a bit confusing? :/ That looks like another improvement of imaplib, which uses threads to achieve some asynchronous execution. Can't say that I like the approach, but I do admire the effort. They even got compression working, but that was at the cost of having to implement readline in python rather than relying on the BufferedReader. That was one of the bigger challenges for me as well, but I opted to write my own SocketIO class for this. - Max From greg.ewing at canterbury.ac.nz Thu Jul 28 03:45:54 2011 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 28 Jul 2011 13:45:54 +1200 Subject: [Python-ideas] Enums In-Reply-To: <87fwlrpfio.fsf@benfinney.id.au> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <87fwlrpfio.fsf@benfinney.id.au> Message-ID: <4E30BF52.6030708@canterbury.ac.nz> Ben Finney wrote: > Barry Warsaw writes: > >>Most enums (at least IME) are discrete objects that don't have a >>natural ordering. > > +1. This is an important property of an enumerated type, IMO. If you mean that it's important for an enum *not* to have an ordering, I disagree. There are some use cases that benefit from having an ordering, and some that don't. For those that don't, personally I don't care whether it has an ordering or not. Raising an exception on ordered comparison could perhaps be an optional feature, but I wouldn't like it to be the only option. -- Greg From guido at python.org Thu Jul 28 04:21:46 2011 From: guido at python.org (Guido van Rossum) Date: Wed, 27 Jul 2011 19:21:46 -0700 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> Message-ID: On Wed, Jul 27, 2011 at 6:12 PM, Nick Coghlan wrote: > On Thu, Jul 28, 2011 at 10:56 AM, Barry Warsaw wrote: >> Again, looking at how I've used them extensively over the last several years, >> I would much rather write >> >> ? ?class Colors(Enum): >> ? ? ? ?red = 1 >> ? ? ? ?green = 2 >> ? ? ? ?blue = 3 >> >> than >> >> ? ?red = NamedValue('red', 1) >> ? ?green = NamedValue('green', 2) >> ? ?blue = NamedValue('blue', 3) >> >> To me, the duplication is jarring and error prone. Agreed, especially in new code, when you just want to get the names defined, and you don't have to worry about compatibility. I think that's why NamedValue would often be wrapped by an enum class. But there is the downside that, for all the elegance of writing class Color(enum): red = 1 ... all the *uses* of colors have to write foo.Color.red instead of foo.red. > Yeah, I'd actually be inclined to define such values programmatically > rather than writing them out manually like that: > > _named_colours = dict( > ?red=0xFF0000, > ?green=0x00FF00, > ?blue=0x0000FF, > ) > globals().update((k, namedvalue(k, v)) for k, v in _named_colours) Eek, no! It will take the average reader way too long to figure out that that does. Static analyzers (which are getting more important) are likely to be fooled by it too. Please remember EIBTI. > (where namedvalue is the value based factory function I mentioned in > the recipe post) > > However, my contention is that the fundamentally interesting operation > is associating names with values (as your EnumValue class does). Enums > and their ilk are then just syntactic sugar for defining groups of > such values without needing to repeat yourself. It's just possible that there's no way to define enums that neither introduced a new (class) scope nor requires a lot of redundant typing. I wish I could write red = Enum(1) and it made the following true: assert red == 1 assert isinstance(red, int) # a subclass assert str(red) == 'red' But we'd first need a non-hacky way for Enum() to know that it is being assigned to something named 'red'. I have a few other use cases for that as well, e.g. a Property class for App Engine that doesn't require a metaclass to patch up the value. A half-solution would be if naturally the definition of red appeared inside some other class (not a class specially created for the enum, but a class that has other functionality) and somehow a mix-in metaclass (if such a beast exists) would pass in the name once the class is being defined. But if you want module-level enum values you'd still have to do something like class Color(Enum): red = 1 ... red, green, blue = Color.red, Color.green, Color.blue That's not ideal since the next programmer could add a new color but forget to also pull it into the outer scope. And no, I don't like solutions based on globals()... TBH, I'm not sure we should hold our breath until we have the perfect solution. Looking over flufl.enum again, I like most of its design decisions except the "enums are not integers" part. For me, after the above definition of class Color, I'd be happy of Color.red == 1, as long as str(Color.red) == 'red'. (Here I am consistent with the behavior of True and False.) If you want the enum values to appear at the module level (e.g. socket constants?) pulling them up a level explicitly, while not ideal (see above), is not the end of the world either. -- --Guido van Rossum (python.org/~guido) From mwm at mired.org Thu Jul 28 05:18:40 2011 From: mwm at mired.org (Mike Meyer) Date: Wed, 27 Jul 2011 20:18:40 -0700 Subject: [Python-ideas] Enums In-Reply-To: <4E30BF52.6030708@canterbury.ac.nz> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <87fwlrpfio.fsf@benfinney.id.au> <4E30BF52.6030708@canterbury.ac.nz> Message-ID: <20110727201840.3b6c0cd3@bhuda.mired.org> On Thu, 28 Jul 2011 13:45:54 +1200 Greg Ewing wrote: > Ben Finney wrote: > > Barry Warsaw writes: > > > >>Most enums (at least IME) are discrete objects that don't have a > >>natural ordering. > > > > +1. This is an important property of an enumerated type, IMO. > > If you mean that it's important for an enum *not* to have > an ordering, I disagree. There are some use cases that > benefit from having an ordering, and some that don't. > For those that don't, personally I don't care whether > it has an ordering or not. Question: how does iterating over the group fit into this? For some enums (colors, for instance) order may be irrelevant or an implementation detail, but you'd still like to be able to iterate over the group. Not sure it matters, but thought it might. http://www.mired.org/ Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From ncoghlan at gmail.com Thu Jul 28 05:57:24 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 28 Jul 2011 13:57:24 +1000 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> Message-ID: On Thu, Jul 28, 2011 at 12:21 PM, Guido van Rossum wrote: > I wish I could write > > ?red = Enum(1) > > and it made the following true: > > ?assert red == 1 > ?assert isinstance(red, int) ?# a subclass > ?assert str(red) == 'red' > > But we'd first need a non-hacky way for Enum() to know that it is > being assigned to something named 'red'. I have a few other use cases > for that as well, e.g. a Property class for App Engine that doesn't > require a metaclass to patch up the value. Yeah, a similar discussion came up in the context of defining namedtuple instances a while back (I don't recall if you were part of that conversation or not - IIRC, the thread started off on the topic of assignment decorators and ended up wandering down this road at some point) Most proposed solutions relied on some form of abuse of the def statement to define arbitrary objects that knew their own name. For example: def red from namedvalue(1) # Rather unnatural phrasing def red as namedvalue(1) # Phrasing is natural, but the name is on the wrong side of the 'as' def red = namedvalue(1) # Simple assignment may not suggest enough magic def as red = namedvalue(1) # Syntax soup! as red def namedvalue(1) # Just throw keywords at the screen and see if anything sticks red def= namedvalue(1) # An alternative inspired by augmented assignment def red << namedvalue(1) # Arbitrary but suggestive A protocol would then be defined to make that work (regardless of the specific syntax). Either the actual call would be transformed into "namedvalue('red', 1)" (which has the virtue of working with existing objects like collections.namedtuple, but prevents provision of useful default behaviour) or else there would be a new protocol like "obj.__named__(name)", with a fallback to something like the NamedValue recipe if __named__ wasn't defined (which has the virtue of working with arbitrary objects as in "def red as 1", but requires adapter classes to work with existing APIs like namedtuple). The bikeshed was painted many different colours before the thread finally wound down without achieving any kind of resolution. I suspect this may end up being another PEP 308, where the only way it will ever happen is if *you* find a syntax you like (or are at least willing to tolerate in order to gain the functionality). Adding an int-specific Enum class eliminates a lot of the use cases for a "named object" feature, which is the main reason I'm trying to hold out for the more general functionality - I still have hope that we'll strike on a syntax for named assignments that is clear enough and clean enough that none of the core devs actively oppose it. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ben+python at benfinney.id.au Thu Jul 28 06:49:56 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 28 Jul 2011 14:49:56 +1000 Subject: [Python-ideas] Enums References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> Message-ID: <87wrf3nh57.fsf@benfinney.id.au> Guido van Rossum writes: > TBH, I'm not sure we should hold our breath until we have the perfect > solution. That's an important principle, certainly. > Looking over flufl.enum again, I like most of its design decisions > except the "enums are not integers" part. For me, after the above > definition of class Color, I'd be happy of Color.red == 1, as long as > str(Color.red) == 'red'. I am displeased if we don't get this:: >>> Color.red == 1 False >>> Fruit.tomato == 1 False but wouldn't want to block an ?enum? implementation waiting for that. What I'd hold out for, though, is:: >>> Color.red == Fruit.tomato False That is, all of the values from Color should compare as inequal with any other value. This is one of the main features to want from an enumerated type, IMO: to have a set of values that are distinct from any other value, that won't be accidentally equal to any other value, and have helpful string representations. > (Here I am consistent with the behavior of True and False.) Do you see that (the behaviour of True and False comparing equal with integers) as anything more than backward-compatible baggage? To me, if I had the time machine, a proper representation of a boolean type would have True and False as distinct values, never comparing equal with any other value. -- \ ?Any intelligent fool can make things bigger and more complex? | `\ It takes a touch of genius ? and a lot of courage ? to move in | _o__) the opposite direction.? ?Albert Einstein | Ben Finney From ben+python at benfinney.id.au Thu Jul 28 06:51:48 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 28 Jul 2011 14:51:48 +1000 Subject: [Python-ideas] Enums References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <87fwlrpfio.fsf@benfinney.id.au> <4E30BF52.6030708@canterbury.ac.nz> <20110727201840.3b6c0cd3@bhuda.mired.org> Message-ID: <87sjprnh23.fsf@benfinney.id.au> Mike Meyer writes: > Question: how does iterating over the group fit into this? For some > enums (colors, for instance) order may be irrelevant or an > implementation detail, but you'd still like to be able to iterate over > the group. You can iterate over members of a set, too. What's the question? -- \ ?To have the choice between proprietary software packages, is | `\ being able to choose your master. Freedom means not having a | _o__) master.? ?Richard M. Stallman, 2007-05-16 | Ben Finney From ron3200 at gmail.com Thu Jul 28 09:08:10 2011 From: ron3200 at gmail.com (ron3200) Date: Thu, 28 Jul 2011 02:08:10 -0500 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> Message-ID: <1311836890.7662.133.camel@Gutsy> On Wed, 2011-07-27 at 19:21 -0700, Guido van Rossum wrote: > It's just possible that there's no way to define enums that neither > introduced a new (class) scope nor requires a lot of redundant typing. > > I wish I could write > > red = Enum(1) > > and it made the following true: > > assert red == 1 > assert isinstance(red, int) # a subclass > assert str(red) == 'red' > > But we'd first need a non-hacky way for Enum() to know that it is > being assigned to something named 'red'. I think the least-hacky way is to live with the redundant typing. When these objects are passed around, (to/from functions or methods), they will most likely be rebound to more generic names that don't match their string repr anyway. The label is not directly tied to the name the object is bound to, although it may match at creation time. What name would you give... colors = [] colors.append(Enum(1)) assert str(colors[0]) == ? The only other nice alternatives I can think of at the moment requires either special syntax or a new keyword. red := 1 # same as... red = Enum('red', 1) enum red, 1 # same as... red = Enum('red', 1) I'd still opt for the more explicit Enum('red', 1) expression first before adding either the syntax or keyword version. Those can come later if it turns out enums are hugely popular. The idea of a new class scope for this hurts my head. To many what if's to consider. Cheers, Ron -------------- next part -------------- An HTML attachment was scrubbed... URL: From ezio.melotti at gmail.com Thu Jul 28 14:17:52 2011 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Thu, 28 Jul 2011 15:17:52 +0300 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> Message-ID: <4E315370.3090306@gmail.com> Hi, On 28/07/2011 4.12, Nick Coghlan wrote: > On Thu, Jul 28, 2011 at 10:56 AM, Barry Warsaw wrote: >> Again, looking at how I've used them extensively over the last several years, >> I would much rather write >> >> class Colors(Enum): >> red = 1 >> green = 2 >> blue = 3 >> >> than >> >> red = NamedValue('red', 1) >> green = NamedValue('green', 2) >> blue = NamedValue('blue', 3) >> >> To me, the duplication is jarring and error prone. > Yeah, I'd actually be inclined to define such values programmatically > rather than writing them out manually like that: > > _named_colours = dict( > red=0xFF0000, > green=0x00FF00, > blue=0x0000FF, > ) > globals().update((k, namedvalue(k, v)) for k, v in _named_colours) A method like Colors.make_global([namespace]) could be added to do the same thing. > > (where namedvalue is the value based factory function I mentioned in > the recipe post) > > However, my contention is that the fundamentally interesting operation > is associating names with values (as your EnumValue class does). Enums > and their ilk are then just syntactic sugar for defining groups of > such values without needing to repeat yourself. > > Cheers, > Nick. > Best Regards, Ezio Melotti From fuzzyman at gmail.com Thu Jul 28 15:00:02 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Thu, 28 Jul 2011 14:00:02 +0100 Subject: [Python-ideas] Enums In-Reply-To: <4E315370.3090306@gmail.com> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <4E315370.3090306@gmail.com> Message-ID: On 28 July 2011 13:17, Ezio Melotti wrote: > Hi, > > > On 28/07/2011 4.12, Nick Coghlan wrote: > >> On Thu, Jul 28, 2011 at 10:56 AM, Barry Warsaw wrote: >> >>> Again, looking at how I've used them extensively over the last several >>> years, >>> I would much rather write >>> >>> class Colors(Enum): >>> red = 1 >>> green = 2 >>> blue = 3 >>> >>> than >>> >>> red = NamedValue('red', 1) >>> green = NamedValue('green', 2) >>> blue = NamedValue('blue', 3) >>> >>> To me, the duplication is jarring and error prone. >>> >> Yeah, I'd actually be inclined to define such values programmatically >> rather than writing them out manually like that: >> >> _named_colours = dict( >> red=0xFF0000, >> green=0x00FF00, >> blue=0x0000FF, >> ) >> globals().update((k, namedvalue(k, v)) for k, v in _named_colours) >> > > A method like Colors.make_global([namespace]**) could be added to do the > same thing. > > Colors.make_global(__name__) Michael > > >> (where namedvalue is the value based factory function I mentioned in >> the recipe post) >> >> However, my contention is that the fundamentally interesting operation >> is associating names with values (as your EnumValue class does). Enums >> and their ilk are then just syntactic sugar for defining groups of >> such values without needing to repeat yourself. >> >> Cheers, >> Nick. >> >> > Best Regards, > Ezio Melotti > > ______________________________**_________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/**mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From eike.welk at gmx.net Thu Jul 28 15:10:39 2011 From: eike.welk at gmx.net (Eike Welk) Date: Thu, 28 Jul 2011 15:10:39 +0200 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725151826.12564b47@resist.wooz.org> Message-ID: <201107281510.39660.eike.welk@gmx.net> On Thursday 28.07.2011 00:24:09 Nick Coghlan wrote: > 1. I find the "enum" concept too limiting. NamedValue (as a mixin > class) is a better building block since it separates the naming aspect > from the "group of values" aspect implied by enum. It would be nice, > for example, if we could do things like "tau = NamedFloat('tau', > 2*math.pi)" where NamedFloat is essentially just "class > NamedFloat(NamedValue, float): pass" (this could, of course, be hidden > behind a factory function that used type(value) and a cache to > dynamically create appropriate subclasses) +1 Very nice idea! From python at mrabarnett.plus.com Thu Jul 28 15:52:36 2011 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 28 Jul 2011 14:52:36 +0100 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> Message-ID: <4E3169A4.8020303@mrabarnett.plus.com> On 28/07/2011 04:57, Nick Coghlan wrote: > On Thu, Jul 28, 2011 at 12:21 PM, Guido van Rossum wrote: >> I wish I could write >> >> red = Enum(1) >> >> and it made the following true: >> >> assert red == 1 >> assert isinstance(red, int) # a subclass >> assert str(red) == 'red' >> >> But we'd first need a non-hacky way for Enum() to know that it is >> being assigned to something named 'red'. I have a few other use cases >> for that as well, e.g. a Property class for App Engine that doesn't >> require a metaclass to patch up the value. > > Yeah, a similar discussion came up in the context of defining > namedtuple instances a while back (I don't recall if you were part of > that conversation or not - IIRC, the thread started off on the topic > of assignment decorators and ended up wandering down this road at some > point) > > Most proposed solutions relied on some form of abuse of the def > statement to define arbitrary objects that knew their own name. For > example: > > def red from namedvalue(1) # Rather unnatural phrasing > def red as namedvalue(1) # Phrasing is natural, but the name is on > the wrong side of the 'as' > def red = namedvalue(1) # Simple assignment may not suggest enough magic > def as red = namedvalue(1) # Syntax soup! > as red def namedvalue(1) # Just throw keywords at the screen and > see if anything sticks > red def= namedvalue(1) # An alternative inspired by augmented assignment > def red<< namedvalue(1) # Arbitrary but suggestive > [snip] You missed out: def red is namedvalue(1) From tjreedy at udel.edu Thu Jul 28 16:39:42 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 28 Jul 2011 10:39:42 -0400 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> Message-ID: On 7/27/2011 10:21 PM, Guido van Rossum wrote: > On Wed, Jul 27, 2011 at 6:12 PM, Nick Coghlan wrote: >> Yeah, I'd actually be inclined to define such values programmatically >> rather than writing them out manually like that: >> >> _named_colours = dict( >> red=0xFF0000, >> green=0x00FF00, >> blue=0x0000FF, >> ) >> globals().update((k, namedvalue(k, v)) for k, v in _named_colours) > > Eek, no! It will take the average reader way too long to figure out > that that does. Eek 2. That could be buried in a global_constants function global_constants(red=1,green=2,blue=3) > Static analyzers (which are getting more important) > are likely to be fooled by it too. Please remember EIBTI. But this would be even worse for analyzers. -- Terry Jan Reedy From p.f.moore at gmail.com Thu Jul 28 16:58:10 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 28 Jul 2011 15:58:10 +0100 Subject: [Python-ideas] Enums In-Reply-To: <4E3169A4.8020303@mrabarnett.plus.com> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <4E3169A4.8020303@mrabarnett.plus.com> Message-ID: On 28 July 2011 14:52, MRAB wrote: > On 28/07/2011 04:57, Nick Coghlan wrote: >> Most proposed solutions relied on some form of abuse of the def >> statement to define arbitrary objects that knew their own name. For >> example: >> >> ? def red from namedvalue(1) ?# Rather unnatural phrasing >> ? def red as namedvalue(1) ?# Phrasing is natural, but the name is on >> the wrong side of the 'as' >> ? def red = namedvalue(1) ?# Simple assignment may not suggest enough >> magic >> ? def as red = namedvalue(1) ?# Syntax soup! >> ? as red def namedvalue(1) ?# Just throw keywords at the screen and >> see if anything sticks >> ? red def= namedvalue(1) ?# An alternative inspired by augmented >> assignment >> ? def red<< ?namedvalue(1) # Arbitrary but suggestive >> > [snip] > You missed out: > > ? ?def red is namedvalue(1) IIRC, there was also red := namedvalue(1) which is probably disallowed because it looks like grit on Tim's screen, but actually has some relevant history in that := is used as an assignment operator in many languages. Paul From guido at python.org Thu Jul 28 16:58:36 2011 From: guido at python.org (Guido van Rossum) Date: Thu, 28 Jul 2011 07:58:36 -0700 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> Message-ID: On Wed, Jul 27, 2011 at 8:57 PM, Nick Coghlan wrote: > On Thu, Jul 28, 2011 at 12:21 PM, Guido van Rossum wrote: >> I wish I could write >> >> ?red = Enum(1) >> >> and it made the following true: >> >> ?assert red == 1 >> ?assert isinstance(red, int) ?# a subclass >> ?assert str(red) == 'red' >> >> But we'd first need a non-hacky way for Enum() to know that it is >> being assigned to something named 'red'. I have a few other use cases >> for that as well, e.g. a Property class for App Engine that doesn't >> require a metaclass to patch up the value. > > Yeah, a similar discussion came up in the context of defining > namedtuple instances a while back (I don't recall if you were part of > that conversation or not - IIRC, the thread started off on the topic > of assignment decorators and ended up wandering down this road at some > point) I recall that, and I think I even contributed. But I agree we never came up with anything acceptable. > Most proposed solutions relied on some form of abuse of the def > statement to define arbitrary objects that knew their own name. For > example: > > ?def red from namedvalue(1) ?# Rather unnatural phrasing > ?def red as namedvalue(1) ?# Phrasing is natural, but the name is on > the wrong side of the 'as' > ?def red = namedvalue(1) ?# Simple assignment may not suggest enough magic > ?def as red = namedvalue(1) ?# Syntax soup! > ?as red def namedvalue(1) ?# Just throw keywords at the screen and > see if anything sticks > ?red def= namedvalue(1) ?# An alternative inspired by augmented assignment > ?def red << namedvalue(1) # Arbitrary but suggestive > > A protocol would then be defined to make that work (regardless of the > specific syntax). Either the actual call would be transformed into > "namedvalue('red', 1)" (which has the virtue of working with existing > objects like collections.namedtuple, but prevents provision of useful > default behaviour) or else there would be a new protocol like > "obj.__named__(name)", with a fallback to something like the > NamedValue recipe if __named__ wasn't defined (which has the virtue of > working with arbitrary objects as in "def red as 1", but requires > adapter classes to work with existing APIs like namedtuple). > > The bikeshed was painted many different colours before the thread > finally wound down without achieving any kind of resolution. I suspect > this may end up being another PEP 308, where the only way it will ever > happen is if *you* find a syntax you like (or are at least willing to > tolerate in order to gain the functionality). My current thinking would be that you could probably hack it in CPython through bytecode inspection: find the STORE_FAST or STORE_GLOBAL opcode right after the CALL_FUNCTION opcode, and see what to name it assigns. This is of course terrible, depending on all sorts of implementation stuff, but we might create a builtin function that returns that variable name, assuming the scope is module-global or a class scope. I know it sounds atrocious, but the alternative is requiring extra mark-up and a hidden implied argument, which does not sound good either. (Maybe extra mark-up could be required for the proposed new builtin to work.) > Adding an int-specific Enum class eliminates a lot of the use cases > for a "named object" feature, which is the main reason I'm trying to > hold out for the more general functionality - I still have hope that > we'll strike on a syntax for named assignments that is clear enough > and clean enough that none of the core devs actively oppose it. I agree that named constants don't have to be ints -- though for most types other than ints, my main requirement for enums (that they print themselves nicely) is not an issue. Regarding Ben Finney's requirement that enums primarily define unique values and that enums from two different classes must compare unequal even if their int values are equal, I really don't care about that. You can use isinstance() if you want to check that. -- --Guido van Rossum (python.org/~guido) From barry at python.org Thu Jul 28 19:12:46 2011 From: barry at python.org (Barry Warsaw) Date: Thu, 28 Jul 2011 13:12:46 -0400 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> Message-ID: <20110728131246.4aff20f8@resist.wooz.org> On Jul 27, 2011, at 07:21 PM, Guido van Rossum wrote: >It's just possible that there's no way to define enums that neither >introduced a new (class) scope nor requires a lot of redundant typing. Ezio had a very nice suggestion, which I've implemented in my experimental branch, e.g.: >>> foo = sys.modules['foo'] >>> Colors.inject(foo) .inject() takes anything that implements the setattr() protocol. So that you could then do: >>> import foo >>> foo.red Colors.red In my experimental branch I've also made EnumValues subclasses of ints. This breaks one test in my Mailman 3 test suite, since I was using a custom encoder for json.dumps() which knew how to encode EnumValues. However, the json module doesn't allow you to override the encoding of basic types, so whereas before my extended encoder's .default() method got called for all the EnumValues being encoded in a REST response, they are now encoded *incorrectly* as their integer values by default. I'd need to figure out a workaround for that, but it does show that EnumValue-as-int is a backward incompatible change. A few other things I've done in my experimental branch: - Added an optional argument `iterable` to make_enum() so that you can use the convenience function to auto-assign integer values other than sequentially incrementing from 1. E.g. >>> def enumiter(): ... start = 1 ... while True: ... yield start ... start <<= 1 >>> make_enum('Flags', 'a b c d e f g', enumiter()) - Renamed attributes .enumclass -> .enum and .enumname -> .name (hey, if you're going to be backward incompatible...) If you want to play with it: $ bzr branch lp:~barry/flufl.enum/asint or take a look at: http://bazaar.launchpad.net/~barry/flufl.enum/asint/files The new using.txt file is at: http://tinyurl.com/3s85oq7 Let me know what you think, while I figure out a workaround for the incompatible change. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Thu Jul 28 19:23:55 2011 From: barry at python.org (Barry Warsaw) Date: Thu, 28 Jul 2011 13:23:55 -0400 Subject: [Python-ideas] Enums References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <87fwlrpfio.fsf@benfinney.id.au> <4E30BF52.6030708@canterbury.ac.nz> Message-ID: <20110728132355.658dd0f8@resist.wooz.org> On Jul 28, 2011, at 01:45 PM, Greg Ewing wrote: >If you mean that it's important for an enum *not* to have an ordering, I >disagree. There are some use cases that benefit from having an ordering, and >some that don't. For those that don't, personally I don't care whether it >has an ordering or not. Based on my recent experiment (see my other follow up), values-as-ints can have unintended side-effects. It's clear that making this change would be backward incompatible to my existing library, and I think it indicates that there are indeed separate use cases for ordered enums and unordered enums. I think if I were to accept values-as-ints as a new feature for my library, I'd want to make it a separate class. Yes, it complicates the library but maybe the two design requirements are just incompatible. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Thu Jul 28 19:27:47 2011 From: barry at python.org (Barry Warsaw) Date: Thu, 28 Jul 2011 13:27:47 -0400 Subject: [Python-ideas] Enums In-Reply-To: <87wrf3nh57.fsf@benfinney.id.au> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <87wrf3nh57.fsf@benfinney.id.au> Message-ID: <20110728132747.78414325@resist.wooz.org> On Jul 28, 2011, at 02:49 PM, Ben Finney wrote: >What I'd hold out for, though, is:: > > >>> Color.red == Fruit.tomato > False > >That is, all of the values from Color should compare as inequal with any >other value. Does it have to be an equality test, or is identity tests enough? >This is one of the main features to want from an enumerated type, IMO: >to have a set of values that are distinct from any other value, that >won't be accidentally equal to any other value, and have helpful string >representations. > >> (Here I am consistent with the behavior of True and False.) > >Do you see that (the behaviour of True and False comparing equal with >integers) as anything more than backward-compatible baggage? > >To me, if I had the time machine, a proper representation of a boolean >type would have True and False as distinct values, never comparing equal >with any other value. Except possibly for ancient modules, does anybody still actually use the int-iness of bools explicitly? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From guido at python.org Thu Jul 28 19:41:05 2011 From: guido at python.org (Guido van Rossum) Date: Thu, 28 Jul 2011 10:41:05 -0700 Subject: [Python-ideas] Enums In-Reply-To: <20110728131246.4aff20f8@resist.wooz.org> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> Message-ID: On Thu, Jul 28, 2011 at 10:12 AM, Barry Warsaw wrote: > On Jul 27, 2011, at 07:21 PM, Guido van Rossum wrote: > >>It's just possible that there's no way to define enums that neither >>introduced a new (class) scope nor requires a lot of redundant typing. > > Ezio had a very nice suggestion, which I've implemented in my experimental > branch, e.g.: > > ? ?>>> foo = sys.modules['foo'] > ? ?>>> Colors.inject(foo) > > .inject() takes anything that implements the setattr() protocol. ?So that you > could then do: > > ? ?>>> import foo > ? ?>>> foo.red > ? ?Colors.red Nice, but it still hides the definitions from static analyzers. It seems I am almost unique in my insistence that Python's dynamic features be used very sparingly. :-) > In my experimental branch I've also made EnumValues subclasses of ints. ?This > breaks one test in my Mailman 3 test suite, since I was using a custom encoder > for json.dumps() which knew how to encode EnumValues. ?However, the json > module doesn't allow you to override the encoding of basic types, so whereas > before my extended encoder's .default() method got called for all the > EnumValues being encoded in a REST response, they are now encoded > *incorrectly* as their integer values by default. ?I'd need to figure out a > workaround for that, but it does show that EnumValue-as-int is a backward > incompatible change. Sure. Almost any change is backward incompatible... > A few other things I've done in my experimental branch: > > - Added an optional argument `iterable` to make_enum() so that you can use the > ?convenience function to auto-assign integer values other than sequentially > ?incrementing from 1. ?E.g. > > ? ?>>> def enumiter(): > ? ?... ? ? start = 1 > ? ?... ? ? while True: > ? ?... ? ? ? ? yield start > ? ?... ? ? ? ? start <<= 1 > ? ?>>> make_enum('Flags', 'a b c d e f g', enumiter()) > ? ? > > - Renamed attributes .enumclass -> .enum and .enumname -> .name (hey, if > ?you're going to be backward incompatible...) > > If you want to play with it: > > ? ?$ bzr branch lp:~barry/flufl.enum/asint > > or take a look at: > > ? ?http://bazaar.launchpad.net/~barry/flufl.enum/asint/files > > The new using.txt file is at: > > ? ?http://tinyurl.com/3s85oq7 > > Let me know what you think, while I figure out a workaround for the > incompatible change. > > Cheers, > -Barry > -- --Guido van Rossum (python.org/~guido) From ethan at stoneleaf.us Thu Jul 28 19:57:01 2011 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 28 Jul 2011 10:57:01 -0700 Subject: [Python-ideas] Enums In-Reply-To: <20110728131246.4aff20f8@resist.wooz.org> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> Message-ID: <4E31A2ED.3030802@stoneleaf.us> Barry Warsaw wrote: > In my experimental branch I've also made EnumValues subclasses of ints. This > breaks one test in my Mailman 3 test suite, since I was using a custom encoder > for json.dumps() which knew how to encode EnumValues. However, the json > module doesn't allow you to override the encoding of basic types, Isn't this a problem with json? An EnumValue is a subclass, after all, not a plain ol' straight-up int -- I would think that json should realize a subclass may be different and go the .default() route. ~Ethan~ From guido at python.org Thu Jul 28 19:42:24 2011 From: guido at python.org (Guido van Rossum) Date: Thu, 28 Jul 2011 10:42:24 -0700 Subject: [Python-ideas] Enums In-Reply-To: <20110728132747.78414325@resist.wooz.org> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <87wrf3nh57.fsf@benfinney.id.au> <20110728132747.78414325@resist.wooz.org> Message-ID: On Thu, Jul 28, 2011 at 10:27 AM, Barry Warsaw wrote: > Except possibly for ancient modules, does anybody still actually use the > int-iness of bools explicitly? I do, but maybe I myself qualify as an ancient module. :-) The bool-ness of ints is still used widely of course... -- --Guido van Rossum (python.org/~guido) From barry at python.org Thu Jul 28 19:48:46 2011 From: barry at python.org (Barry Warsaw) Date: Thu, 28 Jul 2011 13:48:46 -0400 Subject: [Python-ideas] Enums References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <87wrf3nh57.fsf@benfinney.id.au> <20110728132747.78414325@resist.wooz.org> Message-ID: <20110728134846.761c0ca8@resist.wooz.org> On Jul 28, 2011, at 10:42 AM, Guido van Rossum wrote: >I do, but maybe I myself qualify as an ancient module. :-) >>> from __future__ import guido File "", line 1 SyntaxError: future feature guido is not defined Even more troubling though is >>> from __future__ import bdfl File "", line 1 SyntaxError: future feature bdfl is not defined :) >The bool-ness of ints is still used widely of course... Yep, definitely. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Thu Jul 28 19:46:52 2011 From: barry at python.org (Barry Warsaw) Date: Thu, 28 Jul 2011 13:46:52 -0400 Subject: [Python-ideas] Enums References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <4E31A2ED.3030802@stoneleaf.us> Message-ID: <20110728134652.26fdb6c3@resist.wooz.org> On Jul 28, 2011, at 10:57 AM, Ethan Furman wrote: >Barry Warsaw wrote: >> In my experimental branch I've also made EnumValues subclasses of ints. This >> breaks one test in my Mailman 3 test suite, since I was using a custom encoder >> for json.dumps() which knew how to encode EnumValues. However, the json >> module doesn't allow you to override the encoding of basic types, > >Isn't this a problem with json? An EnumValue is a subclass, after all, not a >plain ol' straight-up int -- I would think that json should realize a >subclass may be different and go the .default() route. On first blush, I'd agree. _iterencode() just does an isinstance check, which is why this gets caught in the trap. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From mal at egenix.com Thu Jul 28 20:16:33 2011 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 28 Jul 2011 20:16:33 +0200 Subject: [Python-ideas] Enums In-Reply-To: <20110728132747.78414325@resist.wooz.org> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <87wrf3nh57.fsf@benfinney.id.au> <20110728132747.78414325@resist.wooz.org> Message-ID: <4E31A781.3050909@egenix.com> Barry Warsaw wrote: > Except possibly for ancient modules, does anybody still actually use the > int-iness of bools explicitly? I'm not exactly sure what you mean with "int-iness", but C extensions typically do care about the type and use PyInt_* APIs for checking and processing them. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 28 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From bruce at leapyear.org Thu Jul 28 20:25:48 2011 From: bruce at leapyear.org (Bruce Leban) Date: Thu, 28 Jul 2011 11:25:48 -0700 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> Message-ID: On Thu, Jul 28, 2011 at 10:41 AM, Guido van Rossum wrote: > > Nice, but it still hides the definitions from static analyzers. > > It seems I am almost unique in my insistence that Python's dynamic > features be used very sparingly. :-) Having worked recently on a project that made extensive use of dynamics, I am on your side. In that project, I found it completely impossible to trace a path between any two points in the program short of stepping through it with a debugger. There are classes and functions that are never explicitly referenced anywhere that are found via reflection and then instantiated and called and it is a nightmare. On the other hand, I love dynamic features for writing tests. On Thu, Jul 28, 2011 at 10:12 AM, Barry Warsaw wrote: > > Ezio had a very nice suggestion, which I've implemented in my experimental > branch, e.g.: > > >>> foo = sys.modules['foo'] > >>> Colors.inject(foo) > > .inject() takes anything that implements the setattr() protocol. So that > you > could then do: > > >>> import foo > >>> foo.red > Colors.red So in this example, how would I figure out where foo.red is set? I can't do it by looking at the source unless I know about Colors.inject. And what happens if my Weather class has a value Snow and then someone adds that to the Colors enum and clobbers Weather.Snow. I didn't change my class but suddenly I can't make it Snow. --- Bruce Follow me: http://www.twitter.com/Vroo http://www.vroospeak.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From pyideas at rebertia.com Thu Jul 28 20:44:23 2011 From: pyideas at rebertia.com (Chris Rebert) Date: Thu, 28 Jul 2011 11:44:23 -0700 Subject: [Python-ideas] Enums In-Reply-To: <20110728131246.4aff20f8@resist.wooz.org> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> Message-ID: On Thu, Jul 28, 2011 at 10:12 AM, Barry Warsaw wrote: > A few other things I've done in my experimental branch: > > - Added an optional argument `iterable` to make_enum() so that you can use the > ?convenience function to auto-assign integer values other than sequentially > ?incrementing from 1. ?E.g. > > ? ?>>> def enumiter(): > ? ?... ? ? start = 1 > ? ?... ? ? while True: > ? ?... ? ? ? ? yield start > ? ?... ? ? ? ? start <<= 1 > ? ?>>> make_enum('Flags', 'a b c d e f g', enumiter()) > ? ? Very neat, albeit of questionable enum-ness. > - Renamed attributes .enumclass -> .enum and .enumname -> .name (hey, if > ?you're going to be backward incompatible...) Excellent. That just leaves SomeEnum(...) vs.SomeEnum[...]. Cheers, Chris From steve at pearwood.info Thu Jul 28 22:13:11 2011 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 29 Jul 2011 06:13:11 +1000 Subject: [Python-ideas] Enums In-Reply-To: <20110728131246.4aff20f8@resist.wooz.org> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> Message-ID: <4E31C2D7.7090509@pearwood.info> Barry Warsaw wrote: > On Jul 27, 2011, at 07:21 PM, Guido van Rossum wrote: > >> It's just possible that there's no way to define enums that neither >> introduced a new (class) scope nor requires a lot of redundant typing. > > Ezio had a very nice suggestion, which I've implemented in my experimental > branch, e.g.: > > >>> foo = sys.modules['foo'] > >>> Colors.inject(foo) > > .inject() takes anything that implements the setattr() protocol. So that you > could then do: > > >>> import foo > >>> foo.red > Colors.red I've been reading this thread with a growing sense of dismay. I started off excited by the idea that enums would be added to the language, and thinking that Raymond's opposition was uncalled for. Then, as more and more features and metafeatures have been proposed, I've now changed my mind completely and am now with Raymond. I don't even know what these "enums" actually are any more. Part integer, part string, part namespace, part named constant (only not actually constant)... It seems to me that people are trying to dump a whole lot of only vaguely related functionality into a single concept. Including now a proposal to encourage monkey-patching modules by making it an official enum method. For something that is conceptually so simple, this is becoming awfully complicated. -- Steven From sven at marnach.net Thu Jul 28 22:28:38 2011 From: sven at marnach.net (Sven Marnach) Date: Thu, 28 Jul 2011 21:28:38 +0100 Subject: [Python-ideas] Enums In-Reply-To: <20110728132747.78414325@resist.wooz.org> References: <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <87wrf3nh57.fsf@benfinney.id.au> <20110728132747.78414325@resist.wooz.org> Message-ID: <20110728202838.GS3363@pantoffel-wg.de> Barry Warsaw schrieb am Do, 28. Jul 2011, um 13:27:47 -0400: > Except possibly for ancient modules, does anybody still actually use the > int-iness of bools explicitly? There definitely are applications of using bools as ints. Alex Martelli strongly supports this point of view in this post: http://stackoverflow.com/questions/3174392/is-it-pythonic-to-use-bools-as-ints/3175293#3175293 -- Sven From raymond.hettinger at gmail.com Thu Jul 28 23:16:38 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Thu, 28 Jul 2011 14:16:38 -0700 Subject: [Python-ideas] Enums In-Reply-To: <20110728202838.GS3363@pantoffel-wg.de> References: <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <87wrf3nh57.fsf@benfinney.id.au> <20110728132747.78414325@resist.wooz.org> <20110728202838.GS3363@pantoffel-wg.de> Message-ID: Wikipedia has a nice survey and comparison of enums in other languages: http://en.wikipedia.org/wiki/Enumerated_type Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikegraham at gmail.com Thu Jul 28 23:56:29 2011 From: mikegraham at gmail.com (Mike Graham) Date: Thu, 28 Jul 2011 17:56:29 -0400 Subject: [Python-ideas] A function that gives you the number of items in an arbitrary iterable (not len) Message-ID: There are a couple patterns, some uglier than others, which I have seen occasionally pop up which could be solved by a new builtin or stdlib that works like def exhaust(it): count = 0 for item in it: count += 1 return count Here are some situations where such a function could come in handy: 1. I want to know how many conditions satisfy a condition. I can currently determine this with code like sum(itertools.imap(cond, it)) or one of several other options. Oftentimes these take longer to read than they should. 2. I simply want to evaluate all the items in a lazy iterable. The nicest way to do this is to the effect of "for item in it: pass", though I have seen creative solutions like collections.deque(iterator, 0) (to try to speed up the loop). 3. You simply want to know the length of a non-sequence iterable. One current way to do this is sum(1 for item in it) (although I think I've seen "for count, _ in enumerate(it): pass;; count += 1"). I think the introduction of a function that works like this might make Python easier to read occasionally. I've seen these all pop up, but only occasionally. None are hard to perform as is, but each can feel a little messy. Does anyone have any input? Mike From ncoghlan at gmail.com Fri Jul 29 00:26:04 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 29 Jul 2011 08:26:04 +1000 Subject: [Python-ideas] Enums In-Reply-To: <4E31C2D7.7090509@pearwood.info> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <4E31C2D7.7090509@pearwood.info> Message-ID: On Fri, Jul 29, 2011 at 6:13 AM, Steven D'Aprano wrote: > I don't even know what these "enums" actually are any more. Part integer, > part string, part namespace, part named constant (only not actually > constant)... It seems to me that people are trying to dump a whole lot of > only vaguely related functionality into a single concept. Including now a > proposal to encourage monkey-patching modules by making it an official enum > method. > > For something that is conceptually so simple, this is becoming awfully > complicated. Yeah, until this thread reminded me, I'd actually forgotten my own reasons for objecting to standardising enums in the past and proposing named values instead. I think it mostly has to do with degrees of freedom in the design. With named values, there are only a few elements that require explicit design decisions: - ints only, or arbitrary types? (my recipe is designed to work with arbitrary types) - effect on repr() (my recipe modifies it to include a name without breaking any existing repr/eval round trip support) - effect on ascii() (inherits the repr() change, since ascii() is just repr() with a post-processing step) - effect on str() (my recipe makes sure to leave it alone to maximise compatibility) - effect on serialisation (my recipe makes sure to leave it alone to maximise compatibility) - effect on everything else (my recipe makes sure to leave it alone to maximise compatibility) If any application or library wants to name values retrieved from elsewhere (including previously pickled or serialised named values) then it is up to them to apply the transformation at the input points (e.g. a HTTP support library might create a mapping from server response codes to named values, but those values would still be serialised as ordinary integers). It's a (relatively) simple, flexible technique to improve introspection without being overly prescriptive about how to use it. Most design decisions are handled by the basic rule that named values *are* functionally the same as the bare value, they just happen to have been given a specific name that is available for introspection (and is included in the default repr output). Once you take the step up to the enum level and start dictating the *grouping* behaviour along with the naming behaviour, then a whole host of other design issues arise (in addition to the ones that arise for general purpose named values): - behaviour of identity comparisons - behaviour of equivalence comparisons - behaviour of ordering comparisons - behaviour of isinstance checks - API for explicit conversions to and from the underlying type - whether or not to support implicit conversions to and from the underlying type - how to introspect the group contents By giving up the "this is just an ordinary value with a name" design guideline, enums get into a lot of highly arguable territory. Guido *could* cut through that by BDFL fiat if he wanted to, but I'm not sure he *should* - depending on the use case, the preferred answers to some of the design decisions mentioned above are going to legitimately vary. Regards, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From greg.ewing at canterbury.ac.nz Fri Jul 29 01:24:29 2011 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 29 Jul 2011 11:24:29 +1200 Subject: [Python-ideas] Enums In-Reply-To: <1311836890.7662.133.camel@Gutsy> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <1311836890.7662.133.camel@Gutsy> Message-ID: <4E31EFAD.8000109@canterbury.ac.nz> ron3200 wrote: > When these objects are passed around, (to/from functions or methods), > they will most likely be rebound to more generic names that don't match > their string repr anyway. That doesn't matter, any more than it does for functions and classes. The point is to associate *some* descriptive name with the value for display purposes. > What name would you give... > > colors = [] > colors.append(Enum(1)) > assert str(colors[0]) == ? Obviously you can't infer any descriptive name from that. Automatic name assignment requires a dedicated syntax, IMO. We just need to wait for the BDFL to find one that he doesn't hate. (Or to stop hating one that he currently hates -- it's happened before...) -- Greg From fuzzyman at gmail.com Fri Jul 29 01:42:13 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Fri, 29 Jul 2011 00:42:13 +0100 Subject: [Python-ideas] Enums In-Reply-To: <20110728131246.4aff20f8@resist.wooz.org> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> Message-ID: On 28 July 2011 18:12, Barry Warsaw wrote: > On Jul 27, 2011, at 07:21 PM, Guido van Rossum wrote: > > >It's just possible that there's no way to define enums that neither > >introduced a new (class) scope nor requires a lot of redundant typing. > > Ezio had a very nice suggestion, which I've implemented in my experimental > branch, e.g.: > > >>> foo = sys.modules['foo'] > >>> Colors.inject(foo) > > Why not Colors.inject(__name__)? This is one line less, the same everywhere, and covers the common case. You could dispatch on string / object if you want to support both. Michael > .inject() takes anything that implements the setattr() protocol. So that > you > could then do: > > >>> import foo > >>> foo.red > Colors.red > > In my experimental branch I've also made EnumValues subclasses of ints. > This > breaks one test in my Mailman 3 test suite, since I was using a custom > encoder > for json.dumps() which knew how to encode EnumValues. However, the json > module doesn't allow you to override the encoding of basic types, so > whereas > before my extended encoder's .default() method got called for all the > EnumValues being encoded in a REST response, they are now encoded > *incorrectly* as their integer values by default. I'd need to figure out a > workaround for that, but it does show that EnumValue-as-int is a backward > incompatible change. > > A few other things I've done in my experimental branch: > > - Added an optional argument `iterable` to make_enum() so that you can use > the > convenience function to auto-assign integer values other than sequentially > incrementing from 1. E.g. > > >>> def enumiter(): > ... start = 1 > ... while True: > ... yield start > ... start <<= 1 > >>> make_enum('Flags', 'a b c d e f g', enumiter()) > > > - Renamed attributes .enumclass -> .enum and .enumname -> .name (hey, if > you're going to be backward incompatible...) > > If you want to play with it: > > $ bzr branch lp:~barry/flufl.enum/asint > > or take a look at: > > http://bazaar.launchpad.net/~barry/flufl.enum/asint/files > > The new using.txt file is at: > > http://tinyurl.com/3s85oq7 > > Let me know what you think, while I figure out a workaround for the > incompatible change. > > Cheers, > -Barry > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.hettinger at gmail.com Fri Jul 29 01:58:42 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Thu, 28 Jul 2011 16:58:42 -0700 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> Message-ID: <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> On Jul 28, 2011, at 4:42 PM, Michael Foord wrote: > > Why not Colors.inject(__name__)? Because enums aren't import enough to warrant introducing little atrocities into the language. Raymond P.S. I used the term "language" instead of stdlib because I expect enums to be like decorators and context managers in that they will be used almost everywhere -- you won't be able to ignore them. Effectively, they will become a core language feature. From fuzzyman at gmail.com Fri Jul 29 02:03:47 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Fri, 29 Jul 2011 01:03:47 +0100 Subject: [Python-ideas] Enums In-Reply-To: <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> Message-ID: On 29 July 2011 00:58, Raymond Hettinger wrote: > > On Jul 28, 2011, at 4:42 PM, Michael Foord wrote: > > > > > Why not Colors.inject(__name__)? > > Because enums aren't import enough to warrant introducing little atrocities > into the language. > > Uh ?? It's the same code just taking a module name instead of requiring you to pull the module out of sys.modules yourself. It's not an atrocity at all. Michael > Raymond > > > P.S. I used the term "language" instead of stdlib because I expect > enums to be like decorators and context managers in that they > will be used almost everywhere -- you won't be able to ignore them. > Effectively, they will become a core language feature. -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at gmail.com Fri Jul 29 02:15:46 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Fri, 29 Jul 2011 01:15:46 +0100 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> Message-ID: On 29 July 2011 01:03, Michael Foord wrote: > > > On 29 July 2011 00:58, Raymond Hettinger wrote: > >> >> On Jul 28, 2011, at 4:42 PM, Michael Foord wrote: >> >> > >> > Why not Colors.inject(__name__)? >> >> Because enums aren't import enough to warrant introducing little >> atrocities into the language. >> >> > Uh ?? It's the same code just taking a module name instead of requiring you > to pull the module out of sys.modules yourself. It's not an atrocity at all. > To make it clear, the one-line-works-everywhere alternative people will use is: Colors.inject(sys.modules[__name__]) Which is clearly worse. Michael > > Michael > > >> Raymond >> >> >> P.S. I used the term "language" instead of stdlib because I expect >> enums to be like decorators and context managers in that they >> will be used almost everywhere -- you won't be able to ignore them. >> Effectively, they will become a core language feature. > > > > > -- > > http://www.voidspace.org.uk/ > > May you do good and not evil > > May you find forgiveness for yourself and forgive others > May you share freely, never taking more than you give. > -- the sqlite blessing http://www.sqlite.org/different.html > > > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Jul 29 02:27:44 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 29 Jul 2011 10:27:44 +1000 Subject: [Python-ideas] Enums In-Reply-To: <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> Message-ID: On Fri, Jul 29, 2011 at 9:58 AM, Raymond Hettinger wrote: > P.S. ?I used the term "language" instead of stdlib because I expect > enums to be like decorators and context managers in that they > will be used almost everywhere -- you won't be able to ignore them. > Effectively, they will become a core language feature. I agree with this - and I think it's the named value aspect that will make them so pervasive. It's why I keep hitting on collections.namedtuple as a source of inspiration - it adds as little as possible to ordinary tuples to provide the additional benefits. Enums, on the other hand, come with a lot of excess baggage that many potential use cases simply don't need. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From amcnabb at mcnabbs.org Fri Jul 29 01:56:31 2011 From: amcnabb at mcnabbs.org (Andrew McNabb) Date: Thu, 28 Jul 2011 17:56:31 -0600 Subject: [Python-ideas] A function that gives you the number of items in an arbitrary iterable (not len) In-Reply-To: References: Message-ID: <20110728235630.GG14865@mcnabbs.org> On Thu, Jul 28, 2011 at 05:56:29PM -0400, Mike Graham wrote: > There are a couple patterns, some uglier than others, which I have > seen occasionally pop up which could be solved by a new builtin or > stdlib that works like > > def exhaust(it): > count = 0 > for item in it: > count += 1 > return count I've done this before: sum(1 for x in it) I don't do it frequently enough that a builtin would seem necessary. -- Andrew McNabb http://www.mcnabbs.org/andrew/ PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868 From ncoghlan at gmail.com Fri Jul 29 02:37:46 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 29 Jul 2011 10:37:46 +1000 Subject: [Python-ideas] A function that gives you the number of items in an arbitrary iterable (not len) In-Reply-To: <20110728235630.GG14865@mcnabbs.org> References: <20110728235630.GG14865@mcnabbs.org> Message-ID: On Fri, Jul 29, 2011 at 9:56 AM, Andrew McNabb wrote: > I've done this before: > > sum(1 for x in it) > > I don't do it frequently enough that a builtin would seem necessary. Yeah, sum(1 for x in itr) and sum(1 for x in itr if cond(x)) are the current idiomatic ways to handle the situations mentioned in the OP. You could suggest itertools.exhaust(itr) to Raymond Hettinger, though. The sum() idiom works, but isn't particular obvious to readers. OTOH, it's also pretty trivial to hide the sum() idiom inside a function. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From fuzzyman at gmail.com Fri Jul 29 02:46:39 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Fri, 29 Jul 2011 01:46:39 +0100 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> Message-ID: On 29 July 2011 01:27, Nick Coghlan wrote: > On Fri, Jul 29, 2011 at 9:58 AM, Raymond Hettinger > wrote: > > P.S. I used the term "language" instead of stdlib because I expect > > enums to be like decorators and context managers in that they > > will be used almost everywhere -- you won't be able to ignore them. > > Effectively, they will become a core language feature. > > I agree with this - and I think it's the named value aspect that will > make them so pervasive. It's why I keep hitting on > collections.namedtuple as a source of inspiration - it adds as little > as possible to ordinary tuples to provide the additional benefits. > Enums, on the other hand, come with a lot of excess baggage that many > potential use cases simply don't need. > > I agree "named values" would get us *most* of the value, and would both be less contentious and provide a common building block for third party libraries to build interesting and perhaps esoteric uses on. I *like* grouped named values (I think they make nice apis to read and use when used appropriately), but hey-ho. Michael > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From anacrolix at gmail.com Fri Jul 29 02:56:08 2011 From: anacrolix at gmail.com (Matt Joiner) Date: Fri, 29 Jul 2011 10:56:08 +1000 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <87wrf3nh57.fsf@benfinney.id.au> <20110728132747.78414325@resist.wooz.org> <20110728202838.GS3363@pantoffel-wg.de> Message-ID: I agree with Steven, At first the thought of a standardized enum sounded great. Enums are named constants, this is something Python can already give us. Sometimes the enum values are such that they can be treated as flags. This is a property of integers. In most languages enums are integer constants, because those languages are so much closer to the machine. However Python is a runtime, and it's not Pythonic to attempt to enforce values to be constant, let require that they be integers. Furthermore, it's dangerous to suggest that this be commonplace by adding it to the standard library, when superior Pythonic alternatives exist. If you remove the constraint that enums necessarily correspond to integers, suddenly there is no point: enums for the sake of compatibility and debugging C interfaces, where not even C has that feature built in. While out of tradition, enum like values are still done with integers even in pure Python programs, it seems to me the proper approach is something like: Colors = {'red', 'blue', 'green'} color = 'red' OpenFlags = {'append', 'binary', 'exclusive', 'read', } py_open('/some/path', {'binary', 'exclusive', 'read', 'write'}) Adding enums to the standard library is going to cause a rush to rewrite everything to use enums, where perfectly good alternatives are already in use. I think it should just be accepted that C interfaces work with integers and move on. On Fri, Jul 29, 2011 at 7:16 AM, Raymond Hettinger wrote: > Wikipedia has a nice survey and comparison of enums in other languages: > http://en.wikipedia.org/wiki/Enumerated_type > > Raymond > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > From tjreedy at udel.edu Fri Jul 29 03:12:10 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 28 Jul 2011 21:12:10 -0400 Subject: [Python-ideas] Enums In-Reply-To: <20110728132747.78414325@resist.wooz.org> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <87wrf3nh57.fsf@benfinney.id.au> <20110728132747.78414325@resist.wooz.org> Message-ID: On 7/28/2011 1:27 PM, Barry Warsaw wrote: > Except possibly for ancient modules, does anybody still actually use the > int-iness of bools explicitly? I consider the int-ness of bools to be a positive and useful feature and I will continue to use it when opportune. From guido at python.org Fri Jul 29 05:11:42 2011 From: guido at python.org (Guido van Rossum) Date: Thu, 28 Jul 2011 20:11:42 -0700 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> Message-ID: On Thu, Jul 28, 2011 at 5:46 PM, Michael Foord wrote: > I agree "named values" would get us *most* of the value, and would both be > less contentious and provide a common building block for third party > libraries to build interesting and perhaps esoteric uses on. > > I *like* grouped named values (I think they make nice apis to read and use > when used appropriately), but hey-ho. These could be two separate features. Named values could be a building block for a built-in Enum type -- the named value would take care of the str() and repr(), while the Enum type would only have to take care of the nice syntax ("class Color(Enum): red = 1; ...") and just create a bunch of named values in the class scope. The metaclass would do something like "for key in classdict: classdict[key] = namedvalue(key, classdict[key])" (though it would probably have to skip certain keys). -- --Guido van Rossum (python.org/~guido) From ericsnowcurrently at gmail.com Fri Jul 29 06:40:32 2011 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Thu, 28 Jul 2011 22:40:32 -0600 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> Message-ID: On Thu, Jul 28, 2011 at 9:11 PM, Guido van Rossum wrote: > On Thu, Jul 28, 2011 at 5:46 PM, Michael Foord wrote: >> I agree "named values" would get us *most* of the value, and would both be >> less contentious and provide a common building block for third party >> libraries to build interesting and perhaps esoteric uses on. >> >> I *like* grouped named values (I think they make nice apis to read and use >> when used appropriately), but hey-ho. > > These could be two separate features. Named values could be a building > block for a built-in Enum type -- the named value would take care of > the str() and repr(), while the Enum type would only have to take care > of the nice syntax ("class Color(Enum): red = 1; ...") and just create > a bunch of named values in the class scope. The metaclass would do > something like "for key in classdict: classdict[key] = namedvalue(key, > classdict[key])" (though it would probably have to skip certain keys). ISTM a NamedValue could be either a copy of the original value or a wrapper around the original. Examples of the former include Nick's recipe and Barry's flufl.enum asint branch. Examples of the latter include the flufl.enum trunk. The copy version requires making the copy, which is less convenient if the value's type needs more than the instance to make a copy. The wrapper version probably wouldn't work well as a drop-in replacement for the original value. So if a NamedValue were to make it into the stdlib, which would it be? Does NamedValue add much value if not coupled with grouping? -eric p.s. FWIW I think Nick's recipe for NamedValue is the right approach for the copied-value model. > --Guido van Rossum (python.org/~guido) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From ben+python at benfinney.id.au Fri Jul 29 07:00:26 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jul 2011 15:00:26 +1000 Subject: [Python-ideas] Enums References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> Message-ID: <87zkjxn0k5.fsf@benfinney.id.au> Guido van Rossum writes: > These could be two separate features. Named values could be a building > block for a built-in Enum type -- the named value would take care of > the str() and repr(), while the Enum type would only have to take care > of the nice syntax ("class Color(Enum): red = 1; ...") and just create > a bunch of named values in the class scope. +1. As for EnumValue instances having a defined order, thanks to my reading of I can now say that some of us want an enumerated type that is not ordinal, and others want an ordinal type. And non-ordinal enumerated types, built-in, could then further be used as the basis for a custom ordinal type. Hence: * a built-in ?named value? feature * a built-in, or standard-library, non-ordinal enumerated type feature (using the ?named value? feature) * a standard-library ordinal type feature (using the enumerated type feature) Would that make us all happy? -- \ ?We are stuck with technology when what we really want is just | `\ stuff that works.? ?Douglas Adams | _o__) | Ben Finney From ncoghlan at gmail.com Fri Jul 29 07:29:03 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 29 Jul 2011 15:29:03 +1000 Subject: [Python-ideas] Enums In-Reply-To: <87zkjxn0k5.fsf@benfinney.id.au> References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> <87zkjxn0k5.fsf@benfinney.id.au> Message-ID: On Fri, Jul 29, 2011 at 3:00 PM, Ben Finney wrote: > Guido van Rossum writes: > >> These could be two separate features. Named values could be a building >> block for a built-in Enum type -- the named value would take care of >> the str() and repr(), while the Enum type would only have to take care >> of the nice syntax ("class Color(Enum): red = 1; ...") and just create >> a bunch of named values in the class scope. > > +1. > > As for EnumValue instances having a defined order, thanks to my reading > of I can now say that > some of us want an enumerated type that is not ordinal, and others want > an ordinal type. > > And non-ordinal enumerated types, built-in, could then further be used > as the basis for a custom ordinal type. > > Hence: > > * a built-in ?named value? feature > > * a built-in, or standard-library, non-ordinal enumerated type feature > ?(using the ?named value? feature) > > * a standard-library ordinal type feature > ?(using the enumerated type feature) > > Would that make us all happy? I like the tower, but I'd like to see us approach this incrementally. 1. Design and provide a standard "named value" feature (CPython unfortunately makes it difficult to do this efficiently - we need to add state to hold the name, but doing that in a way that is compatible with the C layout of tuples and similar objects means the state has to be stored in __dict__) 2. Perhaps provide a non-ordinal NamedGroup feature (avoiding the Enum naming with its 'enumerated' ordinal implications) Explicitly naming several values is likely to be clumsy enough that a metaclass based solution may be a beneficial convenience API, but Named Groups wouldn't enforce any additional behaviour on their members. I'm not completely persuaded on this point as yet, but I can see the merit of the argument. As an initial step, I thinks that's the absolute limit of how far we should go. I see it as similar to the handling of function annotations - we can provide a standard infrastructure for naming instances without otherwise affecting their behaviour, and then allow third parties to build on that and see where they take it. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Fri Jul 29 07:36:35 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 29 Jul 2011 15:36:35 +1000 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> Message-ID: On Fri, Jul 29, 2011 at 2:40 PM, Eric Snow wrote: > ISTM a NamedValue could be either a copy of the original value or a > wrapper around the original. ?Examples of the former include Nick's > recipe and Barry's flufl.enum asint branch. ?Examples of the latter > include the flufl.enum trunk. > > The copy version requires making the copy, which is less convenient if > the value's type needs more than the instance to make a copy. ?The > wrapper version probably wouldn't work well as a drop-in replacement > for the original value. A weakref.proxy style API can actually be a fairly effective dropin replacement for a type. However, it's a high overhead and quite complicated solution. > So if a NamedValue were to make it into the stdlib, which would it be? Arguments can be made in both directions. The main benefit of inheritance is that things behave correctly by default, and the only behavioural differences are those we add. Proxying can definitely work, but it's actually hard to do such that special method invocations work properly without giving false positives on hasattr() and ABC isinstance() checks. > ?Does NamedValue add much value if not coupled with grouping? Yeah, it provides the oft-cited debugging benefits of having the name of the value appearing in the representation. That's invaluable for cases where you don't actually control the display operation (e.g logging messages in library code). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ezio.melotti at gmail.com Fri Jul 29 07:50:34 2011 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Fri, 29 Jul 2011 08:50:34 +0300 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> Message-ID: <4E324A2A.8050503@gmail.com> An HTML attachment was scrubbed... URL: From paul at colomiets.name Fri Jul 29 09:55:05 2011 From: paul at colomiets.name (Paul Colomiets) Date: Fri, 29 Jul 2011 10:55:05 +0300 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725133432.3f29960c@resist.wooz.org> <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> Message-ID: On Fri, Jul 29, 2011 at 6:11 AM, Guido van Rossum wrote: > On Thu, Jul 28, 2011 at 5:46 PM, Michael Foord wrote: >> I agree "named values" would get us *most* of the value, and would both be >> less contentious and provide a common building block for third party >> libraries to build interesting and perhaps esoteric uses on. >> >> I *like* grouped named values (I think they make nice apis to read and use >> when used appropriately), but hey-ho. > > These could be two separate features. Named values could be a building > block for a built-in Enum type -- the named value would take care of > the str() and repr(), while the Enum type would only have to take care > of the nice syntax ("class Color(Enum): red = 1; ...") and just create > a bunch of named values in the class scope. The metaclass would do > something like "for key in classdict: classdict[key] = namedvalue(key, > classdict[key])" (though it would probably have to skip certain keys). > The problem with named value is that: namedvalue(1, "red") Will give you are repr of "red". And you don't know whether it's TerminalColors.red or WebSafe.red or BuildbotState.red. And most of us will be too lazy to add group name or module name directly into the named value (partially because it's repeating yourself). So the big feature of flufl.enum is classname (call it group name) in repr and isinstance check. -- Paul From mal at egenix.com Fri Jul 29 11:37:19 2011 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 29 Jul 2011 11:37:19 +0200 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> Message-ID: <4E327F4F.2000104@egenix.com> Paul Colomiets wrote: > On Fri, Jul 29, 2011 at 6:11 AM, Guido van Rossum wrote: >> On Thu, Jul 28, 2011 at 5:46 PM, Michael Foord wrote: >>> I agree "named values" would get us *most* of the value, and would both be >>> less contentious and provide a common building block for third party >>> libraries to build interesting and perhaps esoteric uses on. >>> >>> I *like* grouped named values (I think they make nice apis to read and use >>> when used appropriately), but hey-ho. >> >> These could be two separate features. Named values could be a building >> block for a built-in Enum type -- the named value would take care of >> the str() and repr(), while the Enum type would only have to take care >> of the nice syntax ("class Color(Enum): red = 1; ...") and just create >> a bunch of named values in the class scope. The metaclass would do >> something like "for key in classdict: classdict[key] = namedvalue(key, >> classdict[key])" (though it would probably have to skip certain keys). >> > > The problem with named value is that: > > namedvalue(1, "red") > > Will give you are repr of "red". And you don't know whether it's > TerminalColors.red or WebSafe.red or BuildbotState.red. And > most of us will be too lazy to add group name or module name > directly into the named value (partially because it's repeating > yourself). > > So the big feature of flufl.enum is classname (call it group > name) in repr and isinstance check. A meta class variant could easily add class and module names to the attrbute name, if needed/wanted. Just to make sure: namedvalue() will just change the repr() of the value, not the str() of it, right ? I think that's essential for making namedvalue()s useful in practice, since you mostly need this for debugging and error reporting and don't want the namevalue aspect of a constant to get in the way of its normal use in e.g. conversion to text formats such as JSON, XML, etc. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 29 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From ncoghlan at gmail.com Fri Jul 29 16:55:32 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 30 Jul 2011 00:55:32 +1000 Subject: [Python-ideas] Enums In-Reply-To: <4E327F4F.2000104@egenix.com> References: <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> <4E327F4F.2000104@egenix.com> Message-ID: On Fri, Jul 29, 2011 at 7:37 PM, M.-A. Lemburg wrote: > Paul Colomiets wrote: >> The problem with named value is that: >> >> ? ? namedvalue(1, "red") >> >> Will give you are repr of "red". And you don't know whether it's >> TerminalColors.red or WebSafe.red or BuildbotState.red. And >> most of us will be too lazy to add group name or module name >> directly into the named value (partially because it's repeating >> yourself). >> >> So the big feature of flufl.enum is classname (call it group >> name) in repr and isinstance check. > > A meta class variant could easily add class and module > names to the attrbute name, if needed/wanted. Given that the status quo is for "TerminalColors.red" et al to just display as "1" (or whatever integers they're assigned), I'm finding it hard to consider this a particularly significant criticism. If we don't even know what *kind* of values are being passed to a debugging message, we have bigger problems than whether or not those values have been given names (and searching for all named values of 'red' with a value of '1' is going to create a much smaller resulting set than searching for all values of '1' ). > Just to make sure: > > namedvalue() will just change the repr() of the value, not > the str() of it, right ? Yeah, I figured out the mechanics needed to make that work properly when writing up the recipe for the cookbook (http://code.activestate.com/recipes/577810-named-values/) > I think that's essential for making namedvalue()s useful > in practice, since you mostly need this for debugging > and error reporting and don't want the namevalue aspect > of a constant to get in the way of its normal use in > e.g. conversion to text formats such as JSON, XML, etc. Yeah, the case I first thought of was writing numbers to CSV files, but it's really any data export operation. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From guido at python.org Fri Jul 29 17:09:45 2011 From: guido at python.org (Guido van Rossum) Date: Fri, 29 Jul 2011 08:09:45 -0700 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> <4E327F4F.2000104@egenix.com> Message-ID: On Fri, Jul 29, 2011 at 7:55 AM, Nick Coghlan wrote: > On Fri, Jul 29, 2011 at 7:37 PM, M.-A. Lemburg wrote: >> Paul Colomiets wrote: >>> The problem with named value is that: >>> >>> ? ? namedvalue(1, "red") >>> >>> Will give you are repr of "red". And you don't know whether it's >>> TerminalColors.red or WebSafe.red or BuildbotState.red. And >>> most of us will be too lazy to add group name or module name >>> directly into the named value (partially because it's repeating >>> yourself). >>> >>> So the big feature of flufl.enum is classname (call it group >>> name) in repr and isinstance check. >> >> A meta class variant could easily add class and module >> names to the attrbute name, if needed/wanted. > > Given that the status quo is for "TerminalColors.red" et al to just > display as "1" (or whatever integers they're assigned), I'm finding it > hard to consider this a particularly significant criticism. If we > don't even know what *kind* of values are being passed to a debugging > message, we have bigger problems than whether or not those values have > been given names (and searching for all named values of 'red' with a > value of '1' is going to create a much smaller resulting set than > searching for all values of '1' ). > >> Just to make sure: >> >> namedvalue() will just change the repr() of the value, not >> the str() of it, right ? > > Yeah, I figured out the mechanics needed to make that work properly > when writing up the recipe for the cookbook > (http://code.activestate.com/recipes/577810-named-values/) > >> I think that's essential for making namedvalue()s useful >> in practice, since you mostly need this for debugging >> and error reporting and don't want the namevalue aspect >> of a constant to get in the way of its normal use in >> e.g. conversion to text formats such as JSON, XML, etc. > > Yeah, the case I first thought of was writing numbers to CSV files, > but it's really any data export operation. That's assuming the data export format doesn't support enums. I would fine enums whose str() is just an int pretty annoying. Note that str(True) == repr(True) == 'True' and that's how I'd like it to be for enums in general. If you want the int value, use int(e). If you want the class name, use e.__class__.__name__ (maybe e.__class__ is what you're after anyway). My observation is that most enums have sufficiently unique names that during a debugging session the class is not a big mystery; it's the mapping from value to name that's hard to memorize. As a compromise, I could live with str(e) == 'red' and repr(e) == 'Color.red'. But I don't think I could stomach str(e) == 1. -- --Guido van Rossum (python.org/~guido) From ncoghlan at gmail.com Fri Jul 29 17:26:01 2011 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 30 Jul 2011 01:26:01 +1000 Subject: [Python-ideas] Enums In-Reply-To: References: <20110725151826.12564b47@resist.wooz.org> <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> <4E327F4F.2000104@egenix.com> Message-ID: On Sat, Jul 30, 2011 at 1:09 AM, Guido van Rossum wrote: > That's assuming the data export format doesn't support enums. > > I would fine enums whose str() is just an int pretty annoying. Note > that str(True) == repr(True) == 'True' and that's how I'd like it to > be for enums in general. If you want the int value, use int(e). If you > want the class name, use e.__class__.__name__ (maybe e.__class__ is > what you're after anyway). My observation is that most enums have > sufficiently unique names that during a debugging session the class is > not a big mystery; it's the mapping from value to name that's hard to > memorize. As a compromise, I could live with str(e) == 'red' and > repr(e) == 'Color.red'. But I don't think I could stomach str(e) == 1. For enums that define new values which are not equivalent to the underlying values, I agree with you, but my concept for named values is to allow them to be almost entirely a drop-in replacement for the values they replace. For that more general use case, changing the result of str() would be problematic, since str() is often used to create command line APIs and other things. With a named value 'nv.__name__' would give the name, while 'str(nv)' would give the original unadorned string representation. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ethan at stoneleaf.us Fri Jul 29 18:37:33 2011 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 29 Jul 2011 09:37:33 -0700 Subject: [Python-ideas] change NoneType, NotImplementedType, & ellipses to return the appropriate singleton Message-ID: <4E32E1CD.4020906@stoneleaf.us> Consider: Python 3.2 (r32:88445, Feb 20 2011, 21:29:02) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. --> for ins in ({0:'0'}, (1,), set([2, 3]), [4, 5], 6, 'seven', ... 8.0, True, None): ... print(type(ins)) ... type(ins)() ... {} () set() [] 0 '' 0.0 False Traceback (most recent call last): File "", line 3, in TypeError: cannot create 'NoneType' instances I hit this issue when I had a dictionary of types to instantiate, one of which was NoneType. Is there any reason why we shouldn't change these three to produce the singleton instance instead of raising an error? Pro: They would then be consistent with every other built-in which will return either a new instance (int, str, float, etc), or the singleton instance that already exists (bool, and sometimes int & str), as well as no longer needing the `lambda:None` workaround (or worse, a try-except if you don't know what types are coming in). Con: Might be code that relies on the exception being raised -- is anybody aware of any? ~Ethan~ From guido at python.org Fri Jul 29 19:03:56 2011 From: guido at python.org (Guido van Rossum) Date: Fri, 29 Jul 2011 10:03:56 -0700 Subject: [Python-ideas] change NoneType, NotImplementedType, & ellipses to return the appropriate singleton In-Reply-To: <4E32E1CD.4020906@stoneleaf.us> References: <4E32E1CD.4020906@stoneleaf.us> Message-ID: I think it is fine if type(None)() returns None instead of raising an exception. On Fri, Jul 29, 2011 at 9:37 AM, Ethan Furman wrote: > Consider: > > Python 3.2 (r32:88445, Feb 20 2011, 21:29:02) [MSC v.1500 32 bit (Intel)] on > win32 > Type "help", "copyright", "credits" or "license" for more information. > --> for ins in ({0:'0'}, (1,), set([2, 3]), [4, 5], 6, 'seven', > ... 8.0, True, None): > ... ? print(type(ins)) > ... ? type(ins)() > ... > > {} > > () > > set() > > [] > > 0 > > '' > > 0.0 > > False > > Traceback (most recent call last): > ?File "", line 3, in > TypeError: cannot create 'NoneType' instances > > I hit this issue when I had a dictionary of types to instantiate, one of > which was NoneType. > > Is there any reason why we shouldn't change these three to produce the > singleton instance instead of raising an error? > > Pro: ?They would then be consistent with every other built-in which will > return either a new instance (int, str, float, etc), or the singleton > instance that already exists (bool, and sometimes int & str), as well as no > longer needing the `lambda:None` workaround (or worse, a try-except if you > don't know what types are coming in). > > Con: ?Might be code that relies on the exception being raised -- is anybody > aware of any? > > ~Ethan~ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- --Guido van Rossum (python.org/~guido) From mal at egenix.com Fri Jul 29 19:07:02 2011 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 29 Jul 2011 19:07:02 +0200 Subject: [Python-ideas] Enums In-Reply-To: References: <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> <4E327F4F.2000104@egenix.com> Message-ID: <4E32E8B6.8070005@egenix.com> Guido van Rossum wrote: > On Fri, Jul 29, 2011 at 7:55 AM, Nick Coghlan wrote: >> On Fri, Jul 29, 2011 at 7:37 PM, M.-A. Lemburg wrote: >>> Paul Colomiets wrote: >>>> The problem with named value is that: >>>> >>>> namedvalue(1, "red") >>>> >>>> Will give you are repr of "red". And you don't know whether it's >>>> TerminalColors.red or WebSafe.red or BuildbotState.red. And >>>> most of us will be too lazy to add group name or module name >>>> directly into the named value (partially because it's repeating >>>> yourself). >>>> >>>> So the big feature of flufl.enum is classname (call it group >>>> name) in repr and isinstance check. >>> >>> A meta class variant could easily add class and module >>> names to the attrbute name, if needed/wanted. >> >> Given that the status quo is for "TerminalColors.red" et al to just >> display as "1" (or whatever integers they're assigned), I'm finding it >> hard to consider this a particularly significant criticism. If we >> don't even know what *kind* of values are being passed to a debugging >> message, we have bigger problems than whether or not those values have >> been given names (and searching for all named values of 'red' with a >> value of '1' is going to create a much smaller resulting set than >> searching for all values of '1' ). >> >>> Just to make sure: >>> >>> namedvalue() will just change the repr() of the value, not >>> the str() of it, right ? >> >> Yeah, I figured out the mechanics needed to make that work properly >> when writing up the recipe for the cookbook >> (http://code.activestate.com/recipes/577810-named-values/) >> >>> I think that's essential for making namedvalue()s useful >>> in practice, since you mostly need this for debugging >>> and error reporting and don't want the namevalue aspect >>> of a constant to get in the way of its normal use in >>> e.g. conversion to text formats such as JSON, XML, etc. >> >> Yeah, the case I first thought of was writing numbers to CSV files, >> but it's really any data export operation. > > That's assuming the data export format doesn't support enums. > > I would fine enums whose str() is just an int pretty annoying. Note > that str(True) == repr(True) == 'True' and that's how I'd like it to > be for enums in general. If you want the int value, use int(e). If you > want the class name, use e.__class__.__name__ (maybe e.__class__ is > what you're after anyway). My observation is that most enums have > sufficiently unique names that during a debugging session the class is > not a big mystery; it's the mapping from value to name that's hard to > memorize. As a compromise, I could live with str(e) == 'red' and > repr(e) == 'Color.red'. But I don't think I could stomach str(e) == 1. The situation with enums is different than for booleans: many data interchange formats out there support true/false directly, so there's little to change and little breakage involved. And it's easy to fix, since you only to deal with two such enums. However, when you change existing code constants to enums, chances are high that your interchange libraries are going to fail because of this, or your application will start providing broken data to external resources, without you knowing. Since it's easy to switch from %s to %r, if needed, I don't see much of a problem with having str(e) continue to return the integer string. Regarding the repr(e): This should really be user-defined. There situations where you want to see both the integer code and description in the repr(e), e.g. when dealing with error codes where you quickly need to communicate the error, so a format like '[123] Error contacting server' would be better than just 'Error contacting server'. In other cases, the subsystem is important, so getting '[Server][FTP] Permission error' is more useful than just 'Permission error'. And in yet other circumstances, you want to see the defining module and class. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 29 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From _ at lvh.cc Fri Jul 29 19:08:38 2011 From: _ at lvh.cc (Laurens Van Houtven) Date: Fri, 29 Jul 2011 19:08:38 +0200 Subject: [Python-ideas] change NoneType, NotImplementedType, & ellipses to return the appropriate singleton In-Reply-To: <4E32E1CD.4020906@stoneleaf.us> References: <4E32E1CD.4020906@stoneleaf.us> Message-ID: That looks pretty reasonable to me, and, more importantly, would increase consistency as you point out :) (I have had a similar issue in the past, where I expected this to work but it didn't -- so I replaced NoneType with lambda: None.) cheers lvh -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Fri Jul 29 19:15:25 2011 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 30 Jul 2011 03:15:25 +1000 Subject: [Python-ideas] change NoneType, NotImplementedType, & ellipses to return the appropriate singleton In-Reply-To: <4E32E1CD.4020906@stoneleaf.us> References: <4E32E1CD.4020906@stoneleaf.us> Message-ID: <4E32EAAD.3070403@pearwood.info> Ethan Furman wrote: > > Traceback (most recent call last): > File "", line 3, in > TypeError: cannot create 'NoneType' instances > > I hit this issue when I had a dictionary of types to instantiate, one of > which was NoneType. > > Is there any reason why we shouldn't change these three to produce the > singleton instance instead of raising an error? +0.5 -- Steven From guido at python.org Fri Jul 29 19:20:26 2011 From: guido at python.org (Guido van Rossum) Date: Fri, 29 Jul 2011 10:20:26 -0700 Subject: [Python-ideas] Enums In-Reply-To: <4E32E8B6.8070005@egenix.com> References: <20110727190057.4558d211@resist.wooz.org> <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> <4E327F4F.2000104@egenix.com> <4E32E8B6.8070005@egenix.com> Message-ID: Well, it does sound like we're not going to agree on how enums should behave. There are too many different use cases. So I am giving up. Sorry. (Also I'm going on vacation and will be mostly off-line for the next 2-3 weeks.) --Guido On Fri, Jul 29, 2011 at 10:07 AM, M.-A. Lemburg wrote: > Guido van Rossum wrote: >> On Fri, Jul 29, 2011 at 7:55 AM, Nick Coghlan wrote: >>> On Fri, Jul 29, 2011 at 7:37 PM, M.-A. Lemburg wrote: >>>> Paul Colomiets wrote: >>>>> The problem with named value is that: >>>>> >>>>> ? ? namedvalue(1, "red") >>>>> >>>>> Will give you are repr of "red". And you don't know whether it's >>>>> TerminalColors.red or WebSafe.red or BuildbotState.red. And >>>>> most of us will be too lazy to add group name or module name >>>>> directly into the named value (partially because it's repeating >>>>> yourself). >>>>> >>>>> So the big feature of flufl.enum is classname (call it group >>>>> name) in repr and isinstance check. >>>> >>>> A meta class variant could easily add class and module >>>> names to the attrbute name, if needed/wanted. >>> >>> Given that the status quo is for "TerminalColors.red" et al to just >>> display as "1" (or whatever integers they're assigned), I'm finding it >>> hard to consider this a particularly significant criticism. If we >>> don't even know what *kind* of values are being passed to a debugging >>> message, we have bigger problems than whether or not those values have >>> been given names (and searching for all named values of 'red' with a >>> value of '1' is going to create a much smaller resulting set than >>> searching for all values of '1' ). >>> >>>> Just to make sure: >>>> >>>> namedvalue() will just change the repr() of the value, not >>>> the str() of it, right ? >>> >>> Yeah, I figured out the mechanics needed to make that work properly >>> when writing up the recipe for the cookbook >>> (http://code.activestate.com/recipes/577810-named-values/) >>> >>>> I think that's essential for making namedvalue()s useful >>>> in practice, since you mostly need this for debugging >>>> and error reporting and don't want the namevalue aspect >>>> of a constant to get in the way of its normal use in >>>> e.g. conversion to text formats such as JSON, XML, etc. >>> >>> Yeah, the case I first thought of was writing numbers to CSV files, >>> but it's really any data export operation. >> >> That's assuming the data export format doesn't support enums. >> >> I would fine enums whose str() is just an int pretty annoying. Note >> that str(True) == repr(True) == 'True' and that's how I'd like it to >> be for enums in general. If you want the int value, use int(e). If you >> want the class name, use e.__class__.__name__ (maybe e.__class__ is >> what you're after anyway). My observation is that most enums have >> sufficiently unique names that during a debugging session the class is >> not a big mystery; it's the mapping from value to name that's hard to >> memorize. As a compromise, I could live with str(e) == 'red' and >> repr(e) == 'Color.red'. But I don't think I could stomach str(e) == 1. > > The situation with enums is different than for booleans: many > data interchange formats out there support true/false directly, > so there's little to change and little breakage involved. > And it's easy to fix, since you only to deal with two such > enums. > > However, when you change existing code constants to enums, > chances are high that your interchange libraries are going > to fail because of this, or your application will start > providing broken data to external resources, without you > knowing. > > Since it's easy to switch from %s to %r, if needed, I don't > see much of a problem with having str(e) continue to return > the integer string. > > Regarding the repr(e): This should really be user-defined. > There situations where you want to see both the integer > code and description in the repr(e), e.g. when dealing with > error codes where you quickly need to communicate the > error, so a format like '[123] Error contacting server' > would be better than just 'Error contacting server'. > > In other cases, the subsystem is important, so getting > '[Server][FTP] Permission error' is more useful than > just 'Permission error'. > > And in yet other circumstances, you want to see the > defining module and class. > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source ?(#1, Jul 29 2011) >>>> Python/Zope Consulting and Support ... ? ? ? ?http://www.egenix.com/ >>>> mxODBC.Zope.Database.Adapter ... ? ? ? ? ? ? http://zope.egenix.com/ >>>> mxODBC, mxDateTime, mxTextTools ... ? ? ? ?http://python.egenix.com/ > ________________________________________________________________________ > > ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: > > > ? eGenix.com Software, Skills and Services GmbH ?Pastor-Loeh-Str.48 > ? ?D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > ? ? ? ? ? Registered at Amtsgericht Duesseldorf: HRB 46611 > ? ? ? ? ? ? ? http://www.egenix.com/company/contact/ > -- --Guido van Rossum (python.org/~guido) From mal at egenix.com Fri Jul 29 19:40:14 2011 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 29 Jul 2011 19:40:14 +0200 Subject: [Python-ideas] Enums In-Reply-To: <4E32E8B6.8070005@egenix.com> References: <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> <4E327F4F.2000104@egenix.com> <4E32E8B6.8070005@egenix.com> Message-ID: <4E32F07E.3000107@egenix.com> M.-A. Lemburg wrote: > Guido van Rossum wrote: >> On Fri, Jul 29, 2011 at 7:55 AM, Nick Coghlan wrote: >>> On Fri, Jul 29, 2011 at 7:37 PM, M.-A. Lemburg wrote: >>>> Paul Colomiets wrote: >>>>> The problem with named value is that: >>>>> >>>>> namedvalue(1, "red") >>>>> >>>>> Will give you are repr of "red". And you don't know whether it's >>>>> TerminalColors.red or WebSafe.red or BuildbotState.red. And >>>>> most of us will be too lazy to add group name or module name >>>>> directly into the named value (partially because it's repeating >>>>> yourself). >>>>> >>>>> So the big feature of flufl.enum is classname (call it group >>>>> name) in repr and isinstance check. >>>> >>>> A meta class variant could easily add class and module >>>> names to the attrbute name, if needed/wanted. >>> >>> Given that the status quo is for "TerminalColors.red" et al to just >>> display as "1" (or whatever integers they're assigned), I'm finding it >>> hard to consider this a particularly significant criticism. If we >>> don't even know what *kind* of values are being passed to a debugging >>> message, we have bigger problems than whether or not those values have >>> been given names (and searching for all named values of 'red' with a >>> value of '1' is going to create a much smaller resulting set than >>> searching for all values of '1' ). >>> >>>> Just to make sure: >>>> >>>> namedvalue() will just change the repr() of the value, not >>>> the str() of it, right ? >>> >>> Yeah, I figured out the mechanics needed to make that work properly >>> when writing up the recipe for the cookbook >>> (http://code.activestate.com/recipes/577810-named-values/) >>> >>>> I think that's essential for making namedvalue()s useful >>>> in practice, since you mostly need this for debugging >>>> and error reporting and don't want the namevalue aspect >>>> of a constant to get in the way of its normal use in >>>> e.g. conversion to text formats such as JSON, XML, etc. >>> >>> Yeah, the case I first thought of was writing numbers to CSV files, >>> but it's really any data export operation. >> >> That's assuming the data export format doesn't support enums. >> >> I would fine enums whose str() is just an int pretty annoying. Note >> that str(True) == repr(True) == 'True' and that's how I'd like it to >> be for enums in general. If you want the int value, use int(e). If you >> want the class name, use e.__class__.__name__ (maybe e.__class__ is >> what you're after anyway). My observation is that most enums have >> sufficiently unique names that during a debugging session the class is >> not a big mystery; it's the mapping from value to name that's hard to >> memorize. As a compromise, I could live with str(e) == 'red' and >> repr(e) == 'Color.red'. But I don't think I could stomach str(e) == 1. > > The situation with enums is different than for booleans: many > data interchange formats out there support true/false directly, > so there's little to change and little breakage involved. > And it's easy to fix, since you only to deal with two such > enums. > > However, when you change existing code constants to enums, > chances are high that your interchange libraries are going > to fail because of this, or your application will start > providing broken data to external resources, without you > knowing. > > Since it's easy to switch from %s to %r, if needed, I don't > see much of a problem with having str(e) continue to return > the integer string. > > Regarding the repr(e): This should really be user-defined. > There situations where you want to see both the integer > code and description in the repr(e), e.g. when dealing with > error codes where you quickly need to communicate the > error, so a format like '[123] Error contacting server' > would be better than just 'Error contacting server'. > > In other cases, the subsystem is important, so getting > '[Server][FTP] Permission error' is more useful than > just 'Permission error'. > > And in yet other circumstances, you want to see the > defining module and class. Thinking about this some more: it would be good to have attributes .name - namedvalue name .value - namedvalue value object on enums and have str(e) == str(e.value) and repr(e) == e.name. BTW: Would it be possible to have work for more than just integers ? If not, "namedint" may be a more intuitive name. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 29 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mal at egenix.com Fri Jul 29 19:51:42 2011 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 29 Jul 2011 19:51:42 +0200 Subject: [Python-ideas] Enums In-Reply-To: References: <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> <4E327F4F.2000104@egenix.com> <4E32E8B6.8070005@egenix.com> Message-ID: <4E32F32E.6070709@egenix.com> Guido van Rossum wrote: > Well, it does sound like we're not going to agree on how enums should > behave. There are too many different use cases. So I am giving up. > Sorry. (Also I'm going on vacation and will be mostly off-line for the > next 2-3 weeks.) Perhaps we should try to address those use cases one by one, instead of trying to lump everything into one new object type. OTOH, I also do think we should be conservative about adding such new features to the stdlib. There are enough existing implementations on PyPI and the cookbook (including the one that implements the rejected PEP 354) to choose from and there is no one-fits-all type of named-int-that-belongs-to-a-value-set. Have a nice vacation, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 29 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From guido at python.org Fri Jul 29 19:56:50 2011 From: guido at python.org (Guido van Rossum) Date: Fri, 29 Jul 2011 10:56:50 -0700 Subject: [Python-ideas] Enums In-Reply-To: <4E32F07E.3000107@egenix.com> References: <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> <4E327F4F.2000104@egenix.com> <4E32E8B6.8070005@egenix.com> <4E32F07E.3000107@egenix.com> Message-ID: On Fri, Jul 29, 2011 at 10:40 AM, M.-A. Lemburg wrote: > M.-A. Lemburg wrote: >> Guido van Rossum wrote: >>> On Fri, Jul 29, 2011 at 7:55 AM, Nick Coghlan wrote: >>>> On Fri, Jul 29, 2011 at 7:37 PM, M.-A. Lemburg wrote: >>>>> Paul Colomiets wrote: >>>>>> The problem with named value is that: >>>>>> >>>>>> ? ? namedvalue(1, "red") >>>>>> >>>>>> Will give you are repr of "red". And you don't know whether it's >>>>>> TerminalColors.red or WebSafe.red or BuildbotState.red. And >>>>>> most of us will be too lazy to add group name or module name >>>>>> directly into the named value (partially because it's repeating >>>>>> yourself). >>>>>> >>>>>> So the big feature of flufl.enum is classname (call it group >>>>>> name) in repr and isinstance check. >>>>> >>>>> A meta class variant could easily add class and module >>>>> names to the attrbute name, if needed/wanted. >>>> >>>> Given that the status quo is for "TerminalColors.red" et al to just >>>> display as "1" (or whatever integers they're assigned), I'm finding it >>>> hard to consider this a particularly significant criticism. If we >>>> don't even know what *kind* of values are being passed to a debugging >>>> message, we have bigger problems than whether or not those values have >>>> been given names (and searching for all named values of 'red' with a >>>> value of '1' is going to create a much smaller resulting set than >>>> searching for all values of '1' ). >>>> >>>>> Just to make sure: >>>>> >>>>> namedvalue() will just change the repr() of the value, not >>>>> the str() of it, right ? >>>> >>>> Yeah, I figured out the mechanics needed to make that work properly >>>> when writing up the recipe for the cookbook >>>> (http://code.activestate.com/recipes/577810-named-values/) >>>> >>>>> I think that's essential for making namedvalue()s useful >>>>> in practice, since you mostly need this for debugging >>>>> and error reporting and don't want the namevalue aspect >>>>> of a constant to get in the way of its normal use in >>>>> e.g. conversion to text formats such as JSON, XML, etc. >>>> >>>> Yeah, the case I first thought of was writing numbers to CSV files, >>>> but it's really any data export operation. >>> >>> That's assuming the data export format doesn't support enums. >>> >>> I would fine enums whose str() is just an int pretty annoying. Note >>> that str(True) == repr(True) == 'True' and that's how I'd like it to >>> be for enums in general. If you want the int value, use int(e). If you >>> want the class name, use e.__class__.__name__ (maybe e.__class__ is >>> what you're after anyway). My observation is that most enums have >>> sufficiently unique names that during a debugging session the class is >>> not a big mystery; it's the mapping from value to name that's hard to >>> memorize. As a compromise, I could live with str(e) == 'red' and >>> repr(e) == 'Color.red'. But I don't think I could stomach str(e) == 1. >> >> The situation with enums is different than for booleans: many >> data interchange formats out there support true/false directly, >> so there's little to change and little breakage involved. >> And it's easy to fix, since you only to deal with two such >> enums. >> >> However, when you change existing code constants to enums, >> chances are high that your interchange libraries are going >> to fail because of this, or your application will start >> providing broken data to external resources, without you >> knowing. >> >> Since it's easy to switch from %s to %r, if needed, I don't >> see much of a problem with having str(e) continue to return >> the integer string. >> >> Regarding the repr(e): This should really be user-defined. >> There situations where you want to see both the integer >> code and description in the repr(e), e.g. when dealing with >> error codes where you quickly need to communicate the >> error, so a format like '[123] Error contacting server' >> would be better than just 'Error contacting server'. >> >> In other cases, the subsystem is important, so getting >> '[Server][FTP] Permission error' is more useful than >> just 'Permission error'. >> >> And in yet other circumstances, you want to see the >> defining module and class. > > Thinking about this some more: it would be good > to have attributes > > ?.name - namedvalue name > ?.value - namedvalue value object > > on enums and have str(e) == str(e.value) and > repr(e) == e.name. > > BTW: Would it be possible to have work for more than just > integers ? If not, "namedint" may be a more intuitive name. Nick's proposal for named values is to work for any type. But I think that the things called enums ought to be an int subclass. I guess if enums are named values their value attribute should return a plain int with the same value. -- --Guido van Rossum (python.org/~guido) From solipsis at pitrou.net Fri Jul 29 23:41:41 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Jul 2011 23:41:41 +0200 Subject: [Python-ideas] Enums References: <20110727205642.6317d0b4@resist.wooz.org> <20110728131246.4aff20f8@resist.wooz.org> <1124B58D-8AE6-419E-B2CC-10FD474C9F60@gmail.com> <4E327F4F.2000104@egenix.com> Message-ID: <20110729234141.1435c388@pitrou.net> On Sat, 30 Jul 2011 01:26:01 +1000 Nick Coghlan wrote: > On Sat, Jul 30, 2011 at 1:09 AM, Guido van Rossum wrote: > > That's assuming the data export format doesn't support enums. > > > > I would fine enums whose str() is just an int pretty annoying. Note > > that str(True) == repr(True) == 'True' and that's how I'd like it to > > be for enums in general. If you want the int value, use int(e). If you > > want the class name, use e.__class__.__name__ (maybe e.__class__ is > > what you're after anyway). My observation is that most enums have > > sufficiently unique names that during a debugging session the class is > > not a big mystery; it's the mapping from value to name that's hard to > > memorize. As a compromise, I could live with str(e) == 'red' and > > repr(e) == 'Color.red'. But I don't think I could stomach str(e) == 1. > > For enums that define new values which are not equivalent to the > underlying values, I agree with you, but my concept for named values > is to allow them to be almost entirely a drop-in replacement for the > values they replace. For that more general use case, changing the > result of str() would be problematic, since str() is often used to > create command line APIs and other things. I agree with that. repr() can be used to give further information (and repr() or even ascii() should always be used when debugging anyway, not str()). Realistically, many potential uses of enums are tied to some underlying C library - be it errnos, flags for os.open(), socket families, etc. It would probably be wrong for an enum implementation not to support these use cases. Regards Antoine. From solipsis at pitrou.net Fri Jul 29 23:42:44 2011 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Jul 2011 23:42:44 +0200 Subject: [Python-ideas] change NoneType, NotImplementedType, & ellipses to return the appropriate singleton References: <4E32E1CD.4020906@stoneleaf.us> Message-ID: <20110729234244.7964ca58@pitrou.net> On Fri, 29 Jul 2011 09:37:33 -0700 Ethan Furman wrote: > > Is there any reason why we shouldn't change these three to produce the > singleton instance instead of raising an error? +1 from me. Regards Antoine. From raymond.hettinger at gmail.com Fri Jul 29 23:59:02 2011 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Fri, 29 Jul 2011 14:59:02 -0700 Subject: [Python-ideas] change NoneType, NotImplementedType, & ellipses to return the appropriate singleton In-Reply-To: References: <4E32E1CD.4020906@stoneleaf.us> Message-ID: <98E58713-79D9-43B3-BE27-0ACA73362281@gmail.com> On Jul 29, 2011, at 10:03 AM, Guido van Rossum wrote: > I think it is fine if type(None)() returns None instead of raising an exception. +1 Raymond From python at mrabarnett.plus.com Sat Jul 30 00:04:57 2011 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 29 Jul 2011 23:04:57 +0100 Subject: [Python-ideas] change NoneType, NotImplementedType, & ellipses to return the appropriate singleton In-Reply-To: References: <4E32E1CD.4020906@stoneleaf.us> Message-ID: <4E332E89.4050602@mrabarnett.plus.com> On 29/07/2011 18:03, Guido van Rossum wrote: > I think it is fine if type(None)() returns None instead of raising an exception. > +1 From tjreedy at udel.edu Sat Jul 30 01:41:55 2011 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 29 Jul 2011 19:41:55 -0400 Subject: [Python-ideas] change NoneType, NotImplementedType, & ellipses to return the appropriate singleton In-Reply-To: <4E32E1CD.4020906@stoneleaf.us> References: <4E32E1CD.4020906@stoneleaf.us> Message-ID: On 7/29/2011 12:37 PM, Ethan Furman wrote: > Is there any reason why we shouldn't change these three to produce the > singleton instance instead of raising an error? You have all the approvals you need (congrats), so file a feature request and mention this thread or even better, add a iink mail.python.org/ References: <4E32E1CD.4020906@stoneleaf.us> Message-ID: On 7/29/2011 7:41 PM, Terry Reedy wrote: > On 7/29/2011 12:37 PM, Ethan Furman wrote: > >> Is there any reason why we shouldn't change these three to produce the >> singleton instance instead of raising an error? > > You have all the approvals you need (congrats), so file a feature > request and mention this thread or even better, add a iink > mail.python.org/ References: <4E32E1CD.4020906@stoneleaf.us> Message-ID: <96F24067-4309-43D3-9E39-D40775113D19@gmail.com> On Jul 29, 2011, at 4:41 PM, Terry Reedy wrote: > On 7/29/2011 12:37 PM, Ethan Furman wrote: > >> Is there any reason why we shouldn't change these three to produce the >> singleton instance instead of raising an error? > > You have all the approvals you need (congrats), so file a feature request and mention this thread or even better, add a iink > mail.python.org/ References: <4E32E1CD.4020906@stoneleaf.us> <96F24067-4309-43D3-9E39-D40775113D19@gmail.com> Message-ID: <4E335651.6030909@stoneleaf.us> Raymond Hettinger wrote: > On Jul 29, 2011, at 4:41 PM, Terry Reedy wrote: > >> On 7/29/2011 12:37 PM, Ethan Furman wrote: >> >>> Is there any reason why we shouldn't change these three to produce the >>> singleton instance instead of raising an error? >> You have all the approvals you need (congrats), so file a feature request and mention this thread or even better, add a iink >> mail.python.org/ > Too slow :-) Benjamin already implemented it and checked it in 20 minutes before your post. > Awesome. :) ~Ethan~ From ben+python at benfinney.id.au Sat Jul 30 04:04:20 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 30 Jul 2011 12:04:20 +1000 Subject: [Python-ideas] change NoneType, NotImplementedType, & ellipses to return the appropriate singleton References: <4E32E1CD.4020906@stoneleaf.us> <96F24067-4309-43D3-9E39-D40775113D19@gmail.com> <4E335651.6030909@stoneleaf.us> Message-ID: <87mxfwmsm3.fsf@benfinney.id.au> Ethan Furman writes: > Raymond Hettinger wrote: > > On Jul 29, 2011, at 4:41 PM, Terry Reedy wrote: > >> On 7/29/2011 12:37 PM, Ethan Furman wrote: > >>> Is there any reason why we shouldn't change these three to produce > >>> the singleton instance instead of raising an error? > >> You have all the approvals you need (congrats), so file a feature > >> request [?] > > Too slow :-) Benjamin already implemented it and checked it in 20 > > minutes before your post. > > Awesome. :) Congratulations, and thanks for taking this through the right channels. -- \ ?It is undesirable to believe a proposition when there is no | `\ ground whatever for supposing it true.? ?Bertrand Russell, _The | _o__) Value of Scepticism_, 1928 | Ben Finney From g.nius.ck at gmail.com Sun Jul 31 04:28:03 2011 From: g.nius.ck at gmail.com (Christopher King) Date: Sat, 30 Jul 2011 22:28:03 -0400 Subject: [Python-ideas] +1, +0.5 Message-ID: What does +1 or +0.5 mean. I assume that they are some sort of rating, so how does the rating system work. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From senthil at uthcode.com Sun Jul 31 04:53:11 2011 From: senthil at uthcode.com (Senthil Kumaran) Date: Sun, 31 Jul 2011 10:53:11 +0800 Subject: [Python-ideas] +1, +0.5 In-Reply-To: References: Message-ID: <20110731025311.GB2394@mathmagic> On Sat, Jul 30, 2011 at 10:28:03PM -0400, Christopher King wrote: > What does +1 or +0.5 mean. I assume that they are some sort of rating, so how > does the rating system work. Thanks http://www.python.org/dev/peps/pep-0010/ It is same as what you think it would be. :) -- Senthil