From barry at barrys-emacs.org Sat Jul 2 12:55:13 2016 From: barry at barrys-emacs.org (Barry Scott) Date: Sat, 2 Jul 2016 17:55:13 +0100 Subject: [Python-ideas] documentation for python3 super() Message-ID: <0973EF69-1F00-413A-8781-F9C83119B04D@barrys-emacs.org> I have read the python3.5 docs for super() and https://rhettinger.wordpress.com/2011/05/26/super-considered-super/. Between the two sources I still have no clear idea of what super() will do and why it will doe what it does. I hope to get feedback that can be used as the basis to update the python docs. For single inheritance the use of super() does not seem to have any surprises. But in the case of a class with multiple-inheritance more then 1 function may be called by 1 call to super(). What are the rules for which functions are called by super() in the multiple inheritance case? Is __init__ special or can other super() calls end up calling more then 1 function? What is the role of **kwds with super()? Here is the code I used to show the __init__ multiple calls. The attached example shows that one call to super() causes 2 to __init__ of bases of Person. But describe does not follow the pattern Age.describe is not called. Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: super_example.py Type: text/x-python-script Size: 820 bytes Desc: not available URL: From gvanrossum at gmail.com Sat Jul 2 13:58:30 2016 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sat, 2 Jul 2016 10:58:30 -0700 Subject: [Python-ideas] documentation for python3 super() In-Reply-To: <0973EF69-1F00-413A-8781-F9C83119B04D@barrys-emacs.org> References: <0973EF69-1F00-413A-8781-F9C83119B04D@barrys-emacs.org> Message-ID: No, super() does not (directly) call multiple functions. The function it calls has to call the next with another super() call. Also, __init__() is not special, nor is **kwds. super() itself is special, it knows the class and instance. There are conventions around all of this though. It may be worth documenting those, as long as it is made clear which part of the docs is about conventions (as opposed to how things work). There are also opinions about those conventions. Here I am not so sure that they belong in the docs. --Guido (mobile) On Jul 2, 2016 10:32 AM, "Barry Scott" wrote: > I have read the python3.5 docs for super() and > https://rhettinger.wordpress.com/2011/05/26/super-considered-super/. > > Between the two sources I still have no clear idea of what super() will > do and why it will doe what it does. > > I hope to get feedback that can be used as the basis to update the python > docs. > > For single inheritance the use of super() does not seem to have any > surprises. > > But in the case of a class with multiple-inheritance more then 1 function > may be called by 1 call to super(). > > What are the rules for which functions are called by super() in the > multiple > inheritance case? > > Is __init__ special or can other super() calls end up > calling more then 1 function? > > What is the role of **kwds with super()? > > Here is the code I used to show the __init__ multiple calls. > > The attached example shows that one call to super() causes 2 to > __init__ of bases of Person. But describe does not follow the pattern > Age.describe is not called. > > Barry > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gvanrossum at gmail.com Sat Jul 2 15:28:34 2016 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sat, 2 Jul 2016 12:28:34 -0700 Subject: [Python-ideas] documentation for python3 super() In-Reply-To: References: <0973EF69-1F00-413A-8781-F9C83119B04D@barrys-emacs.org> Message-ID: The trick is in the mro. If you print Person.__mro__ you'll get a list showing that its MRO (method resolution order) is Person, QObject, Age, object. In Person, super() refers to QObject (the next class in the MRO). In QObject, *for a Person instance*, super() refers to Age. In Age, in this case, super() refers to object. The methods on object don't call super. Hope this helps -- you should be able to get more help by Googling for something like MRO. On Sat, Jul 2, 2016 at 11:45 AM, Barry Scott wrote: > > On 2 Jul 2016, at 18:58, Guido van Rossum wrote: > > No, super() does not (directly) call multiple functions. The function it > calls has to call the next with another super() call. Also, __init__() is > not special, nor is **kwds > > > The thing I do not understand is why did super() call 2 __init__ functions > given the code I attached? > The one in QObject and the one in Age. > > This is the output I get with python 3.5: > > $ python3.5 super_example.py > instance of Person.__init__ > instance of QObject.__init__ > instance of Age.__init__ > Person.describe() > name: Barry > QObject.describe() > > I see no obvious code that should call Age.__init__, which is why I > concluded that there is something about super() that is not documented. > > super() itself is special, it knows the class and instance. > > There are conventions around all of this though. It may be worth documenting > those, as long as it is made clear which part of the docs is about > conventions (as opposed to how things work). > > There are also opinions about those conventions. Here I am not so sure that > they belong in the docs. > > Agreed. > > Barry > > --Guido (mobile) > > On Jul 2, 2016 10:32 AM, "Barry Scott" wrote: >> >> I have read the python3.5 docs for super() and >> https://rhettinger.wordpress.com/2011/05/26/super-considered-super/. >> >> Between the two sources I still have no clear idea of what super() will >> do and why it will doe what it does. >> >> I hope to get feedback that can be used as the basis to update the python >> docs. >> >> For single inheritance the use of super() does not seem to have any >> surprises. >> >> But in the case of a class with multiple-inheritance more then 1 function >> may be called by 1 call to super(). >> >> What are the rules for which functions are called by super() in the >> multiple >> inheritance case? >> >> Is __init__ special or can other super() calls end up >> calling more then 1 function? >> >> What is the role of **kwds with super()? >> >> Here is the code I used to show the __init__ multiple calls. >> >> The attached example shows that one call to super() causes 2 to >> __init__ of bases of Person. But describe does not follow the pattern >> Age.describe is not called. >> >> Barry >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ > > -- --Guido van Rossum (python.org/~guido) From barry at barrys-emacs.org Sat Jul 2 14:45:12 2016 From: barry at barrys-emacs.org (Barry Scott) Date: Sat, 2 Jul 2016 19:45:12 +0100 Subject: [Python-ideas] documentation for python3 super() In-Reply-To: References: <0973EF69-1F00-413A-8781-F9C83119B04D@barrys-emacs.org> Message-ID: > On 2 Jul 2016, at 18:58, Guido van Rossum wrote: > > No, super() does not (directly) call multiple functions. The function it calls has to call the next with another super() call. Also, __init__() is not special, nor is **kwds > The thing I do not understand is why did super() call 2 __init__ functions given the code I attached? The one in QObject and the one in Age. This is the output I get with python 3.5: $ python3.5 super_example.py instance of Person.__init__ instance of QObject.__init__ instance of Age.__init__ Person.describe() name: Barry QObject.describe() I see no obvious code that should call Age.__init__, which is why I concluded that there is something about super() that is not documented. > super() itself is special, it knows the class and instance. > > There are conventions around all of this though. It may be worth documenting those, as long as it is made clear which part of the docs is about conventions (as opposed to how things work). > > There are also opinions about those conventions. Here I am not so sure that they belong in the docs. > Agreed. Barry > --Guido (mobile) > > On Jul 2, 2016 10:32 AM, "Barry Scott" > wrote: > I have read the python3.5 docs for super() and > https://rhettinger.wordpress.com/2011/05/26/super-considered-super/ . > > Between the two sources I still have no clear idea of what super() will > do and why it will doe what it does. > > I hope to get feedback that can be used as the basis to update the python docs. > > For single inheritance the use of super() does not seem to have any surprises. > > But in the case of a class with multiple-inheritance more then 1 function > may be called by 1 call to super(). > > What are the rules for which functions are called by super() in the multiple > inheritance case? > > Is __init__ special or can other super() calls end up > calling more then 1 function? > > What is the role of **kwds with super()? > > Here is the code I used to show the __init__ multiple calls. > > The attached example shows that one call to super() causes 2 to > __init__ of bases of Person. But describe does not follow the pattern > Age.describe is not called. > > Barry > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From random832 at fastmail.com Sat Jul 2 16:18:55 2016 From: random832 at fastmail.com (Random832) Date: Sat, 02 Jul 2016 16:18:55 -0400 Subject: [Python-ideas] documentation for python3 super() In-Reply-To: References: <0973EF69-1F00-413A-8781-F9C83119B04D@barrys-emacs.org> Message-ID: <1467490735.1864525.655160481.48019550@webmail.messagingengine.com> On Sat, Jul 2, 2016, at 14:45, Barry Scott wrote: > > > On 2 Jul 2016, at 18:58, Guido van Rossum > > wrote: > > > > No, super() does not (directly) call multiple functions. The > > function it calls has to call the next with another super() call. > > Also, __init__() is not special, nor is **kwds > > > > The thing I do not understand is why did super() call 2 __init__ > functions given the code I attached? Because you called it twice. Well, actually three times: 1. Person calls super().__init__ which calls QObject.__init__. 2. QObject calls it *again*, which calls Age.__init__. 3. Age calls it again, which calls object.__init__, which does nothing and prints nothing. With each method you call, the named keywords are stripped off of **kwds; none of them are left by the time it gets to Age, so none are passed to object - otherwise you would get an error: TypeError: object.__init__() takes no parameters. From daoust.mj at gmail.com Sat Jul 2 19:24:25 2016 From: daoust.mj at gmail.com (Mark Daoust) Date: Sat, 2 Jul 2016 19:24:25 -0400 Subject: [Python-ideas] documentation for python3 super() In-Reply-To: References: <0973EF69-1F00-413A-8781-F9C83119B04D@barrys-emacs.org> Message-ID: >You should be able to get more help by Googling for something like MRO. Raymond's pycon2015 talk is also a good way to understand everything involved. https://www.youtube.com/watch?v=EiOglTERPEo Mark Daoust On Sat, Jul 2, 2016 at 3:28 PM, Guido van Rossum wrote: > The trick is in the mro. If you print Person.__mro__ you'll get a list > showing that its MRO (method resolution order) is Person, QObject, > Age, object. In Person, super() refers to QObject (the next class in > the MRO). In QObject, *for a Person instance*, super() refers to Age. > In Age, in this case, super() refers to object. The methods on object > don't call super. > > Hope this helps -- you should be able to get more help by Googling for > something like MRO. > > On Sat, Jul 2, 2016 at 11:45 AM, Barry Scott > wrote: > > > > On 2 Jul 2016, at 18:58, Guido van Rossum wrote: > > > > No, super() does not (directly) call multiple functions. The function it > > calls has to call the next with another super() call. Also, __init__() is > > not special, nor is **kwds > > > > > > The thing I do not understand is why did super() call 2 __init__ > functions > > given the code I attached? > > The one in QObject and the one in Age. > > > > This is the output I get with python 3.5: > > > > $ python3.5 super_example.py > > instance of Person.__init__ > > instance of QObject.__init__ > > instance of Age.__init__ > > Person.describe() > > name: Barry > > QObject.describe() > > > > I see no obvious code that should call Age.__init__, which is why I > > concluded that there is something about super() that is not documented. > > > > super() itself is special, it knows the class and instance. > > > > There are conventions around all of this though. It may be worth > documenting > > those, as long as it is made clear which part of the docs is about > > conventions (as opposed to how things work). > > > > There are also opinions about those conventions. Here I am not so sure > that > > they belong in the docs. > > > > Agreed. > > > > Barry > > > > --Guido (mobile) > > > > On Jul 2, 2016 10:32 AM, "Barry Scott" wrote: > >> > >> I have read the python3.5 docs for super() and > >> https://rhettinger.wordpress.com/2011/05/26/super-considered-super/. > >> > >> Between the two sources I still have no clear idea of what super() will > >> do and why it will doe what it does. > >> > >> I hope to get feedback that can be used as the basis to update the > python > >> docs. > >> > >> For single inheritance the use of super() does not seem to have any > >> surprises. > >> > >> But in the case of a class with multiple-inheritance more then 1 > function > >> may be called by 1 call to super(). > >> > >> What are the rules for which functions are called by super() in the > >> multiple > >> inheritance case? > >> > >> Is __init__ special or can other super() calls end up > >> calling more then 1 function? > >> > >> What is the role of **kwds with super()? > >> > >> Here is the code I used to show the __init__ multiple calls. > >> > >> The attached example shows that one call to super() causes 2 to > >> __init__ of bases of Person. But describe does not follow the pattern > >> Age.describe is not called. > >> > >> Barry > >> > >> _______________________________________________ > >> Python-ideas mailing list > >> Python-ideas at python.org > >> https://mail.python.org/mailman/listinfo/python-ideas > >> Code of Conduct: http://python.org/psf/codeofconduct/ > > > > > > > > -- > --Guido van Rossum (python.org/~guido) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at barrys-emacs.org Mon Jul 4 10:16:05 2016 From: barry at barrys-emacs.org (Barry Scott) Date: Mon, 4 Jul 2016 15:16:05 +0100 Subject: [Python-ideas] documentation for python3 super() In-Reply-To: References: <0973EF69-1F00-413A-8781-F9C83119B04D@barrys-emacs.org> Message-ID: <5978F8C1-AB17-4212-8448-03E13D43655A@barrys-emacs.org> > On 2 Jul 2016, at 20:28, Guido van Rossum wrote: > > The trick is in the mro. If you print Person.__mro__ you'll get a list > showing that its MRO (method resolution order) is Person, QObject, > Age, object. In Person, super() refers to QObject (the next class in > the MRO). In QObject, *for a Person instance*, super() refers to Age. > In Age, in this case, super() refers to object. The methods on object > don't call super. > > Hope this helps -- you should be able to get more help by Googling for > something like MRO The MRO?s importance is what I managed to not understand from the super() docs. Now I understand that I can explain the behaviour of the code example. The existing docs say: "The search order is same as that used by getattr() except that the type itself is skipped.? But getattr does not document its search order so that does not help. "getattr(object, name[, default]) Return the value of the named attribute of object. name must be a string. If the string is the name of one of the object?s attributes, the result is the value of that attribute. For example, getattr(x, 'foobar') is equivalent to x.foobar. If the named attribute does not exist, default is returned if provided, otherwise AttributeError is raised.? The MRO not mentioned, and not used? So do not mention? I think it would help anyone else that missed the point as I did to document exactly what super() does. In engineer English something like this. "super() uses the __mro__ as the order to search for the attribute to return. super() starts by finding the type in the __mro__ and then examines the following items until the attribute is found and returns it. If the attribute is not found AttributeError is raised.? Thoughts? Barry > . > > On Sat, Jul 2, 2016 at 11:45 AM, Barry Scott wrote: >> >> On 2 Jul 2016, at 18:58, Guido van Rossum wrote: >> >> No, super() does not (directly) call multiple functions. The function it >> calls has to call the next with another super() call. Also, __init__() is >> not special, nor is **kwds >> >> >> The thing I do not understand is why did super() call 2 __init__ functions >> given the code I attached? >> The one in QObject and the one in Age. >> >> This is the output I get with python 3.5: >> >> $ python3.5 super_example.py >> instance of Person.__init__ >> instance of QObject.__init__ >> instance of Age.__init__ >> Person.describe() >> name: Barry >> QObject.describe() >> >> I see no obvious code that should call Age.__init__, which is why I >> concluded that there is something about super() that is not documented. >> >> super() itself is special, it knows the class and instance. >> >> There are conventions around all of this though. It may be worth documenting >> those, as long as it is made clear which part of the docs is about >> conventions (as opposed to how things work). >> >> There are also opinions about those conventions. Here I am not so sure that >> they belong in the docs. >> >> Agreed. >> >> Barry >> >> --Guido (mobile) >> >> On Jul 2, 2016 10:32 AM, "Barry Scott" wrote: >>> >>> I have read the python3.5 docs for super() and >>> https://rhettinger.wordpress.com/2011/05/26/super-considered-super/. >>> >>> Between the two sources I still have no clear idea of what super() will >>> do and why it will doe what it does. >>> >>> I hope to get feedback that can be used as the basis to update the python >>> docs. >>> >>> For single inheritance the use of super() does not seem to have any >>> surprises. >>> >>> But in the case of a class with multiple-inheritance more then 1 function >>> may be called by 1 call to super(). >>> >>> What are the rules for which functions are called by super() in the >>> multiple >>> inheritance case? >>> >>> Is __init__ special or can other super() calls end up >>> calling more then 1 function? >>> >>> What is the role of **kwds with super()? >>> >>> Here is the code I used to show the __init__ multiple calls. >>> >>> The attached example shows that one call to super() causes 2 to >>> __init__ of bases of Person. But describe does not follow the pattern >>> Age.describe is not called. >>> >>> Barry >>> >>> _______________________________________________ >>> Python-ideas mailing list >>> Python-ideas at python.org >>> https://mail.python.org/mailman/listinfo/python-ideas >>> Code of Conduct: http://python.org/psf/codeofconduct/ >> >> > > > > -- > --Guido van Rossum (python.org/~guido) > From rosuav at gmail.com Mon Jul 4 19:06:05 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 5 Jul 2016 09:06:05 +1000 Subject: [Python-ideas] documentation for python3 super() In-Reply-To: <5978F8C1-AB17-4212-8448-03E13D43655A@barrys-emacs.org> References: <0973EF69-1F00-413A-8781-F9C83119B04D@barrys-emacs.org> <5978F8C1-AB17-4212-8448-03E13D43655A@barrys-emacs.org> Message-ID: On Tue, Jul 5, 2016 at 12:16 AM, Barry Scott wrote: > In engineer English something like this. > > "super() uses the __mro__ as the order to search for the attribute to return. > super() starts by finding the type in the __mro__ and then examines the following > items until the attribute is found and returns it. If the attribute is not found > AttributeError is raised.? > > Thoughts? Nitpick: Instead of saying "the __mro__", I'd say something like "the Method Resolution Order (found in cls.__mro__)" first time and "the MRO" second. The MRO is a feature of the class; it happens to be made available in the attribute __mro__, but that's secondary. ChrisA From njs at pobox.com Tue Jul 5 10:56:11 2016 From: njs at pobox.com (Nathaniel Smith) Date: Tue, 5 Jul 2016 07:56:11 -0700 Subject: [Python-ideas] __enter__ and __exit__ on generators In-Reply-To: References: Message-ID: Hi all, This might be too trivial for python-ideas even, but... I recently stumbled slightly over the way that generators define a close method (the one that injects a GeneratorExit), but don't directly implement the context manager protocol. So if you want a generator's internal resources to be cleaned up promptly, you can't write with genfunc() as it: for obj in it: ... but instead have to write with contextlib.closing(genfunc()) as it: for obj in it: ... Is there any particular reason for this, or is it just an oversight? (Why is it important to close generators? Consider a generator that holds an open file descriptor: def genfunc(): with open(...) as f: for line in f: yield ... Python these days very much encourages you to use a "with" statement for the file open, with ResourceWarnings and all that. But in this case, the file won't be closed until someone calls generator.close(), so the with statement above is only really effective if genfunc's caller also uses a "with" statement. Unfortunately, having the "with" statement inside genfunc *is* enough to unconditionally hide the ResourceWarning, even if it's otherwise ineffective, because the warning is triggered when file.__del__ calls file.__close__, and here we have generator.__del__ calling file.__close__.) -n -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Tue Jul 5 11:17:49 2016 From: guido at python.org (Guido van Rossum) Date: Tue, 5 Jul 2016 08:17:49 -0700 Subject: [Python-ideas] __enter__ and __exit__ on generators In-Reply-To: References: Message-ID: Hm, it's probably just an oversight. Context managers and closable generators were developed around the same time (PEP 342 and 343!) for Python 2.5; it probably took a while before either of them became part of the popular culture, so to speak. On Tue, Jul 5, 2016 at 7:56 AM, Nathaniel Smith wrote: > Hi all, > > This might be too trivial for python-ideas even, but... I recently stumbled > slightly over the way that generators define a close method (the one that > injects a GeneratorExit), but don't directly implement the context manager > protocol. So if you want a generator's internal resources to be cleaned up > promptly, you can't write > > with genfunc() as it: > for obj in it: > ... > > but instead have to write > > with contextlib.closing(genfunc()) as it: > for obj in it: > ... > > Is there any particular reason for this, or is it just an oversight? > > (Why is it important to close generators? Consider a generator that holds an > open file descriptor: > > def genfunc(): > with open(...) as f: > for line in f: > yield ... > > Python these days very much encourages you to use a "with" statement for the > file open, with ResourceWarnings and all that. But in this case, the file > won't be closed until someone calls generator.close(), so the with statement > above is only really effective if genfunc's caller also uses a "with" > statement. Unfortunately, having the "with" statement inside genfunc *is* > enough to unconditionally hide the ResourceWarning, even if it's otherwise > ineffective, because the warning is triggered when file.__del__ calls > file.__close__, and here we have generator.__del__ calling file.__close__.) > > -n > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -- --Guido van Rossum (python.org/~guido) From mistersheik at gmail.com Tue Jul 5 15:28:32 2016 From: mistersheik at gmail.com (Neil Girdhar) Date: Tue, 5 Jul 2016 12:28:32 -0700 (PDT) Subject: [Python-ideas] Unpacking a dict In-Reply-To: <3a4c2c02-2546-e115-4876-3c2e3226da20@trueblade.com> References: <5745CE0D.8030300@stoneleaf.us> <5746062A.9010901@trueblade.com> <574607AD.3040207@stoneleaf.us> <3a4c2c02-2546-e115-4876-3c2e3226da20@trueblade.com> Message-ID: <37c806ae-7952-4bc4-a7ca-b0de5a416b69@googlegroups.com> I actually don't think it will be very hard. On Wednesday, May 25, 2016 at 5:02:52 PM UTC-4, Eric V. Smith wrote: > > On 5/25/2016 4:14 PM, Ethan Furman wrote: > > On 05/25/2016 01:08 PM, Eric V. Smith wrote: > > >> x, y = extract(mapping, 'a', 'b') > >> print(x, y) > >> 1, 2 > > > > Let's pretend you wrote: > > > > a, b = extract(mapping, 'a', 'b') > > > > since that's the way I would almost always be using it. > > > > The proposal is this: > > > > a, b = **mapping > > > > The advantages: > > > > - much more readable > > - less duplication > > > > Less duplication might not seem like that big a deal, but it's one of > > the motivators behind decorations and in-place operators, which are both > > wins. > > I agree that would be a win. That's a lot of compiler magic, though. > > Eric. > > > _______________________________________________ > Python-ideas mailing list > Python... at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mistersheik at gmail.com Tue Jul 5 15:27:54 2016 From: mistersheik at gmail.com (Neil Girdhar) Date: Tue, 5 Jul 2016 12:27:54 -0700 (PDT) Subject: [Python-ideas] Unpacking a dict In-Reply-To: References: Message-ID: <6913e4eb-db61-40ec-a102-bb6c976d285c@googlegroups.com> When I worked on the iterable unpacking (PEP 448), I noticed that it would be *pretty easy* to add dict unpacking like what you're asking for to the grammar, parsing, and source code generation. I will leave the practical language development questions to other people, but from a CPython extension standpoint, I don't think this is a lot of work. Best, Neil On Wednesday, May 25, 2016 at 9:12:30 AM UTC-4, Michael Selik wrote: > > Python's iterable unpacking is what Lispers might call a destructuring > bind. > > py> iterable = 1, 2, 3, 4, 5 > py> a, b, *rest = iterable > py> a, b, rest > (1, 2, (3, 4, 5)) > > Clojure also supports mapping destructuring. Let's add that to Python! > > py> mapping = {"a": 1, "b": 2, "c": 3} > py> {"a": x, "b": y, "c": z} = mapping > py> x, y, z > (1, 2, 3) > py> {"a": x, "b": y} = mapping > Traceback: > ValueError: too many keys to unpack > > > This will be approximately as helpful as iterable unpacking was before PEP > 3132 (https://www.python.org/dev/peps/pep-3132/). > > I hope to keep discussion in this thread focused on the most basic form of > dict unpacking, but we could extended mapping unpacking similarly to how > PEP 3132 extended iterable unpacking. Just brainstorming... > > py> mapping = {"a": 1, "b": 2, "c": 3} > py> {"a": x, **rest} = mapping > py> x, rest > (1, {"b": 2, "c": 3}) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mistersheik at gmail.com Tue Jul 5 15:18:00 2016 From: mistersheik at gmail.com (Neil Girdhar) Date: Tue, 5 Jul 2016 12:18:00 -0700 (PDT) Subject: [Python-ideas] should `dict` supply a default `__missing__` implementation? In-Reply-To: References: <57741C88.7030906@stoneleaf.us> <577421F2.7060003@stoneleaf.us> Message-ID: <182153c5-acb1-4881-8532-d49c6d731312@googlegroups.com> But neither UserDict nor MutableMapping defines __missing__ ? What is a subclasser supposed to do? On Wednesday, June 29, 2016 at 4:59:03 PM UTC-4, Guido van Rossum wrote: > > UserDict is superseded by MutableMapping. > > I don't think we should make dict a kitchen sink class. I also don't > think we should particularly encourage subclassing it. So -1 on adding > dict.__missing__. > > On Wed, Jun 29, 2016 at 12:30 PM, Ethan Furman > wrote: > > On 06/29/2016 12:09 PM, Guido van Rossum wrote: > > > >> So providing the comprehensive base class is up to the user, not up to > >> the stdlib. Is that such a big deal? > > > > > > No, it's not. But it makes for a better user experience if the base > class > > has the __missing__ method that raises a KeyError already. > > > > Didn't we add a UserDict that could be subclassed primarily because > > subclassing dict directly was such a poor user experience? > > > > If adding __missing__ to dict is huge (performance hit?), we don't do > it. > > If it's not, I think we should. Maybe add it to UserDict if performance > is > > a concern? > > > > > > -- > > ~Ethan~ > > _______________________________________________ > > Python-ideas mailing list > > Python... at python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ > > > > -- > --Guido van Rossum (python.org/~guido) > _______________________________________________ > Python-ideas mailing list > Python... at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Tue Jul 5 18:03:53 2016 From: guido at python.org (Guido van Rossum) Date: Tue, 5 Jul 2016 15:03:53 -0700 Subject: [Python-ideas] should `dict` supply a default `__missing__` implementation? In-Reply-To: <182153c5-acb1-4881-8532-d49c6d731312@googlegroups.com> References: <57741C88.7030906@stoneleaf.us> <577421F2.7060003@stoneleaf.us> <182153c5-acb1-4881-8532-d49c6d731312@googlegroups.com> Message-ID: What kind of question is that? If you subclass MutableMapping the whole feature doesn't exist (since you're not subclassing dict). It *only* exists for subclasses of dict. On Tue, Jul 5, 2016 at 12:18 PM, Neil Girdhar wrote: > But neither UserDict nor MutableMapping defines __missing__ ? What is a > subclasser supposed to do? > > On Wednesday, June 29, 2016 at 4:59:03 PM UTC-4, Guido van Rossum wrote: >> >> UserDict is superseded by MutableMapping. >> >> I don't think we should make dict a kitchen sink class. I also don't >> think we should particularly encourage subclassing it. So -1 on adding >> dict.__missing__. >> >> On Wed, Jun 29, 2016 at 12:30 PM, Ethan Furman wrote: >> > On 06/29/2016 12:09 PM, Guido van Rossum wrote: >> > >> >> So providing the comprehensive base class is up to the user, not up to >> >> the stdlib. Is that such a big deal? >> > >> > >> > No, it's not. But it makes for a better user experience if the base >> > class >> > has the __missing__ method that raises a KeyError already. >> > >> > Didn't we add a UserDict that could be subclassed primarily because >> > subclassing dict directly was such a poor user experience? >> > >> > If adding __missing__ to dict is huge (performance hit?), we don't do >> > it. >> > If it's not, I think we should. Maybe add it to UserDict if performance >> > is >> > a concern? >> > >> > >> > -- >> > ~Ethan~ >> > _______________________________________________ >> > Python-ideas mailing list >> > Python... at python.org >> > https://mail.python.org/mailman/listinfo/python-ideas >> > Code of Conduct: http://python.org/psf/codeofconduct/ >> >> >> >> -- >> --Guido van Rossum (python.org/~guido) >> _______________________________________________ >> Python-ideas mailing list >> Python... at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ -- --Guido van Rossum (python.org/~guido) From mistersheik at gmail.com Tue Jul 5 18:07:15 2016 From: mistersheik at gmail.com (Neil Girdhar) Date: Tue, 05 Jul 2016 22:07:15 +0000 Subject: [Python-ideas] should `dict` supply a default `__missing__` implementation? In-Reply-To: References: <57741C88.7030906@stoneleaf.us> <577421F2.7060003@stoneleaf.us> <182153c5-acb1-4881-8532-d49c6d731312@googlegroups.com> Message-ID: Okay, that makes sense, but why isn't __missing__ in UserDict? On Tue, Jul 5, 2016 at 6:04 PM Guido van Rossum wrote: > What kind of question is that? If you subclass MutableMapping the > whole feature doesn't exist (since you're not subclassing dict). It > *only* exists for subclasses of dict. > > On Tue, Jul 5, 2016 at 12:18 PM, Neil Girdhar > wrote: > > But neither UserDict nor MutableMapping defines __missing__ ? What is a > > subclasser supposed to do? > > > > On Wednesday, June 29, 2016 at 4:59:03 PM UTC-4, Guido van Rossum wrote: > >> > >> UserDict is superseded by MutableMapping. > >> > >> I don't think we should make dict a kitchen sink class. I also don't > >> think we should particularly encourage subclassing it. So -1 on adding > >> dict.__missing__. > >> > >> On Wed, Jun 29, 2016 at 12:30 PM, Ethan Furman > wrote: > >> > On 06/29/2016 12:09 PM, Guido van Rossum wrote: > >> > > >> >> So providing the comprehensive base class is up to the user, not up > to > >> >> the stdlib. Is that such a big deal? > >> > > >> > > >> > No, it's not. But it makes for a better user experience if the base > >> > class > >> > has the __missing__ method that raises a KeyError already. > >> > > >> > Didn't we add a UserDict that could be subclassed primarily because > >> > subclassing dict directly was such a poor user experience? > >> > > >> > If adding __missing__ to dict is huge (performance hit?), we don't do > >> > it. > >> > If it's not, I think we should. Maybe add it to UserDict if > performance > >> > is > >> > a concern? > >> > > >> > > >> > -- > >> > ~Ethan~ > >> > _______________________________________________ > >> > Python-ideas mailing list > >> > Python... at python.org > >> > https://mail.python.org/mailman/listinfo/python-ideas > >> > Code of Conduct: http://python.org/psf/codeofconduct/ > >> > >> > >> > >> -- > >> --Guido van Rossum (python.org/~guido) > >> _______________________________________________ > >> Python-ideas mailing list > >> Python... at python.org > >> https://mail.python.org/mailman/listinfo/python-ideas > >> Code of Conduct: http://python.org/psf/codeofconduct/ > > > > -- > --Guido van Rossum (python.org/~guido) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Tue Jul 5 18:10:47 2016 From: guido at python.org (Guido van Rossum) Date: Tue, 5 Jul 2016 15:10:47 -0700 Subject: [Python-ideas] should `dict` supply a default `__missing__` implementation? In-Reply-To: References: <57741C88.7030906@stoneleaf.us> <577421F2.7060003@stoneleaf.us> <182153c5-acb1-4881-8532-d49c6d731312@googlegroups.com> Message-ID: Because you shouldn't be using UserDict. On Tue, Jul 5, 2016 at 3:07 PM, Neil Girdhar wrote: > Okay, that makes sense, but why isn't __missing__ in UserDict? > > On Tue, Jul 5, 2016 at 6:04 PM Guido van Rossum wrote: >> >> What kind of question is that? If you subclass MutableMapping the >> whole feature doesn't exist (since you're not subclassing dict). It >> *only* exists for subclasses of dict. >> >> On Tue, Jul 5, 2016 at 12:18 PM, Neil Girdhar >> wrote: >> > But neither UserDict nor MutableMapping defines __missing__ ? What is a >> > subclasser supposed to do? >> > >> > On Wednesday, June 29, 2016 at 4:59:03 PM UTC-4, Guido van Rossum wrote: >> >> >> >> UserDict is superseded by MutableMapping. >> >> >> >> I don't think we should make dict a kitchen sink class. I also don't >> >> think we should particularly encourage subclassing it. So -1 on adding >> >> dict.__missing__. >> >> >> >> On Wed, Jun 29, 2016 at 12:30 PM, Ethan Furman >> >> wrote: >> >> > On 06/29/2016 12:09 PM, Guido van Rossum wrote: >> >> > >> >> >> So providing the comprehensive base class is up to the user, not up >> >> >> to >> >> >> the stdlib. Is that such a big deal? >> >> > >> >> > >> >> > No, it's not. But it makes for a better user experience if the base >> >> > class >> >> > has the __missing__ method that raises a KeyError already. >> >> > >> >> > Didn't we add a UserDict that could be subclassed primarily because >> >> > subclassing dict directly was such a poor user experience? >> >> > >> >> > If adding __missing__ to dict is huge (performance hit?), we don't do >> >> > it. >> >> > If it's not, I think we should. Maybe add it to UserDict if >> >> > performance >> >> > is >> >> > a concern? >> >> > >> >> > >> >> > -- >> >> > ~Ethan~ >> >> > _______________________________________________ >> >> > Python-ideas mailing list >> >> > Python... at python.org >> >> > https://mail.python.org/mailman/listinfo/python-ideas >> >> > Code of Conduct: http://python.org/psf/codeofconduct/ >> >> >> >> >> >> >> >> -- >> >> --Guido van Rossum (python.org/~guido) >> >> _______________________________________________ >> >> Python-ideas mailing list >> >> Python... at python.org >> >> https://mail.python.org/mailman/listinfo/python-ideas >> >> Code of Conduct: http://python.org/psf/codeofconduct/ >> >> >> >> -- >> --Guido van Rossum (python.org/~guido) -- --Guido van Rossum (python.org/~guido) From mistersheik at gmail.com Tue Jul 5 18:13:59 2016 From: mistersheik at gmail.com (Neil Girdhar) Date: Tue, 05 Jul 2016 22:13:59 +0000 Subject: [Python-ideas] should `dict` supply a default `__missing__` implementation? In-Reply-To: References: <57741C88.7030906@stoneleaf.us> <577421F2.7060003@stoneleaf.us> <182153c5-acb1-4881-8532-d49c6d731312@googlegroups.com> Message-ID: Is it deprecated? I've seen this question a lot on stackoverflow: http://stackoverflow.com/questions/7148419/subclass-dict-userdict-dict-or-abc http://stackoverflow.com/questions/2390827/how-to-properly-subclass-dict-and-override-getitem-setitem http://stackoverflow.com/questions/10901048/i-want-to-subclass-dict-and-set-default-values I still have no idea what the right answer is. On Tue, Jul 5, 2016 at 6:11 PM Guido van Rossum wrote: > Because you shouldn't be using UserDict. > > On Tue, Jul 5, 2016 at 3:07 PM, Neil Girdhar > wrote: > > Okay, that makes sense, but why isn't __missing__ in UserDict? > > > > On Tue, Jul 5, 2016 at 6:04 PM Guido van Rossum > wrote: > >> > >> What kind of question is that? If you subclass MutableMapping the > >> whole feature doesn't exist (since you're not subclassing dict). It > >> *only* exists for subclasses of dict. > >> > >> On Tue, Jul 5, 2016 at 12:18 PM, Neil Girdhar > >> wrote: > >> > But neither UserDict nor MutableMapping defines __missing__ ? What > is a > >> > subclasser supposed to do? > >> > > >> > On Wednesday, June 29, 2016 at 4:59:03 PM UTC-4, Guido van Rossum > wrote: > >> >> > >> >> UserDict is superseded by MutableMapping. > >> >> > >> >> I don't think we should make dict a kitchen sink class. I also don't > >> >> think we should particularly encourage subclassing it. So -1 on > adding > >> >> dict.__missing__. > >> >> > >> >> On Wed, Jun 29, 2016 at 12:30 PM, Ethan Furman > >> >> wrote: > >> >> > On 06/29/2016 12:09 PM, Guido van Rossum wrote: > >> >> > > >> >> >> So providing the comprehensive base class is up to the user, not > up > >> >> >> to > >> >> >> the stdlib. Is that such a big deal? > >> >> > > >> >> > > >> >> > No, it's not. But it makes for a better user experience if the > base > >> >> > class > >> >> > has the __missing__ method that raises a KeyError already. > >> >> > > >> >> > Didn't we add a UserDict that could be subclassed primarily because > >> >> > subclassing dict directly was such a poor user experience? > >> >> > > >> >> > If adding __missing__ to dict is huge (performance hit?), we don't > do > >> >> > it. > >> >> > If it's not, I think we should. Maybe add it to UserDict if > >> >> > performance > >> >> > is > >> >> > a concern? > >> >> > > >> >> > > >> >> > -- > >> >> > ~Ethan~ > >> >> > _______________________________________________ > >> >> > Python-ideas mailing list > >> >> > Python... at python.org > >> >> > https://mail.python.org/mailman/listinfo/python-ideas > >> >> > Code of Conduct: http://python.org/psf/codeofconduct/ > >> >> > >> >> > >> >> > >> >> -- > >> >> --Guido van Rossum (python.org/~guido) > >> >> _______________________________________________ > >> >> Python-ideas mailing list > >> >> Python... at python.org > >> >> https://mail.python.org/mailman/listinfo/python-ideas > >> >> Code of Conduct: http://python.org/psf/codeofconduct/ > >> > >> > >> > >> -- > >> --Guido van Rossum (python.org/~guido) > > > > -- > --Guido van Rossum (python.org/~guido) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.selik at gmail.com Tue Jul 5 18:18:33 2016 From: michael.selik at gmail.com (Michael Selik) Date: Tue, 05 Jul 2016 22:18:33 +0000 Subject: [Python-ideas] should `dict` supply a default `__missing__` implementation? In-Reply-To: References: <57741C88.7030906@stoneleaf.us> <577421F2.7060003@stoneleaf.us> <182153c5-acb1-4881-8532-d49c6d731312@googlegroups.com> Message-ID: On Tue, Jul 5, 2016 at 6:15 PM Neil Girdhar wrote: > Is it deprecated? > > I've seen this question a lot on stackoverflow: > http://stackoverflow.com/questions/7148419/subclass-dict-userdict-dict-or-abc > > http://stackoverflow.com/questions/2390827/how-to-properly-subclass-dict-and-override-getitem-setitem > http://stackoverflow.com/questions/10901048/i-want-to-subclass-dict-and-set-default-values > > I still have no idea what the right answer is. > > On Tue, Jul 5, 2016 at 6:11 PM Guido van Rossum wrote: > >> Because you shouldn't be using UserDict. >> > Subclass dict when you only want to change exactly the methods you override, and/or implement __missing__. Subclass MutableMapping when you want to take advantage of the mixin methods interacting with your implementations of the abstract methods. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Tue Jul 5 18:19:19 2016 From: guido at python.org (Guido van Rossum) Date: Tue, 5 Jul 2016 15:19:19 -0700 Subject: [Python-ideas] should `dict` supply a default `__missing__` implementation? In-Reply-To: References: <57741C88.7030906@stoneleaf.us> <577421F2.7060003@stoneleaf.us> <182153c5-acb1-4881-8532-d49c6d731312@googlegroups.com> Message-ID: I think this is a question for Raymond Hettinger. On Tue, Jul 5, 2016 at 3:13 PM, Neil Girdhar wrote: > Is it deprecated? > > I've seen this question a lot on stackoverflow: > http://stackoverflow.com/questions/7148419/subclass-dict-userdict-dict-or-abc > http://stackoverflow.com/questions/2390827/how-to-properly-subclass-dict-and-override-getitem-setitem > http://stackoverflow.com/questions/10901048/i-want-to-subclass-dict-and-set-default-values > I still have no idea what the right answer is. > > On Tue, Jul 5, 2016 at 6:11 PM Guido van Rossum wrote: >> >> Because you shouldn't be using UserDict. >> >> On Tue, Jul 5, 2016 at 3:07 PM, Neil Girdhar >> wrote: >> > Okay, that makes sense, but why isn't __missing__ in UserDict? >> > >> > On Tue, Jul 5, 2016 at 6:04 PM Guido van Rossum >> > wrote: >> >> >> >> What kind of question is that? If you subclass MutableMapping the >> >> whole feature doesn't exist (since you're not subclassing dict). It >> >> *only* exists for subclasses of dict. >> >> >> >> On Tue, Jul 5, 2016 at 12:18 PM, Neil Girdhar >> >> wrote: >> >> > But neither UserDict nor MutableMapping defines __missing__ ? What >> >> > is a >> >> > subclasser supposed to do? >> >> > >> >> > On Wednesday, June 29, 2016 at 4:59:03 PM UTC-4, Guido van Rossum >> >> > wrote: >> >> >> >> >> >> UserDict is superseded by MutableMapping. >> >> >> >> >> >> I don't think we should make dict a kitchen sink class. I also don't >> >> >> think we should particularly encourage subclassing it. So -1 on >> >> >> adding >> >> >> dict.__missing__. >> >> >> >> >> >> On Wed, Jun 29, 2016 at 12:30 PM, Ethan Furman >> >> >> wrote: >> >> >> > On 06/29/2016 12:09 PM, Guido van Rossum wrote: >> >> >> > >> >> >> >> So providing the comprehensive base class is up to the user, not >> >> >> >> up >> >> >> >> to >> >> >> >> the stdlib. Is that such a big deal? >> >> >> > >> >> >> > >> >> >> > No, it's not. But it makes for a better user experience if the >> >> >> > base >> >> >> > class >> >> >> > has the __missing__ method that raises a KeyError already. >> >> >> > >> >> >> > Didn't we add a UserDict that could be subclassed primarily >> >> >> > because >> >> >> > subclassing dict directly was such a poor user experience? >> >> >> > >> >> >> > If adding __missing__ to dict is huge (performance hit?), we don't >> >> >> > do >> >> >> > it. >> >> >> > If it's not, I think we should. Maybe add it to UserDict if >> >> >> > performance >> >> >> > is >> >> >> > a concern? >> >> >> > >> >> >> > >> >> >> > -- >> >> >> > ~Ethan~ >> >> >> > _______________________________________________ >> >> >> > Python-ideas mailing list >> >> >> > Python... at python.org >> >> >> > https://mail.python.org/mailman/listinfo/python-ideas >> >> >> > Code of Conduct: http://python.org/psf/codeofconduct/ >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> --Guido van Rossum (python.org/~guido) >> >> >> _______________________________________________ >> >> >> Python-ideas mailing list >> >> >> Python... at python.org >> >> >> https://mail.python.org/mailman/listinfo/python-ideas >> >> >> Code of Conduct: http://python.org/psf/codeofconduct/ >> >> >> >> >> >> >> >> -- >> >> --Guido van Rossum (python.org/~guido) >> >> >> >> -- >> --Guido van Rossum (python.org/~guido) -- --Guido van Rossum (python.org/~guido) From ncoghlan at gmail.com Tue Jul 5 22:31:39 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 6 Jul 2016 12:31:39 +1000 Subject: [Python-ideas] __enter__ and __exit__ on generators In-Reply-To: References: Message-ID: On 6 July 2016 at 01:17, Guido van Rossum wrote: > Hm, it's probably just an oversight. Context managers and closable > generators were developed around the same time (PEP 342 and 343!) for > Python 2.5; it probably took a while before either of them became part > of the popular culture, so to speak. No, it's deliberate - if generators natively implemented the context management protocol, accidentally leaving out "@contextmanager" would silently call the underlying generator's close() method without ever executing the generator body instead of giving you an immediate exception when you tried to use it in a with statement. When you get the exception, if you actually *do* want to just close the generator, you can wrap it in contextlib.closing, while if you meant to write a custom context manager, you can add the missing decorator. I thought we'd actually added an entry for the design FAQ about that, but it turns out it got stuck on the issue tracker: https://bugs.python.org/issue13814 It does occur to me that it might be worth customising the error reported, though - it's currently genuinely non-obvious why they don't implement the protocol, but it might be clearer if the error was "TypeError: generator requires 'contextlib.contextmanager' or 'contextlib.closing' to support context management" Regards, Nick. > > On Tue, Jul 5, 2016 at 7:56 AM, Nathaniel Smith wrote: >> Hi all, >> >> This might be too trivial for python-ideas even, but... I recently stumbled >> slightly over the way that generators define a close method (the one that >> injects a GeneratorExit), but don't directly implement the context manager >> protocol. So if you want a generator's internal resources to be cleaned up >> promptly, you can't write >> >> with genfunc() as it: >> for obj in it: >> ... >> >> but instead have to write >> >> with contextlib.closing(genfunc()) as it: >> for obj in it: >> ... >> >> Is there any particular reason for this, or is it just an oversight? >> >> (Why is it important to close generators? Consider a generator that holds an >> open file descriptor: >> >> def genfunc(): >> with open(...) as f: >> for line in f: >> yield ... >> >> Python these days very much encourages you to use a "with" statement for the >> file open, with ResourceWarnings and all that. But in this case, the file >> won't be closed until someone calls generator.close(), so the with statement >> above is only really effective if genfunc's caller also uses a "with" >> statement. Unfortunately, having the "with" statement inside genfunc *is* >> enough to unconditionally hide the ResourceWarning, even if it's otherwise >> ineffective, because the warning is triggered when file.__del__ calls >> file.__close__, and here we have generator.__del__ calling file.__close__.) >> >> -n >> >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ > > > > -- > --Guido van Rossum (python.org/~guido) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From michael.selik at gmail.com Tue Jul 5 22:40:04 2016 From: michael.selik at gmail.com (Michael Selik) Date: Wed, 06 Jul 2016 02:40:04 +0000 Subject: [Python-ideas] __enter__ and __exit__ on generators In-Reply-To: References: Message-ID: On Tue, Jul 5, 2016 at 10:31 PM Nick Coghlan wrote: > It does occur to me that it might be worth customising the error > reported, though - it's currently genuinely non-obvious why they don't > implement the protocol, but it might be clearer if the error was > "TypeError: generator requires 'contextlib.contextmanager' or > 'contextlib.closing' to support context management" > +1 for error messages that explain how to solve the problem -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt at getpattern.com Wed Jul 6 01:11:46 2016 From: matt at getpattern.com (Matt Gilson) Date: Tue, 5 Jul 2016 22:11:46 -0700 Subject: [Python-ideas] should `dict` supply a default `__missing__` implementation? In-Reply-To: References: <57741C88.7030906@stoneleaf.us> <577421F2.7060003@stoneleaf.us> <182153c5-acb1-4881-8532-d49c6d731312@googlegroups.com> Message-ID: This discussion has my brain all twisted up... I'm going to try to work my way through it to make some sense of what we're saying here. 1. It is pretty easy to provide a subclass with a base `__missing__` implementation, so why should python be responsible for providing that behavior? 2. We don't want to encourage subclassing `dict`, so let's not make working with `__missing__` slightly nicer (see 1.) 3. Would adding `__missing__` to `dict` actually make working with it slightly nicer anyway? 4. Other places in the standard library for creating mappings easily (`UserDict` and `collections.Mapping`) do not support `__missing__``, but it is easy to build that functionality yourself (see 1. and example code below) Let me know if I left anything out or misrepresented any of the comments ... It definitely wasn't my intent. As I see it, there are three paths forward. - Do nothing. The status quo probably only surprises one or two users a year, so lets not rock the boat. Additionally, we can't add _everything_ everywhere. Things just become a mess if you say "Yes" to every idea and you have to draw the line somewhere. - Add `__missing__` to `dict`. It might be a simple pass-through method for raising `KeyError`, but hey, maybe it'll simplify a code-path or two. - Add `__missing__` support to `collections.abc.Mapping` (or I guess UserDict, but I'd be just as happy if that one went away completely :-). I'm sure I haven't thought through all the pitfalls, but (additional documentation aside) it seems like it could be as easy as: class Mapping(Sized, Iterable, Container): def __missing__(self, key): raise KeyError @abc.abstractmethod def __getitem__(self, key): self.__missing__(key) ... On Tue, Jul 5, 2016 at 3:19 PM, Guido van Rossum wrote: > I think this is a question for Raymond Hettinger. > > On Tue, Jul 5, 2016 at 3:13 PM, Neil Girdhar > wrote: > > Is it deprecated? > > > > I've seen this question a lot on stackoverflow: > > > http://stackoverflow.com/questions/7148419/subclass-dict-userdict-dict-or-abc > > > http://stackoverflow.com/questions/2390827/how-to-properly-subclass-dict-and-override-getitem-setitem > > > http://stackoverflow.com/questions/10901048/i-want-to-subclass-dict-and-set-default-values > > I still have no idea what the right answer is. > > > > On Tue, Jul 5, 2016 at 6:11 PM Guido van Rossum > wrote: > >> > >> Because you shouldn't be using UserDict. > >> > >> On Tue, Jul 5, 2016 at 3:07 PM, Neil Girdhar > >> wrote: > >> > Okay, that makes sense, but why isn't __missing__ in UserDict? > >> > > >> > On Tue, Jul 5, 2016 at 6:04 PM Guido van Rossum > >> > wrote: > >> >> > >> >> What kind of question is that? If you subclass MutableMapping the > >> >> whole feature doesn't exist (since you're not subclassing dict). It > >> >> *only* exists for subclasses of dict. > >> >> > >> >> On Tue, Jul 5, 2016 at 12:18 PM, Neil Girdhar > > >> >> wrote: > >> >> > But neither UserDict nor MutableMapping defines __missing__ ? What > >> >> > is a > >> >> > subclasser supposed to do? > >> >> > > >> >> > On Wednesday, June 29, 2016 at 4:59:03 PM UTC-4, Guido van Rossum > >> >> > wrote: > >> >> >> > >> >> >> UserDict is superseded by MutableMapping. > >> >> >> > >> >> >> I don't think we should make dict a kitchen sink class. I also > don't > >> >> >> think we should particularly encourage subclassing it. So -1 on > >> >> >> adding > >> >> >> dict.__missing__. > >> >> >> > >> >> >> On Wed, Jun 29, 2016 at 12:30 PM, Ethan Furman < > et... at stoneleaf.us> > >> >> >> wrote: > >> >> >> > On 06/29/2016 12:09 PM, Guido van Rossum wrote: > >> >> >> > > >> >> >> >> So providing the comprehensive base class is up to the user, > not > >> >> >> >> up > >> >> >> >> to > >> >> >> >> the stdlib. Is that such a big deal? > >> >> >> > > >> >> >> > > >> >> >> > No, it's not. But it makes for a better user experience if the > >> >> >> > base > >> >> >> > class > >> >> >> > has the __missing__ method that raises a KeyError already. > >> >> >> > > >> >> >> > Didn't we add a UserDict that could be subclassed primarily > >> >> >> > because > >> >> >> > subclassing dict directly was such a poor user experience? > >> >> >> > > >> >> >> > If adding __missing__ to dict is huge (performance hit?), we > don't > >> >> >> > do > >> >> >> > it. > >> >> >> > If it's not, I think we should. Maybe add it to UserDict if > >> >> >> > performance > >> >> >> > is > >> >> >> > a concern? > >> >> >> > > >> >> >> > > >> >> >> > -- > >> >> >> > ~Ethan~ > >> >> >> > _______________________________________________ > >> >> >> > Python-ideas mailing list > >> >> >> > Python... at python.org > >> >> >> > https://mail.python.org/mailman/listinfo/python-ideas > >> >> >> > Code of Conduct: http://python.org/psf/codeofconduct/ > >> >> >> > >> >> >> > >> >> >> > >> >> >> -- > >> >> >> --Guido van Rossum (python.org/~guido) > >> >> >> _______________________________________________ > >> >> >> Python-ideas mailing list > >> >> >> Python... at python.org > >> >> >> https://mail.python.org/mailman/listinfo/python-ideas > >> >> >> Code of Conduct: http://python.org/psf/codeofconduct/ > >> >> > >> >> > >> >> > >> >> -- > >> >> --Guido van Rossum (python.org/~guido) > >> > >> > >> > >> -- > >> --Guido van Rossum (python.org/~guido) > > > > -- > --Guido van Rossum (python.org/~guido) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- [image: pattern-sig.png] Matt Gilson // SOFTWARE ENGINEER E: matt at getpattern.com // P: 603.892.7736 We?re looking for beta testers. Go here to sign up! -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Wed Jul 6 01:51:15 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 05 Jul 2016 22:51:15 -0700 Subject: [Python-ideas] should `dict` supply a default `__missing__` implementation? In-Reply-To: References: <57741C88.7030906@stoneleaf.us> <577421F2.7060003@stoneleaf.us> <182153c5-acb1-4881-8532-d49c6d731312@googlegroups.com> Message-ID: <577C9C53.2070003@stoneleaf.us> On 07/05/2016 10:11 PM, Matt Gilson wrote: > This discussion has my brain all twisted up... I'm going to try to work > my way through it to make some sense of what we're saying here. > > 1. It is pretty easy to provide a subclass with a base `__missing__` > implementation, so why should python be responsible for providing > that behavior? > 2. We don't want to encourage subclassing `dict`, so let's not make > working with `__missing__` slightly nicer (see 1.) > 3. Would adding `__missing__` to `dict` actually make working with it > slightly nicer anyway? > 4. Other places in the standard library for creating mappings easily > (`UserDict` and `collections.Mapping`) do not support > `__missing__``, but it is easy to build that functionality yourself > (see 1. and example code below) > > Let me know if I left anything out or misrepresented any of the comments > ... It definitely wasn't my intent. > As I see it, there are three paths forward. > > * Do nothing. The status quo probably only surprises one or two users > a year, so lets not rock the boat. Additionally, we can't add > _everything_ everywhere. Things just become a mess if you say "Yes" > to every idea and you have to draw the line somewhere. > * Add `__missing__` to `dict`. It might be a simple pass-through > method for raising `KeyError`, but hey, maybe it'll simplify a > code-path or two. > > * Add `__missing__` support to `collections.abc.Mapping` > (or > I guess UserDict, but I'd be just as happy if that one went away > completely :-). I'm sure I haven't thought through all the > pitfalls, but (additional documentation aside) it seems like it > could be as easy as: > > > > class Mapping(Sized, Iterable, Container): > def __missing__(self, key): > raise KeyError > > @abc.abstractmethod > def __getitem__(self, key): > self.__missing__(key) > > ... I will admit I don't understand the push-back against adding __missing__, and I'm mystified about the comment that users shouldn't be subclassing dict. I thought one of the key advances late in the 2.x series was easy(er) subclassing of the builtins. Unless it's a practicality thing, what with dict being written in such a subclass-hostile way (for performance reasions?): class ADict(dict): def __init__(self): super(ADict, self).__init__() self.names = [] def __delitem__(self, name): if name in self.names: self.names.remove(name) super(ADict, self).__delitem__(name) def __setitem__(self, name, value): if name not in self.names: self.names.append(name) super(ADict, self).__setitem__(name, value) my_dict = ADict() my_dict['howdy'] = 'bye' my_dict.update({'hola': 'adios'}) print(my_dict.names) # ['howdy'] print(my_dict.items()) # dict_items([('howdy','bye'), ('hola','adios')]) my_dict.pop('howdy') del my_dict['hola'] print(my_dict.names) # ['howdy'] print(my_dict.items()) # dict_items([]) -- ~Ethan~ From stephen at xemacs.org Wed Jul 6 06:39:04 2016 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 6 Jul 2016 19:39:04 +0900 Subject: [Python-ideas] should `dict` supply a default `__missing__` implementation? In-Reply-To: <577C9C53.2070003@stoneleaf.us> References: <57741C88.7030906@stoneleaf.us> <577421F2.7060003@stoneleaf.us> <182153c5-acb1-4881-8532-d49c6d731312@googlegroups.com> <577C9C53.2070003@stoneleaf.us> Message-ID: <22396.57288.983795.880996@turnbull.sk.tsukuba.ac.jp> Ethan Furman writes: > __missing__, and I'm mystified about the comment that users shouldn't be > subclassing dict. I thought one of the key advances late in the 2.x > series was easy(er) subclassing of the builtins. But that's not what was said, it was conditional. Sometimes you want to be subclassing dict, sometimes MutableMapping. But UserDict is apparently considered to be obsolete. From p.f.moore at gmail.com Wed Jul 6 07:19:24 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 6 Jul 2016 12:19:24 +0100 Subject: [Python-ideas] should `dict` supply a default `__missing__` implementation? In-Reply-To: <577C9C53.2070003@stoneleaf.us> References: <57741C88.7030906@stoneleaf.us> <577421F2.7060003@stoneleaf.us> <182153c5-acb1-4881-8532-d49c6d731312@googlegroups.com> <577C9C53.2070003@stoneleaf.us> Message-ID: On 6 July 2016 at 06:51, Ethan Furman wrote: > > I will admit I don't understand the push-back against adding __missing__ I thought that the idea was that __missing__ *if present* defined an alternative behaviour for missing keys. If it's not present, the default behaviour (key error) applies. Now it's certainly the case that having the base class provide an implementation of __missing__ that had the default behaviour would do the same - and in a "pure object oriented" sense, that's the way something like this "should" be implemented. But historically, it's not that way, and I suspect that it was much easier to bolt on an "if there's a __missing__ use it, otherwise proceed as before" check than it would have been to pull out the existing behaviour into a new (overridable) method. Python's never been particularly concerned about following pure OOP principles - "practicality beats purity" is relevant here. And I think the problem is that there's no obvious gain to making the default behaviour a concrete __missing__ method (and it would result in code churn in one of the most central and performance-sensitive areas of Python). So the push-back is mostly in the nature of "if it ain't broke, don't fix it" - and I don't think there's been a compelling (i.e., founded in practical issues) argument that anything is broken here. The original issue involved super() - and my understanding is that super() is intended for "co-operative" multiple inheritance, not for arbitrary "pass this method on without me needing to know the class hierarchy" boilerplate. Specifically, in class D(dict): def __missing__(self, k): super(D, self).__missing__(k) it's not clear why D needs a __missing__ method at all, if all it's doing is delegating to dict - we know dict has no __missing__ so D can also simply omit the method. (Possibly if D is being designed as a MI base class, that's not acceptable - but in which case there's not enough information here about the requirement to comment - maybe manually raising KeyError here is enough). Paul From michael.selik at gmail.com Wed Jul 6 14:37:05 2016 From: michael.selik at gmail.com (Michael Selik) Date: Wed, 06 Jul 2016 18:37:05 +0000 Subject: [Python-ideas] bytes indexing behavior In-Reply-To: <57572F17.4010009@stoneleaf.us> References: <20160603131730.GA29255@python.ca> <087001d1be4c$99423520$cbc69f60$@hotmail.com> <20160604082715.GB20628@python.ca> <08e901d1beb8$e60915b0$b21b4110$@hotmail.com> <20160606193530.GA27695@python.ca> <5756EF5A.106@stoneleaf.us> <57572F17.4010009@stoneleaf.us> Message-ID: On Tue, Jun 7, 2016 at 4:30 PM Ethan Furman wrote: > On 06/07/2016 12:57 PM, Nick Coghlan wrote: > > > https://www.python.org/dev/peps/pep-0467/#addition-of-optimised-iterator-methods-that-produce-bytes-objects > > > > If someone wanted to take that PEP and drive it through to resolution, > > I'd be happy to hand it over (my recollection is that the sticking > > point in the previous discussion was the proposed constructor changes, > > so dropping those may make it easier to get the additional method > > accepted). > > Split new thread to resolve that PEP. > What's the status of this PEP? Does anyone dislike the suggestion of adding an iterbytes method? I ran into this annoyance again yesterday. The natural way to ensure that all bytes in a bytes object are a particular value (in this case b'z') is: all(byte == b'z' for byte in bytestring) It reads like natural language. Using ``ord`` would work in Python3, but then my code would not be Python2-compatible. And it'd uselessly repeat the call to ord a bunch of times. all(byte == ord(b'z') for byte in bytestring) Instead it seems the best way given the current behavior is to write: len(bytestring) == bytestring.count(b'z') While I wait for PEP 467, can anyone suggest a better way to write that? -------------- next part -------------- An HTML attachment was scrubbed... URL: From leewangzhong+python at gmail.com Fri Jul 8 01:03:45 2016 From: leewangzhong+python at gmail.com (Franklin? Lee) Date: Fri, 8 Jul 2016 01:03:45 -0400 Subject: [Python-ideas] bytes indexing behavior In-Reply-To: References: <20160603131730.GA29255@python.ca> <087001d1be4c$99423520$cbc69f60$@hotmail.com> <20160604082715.GB20628@python.ca> <08e901d1beb8$e60915b0$b21b4110$@hotmail.com> <20160606193530.GA27695@python.ca> <5756EF5A.106@stoneleaf.us> <57572F17.4010009@stoneleaf.us> Message-ID: On Jul 6, 2016 2:40 PM, "Michael Selik" wrote: > > On Tue, Jun 7, 2016 at 4:30 PM Ethan Furman wrote: >> >> On 06/07/2016 12:57 PM, Nick Coghlan wrote: >> > https://www.python.org/dev/peps/pep-0467/#addition-of-optimised-iterator-methods-that-produce-bytes-objects >> > >> > If someone wanted to take that PEP and drive it through to resolution, >> > I'd be happy to hand it over (my recollection is that the sticking >> > point in the previous discussion was the proposed constructor changes, >> > so dropping those may make it easier to get the additional method >> > accepted). >> >> Split new thread to resolve that PEP. > > > What's the status of this PEP? > Does anyone dislike the suggestion of adding an iterbytes method? > > I ran into this annoyance again yesterday. The natural way to ensure that all bytes in a bytes object are a particular value (in this case b'z') is: > > all(byte == b'z' for byte in bytestring) > > It reads like natural language. Using ``ord`` would work in Python3, but then my code would not be Python2-compatible. And it'd uselessly repeat the call to ord a bunch of times. > > all(byte == ord(b'z') for byte in bytestring) > > Instead it seems the best way given the current behavior is to write: > > len(bytestring) == bytestring.count(b'z') > > While I wait for PEP 467, can anyone suggest a better way to write that? How about set(bytestring) == set(b'z') -------------- next part -------------- An HTML attachment was scrubbed... URL: From animalize81 at hotmail.com Fri Jul 8 02:37:51 2016 From: animalize81 at hotmail.com (Ma Lin) Date: Fri, 8 Jul 2016 06:37:51 +0000 Subject: [Python-ideas] Add an option that allow check Type Hints in runtime Message-ID: Hi, all, Is it possible to add an option that allow check Type Hints in runtime? So that I can enable dynamic type checking when debugging. And disable it in final deploy, no performance lose. regards. From mertz at gnosis.cx Fri Jul 8 02:42:57 2016 From: mertz at gnosis.cx (David Mertz) Date: Thu, 7 Jul 2016 23:42:57 -0700 Subject: [Python-ideas] Add an option that allow check Type Hints in runtime In-Reply-To: References: Message-ID: This exists already. All the annotations love on the '__annotations__' attribute is a function or method. Check that at runtime however you like! On Jul 7, 2016 11:39 PM, "Ma Lin" wrote: > Hi, all, > > Is it possible to add an option that allow check Type Hints in > runtime? > So that I can enable dynamic type checking when debugging. And > disable it in final deploy, no performance lose. > > regards. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From animalize81 at hotmail.com Fri Jul 8 04:19:41 2016 From: animalize81 at hotmail.com (Ma Lin) Date: Fri, 8 Jul 2016 08:19:41 +0000 Subject: [Python-ideas] Add an option that allow check Type Hints in runtime In-Reply-To: References: Message-ID: I forget to add python-ideas at python.org in CC, let me retell to the others: I mean real check by the interpreter, not an '__annotations__' attribute. And I just had a look, mypy is a static checker, without actually running the code. My thought is modifying CALL_FUNCTION and RETURN_VALUE of the interpreter, so that allow them to check in each function calling when option enalbed. Cite a case: def fun(a: str, b: int) -> int: return len(a) + b fun('arg1', 'arg2') Whit the option and run this code, the interpreter will give an error prompt: Type of argument b is wrong. (or something like this) This is very useful for development and debugging. Without the option, the interpreter just ignores Type Hints in runtime, so no performance lose in project's final deploy. IMO, this will maximize the benefit of Type Hints within a small cost. ? 7-8 ?? 14:42, David Mertz ??: This exists already. All the annotations love on the '__annotations__' attribute is a function or method. Check that at runtime however you like! On Jul 7, 2016 11:39 PM, "Ma Lin" > wrote: Hi, all, Is it possible to add an option that allow check Type Hints in runtime? So that I can enable dynamic type checking when debugging. And disable it in final deploy, no performance lose. regards. _______________________________________________ Python-ideas mailing list Python-ideas at python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Fri Jul 8 04:30:52 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 8 Jul 2016 18:30:52 +1000 Subject: [Python-ideas] Add an option that allow check Type Hints in runtime In-Reply-To: References: Message-ID: On Fri, Jul 8, 2016 at 6:19 PM, Ma Lin wrote: > I mean real check by the interpreter, not an '__annotations__' attribute. > And I just had a look, mypy is a static checker, without actually running > the code. > > My thought is modifying CALL_FUNCTION and RETURN_VALUE of the interpreter, > so that allow them to check in each function calling when option enalbed. > > Cite a case: > > def fun(a: str, b: int) -> int: > return len(a) + b > fun('arg1', 'arg2') > > Whit the option and run this code, the interpreter will give an error > prompt: > Type of argument b is wrong. (or something like this) > > This is very useful for development and debugging. You should still be able to do it without changing the core interpreter. def enforce_types(func): @functools.wraps(func) def wrapped(*args, **kw): if not args_valid(args, kw, func.__annotations__): raise TypeError ret = func(*args, **kw) if not type_valid(ret, func.__annotations__["return"]): raise TypeError return wrapped @enforce_types def fun(a: str, b: int) -> int: return len(a) + b fun('arg1', 'arg2') Now all you have to do is write the type_valid function (takes an object and an annotation, returns True if it matches) and args_valid (takes the positional and keyword args, pairs them up against the function's params, and returns True if all(type_valid(...)) for the args). The only benefit of hacking the interpreter core for this is that you'd save the hassle of pairing up positional/keyword args against the parameters. ChrisA From p.f.moore at gmail.com Fri Jul 8 04:50:53 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 8 Jul 2016 09:50:53 +0100 Subject: [Python-ideas] Add an option that allow check Type Hints in runtime In-Reply-To: References: Message-ID: On 8 July 2016 at 09:30, Chris Angelico wrote: > On Fri, Jul 8, 2016 at 6:19 PM, Ma Lin wrote: >> I mean real check by the interpreter, not an '__annotations__' attribute. >> And I just had a look, mypy is a static checker, without actually running >> the code. >> >> My thought is modifying CALL_FUNCTION and RETURN_VALUE of the interpreter, >> so that allow them to check in each function calling when option enalbed. [...] > > Now all you have to do is write the type_valid function (takes an > object and an annotation, returns True if it matches) and args_valid > (takes the positional and keyword args, pairs them up against the > function's params, and returns True if all(type_valid(...)) for the > args). The only benefit of hacking the interpreter core for this is > that you'd save the hassle of pairing up positional/keyword args > against the parameters. Just to clarify for the OP - the reason you're getting pushback on making this a built in interpreter function is twofold - first, an external facility could be used *now*, and would not need to be limited to Python 3.6+, and second, adding checks to the function call protocol is not free, and would therefore affect not just the minority of people who want to add checks for debugging, but also all other users (which include people for whom Python's performance is a key issue). The Python interpreter's function calling process is known to be slower than (some) people would like, and making it slower - even marginally - isn't something we would want to do lightly. With the ability to use external static analyzers like mypy, as well as the possibility of writing custom solutions like Chris described above, people wanting such checks have a number of options already, and so the cost of adding the feature to the interpreter (and possibly the language - do you see this as a CPython implementation detail, or would you expect it to be a language-level facility, so other Python implementations such as PyPy and Jython would be expected to have it as well?) needs to be justified. Personally, I'd view this feature as an interesting addition, but not something I'd use myself, and so I'm inclined towards the "why should I pay for something I won't use?" position. Paul From rosuav at gmail.com Fri Jul 8 05:02:21 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 8 Jul 2016 19:02:21 +1000 Subject: [Python-ideas] Add an option that allow check Type Hints in runtime In-Reply-To: References: Message-ID: On Fri, Jul 8, 2016 at 6:50 PM, Paul Moore wrote: > Just to clarify for the OP - the reason you're getting pushback on > making this a built in interpreter function is twofold - first, an > external facility could be used *now*, and would not need to be > limited to Python 3.6+, and second, adding checks to the function call > protocol is not free, and would therefore affect not just the minority > of people who want to add checks for debugging, but also all other > users (which include people for whom Python's performance is a key > issue). The Python interpreter's function calling process is known to > be slower than (some) people would like, and making it slower - even > marginally - isn't something we would want to do lightly. Exactly. The main reason IMO is the first one - you can add this checker to any Python 3 program, rather than wait for it to get coded, debugged, and shipped. Oh, and you can easily have a flag that disables checking for performance: if no_type_checks: def enforce_types(func): return func Now your decorator does nothing, and there is literally ZERO run-time cost (the only cost is a single cheap check when the function is defined, which usually means on module import), closures aside. That wouldn't be possible with a core interpreter change, unless it's made a compile-time option *for the interpreter* - something like "./configure --with-annotations-enforced" - and then you run python3.6 for the fast one, or python3.6tc for the type-checking one. That would be a pain. ChrisA From rosuav at gmail.com Fri Jul 8 05:05:44 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 8 Jul 2016 19:05:44 +1000 Subject: [Python-ideas] Add an option that allow check Type Hints in runtime In-Reply-To: References: Message-ID: On Fri, Jul 8, 2016 at 7:02 PM, Chris Angelico wrote: > Now your decorator does nothing, and there is literally ZERO run-time > cost (the only cost is a single cheap check when the function is > defined, which usually means on module import), closures aside. That > wouldn't be possible with a core interpreter change... I really shouldn't say impossible. Literally the moment I clicked Send, I thought of a way it could be done with minimal run-time cost: just create a completely separate opcode for "type-checked call". I don't think that would materially slow down the regular call operation. As a bonus, any functions that don't have annotations could use the normal call opcode, so they wouldn't be slowed down. But the main point is, this would all entail additional complexity inside CPython, where the decorator method keeps the complexity in your application - and doesn't need nearly as much of it. The one advantage that CPython has is, as mentioned, the matching of arguments to parameters; maybe that could be provided in functools somewhere? ChrisA From animalize81 at hotmail.com Fri Jul 8 05:17:56 2016 From: animalize81 at hotmail.com (Ma Lin) Date: Fri, 8 Jul 2016 09:17:56 +0000 Subject: [Python-ideas] Add an option that allow check Type Hints in runtime In-Reply-To: References: Message-ID: I see, Chris' decorator is amazing, I didn't think that way. I mean add an runtime option, not a build option, it can be only enabled in development/debugging: python -tc a.py Without the option, the interpreter just ignores Type Hints: python a.py Anyway, I just throw an idea to here. Greet to all of you for bring Python to me. From rosuav at gmail.com Fri Jul 8 05:35:36 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 8 Jul 2016 19:35:36 +1000 Subject: [Python-ideas] Add an option that allow check Type Hints in runtime In-Reply-To: References: Message-ID: On Fri, Jul 8, 2016 at 7:17 PM, Ma Lin wrote: > I see, Chris' decorator is amazing, I didn't think that way. That's one of the beauties of Python's model of "everything's an object". Functions can be manipulated (usually in broadly the same way as my example: creating a wrapper function that does stuff before and/or after calling the original), and you can basically do anything you like with the attributes. It's SO handy, in so many places! That said, though, you *can* go too far with that. I discovered recently that Ruby lacks a lot of fairly important features... which are monkey-patched in by Rails. So a Ruby on Rails application has a lot more functionality, even on core types (because you can add methods to the string type by monkey-patching), than a non-Rails Ruby app. There are some things that definitely do need interpreter support, which is why we're now seeing async and await becoming keywords. > I mean add an runtime option, not a build option, it can be only enabled > in development/debugging: > python -tc a.py > Without the option, the interpreter just ignores Type Hints: > python a.py Right, but this means that the code to call a function has to check "was the --tc option specified?" before actually calling the function. Even though the full type checks aren't done, this check has a nonzero cost; and since calling functions happens rather a lot in Python programs (remember, lots of things implicitly become function or method calls), it would add up. Everyone's code would be slowed down slightly to support the occasional case where you want the checking. > Anyway, I just throw an idea to here. Greet to all of you for bring > Python to me. Keep on throwing ideas this way! Ideas are great. Well-thought-out ideas make for interesting discussions, even if they don't get implemented. Often the answer ends up being "here's a way to do this with the current version", but it doesn't carry with it a tag of "and you're an idiot for even suggesting it". We like ideas here - otherwise we wouldn't subscribe to this list :) ChrisA From p.f.moore at gmail.com Fri Jul 8 06:34:11 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 8 Jul 2016 11:34:11 +0100 Subject: [Python-ideas] Add an option that allow check Type Hints in runtime In-Reply-To: References: Message-ID: On 8 July 2016 at 10:35, Chris Angelico wrote: >> Anyway, I just throw an idea to here. Greet to all of you for bring >> Python to me. > > Keep on throwing ideas this way! Ideas are great. Well-thought-out > ideas make for interesting discussions, even if they don't get > implemented. Often the answer ends up being "here's a way to do this > with the current version", but it doesn't carry with it a tag of "and > you're an idiot for even suggesting it". We like ideas here - > otherwise we wouldn't subscribe to this list :) Agreed! I'd have never even thought of Chris' decorator approach before this discussion occurred, so thanks for raising the subject. I may never use it, but knowing how to do something like this is great. Paul From elazarg at gmail.com Fri Jul 8 06:53:25 2016 From: elazarg at gmail.com (=?UTF-8?B?15DXnNei15bXqA==?=) Date: Fri, 08 Jul 2016 10:53:25 +0000 Subject: [Python-ideas] Python-ideas Digest, Vol 116, Issue 12 In-Reply-To: References: Message-ID: Function definition can also be changed inside the interpreter, effectively doing the same work as tge interpreter. The parser can add implicit decorator too. This raises a different approach: adding a module decorator, allowing such implicit decorations accross module functions. Has it been discussed before? ?????? ??? ??, 8 ????' 2016, 12:18, ??? ?: > Send Python-ideas mailing list submissions to > python-ideas at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/python-ideas > or, via email, send a message with subject or body 'help' to > python-ideas-request at python.org > > You can reach the person managing the list at > python-ideas-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-ideas digest..." > > > Today's Topics: > > 1. Re: Add an option that allow check Type Hints in runtime > (Chris Angelico) > 2. Re: Add an option that allow check Type Hints in runtime > (Paul Moore) > 3. Re: Add an option that allow check Type Hints in runtime > (Chris Angelico) > 4. Re: Add an option that allow check Type Hints in runtime > (Chris Angelico) > 5. Re: Add an option that allow check Type Hints in runtime (Ma Lin) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 8 Jul 2016 18:30:52 +1000 > From: Chris Angelico > Cc: python-ideas > Subject: Re: [Python-ideas] Add an option that allow check Type Hints > in runtime > Message-ID: > KbkXXi3Q at mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > On Fri, Jul 8, 2016 at 6:19 PM, Ma Lin wrote: > > I mean real check by the interpreter, not an '__annotations__' attribute. > > And I just had a look, mypy is a static checker, without actually running > > the code. > > > > My thought is modifying CALL_FUNCTION and RETURN_VALUE of the > interpreter, > > so that allow them to check in each function calling when option enalbed. > > > > Cite a case: > > > > def fun(a: str, b: int) -> int: > > return len(a) + b > > fun('arg1', 'arg2') > > > > Whit the option and run this code, the interpreter will give an error > > prompt: > > Type of argument b is wrong. (or something like this) > > > > This is very useful for development and debugging. > > You should still be able to do it without changing the core interpreter. > > def enforce_types(func): > @functools.wraps(func) > def wrapped(*args, **kw): > if not args_valid(args, kw, func.__annotations__): > raise TypeError > ret = func(*args, **kw) > if not type_valid(ret, func.__annotations__["return"]): > raise TypeError > return wrapped > > @enforce_types > def fun(a: str, b: int) -> int: > return len(a) + b > > fun('arg1', 'arg2') > > > Now all you have to do is write the type_valid function (takes an > object and an annotation, returns True if it matches) and args_valid > (takes the positional and keyword args, pairs them up against the > function's params, and returns True if all(type_valid(...)) for the > args). The only benefit of hacking the interpreter core for this is > that you'd save the hassle of pairing up positional/keyword args > against the parameters. > > ChrisA > > > ------------------------------ > > Message: 2 > Date: Fri, 8 Jul 2016 09:50:53 +0100 > From: Paul Moore > To: Chris Angelico > Cc: python-ideas > Subject: Re: [Python-ideas] Add an option that allow check Type Hints > in runtime > Message-ID: > < > CACac1F-+Si-uXwce2CKPiyM576MX7G+Cp4u8n-TXBO_89UTbdQ at mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > On 8 July 2016 at 09:30, Chris Angelico wrote: > > On Fri, Jul 8, 2016 at 6:19 PM, Ma Lin wrote: > >> I mean real check by the interpreter, not an '__annotations__' > attribute. > >> And I just had a look, mypy is a static checker, without actually > running > >> the code. > >> > >> My thought is modifying CALL_FUNCTION and RETURN_VALUE of the > interpreter, > >> so that allow them to check in each function calling when option > enalbed. > [...] > > > > Now all you have to do is write the type_valid function (takes an > > object and an annotation, returns True if it matches) and args_valid > > (takes the positional and keyword args, pairs them up against the > > function's params, and returns True if all(type_valid(...)) for the > > args). The only benefit of hacking the interpreter core for this is > > that you'd save the hassle of pairing up positional/keyword args > > against the parameters. > > Just to clarify for the OP - the reason you're getting pushback on > making this a built in interpreter function is twofold - first, an > external facility could be used *now*, and would not need to be > limited to Python 3.6+, and second, adding checks to the function call > protocol is not free, and would therefore affect not just the minority > of people who want to add checks for debugging, but also all other > users (which include people for whom Python's performance is a key > issue). The Python interpreter's function calling process is known to > be slower than (some) people would like, and making it slower - even > marginally - isn't something we would want to do lightly. > > With the ability to use external static analyzers like mypy, as well > as the possibility of writing custom solutions like Chris described > above, people wanting such checks have a number of options already, > and so the cost of adding the feature to the interpreter (and possibly > the language - do you see this as a CPython implementation detail, or > would you expect it to be a language-level facility, so other Python > implementations such as PyPy and Jython would be expected to have it > as well?) needs to be justified. > > Personally, I'd view this feature as an interesting addition, but not > something I'd use myself, and so I'm inclined towards the "why should > I pay for something I won't use?" position. > > Paul > > > ------------------------------ > > Message: 3 > Date: Fri, 8 Jul 2016 19:02:21 +1000 > From: Chris Angelico > Cc: python-ideas > Subject: Re: [Python-ideas] Add an option that allow check Type Hints > in runtime > Message-ID: > ffM+NB_ysUPNQxvXJUm4UtbFZg at mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > On Fri, Jul 8, 2016 at 6:50 PM, Paul Moore wrote: > > Just to clarify for the OP - the reason you're getting pushback on > > making this a built in interpreter function is twofold - first, an > > external facility could be used *now*, and would not need to be > > limited to Python 3.6+, and second, adding checks to the function call > > protocol is not free, and would therefore affect not just the minority > > of people who want to add checks for debugging, but also all other > > users (which include people for whom Python's performance is a key > > issue). The Python interpreter's function calling process is known to > > be slower than (some) people would like, and making it slower - even > > marginally - isn't something we would want to do lightly. > > Exactly. The main reason IMO is the first one - you can add this > checker to any Python 3 program, rather than wait for it to get coded, > debugged, and shipped. > > Oh, and you can easily have a flag that disables checking for performance: > > if no_type_checks: > def enforce_types(func): > return func > > Now your decorator does nothing, and there is literally ZERO run-time > cost (the only cost is a single cheap check when the function is > defined, which usually means on module import), closures aside. That > wouldn't be possible with a core interpreter change, unless it's made > a compile-time option *for the interpreter* - something like > "./configure --with-annotations-enforced" - and then you run python3.6 > for the fast one, or python3.6tc for the type-checking one. That would > be a pain. > > ChrisA > > > ------------------------------ > > Message: 4 > Date: Fri, 8 Jul 2016 19:05:44 +1000 > From: Chris Angelico > Cc: python-ideas > Subject: Re: [Python-ideas] Add an option that allow check Type Hints > in runtime > Message-ID: > < > CAPTjJmrJstnRzEK48Kj7gOZDi5kkfBPrUCNYQqGds3qje-ZFbA at mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > On Fri, Jul 8, 2016 at 7:02 PM, Chris Angelico wrote: > > Now your decorator does nothing, and there is literally ZERO run-time > > cost (the only cost is a single cheap check when the function is > > defined, which usually means on module import), closures aside. That > > wouldn't be possible with a core interpreter change... > > I really shouldn't say impossible. Literally the moment I clicked > Send, I thought of a way it could be done with minimal run-time cost: > just create a completely separate opcode for "type-checked call". I > don't think that would materially slow down the regular call > operation. As a bonus, any functions that don't have annotations could > use the normal call opcode, so they wouldn't be slowed down. > > But the main point is, this would all entail additional complexity > inside CPython, where the decorator method keeps the complexity in > your application - and doesn't need nearly as much of it. The one > advantage that CPython has is, as mentioned, the matching of arguments > to parameters; maybe that could be provided in functools somewhere? > > ChrisA > > > ------------------------------ > > Message: 5 > Date: Fri, 8 Jul 2016 09:17:56 +0000 > From: Ma Lin > To: "python-ideas at python.org" > Subject: Re: [Python-ideas] Add an option that allow check Type Hints > in runtime > Message-ID: > < > PS1PR06MB1099F602EB64A07AAC2063F0B63C0 at PS1PR06MB1099.apcprd06.prod.outlook.com > > > > Content-Type: text/plain; charset="Windows-1252" > > I see, Chris' decorator is amazing, I didn't think that way. > > I mean add an runtime option, not a build option, it can be only enabled > in development/debugging: > python -tc a.py > Without the option, the interpreter just ignores Type Hints: > python a.py > > Anyway, I just throw an idea to here. Greet to all of you for bring > Python to me. > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > > > ------------------------------ > > End of Python-ideas Digest, Vol 116, Issue 12 > ********************************************* > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Jul 8 09:23:51 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 8 Jul 2016 23:23:51 +1000 Subject: [Python-ideas] Add an option that allow check Type Hints in runtime In-Reply-To: References: Message-ID: On 8 July 2016 at 20:34, Paul Moore wrote: > On 8 July 2016 at 10:35, Chris Angelico wrote: >>> Anyway, I just throw an idea to here. Greet to all of you for bring >>> Python to me. >> >> Keep on throwing ideas this way! Ideas are great. Well-thought-out >> ideas make for interesting discussions, even if they don't get >> implemented. Often the answer ends up being "here's a way to do this >> with the current version", but it doesn't carry with it a tag of "and >> you're an idiot for even suggesting it". We like ideas here - >> otherwise we wouldn't subscribe to this list :) > > Agreed! I'd have never even thought of Chris' decorator approach > before this discussion occurred, so thanks for raising the subject. I > may never use it, but knowing how to do something like this is great. The runtime eval loop switching being considered to enable Pyjion's method JIT would also let folks opt in to running a dynamically typechecked runtime (assuming someone decided to write one). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From mertz at gnosis.cx Fri Jul 8 12:03:43 2016 From: mertz at gnosis.cx (David Mertz) Date: Fri, 8 Jul 2016 09:03:43 -0700 Subject: [Python-ideas] Add an option that allow check Type Hints in runtime In-Reply-To: References: Message-ID: I provided a decorator recipe to do just what Chris Angelico describes a year or two before PEP 484 described static typechecking. I think what I do is a consistent subset of PEP 484 still, modulo the difference between runtime and static. See: http://code.activestate.com/recipes/578528-type-checking-using-python-3x-annotations/ It would be a lot more work to do everything the typing.py module and PEP 484 describes (algebraic typing, basically), but it's all just code that is compatible back to Python 3.0. On Fri, Jul 8, 2016 at 3:34 AM, Paul Moore wrote: > On 8 July 2016 at 10:35, Chris Angelico wrote: > >> Anyway, I just throw an idea to here. Greet to all of you for bring > >> Python to me. > > > > Keep on throwing ideas this way! Ideas are great. Well-thought-out > > ideas make for interesting discussions, even if they don't get > > implemented. Often the answer ends up being "here's a way to do this > > with the current version", but it doesn't carry with it a tag of "and > > you're an idiot for even suggesting it". We like ideas here - > > otherwise we wouldn't subscribe to this list :) > > Agreed! I'd have never even thought of Chris' decorator approach > before this discussion occurred, so thanks for raising the subject. I > may never use it, but knowing how to do something like this is great. > > Paul > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Fri Jul 8 12:12:08 2016 From: wes.turner at gmail.com (Wes Turner) Date: Fri, 8 Jul 2016 11:12:08 -0500 Subject: [Python-ideas] Add an option that allow check Type Hints in runtime In-Reply-To: References: Message-ID: pycontracts supports runtime type and bound checking on Python 2 & 3 There is a global toggle: https://andreacensi.github.io/contracts/overhead.html#overhead Unfortunately, the type specifiers precede and are different (e.g lowercase) from those used for static analysis in mypy. There are orobably additional differences? On Jul 8, 2016 11:04 AM, "David Mertz" wrote: > I provided a decorator recipe to do just what Chris Angelico describes a > year or two before PEP 484 described static typechecking. I think what I > do is a consistent subset of PEP 484 still, modulo the difference between > runtime and static. > > See: > http://code.activestate.com/recipes/578528-type-checking-using-python-3x-annotations/ > > It would be a lot more work to do everything the typing.py module and PEP > 484 describes (algebraic typing, basically), but it's all just code that is > compatible back to Python 3.0. > > On Fri, Jul 8, 2016 at 3:34 AM, Paul Moore wrote: > >> On 8 July 2016 at 10:35, Chris Angelico wrote: >> >> Anyway, I just throw an idea to here. Greet to all of you for bring >> >> Python to me. >> > >> > Keep on throwing ideas this way! Ideas are great. Well-thought-out >> > ideas make for interesting discussions, even if they don't get >> > implemented. Often the answer ends up being "here's a way to do this >> > with the current version", but it doesn't carry with it a tag of "and >> > you're an idiot for even suggesting it". We like ideas here - >> > otherwise we wouldn't subscribe to this list :) >> >> Agreed! I'd have never even thought of Chris' decorator approach >> before this discussion occurred, so thanks for raising the subject. I >> may never use it, but knowing how to do something like this is great. >> >> Paul >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > > -- > Keeping medicines from the bloodstreams of the sick; food > from the bellies of the hungry; books from the hands of the > uneducated; technology from the underdeveloped; and putting > advocates of freedom in prisons. Intellectual property is > to the 21st century what the slave trade was to the 16th. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.selik at gmail.com Fri Jul 8 12:26:08 2016 From: michael.selik at gmail.com (Michael Selik) Date: Fri, 08 Jul 2016 16:26:08 +0000 Subject: [Python-ideas] bytes indexing behavior In-Reply-To: References: <20160603131730.GA29255@python.ca> <087001d1be4c$99423520$cbc69f60$@hotmail.com> <20160604082715.GB20628@python.ca> <08e901d1beb8$e60915b0$b21b4110$@hotmail.com> <20160606193530.GA27695@python.ca> <5756EF5A.106@stoneleaf.us> <57572F17.4010009@stoneleaf.us> Message-ID: On Fri, Jul 8, 2016, 1:03 AM Franklin? Lee wrote: > On Jul 6, 2016 2:40 PM, "Michael Selik" wrote: > > Instead it seems the best way given the current behavior is to write: > > > > len(bytestring) == bytestring.count(b'z') > > > > While I wait for PEP 467, can anyone suggest a better way to write that? > > How about > set(bytestring) == set(b'z') > Ah, that makes sense. I'd thought of using a set literal on the right-hand, but for some reason using set constructor slipped my mind. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From leewangzhong+python at gmail.com Sat Jul 9 16:17:37 2016 From: leewangzhong+python at gmail.com (Franklin? Lee) Date: Sat, 9 Jul 2016 16:17:37 -0400 Subject: [Python-ideas] bytes indexing behavior In-Reply-To: References: <20160603131730.GA29255@python.ca> <087001d1be4c$99423520$cbc69f60$@hotmail.com> <20160604082715.GB20628@python.ca> <08e901d1beb8$e60915b0$b21b4110$@hotmail.com> <20160606193530.GA27695@python.ca> <5756EF5A.106@stoneleaf.us> <57572F17.4010009@stoneleaf.us> Message-ID: On Jul 8, 2016 12:26 PM, "Michael Selik" wrote: > > On Fri, Jul 8, 2016, 1:03 AM Franklin? Lee wrote: >> >> On Jul 6, 2016 2:40 PM, "Michael Selik" wrote: >> > Instead it seems the best way given the current behavior is to write: >> > >> > len(bytestring) == bytestring.count(b'z') >> > >> > While I wait for PEP 467, can anyone suggest a better way to write that? >> >> How about >> set(bytestring) == set(b'z') > > Ah, that makes sense. I'd thought of using a set literal on the right-hand, but for some reason using set constructor slipped my mind. The previous method didn't allow short-circuiting. We can use `all` without calling `ord` each time, by the obvious way. z = b'z'[0] #or next(iter(b'z')) all(byte == z for byte in bytestring) Make it a function. def eqall(iterable, value): return all(x == value for x in iterable) eqall(bytestring, b'z'[0]) Rewriting in functional programming style. def eqall(iterable, value): return all(map(value.__eq__, iterable)) -------------- next part -------------- An HTML attachment was scrubbed... URL: From pavol.lisy at gmail.com Sat Jul 9 16:23:03 2016 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Sat, 9 Jul 2016 22:23:03 +0200 Subject: [Python-ideas] Add an option that allow check Type Hints in runtime In-Reply-To: References: Message-ID: On 7/8/16, Chris Angelico wrote: > On Fri, Jul 8, 2016 at 7:17 PM, Ma Lin wrote: >> I mean add an runtime option, not a build option, it can be only enabled >> in development/debugging: >> python -tc a.py >> Without the option, the interpreter just ignores Type Hints: >> python a.py > > Right, but this means that the code to call a function has to check > "was the --tc option specified?" before actually calling the function. Could not interpreter just internally use "decorator trick" too? From pavol.lisy at gmail.com Sun Jul 10 02:29:53 2016 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Sun, 10 Jul 2016 08:29:53 +0200 Subject: [Python-ideas] bytes indexing behavior In-Reply-To: References: <20160603131730.GA29255@python.ca> <087001d1be4c$99423520$cbc69f60$@hotmail.com> <20160604082715.GB20628@python.ca> <08e901d1beb8$e60915b0$b21b4110$@hotmail.com> <20160606193530.GA27695@python.ca> <5756EF5A.106@stoneleaf.us> <57572F17.4010009@stoneleaf.us> Message-ID: On 7/9/16, Franklin? Lee wrote: > On Jul 8, 2016 12:26 PM, "Michael Selik" wrote: >> >> On Fri, Jul 8, 2016, 1:03 AM Franklin? Lee >> > wrote: >>> >>> On Jul 6, 2016 2:40 PM, "Michael Selik" wrote: >>> > Instead it seems the best way given the current behavior is to write: >>> > >>> > len(bytestring) == bytestring.count(b'z') >>> > >>> > While I wait for PEP 467, can anyone suggest a better way to write > that? >>> >>> How about >>> set(bytestring) == set(b'z') >> >> Ah, that makes sense. I'd thought of using a set literal on the > right-hand, but for some reason using set constructor slipped my mind. > > The previous method didn't allow short-circuiting. We can use `all` without > calling `ord` each time, by the obvious way. > z = b'z'[0] #or next(iter(b'z')) > all(byte == z for byte in bytestring) > > Make it a function. > def eqall(iterable, value): > return all(x == value for x in iterable) > > eqall(bytestring, b'z'[0]) > > Rewriting in functional programming style. > def eqall(iterable, value): > return all(map(value.__eq__, iterable)) > Or if you want to accept more values: def inall(iterable, values): return all(x in values for x in iterable) inall(bytestring, list(b'zZ')) PS. If you got this mail twice pls sorry - something goes wrong in my mail client. From sivachandra at google.com Mon Jul 11 13:42:37 2016 From: sivachandra at google.com (Siva Chandra) Date: Mon, 11 Jul 2016 10:42:37 -0700 Subject: [Python-ideas] Proposal to add const-ness type hints to PEP-484 Message-ID: Hello, I have submitted a proposal to add const-ness type hint constructs here: https://github.com/python/typing/issues/242 Looking for feedback on the proposal. Thanks, Siva Chandra From wes.turner at gmail.com Mon Jul 11 14:08:52 2016 From: wes.turner at gmail.com (Wes Turner) Date: Mon, 11 Jul 2016 13:08:52 -0500 Subject: [Python-ideas] Proposal to add const-ness type hints to PEP-484 In-Reply-To: References: Message-ID: "Persistent / Immutable / Functional data structures for Python" https://github.com/tobgu/pyrsistent IIRC, there are a number of threads discussing adding const as a language feature? It may be worth linking those here / in the gh issue? IIUC, the linked proposal describes adding const as an annotation feature for static analysis only. Would const as a language feature have any effect on the gilectomy? e.g. less locks required Ways to achieve practical runtime const (~immutability): * add a has_key() check in __setattr__ https://gist.github.com/ashwin/e3844158e3d7afd16adb * define an @property without a propname.setter (for a __dunder_prefixed class attr which will otherwise be name-mangled) * const as a language feature . * from typing import Const this would be a great hint for static analysis, but would things really be const? On Jul 11, 2016 12:43 PM, "Siva Chandra via Python-ideas" < python-ideas at python.org> wrote: > Hello, > > I have submitted a proposal to add const-ness type hint constructs > here: https://github.com/python/typing/issues/242 > > Looking for feedback on the proposal. > > Thanks, > Siva Chandra > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephanh42 at gmail.com Tue Jul 12 08:38:22 2016 From: stephanh42 at gmail.com (Stephan Houben) Date: Tue, 12 Jul 2016 14:38:22 +0200 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= Message-ID: Hi list, Here is my speculative language idea for Python: Allow the following alternative spelling of the keyword `lambda': ? (That is "Unicode Character 'GREEK SMALL LETTER LAMDA' (U+03BB).") Background: I have been using the Vim "conceal" functionality with a rule which visually replaces lambda with ? when editing Python files. I find this a great improvement in readability since ? is visually less distracting while still quite distinctive. (The fact that ? is syntax-colored as a keyword also helps with this.) However, at the moment the nice syntax is lost when looking at the file through another editor or viewer. Therefore I would really like this to be an official part of the Python syntax. I know people have been clamoring for shorter lambda-syntax in the past, I think this is a nice minimal extension. Example code: lst.sort(key=lambda x: x.lookup_first_name()) lst.sort(key=? x: x.lookup_first_name()) # Church numerals zero = ? f: ? x: x one = ? f: ? x: f(x) two = ? f: ? x: f(f(x)) (Yes, Python is my favorite Scheme dialect. Why did you ask?) Note that a number of other languages already allow this. (Racket, Haskell). You can judge the aesthetics of this on your own code with the following sed command. sed 's/\/?/g' Advantages: * The lambda keyword is quite long and distracts from the "meat" of the lambda expression. Replacing it by a single-character keyword improves readability. * The resulting code resembles more closely mathematical notation (in particular, lambda-calculus notation), so it brings Python closer to being "executable pseudo-code". * The alternative spelling ?/lambda is quite intuitive (at least to anybody who knows Greek letters.) Disadvantages: For your convenience already noticed here: * Introducing ? is introducing TIMTOWTDI. * Hard to type with certain editors. But note that the old syntax is still available. Easy to fix by upgrading to VIM ;-) * Will turn a pre-existing legal identifier ? into a keyword. So backward-incompatible. Needless to say, my personal opinion is that the advantages outweigh the disadvantages. ;-) Greetings, Stephan -------------- next part -------------- An HTML attachment was scrubbed... URL: From random832 at fastmail.com Tue Jul 12 09:42:25 2016 From: random832 at fastmail.com (Random832) Date: Tue, 12 Jul 2016 09:42:25 -0400 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: Message-ID: <1468330945.481991.663978161.09323860@webmail.messagingengine.com> On Tue, Jul 12, 2016, at 08:38, Stephan Houben wrote: > I know people have been clamoring for shorter lambda-syntax in the > past, I think this is a nice minimal extension. How about a haskell-style backslash? From tritium-list at sdamon.com Tue Jul 12 14:36:10 2016 From: tritium-list at sdamon.com (tritium-list at sdamon.com) Date: Tue, 12 Jul 2016 14:36:10 -0400 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: Message-ID: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> For better or worse, except for string literals which can be anything as long as you set a coding comment, python is pure ascii which simplifies everything. Lambda is not in the first 128 characters of Unicode, so it is highly unlikely to be accepted. * ?Lambda? is exactly as discouraging to type as it needs to be. A more likely to be accepted alternate keyword is ?whyareyounotusingdef? * Python doesn?t attempt to look like mathematical formula * ?Lambda? spelling is intuitive to most people who program * TIMTOWTDI isn?t a religious edict. Python is more pragmatic than that. * It?s hard to type in ALL editors unless your locale is set to (ancient?) Greek. * ? What are you doing to have an identifier outside of ?[A-Za-z_][A-Za-z0-9_]+?? From: Python-ideas [mailto:python-ideas-bounces+tritium-list=sdamon.com at python.org] On Behalf Of Stephan Houben Sent: Tuesday, July 12, 2016 8:38 AM To: python-ideas at python.org Subject: [Python-ideas] allow `lambda' to be spelled ? Hi list, Here is my speculative language idea for Python: Allow the following alternative spelling of the keyword `lambda': ? (That is "Unicode Character 'GREEK SMALL LETTER LAMDA' (U+03BB).") Background: I have been using the Vim "conceal" functionality with a rule which visually replaces lambda with ? when editing Python files. I find this a great improvement in readability since ? is visually less distracting while still quite distinctive. (The fact that ? is syntax-colored as a keyword also helps with this.) However, at the moment the nice syntax is lost when looking at the file through another editor or viewer. Therefore I would really like this to be an official part of the Python syntax. I know people have been clamoring for shorter lambda-syntax in the past, I think this is a nice minimal extension. Example code: lst.sort(key=lambda x: x.lookup_first_name()) lst.sort(key=? x: x.lookup_first_name()) # Church numerals zero = ? f: ? x: x one = ? f: ? x: f(x) two = ? f: ? x: f(f(x)) (Yes, Python is my favorite Scheme dialect. Why did you ask?) Note that a number of other languages already allow this. (Racket, Haskell). You can judge the aesthetics of this on your own code with the following sed command. sed 's/\/?/g' Advantages: * The lambda keyword is quite long and distracts from the "meat" of the lambda expression. Replacing it by a single-character keyword improves readability. * The resulting code resembles more closely mathematical notation (in particular, lambda-calculus notation), so it brings Python closer to being "executable pseudo-code". * The alternative spelling ?/lambda is quite intuitive (at least to anybody who knows Greek letters.) Disadvantages: For your convenience already noticed here: * Introducing ? is introducing TIMTOWTDI. * Hard to type with certain editors. But note that the old syntax is still available. Easy to fix by upgrading to VIM ;-) * Will turn a pre-existing legal identifier ? into a keyword. So backward-incompatible. Needless to say, my personal opinion is that the advantages outweigh the disadvantages. ;-) Greetings, Stephan -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Tue Jul 12 15:28:45 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 13 Jul 2016 05:28:45 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> Message-ID: On Wed, Jul 13, 2016 at 4:36 AM, wrote: > For better or worse, except for string literals which can be anything as > long as you set a coding comment, python is pure ascii which simplifies > everything. Lambda is not in the first 128 characters of Unicode, so it is > highly unlikely to be accepted. Incorrect as of Python 3 - it's pure Unicode :) You can have identifiers that use non-ASCII characters. The core language in the default interpreter is all ASCII in order to make it easy for most people to type, but there are variant Pythons that translate the keywords into other languages (I believe there's a Chinese Python, and possibly Korean?), which are then free to use whatever character set they like. A variant Python would be welcome to translate all the operators and keywords into single-character tokens, using Unicode symbols for NOT EQUAL TO and so on - including using U+03BB in place of 'lambda'. ChrisA From srkunze at mail.de Tue Jul 12 16:21:26 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Tue, 12 Jul 2016 22:21:26 +0200 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> Message-ID: <57855146.80805@mail.de> Completely matches my opinion. Thanks. On 12.07.2016 20:36, tritium-list at sdamon.com wrote: > > For better or worse, except for string literals which can be anything > as long as you set a coding comment, python is pure ascii which > simplifies everything. Lambda is not in the first 128 characters of > Unicode, so it is highly unlikely to be accepted. > > ??Lambda? is exactly as discouraging to type as it needs to be. A > more likely to be accepted alternate keyword is ?whyareyounotusingdef? > > ?Python doesn?t attempt to look like mathematical formula > > ??Lambda? spelling is intuitive to most people who program > > ?TIMTOWTDI isn?t a religious edict. Python is more pragmatic than that. > > ?It?s hard to type in ALL editors unless your locale is set to > (ancient?) Greek. > > ?? What are you doing to have an identifier outside of > ?[A-Za-z_][A-Za-z0-9_]+?? > > *From:*Python-ideas > [mailto:python-ideas-bounces+tritium-list=sdamon.com at python.org] *On > Behalf Of *Stephan Houben > *Sent:* Tuesday, July 12, 2016 8:38 AM > *To:* python-ideas at python.org > *Subject:* [Python-ideas] allow `lambda' to be spelled ? > > Hi list, > > Here is my speculative language idea for Python: > > Allow the following alternative spelling of the keyword `lambda': > > ? > > (That is "Unicode Character 'GREEK SMALL LETTER LAMDA' (U+03BB).") > > Background: > > I have been using the Vim "conceal" functionality with a rule which > visually > > replaces lambda with ? when editing Python files. I find this a great > improvement in > > readability since ? is visually less distracting while still quite > distinctive. > > (The fact that ? is syntax-colored as a keyword also helps with this.) > > However, at the moment the nice syntax is lost when looking at the > file through another editor or viewer. > > Therefore I would really like this to be an official part of the > Python syntax. > > I know people have been clamoring for shorter lambda-syntax in the > past, I think this is > > a nice minimal extension. > > Example code: > > lst.sort(key=lambda x: x.lookup_first_name()) > > lst.sort(key=? x: x.lookup_first_name()) > > # Church numerals > > zero = ? f: ? x: x > > one = ? f: ? x: f(x) > > two = ? f: ? x: f(f(x)) > > (Yes, Python is my favorite Scheme dialect. Why did you ask?) > > Note that a number of other languages already allow this. (Racket, > Haskell). > > You can judge the aesthetics of this on your own code with the > following sed command. > > sed 's/\/?/g' > > Advantages: > > * The lambda keyword is quite long and distracts from the "meat" of > the lambda expression. > > Replacing it by a single-character keyword improves readability. > > * The resulting code resembles more closely mathematical notation (in > particular, lambda-calculus notation), > > so it brings Python closer to being "executable pseudo-code". > > * The alternative spelling ?/lambda is quite intuitive (at least to > anybody who knows Greek letters.) > > Disadvantages: > > For your convenience already noticed here: > > * Introducing ? is introducing TIMTOWTDI. > > * Hard to type with certain editors. > > But note that the old syntax is still available. > > Easy to fix by upgrading to VIM ;-) > > * Will turn a pre-existing legal identifier ? into a keyword. > > So backward-incompatible. > > Needless to say, my personal opinion is that the advantages outweigh > the disadvantages. ;-) > > Greetings, > > Stephan > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mafagafogigante at gmail.com Tue Jul 12 18:11:32 2016 From: mafagafogigante at gmail.com (Bernardo Sulzbach) Date: Tue, 12 Jul 2016 19:11:32 -0300 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: Message-ID: <0dbf3c28-e224-0906-ffe3-760480c743d8@gmail.com> Racket has this. I write lambdas there and I don't use (or know people that use) the symbol anyway. It really does not save a lot. Python is more of a hammer than a beautiful mathematical tool, so the examples don't seem too convincing. Plus, 80 columns and shortness does not make so much sense nowadays. From ben+python at benfinney.id.au Tue Jul 12 19:46:37 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 13 Jul 2016 09:46:37 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= References: Message-ID: <85poqito6a.fsf@benfinney.id.au> Stephan Houben writes: > Here is my speculative language idea for Python: Thank you for raising it. > Allow the following alternative spelling of the keyword `lambda': > ? > [?] > Therefore I would really like this to be an official part of the > Python syntax. > > I know people have been clamoring for shorter lambda-syntax in the > past, I think this is a nice minimal extension. The question is not whether it would be nice, nor how many people have clamoured for it. That you feel this supports the proposal isn't a good sign :-) The question is: What significant improvements to the language are made by this proposal, to counteract the significant cost of *any* such change to the language syntax? > Advantages: > > * The lambda keyword is quite long and distracts from the "meat" of > the lambda expression. Replacing it by a single-character keyword > improves readability. I disagree on this point. Making lambda easier is an attractive nuisance; the ?def? statement is superior (for Python) in most situations, and so lamda expressions should not be easier than that. > * The resulting code resembles more closely mathematical notation (in > particular, lambda-calculus notation), so it brings Python closer to > being "executable pseudo-code". How is that an advantage? > * The alternative spelling ?/lambda is quite intuitive (at least to > anybody who knows Greek letters.) I reject this use of ?intuitive?; no one knows intuitively what lambda is, what ? is, what the correspondence between them is, and what they mean in various contexts. All of that needs to be learned, specifically. so if this is an advantage, it needs to be expressed somehow other than ?intuitive?. Maybe you mean ?familiar?, and avoid that term because it makes for a weaker argument? > Disadvantages: I agree with your assessment of disadvantages, and re-iterate the inherent disadvantage that any language change brings significant cost to the Python core developers and the whole Python community. That's why most such suggestions have a significant hurdle to demonstrate a benefit. -- \ ?Prediction is very difficult, especially of the future.? | `\ ?Niels Bohr | _o__) | Ben Finney From jsbueno at python.org.br Tue Jul 12 20:00:16 2016 From: jsbueno at python.org.br (Joao S. O. Bueno) Date: Tue, 12 Jul 2016 21:00:16 -0300 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <85poqito6a.fsf@benfinney.id.au> References: <85poqito6a.fsf@benfinney.id.au> Message-ID: Stephan: Have you met the coconut project already? https://pypi.python.org/pypi/coconut, http://coconut.readthedocs.io/en/master/DOCS.html They create a superset of Python with the aim to allow writting smaller functional programs, and have a super-featured functional set of operators and such. It is a pure-Python project that pre-compiles their "coconut" program files to .py at compile time - They have a shorter syntax for lambda already - maybe that could be of use to you - and maybe you can get them to accept your suggestion - it certainly would fit there. """ Lambdas Coconut provides the simple, clean -> operator as an alternative to Python?s lambda statements. The operator has the same precedence as the old statement. Rationale In Python, lambdas are ugly and bulky, requiring the entire word lambda to be written out every time one is constructed. This is fine if in-line functions are very rarely needed, but in functional programming in-line functions are an essential tool. Example: dubsums = map((x, y) -> 2*(x+y), range(0, 10), range(10, 20)) """ From rymg19 at gmail.com Tue Jul 12 20:17:33 2016 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Tue, 12 Jul 2016 19:17:33 -0500 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: Message-ID: Doesn't this kind of violate Python's "one way to do it"? (Also, sorry for the top post; I'm on mobile right now...) -- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something?s wrong. http://kirbyfan64.github.io/ On Jul 12, 2016 7:38 AM, "Stephan Houben" wrote: > Hi list, > > Here is my speculative language idea for Python: > > Allow the following alternative spelling of the keyword `lambda': > > ? > > (That is "Unicode Character 'GREEK SMALL LETTER LAMDA' (U+03BB).") > > Background: > > I have been using the Vim "conceal" functionality with a rule which > visually > replaces lambda with ? when editing Python files. I find this a great > improvement in > readability since ? is visually less distracting while still quite > distinctive. > (The fact that ? is syntax-colored as a keyword also helps with this.) > > However, at the moment the nice syntax is lost when looking at the file > through another editor or viewer. > Therefore I would really like this to be an official part of the Python > syntax. > > I know people have been clamoring for shorter lambda-syntax in the past, I > think this is > a nice minimal extension. > > > Example code: > > lst.sort(key=lambda x: x.lookup_first_name()) > lst.sort(key=? x: x.lookup_first_name()) > > # Church numerals > zero = ? f: ? x: x > one = ? f: ? x: f(x) > two = ? f: ? x: f(f(x)) > > (Yes, Python is my favorite Scheme dialect. Why did you ask?) > > Note that a number of other languages already allow this. (Racket, > Haskell). > > You can judge the aesthetics of this on your own code with the following > sed command. > > sed 's/\/?/g' > > Advantages: > > * The lambda keyword is quite long and distracts from the "meat" of the > lambda expression. > Replacing it by a single-character keyword improves readability. > > * The resulting code resembles more closely mathematical notation (in > particular, lambda-calculus notation), > so it brings Python closer to being "executable pseudo-code". > > * The alternative spelling ?/lambda is quite intuitive (at least to > anybody who knows Greek letters.) > > > Disadvantages: > > For your convenience already noticed here: > > * Introducing ? is introducing TIMTOWTDI. > > * Hard to type with certain editors. > But note that the old syntax is still available. > Easy to fix by upgrading to VIM ;-) > > * Will turn a pre-existing legal identifier ? into a keyword. > So backward-incompatible. > > Needless to say, my personal opinion is that the advantages outweigh the > disadvantages. ;-) > > Greetings, > > Stephan > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Tue Jul 12 22:04:19 2016 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 13 Jul 2016 11:04:19 +0900 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> Message-ID: <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> Chris Angelico writes: > A variant Python would be welcome to translate all the operators > and keywords into single-character tokens, using Unicode symbols > for NOT EQUAL TO and so on - including using U+03BB in place of > 'lambda'. Probably it would not be "welcome", except in the usual sense that "Python is open source, you can do what you want". There was extensive discussion about the issues surrounding the natural languages used by programmers in source documentation (eg, identifier choice and comments) at the time of PEP 263. The mojibake (choice of charset) problem has largely improved since then, thanks to Unicode adoption, especially UTF-8. But the "Tower of Babel" issue has not. Fundamentally, it's like women's clothes (they wear them to impress, ie, communicate to, other women -- few men have the interest to understand what is impressive ;-): programming is about programmers communicating to other programmers. Maintaining the traditional spelling of keywords and operators is definitely useful for that purpose. This is not to say that individuals who want a personalized[1] language are wrong, just that it would have a net negative impact on communication in teams. BTW, Barry long advocated use of some variant syntaxes (the one I like to remember inaccurately is "><" instead of "!="), and in fact provided an easter egg import (barry_is_flufl or something like that) that changed the syntax to suit him. I believe that module is pure Python, so people who want to customize the lexical definition of Python at the language level can do so AFAIK. You could probably even spell it "import ??" (to take a very ugly page from the Book of GNU, mixing scripts in a single word -- the Han character means "etc."). Footnotes: [1] I don't have a better word. I mean something like "seasoned to taste", almost "tasteful" but not quite. From rosuav at gmail.com Tue Jul 12 23:25:59 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 13 Jul 2016 13:25:59 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> Message-ID: On Wed, Jul 13, 2016 at 12:04 PM, Stephen J. Turnbull wrote: > Chris Angelico writes: > > > A variant Python would be welcome to translate all the operators > > and keywords into single-character tokens, using Unicode symbols > > for NOT EQUAL TO and so on - including using U+03BB in place of > > 'lambda'. > > Probably it would not be "welcome", except in the usual sense that > "Python is open source, you can do what you want". > > ... programmers communicating to other programmers. ... > > This is not to say that individuals who want a personalized[1] > language are wrong, just that it would have a net negative impact on > communication in teams. A fair point. But Python has a strong mathematical side (look how big the numpy/scipy/matplotlib communities are), and we've already seen how strongly they prefer "a @ b" to "a.matmul(b)". If there's support for a language variant that uses more and shorter symbols, that would be where I'd expect to find it. ChrisA not a mathematician, although I play one on the internet sometimes From steve at pearwood.info Wed Jul 13 00:00:26 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 13 Jul 2016 14:00:26 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> Message-ID: <20160713040025.GN27919@ando.pearwood.info> On Wed, Jul 13, 2016 at 11:04:19AM +0900, Stephen J. Turnbull wrote: > There was extensive discussion about the issues surrounding the > natural languages used by programmers in source documentation (eg, > identifier choice and comments) at the time of PEP 263. The mojibake > (choice of charset) problem has largely improved since then, thanks to > Unicode adoption, especially UTF-8. But the "Tower of Babel" issue > has not. Fundamentally, it's like women's clothes (they wear them to > impress, ie, communicate to, other women -- few men have the interest > to understand what is impressive ;-): programming is about programmers > communicating to other programmers. With respect Stephen, that's codswallop :-) It might be true that the average bogan[1] bloke or socially awkward geek (including myself) might not care about impressive clothes, but many men do dress to compete. The difference is more socio-economic: typically women dress to compete across most s-e groups, while men mostly do so only in the upper-middle and upper classes. And in the upper classes, competition tends to be more understated and subtle ("good taste"), i.e. expensive Italian suits rather than hot pants. Historically, it is usually men who dress like peacocks to impress socially, while women are comparatively restrained. The drab business suit of Anglo-American influence is no more representative of male clothing through the ages as is the Communist Chinese "Mao suit". And as for programmers... the popularity of one-liners, the obfuscated C competition, code golf, "clever coding tricks" etc is rarely for the purposes of communication *about code*. Communication is taking place, but its about social status and cleverness. There's a very popular StackOverflow site dedicated to code golf, where you will see people have written their own custom languages specifically for writing terse code. Nobody expects these languages to be used by more than a handful of people. That's not their point. > Maintaining the traditional > spelling of keywords and operators is definitely useful for that > purpose. Okay, let's put aside the social uses of code-golfing and equivalent, and focus on quote-unquote "real code", where programmers care more about getting the job done and keeping it maintainable rather than competing with other programmers for status, jobs, avoiding being the sacrifical goat in the next round of stack-ranked layoffs, etc. You're right of course that traditional spelling is useful, but perhaps not as much as you think. After all, one person's traditional spelling is another person's confusing notation and a third person's excessively verbose spelling. Not too many people like Cobol-like spelling: add 1 to the_number over "n += 1". So I think that arguments for keeping "traditional spelling" are mostly about familiarity. If we learned lambda calculus in high school, perhaps ? would be less exotic. I think that there is a good argument to be made in favour of increasing the amount of mathematical notation used in code, but I would think that since a lot of my code is mathematical in nature. I can see that makes my code atypical. Coming back to the specific change suggested here, ? as an alternative keyword for lambda, I have a minor and major objection: The minor objection is that I think that ? is too useful a one-letter symbol to waste on a comparatively rare usage, anonymous functions. In mathematical code, I would prefer to keep ? for wavelength, or for the radioactive decay constant, rather than for anonymous functions. The major objection is that I think its still too hard to expect the average programmer to be able to produce the ? symbol on demand. We don't all have a Greek keyboard :-) I *don't* think that expecting programmers to learn ? is too difficult. It's no more difficult than the word "lambda", or that | means bitwise OR. Or for that matter, that * means multiplication. Yes, I've seen beginners stumped by that. (Sometimes we forget that * is not something you learn in maths class.) So overall, I'm a -1 on this specific proposal. [1] Non-Australians will probably recognise similar terms hoser, redneck, chav, gopnik, etc. -- Steve From pavol.lisy at gmail.com Wed Jul 13 00:50:18 2016 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Wed, 13 Jul 2016 06:50:18 +0200 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <85poqito6a.fsf@benfinney.id.au> References: <85poqito6a.fsf@benfinney.id.au> Message-ID: On 7/13/16, Ben Finney wrote: > Stephan Houben > writes: >> * The resulting code resembles more closely mathematical notation (in >> particular, lambda-calculus notation), so it brings Python closer to >> being "executable pseudo-code". > > How is that an advantage? Could help promote python. From pavol.lisy at gmail.com Wed Jul 13 01:43:10 2016 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Wed, 13 Jul 2016 07:43:10 +0200 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: Message-ID: On 7/12/16, Stephan Houben wrote: > Hi list, > > Here is my speculative language idea for Python: > > Allow the following alternative spelling of the keyword `lambda': > > ? > > (That is "Unicode Character 'GREEK SMALL LETTER LAMDA' (U+03BB).") > > Background: > > I have been using the Vim "conceal" functionality with a rule which > visually > replaces lambda with ? when editing Python files. I find this a great > improvement in > readability since ? is visually less distracting while still quite > distinctive. > (The fact that ? is syntax-colored as a keyword also helps with this.) > > However, at the moment the nice syntax is lost when looking at the file > through another editor or viewer. > Therefore I would really like this to be an official part of the Python > syntax. 1. What is future of coding? I feel it is not only language what could translate your ideas into reality. Artificial intelligence in (future) editors (and also vim conceal) is probably right way to enhance your coding power (with lambdas too). 2. If we like to enhance python syntax with Unicode characters then I think it is good to see larger context. There is ocean of possibilities how to make it. (probably good possibilities too). For example Unicode could help to add new operators. But it also brings a lot of questions (how to write Knuth's arrow on my editor?) and difficulties (how to give possibilities to implement class functions for this (*) new operators? How to make for example triple arrow possible?). I propose to be prepared before opening Pandora's box. :) (*) all of it? And it means probably all after future enhancement of Unicode too? 3. Questions around "only one possibilities how to write it" could be probably answered with this? a References: Message-ID: On Wed, Jul 13, 2016 at 3:43 PM, Pavol Lisy wrote: > 3. > Questions around "only one possibilities how to write it" could be > probably answered with this? > > a a.__lt__(b) Those aren't the same, though. One is the interface, the other is the implementation. Dunder methods are for defining, not for calling. Also: rosuav at sikorsky:~$ python3 Python 3.6.0a2+ (default:4ef2404d343e, Jul 11 2016, 12:37:20) [GCC 5.3.1 20160528] on linux Type "help", "copyright", "credits" or "license" for more information. >>> class B: ... def __gt__(self, other): ... print("Am I greater than %s?" % other) ... return False ... >>> a = 5 >>> b = B() >>> a < b Am I greater than 5? False Operators can have multiple implementations (in this case, the interpreter found that "int < B" didn't have an implementation, so it switched it to b > a and re-evaluated). ChrisA From ben+python at benfinney.id.au Wed Jul 13 01:52:13 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 13 Jul 2016 15:52:13 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= References: Message-ID: <85d1mit78y.fsf@benfinney.id.au> Pavol Lisy writes: > Questions around "only one possibilities how to write it" could be > probably answered with this? > > a a.__lt__(b) The maxim is not ?only one way?. That is a common misconception, but it is easily dispelled: read the Zen of Python (by ?import this? in the interactive prompt). Rather, the maxim is ?There should be one obvious way to do it?, with a parenthetical ?and preferably only one?. So the emphasis is on the way being *obvious*, and all other ways being non-obvious. This leads, of course, to choosing the best way to also be the one obvious way to do it. Your example above supports this: the comparison ?a < b? is the one obvious way to compare whether ?a? is less than ?b?. -- \ ?It is forbidden to steal hotel towels. Please if you are not | `\ person to do such is please not to read notice.? ?hotel, | _o__) Kowloon, Hong Kong | Ben Finney From pavol.lisy at gmail.com Wed Jul 13 02:56:55 2016 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Wed, 13 Jul 2016 08:56:55 +0200 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <85d1mit78y.fsf@benfinney.id.au> References: <85d1mit78y.fsf@benfinney.id.au> Message-ID: On 7/13/16, Ben Finney wrote: > Pavol Lisy writes: > >> Questions around "only one possibilities how to write it" could be >> probably answered with this? >> >> a> a.__lt__(b) > > The maxim is not ?only one way?. That is a common misconception, but it > is easily dispelled: read the Zen of Python (by ?import this? in the > interactive prompt). > > Rather, the maxim is ?There should be one obvious way to do it?, with a > parenthetical ?and preferably only one?. > > So the emphasis is on the way being *obvious*, and all other ways being > non-obvious. This leads, of course, to choosing the best way to also be > the one obvious way to do it. > > Your example above supports this: the comparison ?a < b? is the one > obvious way to compare whether ?a? is less than ?b?. > > -- > \ ?It is forbidden to steal hotel towels. Please if you are not | > `\ person to do such is please not to read notice.? ?hotel, | > _o__) Kowloon, Hong Kong | > Ben Finney > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ I don't support this lambda proposal (in this moment - but probably somebody could convince me). But if we will accept it then Unicode version could be the obvious one couldn't be? From matt at getpattern.com Wed Jul 13 03:30:24 2016 From: matt at getpattern.com (Matt Gilson) Date: Wed, 13 Jul 2016 00:30:24 -0700 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <85d1mit78y.fsf@benfinney.id.au> Message-ID: On Tue, Jul 12, 2016 at 11:56 PM, Pavol Lisy wrote: > On 7/13/16, Ben Finney wrote: > > Pavol Lisy writes: > > > >> Questions around "only one possibilities how to write it" could be > >> probably answered with this? > >> > >> a >> a.__lt__(b) > > > > The maxim is not ?only one way?. That is a common misconception, but it > > is easily dispelled: read the Zen of Python (by ?import this? in the > > interactive prompt). > > > > Rather, the maxim is ?There should be one obvious way to do it?, with a > > parenthetical ?and preferably only one?. > > > > So the emphasis is on the way being *obvious*, and all other ways being > > non-obvious. This leads, of course, to choosing the best way to also be > > the one obvious way to do it. > > > > Your example above supports this: the comparison ?a < b? is the one > > obvious way to compare whether ?a? is less than ?b?. > > > > -- > > \ ?It is forbidden to steal hotel towels. Please if you are not | > > `\ person to do such is please not to read notice.? ?hotel, | > > _o__) Kowloon, Hong Kong | > > Ben Finney > > > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas at python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ > > I don't support this lambda proposal (in this moment - but probably > somebody could convince me). > > I don't either, but I'm glad it was brought up regardless... > But if we will accept it then Unicode version could be the obvious one > couldn't be? > I certainly hope not. As a user with no lambda key on my keyboard, the only way that I know to input one is to do a google search for "unicode greek letter lambda" and copy/paste one of those characters into my editor :-). FWIW, I think that the cons here far outweigh the pros -- As a former physicist, when I see a lambda character, I immediately think of a whole host of things (wavelength!) and none of them is "anonymous function". Perhaps someone who works more with lambda calculus (or with a more rigorous comp-sci background) would disagree -- but my point is that this notation would possibly only serve a small community -- and it could possibly break code for another small group of users who are using lambdas as variable names to be clever (which is another practice that I wouldn't support...) and finally, I think it might just be confusing for other people (1-character non-ascii keywords? If my editor's syntax definition wasn't up-to-date, I'd definitely expect a `SyntaxError` from that). All of that aside, it seems like the pros that the original poster mentioned could be gained by writing a plugin for your editor that makes the swap on save and load. Apparently this already exists for some editors -- Why risk breaking existing code to add a syntax that can be handled by an editor extension? > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- [image: pattern-sig.png] Matt Gilson // SOFTWARE ENGINEER E: matt at getpattern.com // P: 603.892.7736 We?re looking for beta testers. Go here to sign up! -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Wed Jul 13 04:51:02 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 13 Jul 2016 09:51:02 +0100 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <20160713040025.GN27919@ando.pearwood.info> References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> Message-ID: On 13 July 2016 at 05:00, Steven D'Aprano wrote: > Not too many people like Cobol-like spelling: > > add 1 to the_number > > over "n += 1". So I think that arguments for keeping "traditional > spelling" are mostly about familiarity. If we learned lambda calculus in > high school, perhaps ? would be less exotic. It's probably also relevant in this context that more "modern" languages tend to avoid the term lambda but embrace "anonymous functions" with syntax such as (x, y) -> x+y or whatever. So while "better syntax for lambda expressions" is potentially a reasonable goal, I don't think that perpetuating the concept/name "lambda" is necessary or valuable. Paul From srkunze at mail.de Wed Jul 13 06:43:56 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Wed, 13 Jul 2016 12:43:56 +0200 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> Message-ID: <57861B6C.1010903@mail.de> On 13.07.2016 10:51, Paul Moore wrote: > On 13 July 2016 at 05:00, Steven D'Aprano wrote: >> Not too many people like Cobol-like spelling: >> >> add 1 to the_number >> >> over "n += 1". So I think that arguments for keeping "traditional >> spelling" are mostly about familiarity. If we learned lambda calculus in >> high school, perhaps ? would be less exotic. > It's probably also relevant in this context that more "modern" > languages tend to avoid the term lambda but embrace "anonymous > functions" with syntax such as > > (x, y) -> x+y > > or whatever. > > So while "better syntax for lambda expressions" is potentially a > reasonable goal, I don't think that perpetuating the concept/name > "lambda" is necessary or valuable. Exactly, there's not much value in having yet another way of writing 'lambda:'. Keeping other languages in mind and the conservative stance Python usually takes, the arrow ('=>') would be the only valid alternative for a "better syntax for lambda expressions". However, IIRC, this has been debated and won't happen. Personally, I have other associations with ?. Thus, I would rather see it as a variable name in such contexts. > > Paul > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From storchaka at gmail.com Wed Jul 13 12:40:02 2016 From: storchaka at gmail.com (Serhiy Storchaka) Date: Wed, 13 Jul 2016 19:40:02 +0300 Subject: [Python-ideas] Use unbound bytes methods with objects supporting the buffer protocol Message-ID: Unbound methods can be used as functions in python. bytes.lower(b) is the same as b.lower() if b is an instance of bytes. Many functions and methods that work with bytes accept not just bytes, but arbitrary objects that support the buffer protocol. Including bytes methods: >>> b'a:b'.split(memoryview(b':')) [b'a', b'b'] But the first argument of unbound bytes method can be only a bytes instance. >>> bytes.split(memoryview(b'a:b'), b':') Traceback (most recent call last): File "", line 1, in TypeError: descriptor 'split' requires a 'bytes' object but received a 'memoryview' I think it would be helpful to allow using unbound bytes methods with arbitrary objects that support the buffer protocol as the first argument. This would allow to avoid unneeded copying (the primary purpose of the buffer protocol). >>> bytes.split(memoryview(b'a:b'), b':') [b'a', b'b'] From mertz at gnosis.cx Wed Jul 13 01:40:55 2016 From: mertz at gnosis.cx (David Mertz) Date: Tue, 12 Jul 2016 22:40:55 -0700 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> Message-ID: I use the vim conceal plugin myself too. It's whimsical, but I like the appearance of it. So I get the sentiment of the original poster. But in my conceal configuration, I substitute a bunch of characters visually (if the attachment works, and screenshot example of some, but not all will be in this message). And honestly, having my text editor make the substitution is exactly what I want. If anyone really wanted, a very simple preprocessor (really just a few lines of sed, or a few str.replace() calls) could transform your *.py+ files into *.py by simple string substitution. There's no need to change the inherent syntax. This would not be nearly as big a change as a superset language like Coconut (which looks interesting), it's just 1-to-1 correspondence between strings. Moreover, even if special characters *were* part of Python, I'd probably want my text editor to provide me shortcuts or aliases. I have no idea how to enter the Unicode GREEK LUNATE EPSILON SYMBOL directly from my keyboard. It would be much more practical for me to type '\epsilon' (a l? LaTeX), or really just 'in', and let my editor alias/substitute that for me. Likewise, to get a GREEK SMALL LETTER LAMDA, a very nice editor shortcut would be 'lambda'. Same goes if someone make a py+-preprocessor that takes the various special symbols?I'd still want mnemonics that are more easily available on my keyboard. On Tue, Jul 12, 2016 at 8:25 PM, Chris Angelico wrote: > A fair point. But Python has a strong mathematical side (look how big > the numpy/scipy/matplotlib communities are), and we've already seen > how strongly they prefer "a @ b" to "a.matmul(b)". If there's support > for a language variant that uses more and shorter symbols, that would > be where I'd expect to find it. > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: vim-screen.jpg Type: image/jpeg Size: 71970 bytes Desc: not available URL: From stephen at xemacs.org Wed Jul 13 14:15:06 2016 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 14 Jul 2016 03:15:06 +0900 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <20160713040025.GN27919@ando.pearwood.info> References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> Message-ID: <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> Steven D'Aprano writes: > And as for programmers... the popularity of one-liners, the > obfuscated C competition, code golf, "clever coding tricks" etc is > rarely for the purposes of communication *about code*. Sure, but *Python* is popular because it's easy to communicate *with* and (usually) *about* Python code, and it does pretty well on "terse" for many algorithmic idioms. (Yes, there are other reasons -- reasonable performance, batteries included, etc. That doesn't make the design of the language *not* a reason for its popularity.) You seem to be understanding my statements to be much more general than they are. I'm only suggesting that this applies to Python as we know and love it, and to Pythonic tradition. > The major objection is that I think its still too hard to expect the > average programmer to be able to produce the ? symbol on demand. We > don't all have a Greek keyboard :-) So what? If you run Mac OS X, Windows, or X11, you do have a keyboard capable of producing Greek. And the same chords work in any Unicode- capable editor, it's just that the Greek letters aren't printed on the keycaps. Neither are emoticons, nor the CUA gestures (bucky-X[1], bucky-C, bucky-V, and the oh-so-useful bucky-Z) but those are everywhere. Any 10-year-old can find them somehow! To the extent that Python would consider such changes (ie, a half-dozen or so one-character replacements for multicharacter operators or keywords), it would be very nearly as learnable to type them as to read them. The problem (if it exists, of course -- obviously, I believe it does but YMMV) is all about overloading people's ability to perceive the meaning of code without reading it token by token. Footnotes: [1] Bucky = Control, Alt, Meta, Command, Option, Windows, etc. keys. From p.f.moore at gmail.com Wed Jul 13 15:20:33 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 13 Jul 2016 20:20:33 +0100 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> Message-ID: On 13 July 2016 at 19:15, Stephen J. Turnbull wrote: > So what? If you run Mac OS X, Windows, or X11, you do have a keyboard > capable of producing Greek. And the same chords work in any Unicode- > capable editor, it's just that the Greek letters aren't printed on the > keycaps. Neither are emoticons, nor the CUA gestures (bucky-X[1], > bucky-C, bucky-V, and the oh-so-useful bucky-Z) but those are > everywhere. Any 10-year-old can find them somehow! Um, as someone significantly older than 10 years old, I don't know how to type a lambda character on my Windows UK keyboard... Note that memorising the Unicode code point, and doing the weird numpad "enter a numeric character code" trick that I can never remember how to do, doesn't count as a realistic option... If it were that easy to type arbitrary characters, why would Vim have the "digraph" facility to insert Unicode characters using 2-character abbreviations? Paul From alexander.belopolsky at gmail.com Wed Jul 13 16:42:01 2016 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 13 Jul 2016 15:42:01 -0500 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> Message-ID: On Wed, Jul 13, 2016 at 2:20 PM, Paul Moore wrote: > > Um, as someone significantly older than 10 years old, I don't know how > to type a lambda character on my Windows UK keyboard... FWIW, in IPython/Jupyter notebooks one can type \lambda followed by a tab to get the ? character. The difficulty of typing is a red herring. Once it is a part of the language, every editor targeted at a Python programmer will provide the means to type ? with fewer than 6 keystrokes (6 is the number of keystrokes needed to type "lambda" without autocompletion.) The unfamiliarity is also not an issue. I am yet to meet a Python programer who knows what the keyword "lambda" is but does not know how the namesake Greek character looks. I am +0 on this feature. I would be +1 if ? was not already a valid identifier. This is one of those features that can easily be ignored by someone who does not need it, but can significantly improve the experience of those who do. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.cristh at gmail.com Wed Jul 13 16:56:41 2016 From: alan.cristh at gmail.com (Alan Cristhian) Date: Wed, 13 Jul 2016 17:56:41 -0300 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> Message-ID: Another identifier could be "/\" that looks like the uppercase lambda. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Wed Jul 13 17:09:14 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 13 Jul 2016 17:09:14 -0400 Subject: [Python-ideas] Use unbound bytes methods with objects supporting the buffer protocol In-Reply-To: References: Message-ID: On 7/13/2016 12:40 PM, Serhiy Storchaka wrote: > Unbound methods can be used as functions in python. According to my naive understanding, in Python 3, there are not supposed to be 'unbound methods'. Functions accessed as a class attribute are supposed to *be* functions, and not just usable as a function. This is true, at least for Python-coded classes. class C(): def f(self, other): return self + other print(C.f) # print(C.f(1,2)) # 3 print(C.f('a', 'b')) # 'ab' This works because the Python code is generic and being accessed as a class attribute does not impose additional restrictions on inputs beyond those inherent in the code itself. >>> C.f('a', 2) Traceback (most recent call last): File "", line 1, in C.f('a', 2) File "", line 3, in f return self + other TypeError: must be str, not int >>> 'a'+2 Traceback (most recent call last): File "", line 1, in 'a'+2 TypeError: must be str, not int If the situation is different for C-coded functions and classes, them it would seem impossible to write a drop-in replacement for Python-coded classes. > bytes.lower(b) is > the same as b.lower() if b is an instance of bytes. Many functions and > methods that work with bytes accept not just bytes, but arbitrary > objects that support the buffer protocol. Including bytes methods: > > >>> b'a:b'.split(memoryview(b':')) > [b'a', b'b'] > > But the first argument of unbound bytes method can be only a bytes > instance. > > >>> bytes.split(memoryview(b'a:b'), b':') > Traceback (most recent call last): > File "", line 1, in > TypeError: descriptor 'split' requires a 'bytes' object but received > a 'memoryview' My naive expectation was that bytes.split should be a built-in function, like open -- but it is not >>> open >>> bytes.split and that the C coded split function would type check both args for being bytes-like, as it does with the second. >>> b'a:b'.split(':') Traceback (most recent call last): File "", line 1, in b'a:b'.split(':') TypeError: a bytes-like object is required, not 'str' Assuming that the descriptor check is not just an unintened holdover from 2.x, it seems that for C-coded functions used as methods, type-checking the first arg was conceptually factored out and replaced by a generic check in the descriptor mechanism. In this case, the descriptor check is stricter that you would like. Is it stricter than necessary? If the memoryview were passed to the code for bytes.check, would the code successfully run to conclusion? Is it sufficiently generic at the machine bytes level? > I think it would be helpful to allow using unbound bytes methods with > arbitrary objects that support the buffer protocol as the first > argument. This would allow to avoid unneeded copying (the primary > purpose of the buffer protocol). > > >>> bytes.split(memoryview(b'a:b'), b':') > [b'a', b'b'] If the descriptor check cannot be selectively loosened, a possible solution might be a base class for all bytes-like buffer protocol classes that would have all method functions that work with all bytes-like objects. -- Terry Jan Reedy From gokoproject at gmail.com Wed Jul 13 17:12:49 2016 From: gokoproject at gmail.com (John Wong) Date: Wed, 13 Jul 2016 17:12:49 -0400 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> Message-ID: On Wed, Jul 13, 2016 at 4:42 PM, Alexander Belopolsky < alexander.belopolsky at gmail.com> wrote: > > On Wed, Jul 13, 2016 at 2:20 PM, Paul Moore wrote: > >> >> Um, as someone significantly older than 10 years old, I don't know how >> to type a lambda character on my Windows UK keyboard... > > > FWIW, in IPython/Jupyter notebooks one can type \lambda followed by a tab > to get the ? character. The difficulty of typing is a red herring. Once > it is a part of the language, every editor targeted at a Python programmer > will provide the means to type ? with fewer than 6 keystrokes (6 is the > number of keystrokes needed to type "lambda" without autocompletion.) The > unfamiliarity is also not an issue. I am yet to meet a Python programer > who knows what the keyword "lambda" is but does not know how the namesake > Greek character looks. > > I am +0 on this feature. I would be +1 if ? was not already a valid > identifier. > > This is one of those features that can easily be ignored by someone who > does not need it, but can significantly improve the experience of those who > do. > > I -1 on this feature. Sorry to be blunt. Are we going to add omega, delta, psilon and the entire Greek alphabet? There should be one and only one way to write code in Python as far as a valid identifier is concerned. Is there an existing exception? I am not saying the experiences of others do not matter, but we should take a step back and see does this actually make sense? Also, how do you exactly enter this character for someone who doesn't really enter unicode character except for the ASCII alphanumeric characters on a daily basis? How many users can we retain and even convert with this approach? Is it really worth the go? Thanks. John On Wed, Jul 13, 2016 at 4:42 PM, Alexander Belopolsky < alexander.belopolsky at gmail.com> wrote: > > On Wed, Jul 13, 2016 at 2:20 PM, Paul Moore wrote: > >> >> Um, as someone significantly older than 10 years old, I don't know how >> to type a lambda character on my Windows UK keyboard... > > > FWIW, in IPython/Jupyter notebooks one can type \lambda followed by a tab > to get the ? character. The difficulty of typing is a red herring. Once > it is a part of the language, every editor targeted at a Python programmer > will provide the means to type ? with fewer than 6 keystrokes (6 is the > number of keystrokes needed to type "lambda" without autocompletion.) The > unfamiliarity is also not an issue. I am yet to meet a Python programer > who knows what the keyword "lambda" is but does not know how the namesake > Greek character looks. > > I am +0 on this feature. I would be +1 if ? was not already a valid > identifier. > > This is one of those features that can easily be ignored by someone who > does not need it, but can significantly improve the experience of those who > do. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.belopolsky at gmail.com Wed Jul 13 21:28:28 2016 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 13 Jul 2016 20:28:28 -0500 Subject: [Python-ideas] =?gb2312?b?YWxsb3cgYGxhbWJkYScgdG8gYmUgc3BlbGxl?= =?gb2312?b?ZCCmyw==?= In-Reply-To: References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> Message-ID: <64A5A9A6-414A-43B1-9011-1DC81B701081@gmail.com> > On Jul 13, 2016, at 4:12 PM, John Wong wrote: > > Sorry to be blunt. Are we going to add omega, delta, psilon and the entire Greek alphabet? Breaking news: the entire Greek alphabet is already available for use in Python. If someone wants to write code that looks like a series of missing character boxes on your screen she already can. PS: There is no letter called "psilon" in the Greek alphabet or anywhere else in the Unicode. From ethan at stoneleaf.us Wed Jul 13 21:57:37 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 13 Jul 2016 18:57:37 -0700 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <64A5A9A6-414A-43B1-9011-1DC81B701081@gmail.com> References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <64A5A9A6-414A-43B1-9011-1DC81B701081@gmail.com> Message-ID: <5786F191.7040805@stoneleaf.us> On 07/13/2016 06:28 PM, Alexander Belopolsky wrote: >> On Jul 13, 2016, at 4:12 PM, John Wong wrote: >> Sorry to be blunt. Are we going to add omega, delta, psilon and the entire Greek alphabet? > > Breaking news: the entire Greek alphabet is already available for use in Python. Indeed. We can all use any letters in unicode for our identifiers. Does Python currently use any one-letter keywords? So why start now? -- ~Ethan~ From steve at pearwood.info Wed Jul 13 22:44:15 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 14 Jul 2016 12:44:15 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> Message-ID: <20160714024414.GP27919@ando.pearwood.info> On Wed, Jul 13, 2016 at 03:42:01PM -0500, Alexander Belopolsky wrote: > On Wed, Jul 13, 2016 at 2:20 PM, Paul Moore wrote: > > > > > Um, as someone significantly older than 10 years old, I don't know how > > to type a lambda character on my Windows UK keyboard... > > > FWIW, in IPython/Jupyter notebooks one can type \lambda followed by a tab > to get the ? character. The difficulty of typing is a red herring. Once > it is a part of the language, every editor targeted at a Python programmer > will provide the means to type ? Great for those using an editor targeted at Python programmers, but most editors are more general than that. Which means that programmers will find themselves split into two camps: those who can easily type ?, and those that cannot. In the 1980s and 90s, I was a Macintosh user, and one nice feature of the Macs at the time was the ease of typing non-ASCII characters. (Of course there were a lot fewer back then: MacRoman is an 8-bit extention to ASCII, compared to Unicode with its thousands of code points.) Consequently I've used an Apple-specific language that included operators like ? ? ? and it is *really nice*. But Apple has the advantage of controlling the entire platform and they could ensure that these characters could be input from any application on any machine using exactly the same key sequence. (By memory, it was option-= to get ?.) We don't have that advantage, and frankly I think you are underestimating the *practical* difficulties for input. I recently discovered (by accident!) the Linux compose key. So now I know how to enter ? at the keyboard: COMPOSE mu does the job. So maybe COMPOSE lambda works? Nope. How about COMPOSE l or shift-l or ll or la or yy (its an upside down y, right, and COMPOSE ee gives ?)? No, none of these things work on my system. They may work on your system: since discovering COMPOSE, I keep coming across people who state "oh, its easy to type such-and-such a character, just type COMPOSE key-sequence, its standard and will work on EVERY LINUX SYSTEM EVERYWHERE". Not a chance. The key bindings for COMPOSE are anything but standard. And COMPOSE is *really* hard to use well: it gives no feedback if you make a mistake except to silently ignore your keypresses (or insert the wrong character). So invariably, every time I want to enter a non-ASCII character, it takes me out of "thinking about code" into "thinking about how to enter characters", sometimes for minutes at a time as I hunt for the character in "Character Map" or google for it on the Internet. It may be reasonable to argue that code is read more than it is written: - suppose that reading ? has a *tiny* benefit of 1% over "lambda" (for those who have learned what it means); - but typing it is (lets say) 50 times harder than typing "lambda"; - but we read code 50 times as often as we type it; - so the total benefit (50*1.01 - 50) is positive. Invent your own numbers, and you'll come up with your own results. I don't think there's any *objective* way to decide this question. And that's why I don't think that Python should take this step: let other languages experiment with non-ASCII keywords first, or let people experiment with translators that transform ? into != and ? into lambda. -- Steve From alexander.belopolsky at gmail.com Wed Jul 13 23:17:15 2016 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 13 Jul 2016 22:17:15 -0500 Subject: [Python-ideas] =?gb2312?b?YWxsb3cgYGxhbWJkYScgdG8gYmUgc3BlbGxl?= =?gb2312?b?ZCCmyw==?= In-Reply-To: <5786F191.7040805@stoneleaf.us> References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <64A5A9A6-414A-43B1-9011-1DC81B701081@gmail.com> <5786F191.7040805@stoneleaf.us> Message-ID: > On Jul 13, 2016, at 8:57 PM, Ethan Furman wrote: > > Does Python currently use any one-letter keywords? No. > So why start now? There is no slippery slope or a hidden agenda in this proposal. The lambda keyword is unique in the Python language. It is the only keyword that does not have an obvious meaning as an English word or an abbreviation of such. There is no other keyword that literally is a name of a letter. (Luckily, iota and rho are spelled range and len in Python and those are not keywords anyways.) Arguably, it would be more Pythonic to spell an anonymous function creation keyword as "function" or "fun" ("func" does not smell right), but Python ended up with an English name for a Greek letter. That was before Unicode became ubiquitous. I don't see much harm in allowing a proper spelling for it now. From alexander.belopolsky at gmail.com Wed Jul 13 23:31:52 2016 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 13 Jul 2016 22:31:52 -0500 Subject: [Python-ideas] =?gb2312?b?YWxsb3cgYGxhbWJkYScgdG8gYmUgc3BlbGxl?= =?gb2312?b?ZCCmyw==?= In-Reply-To: <20160714024414.GP27919@ando.pearwood.info> References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <20160714024414.GP27919@ando.pearwood.info> Message-ID: <8D975331-81E4-479D-B316-37310A2CDBDC@gmail.com> > On Jul 13, 2016, at 9:44 PM, Steven D'Aprano wrote: > > Which means that programmers will > find themselves split into two camps: those who can easily type ?, and > those that cannot. We already have two camps: those who don't mind using "lambda" and those who would only use "def." I would expect that those who will benefit most are people who routinely write expressions that involve a lambda that returns a lambda that returns a lambda. There is a niche for such programming style and using ? instead of lambda will improve the readability of such programs for those who can understand them in the current form. For the "def" camp, the possibility of a non-ascii spelling will serve as yet another argument to avoid using anonymous functions. From alexander.belopolsky at gmail.com Thu Jul 14 00:01:15 2016 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 13 Jul 2016 23:01:15 -0500 Subject: [Python-ideas] =?gb2312?b?YWxsb3cgYGxhbWJkYScgdG8gYmUgc3BlbGxl?= =?gb2312?b?ZCCmyw==?= In-Reply-To: <20160714024414.GP27919@ando.pearwood.info> References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <20160714024414.GP27919@ando.pearwood.info> Message-ID: > On Jul 13, 2016, at 9:44 PM, Steven D'Aprano wrote: > > I think > you are underestimating the *practical* difficulties for input. I appreciate those difficulties (I am typing this on an iPhone), but I think they are irrelevant. I can imagine 3 scenarios: 1. (The 99% case) You will never see ? in the code and never write it yourself. You can be happily unaware of this feature. 2. You see ? occasionally, but don't like it. You continue using spelled out "lambda" (or just use "def") in the code that you write. 3. You work on a project where local coding style mandates that lambda is spelled ?. In this case, there will be plenty of places in the code base to copy and paste ? from. (In the worst case you copy and paste it from the coding style manual.) More likely, however, the project that requires ? would have a precommit hook that translates lambda to ? in all new code and you can continue using the 6-character keyword in your input. From ben+python at benfinney.id.au Thu Jul 14 00:20:19 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 14 Jul 2016 14:20:19 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <20160714024414.GP27919@ando.pearwood.info> <8D975331-81E4-479D-B316-37310A2CDBDC@gmail.com> Message-ID: <858tx4u9z0.fsf@benfinney.id.au> Alexander Belopolsky writes: > We already have two camps: those who don't mind using "lambda" and > those who would only use "def." I don't know anyone in the latter camp, do you? I am in the camp that loves ?lambda? for some narrowly-specified purposes *and* thinks ?def? is generally a better tool. -- \ ?? correct code is great, code that crashes could use | `\ improvement, but incorrect code that doesn?t crash is a | _o__) horrible nightmare.? ?Chris Smith, 2008-08-22 | Ben Finney From jmcs at jsantos.eu Thu Jul 14 02:13:30 2016 From: jmcs at jsantos.eu (=?UTF-8?B?Sm/Do28gU2FudG9z?=) Date: Thu, 14 Jul 2016 08:13:30 +0200 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <20160714024414.GP27919@ando.pearwood.info> Message-ID: > > > 3. You work on a project where local coding style mandates that lambda is > spelled ?. In this case, there will be plenty of places in the code base to* > copy and paste ? from*. (In the worst case you copy and paste it from the > coding style manual.) More likely, however, the project that requires ? > would have a precommit hook that translates lambda to ? in all new code and > you can continue using the 6-character keyword in your input. > > That this your solution makes the proposoal a -100 and if you're going to implement a precommit hook to convert lambda to ? you might has well implement a custom coding that supports ? (like pyxl does for html\xml tags). Python isn't and shouldn't be APL. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mertz at gnosis.cx Thu Jul 14 02:39:26 2016 From: mertz at gnosis.cx (David Mertz) Date: Wed, 13 Jul 2016 23:39:26 -0700 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <20160714024414.GP27919@ando.pearwood.info> References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <20160714024414.GP27919@ando.pearwood.info> Message-ID: On Wed, Jul 13, 2016 at 7:44 PM, Steven D'Aprano wrote: > - suppose that reading ? has a *tiny* benefit of 1% over "lambda" > (for those who have learned what it means); > - but typing it is (lets say) 50 times harder than typing "lambda"; > - but we read code 50 times as often as we type it; > - so the total benefit (50*1.01 - 50) is positive. I actually *do* think ? is a little bit more readable. And I have no idea how to type it directly on my El Capitan system with the ABC Extended keyboard. But I still get 100% of the benefit in readability simply by using vim's conceal feature. If I used a different editor I'd have to hope for a similar feature (or program it myself), but this is purely a display question. Similarly, I think syntax highlighting makes my code much more readable, but I don't want colors for keywords built into the language. That is, and should remain, a matter of tooling not core language (I don't want https://en.wikipedia.org/wiki/ColorForth for Python). FWIW, my conceal configuration is at link I give in a moment. I've customized a bunch of special stuff besides lambda, take it or leave it: http://gnosis.cx/bin/.vim/after/syntax/python.vim -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Thu Jul 14 04:52:19 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 14 Jul 2016 18:52:19 +1000 Subject: [Python-ideas] Use unbound bytes methods with objects supporting the buffer protocol In-Reply-To: References: Message-ID: On 14 July 2016 at 07:09, Terry Reedy wrote: > Assuming that the descriptor check is not just an unintened holdover from > 2.x, it seems that for C-coded functions used as methods, type-checking the > first arg was conceptually factored out and replaced by a generic check in > the descriptor mechanism. It's intentional - the default C level descriptors typecheck their first argument, since getting that wrong may cause a segfault in most cases. > If the descriptor check cannot be selectively loosened, a possible solution > might be a base class for all bytes-like buffer protocol classes that would > have all method functions that work with all bytes-like objects. A custom wrapper descriptor that checks for "supports the buffer protocol" rather than "is a bytes-like object" is certainly possible, so I believe Serhiy's question here is more a design question around "Should they?" than it is a technical question around "Can they?". Given the way this would behave if "bytes" was implemented in Python rather than C (i.e. unbound methods would rely on ducktyping, even for the first argument), +1 from me for making the unbound methods for bytes compatible with arbitrary objects supporting the buffer protocol. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From random832 at fastmail.com Thu Jul 14 09:10:44 2016 From: random832 at fastmail.com (Random832) Date: Thu, 14 Jul 2016 09:10:44 -0400 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <1468330945.481991.663978161.09323860@webmail.messagingengine.com> References: <1468330945.481991.663978161.09323860@webmail.messagingengine.com> Message-ID: <1468501844.2375597.666170897.1B915C53@webmail.messagingengine.com> On Tue, Jul 12, 2016, at 09:42, Random832 wrote: > On Tue, Jul 12, 2016, at 08:38, Stephan Houben wrote: > > I know people have been clamoring for shorter lambda-syntax in the > > past, I think this is a nice minimal extension. > > How about a haskell-style backslash? Nobody has any thoughts at all? From walker_s at hotmail.co.uk Thu Jul 14 09:22:06 2016 From: walker_s at hotmail.co.uk (SW) Date: Thu, 14 Jul 2016 14:22:06 +0100 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <1468501844.2375597.666170897.1B915C53@webmail.messagingengine.com> References: <1468330945.481991.663978161.09323860@webmail.messagingengine.com> <1468501844.2375597.666170897.1B915C53@webmail.messagingengine.com> Message-ID: To me the backslash already has a fairly strong association with "the next character is a literal". Overloading it would feel very strange. S On 14/07/16 14:10, Random832 wrote: > On Tue, Jul 12, 2016, at 09:42, Random832 wrote: >> On Tue, Jul 12, 2016, at 08:38, Stephan Houben wrote: >>> I know people have been clamoring for shorter lambda-syntax in the >>> past, I think this is a nice minimal extension. >> How about a haskell-style backslash? > Nobody has any thoughts at all? > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > From rosuav at gmail.com Thu Jul 14 09:27:35 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 14 Jul 2016 23:27:35 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <1468330945.481991.663978161.09323860@webmail.messagingengine.com> <1468501844.2375597.666170897.1B915C53@webmail.messagingengine.com> Message-ID: On Thu, Jul 14, 2016 at 11:22 PM, SW wrote: > To me the backslash already has a fairly strong association with "the > next character is a literal". Overloading it would feel very strange. But it also has the meaning of "the next character is special", such as \n for newline or \uNNNN for a Unicode escape. However, I suspect there might be a parsing conflict: do_stuff(stuff_with_long_name, more_stuff, what_is_next_arg, \ At that point in the parsing, are you looking at a lambda function or a line continuation? Sure, style guides would decry this (put the backslash with its function, dummy!), but the parser can't depend on style guides being followed. -1 on using backslash for this. -0 on ?. ChrisA From walker_s at hotmail.co.uk Thu Jul 14 09:49:04 2016 From: walker_s at hotmail.co.uk (SW) Date: Thu, 14 Jul 2016 14:49:04 +0100 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <1468330945.481991.663978161.09323860@webmail.messagingengine.com> <1468501844.2375597.666170897.1B915C53@webmail.messagingengine.com> Message-ID: Ah, yes, sorry- it certainly holds that meaning to me as well. I agree with your stated views on this (and ratings): -1 on using backslash for this. -0 on ?. Thanks, S On 14/07/16 14:27, Chris Angelico wrote: > On Thu, Jul 14, 2016 at 11:22 PM, SW wrote: >> To me the backslash already has a fairly strong association with "the >> next character is a literal". Overloading it would feel very strange. > But it also has the meaning of "the next character is special", such > as \n for newline or \uNNNN for a Unicode escape. However, I suspect > there might be a parsing conflict: > > do_stuff(stuff_with_long_name, more_stuff, what_is_next_arg, \ > > At that point in the parsing, are you looking at a lambda function or > a line continuation? Sure, style guides would decry this (put the > backslash with its function, dummy!), but the parser can't depend on > style guides being followed. > > -1 on using backslash for this. > -0 on ?. > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From cody.piersall at gmail.com Thu Jul 14 09:53:51 2016 From: cody.piersall at gmail.com (Cody Piersall) Date: Thu, 14 Jul 2016 08:53:51 -0500 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: Message-ID: On Tue, Jul 12, 2016 at 7:38 AM, Stephan Houben wrote: > Hi list, > > Here is my speculative language idea for Python: > > Allow the following alternative spelling of the keyword `lambda': > > ? > > (That is "Unicode Character 'GREEK SMALL LETTER LAMDA' (U+03BB).") Just to be a small data point, I have written code that uses ? as a variable name (as someone mentioned elsewhere in the thread, Jupyter Notebook makes typing Greek characters easy). Because this would break code that I have written, and I suspect it would break other code as well, I am -1 on the proposal. How selfish of me! Cody From elazarg at gmail.com Thu Jul 14 10:19:35 2016 From: elazarg at gmail.com (=?UTF-8?B?15DXnNei15bXqA==?=) Date: Thu, 14 Jul 2016 14:19:35 +0000 Subject: [Python-ideas] allow `lambda' to be spelled ? In-Reply-To: References: Message-ID: `def` is a much more readable alternative, which will be more familiar to newcomers. The expression/block ambiguity can be resolved using different token instead of colon (perhaps := or an arrow). I can't understand the reaching for the Greek letter - it is a computer-science specific term with no intuitive meaning. On Thu, Jul 14, 2016 at 4:49 PM wrote: > Send Python-ideas mailing list submissions to > python-ideas at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/python-ideas > or, via email, send a message with subject or body 'help' to > python-ideas-request at python.org > > You can reach the person managing the list at > python-ideas-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-ideas digest..." > > > Today's Topics: > > 1. Re: allow `lambda' to be spelled ? (David Mertz) > 2. Re: Use unbound bytes methods with objects supporting the > buffer protocol (Nick Coghlan) > 3. Re: allow `lambda' to be spelled ? (Random832) > 4. Re: allow `lambda' to be spelled ? (SW) > 5. Re: allow `lambda' to be spelled ? (Chris Angelico) > 6. Re: allow `lambda' to be spelled ? (SW) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 13 Jul 2016 23:39:26 -0700 > From: David Mertz > To: "Steven D'Aprano" > Cc: python-ideas > Subject: Re: [Python-ideas] allow `lambda' to be spelled ? > Message-ID: > < > CAEbHw4aVKCXLu2DC7K6ArxVHAjOkc15yk+irtPKfBa5xEy0G2A at mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > On Wed, Jul 13, 2016 at 7:44 PM, Steven D'Aprano > wrote: > > > - suppose that reading ? has a *tiny* benefit of 1% over "lambda" > > (for those who have learned what it means); > > - but typing it is (lets say) 50 times harder than typing "lambda"; > > - but we read code 50 times as often as we type it; > > - so the total benefit (50*1.01 - 50) is positive. > > > I actually *do* think ? is a little bit more readable. And I have no idea > how to type it directly on my El Capitan system with the ABC Extended > keyboard. But I still get 100% of the benefit in readability simply by > using vim's conceal feature. If I used a different editor I'd have to hope > for a similar feature (or program it myself), but this is purely a display > question. Similarly, I think syntax highlighting makes my code much more > readable, but I don't want colors for keywords built into the language. > That is, and should remain, a matter of tooling not core language (I don't > want https://en.wikipedia.org/wiki/ColorForth for Python). > > FWIW, my conceal configuration is at link I give in a moment. I've > customized a bunch of special stuff besides lambda, take it or leave it: > > http://gnosis.cx/bin/.vim/after/syntax/python.vim > > -- > Keeping medicines from the bloodstreams of the sick; food > from the bellies of the hungry; books from the hands of the > uneducated; technology from the underdeveloped; and putting > advocates of freedom in prisons. Intellectual property is > to the 21st century what the slave trade was to the 16th. > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://mail.python.org/pipermail/python-ideas/attachments/20160713/8fafff3f/attachment-0001.html > > > > ------------------------------ > > Message: 2 > Date: Thu, 14 Jul 2016 18:52:19 +1000 > From: Nick Coghlan > To: Terry Reedy > Cc: "python-ideas at python.org" > Subject: Re: [Python-ideas] Use unbound bytes methods with objects > supporting the buffer protocol > Message-ID: > H_DGLZAp2LhgWiWwj97N7Wt31eEEw at mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > On 14 July 2016 at 07:09, Terry Reedy wrote: > > Assuming that the descriptor check is not just an unintened holdover from > > 2.x, it seems that for C-coded functions used as methods, type-checking > the > > first arg was conceptually factored out and replaced by a generic check > in > > the descriptor mechanism. > > It's intentional - the default C level descriptors typecheck their > first argument, since getting that wrong may cause a segfault in most > cases. > > > If the descriptor check cannot be selectively loosened, a possible > solution > > might be a base class for all bytes-like buffer protocol classes that > would > > have all method functions that work with all bytes-like objects. > > A custom wrapper descriptor that checks for "supports the buffer > protocol" rather than "is a bytes-like object" is certainly possible, > so I believe Serhiy's question here is more a design question around > "Should they?" than it is a technical question around "Can they?". > > Given the way this would behave if "bytes" was implemented in Python > rather than C (i.e. unbound methods would rely on ducktyping, even for > the first argument), +1 from me for making the unbound methods for > bytes compatible with arbitrary objects supporting the buffer > protocol. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > > > ------------------------------ > > Message: 3 > Date: Thu, 14 Jul 2016 09:10:44 -0400 > From: Random832 > To: python-ideas at python.org > Subject: Re: [Python-ideas] allow `lambda' to be spelled ? > Message-ID: > <1468501844.2375597.666170897.1B915C53 at webmail.messagingengine.com > > > Content-Type: text/plain > > On Tue, Jul 12, 2016, at 09:42, Random832 wrote: > > On Tue, Jul 12, 2016, at 08:38, Stephan Houben wrote: > > > I know people have been clamoring for shorter lambda-syntax in the > > > past, I think this is a nice minimal extension. > > > > How about a haskell-style backslash? > > Nobody has any thoughts at all? > > > ------------------------------ > > Message: 4 > Date: Thu, 14 Jul 2016 14:22:06 +0100 > From: SW > To: python-ideas at python.org > Subject: Re: [Python-ideas] allow `lambda' to be spelled ? > Message-ID: > Content-Type: text/plain; charset="windows-1252" > > To me the backslash already has a fairly strong association with "the > next character is a literal". Overloading it would feel very strange. > > S > > On 14/07/16 14:10, Random832 wrote: > > On Tue, Jul 12, 2016, at 09:42, Random832 wrote: > >> On Tue, Jul 12, 2016, at 08:38, Stephan Houben wrote: > >>> I know people have been clamoring for shorter lambda-syntax in the > >>> past, I think this is a nice minimal extension. > >> How about a haskell-style backslash? > > Nobody has any thoughts at all? > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas at python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ > > > > > > > > ------------------------------ > > Message: 5 > Date: Thu, 14 Jul 2016 23:27:35 +1000 > From: Chris Angelico > Cc: python-ideas > Subject: Re: [Python-ideas] allow `lambda' to be spelled ? > Message-ID: > < > CAPTjJmqn6RK7FuW9s2+K_t2riYSi3Rpigx5CdUOV6228PV5VTQ at mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > On Thu, Jul 14, 2016 at 11:22 PM, SW wrote: > > To me the backslash already has a fairly strong association with "the > > next character is a literal". Overloading it would feel very strange. > > But it also has the meaning of "the next character is special", such > as \n for newline or \uNNNN for a Unicode escape. However, I suspect > there might be a parsing conflict: > > do_stuff(stuff_with_long_name, more_stuff, what_is_next_arg, \ > > At that point in the parsing, are you looking at a lambda function or > a line continuation? Sure, style guides would decry this (put the > backslash with its function, dummy!), but the parser can't depend on > style guides being followed. > > -1 on using backslash for this. > -0 on ?. > > ChrisA > > > ------------------------------ > > Message: 6 > Date: Thu, 14 Jul 2016 14:49:04 +0100 > From: SW > To: python-ideas at python.org > Subject: Re: [Python-ideas] allow `lambda' to be spelled ? > Message-ID: > Content-Type: text/plain; charset="utf-8" > > Ah, yes, sorry- it certainly holds that meaning to me as well. I agree > with your stated views on this (and ratings): > > -1 on using backslash for this. > -0 on ?. > > Thanks, > S > > On 14/07/16 14:27, Chris Angelico wrote: > > On Thu, Jul 14, 2016 at 11:22 PM, SW wrote: > >> To me the backslash already has a fairly strong association with "the > >> next character is a literal". Overloading it would feel very strange. > > But it also has the meaning of "the next character is special", such > > as \n for newline or \uNNNN for a Unicode escape. However, I suspect > > there might be a parsing conflict: > > > > do_stuff(stuff_with_long_name, more_stuff, what_is_next_arg, \ > > > > At that point in the parsing, are you looking at a lambda function or > > a line continuation? Sure, style guides would decry this (put the > > backslash with its function, dummy!), but the parser can't depend on > > style guides being followed. > > > > -1 on using backslash for this. > > -0 on ?. > > > > ChrisA > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas at python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > > > ------------------------------ > > End of Python-ideas Digest, Vol 116, Issue 28 > ********************************************* > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eryksun at gmail.com Thu Jul 14 12:31:40 2016 From: eryksun at gmail.com (eryk sun) Date: Thu, 14 Jul 2016 16:31:40 +0000 Subject: [Python-ideas] Use unbound bytes methods with objects supporting the buffer protocol In-Reply-To: References: Message-ID: On Thu, Jul 14, 2016 at 8:52 AM, Nick Coghlan wrote: > Given the way this would behave if "bytes" was implemented in Python > rather than C (i.e. unbound methods would rely on ducktyping, even for > the first argument), +1 from me for making the unbound methods for > bytes compatible with arbitrary objects supporting the buffer > protocol. The buffer protocol is a bit generic for duck typing. Instead the bytes methods could check for a memoryview with a format that's "B" or "b". >>> a = np.array([1,2,3,4], dtype='int16') >>> b = np.array([1,2,3,4], dtype='uint8') >>> memoryview(a).format 'h' >>> memoryview(b).format 'B' It's possible to cast if necessary, e.g. memoryview(a).cast('B'). No copy of the data is made, so it's still reasonably efficient. This preserves raising a TypeError for operations that are generally nonsensical, such as attempting to split() an array of short integers as if it's just bytes. From storchaka at gmail.com Thu Jul 14 14:14:41 2016 From: storchaka at gmail.com (Serhiy Storchaka) Date: Thu, 14 Jul 2016 21:14:41 +0300 Subject: [Python-ideas] Use unbound bytes methods with objects supporting the buffer protocol In-Reply-To: References: Message-ID: On 14.07.16 19:31, eryk sun wrote: > On Thu, Jul 14, 2016 at 8:52 AM, Nick Coghlan wrote: >> Given the way this would behave if "bytes" was implemented in Python >> rather than C (i.e. unbound methods would rely on ducktyping, even for >> the first argument), +1 from me for making the unbound methods for >> bytes compatible with arbitrary objects supporting the buffer >> protocol. > > The buffer protocol is a bit generic for duck typing. Instead the > bytes methods could check for a memoryview with a format that's "B" or > "b". > > >>> a = np.array([1,2,3,4], dtype='int16') > >>> b = np.array([1,2,3,4], dtype='uint8') > >>> memoryview(a).format > 'h' > >>> memoryview(b).format > 'B' > > It's possible to cast if necessary, e.g. memoryview(a).cast('B'). No > copy of the data is made, so it's still reasonably efficient. This > preserves raising a TypeError for operations that are generally > nonsensical, such as attempting to split() an array of short integers > as if it's just bytes. This looks reasonable. But for now bytes methods accept arbitrary buffers. >>> a = np.array([1,2,3,4], dtype='int16') >>> b = np.array([1,2,3,4], dtype='uint8') >>> b'.'.join([a, b]) b'\x01\x00\x02\x00\x03\x00\x04\x00.\x01\x02\x03\x04' From srkunze at mail.de Thu Jul 14 16:19:23 2016 From: srkunze at mail.de (Sven R. Kunze) Date: Thu, 14 Jul 2016 22:19:23 +0200 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <20160714024414.GP27919@ando.pearwood.info> Message-ID: <5787F3CB.1080304@mail.de> On 14.07.2016 08:39, David Mertz wrote: > On Wed, Jul 13, 2016 at 7:44 PM, Steven D'Aprano > wrote: > > - suppose that reading ? has a *tiny* benefit of 1% over "lambda" > (for those who have learned what it means); > - but typing it is (lets say) 50 times harder than typing "lambda"; > - but we read code 50 times as often as we type it; > - so the total benefit (50*1.01 - 50) is positive. > > > I actually *do* think ? is a little bit more readable. And I have no > idea how to type it directly on my El Capitan system with the ABC > Extended keyboard. But I still get 100% of the benefit in readability > simply by using vim's conceal feature. If I used a different editor > I'd have to hope for a similar feature (or program it myself), but > this is purely a display question. Similarly, I think syntax > highlighting makes my code much more readable, but I don't want colors > for keywords built into the language. That is, and should remain, a > matter of tooling not core language (I don't want > https://en.wikipedia.org/wiki/ColorForth for Python). Very good point. That now is basically the core argument against it at least for me. So, -100 on the proposal from me. :) > > FWIW, my conceal configuration is at link I give in a moment. I've > customized a bunch of special stuff besides lambda, take it or leave it: > > http://gnosis.cx/bin/.vim/after/syntax/python.vim > Nice thing. This could help those using lambda a lot (whatever reason they might have to do so). I will redirect it to somebody relying on vim heavily for Python development. Thanks. :) > -- > Keeping medicines from the bloodstreams of the sick; food > from the bellies of the hungry; books from the hands of the > uneducated; technology from the underdeveloped; and putting > advocates of freedom in prisons. Intellectual property is > to the 21st century what the slave trade was to the 16th. > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From gokoproject at gmail.com Thu Jul 14 18:13:08 2016 From: gokoproject at gmail.com (John Wong) Date: Thu, 14 Jul 2016 18:13:08 -0400 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <5787F3CB.1080304@mail.de> References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> Message-ID: On Thu, Jul 14, 2016 at 4:19 PM, Sven R. Kunze wrote: > On 14.07.2016 08:39, David Mertz wrote: > > ..... > > That is, and should remain, a matter of tooling not core language (I don't > want > https://en.wikipedia.org/wiki/ColorForth for Python). > > > Very good point. That now is basically the core argument against it at > least for me. So, -100 on the proposal from me. :) > > +1. Editor's job, neither CPython's interpreter nor core grammar. On Wed, Jul 13, 2016 at 9:28 PM, Alexander Belopolsky < alexander.belopolsky at gmail.com> wrote: > > > > On Jul 13, 2016, at 4:12 PM, John Wong wrote: > > > > Sorry to be blunt. Are we going to add omega, delta, psilon and the > entire Greek alphabet? > > Breaking news: the entire Greek alphabet is already available for use in > Python. If someone wants to write code that looks like a series of missing > character boxes on your screen she already can. > You misunderstood my point. I was referring to identifier, what the proposal is asking. Of course unicode is availalble, people always argue about unicode, who wouldn't know Python doesn't support unicode. Why should I write pi in two English characters instead of typing ?? Python is so popular among the science community, so shouldn't we add that as well? Excerpt from the question on http://programmers.stackexchange.com/questions/16010/is-it-bad-to-use-unicode-characters-in-variable-names : t = (?w-?l)/c # those are used in e = ?/c # multiple places. ?w_new = (?w**2 * (1 - (?w**2)/(c**2)*Wwin(t, e)) + ?**2)**.5 If we were to vote on popularity, we'd look at comparing the size of functional programmers vs scientists. Not saying functional programmers don't matter (already stressed this in my previous comment), but this is more like an editor plugin. I personally would love to see such in VIM. Thanks. John -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Thu Jul 14 18:27:20 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 14 Jul 2016 23:27:20 +0100 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> Message-ID: On 14 July 2016 at 23:13, John Wong wrote: > Why should I write pi in two English characters instead of typing ?? Python > is so popular among the science community, so shouldn't we add that as well? > Excerpt from the question on > http://programmers.stackexchange.com/questions/16010/is-it-bad-to-use-unicode-characters-in-variable-names: > > t = (?w-?l)/c # those are used in > e = ?/c # multiple places. > ?w_new = (?w**2 * (1 - (?w**2)/(c**2)*Wwin(t, e)) + ?**2)**.5 I'm not sure what you're saying here. You do realise that the above is perfectly valid Python 3? The SO question you quote is referring to the fact that identifiers are restricted to (Unicode) *letters* and that symbol characters can't be used as variable names. All of which is tangential to the question here which is about using Unicode in a *keyword*. Paul From g.rodola at gmail.com Thu Jul 14 19:18:55 2016 From: g.rodola at gmail.com (Giampaolo Rodola') Date: Fri, 15 Jul 2016 01:18:55 +0200 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> Message-ID: On Fri, Jul 15, 2016 at 12:27 AM, Paul Moore wrote: > On 14 July 2016 at 23:13, John Wong wrote: > > Why should I write pi in two English characters instead of typing ?? > Python > > is so popular among the science community, so shouldn't we add that as > well? > > Excerpt from the question on > > > http://programmers.stackexchange.com/questions/16010/is-it-bad-to-use-unicode-characters-in-variable-names > : > > > > t = (?w-?l)/c # those are used in > > e = ?/c # multiple places. > > ?w_new = (?w**2 * (1 - (?w**2)/(c**2)*Wwin(t, e)) + ?**2)**.5 > > I'm not sure what you're saying here. You do realise that the above is > perfectly valid Python 3? The SO question you quote is referring to > the fact that identifiers are restricted to (Unicode) *letters* and > that symbol characters can't be used as variable names. > > All of which is tangential to the question here which is about using > Unicode in a *keyword*. > I would personally feel bad about using non-ASCII or even non-english variable names in code. Heck, I feel so bad about non-ASCII in code that I even mispell the ? in my last name (Rodol?) and type a' instead. Extending that to a keyword sounds even worse. When Python 3 was cooking I remember there were debates on whether removing "lambda". It stayed, and I'm glad it did, but IMO that should tell it's not important enough to deserve the breakage of a rule which has never been broken (non-ASCII for a keyword). -- Giampaolo - http://grodola.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Jul 15 05:01:05 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 15 Jul 2016 19:01:05 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> Message-ID: On 15 July 2016 at 09:18, Giampaolo Rodola' wrote: > On Fri, Jul 15, 2016 at 12:27 AM, Paul Moore wrote: >> All of which is tangential to the question here which is about using >> Unicode in a *keyword*. > > > I would personally feel bad about using non-ASCII or even non-english > variable names in code. Heck, I feel so bad about non-ASCII in code that I > even mispell the ? in my last name (Rodol?) and type a' instead. Extending > that to a keyword sounds even worse. Unicode-as-identifier makes a lot of sense in situations where you have a data-driven API (like a pandas dataframe or collections.namedtuple) and the data you're working with contains Unicode characters. Hence my choice of example in http://developerblog.redhat.com/2014/09/09/transition-to-multilingual-programming-python/ - it's easy to imagine cases where the named tuple attributes are coming from a data source like headers in a CSV file, and in situations like that, folks shouldn't be forced into awkward workarounds just because their data contains non-ASCII characters. > When Python 3 was cooking I remember there were debates on whether removing > "lambda". It stayed, and I'm glad it did, but IMO that should tell it's not > important enough to deserve the breakage of a rule which has never been > broken (non-ASCII for a keyword). This I largely agree with, though. The *one* argument for improvement I see potentially working is the one I advanced back in March when I suggested that adding support for Java's lambda syntax might be worth doing: https://mail.python.org/pipermail/python-ideas/2016-March/038649.html However, any proposals along those lines need to be couched in terms of how they will advance the Python ecosystem as a whole, rather than "I like using lambda expressions in my code, but I don't like the 'lambda' keyword", as we have a couple of decades worth of evidence informing us that the latter isn't sufficient justification for change. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From antony.lee at berkeley.edu Fri Jul 15 20:51:39 2016 From: antony.lee at berkeley.edu (Antony Lee) Date: Fri, 15 Jul 2016 17:51:39 -0700 Subject: [Python-ideas] Cross-platform pickling of Path objects Message-ID: Hi all, I have opened http://bugs.python.org/issue27175 a month and a half ago but didn't get any feedback there, so I'll copy-paste it here for comments: Currently, pickling Path objects lead to issues when working across platforms: Paths (and, thus, objects that contain Paths) created on a POSIX platform (PosixPaths) cannot be unpickled on Windows and vice versa. There are a few possibilities around this issue. - Don't do anything about it, you should use PurePaths if you care about cross-platform compatibility: this would be pretty awkward, as any call to the Path API would require converting back the PurePath to a Path first. - Silently convert Paths to PurePaths during pickling (a solution that seems to have been adopted by http://docs.menpo.org/en/stable/api/menpo/io/export_pickle.html for example): it would be better if Paths at least roundtripped correctly within a single platform. - Convert Paths to PurePaths at unpickling time, only if the platform is different (and possibly with a warning): this is the least bad solution I came up with so far. Note that calls to the Path API on a "converted" PurePath object would be invalid anyways as the PurePath (being of a different platform) cannot be representing a valid path. Thoughts? Antony -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Fri Jul 15 20:57:50 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 15 Jul 2016 17:57:50 -0700 Subject: [Python-ideas] Cross-platform pickling of Path objects In-Reply-To: References: Message-ID: <5789868E.6010001@stoneleaf.us> On 07/15/2016 05:51 PM, Antony Lee wrote: > I have opened http://bugs.python.org/issue27175 a month and a half ago but didn't get any feedback there, so I'll copy-paste it here for comments: > > Currently, pickling Path objects lead to issues when working across platforms: Paths (and, thus, objects that contain Paths) created on a POSIX platform (PosixPaths) cannot be unpickled on Windows and vice versa. There are a few possibilities around this issue. > > - Don't do anything about it, you should use PurePaths if you care about cross-platform compatibility: this would be pretty awkward, as any call to the Path API would require converting back the PurePath to a Path first. > > - Silently convert Paths to PurePaths during pickling (a solution that seems to have been adopted by http://docs.menpo.org/en/stable/api/menpo/io/export_pickle.html for example): it would be better if Paths at least roundtripped correctly within a single platform. > > - Convert Paths to PurePaths at unpickling time, only if the platform is different (and possibly with a warning): this is the least bad solution I came up with so far. Note that calls to the Path API on a "converted" PurePath object would be invalid anyways as the PurePath (being of a different platform) cannot be representing a valid path. > > Thoughts? Any use-case examples? That would help in deciding on the best path forward. -- ~Ethan~ From antony.lee at berkeley.edu Sat Jul 16 01:51:28 2016 From: antony.lee at berkeley.edu (Antony Lee) Date: Fri, 15 Jul 2016 22:51:28 -0700 Subject: [Python-ideas] Cross-platform pickling of Path objects In-Reply-To: <5789868E.6010001@stoneleaf.us> References: <5789868E.6010001@stoneleaf.us> Message-ID: Sure. I use pickles to store "processed" (scientific) datasets; they typically keep the path to a "raw" dataset as an object attribute (of a Path class, either PosixPath or WindowsPath depending on the "OS" (POSIX/Windows) of the computer where the processing is originally done), as it is sometimes useful to go back and check the raw datasets. However, in general the processed datasets are enough by themselves, so it makes sense to transfer them (independently of the raw datasets) to another computer before further processing. Obviously, once the processed dataset has been transferred to another computer, trying to access the raw dataset will fail with an OSError (e.g., FileNotFoundError); I have code in place to handle that. However, if the file is transferred from a POSIX "OS" to a Windows "OS" or vice-versa, it becomes impossible to even unpickle the file. Converting Path objects to PurePaths upon unpickling on a different "OS", either silently or possibly with a warning, would allow me to just catch both OSErrors and AttributeErrors when trying to access the raw dataset from the path attribute of the processed one. Antony 2016-07-15 17:57 GMT-07:00 Ethan Furman : > On 07/15/2016 05:51 PM, Antony Lee wrote: > > I have opened http://bugs.python.org/issue27175 a month and a half ago >> but didn't get any feedback there, so I'll copy-paste it here for comments: >> >> Currently, pickling Path objects lead to issues when working across >> platforms: Paths (and, thus, objects that contain Paths) created on a POSIX >> platform (PosixPaths) cannot be unpickled on Windows and vice versa. There >> are a few possibilities around this issue. >> >> - Don't do anything about it, you should use PurePaths if you care about >> cross-platform compatibility: this would be pretty awkward, as any call to >> the Path API would require converting back the PurePath to a Path first. >> >> - Silently convert Paths to PurePaths during pickling (a solution that >> seems to have been adopted by >> http://docs.menpo.org/en/stable/api/menpo/io/export_pickle.html for >> example): it would be better if Paths at least roundtripped correctly >> within a single platform. >> >> - Convert Paths to PurePaths at unpickling time, only if the platform is >> different (and possibly with a warning): this is the least bad solution I >> came up with so far. Note that calls to the Path API on a "converted" >> PurePath object would be invalid anyways as the PurePath (being of a >> different platform) cannot be representing a valid path. >> >> Thoughts? >> > > Any use-case examples? That would help in deciding on the best path > forward. > > -- > ~Ethan~ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sat Jul 16 02:58:31 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 16 Jul 2016 16:58:31 +1000 Subject: [Python-ideas] Cross-platform pickling of Path objects In-Reply-To: References: <5789868E.6010001@stoneleaf.us> Message-ID: On 16 July 2016 at 15:51, Antony Lee wrote: > Sure. I use pickles to store "processed" (scientific) datasets; they > typically keep the path to a "raw" dataset as an object attribute (of a Path > class, either PosixPath or WindowsPath depending on the "OS" (POSIX/Windows) > of the computer where the processing is originally done), as it is sometimes > useful to go back and check the raw datasets. However, in general the > processed datasets are enough by themselves, so it makes sense to transfer > them (independently of the raw datasets) to another computer before further > processing. > > Obviously, once the processed dataset has been transferred to another > computer, trying to access the raw dataset will fail with an OSError (e.g., > FileNotFoundError); I have code in place to handle that. However, if the > file is transferred from a POSIX "OS" to a Windows "OS" or vice-versa, it > becomes impossible to even unpickle the file. > > Converting Path objects to PurePaths upon unpickling on a different "OS", > either silently or possibly with a warning, would allow me to just catch > both OSErrors and AttributeErrors when trying to access the raw dataset from > the path attribute of the processed one. The approach of converting to a PurePath with a RuntimeWarning when unpickling seems reasonable to me - if a particular project wants to silence that warning, they can use the warning machinery to suppress it when loading the pickled data. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From storchaka at gmail.com Sun Jul 17 15:22:29 2016 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sun, 17 Jul 2016 22:22:29 +0300 Subject: [Python-ideas] Sequence views Message-ID: Maybe it's time to add a new module for sequence-specific functions (seqtools?). It should contain at least two classes or fabric functions: 1. A view that represents a sliced subsequence. Lazy equivalent of seq[start:end:step]. This feature is implemented in third-party module dataview [1]. 2. A view that represents a linear sequence as 2D array. Iterating this view emits non-intersecting chunks of the sequence. For example it can be used for representing the bytes object as a sequence of 1-byte bytes objects (as in 2.x), a generalized alternative to iterbytes() from PEP 467 [2]. Neither itertools nor collections modules look good place for these features, since they are not concrete classes and work only with sequences, not general iterables or iterators. On other side, mappingproxy and ChainMap look close, maybe new module should be oriented not on sequences, but on views. [1] https://pypi.python.org/pypi/dataview [2] https://www.python.org/dev/peps/pep-0467 From michael.selik at gmail.com Sun Jul 17 16:08:02 2016 From: michael.selik at gmail.com (Michael Selik) Date: Sun, 17 Jul 2016 20:08:02 +0000 Subject: [Python-ideas] Sequence views In-Reply-To: References: Message-ID: On Sun, Jul 17, 2016, 3:22 PM Serhiy Storchaka wrote: > Maybe it's time to add a new module for sequence-specific functions > (seqtools?). It should contain at least two classes or fabric functions: > > 1. A view that represents a sliced subsequence. Lazy equivalent of > seq[start:end:step]. This feature is implemented in third-party module > dataview [1]. > > 2. A view that represents a linear sequence as 2D array. Iterating this > view emits non-intersecting chunks of the sequence. For example it can > be used for representing the bytes object as a sequence of 1-byte bytes > objects (as in 2.x), a generalized alternative to iterbytes() from PEP > 467 [2]. > NumPy slicing and reshaping sounds like it satisfies these requirements. Does it not? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mertz at gnosis.cx Sun Jul 17 17:56:29 2016 From: mertz at gnosis.cx (David Mertz) Date: Sun, 17 Jul 2016 14:56:29 -0700 Subject: [Python-ideas] Sequence views In-Reply-To: References: Message-ID: I don't want to speak for Serhiy, but it seems like he wants NumPy-like behaviors over generic sequences. I think this idea is appealing. For example, Python list has O(1) append, while the equivalent for np.ndarray would be an O(n) copy to a larger array. Expressing those NumPy affordances genetically feels like a good thing. However, maybe this is something that could live in PyPI first top stabilize APIs. On Jul 17, 2016 1:08 PM, "Michael Selik" wrote: > > > On Sun, Jul 17, 2016, 3:22 PM Serhiy Storchaka > wrote: > >> Maybe it's time to add a new module for sequence-specific functions >> (seqtools?). It should contain at least two classes or fabric functions: >> >> 1. A view that represents a sliced subsequence. Lazy equivalent of >> seq[start:end:step]. This feature is implemented in third-party module >> dataview [1]. >> >> 2. A view that represents a linear sequence as 2D array. Iterating this >> view emits non-intersecting chunks of the sequence. For example it can >> be used for representing the bytes object as a sequence of 1-byte bytes >> objects (as in 2.x), a generalized alternative to iterbytes() from PEP >> 467 [2]. >> > > NumPy slicing and reshaping sounds like it satisfies these requirements. > Does it not? > >> > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mertz at gnosis.cx Sun Jul 17 17:57:47 2016 From: mertz at gnosis.cx (David Mertz) Date: Sun, 17 Jul 2016 14:57:47 -0700 Subject: [Python-ideas] Sequence views In-Reply-To: References: Message-ID: Generically, not genetically. Shouldn't reply on phone. On Jul 17, 2016 2:56 PM, "David Mertz" wrote: > I don't want to speak for Serhiy, but it seems like he wants NumPy-like > behaviors over generic sequences. I think this idea is appealing. > > For example, Python list has O(1) append, while the equivalent for > np.ndarray would be an O(n) copy to a larger array. > > Expressing those NumPy affordances genetically feels like a good thing. > However, maybe this is something that could live in PyPI first top > stabilize APIs. > > On Jul 17, 2016 1:08 PM, "Michael Selik" wrote: > >> >> >> On Sun, Jul 17, 2016, 3:22 PM Serhiy Storchaka >> wrote: >> >>> Maybe it's time to add a new module for sequence-specific functions >>> (seqtools?). It should contain at least two classes or fabric functions: >>> >>> 1. A view that represents a sliced subsequence. Lazy equivalent of >>> seq[start:end:step]. This feature is implemented in third-party module >>> dataview [1]. >>> >>> 2. A view that represents a linear sequence as 2D array. Iterating this >>> view emits non-intersecting chunks of the sequence. For example it can >>> be used for representing the bytes object as a sequence of 1-byte bytes >>> objects (as in 2.x), a generalized alternative to iterbytes() from PEP >>> 467 [2]. >>> >> >> NumPy slicing and reshaping sounds like it satisfies these requirements. >> Does it not? >> >>> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Sun Jul 17 18:21:24 2016 From: wes.turner at gmail.com (Wes Turner) Date: Sun, 17 Jul 2016 17:21:24 -0500 Subject: [Python-ideas] Sequence views In-Reply-To: References: Message-ID: There are a number of generic implementations of these sequence algorithms: * http://toolz.readthedocs.io/en/latest/api.html#itertoolz * https://github.com/kachayev/fn.py#itertools-recipes * http://funcy.readthedocs.io/en/stable/seqs.html * http://docs.python.org/2/reference/expressions.html#slicings * https://docs.python.org/2/library/itertools.html#itertools.islice * https://docs.python.org/3/library/itertools.html#itertools.islice On Jul 17, 2016 3:23 PM, "Serhiy Storchaka" wrote: > > Maybe it's time to add a new module for sequence-specific functions (seqtools?). It should contain at least two classes or fabric functions: > > 1. A view that represents a sliced subsequence. Lazy equivalent of seq[start:end:step]. This feature is implemented in third-party module dataview [1]. islice? > > 2. A view that represents a linear sequence as 2D array. Iterating this view emits non-intersecting chunks of the sequence. For example it can be used for representing the bytes object as a sequence of 1-byte bytes objects (as in 2.x), a generalized alternative to iterbytes() from PEP 467 [2]. partition? http://toolz.readthedocs.io/en/latest/api.html#toolz.itertoolz.partition _all > > Neither itertools nor collections modules look good place for these features, since they are not concrete classes and work only with sequences, not general iterables or iterators. On other side, mappingproxy and ChainMap look close, maybe new module should be oriented not on sequences, but on views. > > [1] https://pypi.python.org/pypi/dataview > [2] https://www.python.org/dev/peps/pep-0467 https://docs.python.org/3/library/collections.abc.html > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From grant.jenks at gmail.com Sun Jul 17 19:00:23 2016 From: grant.jenks at gmail.com (Grant Jenks) Date: Sun, 17 Jul 2016 16:00:23 -0700 Subject: [Python-ideas] Sequence views In-Reply-To: References: Message-ID: > > On Jul 17, 2016 3:23 PM, "Serhiy Storchaka" wrote: > > > > Maybe it's time to add a new module for sequence-specific functions (seqtools?). It should contain at least two classes or fabric functions: > > > > 1. A view that represents a sliced subsequence. Lazy equivalent of seq[start:end:step]. This feature is implemented in third-party module dataview [1]. > > On Sun, Jul 17, 2016 at 3:21 PM, Wes Turner wrote: > > islice? SortedContainers implements exactly that as a method on SortedList as: SortedList.islice(start=None, stop=None, reverse=False) Returns an iterator that slices self from start to stop index, inclusive and exclusive respectively. When reverse is True, values are yielded from the iterator in reverse order. Both start and stop default to None which is automatically inclusive of the beginning and end. Return type: iterator Reference: http://www.grantjenks.com/docs/sortedcontainers/sortedlist.html#sortedcontainers.SortedList.L.islice I chose to limit the stride to 1 or -1 using the keyword parameter "reverse." No complaints though it differs from traditional slice syntax. Grant From rustompmody at gmail.com Sun Jul 17 23:41:42 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 17 Jul 2016 20:41:42 -0700 (PDT) Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> Message-ID: On Friday, July 15, 2016 at 2:31:54 PM UTC+5:30, Nick Coghlan wrote: > > However, any proposals along those lines need to be couched in terms > > of how they will advance the Python ecosystem as a whole, rather than > "I like using lambda expressions in my code, but I don't like the > 'lambda' keyword", as we have a couple of decades worth of evidence > informing us that the latter isn't sufficient justification for > change. > > As to the importance of lambdas, on a more conceptual level, most people understand that ?-calculus is theoretically important. A currently running discussion that may indicate that this is true and pragmatic/software engineering levels also http://degoes.net/articles/destroy-all-ifs At the other end of the spectrum on notational/lexical question? On Wednesday, July 13, 2016 at 10:13:39 PM UTC+5:30, David Mertz wrote: I use the vim conceal plugin myself too. It's whimsical, but I like > the appearance of it. So I get the sentiment of the original poster. But > in my conceal configuration, I substitute a bunch of characters visually > (if the attachment works, and screenshot example of some, but not all will > be in this message). And honestly, having my text editor make the > substitution is exactly what I want. > which I find very pretty! More in the same direction: http://blog.languager.org/2014/04/unicoded-python.html Not of course to be taken too literally but rather that the post-ASCII world is any-which-how going that direction As for Nick Coghlan wrote: > Unicode-as-identifier makes a lot of sense in situations > Do consider: >>> ? = 1 >>> A = 2 >>> ? + 1 == A True >>> Can (IMHO) go all the way to https://en.wikipedia.org/wiki/IDN_homograph_attack Discussion on python list at https://mail.python.org/pipermail/python-list/2016-April/706544.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From rustompmody at gmail.com Mon Jul 18 00:20:19 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 17 Jul 2016 21:20:19 -0700 (PDT) Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <858tx4u9z0.fsf@benfinney.id.au> References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <20160714024414.GP27919@ando.pearwood.info> <8D975331-81E4-479D-B316-37310A2CDBDC@gmail.com> <858tx4u9z0.fsf@benfinney.id.au> Message-ID: <6da24b1a-bd5b-49a1-a93e-8f322c2e2f3a@googlegroups.com> On Thursday, July 14, 2016 at 9:51:16 AM UTC+5:30, Ben Finney wrote: > > Alexander Belopolsky > > writes: > > > We already have two camps: those who don't mind using "lambda" and > > those who would only use "def." > > I don't know anyone in the latter camp, do you? > > I am in the camp that loves ?lambda? for some narrowly-specified > purposes *and* thinks ?def? is generally a better tool. > > I suspect the two major camps are those who consider CS to be a branch of math and those who dont Those who dont typically have a strong resistance to the facts of history: http://blog.languager.org/2015/03/cs-history-0.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Mon Jul 18 11:17:42 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 18 Jul 2016 08:17:42 -0700 Subject: [Python-ideas] proper naming of Enum members Message-ID: <578CF316.7040806@stoneleaf.us> There are currently a few locations in the stdlib, such as http and socket, that are now using Enums to replace constants; those names are all upper-case -- those aren't the names I am speaking of. The names I am speaking of are those in brand-new enumerations where we have full control. As an example: class FederalHoliday(AutoNumberEnum): NewYear = "First day of the year.", 'absolute', JANUARY, 1 MartinLutherKingJr = "Birth of Civil Rights leader.", 'relative', JANUARY, MONDAY, 3 President = "Birth of George Washington", 'relative', FEBRUARY, MONDAY, 3 Memorial = "Memory of fallen soldiers", 'relative', MAY, MONDAY, 5 Independence = "Declaration of Independence", 'absolute', JULY, 4 Labor = "American Labor Movement", 'relative', SEPTEMBER, MONDAY, 1 Columbus = "Americas discovered", 'relative', OCTOBER, MONDAY, 2 Veterans = "Recognition of Armed Forces service", 'relative', NOVEMBER, 11, 1 Thanksgiving = "Day of Thanks", 'relative', NOVEMBER, THURSDAY, 4 Christmas = "Birth of Jesus Christ", 'absolute', DECEMBER, 25 def __init__(self, doc, type, month, day, occurance=None): self.__doc__ = doc self.type = type self.month = month self.day = day self.occurance = occurance def date(self, year): """ returns the observed date of the holiday for `year` """" ... @classmethod def next_business_day(cls, date, days=1): """ Return the next `days` business day from date. """ ... @classmethod def count_business_days(cls, date1, date2): """ Return the number of business days between 'date1' and 'date2'. """ ... @classmethod def year(cls, year): """ Return a list of the actual FederalHoliday dates for `year`. """ ... Take the name "NewYear": if it had been a global constant I would have named it "NEWYEAR"; if it had been a normal class attribute I would have named it "new_year"; however, being an Enum member, it is neither of those things. I've written some custom data types as part of my dbf package, and a few of them have instances that are singletons that are created in the global (okay, module) namespace, and for them I followed Python's lead in naming singletons: Python has used Title Case in such things as None, True, and False, so I followed suit and named mine -- Null, NullDate, NullTime, NullDateTime, etc. Given my past history with using and creating singleton objects, I followed suit when creating my own Enum classes. I was recently queried about my apparent break with PEP 8 for naming Enum members, to which I replied: > Considering the strange beast that an Enum is, there is not much precedent for it anywhere. > > Consider: > > - Enum is a class > - but it is a container > - and can be iterated over > - and it has a length (which can be zero) > - but it's always True in a boolean sense > > - Enum members are instances of the Enum class > - but are pre-created > - and new ones cannot be created > - but are available as attributes on the class > > Given all that I have been using Title case (or CamelCase) to name the members as it helps > distinguish an Enum member from an ordinary attribute (which Enum classes can also have). I forgot to include in that reply that I think CamelCase also helps to emphasize the special singleton nature of Enum members. My question for the community: Your thoughts/opinions of my reasoning, and if you don't agree then which casing choice would you recommend and use, and why? (Reminder: this question does not include Enums whose names are replacements for existing constants and so the names cannot be changed.) -- ~Ethan~ From guido at python.org Mon Jul 18 11:41:05 2016 From: guido at python.org (Guido van Rossum) Date: Mon, 18 Jul 2016 08:41:05 -0700 Subject: [Python-ideas] proper naming of Enum members In-Reply-To: <578CF316.7040806@stoneleaf.us> References: <578CF316.7040806@stoneleaf.us> Message-ID: Honestly my own preference would be to use UPPER_CASE, emphasizing the const-ness. CapWords is really only used for classes, which are entirely different beasts. And lower_case is for methods and variables. I think it's useful to emphasize that an enum is neither. On Mon, Jul 18, 2016 at 8:17 AM, Ethan Furman wrote: > There are currently a few locations in the stdlib, such as http and socket, > that are now using > Enums to replace constants; those names are all upper-case -- those aren't > the names I am > speaking of. > > The names I am speaking of are those in brand-new enumerations where we have > full control. > > As an example: > > class FederalHoliday(AutoNumberEnum): > NewYear = "First day of the year.", 'absolute', JANUARY, 1 > MartinLutherKingJr = "Birth of Civil Rights leader.", 'relative', > JANUARY, MONDAY, 3 > President = "Birth of George Washington", 'relative', FEBRUARY, MONDAY, > 3 > Memorial = "Memory of fallen soldiers", 'relative', MAY, MONDAY, 5 > Independence = "Declaration of Independence", 'absolute', JULY, 4 > Labor = "American Labor Movement", 'relative', SEPTEMBER, MONDAY, 1 > Columbus = "Americas discovered", 'relative', OCTOBER, MONDAY, 2 > Veterans = "Recognition of Armed Forces service", 'relative', NOVEMBER, > 11, 1 > Thanksgiving = "Day of Thanks", 'relative', NOVEMBER, THURSDAY, 4 > Christmas = "Birth of Jesus Christ", 'absolute', DECEMBER, 25 > > def __init__(self, doc, type, month, day, occurance=None): > self.__doc__ = doc > self.type = type > self.month = month > self.day = day > self.occurance = occurance > > def date(self, year): > """ > returns the observed date of the holiday for `year` > """" > ... > > @classmethod > def next_business_day(cls, date, days=1): > """ > Return the next `days` business day from date. > """ > ... > @classmethod > def count_business_days(cls, date1, date2): > """ > Return the number of business days between 'date1' and 'date2'. > """ > ... > @classmethod > def year(cls, year): > """ > Return a list of the actual FederalHoliday dates for `year`. > """ > ... > Take the name "NewYear": if it had been a global constant I would have > named it "NEWYEAR"; if > it had been a normal class attribute I would have named it "new_year"; > however, being an Enum > member, it is neither of those things. > > > I've written some custom data types as part of my dbf package, and a few of > them have instances > that are singletons that are created in the global (okay, module) namespace, > and for them I > followed Python's lead in naming singletons: Python has used Title Case in > such things as None, > True, and False, so I followed suit and named mine -- Null, NullDate, > NullTime, NullDateTime, etc. > > > Given my past history with using and creating singleton objects, I followed > suit when creating > my own Enum classes. > > I was recently queried about my apparent break with PEP 8 for naming Enum > members, to which I > replied: > >> Considering the strange beast that an Enum is, there is not much precedent >> for it anywhere. >> >> Consider: >> >> - Enum is a class >> - but it is a container >> - and can be iterated over >> - and it has a length (which can be zero) >> - but it's always True in a boolean sense >> >> - Enum members are instances of the Enum class >> - but are pre-created >> - and new ones cannot be created >> - but are available as attributes on the class >> >> Given all that I have been using Title case (or CamelCase) to name the >> members as it helps >> distinguish an Enum member from an ordinary attribute (which Enum classes >> can also have). > > > I forgot to include in that reply that I think CamelCase also helps to > emphasize the special > singleton nature of Enum members. > > My question for the community: Your thoughts/opinions of my reasoning, and > if you don't agree > then which casing choice would you recommend and use, and why? (Reminder: > this question does > not include Enums whose names are replacements for existing constants and so > the names cannot > be changed.) > > -- > ~Ethan~ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -- --Guido van Rossum (python.org/~guido) From matt at getpattern.com Mon Jul 18 12:53:19 2016 From: matt at getpattern.com (Matt Gilson) Date: Mon, 18 Jul 2016 09:53:19 -0700 Subject: [Python-ideas] proper naming of Enum members In-Reply-To: References: <578CF316.7040806@stoneleaf.us> Message-ID: I also generally think of Enum as a collection of constants so I favor CAP_WORDS for enum members. I will admit that my answer is biased in that I don't typically create Enums that have a whole bunch of methods as you've done in your example. For your example code, I might separate it out into two classes (something representing a Date and an enum representing a Holidays). The former class could use the latter to compute things like "Next business day", etc. On Mon, Jul 18, 2016 at 8:41 AM, Guido van Rossum wrote: > Honestly my own preference would be to use UPPER_CASE, emphasizing the > const-ness. CapWords is really only used for classes, which are > entirely different beasts. And lower_case is for methods and > variables. I think it's useful to emphasize that an enum is neither. > > On Mon, Jul 18, 2016 at 8:17 AM, Ethan Furman wrote: > > There are currently a few locations in the stdlib, such as http and > socket, > > that are now using > > Enums to replace constants; those names are all upper-case -- those > aren't > > the names I am > > speaking of. > > > > The names I am speaking of are those in brand-new enumerations where we > have > > full control. > > > > As an example: > > > > class FederalHoliday(AutoNumberEnum): > > NewYear = "First day of the year.", 'absolute', JANUARY, 1 > > MartinLutherKingJr = "Birth of Civil Rights leader.", 'relative', > > JANUARY, MONDAY, 3 > > President = "Birth of George Washington", 'relative', FEBRUARY, > MONDAY, > > 3 > > Memorial = "Memory of fallen soldiers", 'relative', MAY, MONDAY, 5 > > Independence = "Declaration of Independence", 'absolute', JULY, 4 > > Labor = "American Labor Movement", 'relative', SEPTEMBER, MONDAY, 1 > > Columbus = "Americas discovered", 'relative', OCTOBER, MONDAY, 2 > > Veterans = "Recognition of Armed Forces service", 'relative', > NOVEMBER, > > 11, 1 > > Thanksgiving = "Day of Thanks", 'relative', NOVEMBER, THURSDAY, 4 > > Christmas = "Birth of Jesus Christ", 'absolute', DECEMBER, 25 > > > > def __init__(self, doc, type, month, day, occurance=None): > > self.__doc__ = doc > > self.type = type > > self.month = month > > self.day = day > > self.occurance = occurance > > > > def date(self, year): > > """ > > returns the observed date of the holiday for `year` > > """" > > ... > > > > @classmethod > > def next_business_day(cls, date, days=1): > > """ > > Return the next `days` business day from date. > > """ > > ... > > @classmethod > > def count_business_days(cls, date1, date2): > > """ > > Return the number of business days between 'date1' and 'date2'. > > """ > > ... > > @classmethod > > def year(cls, year): > > """ > > Return a list of the actual FederalHoliday dates for `year`. > > """ > > ... > > Take the name "NewYear": if it had been a global constant I would have > > named it "NEWYEAR"; if > > it had been a normal class attribute I would have named it "new_year"; > > however, being an Enum > > member, it is neither of those things. > > > > > > I've written some custom data types as part of my dbf package, and a few > of > > them have instances > > that are singletons that are created in the global (okay, module) > namespace, > > and for them I > > followed Python's lead in naming singletons: Python has used Title Case > in > > such things as None, > > True, and False, so I followed suit and named mine -- Null, NullDate, > > NullTime, NullDateTime, etc. > > > > > > Given my past history with using and creating singleton objects, I > followed > > suit when creating > > my own Enum classes. > > > > I was recently queried about my apparent break with PEP 8 for naming Enum > > members, to which I > > replied: > > > >> Considering the strange beast that an Enum is, there is not much > precedent > >> for it anywhere. > >> > >> Consider: > >> > >> - Enum is a class > >> - but it is a container > >> - and can be iterated over > >> - and it has a length (which can be zero) > >> - but it's always True in a boolean sense > >> > >> - Enum members are instances of the Enum class > >> - but are pre-created > >> - and new ones cannot be created > >> - but are available as attributes on the class > >> > >> Given all that I have been using Title case (or CamelCase) to name the > >> members as it helps > >> distinguish an Enum member from an ordinary attribute (which Enum > classes > >> can also have). > > > > > > I forgot to include in that reply that I think CamelCase also helps to > > emphasize the special > > singleton nature of Enum members. > > > > My question for the community: Your thoughts/opinions of my reasoning, > and > > if you don't agree > > then which casing choice would you recommend and use, and why? > (Reminder: > > this question does > > not include Enums whose names are replacements for existing constants > and so > > the names cannot > > be changed.) > > > > -- > > ~Ethan~ > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas at python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ > > > > -- > --Guido van Rossum (python.org/~guido) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- [image: pattern-sig.png] Matt Gilson // SOFTWARE ENGINEER E: matt at getpattern.com // P: 603.892.7736 We?re looking for beta testers. Go here to sign up! -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Mon Jul 18 13:22:50 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 18 Jul 2016 10:22:50 -0700 Subject: [Python-ideas] proper naming of Enum members In-Reply-To: References: <578CF316.7040806@stoneleaf.us> Message-ID: <578D106A.7090806@stoneleaf.us> On 07/18/2016 09:53 AM, Matt Gilson wrote: > I also generally think of Enum as a collection of constants so I favor CAP_WORDS for enum members. I will admit that my answer is biased in that I don't typically create Enums that have a whole bunch of methods as you've done in your example. For your example code, I might separate it out into two classes (something representing a Date and an enum representing a Holidays). The former class could use the latter to compute things like "Next business day", etc. Yeah, after I added the 'count_business_days' method I decided to break it apart as BusinessDays for the base behavior, and BankingHolidays for the members themselves; then it won't be US-centric. I just haven't gotten that far. -- ~Ethan~ From shreydesai at me.com Mon Jul 18 12:44:45 2016 From: shreydesai at me.com (Shrey Desai) Date: Mon, 18 Jul 2016 09:44:45 -0700 Subject: [Python-ideas] @Override decorator Message-ID: Python Ideas, While working on some object-oriented projects in Java, I noticed that Python does not have an @override decorator for methods that are derived from a parent class but overridden for the unique purposes of a base class. With the creation of the @abstractmethod decorator, the override decorator could follow in clearly distinguishing the logic and design between parent/base classes. Why I would think an override decorator might be useful: For other people reading the code, it would be great to distinguish between methods that are unique to a class and methods that are inherited from a parent class. Not all methods inherited might be overridden, so keeping track of which inherited methods are overridden and which are not would be nice. With the advent of static typing from mypy: Having the decorator could corroborate the fact that the given method overrides the parent method correctly (correct name + parameter list). When the parent class changes, such as the name or parameter list of an abstract method, the children classes should be updated as well. mypy could easily target the methods that need to be altered with the correct method signature. If you don?t have an override decorator and overrode a parent method, then there could be some error complaining about this. This would be extremely useful to prevent accidental errors. There is some interest for this as expressed on Stack Overflow (http://stackoverflow.com/questions/1167617/in-python-how-do-i-indicate-im-overriding-a-method ) and some people have also made packages for this, but having it in the standard distribution would be nice. Thoughts? Sincerely, Shrey Desai https://github.com/shreydesai -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Mon Jul 18 14:26:04 2016 From: wes.turner at gmail.com (Wes Turner) Date: Mon, 18 Jul 2016 13:26:04 -0500 Subject: [Python-ideas] @Override decorator In-Reply-To: References: Message-ID: It could also be useful to have @override decorator functionality in zope.interface http://zopeinterface.readthedocs.io/en/latest/verify.html zope.interface supports interface verification at test and runtime. AAIU, @override would raise an Exception: - a) when a subclass specifies @override for a method not defined in a superclass - b) when the argspecs are different AFAIU, a)-like functionality does not yet exist in zope.interface On Jul 18, 2016 1:45 PM, "Shrey Desai" wrote: > Python Ideas, > > While working on some object-oriented projects in Java, I noticed that > Python does not have an *@override* decorator for methods that are > derived from a parent class but overridden for the unique purposes of a > base class. With the creation of the *@abstractmethod* decorator, the > override decorator could follow in clearly distinguishing the logic and > design between parent/base classes. > > Why I would think an override decorator might be useful: > > 1. For other people reading the code, it would be great to distinguish > between methods that are unique to a class and methods that are inherited > from a parent class. Not all methods inherited might be overridden, so > keeping track of which inherited methods are overridden and which are not > would be nice. > 2. With the advent of static typing from mypy: > 1. Having the decorator could corroborate the fact that the given > method overrides the parent method correctly (correct name + parameter > list). > 2. When the parent class changes, such as the name or parameter > list of an abstract method, the children classes should be updated as well. > mypy could easily target the methods that need to be altered with the > correct method signature. > 3. If you don?t have an override decorator and overrode a parent > method, then there could be some error complaining about this. This would > be extremely useful to prevent accidental errors. > > > There is some interest for this as expressed on Stack Overflow ( > http://stackoverflow.com/questions/1167617/in-python-how-do-i-indicate-im-overriding-a-method) > and some people have also made packages for this, but having it in the > standard distribution would be nice. Thoughts? > > Sincerely, > Shrey Desai > https://github.com/shreydesai > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Mon Jul 18 14:35:03 2016 From: barry at python.org (Barry Warsaw) Date: Mon, 18 Jul 2016 14:35:03 -0400 Subject: [Python-ideas] proper naming of Enum members References: <578CF316.7040806@stoneleaf.us> Message-ID: <20160718143503.42267702@anarchist.wooz.org> On Jul 18, 2016, at 08:17 AM, Ethan Furman wrote: >Take the name "NewYear": if it had been a global constant I would have named >it "NEWYEAR"; if it had been a normal class attribute I would have named it >"new_year"; however, being an Enum member, it is neither of those things. Since in my code at least, I always include the enum class name at the call site, underscore_words for enum members is just fine. class Action(Enum): hold = 0 reject = 1 discard = 2 accept = 3 defer = 4 ... if action in (Action.defer, Action.hold): # ... elif action is Action.discard: # ... elif action is Action.reject: # ... elif action is Action.accept: # ... and so on. Make the code less shifty :). Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From wes.turner at gmail.com Mon Jul 18 14:45:11 2016 From: wes.turner at gmail.com (Wes Turner) Date: Mon, 18 Jul 2016 13:45:11 -0500 Subject: [Python-ideas] proper naming of Enum members In-Reply-To: <578CF316.7040806@stoneleaf.us> References: <578CF316.7040806@stoneleaf.us> Message-ID: Maybe a bit OT, but there are also holiday singletons defined in https://github.com/pydata/pandas/blob/master/pandas/tseries/holiday.py class USFederalHolidayCalendar(AbstractHolidayCalendar): """ US Federal Government Holiday Calendar based on rules specified by: https://www.opm.gov/policy-data-oversight/ snow-dismissal-procedures/federal-holidays/ """ class Holiday(object): """ Class that defines a holiday with start/end dates and rules for observance. """ def __init__(self, name, year=None, month=None, day=None, offset=None, observance=None, start_date=None, end_date=None, days_of_week=None): On Jul 18, 2016 11:18 AM, "Ethan Furman" wrote: > There are currently a few locations in the stdlib, such as http and > socket, that are now using > Enums to replace constants; those names are all upper-case -- those aren't > the names I am > speaking of. > > The names I am speaking of are those in brand-new enumerations where we > have full control. > > As an example: > > class FederalHoliday(AutoNumberEnum): > NewYear = "First day of the year.", 'absolute', JANUARY, 1 > MartinLutherKingJr = "Birth of Civil Rights leader.", 'relative', > JANUARY, MONDAY, 3 > President = "Birth of George Washington", 'relative', FEBRUARY, > MONDAY, 3 > Memorial = "Memory of fallen soldiers", 'relative', MAY, MONDAY, 5 > Independence = "Declaration of Independence", 'absolute', JULY, 4 > Labor = "American Labor Movement", 'relative', SEPTEMBER, MONDAY, 1 > Columbus = "Americas discovered", 'relative', OCTOBER, MONDAY, 2 > Veterans = "Recognition of Armed Forces service", 'relative', > NOVEMBER, 11, 1 > Thanksgiving = "Day of Thanks", 'relative', NOVEMBER, THURSDAY, 4 > Christmas = "Birth of Jesus Christ", 'absolute', DECEMBER, 25 > > def __init__(self, doc, type, month, day, occurance=None): > self.__doc__ = doc > self.type = type > self.month = month > self.day = day > self.occurance = occurance > > def date(self, year): > """ > returns the observed date of the holiday for `year` > """" > ... > > @classmethod > def next_business_day(cls, date, days=1): > """ > Return the next `days` business day from date. > """ > ... > @classmethod > def count_business_days(cls, date1, date2): > """ > Return the number of business days between 'date1' and 'date2'. > """ > ... > @classmethod > def year(cls, year): > """ > Return a list of the actual FederalHoliday dates for `year`. > """ > ... > Take the name "NewYear": if it had been a global constant I would have > named it "NEWYEAR"; if > it had been a normal class attribute I would have named it "new_year"; > however, being an Enum > member, it is neither of those things. > > > I've written some custom data types as part of my dbf package, and a few > of them have instances > that are singletons that are created in the global (okay, module) > namespace, and for them I > followed Python's lead in naming singletons: Python has used Title Case > in such things as None, > True, and False, so I followed suit and named mine -- Null, NullDate, > NullTime, NullDateTime, etc. > > > Given my past history with using and creating singleton objects, I > followed suit when creating > my own Enum classes. > > I was recently queried about my apparent break with PEP 8 for naming Enum > members, to which I > replied: > > Considering the strange beast that an Enum is, there is not much precedent >> for it anywhere. >> >> Consider: >> >> - Enum is a class >> - but it is a container >> - and can be iterated over >> - and it has a length (which can be zero) >> - but it's always True in a boolean sense >> >> - Enum members are instances of the Enum class >> - but are pre-created >> - and new ones cannot be created >> - but are available as attributes on the class >> >> Given all that I have been using Title case (or CamelCase) to name the >> members as it helps >> distinguish an Enum member from an ordinary attribute (which Enum classes >> can also have). >> > > I forgot to include in that reply that I think CamelCase also helps to > emphasize the special > singleton nature of Enum members. > > My question for the community: Your thoughts/opinions of my reasoning, > and if you don't agree > then which casing choice would you recommend and use, and why? > (Reminder: this question does > not include Enums whose names are replacements for existing constants and > so the names cannot > be changed.) > > -- > ~Ethan~ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Mon Jul 18 14:57:40 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 18 Jul 2016 11:57:40 -0700 Subject: [Python-ideas] @Override decorator In-Reply-To: References: Message-ID: <578D26A4.4080708@stoneleaf.us> On 07/18/2016 09:44 AM, Shrey Desai wrote: > While working on some object-oriented projects in Java, I noticed that Python does not have an *@override* decorator for methods that are derived from a parent class but overridden for the unique purposes of a base class. With the creation of the *@abstractmethod* decorator, the override decorator could follow in clearly distinguishing the logic and design between parent/base classes. I certainly see the usefulness of such a decorator, but I don't think it needs to be in stdlib. -- ~Ethan~ From mistersheik at gmail.com Mon Jul 18 18:15:09 2016 From: mistersheik at gmail.com (Neil Girdhar) Date: Mon, 18 Jul 2016 15:15:09 -0700 (PDT) Subject: [Python-ideas] @Override decorator In-Reply-To: References: Message-ID: <0cae8f09-ff42-4fc4-860f-1f2d6f4ca6eb@googlegroups.com> I would really like this as I currently comment all my overridden methods to remind myself what is an override. However, I suggest two overrides decorators: one for pure virtuals, and one where the pattern is call super. The assertion for argspecs should only apply when the superclass has a pure virtual method. The other pattern is with multiple inheritance where it's common for subclasses to pull off the arguments they need and forward the rest to the superclass. In that case, it would be nice to check (as best as possible, by for example walking the parsed code) that the member calls super. A common bug is to forget to do this, which might not be detected until multiple inheritance causes the mro to change the superclass, and the new superclass has important behavior that's not being run. Best, Neil On Monday, July 18, 2016 at 1:45:40 PM UTC-4, Shrey Desai wrote: > > Python Ideas, > > While working on some object-oriented projects in Java, I noticed that > Python does not have an *@override* decorator for methods that are > derived from a parent class but overridden for the unique purposes of a > base class. With the creation of the *@abstractmethod* decorator, the > override decorator could follow in clearly distinguishing the logic and > design between parent/base classes. > > Why I would think an override decorator might be useful: > > 1. For other people reading the code, it would be great to distinguish > between methods that are unique to a class and methods that are inherited > from a parent class. Not all methods inherited might be overridden, so > keeping track of which inherited methods are overridden and which are not > would be nice. > 2. With the advent of static typing from mypy: > 1. Having the decorator could corroborate the fact that the given > method overrides the parent method correctly (correct name + parameter > list). > 2. When the parent class changes, such as the name or parameter > list of an abstract method, the children classes should be updated as well. > mypy could easily target the methods that need to be altered with the > correct method signature. > 3. If you don?t have an override decorator and overrode a parent > method, then there could be some error complaining about this. This would > be extremely useful to prevent accidental errors. > > > There is some interest for this as expressed on Stack Overflow ( > http://stackoverflow.com/questions/1167617/in-python-how-do-i-indicate-im-overriding-a-method) > and some people have also made packages for this, but having it in the > standard distribution would be nice. Thoughts? > > Sincerely, > Shrey Desai > https://github.com/shreydesai > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Mon Jul 18 18:36:23 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 18 Jul 2016 18:36:23 -0400 Subject: [Python-ideas] @Override decorator In-Reply-To: References: Message-ID: On 7/18/2016 12:44 PM, Shrey Desai wrote: > Python Ideas, > > While working on some object-oriented projects in Java, I noticed that > Python does not have an *@override* decorator for methods that are > derived from a parent class but overridden for the unique purposes of a > base class. With the creation of the *@abstractmethod* decorator, the > override decorator could follow in clearly distinguishing the logic and > design between parent/base classes. If you mean 'override' to have some actual function, then it is not clear to me. If the 'decorator' is strictly a comment, why not use a comment? > Why I would think an override decorator might be useful: > > 1. For other people reading the code, it would be great to distinguish > between methods that are unique to a class and methods that are > inherited from a parent class. Not all methods inherited might be > overridden, so keeping track of which inherited methods are > overridden and which are not would be nice. > 2. With the advent of static typing from mypy: > 1. Having the decorator could corroborate the fact that the given > method overrides the parent method correctly (correct name + > parameter list). > 2. When the parent class changes, such as the name or parameter > list of an abstract method, the children classes should be > updated as well. mypy could easily target the methods that need > to be altered with the correct method signature. > 3. If you don?t have an override decorator and overrode a parent > method, then there could be some error complaining about this. > This would be extremely useful to prevent accidental errors. > > > There is some interest for this as expressed on Stack Overflow > (http://stackoverflow.com/questions/1167617/in-python-how-do-i-indicate-im-overriding-a-method) > and some people have also made packages for this, but having it in the > standard distribution would be nice. Thoughts? > > Sincerely, > Shrey Desai > https://github.com/shreydesai > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Terry Jan Reedy From michael.selik at gmail.com Mon Jul 18 18:53:18 2016 From: michael.selik at gmail.com (Michael Selik) Date: Mon, 18 Jul 2016 22:53:18 +0000 Subject: [Python-ideas] proper naming of Enum members In-Reply-To: <578CF316.7040806@stoneleaf.us> References: <578CF316.7040806@stoneleaf.us> Message-ID: On Mon, Jul 18, 2016 at 10:17 AM Ethan Furman wrote: > There are currently a few locations in the stdlib, such as http and > socket, that are now using > Enums to replace constants; those names are all upper-case -- those aren't > the names I am > speaking of. > > The names I am speaking of are those in brand-new enumerations where we > have full control. > Even if there's a better design as judged in isolation, it still might be best to stay consistent with socket module. To paraphrase "Jakob's Law": programmers spend most of their time using *other* libraries (such as socket, logging, etc.). So it's generally a good idea not to invent new interfaces. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Mon Jul 18 21:10:24 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 19 Jul 2016 11:10:24 +1000 Subject: [Python-ideas] @Override decorator References: Message-ID: <85inw2sa9r.fsf@benfinney.id.au> Terry Reedy writes: > If you mean 'override' to have some actual function, then it is not > clear to me. If the 'decorator' is strictly a comment, why not use a > comment? The primary behaviour I've seen for such suggestions is that ?override? will verify the method is already defined in the specified base class (or, if no class specified, then in at least one of the base classes), and if not will raise AssertionError. The proposal describes the behaviour this way: > > 2. With the advent of static typing from mypy: > > 1. Having the decorator could corroborate the fact that the given > > method overrides the parent method correctly (correct name + > > parameter list). > > 2. When the parent class changes, such as the name or parameter > > list of an abstract method, the children classes should be > > updated as well. mypy could easily target the methods that need > > to be altered with the correct method signature. > > 3. If you don?t have an override decorator and overrode a parent > > method, then there could be some error complaining about this. > > This would be extremely useful to prevent accidental errors. -- \ ?Holy polar ice sheet, Batman!? ?Robin | `\ | _o__) | Ben Finney From steve at pearwood.info Mon Jul 18 23:58:58 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 19 Jul 2016 13:58:58 +1000 Subject: [Python-ideas] @Override decorator In-Reply-To: References: Message-ID: <20160719035858.GW27919@ando.pearwood.info> On Mon, Jul 18, 2016 at 09:44:45AM -0700, Shrey Desai wrote: > Why I would think an override decorator might be useful: > For other people reading the code, it would be great to distinguish > between methods that are unique to a class and methods that are > inherited from a parent class. Not all methods inherited might be > overridden, so keeping track of which inherited methods are overridden > and which are not would be nice. I fail to see that this is a sufficiently great problem to require a decorator. Occasionally I've added a comment to a method "Overriding mixin" or "Overriding class XXX" for example, but mostly I don't even bother. It's either obvious or unimportant or both. And when I do find myself writing such a comment, that's a good sign that my class hierarchy is too hard to understand and needs to be refactored. > With the advent of static typing from mypy: > Having the decorator could corroborate the fact that the given method > overrides the parent method correctly (correct name + parameter list). There is no requirement that the subclass method uses the same parameter names, or even signature, as the superclass method. > When the parent class changes, such as the name or parameter list of > an abstract method, the children classes should be updated as well. > mypy could easily target the methods that need to be altered with the > correct method signature. > If you don?t have an override decorator and overrode a parent method, > then there could be some error complaining about this. This would be > extremely useful to prevent accidental errors. I don't see it being very useful. Perhaps for people who have learned bad habits from Java and write deep class hierarchies, but we shouldn't encourage such anti-patterns. If people wish to write their own @override decorator, they should go right ahead, but I don't think it belongs in the standard library. -- Steve From guido at python.org Tue Jul 19 00:04:18 2016 From: guido at python.org (Guido van Rossum) Date: Mon, 18 Jul 2016 21:04:18 -0700 Subject: [Python-ideas] @Override decorator In-Reply-To: <20160719035858.GW27919@ando.pearwood.info> References: <20160719035858.GW27919@ando.pearwood.info> Message-ID: The same argument would apply against abstractmethod... I have definitely wondered when looking over some code whether it was defining or overriding a method. But requiring people to always decorate overrides seems hard, unless we made it a property that can be enforced by a metaclass, maybe? On Monday, July 18, 2016, Steven D'Aprano wrote: > On Mon, Jul 18, 2016 at 09:44:45AM -0700, Shrey Desai wrote: > > > Why I would think an override decorator might be useful: > > For other people reading the code, it would be great to distinguish > > between methods that are unique to a class and methods that are > > inherited from a parent class. Not all methods inherited might be > > overridden, so keeping track of which inherited methods are overridden > > and which are not would be nice. > > I fail to see that this is a sufficiently great problem to require a > decorator. Occasionally I've added a comment to a method "Overriding > mixin" or "Overriding class XXX" for example, but mostly I don't even > bother. It's either obvious or unimportant or both. And when I do find > myself writing such a comment, that's a good sign that my class > hierarchy is too hard to understand and needs to be refactored. > > > > With the advent of static typing from mypy: > > Having the decorator could corroborate the fact that the given method > > overrides the parent method correctly (correct name + parameter list). > > There is no requirement that the subclass method uses the same parameter > names, or even signature, as the superclass method. > > > > When the parent class changes, such as the name or parameter list of > > an abstract method, the children classes should be updated as well. > > mypy could easily target the methods that need to be altered with the > > correct method signature. > > If you don?t have an override decorator and overrode a parent method, > > then there could be some error complaining about this. This would be > > extremely useful to prevent accidental errors. > > I don't see it being very useful. Perhaps for people who have learned > bad habits from Java and write deep class hierarchies, but we shouldn't > encourage such anti-patterns. If people wish to write their own > @override decorator, they should go right ahead, but I don't think it > belongs in the standard library. > > > > -- > Steve > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -- --Guido (mobile) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Tue Jul 19 00:29:49 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 18 Jul 2016 21:29:49 -0700 Subject: [Python-ideas] @Override decorator In-Reply-To: References: <20160719035858.GW27919@ando.pearwood.info> Message-ID: <578DACBD.6090000@stoneleaf.us> On 07/18/2016 09:04 PM, Guido van Rossum wrote: > The same argument would apply against abstractmethod... I have definitely wondered when looking over some code whether it was defining or overriding a method. But requiring people to always decorate overrides seems hard, unless we made it a property that can be enforced by a metaclass, maybe? Do you mean something like: def __new__(metacls, cls, bases, clsdict): for name, attr in clsdict.items(): if callable(attr): is_override = isinstance(attr, override) # search for name in bases # if found and is_override is False, raise # or if not found and is_override is True, raise -- ~Ethan~ From ncoghlan at gmail.com Tue Jul 19 00:41:51 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 19 Jul 2016 14:41:51 +1000 Subject: [Python-ideas] Sequence views In-Reply-To: References: Message-ID: On 18 July 2016 at 08:21, Wes Turner wrote: > There are a number of generic implementations of these sequence algorithms: > > * http://toolz.readthedocs.io/en/latest/api.html#itertoolz > * https://github.com/kachayev/fn.py#itertools-recipes > * http://funcy.readthedocs.io/en/stable/seqs.html > * http://docs.python.org/2/reference/expressions.html#slicings > * https://docs.python.org/2/library/itertools.html#itertools.islice > * https://docs.python.org/3/library/itertools.html#itertools.islice I think the existence of multiple implementations in the context of larger libraries lends weight to the notion of a "seqtools" standard library module that works with arbitrary sequences, just as itertools works with arbitrary iterables. I don't think combining these algorithms with the "algorithms that work with arbitrary mappings" classes would make sense - for better or for worse, I think the latter is "collections" now, since that's also where the dict variants live (defaultdict, OrderedDict, Counter). Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Tue Jul 19 00:49:41 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 19 Jul 2016 14:49:41 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> Message-ID: On 18 July 2016 at 13:41, Rustom Mody wrote: > Do consider: > >>>> ? = 1 >>>> A = 2 >>>> ? + 1 == A > True >>>> > > Can (IMHO) go all the way to > https://en.wikipedia.org/wiki/IDN_homograph_attack Yes, we know - that dramatic increase in the attack surface is why PyPI is still ASCII only, even though full Unicode support is theoretically possible. It's not a major concern once an attacker already has you running arbitrary code on your system though, as the main problem there is that they're *running arbitrary code on your system*. , That means the usability gains easily outweigh the increased obfuscation potential, as worrying about confusable attacks at that point is like worrying about a dripping tap upstairs when the Brisbane River is already flowing through the ground floor of your house :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Tue Jul 19 00:54:40 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 19 Jul 2016 14:54:40 +1000 Subject: [Python-ideas] proper naming of Enum members In-Reply-To: References: <578CF316.7040806@stoneleaf.us> Message-ID: On 19 July 2016 at 01:41, Guido van Rossum wrote: > Honestly my own preference would be to use UPPER_CASE, emphasizing the > const-ness. CapWords is really only used for classes, which are > entirely different beasts. And lower_case is for methods and > variables. I think it's useful to emphasize that an enum is neither. I haven't used enum much outside the standard library, but my C/C++ background would push me towards CAP_WORDS since they're constants. That also has the virtue of making the standard library uses no longer a special case - they're just CAP_WORDS-because-they're-constants, rather than CAP_WORDS-because-they're-defined-by-a-third-party-C-API Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Tue Jul 19 01:18:34 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 19 Jul 2016 15:18:34 +1000 Subject: [Python-ideas] @Override decorator In-Reply-To: References: <20160719035858.GW27919@ando.pearwood.info> Message-ID: On 19 July 2016 at 14:04, Guido van Rossum wrote: > The same argument would apply against abstractmethod... I have definitely > wondered when looking over some code whether it was defining or overriding a > method. But requiring people to always decorate overrides seems hard, unless > we made it a property that can be enforced by a metaclass, maybe? This is trickier in Python than it is in Java, since we support multiple inheritance, and we don't require that subclasses be drop-in replacements for their parent class. The key difference I see between @abstractmethod and this proposal is that @abstractmethod is a service that base class authors provide to subclass authors to say "Hey, when subclassing this, you *need* to override these, or your subclass won't work properly". In effect, it's an implicit test case for all future subclass definitions. By contrast, this proposal would go in the other direction: it would be an assertion by the author of a *subclass* that that particular method isn't new, it's inherited from a parent class, and they're replacing the existing implementation with a new one. It would only add new information for readers in cases where the method wasn't already calling super() (since the latter already implies you expect there to be at least one further implementation along the MRO to call). In my view, that puts it more in the category of type annotations and the "missing attribute" detection in tools like pylint than it does abstractmethod: rather than checking it at runtime, you'd want static checkers like mypy to flag it as a structural error if a method declared as replacing a base class method wasn't actually doing so. So a "typing.override" to complement "typing.overload" would make sense to me, but a checked-at-runtime decorator wouldn't (doing it that way would also let you put the override annotation in stub files instead of directly in the code). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From rustompmody at gmail.com Tue Jul 19 01:29:34 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 18 Jul 2016 22:29:34 -0700 (PDT) Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> Message-ID: <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> On Tuesday, July 19, 2016 at 10:20:29 AM UTC+5:30, Nick Coghlan wrote: > > On 18 July 2016 at 13:41, Rustom Mody > > wrote: > > Do consider: > > > >>>> ? = 1 > >>>> A = 2 > >>>> ? + 1 == A > > True > >>>> > > > > Can (IMHO) go all the way to > > https://en.wikipedia.org/wiki/IDN_homograph_attack > > Yes, we know - that dramatic increase in the attack surface is why > PyPI is still ASCII only, even though full Unicode support is > theoretically possible. > > It's not a major concern once an attacker already has you running > arbitrary code on your system though, as the main problem there is > that they're *running arbitrary code on your system*. , That means the > usability gains easily outweigh the increased obfuscation potential, > as worrying about confusable attacks at that point is like worrying > about a dripping tap upstairs when the Brisbane River is already > flowing through the ground floor of your house :) > > Cheers, > > There was this question on the python list a few days ago: Subject: SyntaxError: Non-ASCII character Chris Angelico pointed out the offending line: wf = wave.open(?test.wav?, ?rb?) (should be wf = wave.open("test.wav", "rb") instead) Since he also said: > The solution may be as simple as running "python3 script.py" rather than "python script.py". I pointed out that the python2 error was more helpful (to my eyes) than python3s Python3 Traceback (most recent call last): File "", line 1, in File "/home/ariston/foo.py", line 31 wf = wave.open(?test.wav?, ?rb?) ^ SyntaxError: invalid character in identifier Python2 Traceback (most recent call last): File "", line 1, in File "foo.py", line 31 SyntaxError: Non-ASCII character '\xe2' in file foo.py on line 31, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details IOW 1. The lexer is internally (evidently from the error message) so ASCII-oriented that any ?unicode-junk? just defaults out to identifiers (presumably comments are dealt with earlier) and then if that lexing action fails it mistakenly pinpoints a wrong *identifier* rather than just an impermissible character like python 2 combine that with 2. matrix mult (@) Ok to emulate perl but not to go outside ASCII makes it seem (to me) python's unicode support is somewhat wrongheaded. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mistersheik at gmail.com Tue Jul 19 02:44:55 2016 From: mistersheik at gmail.com (Neil Girdhar) Date: Tue, 19 Jul 2016 06:44:55 +0000 Subject: [Python-ideas] @Override decorator In-Reply-To: References: <20160719035858.GW27919@ando.pearwood.info> Message-ID: On Tue, Jul 19, 2016 at 1:19 AM Nick Coghlan wrote: > On 19 July 2016 at 14:04, Guido van Rossum wrote: > > The same argument would apply against abstractmethod... I have definitely > > wondered when looking over some code whether it was defining or > overriding a > > method. But requiring people to always decorate overrides seems hard, > unless > > we made it a property that can be enforced by a metaclass, maybe? > > This is trickier in Python than it is in Java, since we support > multiple inheritance, and we don't require that subclasses be drop-in > replacements for their parent class. > > The key difference I see between @abstractmethod and this proposal is > that @abstractmethod is a service that base class authors provide to > subclass authors to say "Hey, when subclassing this, you *need* to > override these, or your subclass won't work properly". In effect, it's > an implicit test case for all future subclass definitions. > > By contrast, this proposal would go in the other direction: it would > be an assertion by the author of a *subclass* that that particular > method isn't new, it's inherited from a parent class, and they're > replacing the existing implementation with a new one. It would only > add new information for readers in cases where the method wasn't > already calling super() (since the latter already implies you expect > there to be at least one further implementation along the MRO to > call). > > In my view, that puts it more in the category of type annotations and > the "missing attribute" detection in tools like pylint than it does > abstractmethod: rather than checking it at runtime, you'd want static > checkers like mypy to flag it as a structural error if a method > declared as replacing a base class method wasn't actually doing so. > Why can't @overrides be processed at definition time? It could check one of two cases: either you're replacing an abstractmethod, in which case the parameter lists should be the same; or else, you're overriding a base class method in which case, you should probably call super. Finally, it could prepend the __doc__ attribute from the parent class method if there is one to the doc attribute on this method if there is one. > > So a "typing.override" to complement "typing.overload" would make > sense to me, but a checked-at-runtime decorator wouldn't (doing it > that way would also let you put the override annotation in stub files > instead of directly in the code). > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > -- > > --- > You received this message because you are subscribed to a topic in the > Google Groups "python-ideas" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/python-ideas/V6sUaArzGC0/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > python-ideas+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mistersheik at gmail.com Tue Jul 19 03:09:03 2016 From: mistersheik at gmail.com (Neil Girdhar) Date: Tue, 19 Jul 2016 00:09:03 -0700 (PDT) Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> Message-ID: <5593a831-2822-49fa-9fb8-6a25b12f5cd5@googlegroups.com> One solution would be to restrict identifiers to only Unicode characters in appropriate classes. The open quotation mark is in the code class for punctuation, so it doesn't make sense to have it be part of an identifier. http://www.fileformat.info/info/unicode/category/index.htm On Tuesday, July 19, 2016 at 1:29:35 AM UTC-4, Rustom Mody wrote: > > On Tuesday, July 19, 2016 at 10:20:29 AM UTC+5:30, Nick Coghlan wrote: >> >> On 18 July 2016 at 13:41, Rustom Mody wrote: >> > Do consider: >> > >> >>>> ? = 1 >> >>>> A = 2 >> >>>> ? + 1 == A >> > True >> >>>> >> > >> > Can (IMHO) go all the way to >> > https://en.wikipedia.org/wiki/IDN_homograph_attack >> >> Yes, we know - that dramatic increase in the attack surface is why >> PyPI is still ASCII only, even though full Unicode support is >> theoretically possible. >> >> It's not a major concern once an attacker already has you running >> arbitrary code on your system though, as the main problem there is >> that they're *running arbitrary code on your system*. , That means the >> usability gains easily outweigh the increased obfuscation potential, >> as worrying about confusable attacks at that point is like worrying >> about a dripping tap upstairs when the Brisbane River is already >> flowing through the ground floor of your house :) >> >> Cheers, >> >> > There was this question on the python list a few days ago: > Subject: SyntaxError: Non-ASCII character > > Chris Angelico pointed out the offending line: > wf = wave.open(?test.wav?, ?rb?) > (should be wf = wave.open("test.wav", "rb") instead) > > Since he also said: > > The solution may be as simple as running "python3 script.py" rather than > "python script.py". > > I pointed out that the python2 error was more helpful (to my eyes) than > python3s > > > Python3 > > Traceback (most recent call last): > File "", line 1, in > File "/home/ariston/foo.py", line 31 > wf = wave.open(?test.wav?, ?rb?) > ^ > SyntaxError: invalid character in identifier > > Python2 > > > Traceback (most recent call last): > File "", line 1, in > File "foo.py", line 31 > SyntaxError: Non-ASCII character '\xe2' in file foo.py on line 31, but no > encoding declared; see http://python.org/dev/peps/pep-0263/ for details > > IOW > 1. The lexer is internally (evidently from the error message) so > ASCII-oriented that any ?unicode-junk? just defaults out to identifiers > (presumably comments are dealt with earlier) and then if that lexing action > fails it mistakenly pinpoints a wrong *identifier* rather than just an > impermissible character like python 2 > combine that with > 2. matrix mult (@) Ok to emulate perl but not to go outside ASCII > > makes it seem (to me) python's unicode support is somewhat wrongheaded. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rustompmody at gmail.com Tue Jul 19 03:32:39 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 19 Jul 2016 00:32:39 -0700 (PDT) Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <5593a831-2822-49fa-9fb8-6a25b12f5cd5@googlegroups.com> References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <5593a831-2822-49fa-9fb8-6a25b12f5cd5@googlegroups.com> Message-ID: <364890c6-5e3a-4002-b74b-03b98bb9b248@googlegroups.com> On Tuesday, July 19, 2016 at 12:39:04 PM UTC+5:30, Neil Girdhar wrote: > > One solution would be to restrict identifiers to only Unicode characters > in appropriate classes. The open quotation mark is in the code class for > punctuation, so it doesn't make sense to have it be part of an identifier. > > http://www.fileformat.info/info/unicode/category/index.htm > Python (3) is doing that alright as far as I can see: https://docs.python.org/3/reference/lexical_analysis.html#identifiers The point is that when it doesn?t fall in the classification(s) the error it raises suggests that the lexer is not really unicode-aware > > > On Tuesday, July 19, 2016 at 1:29:35 AM UTC-4, Rustom Mody wrote: >> >> On Tuesday, July 19, 2016 at 10:20:29 AM UTC+5:30, Nick Coghlan wrote: >>> >>> On 18 July 2016 at 13:41, Rustom Mody wrote: >>> > Do consider: >>> > >>> >>>> ? = 1 >>> >>>> A = 2 >>> >>>> ? + 1 == A >>> > True >>> >>>> >>> > >>> > Can (IMHO) go all the way to >>> > https://en.wikipedia.org/wiki/IDN_homograph_attack >>> >>> Yes, we know - that dramatic increase in the attack surface is why >>> PyPI is still ASCII only, even though full Unicode support is >>> theoretically possible. >>> >>> It's not a major concern once an attacker already has you running >>> arbitrary code on your system though, as the main problem there is >>> that they're *running arbitrary code on your system*. , That means the >>> usability gains easily outweigh the increased obfuscation potential, >>> as worrying about confusable attacks at that point is like worrying >>> about a dripping tap upstairs when the Brisbane River is already >>> flowing through the ground floor of your house :) >>> >>> Cheers, >>> >>> >> There was this question on the python list a few days ago: >> Subject: SyntaxError: Non-ASCII character >> >> Chris Angelico pointed out the offending line: >> wf = wave.open(?test.wav?, ?rb?) >> (should be wf = wave.open("test.wav", "rb") instead) >> >> Since he also said: >> > The solution may be as simple as running "python3 script.py" rather >> than "python script.py". >> >> I pointed out that the python2 error was more helpful (to my eyes) than >> python3s >> >> >> Python3 >> >> Traceback (most recent call last): >> File "", line 1, in >> File "/home/ariston/foo.py", line 31 >> wf = wave.open(?test.wav?, ?rb?) >> ^ >> SyntaxError: invalid character in identifier >> >> Python2 >> >> >> Traceback (most recent call last): >> File "", line 1, in >> File "foo.py", line 31 >> SyntaxError: Non-ASCII character '\xe2' in file foo.py on line 31, but no >> encoding declared; see http://python.org/dev/peps/pep-0263/ for details >> >> IOW >> 1. The lexer is internally (evidently from the error message) so >> ASCII-oriented that any ?unicode-junk? just defaults out to identifiers >> (presumably comments are dealt with earlier) and then if that lexing action >> fails it mistakenly pinpoints a wrong *identifier* rather than just an >> impermissible character like python 2 >> combine that with >> 2. matrix mult (@) Ok to emulate perl but not to go outside ASCII >> >> makes it seem (to me) python's unicode support is somewhat wrongheaded. >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From mistersheik at gmail.com Tue Jul 19 06:32:07 2016 From: mistersheik at gmail.com (Neil Girdhar) Date: Tue, 19 Jul 2016 03:32:07 -0700 (PDT) Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <364890c6-5e3a-4002-b74b-03b98bb9b248@googlegroups.com> References: <0c7f01d1dc6c$435f9010$ca1eb030$@hotmail.com> <22405.41379.293072.490849@turnbull.sk.tsukuba.ac.jp> <20160713040025.GN27919@ando.pearwood.info> <22406.34090.155898.400535@turnbull.sk.tsukuba.ac.jp> <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <5593a831-2822-49fa-9fb8-6a25b12f5cd5@googlegroups.com> <364890c6-5e3a-4002-b74b-03b98bb9b248@googlegroups.com> Message-ID: <06dda608-8e60-42db-ad3e-a256ecd80ff2@googlegroups.com> Sounds like a bug in the lexer? Or maybe a feature request. On Tuesday, July 19, 2016 at 3:32:39 AM UTC-4, Rustom Mody wrote: > > > > On Tuesday, July 19, 2016 at 12:39:04 PM UTC+5:30, Neil Girdhar wrote: >> >> One solution would be to restrict identifiers to only Unicode characters >> in appropriate classes. The open quotation mark is in the code class for >> punctuation, so it doesn't make sense to have it be part of an identifier. >> >> http://www.fileformat.info/info/unicode/category/index.htm >> > > Python (3) is doing that alright as far as I can see: > https://docs.python.org/3/reference/lexical_analysis.html#identifiers > > The point is that when it doesn?t fall in the classification(s) the error > it raises suggests that the lexer is not really unicode-aware > > >> >> >> On Tuesday, July 19, 2016 at 1:29:35 AM UTC-4, Rustom Mody wrote: >>> >>> On Tuesday, July 19, 2016 at 10:20:29 AM UTC+5:30, Nick Coghlan wrote: >>>> >>>> On 18 July 2016 at 13:41, Rustom Mody wrote: >>>> > Do consider: >>>> > >>>> >>>> ? = 1 >>>> >>>> A = 2 >>>> >>>> ? + 1 == A >>>> > True >>>> >>>> >>>> > >>>> > Can (IMHO) go all the way to >>>> > https://en.wikipedia.org/wiki/IDN_homograph_attack >>>> >>>> Yes, we know - that dramatic increase in the attack surface is why >>>> PyPI is still ASCII only, even though full Unicode support is >>>> theoretically possible. >>>> >>>> It's not a major concern once an attacker already has you running >>>> arbitrary code on your system though, as the main problem there is >>>> that they're *running arbitrary code on your system*. , That means the >>>> usability gains easily outweigh the increased obfuscation potential, >>>> as worrying about confusable attacks at that point is like worrying >>>> about a dripping tap upstairs when the Brisbane River is already >>>> flowing through the ground floor of your house :) >>>> >>>> Cheers, >>>> >>>> >>> There was this question on the python list a few days ago: >>> Subject: SyntaxError: Non-ASCII character >>> >>> Chris Angelico pointed out the offending line: >>> wf = wave.open(?test.wav?, ?rb?) >>> (should be wf = wave.open("test.wav", "rb") instead) >>> >>> Since he also said: >>> > The solution may be as simple as running "python3 script.py" rather >>> than "python script.py". >>> >>> I pointed out that the python2 error was more helpful (to my eyes) than >>> python3s >>> >>> >>> Python3 >>> >>> Traceback (most recent call last): >>> File "", line 1, in >>> File "/home/ariston/foo.py", line 31 >>> wf = wave.open(?test.wav?, ?rb?) >>> ^ >>> SyntaxError: invalid character in identifier >>> >>> Python2 >>> >>> >>> Traceback (most recent call last): >>> File "", line 1, in >>> File "foo.py", line 31 >>> SyntaxError: Non-ASCII character '\xe2' in file foo.py on line 31, but >>> no encoding declared; see http://python.org/dev/peps/pep-0263/ for >>> details >>> >>> IOW >>> 1. The lexer is internally (evidently from the error message) so >>> ASCII-oriented that any ?unicode-junk? just defaults out to identifiers >>> (presumably comments are dealt with earlier) and then if that lexing action >>> fails it mistakenly pinpoints a wrong *identifier* rather than just an >>> impermissible character like python 2 >>> combine that with >>> 2. matrix mult (@) Ok to emulate perl but not to go outside ASCII >>> >>> makes it seem (to me) python's unicode support is somewhat wrongheaded. >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Jul 19 07:20:27 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 19 Jul 2016 21:20:27 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> Message-ID: <20160719112027.GX27919@ando.pearwood.info> On Mon, Jul 18, 2016 at 10:29:34PM -0700, Rustom Mody wrote: > There was this question on the python list a few days ago: > Subject: SyntaxError: Non-ASCII character [...] > I pointed out that the python2 error was more helpful (to my eyes) than > python3s And I pointed out how I thought the Python 3 error message could be improved, but the Python 2 error message was not very good. > Python3 > > Traceback (most recent call last): > File "", line 1, in > File "/home/ariston/foo.py", line 31 > wf = wave.open(?test.wav?, ?rb?) > ^ > SyntaxError: invalid character in identifier It would be much more helpful if the caret lined up with the offending character. Better still, if the offending character was actually stated: wf = wave.open(?test.wav?, ?rb?) ^ SyntaxError: invalid character '?' in identifier > Python2 > > Traceback (most recent call last): > File "", line 1, in > File "foo.py", line 31 > SyntaxError: Non-ASCII character '\xe2' in file foo.py on line 31, but no > encoding declared; see http://python.org/dev/peps/pep-0263/ for details As I pointed out earlier, this is less helpful. The line itself is not shown (although the line number is given), nor is the offending character. (Python 2 can't show the character because it doesn't know what it is -- it only knows the byte value, not the encoding.) But in the person's text editor, chances are they will see what looks to them like a perfectly reasonable character, and have no idea which is the byte \xe2. > IOW > 1. The lexer is internally (evidently from the error message) so > ASCII-oriented that any ?unicode-junk? just defaults out to identifiers > (presumably comments are dealt with earlier) and then if that lexing action > fails it mistakenly pinpoints a wrong *identifier* rather than just an > impermissible character like python 2 You seem to be jumping to a rather large conclusion here. Even if you are right that the lexer considers all otherwise-unexpected characters to be part of an identifier, why is that a problem? I agree that it is mildly misleading to say invalid character '?' in identifier when ? is not part of an identifier: py> '?test'.isidentifier() False but I don't think you can jump from that to your conclusion that Python's unicode support is somewhat "wrongheaded". Surely a much simpler, less inflammatory response would be to say that this one specific error message could be improved? But... is it REALLY so bad? What if we wrote it like this instead: py> result = my?function(arg) File "", line 1 result = my?function(arg) ^ SyntaxError: invalid character in identifier Isn't it more reasonable to consider that "my?function" looks like it is intended as an identifier, but it happens to have an illegal character in it? > combine that with > 2. matrix mult (@) Ok to emulate perl but not to go outside ASCII How does @ emulate Perl? As for your second part, about not going outside of ASCII, yes, that is official policy for Python operators, keywords and builtins. > makes it seem (to me) python's unicode support is somewhat wrongheaded. -- Steve From mistersheik at gmail.com Tue Jul 19 07:36:06 2016 From: mistersheik at gmail.com (Neil Girdhar) Date: Tue, 19 Jul 2016 11:36:06 +0000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <20160719112027.GX27919@ando.pearwood.info> References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> Message-ID: On Tue, Jul 19, 2016 at 7:21 AM Steven D'Aprano wrote: > On Mon, Jul 18, 2016 at 10:29:34PM -0700, Rustom Mody wrote: > > > There was this question on the python list a few days ago: > > Subject: SyntaxError: Non-ASCII character > [...] > > I pointed out that the python2 error was more helpful (to my eyes) than > > python3s > > And I pointed out how I thought the Python 3 error message could be > improved, but the Python 2 error message was not very good. > > > > Python3 > > > > Traceback (most recent call last): > > File "", line 1, in > > File "/home/ariston/foo.py", line 31 > > wf = wave.open(?test.wav?, ?rb?) > > ^ > > SyntaxError: invalid character in identifier > > It would be much more helpful if the caret lined up with the offending > character. Better still, if the offending character was actually stated: > > wf = wave.open(?test.wav?, ?rb?) > ^ > SyntaxError: invalid character '?' in identifier > > > > Python2 > > > > Traceback (most recent call last): > > File "", line 1, in > > File "foo.py", line 31 > > SyntaxError: Non-ASCII character '\xe2' in file foo.py on line 31, but no > > encoding declared; see http://python.org/dev/peps/pep-0263/ for details > > As I pointed out earlier, this is less helpful. The line itself is not > shown (although the line number is given), nor is the offending > character. (Python 2 can't show the character because it doesn't know > what it is -- it only knows the byte value, not the encoding.) But in > the person's text editor, chances are they will see what looks to them > like a perfectly reasonable character, and have no idea which is the > byte \xe2. > > > > > IOW > > 1. The lexer is internally (evidently from the error message) so > > ASCII-oriented that any ?unicode-junk? just defaults out to identifiers > > (presumably comments are dealt with earlier) and then if that lexing > action > > fails it mistakenly pinpoints a wrong *identifier* rather than just an > > impermissible character like python 2 > > You seem to be jumping to a rather large conclusion here. Even if you > are right that the lexer considers all otherwise-unexpected characters > to be part of an identifier, why is that a problem? > It's a problem because those characters could never be part of an identifier. So it seems like a bug. > > I agree that it is mildly misleading to say > > invalid character '?' in identifier > > when ? is not part of an identifier: > > py> '?test'.isidentifier() > False > > but I don't think you can jump from that to your conclusion that > Python's unicode support is somewhat "wrongheaded". Surely a much > simpler, less inflammatory response would be to say that this one > specific error message could be improved? > > But... is it REALLY so bad? What if we wrote it like this instead: > > py> result = my?function(arg) > File "", line 1 > result = my?function(arg) > ^ > SyntaxError: invalid character in identifier > > Isn't it more reasonable to consider that "my?function" looks like it is > intended as an identifier, but it happens to have an illegal character > in it? > > > combine that with > > 2. matrix mult (@) Ok to emulate perl but not to go outside ASCII > > How does @ emulate Perl? > > As for your second part, about not going outside of ASCII, yes, that is > official policy for Python operators, keywords and builtins. > > > > makes it seem (to me) python's unicode support is somewhat wrongheaded. > > > -- > Steve > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > -- > > --- > You received this message because you are subscribed to a topic in the > Google Groups "python-ideas" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/python-ideas/-gsjDSht8VU/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > python-ideas+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Jul 19 07:51:04 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 19 Jul 2016 21:51:04 +1000 Subject: [Python-ideas] @Override decorator In-Reply-To: References: <20160719035858.GW27919@ando.pearwood.info> Message-ID: <20160719115104.GY27919@ando.pearwood.info> On Tue, Jul 19, 2016 at 03:18:34PM +1000, Nick Coghlan wrote: > On 19 July 2016 at 14:04, Guido van Rossum wrote in response to my comments: > > The same argument would apply against abstractmethod... Well, to be honest, I've never been in a situation where I either needed abstractmethod or knowingly had to deal with a class that used it. Maybe I just don't use classes enough to see why this proposal is useful. > > I have definitely > > wondered when looking over some code whether it was defining or overriding a > > method. Isn't that one of the features of IDEs? (Which doesn't help those who aren't using an IDE.) > > But requiring people to always decorate overrides seems hard, unless > > we made it a property that can be enforced by a metaclass, maybe? [Nick] > This is trickier in Python than it is in Java, since we support > multiple inheritance, and we don't require that subclasses be drop-in > replacements for their parent class. Indeed. > The key difference I see between @abstractmethod and this proposal is > that @abstractmethod is a service that base class authors provide to > subclass authors to say "Hey, when subclassing this, you *need* to > override these, or your subclass won't work properly". In effect, it's > an implicit test case for all future subclass definitions. > > By contrast, this proposal would go in the other direction: it would > be an assertion by the author of a *subclass* that that particular > method isn't new, it's inherited from a parent class, and they're > replacing the existing implementation with a new one. It would only > add new information for readers in cases where the method wasn't > already calling super() (since the latter already implies you expect > there to be at least one further implementation along the MRO to > call). > > In my view, that puts it more in the category of type annotations and > the "missing attribute" detection in tools like pylint than it does > abstractmethod: rather than checking it at runtime, you'd want static > checkers like mypy to flag it as a structural error if a method > declared as replacing a base class method wasn't actually doing so. Certainly this is something which static checkers *could* do, but surely a runtime check would be sufficient? The runtime check only needs to occur when the class is defined, not on every call. It could look something like this: def override(method): proxy = get_class_proxy() # MAGIC GOES HERE assert hasattr(method.__name__, proxy) return method I say MAGIC GOES HERE because of course the class doesn't exist yet. The get_class_proxy() function should return some sort of proxy to the soon-to-exist class' MRO. (Maybe this is harder to write than I thought.) Having it work at class definition time rather than compile-time means that it will work with odd corner cases like injected methods: class C: pass ... ... C.method = something() class D(C): @override def method(self): pass I wouldn't expect a purely static checker to necessarily realise that C.method exists, but there's no problem for the runtime check. > So a "typing.override" to complement "typing.overload" would make > sense to me, but a checked-at-runtime decorator wouldn't (doing it > that way would also let you put the override annotation in stub files > instead of directly in the code). -- Steve From mistersheik at gmail.com Tue Jul 19 08:11:50 2016 From: mistersheik at gmail.com (Neil Girdhar) Date: Tue, 19 Jul 2016 12:11:50 +0000 Subject: [Python-ideas] @Override decorator In-Reply-To: <20160719115104.GY27919@ando.pearwood.info> References: <20160719035858.GW27919@ando.pearwood.info> <20160719115104.GY27919@ando.pearwood.info> Message-ID: On Tue, Jul 19, 2016 at 7:51 AM Steven D'Aprano wrote: > On Tue, Jul 19, 2016 at 03:18:34PM +1000, Nick Coghlan wrote: > > > On 19 July 2016 at 14:04, Guido van Rossum wrote > in response to my comments: > > > > The same argument would apply against abstractmethod... > > Well, to be honest, I've never been in a situation where I either needed > abstractmethod or knowingly had to deal with a class that used it. > > Maybe I just don't use classes enough to see why this proposal is > useful. > > > > I have definitely > > > wondered when looking over some code whether it was defining or > overriding a > > > method. > > Isn't that one of the features of IDEs? > > (Which doesn't help those who aren't using an IDE.) > > > > > But requiring people to always decorate overrides seems hard, unless > > > we made it a property that can be enforced by a metaclass, maybe? > > [Nick] > > This is trickier in Python than it is in Java, since we support > > multiple inheritance, and we don't require that subclasses be drop-in > > replacements for their parent class. > > Indeed. > > > > The key difference I see between @abstractmethod and this proposal is > > that @abstractmethod is a service that base class authors provide to > > subclass authors to say "Hey, when subclassing this, you *need* to > > override these, or your subclass won't work properly". In effect, it's > > an implicit test case for all future subclass definitions. > > > > By contrast, this proposal would go in the other direction: it would > > be an assertion by the author of a *subclass* that that particular > > method isn't new, it's inherited from a parent class, and they're > > replacing the existing implementation with a new one. It would only > > add new information for readers in cases where the method wasn't > > already calling super() (since the latter already implies you expect > > there to be at least one further implementation along the MRO to > > call). > > > > In my view, that puts it more in the category of type annotations and > > the "missing attribute" detection in tools like pylint than it does > > abstractmethod: rather than checking it at runtime, you'd want static > > checkers like mypy to flag it as a structural error if a method > > declared as replacing a base class method wasn't actually doing so. > > Certainly this is something which static checkers *could* do, but surely > a runtime check would be sufficient? The runtime check only needs to > occur when the class is defined, not on every call. It could look > something like this: > > def override(method): > proxy = get_class_proxy() # MAGIC GOES HERE > assert hasattr(method.__name__, proxy) > return method > > I say MAGIC GOES HERE because of course the class doesn't exist yet. The > get_class_proxy() function should return some sort of proxy to the > soon-to-exist class' MRO. > > > (Maybe this is harder to write than I thought.) > > I don't think you need any magic. You can either do it like @abstractmethod whereby you put the logic in ABCMeta or given that PEP 487 has been accepted, put the logic in a plain old base class (ABCBase?) that has the logic in __init_subclass__. This would work with your corner case. It also is a bit more flexible. Having it work at class definition time rather than compile-time means > that it will work with odd corner cases like injected methods: > > class C: > pass > > ... > ... > C.method = something() > > class D(C): > @override > def method(self): > pass > > > I wouldn't expect a purely static checker to necessarily realise that > C.method exists, but there's no problem for the runtime check. > > > > So a "typing.override" to complement "typing.overload" would make > > sense to me, but a checked-at-runtime decorator wouldn't (doing it > > that way would also let you put the override annotation in stub files > > instead of directly in the code). > > > > -- > Steve > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > -- > > --- > You received this message because you are subscribed to a topic in the > Google Groups "python-ideas" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/python-ideas/V6sUaArzGC0/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > python-ideas+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rustompmody at gmail.com Tue Jul 19 08:05:11 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 19 Jul 2016 05:05:11 -0700 (PDT) Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> Message-ID: On Tuesday, July 19, 2016 at 5:06:17 PM UTC+5:30, Neil Girdhar wrote: > > > On Tue, Jul 19, 2016 at 7:21 AM Steven D'Aprano wrote: > >> On Mon, Jul 18, 2016 at 10:29:34PM -0700, Rustom Mody wrote: >> >> > IOW >> > 1. The lexer is internally (evidently from the error message) so >> > ASCII-oriented that any ?unicode-junk? just defaults out to identifiers >> > (presumably comments are dealt with earlier) and then if that lexing >> action >> > fails it mistakenly pinpoints a wrong *identifier* rather than just an >> > impermissible character like python 2 >> >> You seem to be jumping to a rather large conclusion here. Even if you >> are right that the lexer considers all otherwise-unexpected characters >> to be part of an identifier, why is that a problem? >> > > It's a problem because those characters could never be part of an > identifier. So it seems like a bug. > An armchair-design solution would say: We should give the most appropriate answer for every possible unicode character category This would need to take all the Unicode character-categories and Python lexical-categories and 'cross-product' them ? a humongous task to little advantage A more practical solution would be to take the best of the python2 and python3 current approaches: "Invalid character XX in line YY" and just reveal nothing about what lexical category ? like identifier ? python thinks the char is coming in. The XX is like python2 and the YY like python3 If it can do better than '\xe2' ? ie a codepoint ? that?s a bonus but not strictly necessary -------------- next part -------------- An HTML attachment was scrubbed... URL: From mistersheik at gmail.com Tue Jul 19 10:11:27 2016 From: mistersheik at gmail.com (Neil Girdhar) Date: Tue, 19 Jul 2016 14:11:27 +0000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> Message-ID: On Tue, Jul 19, 2016 at 8:18 AM Rustom Mody wrote: > > On Tuesday, July 19, 2016 at 5:06:17 PM UTC+5:30, Neil Girdhar wrote: >> >> >> On Tue, Jul 19, 2016 at 7:21 AM Steven D'Aprano wrote: >> > On Mon, Jul 18, 2016 at 10:29:34PM -0700, Rustom Mody wrote: >>> >>> > IOW >>> > 1. The lexer is internally (evidently from the error message) so >>> > ASCII-oriented that any ?unicode-junk? just defaults out to identifiers >>> > (presumably comments are dealt with earlier) and then if that lexing >>> action >>> > fails it mistakenly pinpoints a wrong *identifier* rather than just an >>> > impermissible character like python 2 >>> >>> You seem to be jumping to a rather large conclusion here. Even if you >>> are right that the lexer considers all otherwise-unexpected characters >>> to be part of an identifier, why is that a problem? >>> >> >> It's a problem because those characters could never be part of an >> identifier. So it seems like a bug. >> > > An armchair-design solution would say: We should give the most appropriate > answer for every possible unicode character category > This would need to take all the Unicode character-categories and Python > lexical-categories and 'cross-product' them ? a humongous task to little > advantage > I don't see why this is a "humongous task". Anyway, your solution boils down to the simplest fix in the lexer which is to block some characters from matching any category, does it not? > > A more practical solution would be to take the best of the python2 and > python3 current approaches: > "Invalid character XX in line YY" > and just reveal nothing about what lexical category ? like identifier ? > python thinks the char is coming in. > > The XX is like python2 and the YY like python3 > If it can do better than '\xe2' ? ie a codepoint ? that?s a bonus but not > strictly necessary > > -- > > --- > You received this message because you are subscribed to a topic in the > Google Groups "python-ideas" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/python-ideas/-gsjDSht8VU/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > python-ideas+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > -- > > --- > You received this message because you are subscribed to a topic in the > Google Groups "python-ideas" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/python-ideas/-gsjDSht8VU/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > python-ideas+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rustompmody at gmail.com Tue Jul 19 10:40:42 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 19 Jul 2016 07:40:42 -0700 (PDT) Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> Message-ID: <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> On Tuesday, July 19, 2016 at 7:41:38 PM UTC+5:30, Neil Girdhar wrote: > > On Tue, Jul 19, 2016 at 8:18 AM Rustom Mody wrote: > >> >> On Tuesday, July 19, 2016 at 5:06:17 PM UTC+5:30, Neil Girdhar wrote: >>> >>> >>> On Tue, Jul 19, 2016 at 7:21 AM Steven D'Aprano wrote: >>> >> On Mon, Jul 18, 2016 at 10:29:34PM -0700, Rustom Mody wrote: >>>> >>>> > IOW >>>> > 1. The lexer is internally (evidently from the error message) so >>>> > ASCII-oriented that any ?unicode-junk? just defaults out to >>>> identifiers >>>> > (presumably comments are dealt with earlier) and then if that lexing >>>> action >>>> > fails it mistakenly pinpoints a wrong *identifier* rather than just an >>>> > impermissible character like python 2 >>>> >>>> You seem to be jumping to a rather large conclusion here. Even if you >>>> are right that the lexer considers all otherwise-unexpected characters >>>> to be part of an identifier, why is that a problem? >>>> >>> >>> It's a problem because those characters could never be part of an >>> identifier. So it seems like a bug. >>> >> >> An armchair-design solution would say: We should give the most >> appropriate answer for every possible unicode character category >> This would need to take all the Unicode character-categories and Python >> lexical-categories and 'cross-product' them ? a humongous task to little >> advantage >> > > I don't see why this is a "humongous task". Anyway, your solution boils > down to the simplest fix in the lexer which is to block some characters > from matching any category, does it not? > Block? Not sure what you mean? Nothing should change (in the simplest solution at least) apart from better error messages My suggested solution involved this: Currently the lexer ? basically an automaton ? reveals which state its in when it throws error involving "identifier" Suggested change: if in_ident_state: if current_char is allowable as ident_char: continue as before elif current_char is ASCII: Usual error else: throw error eliding the "in_ident state" else: as is... BTW after last post I tried some things and found other unsatisfactory (to me) behavior in this area; to wit: >>> x = 0o19 File "", line 1 x = 0o19 ^ SyntaxError: invalid syntax Of course the 9 cannot come in an octal constant but "Syntax Error"?? Seems a little over general My preferred fix: make a LexicalError sub exception to SyntaxError Rest should follow for both Disclaimer: I am a teacher and having a LexicalError category makes it nice to explain some core concepts However I understand there are obviously other more pressing priorities than to make python superlative as a CS-teaching language :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Jul 19 10:42:38 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 20 Jul 2016 00:42:38 +1000 Subject: [Python-ideas] @Override decorator In-Reply-To: References: <20160719035858.GW27919@ando.pearwood.info> Message-ID: On 19 July 2016 at 16:44, Neil Girdhar wrote: > Why can't @overrides be processed at definition time? I didn't say it can't, I said I think it shouldn't be, and it shouldn't be for the same reason we don't process type annotations at runtime: this proposal falls into the realm of gradual typing, and gradual typing is an *optional* behaviour for use on projects that are sufficiently large with high enough personnel churn to see the benefits that outweigh the cost in boilerplate. With PEP 484 and typing, we've already established the precedent that people shouldn't go rushing off to their favourite projects and submit low value PRs that just add annotations for the sake of adding annotations. The proposed decorator here is a different kind of semantic annotation, but it's still one that should only be added to a project if the existing contributors to that project think it will help them maintain the code. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From storchaka at gmail.com Tue Jul 19 10:43:49 2016 From: storchaka at gmail.com (Serhiy Storchaka) Date: Tue, 19 Jul 2016 17:43:49 +0300 Subject: [Python-ideas] Sequence views In-Reply-To: References: Message-ID: On 17.07.16 23:08, Michael Selik wrote: > On Sun, Jul 17, 2016, 3:22 PM Serhiy Storchaka > wrote: > > Maybe it's time to add a new module for sequence-specific functions > (seqtools?). It should contain at least two classes or fabric functions: > > 1. A view that represents a sliced subsequence. Lazy equivalent of > seq[start:end:step]. This feature is implemented in third-party module > dataview [1]. > > 2. A view that represents a linear sequence as 2D array. Iterating this > view emits non-intersecting chunks of the sequence. For example it can > be used for representing the bytes object as a sequence of 1-byte bytes > objects (as in 2.x), a generalized alternative to iterbytes() from PEP > 467 [2]. > > > NumPy slicing and reshaping sounds like it satisfies these requirements. > Does it not? NumPy have similar features, but they work with packed arrays of specific numeric types, not with general sequences (such as list or str). And NumPy is a large library, providing a number of features not needed for most Python users. From storchaka at gmail.com Tue Jul 19 10:48:16 2016 From: storchaka at gmail.com (Serhiy Storchaka) Date: Tue, 19 Jul 2016 17:48:16 +0300 Subject: [Python-ideas] Sequence views In-Reply-To: References: Message-ID: On 18.07.16 01:21, Wes Turner wrote: > There are a number of generic implementations of these sequence algorithms: > > * http://toolz.readthedocs.io/en/latest/api.html#itertoolz > * https://github.com/kachayev/fn.py#itertools-recipes > * http://funcy.readthedocs.io/en/stable/seqs.html > * http://docs.python.org/2/reference/expressions.html#slicings > * https://docs.python.org/2/library/itertools.html#itertools.islice > * https://docs.python.org/3/library/itertools.html#itertools.islice Aren't all these implementations works with iterables and iterators? > On Jul 17, 2016 3:23 PM, "Serhiy Storchaka" > > wrote: > > > > Maybe it's time to add a new module for sequence-specific functions > (seqtools?). It should contain at least two classes or fabric functions: > > > > 1. A view that represents a sliced subsequence. Lazy equivalent of > seq[start:end:step]. This feature is implemented in third-party module > dataview [1]. > > islice? The result of itertools.islice() is not a sequence. > > 2. A view that represents a linear sequence as 2D array. Iterating > this view emits non-intersecting chunks of the sequence. For example it > can be used for representing the bytes object as a sequence of 1-byte > bytes objects (as in 2.x), a generalized alternative to iterbytes() from > PEP 467 [2]. > > partition? > > http://toolz.readthedocs.io/en/latest/api.html#toolz.itertoolz.partition > _all toolz.itertoolz.partition() is just a generator. From mistersheik at gmail.com Tue Jul 19 10:51:11 2016 From: mistersheik at gmail.com (Neil Girdhar) Date: Tue, 19 Jul 2016 14:51:11 +0000 Subject: [Python-ideas] @Override decorator In-Reply-To: References: <20160719035858.GW27919@ando.pearwood.info> Message-ID: On Tue, Jul 19, 2016 at 10:42 AM Nick Coghlan wrote: > On 19 July 2016 at 16:44, Neil Girdhar wrote: > > Why can't @overrides be processed at definition time? > > I didn't say it can't, I said I think it shouldn't be, and it > shouldn't be for the same reason we don't process type annotations at > runtime: this proposal falls into the realm of gradual typing, and > gradual typing is an *optional* behaviour for use on projects that are > sufficiently large with high enough personnel churn to see the > benefits that outweigh the cost in boilerplate. > > With PEP 484 and typing, we've already established the precedent that > people shouldn't go rushing off to their favourite projects and submit > low value PRs that just add annotations for the sake of adding > annotations. The proposed decorator here is a different kind of > semantic annotation, but it's still one that should only be added to a > project if the existing contributors to that project think it will > help them maintain the code. > The problem I have with the type annotation behavior that you're proposing is that you lose the definition-time assertions that overrides could raise. Even in a small project like mine (20k lines), that would be a big asset for me to eliminate a whole class of bugs. Your second point about people submitting PRs whereby they add decorators to everything shouldn't be an argument against new features. No one is forced to use @overrides, and projects can decide for themselves whether want to use it. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Jul 19 10:54:13 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 20 Jul 2016 00:54:13 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> Message-ID: On 19 July 2016 at 22:05, Rustom Mody wrote: > A more practical solution would be to take the best of the python2 and > python3 current approaches: > "Invalid character XX in line YY" > and just reveal nothing about what lexical category ? like identifier ? > python thinks the char is coming in. There's historically been relatively little work put into designing the error messages coming out of the lexer, so if it's a task you're interested in stepping up and taking on, you could probably find someone willing to review the patches. But if you perceive "Volunteers used their time as efficiently as possible whilst fully Unicode enabling the CPython compilation toolchain, since it was a dependency that needed to be addressed in order to permit other more interesting changes, rather than an inherently rewarding activity in its own right" as "wrongheaded", you may want to spend some time considering the differences between community-driven and customer-driven development. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From steve at pearwood.info Tue Jul 19 10:55:16 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 20 Jul 2016 00:55:16 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> References: <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> Message-ID: <20160719145515.GZ27919@ando.pearwood.info> On Tue, Jul 19, 2016 at 07:40:42AM -0700, Rustom Mody wrote: > My suggested solution involved this: > Currently the lexer ? basically an automaton ? reveals which state its in > when it throws error involving "identifier" > Suggested change: > > if in_ident_state: > if current_char is allowable as ident_char: > continue as before > elif current_char is ASCII: > Usual error > else: > throw error eliding the "in_ident state" > else: > as is... I'm sorry, you've lost me. Is this pseudo-code (1) of the current CPython lexer, (2) what you imagine the current CPython lexer does, or (3) what you think it should do? Because you call it a "change", but you're only showing one state, so it's not clear if its the beginning or ending state. Basically I guess what I'm saying is that if you are suggesting a concrete change to the lexer, you should be more precise about what needs to actually change. > BTW after last post I tried some things and found other unsatisfactory (to > me) behavior in this area; to wit: > > >>> x = 0o19 > File "", line 1 > x = 0o19 > ^ > SyntaxError: invalid syntax > > Of course the 9 cannot come in an octal constant but "Syntax Error"?? > Seems a little over general > > My preferred fix: > make a LexicalError sub exception to SyntaxError What's the difference between a LexicalError and a SyntaxError? Under what circumstances is it important to distinguish between them? It would be nice to have a more descriptive error message, but why should I care whether the invalid syntax "0o19" is caught by a lexer or a parser or the byte-code generator or the peephole optimizer or something else? Really all I need to care about is: - it is invalid syntax; - why it is invalid syntax (9 is not a legal octal digit); - and preferably, that it is caught at compile-time rather than run-time. -- Steve From ncoghlan at gmail.com Tue Jul 19 11:03:21 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 20 Jul 2016 01:03:21 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> Message-ID: On 20 July 2016 at 00:40, Rustom Mody wrote: > Disclaimer: I am a teacher and having a LexicalError category makes it nice > to explain some core concepts > However I understand there are obviously other more pressing priorities than > to make python superlative as a CS-teaching language :-) Given the motives of some of the volunteers in the community, "I am a teacher, the current error confuses my students, and I'm willing to discuss proposed alternative error messages with them to see if they're improvements" can actually be a good way to get people interested in helping out :) The reason that can help is that the main problem with "improving" error messages, is that it can be really hard to tell whether the improvements are actually improvements or not (in some cases it's obvious, but in others it's hard to decide when you've reached a point of "good enough", so you throw up your hands and say "Eh, it's at least distinctive enough that people will be able to find it on Stack Overflow"). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From rosuav at gmail.com Tue Jul 19 11:13:13 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 20 Jul 2016 01:13:13 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> Message-ID: On Wed, Jul 20, 2016 at 1:03 AM, Nick Coghlan wrote: > The reason that can help is that the main problem with "improving" > error messages, is that it can be really hard to tell whether the > improvements are actually improvements or not (in some cases it's > obvious, but in others it's hard to decide when you've reached a point > of "good enough", so you throw up your hands and say "Eh, it's at > least distinctive enough that people will be able to find it on Stack > Overflow"). Plus, there are all sorts of errors that look very different to humans, but identical to the parser. Steven showed us an example where an invalid character looked like it belonged in the identifier, yet to the parser it's just "this Unicode category isn't valid here", same as the bad-quote one. In a *very* few situations, a single error is common enough to be worth special-casing (eg print without parentheses, in recent Pythons), but otherwise, Stack Overflow or python-list will do a far better job of diagnosis than the lexer ever can. Ultimately, syntax errors represent an error in translation from a human's concept to the text that's given to the interpreter, and trying to reconstruct the concept from the errant text is better done by a human than a computer. Obviously it's great when the computer can read your mind, but it's never going to be perfect. ChrisA From michael.selik at gmail.com Tue Jul 19 11:32:40 2016 From: michael.selik at gmail.com (Michael Selik) Date: Tue, 19 Jul 2016 15:32:40 +0000 Subject: [Python-ideas] @Override decorator In-Reply-To: References: <20160719035858.GW27919@ando.pearwood.info> Message-ID: On Tue, Jul 19, 2016, 9:51 AM Neil Girdhar wrote: > On Tue, Jul 19, 2016 at 10:42 AM Nick Coghlan wrote: > >> On 19 July 2016 at 16:44, Neil Girdhar wrote: >> > Why can't @overrides be processed at definition time? >> >> I didn't say it can't, I said I think it shouldn't be, and it >> shouldn't be for the same reason we don't process type annotations at >> runtime: this proposal falls into the realm of gradual typing, and >> gradual typing is an *optional* behaviour for use on projects that are >> sufficiently large with high enough personnel churn to see the >> benefits that outweigh the cost in boilerplate. >> >> With PEP 484 and typing, we've already established the precedent that >> people shouldn't go rushing off to their favourite projects and submit >> low value PRs that just add annotations for the sake of adding >> annotations. The proposed decorator here is a different kind of >> semantic annotation, but it's still one that should only be added to a >> project if the existing contributors to that project think it will >> help them maintain the code. >> > > The problem I have with the type annotation behavior that you're proposing > is that you lose the definition-time assertions that overrides could > raise. Even in a small project like mine (20k lines), that would be a big > asset for me to eliminate a whole class of bugs. > Could you explain that further? Are you saying you need an override check for dynamically created/modified parent classes such that static analysis would not suffice? Your second point about people submitting PRs whereby they add decorators > to everything shouldn't be an argument against new features. No one is > forced to use @overrides, and projects can decide for themselves whether > want to use it. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Tue Jul 19 13:08:13 2016 From: wes.turner at gmail.com (Wes Turner) Date: Tue, 19 Jul 2016 12:08:13 -0500 Subject: [Python-ideas] @Override decorator In-Reply-To: References: <20160719035858.GW27919@ando.pearwood.info> Message-ID: On Jul 19, 2016 10:43 AM, "Nick Coghlan" wrote: > > On 19 July 2016 at 16:44, Neil Girdhar wrote: > > Why can't @overrides be processed at definition time? > > I didn't say it can't, I said I think it shouldn't be, and it > shouldn't be for the same reason we don't process type annotations at > runtime: this proposal falls into the realm of gradual typing, and > gradual typing is an *optional* behaviour for use on projects that are > sufficiently large with high enough personnel churn to see the > benefits that outweigh the cost in boilerplate. Is there enough information, in all cases, to do the check at definition time? - Obviously this wouldnt work for AOP-style dynamic mixins which are/could be modified by changing {__bases__, __class__, ...?} at runtime (certainly an infrequent use case and a different style of composition than @override-relevant inheritance) - __new__ > > With PEP 484 and typing, we've already established the precedent that > people shouldn't go rushing off to their favourite projects and submit > low value PRs that just add annotations for the sake of adding > annotations. The proposed decorator here is a different kind of > semantic annotation, but it's still one that should only be added to a > project if the existing contributors to that project think it will > help them maintain the code. What reasons are there for @override to be in stdlib?: - no import required (though it would still be necessary for backwards compatibility) - cross-cutting concerns - definition-time type verificayions - runtime type verifications aside: Spring Python may be of interest to people more familiar with Java > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Tue Jul 19 13:27:14 2016 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 19 Jul 2016 18:27:14 +0100 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> Message-ID: On 2016-07-19 16:13, Chris Angelico wrote: > On Wed, Jul 20, 2016 at 1:03 AM, Nick Coghlan wrote: >> The reason that can help is that the main problem with "improving" >> error messages, is that it can be really hard to tell whether the >> improvements are actually improvements or not (in some cases it's >> obvious, but in others it's hard to decide when you've reached a point >> of "good enough", so you throw up your hands and say "Eh, it's at >> least distinctive enough that people will be able to find it on Stack >> Overflow"). > > Plus, there are all sorts of errors that look very different to > humans, but identical to the parser. Steven showed us an example where > an invalid character looked like it belonged in the identifier, yet to > the parser it's just "this Unicode category isn't valid here", same as > the bad-quote one. In a *very* few situations, a single error is > common enough to be worth special-casing (eg print without > parentheses, in recent Pythons), but otherwise, Stack Overflow or > python-list will do a far better job of diagnosis than the lexer ever > can. Ultimately, syntax errors represent an error in translation from > a human's concept to the text that's given to the interpreter, and > trying to reconstruct the concept from the errant text is better done > by a human than a computer. Obviously it's great when the computer can > read your mind, but it's never going to be perfect. > Unicode has a couple of properties called "ID_Start" and "ID_Continue". The codepoint '?' doesn't match either of them, which is a good hint that Python shouldn't really be saying "invalid character in identifier" (it's the first character, but it can't be part of an identifier). From matt at getpattern.com Tue Jul 19 14:25:52 2016 From: matt at getpattern.com (Matt Gilson) Date: Tue, 19 Jul 2016 11:25:52 -0700 Subject: [Python-ideas] Sequence views In-Reply-To: References: Message-ID: On Tue, Jul 19, 2016 at 7:48 AM, Serhiy Storchaka wrote: > On 18.07.16 01:21, Wes Turner wrote: > >> There are a number of generic implementations of these sequence >> algorithms: >> >> * http://toolz.readthedocs.io/en/latest/api.html#itertoolz >> * https://github.com/kachayev/fn.py#itertools-recipes >> * http://funcy.readthedocs.io/en/stable/seqs.html >> * http://docs.python.org/2/reference/expressions.html#slicings >> * https://docs.python.org/2/library/itertools.html#itertools.islice >> * https://docs.python.org/3/library/itertools.html#itertools.islice >> > > Aren't all these implementations works with iterables and iterators? > > On Jul 17, 2016 3:23 PM, "Serhiy Storchaka" >> > > wrote: >> > >> > Maybe it's time to add a new module for sequence-specific functions >> (seqtools?). It should contain at least two classes or fabric functions: >> > >> > 1. A view that represents a sliced subsequence. Lazy equivalent of >> seq[start:end:step]. This feature is implemented in third-party module >> dataview [1]. >> >> islice? >> > > The result of itertools.islice() is not a sequence. Additionally, itertools.islice doesn't support negative indexing and it's O(start) to get the first element rather than the O(1) that it could be for sequences. >>> import itertools >>> x = range(30) >>> itertools.islice(x, -2, None) Traceback (most recent call last): File "", line 1, in ValueError: Indices for islice() must be None or an integer: 0 <= x <= maxint. With that said, I've never really worked with sequences that are big enough for the runtime complexity of normal python slicing to actually matter. I have a feeling that in the typical case, wrapping more abstraction around slicing and creating lazy "views" wouldn't lead to any practical performance benefits. For the cases where it *does* lead to practical performance benefits, `numpy` starts to look a whole lot more attractive as an option. I wonder how many applications would actually benefit from this but can't/shouldn't switch to `numpy` due to other constraints? > > > 2. A view that represents a linear sequence as 2D array. Iterating >> this view emits non-intersecting chunks of the sequence. For example it >> can be used for representing the bytes object as a sequence of 1-byte >> bytes objects (as in 2.x), a generalized alternative to iterbytes() from >> PEP 467 [2]. >> >> partition? >> >> http://toolz.readthedocs.io/en/latest/api.html#toolz.itertoolz.partition >> _all >> > > toolz.itertoolz.partition() is just a generator. > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- [image: pattern-sig.png] Matt Gilson // SOFTWARE ENGINEER E: matt at getpattern.com // P: 603.892.7736 We?re looking for beta testers. Go here to sign up! -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.selik at gmail.com Tue Jul 19 14:53:54 2016 From: michael.selik at gmail.com (Michael Selik) Date: Tue, 19 Jul 2016 18:53:54 +0000 Subject: [Python-ideas] @Override decorator In-Reply-To: References: <20160719035858.GW27919@ando.pearwood.info> Message-ID: On Tue, Jul 19, 2016, 12:08 PM Wes Turner wrote: > On Jul 19, 2016 10:43 AM, "Nick Coghlan" wrote: > > > > On 19 July 2016 at 16:44, Neil Girdhar wrote: > > > Why can't @overrides be processed at definition time? > > > > I didn't say it can't, I said I think it shouldn't be, and it > > shouldn't be for the same reason we don't process type annotations at > > runtime: this proposal falls into the realm of gradual typing, and > > gradual typing is an *optional* behaviour for use on projects that are > > sufficiently large with high enough personnel churn to see the > > benefits that outweigh the cost in boilerplate. > > Is there enough information, in all cases, to do the check at definition > time? > > - Obviously this wouldnt work for AOP-style dynamic mixins which are/could > be modified by changing {__bases__, __class__, ...?} at runtime > This is an argument for adding this feature to the typing module as opposed to a decorator, correct? Changing the base dynamically would not trigger another call to @overrides, even with some fancy metaclassing. The check wouldn't occur until instantiation or calling the method. Going for instantiation-time checks could get awkward via metaclass. To avoid conflicts I guess OverrideMeta would be an extension of ABCMeta. If just using __new__ it's too easy to accidentally bypass. Call-time checks would add overhead to every method call. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Tue Jul 19 15:14:50 2016 From: wes.turner at gmail.com (Wes Turner) Date: Tue, 19 Jul 2016 14:14:50 -0500 Subject: [Python-ideas] Sequence views In-Reply-To: References: Message-ID: On Jul 19, 2016 10:51 AM, "Serhiy Storchaka" wrote: > > On 18.07.16 01:21, Wes Turner wrote: >> >> There are a number of generic implementations of these sequence algorithms: >> >> * http://toolz.readthedocs.io/en/latest/api.html#itertoolz >> * https://github.com/kachayev/fn.py#itertools-recipes >> * http://funcy.readthedocs.io/en/stable/seqs.html >> * http://docs.python.org/2/reference/expressions.html#slicings >> * https://docs.python.org/2/library/itertools.html#itertools.islice >> * https://docs.python.org/3/library/itertools.html#itertools.islice > > > Aren't all these implementations works with iterables and iterators? > >> On Jul 17, 2016 3:23 PM, "Serhiy Storchaka" >> > > wrote: >> > >> > Maybe it's time to add a new module for sequence-specific functions >> (seqtools?). It should contain at least two classes or fabric functions: >> > >> > 1. A view that represents a sliced subsequence. Lazy equivalent of >> seq[start:end:step]. This feature is implemented in third-party module >> dataview [1]. >> >> islice? > > > The result of itertools.islice() is not a sequence. > > >> > 2. A view that represents a linear sequence as 2D array. Iterating >> this view emits non-intersecting chunks of the sequence. For example it >> can be used for representing the bytes object as a sequence of 1-byte >> bytes objects (as in 2.x), a generalized alternative to iterbytes() from >> PEP 467 [2]. >> >> partition? >> >> http://toolz.readthedocs.io/en/latest/api.html#toolz.itertoolz.partition >> _all > > > toolz.itertoolz.partition() is just a generator. so you're looking for something like strided memoryviews for nonsequential access over sequences/iterables which are sequential? sort of like a bitmask? https://docs.python.org/3/library/stdtypes.html#memoryview > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mistersheik at gmail.com Wed Jul 20 00:01:02 2016 From: mistersheik at gmail.com (Neil Girdhar) Date: Wed, 20 Jul 2016 04:01:02 +0000 Subject: [Python-ideas] @Override decorator In-Reply-To: References: <20160719035858.GW27919@ando.pearwood.info> Message-ID: You're right (and so is Nick): the static analysis would suffice. I guess you don't get the automatic docstring copying. My personal feeling is that it is also nice to have (even if I don't use it) support for dynamically created/modified parent classes. Best, Neil On Tue, Jul 19, 2016 at 11:32 AM Michael Selik wrote: > > > On Tue, Jul 19, 2016, 9:51 AM Neil Girdhar wrote: > >> On Tue, Jul 19, 2016 at 10:42 AM Nick Coghlan wrote: >> >>> On 19 July 2016 at 16:44, Neil Girdhar wrote: >>> > Why can't @overrides be processed at definition time? >>> >>> I didn't say it can't, I said I think it shouldn't be, and it >>> shouldn't be for the same reason we don't process type annotations at >>> runtime: this proposal falls into the realm of gradual typing, and >>> gradual typing is an *optional* behaviour for use on projects that are >>> sufficiently large with high enough personnel churn to see the >>> benefits that outweigh the cost in boilerplate. >>> >>> With PEP 484 and typing, we've already established the precedent that >>> people shouldn't go rushing off to their favourite projects and submit >>> low value PRs that just add annotations for the sake of adding >>> annotations. The proposed decorator here is a different kind of >>> semantic annotation, but it's still one that should only be added to a >>> project if the existing contributors to that project think it will >>> help them maintain the code. >>> >> >> The problem I have with the type annotation behavior that you're >> proposing is that you lose the definition-time assertions that overrides >> could raise. Even in a small project like mine (20k lines), that would be >> a big asset for me to eliminate a whole class of bugs. >> > > Could you explain that further? Are you saying you need an override check > for dynamically created/modified parent classes such that static analysis > would not suffice? > > Your second point about people submitting PRs whereby they add decorators >> to everything shouldn't be an argument against new features. No one is >> forced to use @overrides, and projects can decide for themselves whether >> want to use it. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From turnbull.stephen.fw at u.tsukuba.ac.jp Wed Jul 20 12:44:12 2016 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 21 Jul 2016 01:44:12 +0900 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> Message-ID: <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> Nick Coghlan writes: > The reason that can help is that the main problem with "improving" > error messages, is that it can be really hard to tell whether the > improvements are actually improvements or not Personally, I think the real issue here is that the curly quote (and things like mathematical PRIME character) are easily confused with Python syntax and it all looks like grit on Tim's monitor. I tried substituting an emoticon and the DOUBLE INTEGRAL, and it was quite obvious what was wrong from the Python 3 error message. However, in this case, as far as I can tell from the error messages induced by playing with ASCII, Python 3.5 thinks that all non- identifier ASCII characters are syntactic (so for example it says that with open($file.txt") as f: is "invalid syntax"). But for non-ASCII characters (I guess including the Latin 1 set?) they are either letters, numerals, or just plain not valid in a Python program AIUI (outside of strings and comments, of course). I would think the lexer could just treat each invalid character as an invalid_token, which is always invalid in Python syntax, and the error would be a SyntaxError with the message formatted something like "invalid character {} = U+{:04X}".format(ch, ord(ch)) This should avoid the strange placement of the position indicator, too. If someday we decide to use an non-ASCII character for a syntactic purpose, that's a big enough compatibility break in itself that changing the invalid character set (and thus the definition of invalid_token) is insignificant. I'm pretty sure this is what a couple of earlier posters have in mind, too. From danilo.bellini at gmail.com Wed Jul 20 17:16:10 2016 From: danilo.bellini at gmail.com (Danilo J. S. Bellini) Date: Wed, 20 Jul 2016 18:16:10 -0300 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> Message-ID: 1. Using SyntaxError for lexical errors sounds as strange as saying a misspell/typo is a syntax mistake in a natural language. A new "LexicalError" or "TokenizerError" for that makes sense. Perhaps both this new exception and SyntaxError should inherit from a new CompileError class. But the SyntaxError is already covering cases alike with the TabError (an IndentationError), which is a lexical analysis error, not a parser one [1]. To avoid such changes while keeping the name, at least the SyntaxError docstring should be "Compile-time error." instead of "Invalid Syntax.", and the documentation should be explicit that it isn't only about parsing/syntax/grammar but also about lexical analysis errors. 2. About those lexical error messages, the caret is worse than the lack of it when it's not aligned, but unless I'm missing something, one can't guarantee that the terminal is printing the error message with the right encoding. Including the row and column numbers in the message would be helpful. 3. There are people who like and use unicode chars in identifiers. Usually I don't like to translate comments/identifiers to another language, but I did so myself, using variable names with accents in Portuguese for a talk [2], mostly to give it a try. Surprisingly, few people noticed that until I said. The same can be said about Sympy scripts, where symbols like Greek letters would be meaningful (e.g. ? for the mean, ? for the standard deviation and ? for the covariance matrix), so I'd argue it's quite natural. 4. Unicode have more than one codepoint for some symbols that look alike, for example "??????" are all valid uppercase sigmas. There's also "?", but this one is invalid in Python 3. The italic/bold/serif distinction seems enough for a distinction, and when editing a code with an Unicode char like that, most people would probably copy and paste the symbol instead of typing it, leading to a consistent use of the same symbol. 5. New keywords, no matter whether they fit into the 7-bit ASCII or requires Unicode, unavoidably breaks backwards compatibility at least to some degree. That happened with the "nonlocal" keyword in Python 3, for example. 6. Python 3 code is UTF-8 and Unicode identifiers are allowed. Not having Unicode keywords is merely contingent on Python 2 behavior that emphasized ASCII-only code (besides comments and strings). 7. The discussion isn't about lambda or anti-lambda bias, it's about keyword naming and readability. Who gains/loses with that resource? It won't hurt those who never uses lambda and never uses Unicode identifiers. Perhaps Sympy users would feel harmed by that, as well as other scientific packages users, but looking for the "?" char in GitHub I found no one using it alone within Python code. The online Python books written in Greek that I found were using only English identifiers. 8. I don't know if any consensus can emerge in this matter about lambdas, but there's another subject that can be discussed together: macros. What OP wants is exactly a "#define ? lambda", which would be only in the code that uses/needs such symbol with that meaning. A minimal lexical macro that just apply a single keyword token replacement by a identifier-like token is enough for him. I don't know a nice way to do that, something like "from __replace__ import lambda_to_?" or even "def ? is lambda" would avoid new keywords, but I also don't know how desired this resource is (perhaps to translate the language keywords to another language?). 7. I really don't like the editor "magic", it would be better to create a packaging/setup.py translation script than that (something like 2to3). It's not about coloring/highlighting, nor about editors/IDEs features, it's about seeing the object/file itself, and colors never change that AFAIK. Also, most code I read isn't using my editor, sometimes it comes from cat/diff (terminal stdout output), vim/gedit/pluma (editor), GitHub/BitBucket (web), blogs/forums/e-mails, gitk, Spyder (IDE), etc.. That kind of "view" replacement would compromise some code alignment (e.g. multiline strings/comments) and line length, besides being a problem to look for code with tools like find + grep/sed/awk (which I use all the time). Still worse are the git hooks to perform the replacement before/after a commit: how should one test a code that uses that? It somehow feels out of control. [1] https://docs.python.org/3/reference/lexical_analysis.html [2] http://www.slideshare.net/djsbellini/20140416-garoa-hc-strategy 2016-07-20 13:44 GMT-03:00 Stephen J. Turnbull < turnbull.stephen.fw at u.tsukuba.ac.jp>: > Nick Coghlan writes: > > > The reason that can help is that the main problem with "improving" > > error messages, is that it can be really hard to tell whether the > > improvements are actually improvements or not > > Personally, I think the real issue here is that the curly quote (and > things like mathematical PRIME character) are easily confused with > Python syntax and it all looks like grit on Tim's monitor. I tried > substituting an emoticon and the DOUBLE INTEGRAL, and it was quite > obvious what was wrong from the Python 3 error message. > > However, in this case, as far as I can tell from the error messages > induced by playing with ASCII, Python 3.5 thinks that all non- > identifier ASCII characters are syntactic (so for example it says that > > with open($file.txt") as f: > > is "invalid syntax"). But for non-ASCII characters (I guess including > the Latin 1 set?) they are either letters, numerals, or just plain not > valid in a Python program AIUI (outside of strings and comments, of > course). > > I would think the lexer could just treat each invalid character as an > invalid_token, which is always invalid in Python syntax, and the error > would be a SyntaxError with the message formatted something like > > "invalid character {} = U+{:04X}".format(ch, ord(ch)) > > This should avoid the strange placement of the position indicator, > too. > > If someday we decide to use an non-ASCII character for a syntactic > purpose, that's a big enough compatibility break in itself that > changing the invalid character set (and thus the definition of > invalid_token) is insignificant. > > I'm pretty sure this is what a couple of earlier posters have in mind, > too. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Danilo J. S. Bellini --------------- "*It is not our business to set up prohibitions, but to arrive at conventions.*" (R. Carnap) -------------- next part -------------- An HTML attachment was scrubbed... URL: From pavol.lisy at gmail.com Thu Jul 21 00:53:52 2016 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Thu, 21 Jul 2016 06:53:52 +0200 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> Message-ID: On 7/20/16, Danilo J. S. Bellini wrote: > 4. Unicode have more than one codepoint for some symbols that look alike, > for example "??????" are all valid uppercase sigmas. There's also "?", > but this one is invalid in Python 3. The italic/bold/serif distinction > seems enough for a distinction, and when editing a code with an Unicode > char like that, most people would probably copy and paste the symbol > instead of typing it, leading to a consistent use of the same symbol. I am not sure what do you like to say, so for sure some info: PEP-3131 (https://www.python.org/dev/peps/pep-3131/): "All identifiers are converted into the normal form NFKC while parsing; comparison of identifiers is based on NFKC." >From this point of view all sigmas are same: set(unicodedata.normalize('NFKC', i) for i in "??????") == {'?'} From rustompmody at gmail.com Thu Jul 21 01:08:33 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 20 Jul 2016 22:08:33 -0700 (PDT) Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> Message-ID: <423dccda-a934-4aac-9ec4-55da54a01f6e@googlegroups.com> On Tuesday, July 19, 2016 at 8:26:41 PM UTC+5:30, Nick Coghlan wrote: > > But if you perceive "Volunteers used their time as efficiently as > > possible whilst fully Unicode enabling the CPython compilation > toolchain, since it was a dependency that needed to be addressed in > order to permit other more interesting changes, rather than an > inherently rewarding activity in its own right" as "wrongheaded", you > may want to spend some time considering the differences between > community-driven and customer-driven development. > > Hi Nick Sorry if I caused offense. Ive been using Python since around 2001 and its been a strikingly pleasant relationship. There have been surprisingly few times when python let me down in a class (Only exception I remember in all these years: https://mail.python.org/pipermail/python-list/2011-July/609369.html ) Which is a generally better record than most other languages. So I remain grateful to Guido and the devs for this pleasing creation. My ?wrongheaded? was (intended) quite narrow and technical: - The embargo on non-ASCII everywhere in the language except identifiers (strings and comments obviously dont count as ?in? the language - The opening of identifiers to large swathes of Unicode widens as you say hugely the surface area of attack This was solely the contradiction I was pointing out. -------------- next part -------------- An HTML attachment was scrubbed... URL: From danilo.bellini at gmail.com Thu Jul 21 02:13:14 2016 From: danilo.bellini at gmail.com (Danilo J. S. Bellini) Date: Thu, 21 Jul 2016 03:13:14 -0300 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> Message-ID: 2016-07-21 1:53 GMT-03:00 Pavol Lisy : > On 7/20/16, Danilo J. S. Bellini wrote: > > > 4. Unicode have more than one codepoint for some symbols that look alike, > > for example "??????" are all valid uppercase sigmas. There's also > "?", > > but this one is invalid in Python 3. The italic/bold/serif distinction > > seems enough for a distinction, and when editing a code with an Unicode > > char like that, most people would probably copy and paste the symbol > > instead of typing it, leading to a consistent use of the same symbol. > > I am not sure what do you like to say, so for sure some info: > > PEP-3131 (https://www.python.org/dev/peps/pep-3131/): "All identifiers > are converted into the normal form NFKC while parsing; comparison of > identifiers is based on NFKC." > > From this point of view all sigmas are same: > > set(unicodedata.normalize('NFKC', i) for i in "??????") == {'?'} > In this item I just said that most programmers would probably keep the same character in a source code file due to copying and pasting, and that even when it doesn't happen (the copy-and-paste action), visual differences like italic/bold/serif are enough for one to notice (when using another input method). At first, I was thinking on a code with one of those symbols as a variable name (any of them), but PEP3131 challenges that. Actually, any conversion to a normal form means that one should never use unicode identifiers outside the chosen normal form. It would be better to raise an error instead of converting. If there isn't any lint tool already complaining about that, I strongly believe that's something that should be done. When mixing strings and identifier names, that's not so predictable: >>> obj = type("SomeClass", (object,), {c: i for i, c in enumerate("??????")})() >>> obj.? == getattr(obj, "?") False >>> obj.? == getattr(obj, "?") True >>> dir(obj) [..., '?', '?', '?', '?', '?', '?'] -- Danilo J. S. Bellini --------------- "*It is not our business to set up prohibitions, but to arrive at conventions.*" (R. Carnap) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rustompmody at gmail.com Thu Jul 21 02:14:24 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 20 Jul 2016 23:14:24 -0700 (PDT) Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> Message-ID: <0672eafb-7d89-4769-a32f-2f075a9b3dc2@googlegroups.com> On Thursday, July 21, 2016 at 10:24:42 AM UTC+5:30, Pavol Lisy wrote: > > On 7/20/16, Danilo J. S. Bellini > > wrote: > > > 4. Unicode have more than one codepoint for some symbols that look alike, > > for example "??????" are all valid uppercase sigmas. There's also > "?", > > but this one is invalid in Python 3. The italic/bold/serif distinction > > seems enough for a distinction, and when editing a code with an Unicode > > char like that, most people would probably copy and paste the symbol > > instead of typing it, leading to a consistent use of the same symbol. > > I am not sure what do you like to say, so for sure some info: > > PEP-3131 (https://www.python.org/dev/peps/pep-3131/): "All identifiers > are converted into the normal form NFKC while parsing; comparison of > identifiers is based on NFKC." > > From this point of view all sigmas are same: > > set(unicodedata.normalize('NFKC', i) for i in "??????") == {'?'} > > Nice! >>> ? = 1 >>> ? = ? + 1 >>> ? 2 But not enough >>> ? = 1 >>> A = A + 1 Traceback (most recent call last): File "", line 1, in NameError: name 'A' is not defined Moral: The coarser the equivalence-relation the better (within reasonable limits!) NFKC-equality i coarser than literal-codepoint equality. ? Better But not coarse enough. After all identifiers are meant to identify! -------------- next part -------------- An HTML attachment was scrubbed... URL: From rustompmody at gmail.com Thu Jul 21 02:26:58 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 20 Jul 2016 23:26:58 -0700 (PDT) Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> Message-ID: <9b707450-5a57-4677-bd20-48e9f53cb87d@googlegroups.com> On Thursday, July 21, 2016 at 11:45:11 AM UTC+5:30, Danilo J. S. Bellini wrote: > > > 2016-07-21 1:53 GMT-03:00 Pavol Lisy >: > >> On 7/20/16, Danilo J. S. Bellini > >> wrote: >> >> > 4. Unicode have more than one codepoint for some symbols that look >> alike, >> > for example "??????" are all valid uppercase sigmas. There's also >> "?", >> > but this one is invalid in Python 3. The italic/bold/serif distinction >> > seems enough for a distinction, and when editing a code with an Unicode >> > char like that, most people would probably copy and paste the symbol >> > instead of typing it, leading to a consistent use of the same symbol. >> >> I am not sure what do you like to say, so for sure some info: >> >> PEP-3131 (https://www.python.org/dev/peps/pep-3131/): "All identifiers >> are converted into the normal form NFKC while parsing; comparison of >> identifiers is based on NFKC." >> >> From this point of view all sigmas are same: >> >> set(unicodedata.normalize('NFKC', i) for i in "??????") == {'?'} >> > > In this item I just said that most programmers would probably keep the > same character in a source code file due to copying and pasting, and that > even when it doesn't happen (the copy-and-paste action), visual differences > like italic/bold/serif are enough for one to notice (when using another > input method). > > At first, I was thinking on a code with one of those symbols as a variable > name (any of them), but PEP3131 challenges that. Actually, any conversion > to a normal form means that one should never use unicode identifiers > outside the chosen normal form. It would be better to raise an error > instead of converting. > Yes Agree I said ?Nice!? for >>> ? = 1 >>> ? = ? + 1 >>> ? 2 in comparison to: >>> ? = 1 >>> A = A + 1 because the A's look more indistinguishable than the sigmas and are internally more distinct If the choice is to simply disallow the confusables that?s probably the best choice IOW 1. Disallow co-existence of confusables (in identifiers) 2. Identify confusables to a normal form ? like case-insensitive comparison and like NKFC 3. Leave the confusables to confuse My choice 1 better than 2 better than 3 -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Thu Jul 21 03:20:38 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 21 Jul 2016 17:20:38 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <9b707450-5a57-4677-bd20-48e9f53cb87d@googlegroups.com> References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> <9b707450-5a57-4677-bd20-48e9f53cb87d@googlegroups.com> Message-ID: On Thu, Jul 21, 2016 at 4:26 PM, Rustom Mody wrote: > IOW > 1. Disallow co-existence of confusables (in identifiers) > 2. Identify confusables to a normal form ? like case-insensitive comparison > and like NKFC > 3. Leave the confusables to confuse > > My choice > 1 better than 2 better than 3 So should we disable the lowercase 'l', the uppercase 'I', and the digit '1', because they can be confused? What about the confusability of "m" and "rn"? O and 0 are similar in some fonts. And case insensitivity brings its own problems - is "ss" equivalent to "?", and is "?" equivalent to either? Turkish distinguishes between "i", which upper-cases to "?", and "?", which upper-cases to "I". We already have interminable debates about letter similarities across scripts. I'm sure everyone agrees that Cyrillic "?" is not the same letter as Latin "i", but we have "A??" in three different scripts. Should they be considered equivalent? I think not, because in any non-trivial context, you'll know whether the program's been written in Greek, a Slavic language, or something using the Latin script. But maybe you disagree. Okay; are "B??" all to be considered equivalent too? What about "?C"? "X???"? They're visually similar, but they're not equivalent in any other way. And if you're going to say things should be considered equivalent solely on the basis of visuals, you get into a minefield - should U+200B ZERO WIDTH SPACE be completely ignored, allowing "AB" to be equivalent to "A\u200bB" as an identifier? This debate should probably continue on python-list (if anywhere). I doubt Python is going to change its normalization rules any time soon, and if it does, it'll need a very solid reason (and probably a PEP with all the pros and cons). ChrisA From turnbull.stephen.fw at u.tsukuba.ac.jp Thu Jul 21 03:36:24 2016 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 21 Jul 2016 16:36:24 +0900 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> Message-ID: <22416.31608.704953.868567@turnbull.sk.tsukuba.ac.jp> Danilo J. S. Bellini writes: > 1. Using SyntaxError for lexical errors sounds as strange as saying a > misspell/typo is a syntax mistake in a natural language. Well, I find that many typos are discovered even though they look like (and often enough are) real words, with unacceptable semantics (sometimes even the same part of speech). So I don't find that analogy at all compelling -- human recognition of typos is far more complex than computer recognition of parse errors. And the Python lexer is very simple, even among translators. It creates tokens for operators which are more or less self-delimiting, indentation, strings, and failing that sequences of characters delimited by spaces, newlines, and operators. Token recognition is now complete. For tokens of as-yet unknown type, it then checks whether the token is a keyword, if not, is it a number. If not, in a syntactically correct program, what's left is an identifier (and I suppose that's why this error message says "identifier", and why it points to the end of the token, not the "bad" character). It then checks the putative identifier and discovers that the token isn't well-formed as an identifier. I think it's a very good idea to keep this tokenization process simple. So in my proposal, it's intentionally not a lexical error, but rather a new kind of self-delimiting token (with no syntactic role in correct programs). A lexical error means that the translator failed to construct (including identifying the syntactic role) a token. That's very bad. Theoretically speaking, that means all bets are off, who knows what the rest of the program might mean? Pragmatically, you can use heuristics to generate error messages and reset the lexer to an "appropriate" state, but as Nick points out, those heuristics are unreliable and may do more harm than good, and it's not clear what the appropriate reset state is. Making an invalid_token (perhaps a better name for current purposes would be invalid_character_token) means that there are no lexical errors (except for UnicodeErrors, but they are "below" the level of the language definition). This is consistent with current Python practice for pure ASCII programs: >>> a$b File "", line 1 a$b ^ SyntaxError: invalid syntax Note that the caret is in the right place, so '$' is being treated as an operator. (The same happens with '?', the other non-identifier non-control ASCII character without specified semantics.) The advantage is that the tokenized program has much more structure, and much more restricted valid structure that it can match (correct positioning of the caret is an immediate benefit, see below), than an untokenized string (remember, it's already known to contain errors). Of course you could implicitly do the same thing at the lexical level, but "explicit is better than implicit". Since we're trying to reason about invalid programs, the motivation is heuristic either way, but an explicit definition of invalid_token means that the processing by the translator is easier to understand, and it would restrict the ways that handling of this error could change in the future. I consider that restriction to be a good thing in this context, YMMV. > 2. About those lexical error messages, the caret is worse than the lack of > it when it's not aligned, but unless I'm missing something, one can't > guarantee that the terminal is printing the error message with the right > encoding. But it will print the character in the erroneous line and that character in the error message the same way, which should be enough (certainly will be enough in the "curly quotes" example). To identify the exact character that Python is concerned with (regardless of whether the glyphs in the error message are what the user sees in her editor) the Unicode scalar (or even the Unicode name, but that requires importing the Unicode character database which might be undesirable) is included. > Including the row and column numbers in the message would be > helpful. The line number is already there, the current tokenization process will set the column number to the place where the caret is. My proposal fixes this automatically without requiring Python to do more analysis than "end of token", which it already knows. > 6. Python 3 code is UTF-8 and Unicode identifiers are allowed. Not having > Unicode keywords is merely contingent on Python 2 behavior that > emphasized ASCII-only code (besides comments and strings). It's more than that. For better or worse, English is the natural language source for Python keywords (even "elif" is a contraction, and feels natural to this native speaker), and I can think of no variant of English where (plausible) candidate keywords can't be spelled with ASCII. "lambda" itself is the only plausible exception as far as I know, and even there "lambda calculus" is perfectly good English now. > 7. I really don't like the editor "magic", it would be better to create a > packaging/setup.py translation script than that (something like > 2to3). 2to3 can be used for this purpose, it's quite flexible about the rulesets that can be defined and specified. But note that that implies that adding this capability to the stdlib would fork the language within the CPython implemention, just as Python 3 is a fork from Python 2. That sounds like a bad idea to me -- some people have always complained that porting to Python 3 is almost like learning a new language, many people are already complaining that Python 3 is getting bigger than they like, and it would impose a burden on other implementations. > Still worse are the git hooks to perform the replacement > before/after a commit: how should one test a code that uses that? > It somehow feels out of control. Exactly. All of this discussion about providing an alias for "lambda" seems out of control, and as a 20-year veteran of Emacs development (where there is no way to make a clean distinction between language and stdlib, apparently nobody has ever heard of TOOWTDI, and 3-line hacks are regularly committed to the core code), it gives me a terrifying feeling of deja vu. Improving the message for invalid identifiers of this particular kind, OTOH, is a straightforward extension of the existing mechanism. From pavol.lisy at gmail.com Thu Jul 21 03:38:05 2016 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Thu, 21 Jul 2016 09:38:05 +0200 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> Message-ID: On 7/21/16, Danilo J. S. Bellini wrote: > 2016-07-21 1:53 GMT-03:00 Pavol Lisy : > >> set(unicodedata.normalize('NFKC', i) for i in "??????") == {'?'} >> > > In this item I just said that most programmers would probably keep the same > character in a source code file due to copying and pasting, and that even > when it doesn't happen (the copy-and-paste action), visual differences like > italic/bold/serif are enough for one to notice (when using another input > method). > > At first, I was thinking on a code with one of those symbols as a variable > name (any of them), but PEP3131 challenges that. Actually, any conversion > to a normal form means that one should never use unicode identifiers > outside the chosen normal form. It would be better to raise an error > instead of converting. If there isn't any lint tool already complaining > about that, I strongly believe that's something that should be done. When > mixing strings and identifier names, that's not so predictable: > >>>> obj = type("SomeClass", (object,), {c: i for i, c in > enumerate("??????")})() >>>> obj.? == getattr(obj, "?") > False >>>> obj.? == getattr(obj, "?") > True >>>> dir(obj) > [..., '?', '?', '?', '?', '?', '?'] [getattr(obj, i) for i in dir(obj) if i in "??????"] # [0, 1, 2, 3, 4, 5] but: [obj.?, obj.?, obj.?, obj.?, obj.?, obj.?, ] # [0, 0, 0, 0, 0, 0] So you could mix any of them while editing identifiers. (but you could not mix them while writing parameters in getattr, setattr and type) But getattr, setattr and type are other beasts, because they can use "non identifiers", non letter characters too: setattr(obj,'+', 7) dir(obj) # ['+', ...] # but obj.+ is syntax error setattr(obj,u"\udcb4", 7) dir(obj) # [..., '\udcb4' ,...] obj = type("SomeClass", (object,), {c: i for i, c in enumerate("+-*/")})() Maybe there is still some Babel curse here and some sort of normalize_dir, normalize_getattr, normalize_setattr, normalize_type could help? I am not sure. They probably make things more complicated than simpler. From ncoghlan at gmail.com Thu Jul 21 03:41:49 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 21 Jul 2016 17:41:49 +1000 Subject: [Python-ideas] =?utf-8?q?Error_handling_for_unknown_Unicode_chara?= =?utf-8?q?cters_=28was_Re=3A__allow_=60lambda=27_to_be_spelled_?= =?utf-8?b?zrsp?= Message-ID: On 21 July 2016 at 15:08, Rustom Mody wrote: > My ?wrongheaded? was (intended) quite narrow and technical: > > - The embargo on non-ASCII everywhere in the language except identifiers > (strings > and comments obviously dont count as ?in? the language > - The opening of identifiers to large swathes of Unicode widens as you say > hugely the surface area of attack > > This was solely the contradiction I was pointing out. OK, thanks for the clarification, and my apologies for jumping on you. I can be a bit hypersensitive on this topic, as my day job sometimes includes encouraging commercial redistributors and end users to stop taking community volunteers for granted and instead help find ways to ensure their work is sustainable :) As it is, I think there are some possible checks that could be added to the code generator pipeline to help clarify matters: - for the "invalid character" error message, we should be able to always report both the printed symbol *and* the ASCII hex escape, rather than assuming the caret will point to the correct place - the caret positioning logic for syntax errors needs to be checked to see if it's currently counting encoded UTF-8 bytes instead of code points (as that will consistently do the wrong thing on a correctly configured UTF-8 terminal) - (more speculatively) when building the symbol table, we may be able to look for identifiers referenced in a namespace that are not NKFC equivalent, but nevertheless qualify as Unicode confusables, and emit a SyntaxWarning (this is speculative, as I'm not sure what degree of performance hit would be associated with it) As far as Danilo's observation regarding the CPython code generator always emitting SyntaxError and SyntaxWarning (regardless of which part of the code generation actually failed) goes, I wouldn't be opposed to our getting more precise about that by defining additional subclasses, but one of the requirements would be for documentation in https://docs.python.org/devguide/compiler.html or helper functions in the source to clearly define "when working on part of the code generation pipeline, raise kind of error if something goes wrong". Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From rustompmody at gmail.com Thu Jul 21 03:47:37 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Thu, 21 Jul 2016 00:47:37 -0700 (PDT) Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> <9b707450-5a57-4677-bd20-48e9f53cb87d@googlegroups.com> Message-ID: On Thursday, July 21, 2016 at 12:51:27 PM UTC+5:30, Chris Angelico wrote: > > On Thu, Jul 21, 2016 at 4:26 PM, Rustom Mody > wrote: > > IOW > > 1. Disallow co-existence of confusables (in identifiers) > > 2. Identify confusables to a normal form ? like case-insensitive > comparison > > and like NKFC > > 3. Leave the confusables to confuse > > > > My choice > > 1 better than 2 better than 3 > > So should we disable the lowercase 'l', the uppercase 'I', and the > digit '1', because they can be confused? What about the confusability > of "m" and "rn"? O and 0 are similar in some fonts. And case > insensitivity brings its own problems - is "ss" equivalent to "?", and > is "?" equivalent to either? Turkish distinguishes between "i", which > upper-cases to "?", and "?", which upper-cases to "I". > > We already have interminable debates about letter similarities across > scripts. I'm sure everyone agrees that Cyrillic "?" is not the same > letter as Latin "i", but we have "A??" in three different scripts. > Should they be considered equivalent? I think not, because in any > non-trivial context, you'll know whether the program's been written in > Greek, a Slavic language, or something using the Latin script. But > maybe you disagree. Okay; are "B??" all to be considered equivalent > too? What about "?C"? "X???"? They're visually similar, but they're > not equivalent in any other way. And if you're going to say things > should be considered equivalent solely on the basis of visuals, you > get into a minefield - should U+200B ZERO WIDTH SPACE be completely > ignored, allowing "AB" to be equivalent to "A\u200bB" as an > identifier? > > I said 1 better than 2 better than 3 Maybe you also want to add: Special cases aren't special enough to break the rules. Although practicality beats purity. followed by Errors should never pass silently. IOW setting out 1 better than 2 better than 3 does not necessarily imply its completely achievable -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Thu Jul 21 03:54:27 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 21 Jul 2016 17:54:27 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> <9b707450-5a57-4677-bd20-48e9f53cb87d@googlegroups.com> Message-ID: On Thu, Jul 21, 2016 at 5:47 PM, Rustom Mody wrote: >> On Thu, Jul 21, 2016 at 4:26 PM, Rustom Mody wrote: >> > IOW >> > 1. Disallow co-existence of confusables (in identifiers) >> > 2. Identify confusables to a normal form ? like case-insensitive >> > comparison >> > and like NKFC >> > 3. Leave the confusables to confuse >> > >> > My choice >> > 1 better than 2 better than 3 >> >> So should we disable the lowercase 'l', the uppercase 'I', and the >> digit '1', because they can be confused? What about the confusability >> of "m" and "rn"? O and 0 are similar in some fonts. And case >> insensitivity brings its own problems - is "ss" equivalent to "?", and >> is "?" equivalent to either? Turkish distinguishes between "i", which >> upper-cases to "?", and "?", which upper-cases to "I". >> >> We already have interminable debates about letter similarities across >> scripts. I'm sure everyone agrees that Cyrillic "?" is not the same >> letter as Latin "i", but we have "A??" in three different scripts. >> Should they be considered equivalent? I think not, because in any >> non-trivial context, you'll know whether the program's been written in >> Greek, a Slavic language, or something using the Latin script. But >> maybe you disagree. Okay; are "B??" all to be considered equivalent >> too? What about "?C"? "X???"? They're visually similar, but they're >> not equivalent in any other way. And if you're going to say things >> should be considered equivalent solely on the basis of visuals, you >> get into a minefield - should U+200B ZERO WIDTH SPACE be completely >> ignored, allowing "AB" to be equivalent to "A\u200bB" as an >> identifier? >> >> > > I said 1 better than 2 better than 3 > Maybe you also want to add: > > Special cases aren't special enough to break the rules. > Although practicality beats purity. > > followed by > > Errors should never pass silently. > > IOW setting out 1 better than 2 better than 3 does not necessarily imply its > completely achievable No; I'm not saying that. I'm completely disagreeing with #1's value. I don't think the language interpreter should concern itself with visually-confusing identifiers. Unicode normalization is about *equivalent characters*, not confusability, and I think that's as far as Python should go. ChrisA From p.f.moore at gmail.com Thu Jul 21 04:37:06 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 21 Jul 2016 09:37:06 +0100 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> <9b707450-5a57-4677-bd20-48e9f53cb87d@googlegroups.com> Message-ID: On 21 July 2016 at 08:54, Chris Angelico wrote: > No; I'm not saying that. I'm completely disagreeing with #1's value. I > don't think the language interpreter should concern itself with > visually-confusing identifiers. Unicode normalization is about > *equivalent characters*, not confusability, and I think that's as far > as Python should go. +1. There are plenty of ways of writing programs that don't do what the reader expects. (Non-malicious) people writing Python code shouldn't be using visually ambiguous identifiers. People running code they don't trust should check it (and yes, "are there any non-ASCII/confusable characters used in identifiers" is one check they should make, among many). Avoiding common mistakes is a good thing. But that's about as far as we should go. On that note, though, "smart quotes" do find their way into code, usually via cut and paste from documents in tools like MS Word that "helpfully" change straight quotes to smart ones. So it *may* be worth special casing a check for smart quotes in identifiers, and reporting something like "was this meant to be a string, but you accidentally used smart quotes"? On the other hand, people who routinely copy code samples from sources that mangle the quotes are probably used to errors of this nature, and know what went wrong even if the error is unclear. After all, I don't know *any* language that explicitly checks for this specific error. Personally, I don't think that the effort required is justified by the minimal benefit. But if someone were to be inclined to make that effort and produce a patch, an argument along the above lines (catching a common cut and paste error) might be sufficient to persuade someone to commit it. Paul From ncoghlan at gmail.com Thu Jul 21 07:55:57 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 21 Jul 2016 21:55:57 +1000 Subject: [Python-ideas] =?utf-8?q?Error_handling_for_unknown_Unicode_char?= =?utf-8?q?acters_=28was_Re=3A_allow_=60lambda=27_to_be_spelled_?= =?utf-8?b?zrsp?= In-Reply-To: References: Message-ID: On 21 July 2016 at 17:41, Nick Coghlan wrote: > - the caret positioning logic for syntax errors needs to be checked to > see if it's currently counting encoded UTF-8 bytes instead of code > points (as that will consistently do the wrong thing on a correctly > configured UTF-8 terminal) Prompted by Chris Angelico, I took a closer a look at the behaviour here, and it seems to be due to a problem with the caret being positioned at the end of a candidate "identifier" token, rather than at the beginning: >>> varname = ?d?a?t?apoint File "", line 1 varname = ?d?a?t?apoint ^ SyntaxError: invalid character in identifier >>> varname = ?d?a?t?apoint.evidence File "", line 1 varname = ?d?a?t?apoint.evidence ^ SyntaxError: invalid character in identifier >>> varname = ?d?a?t?apoint[evidence] File "", line 1 varname = ?d?a?t?apoint[evidence] ^ SyntaxError: invalid character in identifier >>> varname = ?d?a?t?apoint(evidence) File "", line 1 varname = ?d?a?t?apoint(evidence) ^ SyntaxError: invalid character in identifier If you view those examples in a fixed width font, you'll see the caret is pointing at the "t" in each case, rather than at the first problematic code point. (Even in a proportional font, while you can't see the actual alignment, you *can* see that the alignment isn't right) By contrast, if you put an impermissible ASCII character into the "identifier" the caret points right at it. If anyone's inclined to dig into the compilation toolchain to try to figure out what's going on, I filed on issue for this particular misbehaviour at http://bugs.python.org/issue27582 Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From spitz.dan.l at gmail.com Thu Jul 21 10:06:51 2016 From: spitz.dan.l at gmail.com (Daniel Spitz) Date: Thu, 21 Jul 2016 14:06:51 +0000 Subject: [Python-ideas] Addition to operator module: starcaller Message-ID: Hello, I have run into a use case where I want to create a wrapper object that will star-unpack a sequence of arguments to pass to a function. Currently one way to construct this is to create a function: def starcaller(f): def wrapper(args): return f(*args) return wrapper Such a function feels simple enough, and specific enough to the language, that it would fit in well in the operator module of the standard library. We already have a similar representation of this functionality in itertools.starmap. Whereas the nested function implementation above will produce unpickleable objects (due to the closure), a straightforward c implementation will produce pickleable ones, making them useful for parallel applications. Thanks, Dan Spitz -------------- next part -------------- An HTML attachment was scrubbed... URL: From vgr255 at live.ca Thu Jul 21 10:19:46 2016 From: vgr255 at live.ca (Emanuel Barry) Date: Thu, 21 Jul 2016 14:19:46 +0000 Subject: [Python-ideas] Addition to operator module: starcaller In-Reply-To: References: Message-ID: Hello, You?re presumably doing something like ?star = starcaller(f); star((?foo?, ?bar?, ?baz?))? ? how is it different from ?f(*(?foo?, ?bar?, ?baz?))? ? I don?t see any difference? -Emanuel From: Python-ideas [mailto:python-ideas-bounces+vgr255=live.ca at python.org] On Behalf Of Daniel Spitz Sent: Thursday, July 21, 2016 10:07 AM To: python-ideas at python.org Subject: [Python-ideas] Addition to operator module: starcaller Hello, I have run into a use case where I want to create a wrapper object that will star-unpack a sequence of arguments to pass to a function. Currently one way to construct this is to create a function: def starcaller(f): def wrapper(args): return f(*args) return wrapper Such a function feels simple enough, and specific enough to the language, that it would fit in well in the operator module of the standard library. We already have a similar representation of this functionality in itertools.starmap. Whereas the nested function implementation above will produce unpickleable objects (due to the closure), a straightforward c implementation will produce pickleable ones, making them useful for parallel applications. Thanks, Dan Spitz -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Thu Jul 21 10:25:37 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 22 Jul 2016 00:25:37 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> Message-ID: <20160721142537.GF27919@ando.pearwood.info> On Wed, Jul 20, 2016 at 06:16:10PM -0300, Danilo J. S. Bellini wrote: > 1. Using SyntaxError for lexical errors sounds as strange as saying a > misspell/typo is a syntax mistake in a natural language. Why? Regardless of whether the error is found by the tokeniser, the lexer, the parser, or something else, it is still a *syntax error*. Why would the programmer need to know, or care, what part of the compiler/interpreter detects the error? Also consider that not all Python interpreters will divide up the task of interpreting code exactly the same way. Tokenisers, lexers and parsers are very closely related and not necessarily distinct. Should the *exact same typo* generate TokenError in one Python, LexerError in another, and ParserError in a third? What is the advantage of that? > 2. About those lexical error messages, the caret is worse than the lack of > it when it's not aligned, but unless I'm missing something, one can't > guarantee that the terminal is printing the error message with the right > encoding. Including the row and column numbers in the message would be > helpful. It would be nice for the caret to point to the illegal character, but it's not *wrong* to point past it to the end of the token that contains the illegal character. > 4. Unicode have more than one codepoint for some symbols that look alike, > for example "??????" are all valid uppercase sigmas. Ther Not really. Look at their names: GREEK CAPITAL LETTER SIGMA MATHEMATICAL BOLD CAPITAL SIGMA MATHEMATICAL ITALIC CAPITAL SIGMA MATHEMATICAL BOLD ITALIC CAPITAL SIGMA MATHEMATICAL SANS-SERIF BOLD CAPITAL SIGMA MATHEMATICAL SANS-SERIF BOLD ITALIC CAPITAL SIGMA Personally, I don't understand why the Unicode Consortium has included all these variants. But whatever the reason, the names hint strongly that they have specialised purposes, and shouldn't be used when you want the letter ?. But, if you do, Python will normalise them all to ?, so there's no real harm done, except to the readability of your code. [...] > when editing a code with an Unicode > char like that, most people would probably copy and paste the symbol > instead of typing it, leading to a consistent use of the same symbol. You are assuming that the programmer's font includes glyphs for all of six of those code points. More likely, the programmer will see ? for the first code point, and the other five will display as a pair of "missing glyph" boxes. (That's exactly what I see in my mail client, and in the Python interpreter.) Why a pair of boxes? Because they are code points in the Supplementary Multilingual Planes, and require *two* 16-bit code units in UTF-16. So naive Unicode software with poor support for the SMPs will display two boxes, one for each surrogate code point. Even if the code points display correctly, with distinct glyphs, your comment that most people will be forced to copy and paste the symbol is precisely why I am reluctant to see Python introduce non-ASCII keywords or operators. It's a pity, because I think that non-ASCII operators at least can make a much richer language (although I wouldn't want to see anything as extreme as APL). Perhaps I will change my mind in a few more years, as the popularity of emoji encourage more applications to have better support for non-ASCII and the SMPs. [...] > 6. Python 3 code is UTF-8 and Unicode identifiers are allowed. Not having > Unicode keywords is merely contingent on Python 2 behavior that emphasized > ASCII-only code (besides comments and strings). No, it is a *policy decision*. It is not because Python 2 didn't support them. Python 2 didn't support non-ASCII identifiers either, but Python 3 intentionally broke with that. > 7. The discussion isn't about lambda or anti-lambda bias, it's about > keyword naming and readability. Who gains/loses with that resource? It > won't hurt those who never uses lambda and never uses Unicode identifiers. It will hurt those who have to read code with a mystery ? that they don't know what it means and they have no idea how to search for it. At least "python lambda" is easy to search for. It will hurt those who want to use ? as an identifier. I include myself in that category. I don't want ? to be reserved as a keyword. I look at it like this: use ? as a keyword makes as much sense as making f a keyword so that we can save a few characters by writing: f myfunction(arg, x, y): pass instead of def. I use f as an identifier in many places, e.g.: for f in list_of_functions: ... or in functional code: compose(f, g) Yes, I can *work around it* by naming things f_ instead of f, but that's ugly. Even though it saves a few keystrokes, I wouldn't want f to be reserved as a keyword, and the same goes for ? as lambda. > 8. I don't know if any consensus can emerge in this matter about lambdas, > but there's another subject that can be discussed together: macros. I'm pretty sure that Guido has ruled "Over My Dead Body" to anything resembling macros in Python. However, we can experiment with adding keywords and macro-like facilities without Guido's permission. For example: http://www.staringispolite.com/likepython/ It's a joke, of course, but the technology is real. Imagine, if you will, that somebody you could declare a "dialect" at the start of Python modules, just after the optional language cookie: # -*- coding: utf-8 -*- # -*- dialect math -*- which would tell importlib to run the code through some sort of source/AST transformation before importing it. That will allow us to localise the keywords, introduce new operators, and all the other things Guido hates *wink* and still be able to treat the code as normal Python. A bad idea? Probably an awful one. But it's worth experimenting with it, It will be fun, and it *just might* turn out to be a good idea. For the record, in the 1980s and 1990s, Apple used a similar idea for two of their scripting languages, Hypertalk and Applescript, allowing users to localise keywords. Hypertalk is now defunct, and Applescript has dropped that feature, which suggests that it is a bad idea. Or maybe it was just ahead of its time. -- Steve From rosuav at gmail.com Thu Jul 21 10:34:32 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 22 Jul 2016 00:34:32 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <20160721142537.GF27919@ando.pearwood.info> References: <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> <20160721142537.GF27919@ando.pearwood.info> Message-ID: On Fri, Jul 22, 2016 at 12:25 AM, Steven D'Aprano wrote: > On Wed, Jul 20, 2016 at 06:16:10PM -0300, Danilo J. S. Bellini wrote: >> 2. About those lexical error messages, the caret is worse than the lack of >> it when it's not aligned, but unless I'm missing something, one can't >> guarantee that the terminal is printing the error message with the right >> encoding. Including the row and column numbers in the message would be >> helpful. > > It would be nice for the caret to point to the illegal character, but > it's not *wrong* to point past it to the end of the token that contains > the illegal character. And it's currently being explored here: http://bugs.python.org/issue27582 If you like the idea of the caret pointing somewhere else, join the discussion. ChrisA From mertz at gnosis.cx Thu Jul 21 10:59:27 2016 From: mertz at gnosis.cx (David Mertz) Date: Thu, 21 Jul 2016 07:59:27 -0700 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: References: <20160714024414.GP27919@ando.pearwood.info> <5787F3CB.1080304@mail.de> <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> <9b707450-5a57-4677-bd20-48e9f53cb87d@googlegroups.com> Message-ID: This idea of "visually confusable" seems like a very silly thing to worry about, as others have noted. It's not just that completely different letters from different alphabets may "look similar", it's also that the similarity is completely dependent on the specific font used for display. My favorite font might have clearly distinguished glyphs for the Cyrillic, Roman, and Greek "A", even if your font uses identical glyphs. So in this crazy scenario, Python would have to gain awareness of the fonts installed in every text editor and display device of every user. On Jul 21, 2016 12:55 AM, "Chris Angelico" wrote: On Thu, Jul 21, 2016 at 5:47 PM, Rustom Mody wrote: >> On Thu, Jul 21, 2016 at 4:26 PM, Rustom Mody wrote: >> > IOW >> > 1. Disallow co-existence of confusables (in identifiers) >> > 2. Identify confusables to a normal form ? like case-insensitive >> > comparison >> > and like NKFC >> > 3. Leave the confusables to confuse >> > >> > My choice >> > 1 better than 2 better than 3 >> >> So should we disable the lowercase 'l', the uppercase 'I', and the >> digit '1', because they can be confused? What about the confusability >> of "m" and "rn"? O and 0 are similar in some fonts. And case >> insensitivity brings its own problems - is "ss" equivalent to "?", and >> is "?" equivalent to either? Turkish distinguishes between "i", which >> upper-cases to "?", and "?", which upper-cases to "I". >> >> We already have interminable debates about letter similarities across >> scripts. I'm sure everyone agrees that Cyrillic "?" is not the same >> letter as Latin "i", but we have "A??" in three different scripts. >> Should they be considered equivalent? I think not, because in any >> non-trivial context, you'll know whether the program's been written in >> Greek, a Slavic language, or something using the Latin script. But >> maybe you disagree. Okay; are "B??" all to be considered equivalent >> too? What about "?C"? "X???"? They're visually similar, but they're >> not equivalent in any other way. And if you're going to say things >> should be considered equivalent solely on the basis of visuals, you >> get into a minefield - should U+200B ZERO WIDTH SPACE be completely >> ignored, allowing "AB" to be equivalent to "A\u200bB" as an >> identifier? >> >> > > I said 1 better than 2 better than 3 > Maybe you also want to add: > > Special cases aren't special enough to break the rules. > Although practicality beats purity. > > followed by > > Errors should never pass silently. > > IOW setting out 1 better than 2 better than 3 does not necessarily imply its > completely achievable No; I'm not saying that. I'm completely disagreeing with #1's value. I don't think the language interpreter should concern itself with visually-confusing identifiers. Unicode normalization is about *equivalent characters*, not confusability, and I think that's as far as Python should go. ChrisA _______________________________________________ Python-ideas mailing list Python-ideas at python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mertz at gnosis.cx Thu Jul 21 11:09:59 2016 From: mertz at gnosis.cx (David Mertz) Date: Thu, 21 Jul 2016 08:09:59 -0700 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <20160721142537.GF27919@ando.pearwood.info> References: <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> <20160721142537.GF27919@ando.pearwood.info> Message-ID: On Jul 21, 2016 7:26 AM, "Steven D'Aprano" wrote: > You are assuming that the programmer's font includes glyphs for all of > six of those code points. More likely, the programmer will see ? for the > first code point, and the other five will display as a pair of "missing > glyph" boxes. (That's exactly what I see in my mail client, and in the > Python interpreter.) Fwiw, on my OSX laptop, with whatever particular fonts I have installed there, using a particular webmail service in the particular browser I use, I see all six glyphs. If I were to copy-paste into a text editor, all bets would be off, and depend on the editor and it its settings. Same for interactive shells run in particular terminal apps. Viewing right now, on my Android tablet and the Gmail app, I see a bunch of missing glyph markers. But quite likely I could install fonts or change settings on this device to render them. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Thu Jul 21 11:20:38 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 22 Jul 2016 01:20:38 +1000 Subject: [Python-ideas] Addition to operator module: starcaller In-Reply-To: References: Message-ID: On 22 July 2016 at 00:19, Emanuel Barry wrote: > Hello, > > You?re presumably doing something like ?star = starcaller(f); star((?foo?, > ?bar?, ?baz?))? ? how is it different from ?f(*(?foo?, ?bar?, ?baz?))? ? I > don?t see any difference? Similar to functools.partial, it lets you more easily separate the process of changing a function's signature from actually calling it. Compare: def f(a, b): print(a, b) accepts_only_b = functools.partial(f, "predefined_a") accepts_args_as_tuple = operator.starcaller(f) >From the point of view of subsequent code, "args_as_tuple" now works as a callable that accepts a 2-tuple and prints the elements. However, the similarity to partial does make me wonder whether we might be better off resurrecting the old apply() builtin as a functools function: https://docs.python.org/2/library/functions.html#apply Then the requested operation would just be: accepts_args_as_tuple = functools.partial(functools.apply, f) (with the added bonus of also accepting an optional keyword dict) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From steve at pearwood.info Thu Jul 21 11:45:43 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 22 Jul 2016 01:45:43 +1000 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <9b707450-5a57-4677-bd20-48e9f53cb87d@googlegroups.com> References: <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> <9b707450-5a57-4677-bd20-48e9f53cb87d@googlegroups.com> Message-ID: <20160721154543.GG27919@ando.pearwood.info> On Wed, Jul 20, 2016 at 11:26:58PM -0700, Rustom Mody wrote: > >>> ? = 1 > >>> A = A + 1 > > because the A's look more indistinguishable than the sigmas and are > internally more distinct > If the choice is to simply disallow the confusables that?s probably the > best choice > > IOW > 1. Disallow co-existence of confusables (in identifiers) That would require disallowing 1 l and I, as well as O and 0. Or are you, after telling us off for taking an ASCII-centric perspective, going to exempt ASCII confusables? In a dynamic language like Python, how do you prohibit these confusables? Every time Python does a name binding operation, is it supposed to search the entire namespace for potential confusables? That's going to be awful expensive. Confusables are a real problem in URLs, because they can be used for phishing attacks. While even the most tech-savvy user is vulnerable, it is especially the *least* savvy users who are at risk, which makes it all the more important to protect against confusables in URLs. But in programming code? Your demonstration with the Latin A and the Greek alpha ? or Cyrillic ? is just a party trick. In a world where most developers do something like: pip install randompackage python -m randompackage without ever once looking at the source code, I think we have bigger problems. Or rather, even the bigger problems are not that big. If you're worried about confusables, there are alternatives other than banning them: your editor or linter might highlight them. Or rather than syntax highlighting, perhaps editors should use *semantic highlighting* and colour-code variables: https://medium.com/@evnbr/coding-in-color-3a6db2743a1e in which case your A and A will be highlighted in completely different colours, completely ruining the trick. (Aside: this may also help with the "oops I misspelled my variable and the compiler didn't complain" problem. If "self.dashes" is green and "self.dahses" is blue, you're more likely to notice the typo.) -- Steve From vgr255 at live.ca Thu Jul 21 11:48:15 2016 From: vgr255 at live.ca (Emanuel Barry) Date: Thu, 21 Jul 2016 15:48:15 +0000 Subject: [Python-ideas] Addition to operator module: starcaller In-Reply-To: References: Message-ID: > From: Nick Coghlan > Sent: Thursday, July 21, 2016 11:21 AM > > On 22 July 2016 at 00:19, Emanuel Barry wrote: > > Hello, > > > > You?re presumably doing something like ?star = starcaller(f); star((?foo?, > > ?bar?, ?baz?))? ? how is it different from ?f(*(?foo?, ?bar?, ?baz?))? ? I > > don?t see any difference? > > Similar to functools.partial, it lets you more easily separate the > process of changing a function's signature from actually calling it. > > Compare: > > def f(a, b): > print(a, b) > > accepts_only_b = functools.partial(f, "predefined_a") > accepts_args_as_tuple = operator.starcaller(f) Ah, of course - I didn't think of the multiple arguments case, thanks :) -Emanuel From spitz.dan.l at gmail.com Thu Jul 21 12:36:37 2016 From: spitz.dan.l at gmail.com (Daniel Spitz) Date: Thu, 21 Jul 2016 16:36:37 +0000 Subject: [Python-ideas] Addition to operator module: starcaller In-Reply-To: References: Message-ID: Well, partial + apply seems like the most powerful solution (supports **keywords, opens up the possibility of using apply in other ways). Perhaps adding apply back to functools rather than as a builtin would evade some of the antipatterns it used to encourage (a la reduce). I'm not sure and don't have a strong opinion about the original removal of apply. Alternatively a form of "starcaller" or "unpackcaller" defined roughly as partial(apply) could be added to the operator module in the capacity I proposed above. The only possible advantage here is aesthetic; it fits conceptually with the rest of the module as a callable capturing a repeatable operation that is normally only possible with special syntax. It also disallows using apply in "non-curried" contexts. Thanks, Dan PS. Emanuel, my use case is one where I am creating data transformations by composing chains of function calls. The functions are executed in parallel across multiple processes. Specifically I'm using the Dask Distributed library. On Thu, Jul 21, 2016 at 11:51 AM Emanuel Barry wrote: > > From: Nick Coghlan > > Sent: Thursday, July 21, 2016 11:21 AM > > > > On 22 July 2016 at 00:19, Emanuel Barry wrote: > > > Hello, > > > > > > You?re presumably doing something like ?star = starcaller(f); > star((?foo?, > > > ?bar?, ?baz?))? ? how is it different from ?f(*(?foo?, ?bar?, ?baz?))? > ? I > > > don?t see any difference? > > > > Similar to functools.partial, it lets you more easily separate the > > process of changing a function's signature from actually calling it. > > > > Compare: > > > > def f(a, b): > > print(a, b) > > > > accepts_only_b = functools.partial(f, "predefined_a") > > accepts_args_as_tuple = operator.starcaller(f) > > Ah, of course - I didn't think of the multiple arguments case, thanks :) > > -Emanuel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Thu Jul 21 13:48:36 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 21 Jul 2016 13:48:36 -0400 Subject: [Python-ideas] =?utf-8?q?Error_handling_for_unknown_Unicode_char?= =?utf-8?q?acters_=28was_Re=3A_allow_=60lambda=27_to_be_spelled_=CE=BB=29?= In-Reply-To: References: Message-ID: On 7/21/2016 3:41 AM, Nick Coghlan wrote: > On 21 July 2016 at 15:08, Rustom Mody wrote: >> My ?wrongheaded? was (intended) quite narrow and technical: >> >> - The embargo on non-ASCII everywhere in the language except identifiers >> (strings >> and comments obviously dont count as ?in? the language >> - The opening of identifiers to large swathes of Unicode widens as you say >> hugely the surface area of attack >> >> This was solely the contradiction I was pointing out. > > OK, thanks for the clarification, and my apologies for jumping on you. > I can be a bit hypersensitive on this topic, as my day job sometimes > includes encouraging commercial redistributors and end users to stop > taking community volunteers for granted and instead help find ways to > ensure their work is sustainable :) > > As it is, I think there are some possible checks that could be added > to the code generator pipeline to help clarify matters: > > - for the "invalid character" error message, we should be able to > always report both the printed symbol *and* the ASCII hex escape, > rather than assuming the caret will point to the correct place > - the caret positioning logic for syntax errors needs to be checked to > see if it's currently counting encoded UTF-8 bytes instead of code > points (as that will consistently do the wrong thing on a correctly > configured UTF-8 terminal) > - (more speculatively) when building the symbol table, we may be able > to look for identifiers referenced in a namespace that are not NKFC > equivalent, but nevertheless qualify as Unicode confusables, and emit > a SyntaxWarning (this is speculative, as I'm not sure what degree of > performance hit would be associated with it) > > As far as Danilo's observation regarding the CPython code generator > always emitting SyntaxError and SyntaxWarning (regardless of which > part of the code generation actually failed) goes, I wouldn't be > opposed to our getting more precise about that by defining additional > subclasses, but one of the requirements would be for documentation in > https://docs.python.org/devguide/compiler.html or helper functions in > the source to clearly define "when working on part of the code > generation pipeline, raise kind of error if something goes > wrong". > > Cheers, > Nick. > -- Terry Jan Reedy From tjreedy at udel.edu Thu Jul 21 14:01:31 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 21 Jul 2016 14:01:31 -0400 Subject: [Python-ideas] =?utf-8?q?Error_handling_for_unknown_Unicode_char?= =?utf-8?q?acters_=28was_Re=3A_allow_=60lambda=27_to_be_spelled_=CE=BB=29?= In-Reply-To: References: Message-ID: [Apologies for the previous premature send] On 7/21/2016 3:41 AM, Nick Coghlan wrote: > On 21 July 2016 at 15:08, Rustom Mody wrote: >> My ?wrongheaded? was (intended) quite narrow and technical: >> >> - The embargo on non-ASCII everywhere in the language except identifiers >> (strings >> and comments obviously dont count as ?in? the language >> - The opening of identifiers to large swathes of Unicode widens as you say >> hugely the surface area of attack >> >> This was solely the contradiction I was pointing out. > > OK, thanks for the clarification, and my apologies for jumping on you. > I can be a bit hypersensitive on this topic, as my day job sometimes > includes encouraging commercial redistributors and end users to stop > taking community volunteers for granted and instead help find ways to > ensure their work is sustainable :) Like reporting errors on the tracker or at least python-list or even here, instead of only in books, articles, posts, other forum questions. I found a few unreported errors on SO questions (and have fixed a couple). > As it is, I think there are some possible checks that could be added > to the code generator pipeline to help clarify matters: > > - for the "invalid character" error message, we should be able to > always report both the printed symbol *and* the ASCII hex escape, > rather than assuming the caret will point to the correct place > - the caret positioning logic for syntax errors needs to be checked to > see if it's currently counting encoded UTF-8 bytes instead of code > points (as that will consistently do the wrong thing on a correctly > configured UTF-8 terminal) > - (more speculatively) when building the symbol table, we may be able > to look for identifiers referenced in a namespace that are not NKFC > equivalent, but nevertheless qualify as Unicode confusables, and emit > a SyntaxWarning (this is speculative, as I'm not sure what degree of > performance hit would be associated with it) > > As far as Danilo's observation regarding the CPython code generator > always emitting SyntaxError and SyntaxWarning (regardless of which > part of the code generation actually failed) goes, It is not true. https://bugs.python.org/issue25733 All these are (supposedly) possible: SyntaxError -- obvious NameError -- ?, already caught in code module OverflowError-- ?, already caught in code module SystemError - 22 nested for loops ('deeply nested blocks') TypeError -- chr(0), 2.7 ValueError -- chr(0), 3.x; bytes(0), 2.7 Emitting SystemError was changed in 27514. I wish Danilo's observation were true. This issue is about the fact that the code module and IDLE do not catch all possible compile errors because there was no documented list until I compiled the above, which may still not be complete. > I wouldn't be > opposed to our getting more precise about that by defining additional > subclasses, but one of the requirements would be for documentation in > https://docs.python.org/devguide/compiler.html or helper functions in > the source to clearly define "when working on part of the code > generation pipeline, raise kind of error if something goes > wrong". Or I wish that the compile doc gave a complete list. -- Terry Jan Reedy From michael.selik at gmail.com Thu Jul 21 14:04:32 2016 From: michael.selik at gmail.com (Michael Selik) Date: Thu, 21 Jul 2016 18:04:32 +0000 Subject: [Python-ideas] Addition to operator module: starcaller In-Reply-To: References: Message-ID: On Thu, Jul 21, 2016 at 12:37 PM Daniel Spitz wrote: > Alternatively a form of "starcaller" or "unpackcaller" defined roughly as > partial(apply) could be added to the operator module in the capacity I > proposed above. The only possible advantage here is aesthetic; it fits > conceptually with the rest of the module as a callable capturing a > repeatable operation that is normally only possible with special syntax. It > also disallows using apply in "non-curried" contexts. > It does seem to fit with the theme of the operator module. One major use of the operator module is providing an alternative to lambdas. In this case, the clear alternative is ``g = lambda args, kwargs: f(*args, **kwargs)`` -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Thu Jul 21 14:50:52 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 21 Jul 2016 14:50:52 -0400 Subject: [Python-ideas] =?utf-8?q?allow_=60lambda=27_to_be_spelled_=CE=BB?= In-Reply-To: <20160721142537.GF27919@ando.pearwood.info> References: <95cca6ee-07a4-4393-8146-7ed9bf68e15d@googlegroups.com> <20160719112027.GX27919@ando.pearwood.info> <220634be-2c03-4c88-9818-0191d757b35d@googlegroups.com> <22415.43612.173013.196705@turnbull.sk.tsukuba.ac.jp> <20160721142537.GF27919@ando.pearwood.info> Message-ID: On 7/21/2016 10:25 AM, Steven D'Aprano wrote: > Imagine, if you will, that somebody you could declare a "dialect" at the > start of Python modules, just after the optional language cookie: > > # -*- coding: utf-8 -*- > # -*- dialect math -*- > > which would tell importlib to run the code through some sort of > source/AST transformation before importing it. That will allow us to > localise the keywords, introduce new operators, and all the other things > Guido hates *wink* and still be able to treat the code as normal Python. Or one could write a 'unipy' extension to an IDE like IDLE that would translate an entire editor buffer either way. It would take less time than has been expended pushing for a change that will not happen in the near future. > A bad idea? Probably an awful one. But it's worth experimenting with it, > It will be fun, and it *just might* turn out to be a good idea. > > For the record, in the 1980s and 1990s, Apple used a similar idea for > two of their scripting languages, Hypertalk and Applescript, allowing > users to localise keywords. Hypertalk is now defunct, and Applescript > has dropped that feature, which suggests that it is a bad idea. Or maybe > it was just ahead of its time. > > -- Terry Jan Reedy From guido at python.org Thu Jul 21 16:43:01 2016 From: guido at python.org (Guido van Rossum) Date: Thu, 21 Jul 2016 13:43:01 -0700 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" Message-ID: At Dropbox I see a lot of people who want to start using type annotations (PEP 484, mypy) struggle with the collection ABCs. It's pretty common to want to support both sets and lists of values, as these have a lot of useful behavior: they are (re)iterable, have a size, and implement `__contains__` (i.e. x in c, x not in c). It's pretty common for people to think that Iterable is the answer, but it's not. I'm beginning to think that it would be useful to add another ABC to collections.abc (and to typing) that represents the intersection of these three ABCs, and to "upgrade" Set and Sequence to inherit from it. (And Mapping.) Thoughts? (Another useful concept is "reiterable", i.e. an Iterable that can be iterated over multiple times -- there's no ABC to indicate this concept. Sequence, Set and Mapping all support this concept, Iterator does not include it, but Iterable is wishy-washy.) -- --Guido van Rossum (python.org/~guido) From michael.selik at gmail.com Thu Jul 21 17:14:21 2016 From: michael.selik at gmail.com (Michael Selik) Date: Thu, 21 Jul 2016 21:14:21 +0000 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: Message-ID: On Thu, Jul 21, 2016 at 4:43 PM Guido van Rossum wrote: > At Dropbox I see a lot of people who want to start using type > annotations (PEP 484, mypy) struggle with the collection ABCs. > > It's pretty common to want to support both sets and lists of values > Are these folks dissatisfied with a union type: ``Union[Sequence, Set]``? (Another useful concept is "reiterable", i.e. an Iterable that can be > iterated over multiple times -- there's no ABC to indicate this > concept. Sequence, Set and Mapping all support this concept, Iterator > does not include it, but Iterable is wishy-washy.) > "Reiterable" seems to cause vocabulary confusion regularly on this list. It also has a common case for runtime checking: if not isinstance(iterable, Reiterable): iterable = list(iterable) It's hard to think of a good name to describe ``Union[Iterable, Sized, Container]`` and it'd still have the confusion of whether it's re-iterable. Does the concept of "reiterable" imply that an iteration does not mutate the object? If so, then if something is both Reiterable and Sized, it should also be a Container. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Thu Jul 21 17:28:55 2016 From: guido at python.org (Guido van Rossum) Date: Thu, 21 Jul 2016 14:28:55 -0700 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: Message-ID: On Thu, Jul 21, 2016 at 2:14 PM, Michael Selik wrote: > > > On Thu, Jul 21, 2016 at 4:43 PM Guido van Rossum wrote: >> >> At Dropbox I see a lot of people who want to start using type >> annotations (PEP 484, mypy) struggle with the collection ABCs. >> >> It's pretty common to want to support both sets and lists of values > > > Are these folks dissatisfied with a union type: ``Union[Sequence, Set]``? I don't know if they are, but I am. :-) Unions look pretty ugly, and there are obvious implementations that satisfy all three criteria but are neither sequence not set (e.g. mapping). >> (Another useful concept is "reiterable", i.e. an Iterable that can be >> iterated over multiple times -- there's no ABC to indicate this >> concept. Sequence, Set and Mapping all support this concept, Iterator >> does not include it, but Iterable is wishy-washy.) > > > "Reiterable" seems to cause vocabulary confusion regularly on this list. What kind of confusion? > It also has a common case for runtime checking: > > if not isinstance(iterable, Reiterable): > iterable = list(iterable) > > It's hard to think of a good name to describe ``Union[Iterable, Sized, > Container]`` and it'd still have the confusion of whether it's re-iterable. It's hard to imagine it having a size and not being reiterable. FWIW IIUC the most common idiom to test for this is: if iter(iterable) is iterable: iterable = list(iterable) > Does the concept of "reiterable" imply that an iteration does not mutate the > object? I think that's generally true, though it's not the key property. The key property is that after iterating until the end, you can start over from the beginning just by calling iter() again. Example types that don't have this property include streams and anything that is itself the result of calling iter() on something more stable. > If so, then if something is both Reiterable and Sized, it should > also be a Container. Agreed. (Container is not implied by Iterable mostly because for a once-iterable, "x in c" consumes the iterable.) -- --Guido van Rossum (python.org/~guido) From michael.selik at gmail.com Thu Jul 21 18:42:30 2016 From: michael.selik at gmail.com (Michael Selik) Date: Thu, 21 Jul 2016 22:42:30 +0000 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: Message-ID: On Thu, Jul 21, 2016 at 5:29 PM Guido van Rossum wrote: > On Thu, Jul 21, 2016 at 2:14 PM, Michael Selik > wrote: > > On Thu, Jul 21, 2016 at 4:43 PM Guido van Rossum > wrote: > > >> (Another useful concept is "reiterable", i.e. an Iterable that can be > >> iterated over multiple times -- there's no ABC to indicate this > >> concept. Sequence, Set and Mapping all support this concept, Iterator > >> does not include it, but Iterable is wishy-washy.) > > > > "Reiterable" seems to cause vocabulary confusion regularly on this list. > > What kind of confusion? > Someone accidentally says iterable when they mean reiterable, and then there's a handful of emails pointing that out and arguing about what the right word should be to describe something (like range) that can be iterated over more than once, but isn't necessarily a sequence, set, or mapping. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Thu Jul 21 19:38:25 2016 From: guido at python.org (Guido van Rossum) Date: Thu, 21 Jul 2016 16:38:25 -0700 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: Message-ID: Ah. Well, there's no official term "reiterable" AFAIK. This confusion also lies at the root of the issues I started with. So let's try to find a new term! On Thu, Jul 21, 2016 at 3:42 PM, Michael Selik wrote: > > > On Thu, Jul 21, 2016 at 5:29 PM Guido van Rossum wrote: >> >> On Thu, Jul 21, 2016 at 2:14 PM, Michael Selik >> wrote: >> > On Thu, Jul 21, 2016 at 4:43 PM Guido van Rossum >> > wrote: >> >> >> (Another useful concept is "reiterable", i.e. an Iterable that can be >> >> iterated over multiple times -- there's no ABC to indicate this >> >> concept. Sequence, Set and Mapping all support this concept, Iterator >> >> does not include it, but Iterable is wishy-washy.) >> > >> > "Reiterable" seems to cause vocabulary confusion regularly on this list. >> >> What kind of confusion? > > > Someone accidentally says iterable when they mean reiterable, and then > there's a handful of emails pointing that out and arguing about what the > right word should be to describe something (like range) that can be iterated > over more than once, but isn't necessarily a sequence, set, or mapping. -- --Guido van Rossum (python.org/~guido) From greg.ewing at canterbury.ac.nz Thu Jul 21 19:51:35 2016 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 22 Jul 2016 11:51:35 +1200 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: Message-ID: <57916007.5030504@canterbury.ac.nz> Guido van Rossum wrote: > Ah. Well, there's no official term "reiterable" AFAIK. This confusion > also lies at the root of the issues I started with. So let's try to > find a new term! Container or Collection sounds about right to me. -- Greg From guido at python.org Thu Jul 21 19:58:50 2016 From: guido at python.org (Guido van Rossum) Date: Thu, 21 Jul 2016 16:58:50 -0700 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: <57916007.5030504@canterbury.ac.nz> References: <57916007.5030504@canterbury.ac.nz> Message-ID: But there's already Container which means "supports __contains__". Collection might cause confusing with the module name collections. Otherwise either would be a good candidate... On Thu, Jul 21, 2016 at 4:51 PM, Greg Ewing wrote: > Guido van Rossum wrote: >> >> Ah. Well, there's no official term "reiterable" AFAIK. This confusion >> also lies at the root of the issues I started with. So let's try to >> find a new term! > > > Container or Collection sounds about right to me. > > -- > Greg > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -- --Guido van Rossum (python.org/~guido) From tomuxiong at gmail.com Thu Jul 21 20:05:25 2016 From: tomuxiong at gmail.com (Thomas Nyberg) Date: Thu, 21 Jul 2016 20:05:25 -0400 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: On 07/21/2016 07:58 PM, Guido van Rossum wrote: > But there's already Container which means "supports __contains__". > > Collection might cause confusing with the module name collections. > > Otherwise either would be a good candidate... Coming out of lurking... StaticIterable? ConstIterable? Something to indicate that if you just iterate over it you keep getting the same thing? Personally I think Reiterable is about as clear as it ever will be... Thomas From guido at python.org Thu Jul 21 20:29:08 2016 From: guido at python.org (Guido van Rossum) Date: Thu, 21 Jul 2016 17:29:08 -0700 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: On Thu, Jul 21, 2016 at 5:05 PM, Thomas Nyberg wrote: > On 07/21/2016 07:58 PM, Guido van Rossum wrote: >> >> But there's already Container which means "supports __contains__". >> >> Collection might cause confusing with the module name collections. >> >> Otherwise either would be a good candidate... > > > Coming out of lurking... > > StaticIterable? ConstIterable? Something to indicate that if you just > iterate over it you keep getting the same thing? Neither "static" nor "const" convey the right meaning. > Personally I think Reiterable is about as clear as it ever will be... Yeah, I think that's my conclusion as well. -- --Guido van Rossum (python.org/~guido) From steve at pearwood.info Thu Jul 21 22:29:21 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 22 Jul 2016 12:29:21 +1000 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: Message-ID: <20160722022919.GI27919@ando.pearwood.info> On Thu, Jul 21, 2016 at 01:43:01PM -0700, Guido van Rossum wrote: > At Dropbox I see a lot of people who want to start using type > annotations (PEP 484, mypy) struggle with the collection ABCs. > > It's pretty common to want to support both sets and lists of values, > as these have a lot of useful behavior: they are (re)iterable, have a > size, and implement `__contains__` (i.e. x in c, x not in c). It's > pretty common for people to think that Iterable is the answer, but > it's not. I'm beginning to think that it would be useful to add > another ABC to collections.abc (and to typing) that represents the > intersection of these three ABCs, and to "upgrade" Set and Sequence to > inherit from it. (And Mapping.) Are you talking about giving a nice name to Union[Set, Sequence, Mapping]? I'm not sure that there is one, unless we make up a word. How do you feel about abbreviations? SSM? MSS? > Thoughts? > > (Another useful concept is "reiterable", i.e. an Iterable that can be > iterated over multiple times -- there's no ABC to indicate this > concept. Sequence, Set and Mapping all support this concept, Iterator > does not include it, but Iterable is wishy-washy.) Now I'm having flash-backs to this thread: https://mail.python.org/pipermail/python-ideas/2013-September/023239.html As I recall from that thread: isinstance(obj, Reiterable) is *almost* equivalent to: isinstance(obj, Iterable) and not isinstance(obj, Iterator) Back at the time, I wasn't convinced that the stdlib needed a special name for this concept, but I wasn't thinking of the typing module. -- Steve From ncoghlan at gmail.com Thu Jul 21 22:39:56 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 22 Jul 2016 12:39:56 +1000 Subject: [Python-ideas] Addition to operator module: starcaller In-Reply-To: References: Message-ID: On 22 July 2016 at 02:36, Daniel Spitz wrote: > Well, partial + apply seems like the most powerful solution (supports > **keywords, opens up the possibility of using apply in other ways). Perhaps > adding apply back to functools rather than as a builtin would evade some of > the antipatterns it used to encourage (a la reduce). I'm not sure and don't > have a strong opinion about the original removal of apply. > > Alternatively a form of "starcaller" or "unpackcaller" defined roughly as > partial(apply) could be added to the operator module in the capacity I > proposed above. The only possible advantage here is aesthetic; it fits > conceptually with the rest of the module as a callable capturing a > repeatable operation that is normally only possible with special syntax. It > also disallows using apply in "non-curried" contexts. apply was removed primarily for reasons of redundancy rather than there being anything particularly objectionable about the function itself, so I wouldn't anticipate any major objections to giving it the reduce treatment. The answer to "Which approach should I use when?" is pretty straightforward: "use the builtin syntax in most situations, but use functools.apply instead if you specifically need a higher-order function". Where I've personally seen that need come up is in distributed task processing where you normally want the actual computation to happen *somewhere else* (e.g. on another machine, in another process or thread, or even as a callback within the current thread), but the mechanism for actually invoking the callback is itself a callback that accepts a "(callable, args, kwds)" triple. "functools.apply" then slots in neatly as a lowest common denominator "just call it immediately" baseline implementation. There's also an interesting potential synergy with PEP 484 type annotations when it comes to manipulating higher order functions: at the moment, there's no way to explicitly type callables that accept keyword arguments, but you *can* type a callable that accepts a tuple and a dict with particular contents as positional arguments. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Thu Jul 21 22:59:42 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 22 Jul 2016 12:59:42 +1000 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: On 22 July 2016 at 10:29, Guido van Rossum wrote: > On Thu, Jul 21, 2016 at 5:05 PM, Thomas Nyberg wrote: >> On 07/21/2016 07:58 PM, Guido van Rossum wrote: >>> >>> But there's already Container which means "supports __contains__". >>> >>> Collection might cause confusing with the module name collections. >>> >>> Otherwise either would be a good candidate... As far as the original "Sized Iterable Container" question goes, would FiniteContainer work? - it's countable/iterable - it has a quantifiable size - you can do membership tests on it Countable & quantifiable = finite, membership testing = container, hence FiniteContainer >> Coming out of lurking... >> >> StaticIterable? ConstIterable? Something to indicate that if you just >> iterate over it you keep getting the same thing? > > Neither "static" nor "const" convey the right meaning. > >> Personally I think Reiterable is about as clear as it ever will be... > > Yeah, I think that's my conclusion as well. With the semantics being "iter(x) is not x"? That seems reasonable to me, as I spent some time think about whether or not this is a distinct idea from the "Sized Iterable Container" question, and it's pretty easy to demonstrate that it is: >>> class ReiterableCounter: ... def __iter__(self): ... x = 0 ... while True: ... yield x ... x += 1 ... >>> ctr = ReiterableCounter() >>> for x in ctr: ... print(x) ... if x > 1: break ... 0 1 2 >>> for x in ctr: ... print(x) ... if x > 1: break ... 0 1 2 (The same example works by returning any non-terminating iterator from __iter__) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From guido at python.org Fri Jul 22 00:04:55 2016 From: guido at python.org (Guido van Rossum) Date: Thu, 21 Jul 2016 21:04:55 -0700 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: Yeah, I know that you can define something that's Iterable and Container but not Sized. But has anyone ever done that outside a test or demonstration? And PEP 484 can't talk about "this type except if it's that type", so the operational definition "Iterable except Iterator" can't be expressed as a type. If PEP 484 had the concept of Intersection type, I'd probably define X = Intersection[Sized, Iterable, Container]. But even then I'd like a name that rolls a little easier off the tongue than FiniteContainer. If it really must be two words, how about SizedIterable? That suggests it's a subclass of Sized and Iterable, which it is. Container doesn't need separate mention, it's pretty obvious that any Iterable can implement __contains__, and more people know that Iterable is an ABC than Container. The new ABC could define a default concrete __contains__ implementation: class SizedIterable(Sized, Iterable, Container): def __contains__(self, value): # Same as Sequence for v in self: if v == value: return True return False TBH if I had known about ABCs and PEP 484 when we defined the collections ABCs, I probably would have defined it like that on Iterable as well, and completely done away with Container as a separate one-trick pony ABC. Set and Mapping would just override the default implementation. (Hm... not sure how the default Set.__contains__ would work though. Can you make a concrete method abstract in a subclass?) On Thu, Jul 21, 2016 at 7:59 PM, Nick Coghlan wrote: > On 22 July 2016 at 10:29, Guido van Rossum wrote: >> On Thu, Jul 21, 2016 at 5:05 PM, Thomas Nyberg wrote: >>> On 07/21/2016 07:58 PM, Guido van Rossum wrote: >>>> >>>> But there's already Container which means "supports __contains__". >>>> >>>> Collection might cause confusing with the module name collections. >>>> >>>> Otherwise either would be a good candidate... > > As far as the original "Sized Iterable Container" question goes, would > FiniteContainer work? > > - it's countable/iterable > - it has a quantifiable size > - you can do membership tests on it > > Countable & quantifiable = finite, membership testing = container, > hence FiniteContainer > >>> Coming out of lurking... >>> >>> StaticIterable? ConstIterable? Something to indicate that if you just >>> iterate over it you keep getting the same thing? >> >> Neither "static" nor "const" convey the right meaning. >> >>> Personally I think Reiterable is about as clear as it ever will be... >> >> Yeah, I think that's my conclusion as well. > > With the semantics being "iter(x) is not x"? > > That seems reasonable to me, as I spent some time think about whether > or not this is a distinct idea from the "Sized Iterable Container" > question, and it's pretty easy to demonstrate that it is: > > >>> class ReiterableCounter: > ... def __iter__(self): > ... x = 0 > ... while True: > ... yield x > ... x += 1 > ... > >>> ctr = ReiterableCounter() > >>> for x in ctr: > ... print(x) > ... if x > 1: break > ... > 0 > 1 > 2 > >>> for x in ctr: > ... print(x) > ... if x > 1: break > ... > 0 > 1 > 2 > > (The same example works by returning any non-terminating iterator from __iter__) > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -- --Guido van Rossum (python.org/~guido) From greg.ewing at canterbury.ac.nz Thu Jul 21 19:45:40 2016 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 22 Jul 2016 11:45:40 +1200 Subject: [Python-ideas] Addition to operator module: starcaller In-Reply-To: References: Message-ID: <57915EA4.6080602@canterbury.ac.nz> Daniel Spitz wrote: > I'm > not sure and don't have a strong opinion about the original removal of > apply. I expect it was removed simply because the * and ** calling syntax makes it unecessary (before that was introduced, apply was the only way of getting that funcionality). If it's to be reintroduced, the operator module would seem to be the right place for it. -- Greg From ncoghlan at gmail.com Fri Jul 22 02:36:00 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 22 Jul 2016 16:36:00 +1000 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: On 22 July 2016 at 14:04, Guido van Rossum wrote: > If it really must be two words, how about SizedIterable? That suggests > it's a subclass of Sized and Iterable, which it is. Container doesn't > need separate mention, it's pretty obvious that any Iterable can > implement __contains__, and more people know that Iterable is an ABC > than Container. SizedIterable works for me, as it's the __len__ that drives most of the other interesting properties beyond an arbitrary iterable (like using iteration for the default containment check without worrying about consuming a one-shot iterator or creating an infinite loop). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From matt at getpattern.com Fri Jul 22 02:38:15 2016 From: matt at getpattern.com (Matt Gilson) Date: Thu, 21 Jul 2016 23:38:15 -0700 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: I like this proposal from a book-keeping perspective... the interface provided by sequences/mappings/sets is definitely different than the interface provided by generic iterables (mostly due to the fact that the former group can be iterated repeatedly). With that said, I'm not really sure what how the `Reiterable` (or whatever suitable name the smart people around here come up with) class would be implemented. class Reiterable(Iterable): pass This seems a bit weird. All of the other ABCs make some guarantees. If I fill out a set of methods then my object will be a . There doesn't seem to be a set of methods that exist to distinguish between an object that is Iterable and an object that is Reiterable. I suppose that the counter argument is that we always assume that the methods on the object are implemented "reasonably". Perhaps the main goal for this class is just book-keeping/annotating/loose "if you say so" typechecking and we're trusting that if someone subclasses `Reiterable`, they're going to implement it in such a way that it indeed can be iterated multiple times and that they didn't just mean `Iterable`. If that's the intent, I think that is also fine with me -- it just seemed like someone ought to mention it once... On Thu, Jul 21, 2016 at 5:29 PM, Guido van Rossum wrote: > On Thu, Jul 21, 2016 at 5:05 PM, Thomas Nyberg > wrote: > > On 07/21/2016 07:58 PM, Guido van Rossum wrote: > >> > >> But there's already Container which means "supports __contains__". > >> > >> Collection might cause confusing with the module name collections. > >> > >> Otherwise either would be a good candidate... > > > > > > Coming out of lurking... > > > > StaticIterable? ConstIterable? Something to indicate that if you just > > iterate over it you keep getting the same thing? > > Neither "static" nor "const" convey the right meaning. > > > Personally I think Reiterable is about as clear as it ever will be... > > Yeah, I think that's my conclusion as well. > > -- > --Guido van Rossum (python.org/~guido) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- [image: pattern-sig.png] Matt Gilson // SOFTWARE ENGINEER E: matt at getpattern.com // P: 603.892.7736 We?re looking for beta testers. Go here to sign up! -------------- next part -------------- An HTML attachment was scrubbed... URL: From shoyer at gmail.com Fri Jul 22 02:44:32 2016 From: shoyer at gmail.com (Stephan Hoyer) Date: Thu, 21 Jul 2016 23:44:32 -0700 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: On Thu, Jul 21, 2016 at 9:04 PM, Guido van Rossum wrote: > If it really must be two words, how about SizedIterable? That suggests > it's a subclass of Sized and Iterable, which it is. Container doesn't > need separate mention, it's pretty obvious that any Iterable can > implement __contains__, and more people know that Iterable is an ABC > than Container. In my experiments with type annotations, we recently encountered this exact same issue (variables for which either sets or lists should be valid). My initial solution was almost exactly what you propose here: we wrote a SizedIterable class, simply inheriting from Sized and Iterable. Ultimately, we didn't find this very satisfying, because we realized that strings are SizedIterables (they're sequences, too), and we really didn't want to allow accidentally passing strings that would be interpreted as iterables of characters. Unfortunately, I don't think there's any good way in Python to specify "any sized iterable that isn't a string". -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmoisset at machinalis.com Fri Jul 22 10:20:18 2016 From: dmoisset at machinalis.com (Daniel Moisset) Date: Fri, 22 Jul 2016 11:20:18 -0300 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: This is a bit of a tangent, hopefully convergent, but hey, this is python-ideas: Would there be interest in some kind of method/API for restarting iterators? If there was a it.restart() (or reset(it)), it would probably be a good idea to name the concept in the same way as the method ( "Restartable" or "Resettable"). And having that as an actual protocol would simplify the discussion on both what the type should contain and how it should be named, while having actal applications to simplify code (and reduce memory usage in cases when creating a list is not needed, just making multiple passes). I'm not proposing one of those names in particular, discussing whether this concept makes sense and is useful should be before the naming On Fri, Jul 22, 2016 at 3:44 AM, Stephan Hoyer wrote: > On Thu, Jul 21, 2016 at 9:04 PM, Guido van Rossum > wrote: > >> If it really must be two words, how about SizedIterable? That suggests >> it's a subclass of Sized and Iterable, which it is. Container doesn't >> need separate mention, it's pretty obvious that any Iterable can >> implement __contains__, and more people know that Iterable is an ABC >> than Container. > > > In my experiments with type annotations, we recently encountered this > exact same issue (variables for which either sets or lists should be > valid). My initial solution was almost exactly what you propose here: we > wrote a SizedIterable class, simply inheriting from Sized and Iterable. > > Ultimately, we didn't find this very satisfying, because we realized that > strings are SizedIterables (they're sequences, too), and we really didn't > want to allow accidentally passing strings that would be interpreted as > iterables of characters. Unfortunately, I don't think there's any good way > in Python to specify "any sized iterable that isn't a string". > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Daniel F. Moisset - UK Country Manager www.machinalis.com Skype: @dmoisset -------------- next part -------------- An HTML attachment was scrubbed... URL: From egregius313 at gmail.com Fri Jul 22 10:31:13 2016 From: egregius313 at gmail.com (Edward Minnix) Date: Fri, 22 Jul 2016 10:31:13 -0400 Subject: [Python-ideas] Good uses for staticmethod Message-ID: Hello, I have been reading Luciano Ramalho?s Fluent Python, and in the book he says he cannot think of a good use for staticmethod, I have been thinking about that, and so far I have only thought of one good use case: when you have a version of a common function, e.g., sum, and you want to qualify it with a better name but something like sumLineItems would sound wrong (to me at least). Would this be a good choice for something like: class LineItem: . . . . . . . . def __add__(self, other): ### Function body ### . . . . . . . . @staticmethod def sum(items: Sequence[LineItem]) -> LineItem: ### Add up LineItems ### Does this seem like a reasonable use for staticmethod? I think its a good way to qualify a name in a more concise way. Thanks, Ed M. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Fri Jul 22 11:48:34 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 23 Jul 2016 01:48:34 +1000 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: On Sat, Jul 23, 2016 at 12:20 AM, Daniel Moisset wrote: > Would there be interest in some kind of method/API for restarting iterators? > If there was a it.restart() (or reset(it)), it would probably be a good idea > to name the concept in the same way as the method ( "Restartable" or > "Resettable"). And having that as an actual protocol would simplify the > discussion on both what the type should contain and how it should be named, > while having actal applications to simplify code (and reduce memory usage in > cases when creating a list is not needed, just making multiple passes). IMO, no. Some iterators can be restarted by going back to the original iterable and requesting another iterator, but with no guarantee that it will result in the exact same sequence (eg dict/set iterators). Does that count as restarting? And what if you start by consuming a header or two, then pass the resulting iterator to another function. Based on type, it would be restartable (say it's a list_iterator), but the called function would be highly surprised to suddenly get back more results. What you might be looking at is a protocol for "bookmarking" or "forking" an iterator. That might be more useful. For example: # Like itertools.cycle() def cycle(it): it = iter(it) while True: yield from fork(it) # and then, in effect, "restart" it With list_iterator, forking is straight-forward - make another iterator off the same list, at the same point this one is. Compared to the normal implementation of itertools.cycle, this wouldn't need any auxiliary storage, as it'd just reference the base list every time. It's kinda like restarting the iterator, but if you give this a partly-consumed list_iterator, it would cycle through the remaining elements only. (How many iterators would be forkable, I don't know, but I expect it'd be the exact same ones that would be restartable by any other definition.) ChrisA From brett at python.org Fri Jul 22 12:46:18 2016 From: brett at python.org (Brett Cannon) Date: Fri, 22 Jul 2016 16:46:18 +0000 Subject: [Python-ideas] Good uses for staticmethod In-Reply-To: References: Message-ID: On Fri, 22 Jul 2016 at 07:34 Edward Minnix wrote: > Hello, > > I have been reading Luciano Ramalho?s *Fluent Python, *and in the book he > says he cannot think of a good use for staticmethod, I have been thinking > about that, and so far I have only thought of one good use case: when you > have a version of a common function, e.g., sum, and you want to qualify > it with a better name but something like sumLineItems would sound wrong > (to me at least). Would this be a good choice for something like: > > class LineItem: > . . . . . . . . > def __add__(self, other): > ### Function body ### > > . . . . . . . . > > @staticmethod > def sum(items: Sequence[LineItem]) -> LineItem: > ### Add up LineItems ### > > Does this seem like a reasonable use for staticmethod? I think its a good > way to qualify a name in a more concise way. > I don't think it is as I don't think it's any more concise than sum_line_items(). The only reason I ever use staticmethod is if I have a method that doesn't use self or cls but I expect subclasses to potentially need/want to override the method. Otherwise whatever the staticmethod is should be a function. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kramm at google.com Fri Jul 22 13:49:27 2016 From: kramm at google.com (Matthias Kramm) Date: Fri, 22 Jul 2016 10:49:27 -0700 (PDT) Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: Message-ID: On Thursday, July 21, 2016 at 1:44:11 PM UTC-7, Guido van Rossum wrote: > > I'm beginning to think that it would be useful to add > another ABC to collections.abc (and to typing) that represents the > intersection of these three ABCs, and to "upgrade" Set and Sequence to > inherit from it. (And Mapping.) > +1 on this, from the pytype side of things. pytype, during type-inference, always struggles to give names to things, and having additional structural types around helps. Also, I like "Collection" better than Reiterable, since the latter sounds too intricate, possibly scaring users away. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Fri Jul 22 13:51:19 2016 From: guido at python.org (Guido van Rossum) Date: Fri, 22 Jul 2016 10:51:19 -0700 Subject: [Python-ideas] Good uses for staticmethod In-Reply-To: References: Message-ID: Honestly, staticmethod was something of a mistake -- I was trying to do something like Java class methods but once it was released I found what was really needed was classmethod. But it was too late to get rid of staticmethod. From kramm at google.com Fri Jul 22 13:49:27 2016 From: kramm at google.com (Matthias Kramm) Date: Fri, 22 Jul 2016 10:49:27 -0700 (PDT) Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: Message-ID: On Thursday, July 21, 2016 at 1:44:11 PM UTC-7, Guido van Rossum wrote: > > I'm beginning to think that it would be useful to add > another ABC to collections.abc (and to typing) that represents the > intersection of these three ABCs, and to "upgrade" Set and Sequence to > inherit from it. (And Mapping.) > +1 on this, from the pytype side of things. pytype, during type-inference, always struggles to give names to things, and having additional structural types around helps. Also, I like "Collection" better than Reiterable, since the latter sounds too intricate, possibly scaring users away. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mertz at gnosis.cx Fri Jul 22 13:58:36 2016 From: mertz at gnosis.cx (David Mertz) Date: Fri, 22 Jul 2016 10:58:36 -0700 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: On Fri, Jul 22, 2016 at 8:48 AM, Chris Angelico wrote: > IMO, no. Some iterators can be restarted by going back to the original > iterable and requesting another iterator, but with no guarantee that > it will result in the exact same sequence (eg dict/set iterators). > Sequences don't give you this *guarantee* either. A trivial example: class MyList(list): def __getitem__(self, ndx): # In "real world" this might be meaningful condition and update if random() < .333: if isinstance(ndx, slice): for n in range(ndx.start, ndx.stop, ndx.step or 1): self[n] += 1 else: self[ndx] += 1 return super().__getitem__(ndx) > What you might be looking at is a protocol for "bookmarking" or > "forking" an iterator. That might be more useful. For example: > How is this different from itertools.tee()? -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: From egregius313 at gmail.com Fri Jul 22 14:31:17 2016 From: egregius313 at gmail.com (Edward Minnix) Date: Fri, 22 Jul 2016 14:31:17 -0400 Subject: [Python-ideas] Good uses for staticmethod In-Reply-To: References: Message-ID: Okay, thanks > On Jul 22, 2016, at 1:51 PM, Guido van Rossum wrote: > > Honestly, staticmethod was something of a mistake -- I was trying to > do something like Java class methods but once it was released I found > what was really needed was classmethod. But it was too late to get rid > of staticmethod. From random832 at fastmail.com Fri Jul 22 14:41:16 2016 From: random832 at fastmail.com (Random832) Date: Fri, 22 Jul 2016 14:41:16 -0400 Subject: [Python-ideas] Good uses for staticmethod In-Reply-To: References: Message-ID: <1469212876.1554957.674095473.1A5AB732@webmail.messagingengine.com> On Fri, Jul 22, 2016, at 13:51, Guido van Rossum wrote: > Honestly, staticmethod was something of a mistake -- I was trying to > do something like Java class methods but once it was released I found > what was really needed was classmethod. But it was too late to get rid > of staticmethod. Regardless of its utility or lack thereof for methods, staticmethod seems to me to be the best way to make an attribute of a class that happens to be a function (or, theoretically, some other other descriptor object). I like to think of it as an "implicit-magic remover". Probably if it didn't exist something like it would have been proposed eventually, under some other name like @funcattr or something. From rosuav at gmail.com Fri Jul 22 15:05:50 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 23 Jul 2016 05:05:50 +1000 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: On Sat, Jul 23, 2016 at 3:58 AM, David Mertz wrote: >> What you might be looking at is a protocol for "bookmarking" or >> "forking" an iterator. That might be more useful. For example: > > > How is this different from itertools.tee()? Functionally? Not at all. In fact, itertools.tee could make use of the fork protocol. The idea would be efficiency; iterators that are "restartable" by your definition would be "forkable", which would effectively permit tee to just ask the iterator to tee itself. The simple and naive implementation of tee() is "save all the elements until they get asked for", which requires extra storage for those elements; if your iterator can fork, that implies that you can get those elements right back again from the original. Imagine a list_iterator consists of two things, a base list and a current index. Tee'ing that iterator means retrieving elements and hanging onto them until needed; forking means creating another iterator with the same index. Which means that a forked iterator would reflect changes to the original list, for better or for worse. Not saying it's necessarily a good thing, but it's a more general version of the concept of "restarting" an iterator, being more compatible with trimmed iterators and such. Ultimately, your "restart" is no different from itertools.tee either. ChrisA From guido at python.org Fri Jul 22 15:08:13 2016 From: guido at python.org (Guido van Rossum) Date: Fri, 22 Jul 2016 12:08:13 -0700 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: Can someone create a bug and a patch to add SizedIterable to collections.abc and typing in 3.6? From brett at python.org Sat Jul 23 11:47:34 2016 From: brett at python.org (Brett Cannon) Date: Sat, 23 Jul 2016 15:47:34 +0000 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: On Fri, 22 Jul 2016 at 12:09 Guido van Rossum wrote: > Can someone create a bug Sure: http://bugs.python.org/issue27598 > and a patch to add SizedIterable to > collections.abc and typing in 3.6? > That's something I will leave up to someone else as I have file system path stuff to do. :) -Brett > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From storchaka at gmail.com Sat Jul 23 12:57:08 2016 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sat, 23 Jul 2016 19:57:08 +0300 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: On 22.07.16 18:48, Chris Angelico wrote: > On Sat, Jul 23, 2016 at 12:20 AM, Daniel Moisset > wrote: >> Would there be interest in some kind of method/API for restarting iterators? >> If there was a it.restart() (or reset(it)), it would probably be a good idea >> to name the concept in the same way as the method ( "Restartable" or >> "Resettable"). And having that as an actual protocol would simplify the >> discussion on both what the type should contain and how it should be named, >> while having actal applications to simplify code (and reduce memory usage in >> cases when creating a list is not needed, just making multiple passes). > > IMO, no. Some iterators can be restarted by going back to the original > iterable and requesting another iterator, but with no guarantee that > it will result in the exact same sequence (eg dict/set iterators). > Does that count as restarting? And what if you start by consuming a > header or two, then pass the resulting iterator to another function. > Based on type, it would be restartable (say it's a list_iterator), but > the called function would be highly surprised to suddenly get back > more results. > > What you might be looking at is a protocol for "bookmarking" or > "forking" an iterator. That might be more useful. There is such a protocol. Use copy.copy(). From rosuav at gmail.com Sat Jul 23 19:48:44 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 24 Jul 2016 09:48:44 +1000 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: On Sun, Jul 24, 2016 at 2:57 AM, Serhiy Storchaka wrote: > On 22.07.16 18:48, Chris Angelico wrote: >> >> On Sat, Jul 23, 2016 at 12:20 AM, Daniel Moisset >> wrote: >>> >>> Would there be interest in some kind of method/API for restarting >>> iterators? >>> If there was a it.restart() (or reset(it)), it would probably be a good >>> idea >>> to name the concept in the same way as the method ( "Restartable" or >>> "Resettable"). And having that as an actual protocol would simplify the >>> discussion on both what the type should contain and how it should be >>> named, >>> while having actal applications to simplify code (and reduce memory usage >>> in >>> cases when creating a list is not needed, just making multiple passes). >> >> >> IMO, no. Some iterators can be restarted by going back to the original >> iterable and requesting another iterator, but with no guarantee that >> it will result in the exact same sequence (eg dict/set iterators). >> Does that count as restarting? And what if you start by consuming a >> header or two, then pass the resulting iterator to another function. >> Based on type, it would be restartable (say it's a list_iterator), but >> the called function would be highly surprised to suddenly get back >> more results. >> >> What you might be looking at is a protocol for "bookmarking" or >> "forking" an iterator. That might be more useful. > > > There is such a protocol. Use copy.copy(). Time machine strikes again! Restarting of iterators exists. ChrisA From gvanrossum at gmail.com Sat Jul 23 20:15:32 2016 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sat, 23 Jul 2016 17:15:32 -0700 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: Does it work? For what categories of iterators? Surely not for streams... --Guido (mobile) On Jul 23, 2016 4:49 PM, "Chris Angelico" wrote: > On Sun, Jul 24, 2016 at 2:57 AM, Serhiy Storchaka > wrote: > > On 22.07.16 18:48, Chris Angelico wrote: > >> > >> On Sat, Jul 23, 2016 at 12:20 AM, Daniel Moisset > >> wrote: > >>> > >>> Would there be interest in some kind of method/API for restarting > >>> iterators? > >>> If there was a it.restart() (or reset(it)), it would probably be a good > >>> idea > >>> to name the concept in the same way as the method ( "Restartable" or > >>> "Resettable"). And having that as an actual protocol would simplify the > >>> discussion on both what the type should contain and how it should be > >>> named, > >>> while having actal applications to simplify code (and reduce memory > usage > >>> in > >>> cases when creating a list is not needed, just making multiple passes). > >> > >> > >> IMO, no. Some iterators can be restarted by going back to the original > >> iterable and requesting another iterator, but with no guarantee that > >> it will result in the exact same sequence (eg dict/set iterators). > >> Does that count as restarting? And what if you start by consuming a > >> header or two, then pass the resulting iterator to another function. > >> Based on type, it would be restartable (say it's a list_iterator), but > >> the called function would be highly surprised to suddenly get back > >> more results. > >> > >> What you might be looking at is a protocol for "bookmarking" or > >> "forking" an iterator. That might be more useful. > > > > > > There is such a protocol. Use copy.copy(). > > Time machine strikes again! Restarting of iterators exists. > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Sat Jul 23 20:54:42 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 24 Jul 2016 10:54:42 +1000 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: On Sun, Jul 24, 2016 at 10:15 AM, Guido van Rossum wrote: > Does it work? For what categories of iterators? Surely not for streams... I tried it with list and set iterators and it worked fine. With a generator, it threw an exception (though not something tidy like UncopyableObjectException - it complained about calling object.__new__(generator) instead of generator.__new__). Presumably it'll fail, one way or another, with non-duplicable objects (eg an open file, which is its own iterator, simply raises TypeError "cannot serialize"); those are non-restartable, non-bookmarkable, non-forkable iterators (take your pick of terminology). ChrisA From steve at pearwood.info Sat Jul 23 23:07:07 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 24 Jul 2016 13:07:07 +1000 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: Message-ID: <20160724030706.GN27919@ando.pearwood.info> On Sun, Jul 24, 2016 at 09:48:44AM +1000, Chris Angelico wrote: > On Sun, Jul 24, 2016 at 2:57 AM, Serhiy Storchaka wrote: > > There is such a protocol. Use copy.copy(). > > Time machine strikes again! Restarting of iterators exists. You can copy *some* iterators, but it doesn't restart them. py> import copy py> it = iter([1, 2, 3]) py> next(it) 1 py> next(it) 2 py> a = copy.copy(it) py> next(a) 3 py> next(it) 3 I wouldn't expect that copying an iterator would, in general, restart it. In general, there's no way to restart an iterator. Once seen, values are gone and cannot be recreated. -- Steve From wjun77 at gmail.com Sat Jul 23 23:39:56 2016 From: wjun77 at gmail.com (=?UTF-8?B?546L54+6?=) Date: Sun, 24 Jul 2016 11:39:56 +0800 Subject: [Python-ideas] Good uses for staticmethod In-Reply-To: <1469212876.1554957.674095473.1A5AB732@webmail.messagingengine.com> References: <1469212876.1554957.674095473.1A5AB732@webmail.messagingengine.com> Message-ID: There is no module or namespace keyword in python. One module consists at least one file. But sometimes I'm too lazy to create multiple files, or I simply cannot, e.g. when writing a plugin for something. I write staticmethod as a replacement of module function in these cases. 2016-07-23 2:41 GMT+08:00 Random832 : > On Fri, Jul 22, 2016, at 13:51, Guido van Rossum wrote: > > Honestly, staticmethod was something of a mistake -- I was trying to > > do something like Java class methods but once it was released I found > > what was really needed was classmethod. But it was too late to get rid > > of staticmethod. > > Regardless of its utility or lack thereof for methods, staticmethod > seems to me to be the best way to make an attribute of a class that > happens to be a function (or, theoretically, some other other descriptor > object). I like to think of it as an "implicit-magic remover". > > Probably if it didn't exist something like it would have been proposed > eventually, under some other name like @funcattr or something. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From storchaka at gmail.com Sun Jul 24 01:03:31 2016 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sun, 24 Jul 2016 08:03:31 +0300 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: On 24.07.16 03:15, Guido van Rossum wrote: > Does it work? For what categories of iterators? Surely not for streams... It works for iterators of all standard collections except Python implementation of OrderedDict. Not for streams, not for generators. From guido at python.org Sun Jul 24 01:48:42 2016 From: guido at python.org (Guido van Rossum) Date: Sat, 23 Jul 2016 22:48:42 -0700 Subject: [Python-ideas] Good uses for staticmethod In-Reply-To: References: <1469212876.1554957.674095473.1A5AB732@webmail.messagingengine.com> Message-ID: What you do in the privacy of your own code is your business, but please don't pretend it's good style... On Saturday, July 23, 2016, ?? wrote: > There is no module or namespace keyword in python. One module consists at > least one file. > But sometimes I'm too lazy to create multiple files, or I simply cannot, > e.g. when writing a plugin for something. > I write staticmethod as a replacement of module function in these cases. > > 2016-07-23 2:41 GMT+08:00 Random832 >: > >> On Fri, Jul 22, 2016, at 13:51, Guido van Rossum wrote: >> > Honestly, staticmethod was something of a mistake -- I was trying to >> > do something like Java class methods but once it was released I found >> > what was really needed was classmethod. But it was too late to get rid >> > of staticmethod. >> >> Regardless of its utility or lack thereof for methods, staticmethod >> seems to me to be the best way to make an attribute of a class that >> happens to be a function (or, theoretically, some other other descriptor >> object). I like to think of it as an "implicit-magic remover". >> >> Probably if it didn't exist something like it would have been proposed >> eventually, under some other name like @funcattr or something. >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > -- --Guido (mobile) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sun Jul 24 06:41:58 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 24 Jul 2016 20:41:58 +1000 Subject: [Python-ideas] Addition to operator module: starcaller In-Reply-To: <57915EA4.6080602@canterbury.ac.nz> References: <57915EA4.6080602@canterbury.ac.nz> Message-ID: On 22 July 2016 at 09:45, Greg Ewing wrote: > Daniel Spitz wrote: >> >> I'm not sure and don't have a strong opinion about the original removal of >> apply. > > > I expect it was removed simply because the * and ** calling > syntax makes it unecessary (before that was introduced, > apply was the only way of getting that funcionality). > > If it's to be reintroduced, the operator module would seem > to be the right place for it. I guess folks doing higher order functions are already used to importing the operator module for access to operator.add (etc), and this is a pretty thin wrapper around the __call__ protocol (unlike reduce(), which has no syntactic counterpart). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From dev at roznowski.net Mon Jul 25 04:04:27 2016 From: dev at roznowski.net (Achim Andreas von Roznowski) Date: Mon, 25 Jul 2016 10:04:27 +0200 Subject: [Python-ideas] add jsonrpc to stdlib "Internet Protocols and Support" akin to xmlrpc Message-ID: Hi I am new to this mailing list, hopefully this idea hasn't been raised already - I searched the PEPs and python-ideas and couldn't find anything similar, so decided to post this idea. Please forgive me if it is a duplicate. Idea: add jsonprc server and client to stdlib "Internet Protocols and Support" Details I would like to suggest adding a jsonrpc server and client to the stdlib "Internet Protocols and Support" with identical functionality to the existing xmlrpc.server and client but using json instead of xml, including a subclass SimpleJSONRPCServer just like SimpleXMLRPCServer based on socketserver.TCPServer. Rationale JSON is a very common data encoding format, widely established and very close to python's dictionaries, yet there exists today no jsonrpc server in stdlib. Many jsonrpc implementations using external libraries are available, so there is clearly demand, but no standard python library. All of these external implementations seem to require other external libraries for transport, eg. Werkzeug - no way to do this just using stdlib classes. Envisaged Features Identical to existing xmlrpc, just using json instead of xml encoding: --> https://docs.python.org/3/library/xmlrpc.server.html#simplexmlrpcserver-objects Effort I am too new to python-dev so others should estimate this, but it seems to me that all the hard work has already been done with xmlrpc - only the encoding would have to be adapted to encode/decode json instead of xml, which seems trivial, given that a json module already exists in stdlib. I appreciate your feedback. Kind regards, Achim Andreas von Roznowski From brett at python.org Mon Jul 25 12:50:27 2016 From: brett at python.org (Brett Cannon) Date: Mon, 25 Jul 2016 16:50:27 +0000 Subject: [Python-ideas] add jsonrpc to stdlib "Internet Protocols and Support" akin to xmlrpc In-Reply-To: References: Message-ID: On Mon, 25 Jul 2016 at 01:05 Achim Andreas von Roznowski wrote: > Hi > > I am new to this mailing list, hopefully this idea hasn't been raised > already - I searched the PEPs and python-ideas and couldn't find anything > similar, so decided to post this idea. Please forgive me if it is a > duplicate. > > Idea: add jsonprc server and client to stdlib "Internet Protocols and > Support" > > Details > I would like to suggest adding a jsonrpc server and client to the stdlib > "Internet Protocols and Support" with identical functionality to the > existing xmlrpc.server and client but using json instead of xml, including > a subclass SimpleJSONRPCServer just like SimpleXMLRPCServer based on > socketserver.TCPServer. > > Rationale > JSON is a very common data encoding format, widely established and very > close to python's dictionaries, yet there exists today no jsonrpc server > in stdlib. > Many jsonrpc implementations using external libraries are available, so > there is clearly demand, but no standard python library. > All of these external implementations seem to require other external > libraries for transport, eg. Werkzeug - no way to do this just using > stdlib classes. > > > Envisaged Features > Identical to existing xmlrpc, just using json instead of xml encoding: > --> > > https://docs.python.org/3/library/xmlrpc.server.html#simplexmlrpcserver-objects > > > Effort > I am too new to python-dev so others should estimate this, but it seems to > me that all the hard work has already been done with xmlrpc - only the > encoding would have to be adapted to encode/decode json instead of xml, > which seems trivial, given that a json module already exists in stdlib. > > > I appreciate your feedback. > Thanks for the idea, Achim, but I don't think a JSON-RPC library should be in the stdlib. Honestly, if someone was proposing an XML-RPC library to go into the stdlib today I would argue for it not to be included. I don't think JSON-RPC is used by the vast majority of Python users nor difficult enough to implement correctly to warrant being in the stdlib. And even if we thought a JSON-RPC library should be included, I would argue it should be asynchronous, meaning that the xmlrpc library couldn't be reused in any way for the implementation. -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg at krypto.org Mon Jul 25 13:19:53 2016 From: greg at krypto.org (Gregory P. Smith) Date: Mon, 25 Jul 2016 17:19:53 +0000 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: On Thu, Jul 21, 2016 at 11:45 PM Stephan Hoyer wrote: > On Thu, Jul 21, 2016 at 9:04 PM, Guido van Rossum > wrote: > >> If it really must be two words, how about SizedIterable? That suggests >> it's a subclass of Sized and Iterable, which it is. Container doesn't >> need separate mention, it's pretty obvious that any Iterable can >> implement __contains__, and more people know that Iterable is an ABC >> than Container. > > > In my experiments with type annotations, we recently encountered this > exact same issue (variables for which either sets or lists should be > valid). My initial solution was almost exactly what you propose here: we > wrote a SizedIterable class, simply inheriting from Sized and Iterable. > > Ultimately, we didn't find this very satisfying, because we realized that > strings are SizedIterables (they're sequences, too), and we really didn't > want to allow accidentally passing strings that would be interpreted as > iterables of characters. Unfortunately, I don't think there's any good way > in Python to specify "any sized iterable that isn't a string". > Given how often the str problem comes up in this context, I'm thinking a common special case in type annotation checkers that explicitly exclude str from consideration for iteration. It's behavior is rarely what the programmer intended and more likely to be a bug. -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Jul 25 13:58:04 2016 From: guido at python.org (Guido van Rossum) Date: Mon, 25 Jul 2016 10:58:04 -0700 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: On Mon, Jul 25, 2016 at 10:19 AM, Gregory P. Smith wrote: > Given how often the str problem comes up in this context, I'm thinking a > common special case in type annotation checkers that explicitly exclude str > from consideration for iteration. It's behavior is rarely what the > programmer intended and more likely to be a bug. Should we have IterableButNotString, CollectionExceptString, NonStringSequence, or all three? I suppose the runtime check would be e.g. isinstance(x, Iterable) and not isinstance(x, (str, bytes, bytearray, memoryview))? And the type checker should special-case the snot out of it? -- --Guido van Rossum (python.org/~guido) From ralph at ralphbroenink.net Mon Jul 25 13:55:27 2016 From: ralph at ralphbroenink.net (Ralph Broenink) Date: Mon, 25 Jul 2016 17:55:27 +0000 Subject: [Python-ideas] Making the stdlib consistent again Message-ID: Hi python-ideas, As you all know, the Python stdlib can sometimes be a bit of an inconsistent mess that can be surprising in how it names things. This is mostly caused by the fact that several modules were developed before the introduction of PEP-8, and now we're stuck with the older naming within these modules. It has been said and discussed in the past [1][2] that the stdlib is in fact inconsistent, but fixing this has almost always been disregarded as being too painful (after all, we don't want a new Python 3 all over again). However, this way, we will never move away from these inconsistencies. Perhaps this is fine, but I think we should at least consider providing function and class names that are unsurprising for developers. While maintaining full backwards compatibility, my idea is that we should offer consistently named aliases in -eventually- all stdlib modules. For instance, with Python 2.6, the threading module received this treatment, but unfortunately this was not expanded to all modules. What am I speaking of precisely? I have done a quick survey of the stdlib and found the following examples. Please note, this is a highly opinionated list; some names may have been chosen with a very good reason, and others are just a matter of taste. Hopefully you agree with at least some of them: * The CamelCasing in some modules are the most obvious culprits, e.g. logging and unittest. There is obviously an issue regarding subclasses and methods that are supposed to be overridden, but I feel we could make it work. * All lower case class names, such as collections.defaultdict and collections.deque, should be CamelCased. Another example is datetime, which uses names such as timedelta instead of TimeDelta. * Inconsistent names all together, such as re.sub, which I feel should be re.replace (cf. str.replace). But also re.finditer and re.findall, but no re.find. * Names that do not reflect actual usage, such as ssl.PROTOCOL_SSLv23, which can in fact not be used as client for SSLv2. * Underscore usage, such as tarfile.TarFile.gettarinfo (should it not be get_tar_info?), http.client.HTTPConnection.getresponse vs set_debuglevel, and pathlib.Path.samefile vs pathlib.Path.read_text. And is it pkgutil.iter_modules or is it pathlib.Path.iterdir (or re.finditer)? * Usage of various abbreviations, such as in filecmp.cmp * Inconsistencies between similar modules, e.g. between tarfile.TarFile.add and zipfile.ZipFile.write. These are just some examples of inconsistent and surprising naming I could find, other categories are probably also conceivable. Another subject for reconsideration would be attribute and argument names, but I haven't looked for those in my quick survey. For all of these inconsistencies, I think we should make a 'consistently' named alternative, and alias the original variant with them (or the other way around), without defining a deprecation timeline for the original names. This should make it possible to eventually make the stdlib consistent, Pythonic and unsurprising. What would you think of such an effort? Regards, Ralph Broenink [1] https://mail.python.org/pipermail/python-ideas/2010-January/006755.html [2] https://mail.python.org/pipermail/python-dev/2009-March/086646.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Mon Jul 25 15:23:26 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 25 Jul 2016 20:23:26 +0100 Subject: [Python-ideas] Making the stdlib consistent again In-Reply-To: References: Message-ID: On 25 July 2016 at 18:55, Ralph Broenink wrote: > For all of these inconsistencies, I think we should make a 'consistently' > named alternative, and alias the original variant with them (or the other > way around), without defining a deprecation timeline for the original names. > This should make it possible to eventually make the stdlib consistent, > Pythonic and unsurprising. > > What would you think of such an effort? It sounds to me like rather a lot of effort, for limited benefit. I suspect few people will actually start using the new names (as they likely have to continue supporting older versions of Python for a long time yet) and so the benefit will actually be lower than we'd like. Also, you'd need to put a lot of effort into updating the tests - should they use the new or the old names, and how would they test that both names behave identically? So there's probably more effort than you'd think. Also, there's a lot of tutorial material - books, courses, websites - that would need to change. So overall, I think practicality beats purity here, and such a change is not worth the effort. Paul From jelle.zijlstra at gmail.com Mon Jul 25 15:29:53 2016 From: jelle.zijlstra at gmail.com (Jelle Zijlstra) Date: Mon, 25 Jul 2016 12:29:53 -0700 Subject: [Python-ideas] Making the stdlib consistent again In-Reply-To: References: Message-ID: 2016-07-25 10:55 GMT-07:00 Ralph Broenink : > Hi python-ideas, > > As you all know, the Python stdlib can sometimes be a bit of an > inconsistent mess that can be surprising in how it names things. This is > mostly caused by the fact that several modules were developed before the > introduction of PEP-8, and now we're stuck with the older naming within > these modules. > > It has been said and discussed in the past [1][2] that the stdlib is in > fact inconsistent, but fixing this has almost always been disregarded as > being too painful (after all, we don't want a new Python 3 all over again). > However, this way, we will never move away from these inconsistencies. > Perhaps this is fine, but I think we should at least consider providing > function and class names that are unsurprising for developers. > > While maintaining full backwards compatibility, my idea is that we should > offer consistently named aliases in -eventually- all stdlib modules. For > instance, with Python 2.6, the threading module received this treatment, > but unfortunately this was not expanded to all modules. > > What am I speaking of precisely? I have done a quick survey of the stdlib > and found the following examples. Please note, this is a highly opinionated > list; some names may have been chosen with a very good reason, and others > are just a matter of taste. Hopefully you agree with at least some of them: > > * The CamelCasing in some modules are the most obvious culprits, e.g. > logging and unittest. There is obviously an issue regarding subclasses and > methods that are supposed to be overridden, but I feel we could make it > work. > > * All lower case class names, such as collections.defaultdict and > collections.deque, should be CamelCased. Another example is datetime, which > uses names such as timedelta instead of TimeDelta. > > Does that also go for builtin classes, such as int and str and object and type? > * Inconsistent names all together, such as re.sub, which I feel should > be re.replace (cf. str.replace). But also re.finditer and re.findall, but > no re.find. > > * Names that do not reflect actual usage, such as ssl.PROTOCOL_SSLv23, > which can in fact not be used as client for SSLv2. > > * Underscore usage, such as tarfile.TarFile.gettarinfo (should it not be > get_tar_info?), http.client.HTTPConnection.getresponse vs set_debuglevel, > and pathlib.Path.samefile vs pathlib.Path.read_text. And is it > pkgutil.iter_modules or is it pathlib.Path.iterdir (or re.finditer)? > > * Usage of various abbreviations, such as in filecmp.cmp > > * Inconsistencies between similar modules, e.g. between > tarfile.TarFile.add and zipfile.ZipFile.write. > > These are just some examples of inconsistent and surprising naming I could > find, other categories are probably also conceivable. Another subject for > reconsideration would be attribute and argument names, but I haven't looked > for those in my quick survey. > > For all of these inconsistencies, I think we should make a 'consistently' > named alternative, and alias the original variant with them (or the other > way around), without defining a deprecation timeline for the original > names. This should make it possible to eventually make the stdlib > consistent, Pythonic and unsurprising. > > What would you think of such an effort? > > Regards, > Ralph Broenink > > [1] > https://mail.python.org/pipermail/python-ideas/2010-January/006755.html > [2] https://mail.python.org/pipermail/python-dev/2009-March/086646.html > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Jul 25 15:58:28 2016 From: guido at python.org (Guido van Rossum) Date: Mon, 25 Jul 2016 12:58:28 -0700 Subject: [Python-ideas] Making the stdlib consistent again In-Reply-To: References: Message-ID: Ralph, You seem to be vastly underestimating the cost of making backwards incompatible changes like these, while sounding naively optimistic about the value of consistency. Please understand the second section heading of PEP 8. --Guido On Mon, Jul 25, 2016 at 10:55 AM, Ralph Broenink wrote: > Hi python-ideas, > > As you all know, the Python stdlib can sometimes be a bit of an inconsistent > mess that can be surprising in how it names things. This is mostly caused by > the fact that several modules were developed before the introduction of > PEP-8, and now we're stuck with the older naming within these modules. > > It has been said and discussed in the past [1][2] that the stdlib is in fact > inconsistent, but fixing this has almost always been disregarded as being > too painful (after all, we don't want a new Python 3 all over again). > However, this way, we will never move away from these inconsistencies. > Perhaps this is fine, but I think we should at least consider providing > function and class names that are unsurprising for developers. > > While maintaining full backwards compatibility, my idea is that we should > offer consistently named aliases in -eventually- all stdlib modules. For > instance, with Python 2.6, the threading module received this treatment, but > unfortunately this was not expanded to all modules. > > What am I speaking of precisely? I have done a quick survey of the stdlib > and found the following examples. Please note, this is a highly opinionated > list; some names may have been chosen with a very good reason, and others > are just a matter of taste. Hopefully you agree with at least some of them: > > * The CamelCasing in some modules are the most obvious culprits, e.g. > logging and unittest. There is obviously an issue regarding subclasses and > methods that are supposed to be overridden, but I feel we could make it > work. > > * All lower case class names, such as collections.defaultdict and > collections.deque, should be CamelCased. Another example is datetime, which > uses names such as timedelta instead of TimeDelta. > > * Inconsistent names all together, such as re.sub, which I feel should be > re.replace (cf. str.replace). But also re.finditer and re.findall, but no > re.find. > > * Names that do not reflect actual usage, such as ssl.PROTOCOL_SSLv23, > which can in fact not be used as client for SSLv2. > > * Underscore usage, such as tarfile.TarFile.gettarinfo (should it not be > get_tar_info?), http.client.HTTPConnection.getresponse vs set_debuglevel, > and pathlib.Path.samefile vs pathlib.Path.read_text. And is it > pkgutil.iter_modules or is it pathlib.Path.iterdir (or re.finditer)? > > * Usage of various abbreviations, such as in filecmp.cmp > > * Inconsistencies between similar modules, e.g. between > tarfile.TarFile.add and zipfile.ZipFile.write. > > These are just some examples of inconsistent and surprising naming I could > find, other categories are probably also conceivable. Another subject for > reconsideration would be attribute and argument names, but I haven't looked > for those in my quick survey. > > For all of these inconsistencies, I think we should make a 'consistently' > named alternative, and alias the original variant with them (or the other > way around), without defining a deprecation timeline for the original names. > This should make it possible to eventually make the stdlib consistent, > Pythonic and unsurprising. > > What would you think of such an effort? > > Regards, > Ralph Broenink > > [1] https://mail.python.org/pipermail/python-ideas/2010-January/006755.html > [2] https://mail.python.org/pipermail/python-dev/2009-March/086646.html > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -- --Guido van Rossum (python.org/~guido) From bufordsharkley at gmail.com Mon Jul 25 16:02:55 2016 From: bufordsharkley at gmail.com (Mark Mollineaux) Date: Mon, 25 Jul 2016 13:02:55 -0700 Subject: [Python-ideas] Making the stdlib consistent again In-Reply-To: References: Message-ID: I've pined for this (and feel a real mental pain every time I use one of those poorlyCased names)-- I end up using a lot of mental space remembering exactly HOW each stdlib isn't consistent. Aliasing consistent names in each case seems like a real win all around, personally. On Mon, Jul 25, 2016 at 10:55 AM, Ralph Broenink wrote: > Hi python-ideas, > > As you all know, the Python stdlib can sometimes be a bit of an inconsistent > mess that can be surprising in how it names things. This is mostly caused by > the fact that several modules were developed before the introduction of > PEP-8, and now we're stuck with the older naming within these modules. > > It has been said and discussed in the past [1][2] that the stdlib is in fact > inconsistent, but fixing this has almost always been disregarded as being > too painful (after all, we don't want a new Python 3 all over again). > However, this way, we will never move away from these inconsistencies. > Perhaps this is fine, but I think we should at least consider providing > function and class names that are unsurprising for developers. > > While maintaining full backwards compatibility, my idea is that we should > offer consistently named aliases in -eventually- all stdlib modules. For > instance, with Python 2.6, the threading module received this treatment, but > unfortunately this was not expanded to all modules. > > What am I speaking of precisely? I have done a quick survey of the stdlib > and found the following examples. Please note, this is a highly opinionated > list; some names may have been chosen with a very good reason, and others > are just a matter of taste. Hopefully you agree with at least some of them: > > * The CamelCasing in some modules are the most obvious culprits, e.g. > logging and unittest. There is obviously an issue regarding subclasses and > methods that are supposed to be overridden, but I feel we could make it > work. > > * All lower case class names, such as collections.defaultdict and > collections.deque, should be CamelCased. Another example is datetime, which > uses names such as timedelta instead of TimeDelta. > > * Inconsistent names all together, such as re.sub, which I feel should be > re.replace (cf. str.replace). But also re.finditer and re.findall, but no > re.find. > > * Names that do not reflect actual usage, such as ssl.PROTOCOL_SSLv23, > which can in fact not be used as client for SSLv2. > > * Underscore usage, such as tarfile.TarFile.gettarinfo (should it not be > get_tar_info?), http.client.HTTPConnection.getresponse vs set_debuglevel, > and pathlib.Path.samefile vs pathlib.Path.read_text. And is it > pkgutil.iter_modules or is it pathlib.Path.iterdir (or re.finditer)? > > * Usage of various abbreviations, such as in filecmp.cmp > > * Inconsistencies between similar modules, e.g. between > tarfile.TarFile.add and zipfile.ZipFile.write. > > These are just some examples of inconsistent and surprising naming I could > find, other categories are probably also conceivable. Another subject for > reconsideration would be attribute and argument names, but I haven't looked > for those in my quick survey. > > For all of these inconsistencies, I think we should make a 'consistently' > named alternative, and alias the original variant with them (or the other > way around), without defining a deprecation timeline for the original names. > This should make it possible to eventually make the stdlib consistent, > Pythonic and unsurprising. > > What would you think of such an effort? > > Regards, > Ralph Broenink > > [1] https://mail.python.org/pipermail/python-ideas/2010-January/006755.html > [2] https://mail.python.org/pipermail/python-dev/2009-March/086646.html > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From rdmurray at bitdance.com Mon Jul 25 16:20:34 2016 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 25 Jul 2016 16:20:34 -0400 Subject: [Python-ideas] Making the stdlib consistent again In-Reply-To: References: Message-ID: <20160725202035.C45EBB1401C@webabinitio.net> On Mon, 25 Jul 2016 17:55:27 -0000, Ralph Broenink wrote: > While maintaining full backwards compatibility, my idea is that we should > offer consistently named aliases in -eventually- all stdlib modules. For > instance, with Python 2.6, the threading module received this treatment, > but unfortunately this was not expanded to all modules. Right. We tried it. It didn't work out very well (really bad cost/benefit ratio), so we stopped.[*] --David [*] At least, that's my understanding, I wasn't actually around yet when then threading changes were made. From bzvi7919 at gmail.com Mon Jul 25 17:29:08 2016 From: bzvi7919 at gmail.com (Bar Harel) Date: Mon, 25 Jul 2016 21:29:08 +0000 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: Guido, I'm not entirely sure if you're joking or not, but if we'll start with this, we will never finish. I believe that although str/bytes truly are special regarding iterables, the combinations are endless. Soon enough we'll have NotBytesSequence, NumericSequenceWithoutFloat and BuiltinNotCustomMadeIterable. We cannot guarantee a "meaning". We can however guarantee an interface or API, in this case __iter__, and promote duck-typing in order to support even the craziest custom iterables. I think breaking the design or the state-of-mind in case of " IterableButNotString" would be a mistake, unless we're planning to remove str's __iter__ altogether (which I don't support either but who knows). Nevertheless, +1 for the unification of Iterable, Sized and Container or "Reiterables" as it declares a clearer interface. On Mon, Jul 25, 2016 at 9:03 PM Guido van Rossum wrote: > On Mon, Jul 25, 2016 at 10:19 AM, Gregory P. Smith > wrote: > > Given how often the str problem comes up in this context, I'm thinking a > > common special case in type annotation checkers that explicitly exclude > str > > from consideration for iteration. It's behavior is rarely what the > > programmer intended and more likely to be a bug. > > Should we have IterableButNotString, CollectionExceptString, > NonStringSequence, or all three? I suppose the runtime check would be > e.g. isinstance(x, Iterable) and not isinstance(x, (str, bytes, > bytearray, memoryview))? And the type checker should special-case the > snot out of it? > > -- > --Guido van Rossum (python.org/~guido) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Jul 25 18:59:03 2016 From: guido at python.org (Guido van Rossum) Date: Mon, 25 Jul 2016 15:59:03 -0700 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: I'm sorry, I was using Socratic questions to get people to think mount about the name and semantics for this one type. I certainly don't think we should have more of these! At least not until we've added an "except" type to PEP 484. On Mon, Jul 25, 2016 at 2:29 PM, Bar Harel wrote: > Guido, I'm not entirely sure if you're joking or not, but if we'll start > with this, we will never finish. > > I believe that although str/bytes truly are special regarding iterables, the > combinations are endless. > Soon enough we'll have NotBytesSequence, NumericSequenceWithoutFloat and > BuiltinNotCustomMadeIterable. > > We cannot guarantee a "meaning". We can however guarantee an interface or > API, in this case __iter__, > and promote duck-typing in order to support even the craziest custom > iterables. > > I think breaking the design or the state-of-mind in case of > "IterableButNotString" would be a mistake, unless > we're planning to remove str's __iter__ altogether (which I don't support > either but who knows). > > Nevertheless, +1 for the unification of Iterable, Sized and Container or > "Reiterables" as it declares a clearer interface. > > On Mon, Jul 25, 2016 at 9:03 PM Guido van Rossum wrote: >> >> On Mon, Jul 25, 2016 at 10:19 AM, Gregory P. Smith >> wrote: >> > Given how often the str problem comes up in this context, I'm thinking a >> > common special case in type annotation checkers that explicitly exclude >> > str >> > from consideration for iteration. It's behavior is rarely what the >> > programmer intended and more likely to be a bug. >> >> Should we have IterableButNotString, CollectionExceptString, >> NonStringSequence, or all three? I suppose the runtime check would be >> e.g. isinstance(x, Iterable) and not isinstance(x, (str, bytes, >> bytearray, memoryview))? And the type checker should special-case the >> snot out of it? >> >> -- >> --Guido van Rossum (python.org/~guido) >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ -- --Guido van Rossum (python.org/~guido) From greg.ewing at canterbury.ac.nz Mon Jul 25 19:58:26 2016 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 26 Jul 2016 11:58:26 +1200 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: <5796A7A2.4090409@canterbury.ac.nz> Bar Harel wrote: > Soon enough we'll have NotBytesSequence, NumericSequenceWithoutFloat and > BuiltinNotCustomMadeIterable. Is there some way we could provide a general way of constructing a "but not" ABC, so that people could build their own combinations as needed? -- Greg From rymg19 at gmail.com Mon Jul 25 20:22:23 2016 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Mon, 25 Jul 2016 19:22:23 -0500 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: <5796A7A2.4090409@canterbury.ac.nz> References: <57916007.5030504@canterbury.ac.nz> <5796A7A2.4090409@canterbury.ac.nz> Message-ID: I can't recall which, but I remember reading about a language with a relative complement operator on types. The idea is, where T is a set of types, then: T \ {A, B, ...} would be all types in T *except* A, B, .... I think maybe there could be a type variable Except that could be used like: Except[T, A, B, ...] where T is a type variable. (Other name ideas: Complement, Difference). -- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something?s wrong. http://kirbyfan64.github.io/ On Jul 25, 2016 6:58 PM, "Greg Ewing" wrote: > Bar Harel wrote: > >> Soon enough we'll have NotBytesSequence, NumericSequenceWithoutFloat and >> BuiltinNotCustomMadeIterable. >> > > Is there some way we could provide a general way of > constructing a "but not" ABC, so that people could > build their own combinations as needed? > > -- > Greg > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Jul 26 00:21:30 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 26 Jul 2016 14:21:30 +1000 Subject: [Python-ideas] Making the stdlib consistent again In-Reply-To: References: Message-ID: On 26 July 2016 at 05:23, Paul Moore wrote: > So overall, I think practicality beats purity here, and such a change > is not worth the effort. We did go through the effort for the threading module, as we had the problem there that multiprocessing was PEP-8 compliant (with aliases for threading compatibility), but threading only had the old pre-PEP-8 names. A couple of the conclusions that came out of that were: - it was probably worth it in threading's case due to the improved alignment with multiprocessing - it isn't worth the disruption in the general case The "isn't worth it" mainly comes from the fact that these APIs generally *were* compliant with the coding guidelines that existed at the time they were first written, it's just that the guidelines and community expectations have changed since then, so they represent things like "good Python design style circa 1999" (e.g. unittest, logging) rather than "good Python style for today". So if you say "let's update them to 2016 conventions" today, by 2026 you'll just have the same problem again. It ends up being one of those cases where "invest development time now to reduce cognitive burden later" doesn't actually work in practice, as you have to keep the old API around anyway if you don't want to break working code, and that means future learners now have two APIs to learn instead of one (and if you refresh the API to the design du jour ever decade or so, that number keeps going up). Instead, for these older parts of the standard library, it's useful to view them as "interoperability plumbing" (e.g. logging for event stream reporting and management, unittest for test case reporting and management), and look for more modern version independent 3rd party facades if you find the mix of API design eras in the standard library annoying (e.g. pytest as an alternative to raw unittest, structlog over direct use of the logging module) Cheers, Nick. P.S. I sometimes wonder if you could analyse Python API designs over time and group them into eras the way people do with building architectural styles - "This API is a fine example of the stylistic leanings of Python's Java period, which ran from to . As the community moved into the subsequent Haskell period, composition came to be favoured over inheritance, leading to the emergence of alternative APIs like X, Y, and Z, of which, Y ultimately proved to be most popular" :) -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From rosuav at gmail.com Tue Jul 26 00:38:56 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 26 Jul 2016 14:38:56 +1000 Subject: [Python-ideas] Making the stdlib consistent again In-Reply-To: References: Message-ID: On Tue, Jul 26, 2016 at 2:21 PM, Nick Coghlan wrote: > P.S. I sometimes wonder if you could analyse Python API designs over > time and group them into eras the way people do with building > architectural styles - "This API is a fine example of the stylistic > leanings of Python's Java period, which ran from to . As > the community moved into the subsequent Haskell period, composition > came to be favoured over inheritance, leading to the emergence of > alternative APIs like X, Y, and Z, of which, Y ultimately proved to be > most popular" :) This is yet another example of the neo-classic baroque Python code, and as I always say, if it's not baroque, don't fix it! -- Cogsworth ChrisA From ncoghlan at gmail.com Tue Jul 26 00:52:08 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 26 Jul 2016 14:52:08 +1000 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: On 26 July 2016 at 03:58, Guido van Rossum wrote: > On Mon, Jul 25, 2016 at 10:19 AM, Gregory P. Smith wrote: >> Given how often the str problem comes up in this context, I'm thinking a >> common special case in type annotation checkers that explicitly exclude str >> from consideration for iteration. It's behavior is rarely what the >> programmer intended and more likely to be a bug. > > Should we have IterableButNotString, CollectionExceptString, > NonStringSequence, or all three? I suppose the runtime check would be > e.g. isinstance(x, Iterable) and not isinstance(x, (str, bytes, > bytearray, memoryview))? And the type checker should special-case the > snot out of it? For cases where str instances are handled differently from other iterables, but still within the same function, a convention of writing "Union[str, Iterable]" as the input parameter type may suffice - while technically the union is redundant, the implication would be that str instances are treated as str objects, rather than as an iterable of length-1 str objects. The other case where this would come up is in signature overloading, where one overload may be typed as "str" and the other as "Iterable". Presumably in those cases the more specific type already wins. Anything more than that would need to be defined in the context of a particular algorithm, such as the oft-requested generic "flatten" operation, where you typically want to consider types like str, bytes, bytearray and memoryview as atomic objects, rather than as containers, but then things get blurry around iterable non-builtin types like array.array, numpy.ndarray, Pandas data frames, image formats with pixel addressing, etc, as well as builtins with multiple iteration behaviours like dict. One possible way to tackle that would be to declare a new collections.abc.AtomicIterable ABC, as well as a collections.flatten operation defined as something like: def flatten(iterable): for item in iterable: if isinstance(item, AtomicIterable): yield item continue if isinstance(item, Mapping): yield from item.items() continue try: subiter = iter(item) except TypeError: yield item else: yield from flatten(subiter) Over time, different types would get explicitly registered with AtomicIterable based on what their developers considered the most appropriate behaviour to be when asked to flatten them. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From greg.ewing at canterbury.ac.nz Tue Jul 26 02:53:36 2016 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 26 Jul 2016 18:53:36 +1200 Subject: [Python-ideas] Making the stdlib consistent again In-Reply-To: References: Message-ID: <579708F0.9060308@canterbury.ac.nz> Nick Coghlan wrote: > The "isn't worth it" mainly comes from the fact that these APIs > generally *were* compliant with the coding guidelines that existed at > the time they were first written Is that true? Did we really actively advise people to use Java-style APIs in some cases, or was it just that nobody told them otherwise? > So if you say > "let's update them to 2016 conventions" today, by 2026 you'll just > have the same problem again. You're assuming that the conventions will change just as much in the next 10 years as they did in the last 10. I don't think that's likely -- I would hope we're converging on a set of conventions that's good enough to endure. > and look for more modern version independent 3rd party > facades if you find the mix of API design eras in the standard library > annoying It would be sad if Python's motto became "Batteries included (as long as you're happy with Leclanche cells; if you want anything more modern you'll have to look elsewhere)". -- Greg From bzvi7919 at gmail.com Tue Jul 26 02:56:28 2016 From: bzvi7919 at gmail.com (Bar Harel) Date: Tue, 26 Jul 2016 06:56:28 +0000 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: I like your idea with AtomicIterable, but I still don't think it fully solves the problem. How can one categorize array.array? Consider an array of bytes. Sometimes you'll want to iterate over them as ints sometimes as 1 byte objects. In one case they can be used in order to conserve space as a sequence of ints in which case you'll want them one by one, and in the other case they'll be atomic just like bytes. Giving the register option on the other hand to the end user would pose a different problem - ABC.register is global. If you're developing a module and register array.array as an AtomicIterable, you might affect other parts of the program in unexpected ways. I think it's a tough one to solve :-/ On Tue, Jul 26, 2016, 7:52 AM Nick Coghlan wrote: > On 26 July 2016 at 03:58, Guido van Rossum wrote: > > On Mon, Jul 25, 2016 at 10:19 AM, Gregory P. Smith > wrote: > >> Given how often the str problem comes up in this context, I'm thinking a > >> common special case in type annotation checkers that explicitly exclude > str > >> from consideration for iteration. It's behavior is rarely what the > >> programmer intended and more likely to be a bug. > > > > Should we have IterableButNotString, CollectionExceptString, > > NonStringSequence, or all three? I suppose the runtime check would be > > e.g. isinstance(x, Iterable) and not isinstance(x, (str, bytes, > > bytearray, memoryview))? And the type checker should special-case the > > snot out of it? > > For cases where str instances are handled differently from other > iterables, but still within the same function, a convention of writing > "Union[str, Iterable]" as the input parameter type may suffice - while > technically the union is redundant, the implication would be that str > instances are treated as str objects, rather than as an iterable of > length-1 str objects. > > The other case where this would come up is in signature overloading, > where one overload may be typed as "str" and the other as "Iterable". > Presumably in those cases the more specific type already wins. > > Anything more than that would need to be defined in the context of a > particular algorithm, such as the oft-requested generic "flatten" > operation, where you typically want to consider types like str, bytes, > bytearray and memoryview as atomic objects, rather than as containers, > but then things get blurry around iterable non-builtin types like > array.array, numpy.ndarray, Pandas data frames, image formats with > pixel addressing, etc, as well as builtins with multiple iteration > behaviours like dict. > > One possible way to tackle that would be to declare a new > collections.abc.AtomicIterable ABC, as well as a collections.flatten > operation defined as something like: > > def flatten(iterable): > for item in iterable: > if isinstance(item, AtomicIterable): > yield item > continue > if isinstance(item, Mapping): > yield from item.items() > continue > try: > subiter = iter(item) > except TypeError: > yield item > else: > yield from flatten(subiter) > > Over time, different types would get explicitly registered with > AtomicIterable based on what their developers considered the most > appropriate behaviour to be when asked to flatten them. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pavol.lisy at gmail.com Tue Jul 26 03:02:58 2016 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Tue, 26 Jul 2016 09:02:58 +0200 Subject: [Python-ideas] Making the stdlib consistent again In-Reply-To: References: Message-ID: On 7/26/16, Nick Coghlan wrote: > On 26 July 2016 at 05:23, Paul Moore wrote: >> So overall, I think practicality beats purity here, and such a change >> is not worth the effort. > We did go through the effort for the threading module, as we had the > problem there that multiprocessing was PEP-8 compliant (with aliases > for threading compatibility), but threading only had the old pre-PEP-8 > names. > The "isn't worth it" mainly comes from the fact that these APIs > generally *were* compliant with the coding guidelines that existed at > the time they were first written, it's just that the guidelines and > community expectations have changed since then, so they represent > things like "good Python design style circa 1999" (e.g. unittest, > logging) rather than "good Python style for today". So if you say > "let's update them to 2016 conventions" today, by 2026 you'll just > have the same problem again. How could snake survive if not moulting regularly, when old skin is outgrown? I am not argue with you! Just thinking about future of this language and see some analogies in biology. And sorry just could not resist to share this little idea! :) From storchaka at gmail.com Tue Jul 26 03:42:58 2016 From: storchaka at gmail.com (Serhiy Storchaka) Date: Tue, 26 Jul 2016 10:42:58 +0300 Subject: [Python-ideas] Making the stdlib consistent again In-Reply-To: <579708F0.9060308@canterbury.ac.nz> References: <579708F0.9060308@canterbury.ac.nz> Message-ID: On 26.07.16 09:53, Greg Ewing wrote: > Nick Coghlan wrote: >> The "isn't worth it" mainly comes from the fact that these APIs >> generally *were* compliant with the coding guidelines that existed at >> the time they were first written > > Is that true? Did we really actively advise people to use > Java-style APIs in some cases, or was it just that > nobody told them otherwise? logging and xml.dom are exact reproducing of Java libraries on Python. These APIs are something like a standard and are reproduced in many other languages. From cory at lukasa.co.uk Tue Jul 26 03:53:44 2016 From: cory at lukasa.co.uk (Cory Benfield) Date: Tue, 26 Jul 2016 09:53:44 +0200 Subject: [Python-ideas] Making the stdlib consistent again In-Reply-To: References: Message-ID: <9167665A-CEF9-46B2-9326-BD5CAEF6CBD4@lukasa.co.uk> > On 25 Jul 2016, at 19:55, Ralph Broenink wrote: > > * Names that do not reflect actual usage, such as ssl.PROTOCOL_SSLv23, which can in fact not be used as client for SSLv2. PROTOCOL_SSLv23 *can* be used as a client for SSLv2, in some circumstances. If you want to fight about names, then we should talk about the fact that PROTOCOL_SSLv23 means ?use a SSLv3 handshake and then negotiate the highest commonly supported TLS version?. However, this naming scheme comes from OpenSSL (what we call PROTOCOL_SSLv23 OpenSSL calls SSLv23_METHOD, and so on). This naming scheme *is* terrible, but the stdlib ssl module is really more appropriately called the openssl module: it exposes a substantial number of OpenSSL internals and names. Happily, OpenSSL is changing this name to TLS_METHOD, which is what it should have been called all along. Cory From ncoghlan at gmail.com Tue Jul 26 10:30:18 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 27 Jul 2016 00:30:18 +1000 Subject: [Python-ideas] Making the stdlib consistent again In-Reply-To: <579708F0.9060308@canterbury.ac.nz> References: <579708F0.9060308@canterbury.ac.nz> Message-ID: On 26 July 2016 at 16:53, Greg Ewing wrote: > Nick Coghlan wrote: >> So if you say >> "let's update them to 2016 conventions" today, by 2026 you'll just >> have the same problem again. > > You're assuming that the conventions will change just as > much in the next 10 years as they did in the last 10. I > don't think that's likely -- I would hope we're converging > on a set of conventions that's good enough to endure. Conventions can really only remain stable if their environment doesn't change, and the software development world isn't that much more stable now than it was 20 years ago. >> and look for more modern version independent 3rd party >> facades if you find the mix of API design eras in the standard library >> annoying > > It would be sad if Python's motto became "Batteries > included (as long as you're happy with Leclanche cells; > if you want anything more modern you'll have to look > elsewhere)". New libraries can still make their way into the standard set, but it makes more sense to do that as *actual new libraries*, rather than simply PEP-8-ifying old ones. It's relatively easy to manage the cognitive burden of argparse vs optparse vs getopt, for example - you know which one people are using based on what they import, and they have different names so it's easy to talk about the trade-offs between them and tell people which one is recommended for new code. Ditto for asyncio vs asynchat and asyncore. We even have a section in the docs specifically for modules that are still around for backwards compatibility, but we don't recommend to new users: https://docs.python.org/3/library/superseded.html So evolving the standard library by introducing new ways of doing things as the community learns better alternatives, and requiring those new ways to be compliant with PEP 8 at the time they're standardised *absolutely* makes sense. The only part I don't think makes sense is modernising the APIs of existing modules purely for the sake of modernising them - there needs to be a stronger justification for expending that much effort (as there was with properly aligning the threading and multiprocessing APIs). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From brett at python.org Tue Jul 26 12:29:24 2016 From: brett at python.org (Brett Cannon) Date: Tue, 26 Jul 2016 16:29:24 +0000 Subject: [Python-ideas] Making the stdlib consistent again In-Reply-To: References: Message-ID: On Mon, 25 Jul 2016 at 13:03 Mark Mollineaux wrote: > I've pined for this (and feel a real mental pain every time I use one > of those poorlyCased names)-- I end up using a lot of mental space > remembering exactly HOW each stdlib isn't consistent. > > Aliasing consistent names in each case seems like a real win all > around, personally. > For those that want consistent names, you could create a PyPI package that is nothing more than the aliased names as suggested. Otherwise I get the desire for consistency, but as pointed out by a bunch of other core devs, we have thought about this many times and always reach the same conclusion that the amount of work and potential code breakage is too great. -Brett > > On Mon, Jul 25, 2016 at 10:55 AM, Ralph Broenink > wrote: > > Hi python-ideas, > > > > As you all know, the Python stdlib can sometimes be a bit of an > inconsistent > > mess that can be surprising in how it names things. This is mostly > caused by > > the fact that several modules were developed before the introduction of > > PEP-8, and now we're stuck with the older naming within these modules. > > > > It has been said and discussed in the past [1][2] that the stdlib is in > fact > > inconsistent, but fixing this has almost always been disregarded as being > > too painful (after all, we don't want a new Python 3 all over again). > > However, this way, we will never move away from these inconsistencies. > > Perhaps this is fine, but I think we should at least consider providing > > function and class names that are unsurprising for developers. > > > > While maintaining full backwards compatibility, my idea is that we should > > offer consistently named aliases in -eventually- all stdlib modules. For > > instance, with Python 2.6, the threading module received this treatment, > but > > unfortunately this was not expanded to all modules. > > > > What am I speaking of precisely? I have done a quick survey of the stdlib > > and found the following examples. Please note, this is a highly > opinionated > > list; some names may have been chosen with a very good reason, and others > > are just a matter of taste. Hopefully you agree with at least some of > them: > > > > * The CamelCasing in some modules are the most obvious culprits, e.g. > > logging and unittest. There is obviously an issue regarding subclasses > and > > methods that are supposed to be overridden, but I feel we could make it > > work. > > > > * All lower case class names, such as collections.defaultdict and > > collections.deque, should be CamelCased. Another example is datetime, > which > > uses names such as timedelta instead of TimeDelta. > > > > * Inconsistent names all together, such as re.sub, which I feel should > be > > re.replace (cf. str.replace). But also re.finditer and re.findall, but no > > re.find. > > > > * Names that do not reflect actual usage, such as ssl.PROTOCOL_SSLv23, > > which can in fact not be used as client for SSLv2. > > > > * Underscore usage, such as tarfile.TarFile.gettarinfo (should it not > be > > get_tar_info?), http.client.HTTPConnection.getresponse vs set_debuglevel, > > and pathlib.Path.samefile vs pathlib.Path.read_text. And is it > > pkgutil.iter_modules or is it pathlib.Path.iterdir (or re.finditer)? > > > > * Usage of various abbreviations, such as in filecmp.cmp > > > > * Inconsistencies between similar modules, e.g. between > > tarfile.TarFile.add and zipfile.ZipFile.write. > > > > These are just some examples of inconsistent and surprising naming I > could > > find, other categories are probably also conceivable. Another subject for > > reconsideration would be attribute and argument names, but I haven't > looked > > for those in my quick survey. > > > > For all of these inconsistencies, I think we should make a 'consistently' > > named alternative, and alias the original variant with them (or the other > > way around), without defining a deprecation timeline for the original > names. > > This should make it possible to eventually make the stdlib consistent, > > Pythonic and unsurprising. > > > > What would you think of such an effort? > > > > Regards, > > Ralph Broenink > > > > [1] > https://mail.python.org/pipermail/python-ideas/2010-January/006755.html > > [2] https://mail.python.org/pipermail/python-dev/2009-March/086646.html > > > > > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas at python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From xdegaye at gmail.com Thu Jul 28 12:30:59 2016 From: xdegaye at gmail.com (Xavier de Gaye) Date: Thu, 28 Jul 2016 18:30:59 +0200 Subject: [Python-ideas] size of the installation of Python on mobile devices Message-ID: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> Issue 26852 [1] proposes an enhancement to reduce the size of the Python installation by using a sourceless distribution. It seems that discussion on this problem (the size of the Python installation on mobile devices) should be moved here. IMHO there is no need to debate on the fact that a full install of Python on a mobile device today is a problem. As a reference, here is a raw estimation of the sizes of the different installs: full install: 111M without the test suite: 53M without the test suite and as a sourceless distribution: 23M same as above but also without ensurepip: 14M It would be useful to find the references to the previous discussions related to sourceless distributions, especially the reasons why their use is discouraged. Xavier [1] http://bugs.python.org/issue26852 From victor.stinner at gmail.com Thu Jul 28 13:00:12 2016 From: victor.stinner at gmail.com (Victor Stinner) Date: Thu, 28 Jul 2016 19:00:12 +0200 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> Message-ID: 2016-07-28 18:30 GMT+02:00 Xavier de Gaye : > Issue 26852 [1] proposes an enhancement to reduce the size of the Python > installation by using a sourceless distribution. It seems that > discussion on this problem (the size of the Python installation on > mobile devices) should be moved here. > > IMHO there is no need to debate on the fact that a full install of > Python on a mobile device today is a problem. FYI size does also matter on server: it became an important for cloud and container images. See for example the "System Python" idea of Fedora which comes with a truncated standard library: https://fedoraproject.org/wiki/Changes/System_Python Extract: """ - system-python-libs - a subset of standard Python library considered essential to run "system tools" requiring Python - system-python - /usr/libexec/system-python binary (interpreter) for the tools to be able to run, provides system-python(abi) - python3-libs brings in the remaining subset of standard Python library (...) (...) """ > It would be useful to find the references to the previous discussions > related to sourceless distributions, especially the reasons why their > use is discouraged. I worked on embeded devices, and truncating the stanard library is a common and *mandatory* practice. Victor From mal at egenix.com Thu Jul 28 14:07:52 2016 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 28 Jul 2016 20:07:52 +0200 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> Message-ID: <579A49F8.9080708@egenix.com> On 28.07.2016 19:00, Victor Stinner wrote: > 2016-07-28 18:30 GMT+02:00 Xavier de Gaye : >> Issue 26852 [1] proposes an enhancement to reduce the size of the Python >> installation by using a sourceless distribution. It seems that >> discussion on this problem (the size of the Python installation on >> mobile devices) should be moved here. >> >> IMHO there is no need to debate on the fact that a full install of >> Python on a mobile device today is a problem. You may want to have a look at what we did for eGenix PyRun: http://www.egenix.com/products/python/PyRun/ for how to create a minified version of the Python stdlib. > FYI size does also matter on server: it became an important for cloud > and container images. See for example the "System Python" idea of > Fedora which comes with a truncated standard library: > https://fedoraproject.org/wiki/Changes/System_Python > > Extract: > """ > - system-python-libs - a subset of standard Python library considered > essential to run "system tools" requiring Python > - system-python - /usr/libexec/system-python binary (interpreter) for > the tools to be able to run, provides system-python(abi) > - python3-libs brings in the remaining subset of standard Python library (...) > (...) > """ > > >> It would be useful to find the references to the previous discussions >> related to sourceless distributions, especially the reasons why their >> use is discouraged. Sourceless distributions are not discouraged. They are essential for commercial use cases and I would strongly oppose removing the possibility to ship byte code only applications. Why do you think that this doesn't work anymore ? > I worked on embeded devices, and truncating the stanard library is a > common and *mandatory* practice. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Jul 28 2016) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ From kevin-lists at theolliviers.com Thu Jul 28 14:27:55 2016 From: kevin-lists at theolliviers.com (Kevin Ollivier) Date: Thu, 28 Jul 2016 11:27:55 -0700 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> Message-ID: <1E6C2DF9-388F-484A-990D-3CB98ECF1B2A@theolliviers.com> On 7/28/16, 9:30 AM, "Python-ideas on behalf of Xavier de Gaye" wrote: >Issue 26852 [1] proposes an enhancement to reduce the size of the Python >installation by using a sourceless distribution. It seems that >discussion on this problem (the size of the Python installation on >mobile devices) should be moved here. Is the reason you're doing this as a configure flag rather than some tool that can run on a full Python distro because this is a cross-compile target? I ask because as Victor points out, this concern is really not unique to Android or mobile development. There are numerous use cases for wanting a sourceless distribution on just about any mobile, desktop or server platform. The abundance of tools out there that try to solve this problem is a testament to how common the need is. If Python could spit out a sourceless version of itself, complete with site-packages, that alone would go a long way towards consolidating the packaging logic into one place, and would likely solve the mobile and embedded problem as well. >IMHO there is no need to debate on the fact that a full install of >Python on a mobile device today is a problem. > >As a reference, here is a raw estimation of the sizes of the different >installs: > > full install: 111M > without the test suite: 53M > without the test suite and as a sourceless distribution: 23M > same as above but also without ensurepip: 14M > >It would be useful to find the references to the previous discussions >related to sourceless distributions, especially the reasons why their >use is discouraged. I suspect the concerns against it are the complexity of implementation, but after having done this for many years, I rather want something that is less 'smart' than the existing packaging tools I've used try to be. What I'd love personally is the ability to simply take a full Python install - system distro, venv or cross-compile target - and just package the whole thing up minus sources. I can be careful to make sure I have only what I need in it before building the release. At most I'd like to be able to specify a whitelist / blacklist file to include / exclude specific things. Thanks, Kevin >Xavier > >[1] http://bugs.python.org/issue26852 >_______________________________________________ >Python-ideas mailing list >Python-ideas at python.org >https://mail.python.org/mailman/listinfo/python-ideas >Code of Conduct: http://python.org/psf/codeofconduct/ From vgr255 at live.ca Thu Jul 28 18:01:08 2016 From: vgr255 at live.ca (Emanuel Barry) Date: Thu, 28 Jul 2016 22:01:08 +0000 Subject: [Python-ideas] Make types.MappingProxyType hashable Message-ID: Hello Python-ideas, While answering a question earlier today, I realized that types.MappingProxyType is immutable and unhashable. It seems to me that if all values are hashable (all keys are already guaranteed to be hashable), then the mappingproxy itself should be hashable. Should I go ahead and make a patch for this, or is this a bad idea? Cheers, -Emanuel From victor.stinner at gmail.com Thu Jul 28 18:11:57 2016 From: victor.stinner at gmail.com (Victor Stinner) Date: Fri, 29 Jul 2016 00:11:57 +0200 Subject: [Python-ideas] Make types.MappingProxyType hashable In-Reply-To: References: Message-ID: Good idea. Victor Le 29 juil. 2016 12:01 AM, "Emanuel Barry" a ?crit : > Hello Python-ideas, > While answering a question earlier today, I realized that > types.MappingProxyType is immutable and unhashable. It seems to me that if > all values are hashable (all keys are already guaranteed to be hashable), > then the mappingproxy itself should be hashable. Should I go ahead and make > a patch for this, or is this a bad idea? > > Cheers, > -Emanuel > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Thu Jul 28 18:06:07 2016 From: guido at python.org (Guido van Rossum) Date: Thu, 28 Jul 2016 15:06:07 -0700 Subject: [Python-ideas] Make types.MappingProxyType hashable In-Reply-To: References: Message-ID: I think you misunderstand the distinction between immutable and read-only. MappingProxyType is read-only -- you cannot modify the underlying dict through the proxy. But it's not immutable -- if the underlying dict is changed directly (e.g. by assignment to a class attribute) the proxy is changed too. So making the proxy hashable would be a mistake. What real-world use case do you have for this? On Thu, Jul 28, 2016 at 3:01 PM, Emanuel Barry wrote: > Hello Python-ideas, > While answering a question earlier today, I realized that > types.MappingProxyType is immutable and unhashable. It seems to me that if > all values are hashable (all keys are already guaranteed to be hashable), > then the mappingproxy itself should be hashable. Should I go ahead and make > a patch for this, or is this a bad idea? > > Cheers, > -Emanuel > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -- --Guido van Rossum (python.org/~guido) From vgr255 at live.ca Thu Jul 28 18:23:35 2016 From: vgr255 at live.ca (Emanuel Barry) Date: Thu, 28 Jul 2016 22:23:35 +0000 Subject: [Python-ideas] Make types.MappingProxyType hashable In-Reply-To: References: Message-ID: > From: Guido van Rossum > Sent: Thursday, July 28, 2016 6:06 PM > To: Emanuel Barry > Cc: python-ideas at python.org > Subject: Re: [Python-ideas] Make types.MappingProxyType hashable > > I think you misunderstand the distinction between immutable and > read-only. MappingProxyType is read-only -- you cannot modify the > underlying dict through the proxy. But it's not immutable -- if the > underlying dict is changed directly (e.g. by assignment to a class > attribute) the proxy is changed too. So making the proxy hashable > would be a mistake. Oh, I didn't know that this was just a view over an actual dict. I guess that solves the question :) > > What real-world use case do you have for this? Not me, actually. Someone in #python asked about an immutable dict, and I pointed them towards mappingproxy, only to realize it wasn't hashable. Maybe something like frozendict could be added, though? At this point I don't really have an opinion, so it's up to the list to decide if they want it or not :) -Emanuel From tjreedy at udel.edu Fri Jul 29 03:07:45 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 29 Jul 2016 03:07:45 -0400 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> Message-ID: On 7/28/2016 12:30 PM, Xavier de Gaye wrote: > Issue 26852 [1] proposes an enhancement to reduce the size of the Python > installation by using a sourceless distribution. It seems that > discussion on this problem (the size of the Python installation on > mobile devices) should be moved here. > > IMHO there is no need to debate on the fact that a full install of > Python on a mobile device today is a problem. > > As a reference, here is a raw estimation of the sizes of the different > installs: > > full install: 111M > without the test suite: 53M > without the test suite and as a sourceless distribution: 23M > same as above but also without ensurepip: 14M I believe unicodedata is at least a couple more MB, and, I believe, dispensible for more purposes. > It would be useful to find the references to the previous discussions > related to sourceless distributions, especially the reasons why their > use is discouraged. > > Xavier > > [1] http://bugs.python.org/issue26852 > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Terry Jan Reedy From xdegaye at gmail.com Fri Jul 29 05:51:26 2016 From: xdegaye at gmail.com (Xavier de Gaye) Date: Fri, 29 Jul 2016 11:51:26 +0200 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: <579A49F8.9080708@egenix.com> References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> <579A49F8.9080708@egenix.com> Message-ID: <7bb89d4a-3930-d2fd-9736-2360008f561e@gmail.com> On 07/28/2016 08:07 PM, M.-A. Lemburg wrote: > Sourceless distributions are not discouraged. They are essential > for commercial use cases and I would strongly oppose removing > the possibility to ship byte code only applications. > > Why do you think that this doesn't work anymore ? This refers to this message [1] from the discussion in issue 26852. I would not think that sourceless distributions are discouraged if I am submitting a patch that adds an option to build a sourceless distribution, there are better ways to waste time :) The drawbacks of sourceless distributions: * Quote from PEP 3147 "In practice, it is well known that pyc files are not compatible across Python major releases. A reading of import.c [3] in the Python source code proves that within recent memory, every new CPython major release has bumped the pyc magic number.". * The test suite cannot be run on a sourceless distribution as many tests use the linecache module. It would be useful to know the other reasons why a sourceless distribution is discouraged, so we might fix them. And if they cannot be fixed then issue 26852 can be closed. [1] http://bugs.python.org/issue26852#msg271535 From ncoghlan at gmail.com Fri Jul 29 07:04:02 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 29 Jul 2016 21:04:02 +1000 Subject: [Python-ideas] Make types.MappingProxyType hashable In-Reply-To: References: Message-ID: On 29 July 2016 at 08:23, Emanuel Barry wrote: > Not me, actually. Someone in #python asked about an immutable dict, and I pointed them towards mappingproxy, only to realize it wasn't hashable. Maybe something like frozendict could be added, though? At this point I don't really have an opinion, so it's up to the list to decide if they want it or not :) For most cases where this kind of capability is needed, it will be sufficient to model a frozen dict as a frozen set of 2-tuples: >>> data = {k: str(k) for k in range(5)} >>> data {0: '0', 1: '1', 2: '2', 3: '3', 4: '4'} >>> frozenset(data.items()) frozenset({(4, '4'), (3, '3'), (1, '1'), (0, '0'), (2, '2')}) >>> dict(frozenset(data.items())) {0: '0', 1: '1', 2: '2', 3: '3', 4: '4'} The fact the frozen set is keyed by the key-value pairs rather than just the keys doesn't matter, as you can't add any new entries at all, and the same goes for any other discrepancies between the invariants of the two data structures: even though the frozen set doesn't enforce any dict invariants, their preservation is ensured by the stricter immutability and hashability invariants. And in cases where you want to expose a read-only view to a normal dictionary, then you have types.MappingProxyType. So it's not that a frozendict type couldn't be provided, it's just of highly questionable utility given other existing features. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Fri Jul 29 07:13:28 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 29 Jul 2016 21:13:28 +1000 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: <7bb89d4a-3930-d2fd-9736-2360008f561e@gmail.com> References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> <579A49F8.9080708@egenix.com> <7bb89d4a-3930-d2fd-9736-2360008f561e@gmail.com> Message-ID: On 29 July 2016 at 19:51, Xavier de Gaye wrote: > It would be useful to know the other reasons why a sourceless > distribution is discouraged, so we might fix them. And if they cannot be > fixed then issue 26852 can be closed. "It currently makes debugging a pain" is the reason why we (Fedora & Red Hat) chose not to go for a sourceless distribution of the standard library as a way of reducing the default image size for the base Fedora Cloud and Atomic images. However, that's technically a solvable problem - given a traceback without source contents, we (python-dev) could provide a way to regenerate it against a full distribution with source code (similar to the way folks are able to map stack dumps in other compiled languages back to the relevant pieces of source code). It's also only an argument for folks to take into account when they consider "sourceless or not?" for their particular use case, rather than an argument against making sourceless distribution for size optimisation purposes easier in general. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From mal at egenix.com Fri Jul 29 07:44:13 2016 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 29 Jul 2016 13:44:13 +0200 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: <7bb89d4a-3930-d2fd-9736-2360008f561e@gmail.com> References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> <579A49F8.9080708@egenix.com> <7bb89d4a-3930-d2fd-9736-2360008f561e@gmail.com> Message-ID: <579B418D.6060004@egenix.com> On 29.07.2016 11:51, Xavier de Gaye wrote: > On 07/28/2016 08:07 PM, M.-A. Lemburg wrote: >> Sourceless distributions are not discouraged. They are essential >> for commercial use cases and I would strongly oppose removing >> the possibility to ship byte code only applications. >> >> Why do you think that this doesn't work anymore ? > > This refers to this message [1] from the discussion in issue 26852. > I would not think that sourceless distributions are discouraged if I > am submitting a patch that adds an option to build a sourceless > distribution, there are better ways to waste time :) As I read it, David was referring to sourceless distribution of the Python stdlib itself. That's a different use case than the one I meant, which focuses on shipping application code without source code. > The drawbacks of sourceless distributions: > * Quote from PEP 3147 "In practice, it is well known that pyc files > are not compatible across Python major releases. A reading of > import.c [3] in the Python source code proves that within recent > memory, every new CPython major release has bumped the pyc magic > number.". This argument is only valid if you are distributing application code without also shipping a Python interpreter to go with it. > * The test suite cannot be run on a sourceless distribution as many > tests use the linecache module. That's true, but in production, you are typically not interested in the test suite anyway. The same is true for the ensurepip package, since this comes with wheel files living inside the package directory structure. BTW: For distutils compilation, you also need the include files, but again, depending on your use case, you may not be interested in supporting this. > It would be useful to know the other reasons why a sourceless > distribution is discouraged, so we might fix them. And if they cannot be > fixed then issue 26852 can be closed. > > [1] http://bugs.python.org/issue26852#msg271535 There are certainly a lot of use cases for wanting to have a Python distribution which is small, but in most cases, you need to do more than just remove the .py files to address these, so I'm not sure whether we need an option in configure for this. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Jul 29 2016) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ From vgr255 at live.ca Fri Jul 29 08:02:32 2016 From: vgr255 at live.ca (Emanuel Barry) Date: Fri, 29 Jul 2016 12:02:32 +0000 Subject: [Python-ideas] Make types.MappingProxyType hashable In-Reply-To: References: Message-ID: All right, this makes sense. Thanks for the reply! -Emanuel > From: Nick Coghlan [mailto:ncoghlan at gmail.com] > Subject: Re: [Python-ideas] Make types.MappingProxyType hashable > > For most cases where this kind of capability is needed, it will be > sufficient to model a frozen dict as a frozen set of 2-tuples: > > >>> data = {k: str(k) for k in range(5)} > >>> data > {0: '0', 1: '1', 2: '2', 3: '3', 4: '4'} > >>> frozenset(data.items()) > frozenset({(4, '4'), (3, '3'), (1, '1'), (0, '0'), (2, '2')}) > >>> dict(frozenset(data.items())) > {0: '0', 1: '1', 2: '2', 3: '3', 4: '4'} > > The fact the frozen set is keyed by the key-value pairs rather than > just the keys doesn't matter, as you can't add any new entries at all, > and the same goes for any other discrepancies between the invariants > of the two data structures: even though the frozen set doesn't enforce > any dict invariants, their preservation is ensured by the stricter > immutability and hashability invariants. > > And in cases where you want to expose a read-only view to a normal > dictionary, then you have types.MappingProxyType. > > So it's not that a frozendict type couldn't be provided, it's just of > highly questionable utility given other existing features. > > Cheers, > Nick. From xdegaye at gmail.com Fri Jul 29 08:37:40 2016 From: xdegaye at gmail.com (Xavier de Gaye) Date: Fri, 29 Jul 2016 14:37:40 +0200 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: <579B418D.6060004@egenix.com> References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> <579A49F8.9080708@egenix.com> <7bb89d4a-3930-d2fd-9736-2360008f561e@gmail.com> <579B418D.6060004@egenix.com> Message-ID: On 07/29/2016 01:44 PM, M.-A. Lemburg wrote: > There are certainly a lot of use cases for wanting to have > a Python distribution which is small, but in most cases, > you need to do more than just remove the .py files to address > these, so I'm not sure whether we need an option in configure > for this. No, a sourceless distribution does not "just remove the .py files", it also does not install the __pycache__ files. The gain in size is quite significant, more than 50%. Right, there are more steps to a small Python than a configure option to build a sourceless distribution, but obviously this first step is the one that would provide the biggest gain in size. Xavier From storchaka at gmail.com Fri Jul 29 08:50:40 2016 From: storchaka at gmail.com (Serhiy Storchaka) Date: Fri, 29 Jul 2016 15:50:40 +0300 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> Message-ID: On 28.07.16 19:30, Xavier de Gaye wrote: > Issue 26852 [1] proposes an enhancement to reduce the size of the Python > installation by using a sourceless distribution. It seems that > discussion on this problem (the size of the Python installation on > mobile devices) should be moved here. > > IMHO there is no need to debate on the fact that a full install of > Python on a mobile device today is a problem. > > As a reference, here is a raw estimation of the sizes of the different > installs: > > full install: 111M > without the test suite: 53M > without the test suite and as a sourceless distribution: 23M > same as above but also without ensurepip: 14M > > It would be useful to find the references to the previous discussions > related to sourceless distributions, especially the reasons why their > use is discouraged. Try to pack the stdlib in the ZIP file and use zipimport. From victor.stinner at gmail.com Fri Jul 29 08:57:14 2016 From: victor.stinner at gmail.com (Victor Stinner) Date: Fri, 29 Jul 2016 14:57:14 +0200 Subject: [Python-ideas] Make types.MappingProxyType hashable In-Reply-To: References: Message-ID: 2016-07-29 0:06 GMT+02:00 Guido van Rossum : > I think you misunderstand the distinction between immutable and > read-only. MappingProxyType is read-only -- you cannot modify the > underlying dict through the proxy. But it's not immutable -- if the > underlying dict is changed directly (e.g. by assignment to a class > attribute) the proxy is changed too. Oh right. That's why I'm not a big fan of MappingProxyType :-) -- It reminds me that I wrote a frozendict PEP which was rejected :-) https://www.python.org/dev/peps/pep-0416/ Victor From victor.stinner at gmail.com Fri Jul 29 09:04:02 2016 From: victor.stinner at gmail.com (Victor Stinner) Date: Fri, 29 Jul 2016 15:04:02 +0200 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> <579A49F8.9080708@egenix.com> <7bb89d4a-3930-d2fd-9736-2360008f561e@gmail.com> Message-ID: 2016-07-29 13:13 GMT+02:00 Nick Coghlan : > "It currently makes debugging a pain" is the reason why we (Fedora & > Red Hat) chose not to go for a sourceless distribution of the standard > library as a way of reducing the default image size for the base > Fedora Cloud and Atomic images. In the embeded world, you have hard limit. For example, the firmware (boot load, kernel, OS, application: everything) must be exceed 128 MB. It takes time to reduce the firmware size by 1 MB. So such obvious solution of not embeding .py files but only .pyc is not really an option, but just mandatory. On the production, usually you just cannot run a debugger just for practical reasons. For example, you cannot open a TCP connection to the device. Only the device can open connections to a known server. Traceback are not logged neither to not waste limited storage. So .py files are just useless. Victor From stefan_ml at behnel.de Fri Jul 29 10:48:05 2016 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 29 Jul 2016 16:48:05 +0200 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> Message-ID: Serhiy Storchaka schrieb am 29.07.2016 um 14:50: > On 28.07.16 19:30, Xavier de Gaye wrote: >> Issue 26852 [1] proposes an enhancement to reduce the size of the Python >> installation by using a sourceless distribution. It seems that >> discussion on this problem (the size of the Python installation on >> mobile devices) should be moved here. >> >> IMHO there is no need to debate on the fact that a full install of >> Python on a mobile device today is a problem. >> >> As a reference, here is a raw estimation of the sizes of the different >> installs: >> >> full install: 111M >> without the test suite: 53M >> without the test suite and as a sourceless distribution: 23M >> same as above but also without ensurepip: 14M >> >> It would be useful to find the references to the previous discussions >> related to sourceless distributions, especially the reasons why their >> use is discouraged. > > Try to pack the stdlib in the ZIP file and use zipimport. Speaking of compression, I just saw that the ticket about compressing .pyc files is still open, so I guess that nothing has been done in that regard. It still surprises me that they aren't. They are classical write-once, read-many kind of files, so not compressing them seems counter-intuitive. https://bugs.python.org/issue22789 Obviously, compressing everything into an archive instead of compressing each file separately is bound to provide even larger gains, but that doesn't deal with the fact that some things usually get compiled and installed later than others. Stefan From elazarg at gmail.com Fri Jul 29 10:50:25 2016 From: elazarg at gmail.com (=?UTF-8?B?15DXnNei15bXqA==?=) Date: Fri, 29 Jul 2016 14:50:25 +0000 Subject: [Python-ideas] Make types.MappingProxyType hashable In-Reply-To: References: Message-ID: > Date: Thu, 28 Jul 2016 22:23:35 +0000 > From: Emanuel Barry > To: "guido at python.org" > Cc: "python-ideas at python.org" > Subject: Re: [Python-ideas] Make types.MappingProxyType hashable >> What real-world use case do you have for this? > Not me, actually. Someone in #python asked about an immutable dict, > and I pointed them towards mappingproxy, only to realize it wasn't hashable. Maybe something like frozendict could be added, though? At this point I don't really have an opinion, so it's up to the list to decide if they want it or not :) If the keys can be converted to/from valid identifiers, and the values are hashable, then namedtuple can give you your frozendict. - Elazar -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Fri Jul 29 15:18:11 2016 From: chris.barker at noaa.gov (Chris Barker - NOAA Federal) Date: Fri, 29 Jul 2016 12:18:11 -0700 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: References: <57916007.5030504@canterbury.ac.nz> Message-ID: <-8182430984798288380@unknownmsgid> > For cases where str instances are handled differently from other > iterables, but still within the same function, a convention of writing > "Union[str, Iterable]" as the input parameter type may suffice - The problem here is that often folks want to NOT allow single strings, but rather want only an iterable of strings. But, of course, a single string is an iterable of strings... I'm pretty sure this is by far the most common dynamic typing gotcha in Python ( at least after true division). Which means it may well be worth special casing all over the place to make it easy to do right. Personally, I'd like a solution that helps folks write correct code even without a type checker. AFAIK, the "solution" at this point is for every function that takes an iterable of strings to have code like: in = [in] if type(in) is str else in But this is ugly and fragile in so many ways... Perhaps there should be (maybe only in my private lib) a "as_sequence_of_strings()" function that does this, but more completely and robustly. Similar to numpy's asarray(), which passes back the object of it's already an array of the type specified, and tries to convert it otherwise. > typically want to consider types like str, bytes, > bytearray and memoryview as atomic objects, rather than as containers, There are all kinds of things that one might want to treat as atomic, rather than a sequence -- but what is special about str is that it is infinitely recurs able---no matter how many times you index it, you always get back a sequence -- never an atomic object. Most (every?) other sequence returns an atomic object (eventually) when you index it. So maybe the solution really is to have a character type -- then a string is a sequence of characters, rather that a sequence of string, and all these problems go away. If a character object has all the methods of a string except indexing (and iterating) it might even be mostly backward compatible.... Otherwise, special casing in multiple cases may be the only option. Or maybe all we need is a way to spell "any iterable of strings except a string", though that would only help type checking. But either way, it would be great to support this better. -CHB From michael.selik at gmail.com Fri Jul 29 22:59:09 2016 From: michael.selik at gmail.com (Michael Selik) Date: Sat, 30 Jul 2016 02:59:09 +0000 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: <-8182430984798288380@unknownmsgid> References: <57916007.5030504@canterbury.ac.nz> <-8182430984798288380@unknownmsgid> Message-ID: On Fri, Jul 29, 2016, 2:18 PM Chris Barker - NOAA Federal < chris.barker at noaa.gov> wrote: > The problem here is that often folks want to NOT allow single strings, > but rather want only an iterable of strings. > The module author might, but sometimes the user might want to use a str as an iterable of 1-len strs. It bugs me that Pandas forces me to wrap a str of indices in a list constructor before passing it into the DataFrame constructor. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.selik at gmail.com Fri Jul 29 22:59:18 2016 From: michael.selik at gmail.com (Michael Selik) Date: Sat, 30 Jul 2016 02:59:18 +0000 Subject: [Python-ideas] Addition to operator module: starcaller In-Reply-To: <57915EA4.6080602@canterbury.ac.nz> References: <57915EA4.6080602@canterbury.ac.nz> Message-ID: On Fri, Jul 22, 2016 at 12:52 AM Greg Ewing wrote: > Daniel Spitz wrote: > > I'm > > not sure and don't have a strong opinion about the original removal of > > apply. > > I expect it was removed simply because the * and ** calling > syntax makes it unecessary (before that was introduced, > apply was the only way of getting that funcionality). > > If it's to be reintroduced, the operator module would seem > to be the right place for it. > If it goes in the operator module, then the name ``operator.starcall`` seems better than ``starcaller``. I'd say ``operator.unpack`` would be better, except for the confusion between unpacking assignment and unpacking as arguments. I suppose ``operator.apply`` would be OK, too. Is there a better vocabulary word for unpack-as-args? -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at 2sn.net Fri Jul 29 23:57:21 2016 From: python at 2sn.net (Alexander Heger) Date: Sat, 30 Jul 2016 13:57:21 +1000 Subject: [Python-ideas] An ABC representing "Iterable, Sized, Container" In-Reply-To: <-8182430984798288380@unknownmsgid> References: <57916007.5030504@canterbury.ac.nz> <-8182430984798288380@unknownmsgid> Message-ID: > > So maybe the solution really is to have a character type -- then a > string is a sequence of characters, rather that a sequence of string, > and all these problems go away. > > If a character object has all the methods of a string except indexing > (and iterating) it might even be mostly backward compatible.... > > Otherwise, special casing in multiple cases may be the only option. > > Or maybe all we need is a way to spell "any iterable of strings except > a string", though that would only help type checking. > I agree, the fact that a string consists of strings has bothered me as well. It makes some things work well but some recursive algorithms requires special checks. There should be a way to iterate over a string as characters. AtomicIterable seems a good idea, but maybe it could be just the conceptual starting point of building up a class hierarchy of "iterability", and such a class could be passed to the 'flatten' method proposed above in the thread by Nick Coghlan. Some conceptual inspiration may be taken from numpy where one can have 0-dimensional arrays which are different from scalars - and different from n-D array with 1 element (n > 0) ``` In [7]: x = np.ndarray(()) In [8]: x Out[8]: array(21.0) In [9]: type(x) Out[9]: numpy.ndarray In [10]: x[()] Out[10]: 21.0 In [11]: np.asscalar(x) Out[11]: 21.0 In [13]: x = np.ndarray((1,)) In [14]: x Out[14]: array([ 17.5]) ``` -------------- next part -------------- An HTML attachment was scrubbed... URL: From brenbarn at brenbarn.net Sat Jul 30 01:57:14 2016 From: brenbarn at brenbarn.net (Brendan Barnwell) Date: Fri, 29 Jul 2016 22:57:14 -0700 Subject: [Python-ideas] Addition to operator module: starcaller In-Reply-To: References: <57915EA4.6080602@canterbury.ac.nz> Message-ID: <579C41BA.1040205@brenbarn.net> On 2016-07-29 19:59, Michael Selik wrote: > On Fri, Jul 22, 2016 at 12:52 AM Greg Ewing > wrote: > > Daniel Spitz wrote: > > I'm > > not sure and don't have a strong opinion about the original > removal of > > apply. > > I expect it was removed simply because the * and ** calling > syntax makes it unecessary (before that was introduced, > apply was the only way of getting that funcionality). > > If it's to be reintroduced, the operator module would seem > to be the right place for it. > > > If it goes in the operator module, then the name ``operator.starcall`` > seems better than ``starcaller``. I'd say ``operator.unpack`` would be > better, except for the confusion between unpacking assignment and > unpacking as arguments. I suppose ``operator.apply`` would be OK, too. > Is there a better vocabulary word for unpack-as-args? Why not just operator.call? I suppose actually operator.caller would be more consistent with the existing attrgetter and methodcaller? -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown From turnbull.stephen.fw at u.tsukuba.ac.jp Sat Jul 30 03:25:40 2016 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Sat, 30 Jul 2016 16:25:40 +0900 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: <7bb89d4a-3930-d2fd-9736-2360008f561e@gmail.com> References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> <579A49F8.9080708@egenix.com> <7bb89d4a-3930-d2fd-9736-2360008f561e@gmail.com> Message-ID: <22428.22132.611219.145348@turnbull.sk.tsukuba.ac.jp> Xavier de Gaye writes: > It would be useful to know the other reasons why a sourceless > distribution is discouraged, I wouldn't say it's discouraged, except for the ethical reason that Python is open source, and we love the idea that our users can hack Python for their own needs. That, of course, is balanced by the fact that sourceless distribution is exactly such a derivative. But I don't see why maintaining a sourceless distribution is an appropriate use of core developer time. On the one hand, it seems to be something that can be done separately from the core, and there are already many examples. On the other hand, there doesn't seem to be demand for a "generic" sourceless distribution -- removing source is the *easy* part. In fact there are many use cases that *need* variants, so it seems to me that it would be likely to be a real hairball of optional features. For example, as Terry mentioned not all applications need the unicodedata module, and it's *big*, but many applications I can imagine doing myself (spam-checking and other validation of textual data) would include it. Steve From ncoghlan at gmail.com Sat Jul 30 04:28:05 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 30 Jul 2016 18:28:05 +1000 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: <22428.22132.611219.145348@turnbull.sk.tsukuba.ac.jp> References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> <579A49F8.9080708@egenix.com> <7bb89d4a-3930-d2fd-9736-2360008f561e@gmail.com> <22428.22132.611219.145348@turnbull.sk.tsukuba.ac.jp> Message-ID: On 30 July 2016 at 17:25, Stephen J. Turnbull wrote: > Xavier de Gaye writes: > On the other hand, there doesn't seem to be demand for a "generic" > sourceless distribution -- removing source is the *easy* part. In > fact there are many use cases that *need* variants, so it seems to me > that it would be likely to be a real hairball of optional features. While you're not wrong about the complexity of the problem at hand, there are actually a few different problem domains pushing towards offering better mainline support for stripped down special purpose builds of the CPython runtime: - mobile devices - embedded devices - self-contained binaries - Linux container images (& even unikernels) These are all cases where "How do we make the core Python installation smaller?" is a reasonable question to ask (to either reduce storage requirements or to reduce network usage), and "use MicroPython instead of CPython" isn't always going to be the right answer (e.g. if you need CPython C API support). These are also all cases where the ability to automate builds and updates encourages pushing more complexity into the build process in pursuit of simplification and minimisation of the runtime components. If we say "pursue this problem outside the core", then odds are pretty good that those communities will all settle on different approaches, and we'll end up with four (or more) redundant sets of tooling and techniques for installation minimisation. By contrast, if we say "pursue improved solutions to this problem as part of CPython, but you'll need to keep these other potential use cases in mind", we increase the changes for "one obvious way" to be formulated, even if people have different motives for caring about the problem. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From xdegaye at gmail.com Sat Jul 30 08:18:56 2016 From: xdegaye at gmail.com (Xavier de Gaye) Date: Sat, 30 Jul 2016 14:18:56 +0200 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: <22428.22132.611219.145348@turnbull.sk.tsukuba.ac.jp> References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> <579A49F8.9080708@egenix.com> <7bb89d4a-3930-d2fd-9736-2360008f561e@gmail.com> <22428.22132.611219.145348@turnbull.sk.tsukuba.ac.jp> Message-ID: <890078b8-3c00-5560-f4e0-a30943bde568@gmail.com> On 07/30/2016 09:25 AM, Stephen J. Turnbull wrote: > For example, as Terry mentioned not all applications need the > unicodedata module, and it's *big*, but many applications I can > imagine doing myself (spam-checking and other validation of textual > data) would include it. How *big* exactly ? The size of the unicodedata shared library is 843K on archlinux when a sourceless distribution saves many tens of Mega bytes and a distribution without the test suite saves about 60M. IMHO only three options should be implemented, if the Python community really thinks seriously about porting Python to mobile devices: * --enable-sourceless-distribution * --disable-test-suite * --disable-distutils-builds The last option is to get rid of the config (LIBPL) directory that is used to build extension modules, as one cannot expect most mobile devices to have a proper build environment anyway. The result would be a distribution in the 10M order of magnitude, ten times smaller than the current size. Then we can look into compression, the impacts of compression on the size of the memory used by the python process and the impacts of compression on performance in mobile devices. The remaining savings (unicodedata, turtledemo, tkinter, unittest, ...) should be left to packagers. From mal at egenix.com Sat Jul 30 08:29:15 2016 From: mal at egenix.com (M.-A. Lemburg) Date: Sat, 30 Jul 2016 14:29:15 +0200 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: <890078b8-3c00-5560-f4e0-a30943bde568@gmail.com> References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> <579A49F8.9080708@egenix.com> <7bb89d4a-3930-d2fd-9736-2360008f561e@gmail.com> <22428.22132.611219.145348@turnbull.sk.tsukuba.ac.jp> <890078b8-3c00-5560-f4e0-a30943bde568@gmail.com> Message-ID: <579C9D9B.2070103@egenix.com> On 30.07.2016 14:18, Xavier de Gaye wrote: > On 07/30/2016 09:25 AM, Stephen J. Turnbull wrote: >> For example, as Terry mentioned not all applications need the >> unicodedata module, and it's *big*, but many applications I can >> imagine doing myself (spam-checking and other validation of textual >> data) would include it. > > How *big* exactly ? > The size of the unicodedata shared library is 843K on archlinux when a > sourceless distribution saves many tens of Mega bytes and a > distribution without the test suite saves about 60M. > > IMHO only three options should be implemented, if the Python community > really thinks seriously about porting Python to mobile devices: > * --enable-sourceless-distribution > * --disable-test-suite > * --disable-distutils-builds > The last option is to get rid of the config (LIBPL) directory that is > used to build extension modules, as one cannot expect most mobile > devices to have a proper build environment anyway. > > The result would be a distribution in the 10M order of magnitude, ten > times smaller than the current size. Then we can look into > compression, the impacts of compression on the size of the memory used > by the python process and the impacts of compression on performance in > mobile devices. > > The remaining savings (unicodedata, turtledemo, tkinter, unittest, > ...) should be left to packagers. IMO, this should all be left to packagers. They will have to patch the code base anyway to get things working in such an environment, so there isn't much point in adding some bits to the configure script which everyone uses, while leaving out other important parts. This doesn't simplify the life of packagers much, but does add maintenance overhead for the core developers (since these new options would have to be tested as well before release). BTW: By not having these options in configure, you also gain a lot more freedom as packager, since you can have the build behave you would like it to instead of going through the whole change request discussion process that updates/changes to these options would entail going forward. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Jul 30 2016) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ From ncoghlan at gmail.com Sat Jul 30 10:42:27 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 31 Jul 2016 00:42:27 +1000 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: <579C9D9B.2070103@egenix.com> References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> <579A49F8.9080708@egenix.com> <7bb89d4a-3930-d2fd-9736-2360008f561e@gmail.com> <22428.22132.611219.145348@turnbull.sk.tsukuba.ac.jp> <890078b8-3c00-5560-f4e0-a30943bde568@gmail.com> <579C9D9B.2070103@egenix.com> Message-ID: On 30 July 2016 at 22:29, M.-A. Lemburg wrote: > IMO, this should all be left to packagers. They will have to > patch the code base anyway to get things working in such > an environment, so there isn't much point in adding some > bits to the configure script which everyone uses, while > leaving out other important parts. This doesn't simplify > the life of packagers much, but does add maintenance > overhead for the core developers (since these new options > would have to be tested as well before release). > > BTW: By not having these options in configure, you also > gain a lot more freedom as packager, since you can have > the build behave you would like it to instead of going > through the whole change request discussion process that > updates/changes to these options would entail going > forward. Encouraging that divergence isn't free, though - it just pushes the testing burden outward by creating a larger compatibility testing matrix for folks wanting to support sourceless environments. And, to be clear, I'm not suggesting we start making official sourceless releases of CPython, nor am I suggesting we make "sourceless builds are working" a gating criterion for versioned releases - instead, I'm suggesting we allow interested folks like Xavier to add the basic knobs and dials to the standard distribution, with the awareness that they might break sometimes and requiring patching after other folks make changes. Since the assumption is that folks interested in this will be building from source anyway, it shouldn't be a problem to run directly off a maintenance branch when they need to, rather than using a particular release tag. Cheers, NIck. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From xdegaye at gmail.com Sat Jul 30 17:26:18 2016 From: xdegaye at gmail.com (Xavier de Gaye) Date: Sat, 30 Jul 2016 23:26:18 +0200 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> <579A49F8.9080708@egenix.com> <7bb89d4a-3930-d2fd-9736-2360008f561e@gmail.com> <22428.22132.611219.145348@turnbull.sk.tsukuba.ac.jp> <890078b8-3c00-5560-f4e0-a30943bde568@gmail.com> <579C9D9B.2070103@egenix.com> Message-ID: <45ed9d6d-7df7-b83d-1f0a-73496ebf3307@gmail.com> On 07/30/2016 04:42 PM, Nick Coghlan wrote: > On 30 July 2016 at 22:29, M.-A. Lemburg wrote: >> IMO, this should all be left to packagers. They will have to >> patch the code base anyway to get things working in such >> an environment, so there isn't much point in adding some >> bits to the configure script which everyone uses, while >> leaving out other important parts. This doesn't simplify >> the life of packagers much, but does add maintenance >> overhead for the core developers (since these new options >> would have to be tested as well before release). >> >> BTW: By not having these options in configure, you also >> gain a lot more freedom as packager, since you can have >> the build behave you would like it to instead of going >> through the whole change request discussion process that >> updates/changes to these options would entail going >> forward. > > Encouraging that divergence isn't free, though - it just pushes the > testing burden outward by creating a larger compatibility testing > matrix for folks wanting to support sourceless environments. > > And, to be clear, I'm not suggesting we start making official > sourceless releases of CPython, nor am I suggesting we make > "sourceless builds are working" a gating criterion for versioned > releases - instead, I'm suggesting we allow interested folks like > Xavier to add the basic knobs and dials to the standard distribution, > with the awareness that they might break sometimes and requiring > patching after other folks make changes. Since the assumption is that > folks interested in this will be building from source anyway, it > shouldn't be a problem to run directly off a maintenance branch when > they need to, rather than using a particular release tag. Then I propose that we vote to allow issue 26852 [1] to add the '--enable-sourceless-distribution' option to configure. I am +1 for the reasons given in my previous posts. Xavier [1] http://bugs.python.org/issue26852 From guido at python.org Sat Jul 30 18:22:31 2016 From: guido at python.org (Guido van Rossum) Date: Sat, 30 Jul 2016 15:22:31 -0700 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: <45ed9d6d-7df7-b83d-1f0a-73496ebf3307@gmail.com> References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> <579A49F8.9080708@egenix.com> <7bb89d4a-3930-d2fd-9736-2360008f561e@gmail.com> <22428.22132.611219.145348@turnbull.sk.tsukuba.ac.jp> <890078b8-3c00-5560-f4e0-a30943bde568@gmail.com> <579C9D9B.2070103@egenix.com> <45ed9d6d-7df7-b83d-1f0a-73496ebf3307@gmail.com> Message-ID: You misunderstand. There is no voting. On Sat, Jul 30, 2016 at 2:26 PM, Xavier de Gaye wrote: > On 07/30/2016 04:42 PM, Nick Coghlan wrote: >> On 30 July 2016 at 22:29, M.-A. Lemburg wrote: >>> IMO, this should all be left to packagers. They will have to >>> patch the code base anyway to get things working in such >>> an environment, so there isn't much point in adding some >>> bits to the configure script which everyone uses, while >>> leaving out other important parts. This doesn't simplify >>> the life of packagers much, but does add maintenance >>> overhead for the core developers (since these new options >>> would have to be tested as well before release). >>> >>> BTW: By not having these options in configure, you also >>> gain a lot more freedom as packager, since you can have >>> the build behave you would like it to instead of going >>> through the whole change request discussion process that >>> updates/changes to these options would entail going >>> forward. >> >> Encouraging that divergence isn't free, though - it just pushes the >> testing burden outward by creating a larger compatibility testing >> matrix for folks wanting to support sourceless environments. >> >> And, to be clear, I'm not suggesting we start making official >> sourceless releases of CPython, nor am I suggesting we make >> "sourceless builds are working" a gating criterion for versioned >> releases - instead, I'm suggesting we allow interested folks like >> Xavier to add the basic knobs and dials to the standard distribution, >> with the awareness that they might break sometimes and requiring >> patching after other folks make changes. Since the assumption is that >> folks interested in this will be building from source anyway, it >> shouldn't be a problem to run directly off a maintenance branch when >> they need to, rather than using a particular release tag. > > > Then I propose that we vote to allow issue 26852 [1] to add the > '--enable-sourceless-distribution' option to configure. > > I am +1 for the reasons given in my previous posts. > > Xavier > > [1] http://bugs.python.org/issue26852 > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -- --Guido van Rossum (python.org/~guido) From mistersheik at gmail.com Sat Jul 30 17:57:53 2016 From: mistersheik at gmail.com (Neil Girdhar) Date: Sat, 30 Jul 2016 14:57:53 -0700 (PDT) Subject: [Python-ideas] Consider adding clip or clamp function to math Message-ID: <8457d50f-2ea4-47dd-ab44-f3bf28fe8bb3@googlegroups.com> It's common to want to clip (or clamp) a number to a range. This feature is commonly needed for both floating point numbers and integers: http://stackoverflow.com/questions/9775731/clamping-floating-numbers-in-python http://stackoverflow.com/questions/4092528/how-to-clamp-an-integer-to-some-range-in-python There are a few approaches: * use a couple ternary operators (e.g. https://github.com/scipy/scipy/pull/5944/files line 98, which generated a lot of discussion) * use a min/max construction, * call sorted on a list of the three numbers and pick out the first, or * use numpy.clip. Am I right that there is no *obvious* way to do this? If so, I suggest adding math.clip (or math.clamp) to the standard library that has the meaning: def clip(number, lower, upper): return lower if number < lower else upper if number > upper else number This would work for non-numeric types so long as the non-numeric types support comparison. It might also be worth adding assert lower < upper to catch some bugs. Best, Neil -------------- next part -------------- An HTML attachment was scrubbed... URL: From mertz at gnosis.cx Sun Jul 31 00:41:07 2016 From: mertz at gnosis.cx (David Mertz) Date: Sat, 30 Jul 2016 21:41:07 -0700 Subject: [Python-ideas] Consider adding clip or clamp function to math In-Reply-To: <8457d50f-2ea4-47dd-ab44-f3bf28fe8bb3@googlegroups.com> References: <8457d50f-2ea4-47dd-ab44-f3bf28fe8bb3@googlegroups.com> Message-ID: Is there some special subtlety or edge case where a hand rolled function will go wrong? I like the SO version spelled like this (a little fleshed out): def clamp(val, min_val=None, max_val=None): min_val = val if min_val is None else min_val max_val = val if max_val is None else max_val assert min_val <= max_val return max(min(val , max_val), min_val) On Sat, Jul 30, 2016 at 2:57 PM, Neil Girdhar wrote: > It's common to want to clip (or clamp) a number to a range. This feature > is commonly needed for both floating point numbers and integers: > > > http://stackoverflow.com/questions/9775731/clamping-floating-numbers-in-python > > http://stackoverflow.com/questions/4092528/how-to-clamp-an-integer-to-some-range-in-python > > There are a few approaches: > > * use a couple ternary operators (e.g. > https://github.com/scipy/scipy/pull/5944/files line 98, which generated > a lot of discussion) > * use a min/max construction, > * call sorted on a list of the three numbers and pick out the first, or > * use numpy.clip. > > Am I right that there is no *obvious* way to do this? If so, I suggest > adding math.clip (or math.clamp) to the standard library that has the > meaning: > > def clip(number, lower, upper): > return lower if number < lower else upper if number > upper else number > > This would work for non-numeric types so long as the non-numeric types > support comparison. It might also be worth adding > > assert lower < upper > > to catch some bugs. > > Best, > > Neil > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sun Jul 31 04:19:39 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 31 Jul 2016 18:19:39 +1000 Subject: [Python-ideas] Consider adding clip or clamp function to math In-Reply-To: References: <8457d50f-2ea4-47dd-ab44-f3bf28fe8bb3@googlegroups.com> Message-ID: <20160731081939.GB13777@ando.pearwood.info> On Sat, Jul 30, 2016 at 09:41:07PM -0700, David Mertz wrote: > Is there some special subtlety or edge case where a hand rolled function > will go wrong? Depends on the person doing the hand rolling :-) I'm not sure how your version would work with NANs, and I haven't bothered to try it to find out, but I like this version: def clamp(value, lower=None, upper=None): """Clamp value to the closed interval lower...upper. The limits lower and upper can be set to None to mean -? and +? respectively. """ if not (lower is None or upper is None): if lower > upper: raise ValueError('lower must be <= upper') if lower is not None and value < lower: value = lower elif upper is not None and value > upper: value = upper return value which does support NANs for the value being clamped. (It returns the NAN unchanged.) It also avoids the expense of function calls to min and max, and will check that the lower and upper bounds are in the right order even if you run with assertions turned off. I don't think I've missed any cases. -- Steve From ian.g.kelly at gmail.com Sun Jul 31 09:10:59 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sun, 31 Jul 2016 07:10:59 -0600 Subject: [Python-ideas] Consider adding clip or clamp function to math In-Reply-To: <20160731081939.GB13777@ando.pearwood.info> References: <8457d50f-2ea4-47dd-ab44-f3bf28fe8bb3@googlegroups.com> <20160731081939.GB13777@ando.pearwood.info> Message-ID: On Sun, Jul 31, 2016 at 2:19 AM, Steven D'Aprano wrote: > I'm not sure how your version would work with NANs, and I haven't > bothered to try it to find out, but I like this version: Answer: py> min(10, float('nan')) 10 From ian.g.kelly at gmail.com Sun Jul 31 09:14:03 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sun, 31 Jul 2016 07:14:03 -0600 Subject: [Python-ideas] Consider adding clip or clamp function to math In-Reply-To: References: <8457d50f-2ea4-47dd-ab44-f3bf28fe8bb3@googlegroups.com> <20160731081939.GB13777@ando.pearwood.info> Message-ID: On Sun, Jul 31, 2016 at 7:10 AM, Ian Kelly wrote: > On Sun, Jul 31, 2016 at 2:19 AM, Steven D'Aprano wrote: >> I'm not sure how your version would work with NANs, and I haven't >> bothered to try it to find out, but I like this version: > > Answer: > > py> min(10, float('nan')) > 10 Ah, but the original passed the value before the boundary. py> min(float('nan'), 10) nan py> max(_, 0) nan So it works, but it's somewhat fragile because it depends on the order of the arguments to min and max. From mistersheik at gmail.com Sat Jul 30 18:02:18 2016 From: mistersheik at gmail.com (Neil Girdhar) Date: Sat, 30 Jul 2016 15:02:18 -0700 (PDT) Subject: [Python-ideas] Consider adding clip or clamp function to math In-Reply-To: <8457d50f-2ea4-47dd-ab44-f3bf28fe8bb3@googlegroups.com> References: <8457d50f-2ea4-47dd-ab44-f3bf28fe8bb3@googlegroups.com> Message-ID: <29bf0247-57cf-4849-88e8-7eb9f325c2d9@googlegroups.com> Rather: assert lower <= upper And apologies if this has been requested before. My search turned up nothing. On Saturday, July 30, 2016 at 5:57:53 PM UTC-4, Neil Girdhar wrote: > > It's common to want to clip (or clamp) a number to a range. This feature > is commonly needed for both floating point numbers and integers: > > > http://stackoverflow.com/questions/9775731/clamping-floating-numbers-in-python > > http://stackoverflow.com/questions/4092528/how-to-clamp-an-integer-to-some-range-in-python > > There are a few approaches: > > * use a couple ternary operators (e.g. > https://github.com/scipy/scipy/pull/5944/files line 98, which generated > a lot of discussion) > * use a min/max construction, > * call sorted on a list of the three numbers and pick out the first, or > * use numpy.clip. > > Am I right that there is no *obvious* way to do this? If so, I suggest > adding math.clip (or math.clamp) to the standard library that has the > meaning: > > def clip(number, lower, upper): > return lower if number < lower else upper if number > upper else number > > This would work for non-numeric types so long as the non-numeric types > support comparison. It might also be worth adding > > assert lower < upper > > to catch some bugs. > > Best, > > Neil > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralph at ralphbroenink.net Sun Jul 31 14:46:32 2016 From: ralph at ralphbroenink.net (Ralph Broenink) Date: Sun, 31 Jul 2016 18:46:32 +0000 Subject: [Python-ideas] Making the stdlib consistent again In-Reply-To: References: Message-ID: I'm a bit sad that I'm clearly on the loosing side of the argument. I now believe that I must be grossly underestimating the amount of effort and overestimating the potential gain. However, I still feel that we should strive for consistency in the long run. I do not propose to do this at once, but I feel that at least some collaborated effort would be nice. (If not only for this kind of mail threads.) If I would start an effort to - for instance - 'fix' some camelCased modules, and attempt to make it 100% backwards compatible, including tests, would there be any chance it could be merged at some point? Otherwise, I feel it would be a totally pointless effort ;). On Tue, 26 Jul 2016 at 18:29 Brett Cannon wrote: > On Mon, 25 Jul 2016 at 13:03 Mark Mollineaux > wrote: > >> I've pined for this (and feel a real mental pain every time I use one >> of those poorlyCased names)-- I end up using a lot of mental space >> remembering exactly HOW each stdlib isn't consistent. >> >> Aliasing consistent names in each case seems like a real win all >> around, personally. >> > > For those that want consistent names, you could create a PyPI package that > is nothing more than the aliased names as suggested. > > Otherwise I get the desire for consistency, but as pointed out by a bunch > of other core devs, we have thought about this many times and always reach > the same conclusion that the amount of work and potential code breakage is > too great. > > -Brett > > >> >> On Mon, Jul 25, 2016 at 10:55 AM, Ralph Broenink >> wrote: >> > Hi python-ideas, >> > >> > As you all know, the Python stdlib can sometimes be a bit of an >> inconsistent >> > mess that can be surprising in how it names things. This is mostly >> caused by >> > the fact that several modules were developed before the introduction of >> > PEP-8, and now we're stuck with the older naming within these modules. >> > >> > It has been said and discussed in the past [1][2] that the stdlib is in >> fact >> > inconsistent, but fixing this has almost always been disregarded as >> being >> > too painful (after all, we don't want a new Python 3 all over again). >> > However, this way, we will never move away from these inconsistencies. >> > Perhaps this is fine, but I think we should at least consider providing >> > function and class names that are unsurprising for developers. >> > >> > While maintaining full backwards compatibility, my idea is that we >> should >> > offer consistently named aliases in -eventually- all stdlib modules. For >> > instance, with Python 2.6, the threading module received this >> treatment, but >> > unfortunately this was not expanded to all modules. >> > >> > What am I speaking of precisely? I have done a quick survey of the >> stdlib >> > and found the following examples. Please note, this is a highly >> opinionated >> > list; some names may have been chosen with a very good reason, and >> others >> > are just a matter of taste. Hopefully you agree with at least some of >> them: >> > >> > * The CamelCasing in some modules are the most obvious culprits, e.g. >> > logging and unittest. There is obviously an issue regarding subclasses >> and >> > methods that are supposed to be overridden, but I feel we could make it >> > work. >> > >> > * All lower case class names, such as collections.defaultdict and >> > collections.deque, should be CamelCased. Another example is datetime, >> which >> > uses names such as timedelta instead of TimeDelta. >> > >> > * Inconsistent names all together, such as re.sub, which I feel >> should be >> > re.replace (cf. str.replace). But also re.finditer and re.findall, but >> no >> > re.find. >> > >> > * Names that do not reflect actual usage, such as ssl.PROTOCOL_SSLv23, >> > which can in fact not be used as client for SSLv2. >> > >> > * Underscore usage, such as tarfile.TarFile.gettarinfo (should it not >> be >> > get_tar_info?), http.client.HTTPConnection.getresponse vs >> set_debuglevel, >> > and pathlib.Path.samefile vs pathlib.Path.read_text. And is it >> > pkgutil.iter_modules or is it pathlib.Path.iterdir (or re.finditer)? >> > >> > * Usage of various abbreviations, such as in filecmp.cmp >> > >> > * Inconsistencies between similar modules, e.g. between >> > tarfile.TarFile.add and zipfile.ZipFile.write. >> > >> > These are just some examples of inconsistent and surprising naming I >> could >> > find, other categories are probably also conceivable. Another subject >> for >> > reconsideration would be attribute and argument names, but I haven't >> looked >> > for those in my quick survey. >> > >> > For all of these inconsistencies, I think we should make a >> 'consistently' >> > named alternative, and alias the original variant with them (or the >> other >> > way around), without defining a deprecation timeline for the original >> names. >> > This should make it possible to eventually make the stdlib consistent, >> > Pythonic and unsurprising. >> > >> > What would you think of such an effort? >> > >> > Regards, >> > Ralph Broenink >> > >> > [1] >> https://mail.python.org/pipermail/python-ideas/2010-January/006755.html >> > [2] >> https://mail.python.org/pipermail/python-dev/2009-March/086646.html >> > >> > >> > _______________________________________________ >> > Python-ideas mailing list >> > Python-ideas at python.org >> > https://mail.python.org/mailman/listinfo/python-ideas >> > Code of Conduct: http://python.org/psf/codeofconduct/ >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From victor.stinner at gmail.com Sun Jul 31 15:38:44 2016 From: victor.stinner at gmail.com (Victor Stinner) Date: Sun, 31 Jul 2016 21:38:44 +0200 Subject: [Python-ideas] size of the installation of Python on mobile devices In-Reply-To: <22428.22132.611219.145348@turnbull.sk.tsukuba.ac.jp> References: <8d577973-eaaf-21cf-3604-8ffd28773ba4@gmail.com> <579A49F8.9080708@egenix.com> <7bb89d4a-3930-d2fd-9736-2360008f561e@gmail.com> <22428.22132.611219.145348@turnbull.sk.tsukuba.ac.jp> Message-ID: Xavier is a core developer. He is free to dedicate his time to supporting sourceless distribution :-) Victor -------------- next part -------------- An HTML attachment was scrubbed... URL: From victor.stinner at gmail.com Sun Jul 31 15:38:44 2016 From: victor.stinner at gmail.com (Victor Stinner) Date: Sun, 31 Jul 2016 21:38:44 +0200 Subject: [Python-ideas] Consider adding clip or clamp function to math In-Reply-To: References: <8457d50f-2ea4-47dd-ab44-f3bf28fe8bb3@googlegroups.com> Message-ID: I dislike this API. What's the point of calling clamp(x)? clamp(b, a) is min(a, b) and clamp(a, max_val=b) is just max(a, b). My point is that all parameters must be mandatory. Victor Le 31 juil. 2016 6:41 AM, "David Mertz" a ?crit : > Is there some special subtlety or edge case where a hand rolled function > will go wrong? I like the SO version spelled like this (a little fleshed > out): > > def clamp(val, min_val=None, max_val=None): > min_val = val if min_val is None else min_val > max_val = val if max_val is None else max_val > assert min_val <= max_val > return max(min(val , max_val), min_val) > > > On Sat, Jul 30, 2016 at 2:57 PM, Neil Girdhar > wrote: > >> It's common to want to clip (or clamp) a number to a range. This feature >> is commonly needed for both floating point numbers and integers: >> >> >> http://stackoverflow.com/questions/9775731/clamping-floating-numbers-in-python >> >> http://stackoverflow.com/questions/4092528/how-to-clamp-an-integer-to-some-range-in-python >> >> There are a few approaches: >> >> * use a couple ternary operators (e.g. >> https://github.com/scipy/scipy/pull/5944/files line 98, which generated >> a lot of discussion) >> * use a min/max construction, >> * call sorted on a list of the three numbers and pick out the first, or >> * use numpy.clip. >> >> Am I right that there is no *obvious* way to do this? If so, I suggest >> adding math.clip (or math.clamp) to the standard library that has the >> meaning: >> >> def clip(number, lower, upper): >> return lower if number < lower else upper if number > upper else >> number >> >> This would work for non-numeric types so long as the non-numeric types >> support comparison. It might also be worth adding >> >> assert lower < upper >> >> to catch some bugs. >> >> Best, >> >> Neil >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > > -- > Keeping medicines from the bloodstreams of the sick; food > from the bellies of the hungry; books from the hands of the > uneducated; technology from the underdeveloped; and putting > advocates of freedom in prisons. Intellectual property is > to the 21st century what the slave trade was to the 16th. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From p1himik at gmail.com Sun Jul 31 15:47:54 2016 From: p1himik at gmail.com (Eugene Pakhomov) Date: Mon, 1 Aug 2016 02:47:54 +0700 Subject: [Python-ideas] Making the stdlib consistent again In-Reply-To: References: Message-ID: I'm on Ralph's side here. "Why is this thing named the other way?" was one of the first questions I asked. And people whom I occasionally teach about Python, ask the same question over and over again. Code breakage happens (PEP 3151 - didn't know about it till it almost bit my leg off), so we can't shy away from it completely. Is there any link on the previous thoughts of the core devs on the matter? Especially regarding the amount of potential code breakage. I'm genuinely interested, as I think that this amount is negligible if the new names will be gradually introduced along with a deprecation notice on (end eventual removal of) the old ones. As far as I can see, it can only do some harm if someone uses a discouraged "import *" or monkey-patches some new methods into Python standard classes/modules, and updates his Python installation. Regards, Eugene On Mon, Aug 1, 2016 at 1:46 AM, Ralph Broenink wrote: > I'm a bit sad that I'm clearly on the loosing side of the argument. I now > believe that I must be grossly underestimating the amount of effort and > overestimating the potential gain. However, I still feel that we should > strive for consistency in the long run. I do not propose to do this at > once, but I feel that at least some collaborated effort would be nice. (If > not only for this kind of mail threads.) > > If I would start an effort to - for instance - 'fix' some camelCased > modules, and attempt to make it 100% backwards compatible, including tests, > would there be any chance it could be merged at some point? Otherwise, I > feel it would be a totally pointless effort ;). > > On Tue, 26 Jul 2016 at 18:29 Brett Cannon wrote: > >> On Mon, 25 Jul 2016 at 13:03 Mark Mollineaux >> wrote: >> >>> I've pined for this (and feel a real mental pain every time I use one >>> of those poorlyCased names)-- I end up using a lot of mental space >>> remembering exactly HOW each stdlib isn't consistent. >>> >>> Aliasing consistent names in each case seems like a real win all >>> around, personally. >>> >> >> For those that want consistent names, you could create a PyPI package >> that is nothing more than the aliased names as suggested. >> >> Otherwise I get the desire for consistency, but as pointed out by a bunch >> of other core devs, we have thought about this many times and always reach >> the same conclusion that the amount of work and potential code breakage is >> too great. >> >> -Brett >> >> >>> >>> On Mon, Jul 25, 2016 at 10:55 AM, Ralph Broenink >>> wrote: >>> > Hi python-ideas, >>> > >>> > As you all know, the Python stdlib can sometimes be a bit of an >>> inconsistent >>> > mess that can be surprising in how it names things. This is mostly >>> caused by >>> > the fact that several modules were developed before the introduction of >>> > PEP-8, and now we're stuck with the older naming within these modules. >>> > >>> > It has been said and discussed in the past [1][2] that the stdlib is >>> in fact >>> > inconsistent, but fixing this has almost always been disregarded as >>> being >>> > too painful (after all, we don't want a new Python 3 all over again). >>> > However, this way, we will never move away from these inconsistencies. >>> > Perhaps this is fine, but I think we should at least consider providing >>> > function and class names that are unsurprising for developers. >>> > >>> > While maintaining full backwards compatibility, my idea is that we >>> should >>> > offer consistently named aliases in -eventually- all stdlib modules. >>> For >>> > instance, with Python 2.6, the threading module received this >>> treatment, but >>> > unfortunately this was not expanded to all modules. >>> > >>> > What am I speaking of precisely? I have done a quick survey of the >>> stdlib >>> > and found the following examples. Please note, this is a highly >>> opinionated >>> > list; some names may have been chosen with a very good reason, and >>> others >>> > are just a matter of taste. Hopefully you agree with at least some of >>> them: >>> > >>> > * The CamelCasing in some modules are the most obvious culprits, e.g. >>> > logging and unittest. There is obviously an issue regarding subclasses >>> and >>> > methods that are supposed to be overridden, but I feel we could make it >>> > work. >>> > >>> > * All lower case class names, such as collections.defaultdict and >>> > collections.deque, should be CamelCased. Another example is datetime, >>> which >>> > uses names such as timedelta instead of TimeDelta. >>> > >>> > * Inconsistent names all together, such as re.sub, which I feel >>> should be >>> > re.replace (cf. str.replace). But also re.finditer and re.findall, but >>> no >>> > re.find. >>> > >>> > * Names that do not reflect actual usage, such as >>> ssl.PROTOCOL_SSLv23, >>> > which can in fact not be used as client for SSLv2. >>> > >>> > * Underscore usage, such as tarfile.TarFile.gettarinfo (should it >>> not be >>> > get_tar_info?), http.client.HTTPConnection.getresponse vs >>> set_debuglevel, >>> > and pathlib.Path.samefile vs pathlib.Path.read_text. And is it >>> > pkgutil.iter_modules or is it pathlib.Path.iterdir (or re.finditer)? >>> > >>> > * Usage of various abbreviations, such as in filecmp.cmp >>> > >>> > * Inconsistencies between similar modules, e.g. between >>> > tarfile.TarFile.add and zipfile.ZipFile.write. >>> > >>> > These are just some examples of inconsistent and surprising naming I >>> could >>> > find, other categories are probably also conceivable. Another subject >>> for >>> > reconsideration would be attribute and argument names, but I haven't >>> looked >>> > for those in my quick survey. >>> > >>> > For all of these inconsistencies, I think we should make a >>> 'consistently' >>> > named alternative, and alias the original variant with them (or the >>> other >>> > way around), without defining a deprecation timeline for the original >>> names. >>> > This should make it possible to eventually make the stdlib consistent, >>> > Pythonic and unsurprising. >>> > >>> > What would you think of such an effort? >>> > >>> > Regards, >>> > Ralph Broenink >>> > >>> > [1] >>> https://mail.python.org/pipermail/python-ideas/2010-January/006755.html >>> > [2] >>> https://mail.python.org/pipermail/python-dev/2009-March/086646.html >>> > >>> > >>> > _______________________________________________ >>> > Python-ideas mailing list >>> > Python-ideas at python.org >>> > https://mail.python.org/mailman/listinfo/python-ideas >>> > Code of Conduct: http://python.org/psf/codeofconduct/ >>> _______________________________________________ >>> Python-ideas mailing list >>> Python-ideas at python.org >>> https://mail.python.org/mailman/listinfo/python-ideas >>> Code of Conduct: http://python.org/psf/codeofconduct/ >>> >> > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mertz at gnosis.cx Sun Jul 31 16:16:55 2016 From: mertz at gnosis.cx (David Mertz) Date: Sun, 31 Jul 2016 13:16:55 -0700 Subject: [Python-ideas] Consider adding clip or clamp function to math In-Reply-To: References: <8457d50f-2ea4-47dd-ab44-f3bf28fe8bb3@googlegroups.com> Message-ID: On Sun, Jul 31, 2016 at 12:38 PM, Victor Stinner wrote: > I dislike this API. What's the point of calling clamp(x)? clamp(b, a) is > min(a, b) and clamp(a, max_val=b) is just max(a, b). My point is that all > parameters must be mandatory. > Fair enough. I was envisioning a usage like: bottom, top = None, None # ... some code that might derive values for bottom and/or top x = clamp(x, bottom, top) But this also lets us use the same signature for, e.g.: y = clamp(y, max_val=100) Still, my point wasn't to argue for a signature or implementation, but just opining that the utility is easy enough to write that users can put it in their own library/code. Le 31 juil. 2016 6:41 AM, "David Mertz" a ?crit : > >> Is there some special subtlety or edge case where a hand rolled function >> will go wrong? I like the SO version spelled like this (a little fleshed >> out): >> >> def clamp(val, min_val=None, max_val=None): >> min_val = val if min_val is None else min_val >> max_val = val if max_val is None else max_val >> assert min_val <= max_val >> return max(min(val , max_val), min_val) >> >> >> On Sat, Jul 30, 2016 at 2:57 PM, Neil Girdhar >> wrote: >> >>> Am I right that there is no *obvious* way to do this? If so, I suggest >>> adding math.clip (or math.clamp) to the standard library that has the >>> meaning: >>> >> -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Sun Jul 31 20:39:39 2016 From: guido at python.org (Guido van Rossum) Date: Sun, 31 Jul 2016 17:39:39 -0700 Subject: [Python-ideas] Making the stdlib consistent again In-Reply-To: References: Message-ID: Thoughts of the core dev were expressed clearly earlier in this thread. On Sun, Jul 31, 2016 at 12:47 PM, Eugene Pakhomov wrote: > I'm on Ralph's side here. "Why is this thing named the other way?" was one > of the first questions I asked. And people whom I occasionally teach about > Python, ask the same question over and over again. > > Code breakage happens (PEP 3151 - didn't know about it till it almost bit my > leg off), so we can't shy away from it completely. > Is there any link on the previous thoughts of the core devs on the matter? > Especially regarding the amount of potential code breakage. > I'm genuinely interested, as I think that this amount is negligible if the > new names will be gradually introduced along with a deprecation notice on > (end eventual removal of) the old ones. > As far as I can see, it can only do some harm if someone uses a discouraged > "import *" or monkey-patches some new methods into Python standard > classes/modules, and updates his Python installation. > > Regards, > Eugene > > > On Mon, Aug 1, 2016 at 1:46 AM, Ralph Broenink > wrote: >> >> I'm a bit sad that I'm clearly on the loosing side of the argument. I now >> believe that I must be grossly underestimating the amount of effort and >> overestimating the potential gain. However, I still feel that we should >> strive for consistency in the long run. I do not propose to do this at once, >> but I feel that at least some collaborated effort would be nice. (If not >> only for this kind of mail threads.) >> >> If I would start an effort to - for instance - 'fix' some camelCased >> modules, and attempt to make it 100% backwards compatible, including tests, >> would there be any chance it could be merged at some point? Otherwise, I >> feel it would be a totally pointless effort ;). >> >> On Tue, 26 Jul 2016 at 18:29 Brett Cannon wrote: >>> >>> On Mon, 25 Jul 2016 at 13:03 Mark Mollineaux >>> wrote: >>>> >>>> I've pined for this (and feel a real mental pain every time I use one >>>> of those poorlyCased names)-- I end up using a lot of mental space >>>> remembering exactly HOW each stdlib isn't consistent. >>>> >>>> Aliasing consistent names in each case seems like a real win all >>>> around, personally. >>> >>> >>> For those that want consistent names, you could create a PyPI package >>> that is nothing more than the aliased names as suggested. >>> >>> Otherwise I get the desire for consistency, but as pointed out by a bunch >>> of other core devs, we have thought about this many times and always reach >>> the same conclusion that the amount of work and potential code breakage is >>> too great. >>> >>> -Brett >>> >>>> >>>> >>>> On Mon, Jul 25, 2016 at 10:55 AM, Ralph Broenink >>>> wrote: >>>> > Hi python-ideas, >>>> > >>>> > As you all know, the Python stdlib can sometimes be a bit of an >>>> > inconsistent >>>> > mess that can be surprising in how it names things. This is mostly >>>> > caused by >>>> > the fact that several modules were developed before the introduction >>>> > of >>>> > PEP-8, and now we're stuck with the older naming within these modules. >>>> > >>>> > It has been said and discussed in the past [1][2] that the stdlib is >>>> > in fact >>>> > inconsistent, but fixing this has almost always been disregarded as >>>> > being >>>> > too painful (after all, we don't want a new Python 3 all over again). >>>> > However, this way, we will never move away from these inconsistencies. >>>> > Perhaps this is fine, but I think we should at least consider >>>> > providing >>>> > function and class names that are unsurprising for developers. >>>> > >>>> > While maintaining full backwards compatibility, my idea is that we >>>> > should >>>> > offer consistently named aliases in -eventually- all stdlib modules. >>>> > For >>>> > instance, with Python 2.6, the threading module received this >>>> > treatment, but >>>> > unfortunately this was not expanded to all modules. >>>> > >>>> > What am I speaking of precisely? I have done a quick survey of the >>>> > stdlib >>>> > and found the following examples. Please note, this is a highly >>>> > opinionated >>>> > list; some names may have been chosen with a very good reason, and >>>> > others >>>> > are just a matter of taste. Hopefully you agree with at least some of >>>> > them: >>>> > >>>> > * The CamelCasing in some modules are the most obvious culprits, >>>> > e.g. >>>> > logging and unittest. There is obviously an issue regarding subclasses >>>> > and >>>> > methods that are supposed to be overridden, but I feel we could make >>>> > it >>>> > work. >>>> > >>>> > * All lower case class names, such as collections.defaultdict and >>>> > collections.deque, should be CamelCased. Another example is datetime, >>>> > which >>>> > uses names such as timedelta instead of TimeDelta. >>>> > >>>> > * Inconsistent names all together, such as re.sub, which I feel >>>> > should be >>>> > re.replace (cf. str.replace). But also re.finditer and re.findall, but >>>> > no >>>> > re.find. >>>> > >>>> > * Names that do not reflect actual usage, such as >>>> > ssl.PROTOCOL_SSLv23, >>>> > which can in fact not be used as client for SSLv2. >>>> > >>>> > * Underscore usage, such as tarfile.TarFile.gettarinfo (should it >>>> > not be >>>> > get_tar_info?), http.client.HTTPConnection.getresponse vs >>>> > set_debuglevel, >>>> > and pathlib.Path.samefile vs pathlib.Path.read_text. And is it >>>> > pkgutil.iter_modules or is it pathlib.Path.iterdir (or re.finditer)? >>>> > >>>> > * Usage of various abbreviations, such as in filecmp.cmp >>>> > >>>> > * Inconsistencies between similar modules, e.g. between >>>> > tarfile.TarFile.add and zipfile.ZipFile.write. >>>> > >>>> > These are just some examples of inconsistent and surprising naming I >>>> > could >>>> > find, other categories are probably also conceivable. Another subject >>>> > for >>>> > reconsideration would be attribute and argument names, but I haven't >>>> > looked >>>> > for those in my quick survey. >>>> > >>>> > For all of these inconsistencies, I think we should make a >>>> > 'consistently' >>>> > named alternative, and alias the original variant with them (or the >>>> > other >>>> > way around), without defining a deprecation timeline for the original >>>> > names. >>>> > This should make it possible to eventually make the stdlib consistent, >>>> > Pythonic and unsurprising. >>>> > >>>> > What would you think of such an effort? >>>> > >>>> > Regards, >>>> > Ralph Broenink >>>> > >>>> > [1] >>>> > https://mail.python.org/pipermail/python-ideas/2010-January/006755.html >>>> > [2] >>>> > https://mail.python.org/pipermail/python-dev/2009-March/086646.html >>>> > >>>> > >>>> > _______________________________________________ >>>> > Python-ideas mailing list >>>> > Python-ideas at python.org >>>> > https://mail.python.org/mailman/listinfo/python-ideas >>>> > Code of Conduct: http://python.org/psf/codeofconduct/ >>>> _______________________________________________ >>>> Python-ideas mailing list >>>> Python-ideas at python.org >>>> https://mail.python.org/mailman/listinfo/python-ideas >>>> Code of Conduct: http://python.org/psf/codeofconduct/ >> >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -- --Guido van Rossum (python.org/~guido) From steve at pearwood.info Sun Jul 31 22:47:39 2016 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 1 Aug 2016 12:47:39 +1000 Subject: [Python-ideas] Consider adding clip or clamp function to math In-Reply-To: References: <8457d50f-2ea4-47dd-ab44-f3bf28fe8bb3@googlegroups.com> Message-ID: <20160801024738.GC13777@ando.pearwood.info> On Sun, Jul 31, 2016 at 09:38:44PM +0200, Victor Stinner wrote: > I dislike this API. What's the point of calling clamp(x)? clamp(b, a) is > min(a, b) and clamp(a, max_val=b) is just max(a, b). You have that the wrong way around. If you supply a lower-bounds, you must take the max(), not the min(). If you supply a upper-bounds, you take the min(), not the max(). It's easy to get wrong. > My point is that all parameters must be mandatory. I don't care too much whether the parameters are mandatory or have defaults, so long as it is *possible* to pass something for the lower and upper bounds which mean "unbounded". There are four obvious alternatives (well three obvious ones and one surprising one): (1) Explicitly pass -INFINITY or +INFINITY as needed; but which infinity, float or Decimal? If you pass the wrong one, you may have to pay the cost of converting your values to float/Decimal, which could end up expensive if you have a lot of them. (2) Pass a NAN as the bounds. With my implementation, that actually works! But it's a surprising accident of implementation, it feels wrong and looks weird, and again, it may require converting the values to float/Decimal. (3) Use some special Infimum and Supremum objects which are smaller than, and greater than, every other value. But we don't have such objects, so you'd need to create your own. (4) Use None as a placeholder for "no limit". That's my preferred option. Of course, even if None is accepted as "no limit", the caller can still explicitly provide an infinity if they prefer. As I said, I don't particularly care whether the lower and upper bounds have default values. But I think it is useful and elegant to accept None (as well as infinity) to mean "no limit". -- Steve