From p.f.moore at gmail.com  Thu Feb  1 04:30:04 2018
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 1 Feb 2018 09:30:04 +0000
Subject: [Python-Dev] Why is Python for Windows compiled with MSVC?
In-Reply-To: <CAGE7PN+6iZObfjHHv1q+Vnd1Y8dpmNTA6HBUt+piy1TrYxxP6g@mail.gmail.com>
References: <mailman.1199.1517397240.2815.python-dev@python.org>
 <AM4PR04MB21150497592D868BAF9BB308A7FB0@AM4PR04MB2115.eurprd04.prod.outlook.com>
 <3zWzR85rP2zFrJV@mail.python.org>
 <CAGE7PN+6iZObfjHHv1q+Vnd1Y8dpmNTA6HBUt+piy1TrYxxP6g@mail.gmail.com>
Message-ID: <CACac1F8O3o0CUfJfTbWUjsoFwwkVt7fFhS0wxjBJFnRxqN4Zgg@mail.gmail.com>

On 1 February 2018 at 00:42, Gregory P. Smith <greg at krypto.org> wrote:
> TL;DR of Steve's post - MSVC is the compiler of choice for most serious
> software on Windows. So we use it to best integrate with the world. There is
> no compelling reason to change that.
>
> The free-as-in-beer MSVC community edition is finally non-sucky (their
> earlier efforts were crippled, they seem to have learned the lesson)
>
> There are other viable Windows compilers.  If we want to support those in
> CPython someone needs to contribute the work to do so, ongoing maintenance,
> and buildbots.  I'd love to see a Clang based Windows build (Google Chrome
> is built using that).  But I have no motivating reason to do the work.  I
> believe such a build could be made to integrate and inter-operate fully with
> MSVC builds and ABIs.  We could probably even make cross-compilation of
> extensions from Linux -> Windows work that way.
>
> We're highly unlikely to ever stop shipping python.org Windows binaries
> built with anything other than MSVC unless Microsoft takes a turn toward the
> dark side again.

I would also point out that CPython (distutils, specifically)
supported mingw builds (that's the original mingw 32-bit version) for
a long time. Support for that bit-rotted as the mingw project
fragmented with various 64-bit versions, and slow progress from the
mingw project(s) for supporting newer CRTs on Windows. No-one from the
community who used mingw was providing patches back to distutils or
Python, and so the support for mingw was (I believe) dropped. It
sounds as if people have since then got things to a point where
building extensions with (some form of ) mingw is possible, but I
don't know the details, and I'm not aware of any documentation that's
been contributed back to Python on how "mingw support for building
extensions" works these days.

We'd be more than happy to support building extensions with
alternative compilers (such as one of the various gcc builds that go
under the banner of "mingw") - that's a completely different matter
than switching the CPython build process to use mingw - but it's down
to the community of users of such a compiler to contribute that
support. Expecting "someone else", and particularly someone who
doesn't need it, to provide that support, is unreasonable.

Disclaimer: This all comes under "as far as I know" - I did a lot of
work on mingw support back when it was supported (including trying to
push patches back to the mingw people) but haven't been following it
for a long time. Things could be very different now than I remember.

Paul

From j.orponen at 4teamwork.ch  Thu Feb  1 06:35:35 2018
From: j.orponen at 4teamwork.ch (Joni Orponen)
Date: Thu, 1 Feb 2018 12:35:35 +0100
Subject: [Python-Dev] OS-X builds for 3.7.0
In-Reply-To: <CALGmxEJ5Jjdrsbf4TGFsV1UP+y-r1BAMxg2EcjpNwLDX=U+ecw@mail.gmail.com>
References: <CALGmxE+RxK+YMFax9J=5c7VRUhnfQCieayw-Q2KhxJRk3Fvghg@mail.gmail.com>
 <CA+LW3i0Xto+Jf8_r8Dda2EWD3OcLBsH-XNTPb3Q-qw2PMJFTEw@mail.gmail.com>
 <CALGmxEKVER+5TD36XXJd6dnmpkbtn3M4Wn0Zcm5NesHT7JVKwA@mail.gmail.com>
 <CA+LW3i36DxUbYGDhUPTL0VyQ+_pMji2aSkven1EA01cSBf-XPg@mail.gmail.com>
 <CALGmxEJ5Jjdrsbf4TGFsV1UP+y-r1BAMxg2EcjpNwLDX=U+ecw@mail.gmail.com>
Message-ID: <CA+LW3i2-D1xJXyYbX7yc8ptU7ik3=7P6BuXVOyJ4D5JyzBWF0A@mail.gmail.com>

On Thu, Feb 1, 2018 at 12:18 AM, Chris Barker <chris.barker at noaa.gov> wrote:

> On Wed, Jan 31, 2018 at 3:13 AM, Joni Orponen <j.orponen at 4teamwork.ch>
> wrote:
>
>> On Wed, Jan 31, 2018 at 12:43 AM, Chris Barker - NOAA Federal <
>> chris.barker at noaa.gov> wrote:
>>
>>> And maybe we could even get rid of the "Framework" builds......
>>>>
>>>
>>> Please do not. These make life easier for doing things the Apple way for
>>> signed sandboxed applications.
>>>
>>> For the record, are you re-distributing the python.org builds, or
>>> re-building yourself?
>>>
>>
>> We are re-building ourselves.
>>
>
> Then it makes no difference to you if the pyton.org binaries are
> Framework builds... though maybe you want the configure target available.
>

And if the official distribution distributes a Framework, the configure
target and build for that are also actually maintained, which will keep
them likelier usable.

Would we not be doing a couple of very specific things, we'd be able to use
the distributed Framework as well. It is not unconceivable we could use it
as-is in the future or for other projects.

-- 
Joni Orponen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180201/1d6855c6/attachment.html>

From olegs at traiana.com  Thu Feb  1 04:19:31 2018
From: olegs at traiana.com (Oleg Sivokon)
Date: Thu, 1 Feb 2018 09:19:31 +0000
Subject: [Python-Dev] Why is Python for Windows compiled with MSVC?
In-Reply-To: <69628ee3-d220-4934-aa73-f652cb5c6f2d@HE1EUR02FT027.eop-EUR02.prod.protection.outlook.com>
References: <mailman.1199.1517397240.2815.python-dev@python.org>
 <AM4PR04MB21150497592D868BAF9BB308A7FB0@AM4PR04MB2115.eurprd04.prod.outlook.com>,
 <69628ee3-d220-4934-aa73-f652cb5c6f2d@HE1EUR02FT027.eop-EUR02.prod.protection.outlook.com>
Message-ID: <AM4PR04MB2115C2BF009DFF3A066C16B8A7FA0@AM4PR04MB2115.eurprd04.prod.outlook.com>


> so why shouldn?t the one with the most users?

Because it makes compilation difficult, and cross-compilatin completely impossible?  Why is it difficult: a package maintainer needs to (1) buy MS Windows (2) create a special workflow for compiling on a different machine.  This is both costly and inconsistent with free-as-in-freedom...  It makes cross-compilation impossible because libraries produced by any tool that can run on all target platforms are incompatible with Python binaries on MS Windows.

Again, many languages (i.e. projects similar in size an purpose to CPython) took a different approach: they use GNU compilers to be able to compile cross-platform.  This is true for Ruby and Go at least.  I would need to investigate further, but I think these two examples should be enough.

> I?m likely biased because I work there and I?m the main intermediary with python-dev, but these days Microsoft is one of the strongest supporters of CPython. We employ the most core developers of any private company and we all are allowed work time to contribute, we provide full access to our development tools and platforms to all core developers and some prominent projects, we?ve made fixes, enhancements and releases or core products such as the CRT, MSVC, Visual Studio, Visual Studio Code, and Azure SPECIFICALLY to support CPython development and users. As far as I know, ALL the Windows buildbots are running on Azure subscriptions that Microsoft provides (though managed by some awesome volunteers). You?ll see us at PyCon US under the biggest banner and we?ll have a booth filled with engineers and not recruiters. Crash reports from thousands of opted-in users come into our systems and have directly lead to both CPython and Windows bug fixes.

Oh, so this is the real reason... well, corporate interests are hard to argue against.  But, this is an interesting statistic nevertheless.  Thanks for letting me know.

Best.

Oleg
This communication and all information contained in or attached to it is confidential, intended solely for the addressee, may be legally privileged and is the intellectual property of one of the companies of NEX Group plc ("NEX") or third parties. If you are not the intended addressee or receive this message in error, please immediately delete all copies of it and notify the sender. We have taken precautions to minimise the risk of transmitting software viruses, but we advise you to carry out your own virus checks on any attachments. We do not accept liability for any loss or damage caused by software viruses. NEX reserves the right to monitor all communications. We do not accept any legal responsibility for the content of communications, and no communication shall be considered legally binding. Furthermore, if the content of this communication is personal or unconnected with our business, we accept no liability or responsibility for it. NEX Group plc is a public limited company registered in England and Wales under number 10013770 and certain of its affiliates are authorised and regulated by regulatory authorities. For further regulatory information please see www.NEX.com.

From christian at python.org  Thu Feb  1 07:14:43 2018
From: christian at python.org (Christian Heimes)
Date: Thu, 1 Feb 2018 13:14:43 +0100
Subject: [Python-Dev] Why is Python for Windows compiled with MSVC?
In-Reply-To: <AM4PR04MB2115C2BF009DFF3A066C16B8A7FA0@AM4PR04MB2115.eurprd04.prod.outlook.com>
References: <mailman.1199.1517397240.2815.python-dev@python.org>
 <AM4PR04MB21150497592D868BAF9BB308A7FB0@AM4PR04MB2115.eurprd04.prod.outlook.com>
 <69628ee3-d220-4934-aa73-f652cb5c6f2d@HE1EUR02FT027.eop-EUR02.prod.protection.outlook.com>
 <AM4PR04MB2115C2BF009DFF3A066C16B8A7FA0@AM4PR04MB2115.eurprd04.prod.outlook.com>
Message-ID: <p4v07l$khh$1@blaine.gmane.org>

On 2018-02-01 10:19, Oleg Sivokon wrote:
> 
>> so why shouldn?t the one with the most users?
> 
> Because it makes compilation difficult, and cross-compilatin completely impossible?  Why is it difficult: a package maintainer needs to (1) buy MS Windows (2) create a special workflow for compiling on a different machine.  This is both costly and inconsistent with free-as-in-freedom...  It makes cross-compilation impossible because libraries produced by any tool that can run on all target platforms are incompatible with Python binaries on MS Windows.
> 
> Again, many languages (i.e. projects similar in size an purpose to CPython) took a different approach: they use GNU compilers to be able to compile cross-platform.  This is true for Ruby and Go at least.  I would need to investigate further, but I think these two examples should be enough.
> 
>> I?m likely biased because I work there and I?m the main intermediary with python-dev, but these days Microsoft is one of the strongest supporters of CPython. We employ the most core developers of any private company and we all are allowed work time to contribute, we provide full access to our development tools and platforms to all core developers and some prominent projects, we?ve made fixes, enhancements and releases or core products such as the CRT, MSVC, Visual Studio, Visual Studio Code, and Azure SPECIFICALLY to support CPython development and users. As far as I know, ALL the Windows buildbots are running on Azure subscriptions that Microsoft provides (though managed by some awesome volunteers). You?ll see us at PyCon US under the biggest banner and we?ll have a booth filled with engineers and not recruiters. Crash reports from thousands of opted-in users come into our systems and have directly lead to both CPython and Windows bug fixes.
> 
> Oh, so this is the real reason... well, corporate interests are hard to argue against.  But, this is an interesting statistic nevertheless.  Thanks for letting me know.

You are drawing the wrong conclusion here. CPython has been using MSVC
many years before Microsoft even started to sponsor MSDN subscriptions
for core developers. MSVC is the default Windows compiler for over 20
years now. IIRC Microsoft started to donate MSDN subscription for about
5 years and Steve has been helping out with Windows improvement for
about 5 years.

Christian




From brian at python.org  Thu Feb  1 08:15:32 2018
From: brian at python.org (Brian Curtin)
Date: Thu, 1 Feb 2018 08:15:32 -0500
Subject: [Python-Dev] Why is Python for Windows compiled with MSVC?
In-Reply-To: <AM4PR04MB2115C2BF009DFF3A066C16B8A7FA0@AM4PR04MB2115.eurprd04.prod.outlook.com>
References: <mailman.1199.1517397240.2815.python-dev@python.org>
 <AM4PR04MB21150497592D868BAF9BB308A7FB0@AM4PR04MB2115.eurprd04.prod.outlook.com>
 <69628ee3-d220-4934-aa73-f652cb5c6f2d@HE1EUR02FT027.eop-EUR02.prod.protection.outlook.com>
 <AM4PR04MB2115C2BF009DFF3A066C16B8A7FA0@AM4PR04MB2115.eurprd04.prod.outlook.com>
Message-ID: <CAD+XWwoa5CB_hpwVC+wEJ6px5e-ECZA6AijA3nbi2KBhY3fKNw@mail.gmail.com>

On Thu, Feb 1, 2018 at 4:19 AM, Oleg Sivokon <olegs at traiana.com> wrote:

>
> > so why shouldn?t the one with the most users?
>
> Because it makes compilation difficult, and cross-compilatin completely
> impossible?  Why is it difficult: a package maintainer needs to (1) buy MS
> Windows (2) create a special workflow for compiling on a different
> machine.  This is both costly and inconsistent with free-as-in-freedom...
> It makes cross-compilation impossible because libraries produced by any
> tool that can run on all target platforms are incompatible with Python
> binaries on MS Windows.
>
> Again, many languages (i.e. projects similar in size an purpose to
> CPython) took a different approach: they use GNU compilers to be able to
> compile cross-platform.  This is true for Ruby and Go at least.  I would
> need to investigate further, but I think these two examples should be
> enough.
>

They should be enough for *what*, though? You can tell people what everyone
else is doing, but the difference between that and what we have is
someone's time and effort.

> I?m likely biased because I work there and I?m the main intermediary with
> python-dev, but these days Microsoft is one of the strongest supporters of
> CPython. We employ the most core developers of any private company and we
> all are allowed work time to contribute, we provide full access to our
> development tools and platforms to all core developers and some prominent
> projects, we?ve made fixes, enhancements and releases or core products such
> as the CRT, MSVC, Visual Studio, Visual Studio Code, and Azure SPECIFICALLY
> to support CPython development and users. As far as I know, ALL the Windows
> buildbots are running on Azure subscriptions that Microsoft provides
> (though managed by some awesome volunteers). You?ll see us at PyCon US
> under the biggest banner and we?ll have a booth filled with engineers and
> not recruiters. Crash reports from thousands of opted-in users come into
> our systems and have directly lead to both CPython and Windows bug fixes.
>
> Oh, so this is the real reason... well, corporate interests are hard to
> argue against.  But, this is an interesting statistic nevertheless.  Thanks
> for letting me know.


I think that's a mischaracterization of the situation. The MS toolchain was
chosen some time long before I (or Steve) got involved, and when I upgraded
us from VS2008 to VS2010 for 3.3 ~6 years ago I had several messages
similar to this thread. As much as Steve is unlikely to do the work to
initiate and maintain support of these other tools?whether due to his
employer's interests or his own?I too was unlikely to do work like this
thread is asking. In fact, the chances I would have done it were zero
because I was sitting on my couch upgrading our Visual Studio versions
because it let me do better stuff at my day job, though I was always open
to review patches that supported alternatives without major disruption.
However, they never came. I suspect the same could be said of Martin and
anyone else working in this area prior to that, because nothing has really
changed.

Like the previous times this sort of question has come up?and really, for
any question on this list?it ultimately turns into a matter of how much the
solution is wanted and how much effort people are willing to give to make
it happen. Historically, the former has had small amounts, and the latter
has had much smaller amounts. Without a change there I don't think one will
materialize in a released version of Python.

Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180201/ce6e0f3d/attachment.html>

From stefanrin at gmail.com  Thu Feb  1 08:55:27 2018
From: stefanrin at gmail.com (Stefan Ring)
Date: Thu, 1 Feb 2018 14:55:27 +0100
Subject: [Python-Dev] Why is Python for Windows compiled with MSVC?
In-Reply-To: <CAD+XWwoa5CB_hpwVC+wEJ6px5e-ECZA6AijA3nbi2KBhY3fKNw@mail.gmail.com>
References: <mailman.1199.1517397240.2815.python-dev@python.org>
 <AM4PR04MB21150497592D868BAF9BB308A7FB0@AM4PR04MB2115.eurprd04.prod.outlook.com>
 <69628ee3-d220-4934-aa73-f652cb5c6f2d@HE1EUR02FT027.eop-EUR02.prod.protection.outlook.com>
 <AM4PR04MB2115C2BF009DFF3A066C16B8A7FA0@AM4PR04MB2115.eurprd04.prod.outlook.com>
 <CAD+XWwoa5CB_hpwVC+wEJ6px5e-ECZA6AijA3nbi2KBhY3fKNw@mail.gmail.com>
Message-ID: <CAAxjCEyHsEAhJNWi9P4_8x-P_W0i2nKxNHS6vpf0Ya1v8_voJA@mail.gmail.com>

> As much as Steve is unlikely to do the work to initiate and
> maintain support of these other tools?whether due to his employer's
> interests or his own?I too was unlikely to do work like this thread is
> asking. In fact, the chances I would have done it were zero because I was
> sitting on my couch upgrading our Visual Studio versions because it let me
> do better stuff at my day job, though I was always open to review patches
> that supported alternatives without major disruption. However, they never
> came. I suspect the same could be said of Martin and anyone else working in
> this area prior to that, because nothing has really changed.

It would be cool though if Microsoft started providing a
cross-compiler running on Linux. This could even be the only compiler
shipped with Visual Studio, now that Windows can run Linux userland.
Cross-compilers from Microsoft would not be totally unheard of. IIRC,
the last DOS versions (Visual C++ 1.5x) were Win32 binaries building
for DOS 16 bit. Technically speaking, using a 32 bit compiler for
building for 64 bit Windows or the other way around would probably
count as cross-compilation anyway.

From barry at python.org  Thu Feb  1 09:40:56 2018
From: barry at python.org (Barry Warsaw)
Date: Thu, 1 Feb 2018 09:40:56 -0500
Subject: [Python-Dev] Why is Python for Windows compiled with MSVC?
In-Reply-To: <AM4PR04MB2115C2BF009DFF3A066C16B8A7FA0@AM4PR04MB2115.eurprd04.prod.outlook.com>
References: <mailman.1199.1517397240.2815.python-dev@python.org>
 <AM4PR04MB21150497592D868BAF9BB308A7FB0@AM4PR04MB2115.eurprd04.prod.outlook.com>
 <69628ee3-d220-4934-aa73-f652cb5c6f2d@HE1EUR02FT027.eop-EUR02.prod.protection.outlook.com>
 <AM4PR04MB2115C2BF009DFF3A066C16B8A7FA0@AM4PR04MB2115.eurprd04.prod.outlook.com>
Message-ID: <59693230-FC88-430A-9887-D50B49E196CE@python.org>

On Feb 1, 2018, at 04:19, Oleg Sivokon <olegs at traiana.com> wrote:
> 
> Oh, so this is the real reason... well, corporate interests are hard to argue against.  But, this is an interesting statistic nevertheless.  Thanks for letting me know.

Maybe it hasn?t happened because no volunteer has stepped up to do it.  Or maybe no corporation thinks it a good business investment to pay employees to do it.  What other options are there, and if none, then which of those can you solve?

Or to paraphrase a timeless quote: Ask not what Python can do for you, ask what you can do for Python.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180201/01fbe3a2/attachment.sig>

From olegs at traiana.com  Thu Feb  1 04:50:01 2018
From: olegs at traiana.com (Oleg Sivokon)
Date: Thu, 1 Feb 2018 09:50:01 +0000
Subject: [Python-Dev] Why is Python for Windows compiled with MSVC?
In-Reply-To: <CACac1F8O3o0CUfJfTbWUjsoFwwkVt7fFhS0wxjBJFnRxqN4Zgg@mail.gmail.com>
References: <mailman.1199.1517397240.2815.python-dev@python.org>
 <AM4PR04MB21150497592D868BAF9BB308A7FB0@AM4PR04MB2115.eurprd04.prod.outlook.com>
 <3zWzR85rP2zFrJV@mail.python.org>
 <CAGE7PN+6iZObfjHHv1q+Vnd1Y8dpmNTA6HBUt+piy1TrYxxP6g@mail.gmail.com>,
 <CACac1F8O3o0CUfJfTbWUjsoFwwkVt7fFhS0wxjBJFnRxqN4Zgg@mail.gmail.com>
Message-ID: <AM4PR04MB211569A4CFEBBCA36D334F6EA7FA0@AM4PR04MB2115.eurprd04.prod.outlook.com>

> I would also point out that CPython (distutils, specifically)
supported mingw builds (that's the original mingw 32-bit version) for
a long time. Support for that bit-rotted as the mingw project
fragmented with various 64-bit versions, and slow progress from the
mingw project(s) for supporting newer CRTs on Windows. No-one from the
community who used mingw was providing patches back to distutils or
Python, and so the support for mingw was (I believe) dropped. It
sounds as if people have since then got things to a point where
building extensions with (some form of ) mingw is possible, but I
don't know the details, and I'm not aware of any documentation that's
been contributed back to Python on how "mingw support for building
extensions" works these days.

The question isn't whether I can build Python with MinGW: I know I can.  The problem is that if I do that, I make the users of my package depend on my special build of Python.  This also means that the packages built by others (who are unaware of my special build of Python) may or may not work for them.  In practice, if I want to make sure that the users of my packages can actually use them, I cannot allow myself to link them against a Python binary of my choice.  It has to be their choice.

We'd be more than happy to support building extensions with
alternative compilers (such as one of the various gcc builds that go
under the banner of "mingw") - that's a completely different matter
than switching the CPython build process to use mingw - but it's down
to the community of users of such a compiler to contribute that
support. Expecting "someone else", and particularly someone who
doesn't need it, to provide that support, is unreasonable.

Well, guys, since you are here, I assumed you were in the business of deciding which compiler to compile your code with.  I really, just asked a question.  It's obvious that I don't agree with your decision, but I wanted to hear your argument.  Nowhere did I suggest that you should do any work for me or anything of that kind.  All I wanted was information to make an informed decision about using Python and its extensions ecosystem.  I've got my answers, for which I'm thankful.

I'm sorry for the disclaimer that follows this email.  Unfortunately, I'm too a slave of circumstances, s.a. my employer and the IT department that would not let me send an email without this legal mumbo-jumbo.

Best.

Oleg
This communication and all information contained in or attached to it is confidential, intended solely for the addressee, may be legally privileged and is the intellectual property of one of the companies of NEX Group plc ("NEX") or third parties. If you are not the intended addressee or receive this message in error, please immediately delete all copies of it and notify the sender. We have taken precautions to minimise the risk of transmitting software viruses, but we advise you to carry out your own virus checks on any attachments. We do not accept liability for any loss or damage caused by software viruses. NEX reserves the right to monitor all communications. We do not accept any legal responsibility for the content of communications, and no communication shall be considered legally binding. Furthermore, if the content of this communication is personal or unconnected with our business, we accept no liability or responsibility for it. NEX Group plc is a public limited company registered in England and Wales under number 10013770 and certain of its affiliates are authorised and regulated by regulatory authorities. For further regulatory information please see www.NEX.com.

From chris.barker at noaa.gov  Thu Feb  1 15:24:53 2018
From: chris.barker at noaa.gov (Chris Barker)
Date: Thu, 1 Feb 2018 12:24:53 -0800
Subject: [Python-Dev] OS-X builds for 3.7.0
In-Reply-To: <p4tmp1$57e$1@blaine.gmane.org>
References: <CALGmxE+RxK+YMFax9J=5c7VRUhnfQCieayw-Q2KhxJRk3Fvghg@mail.gmail.com>
 <CAOYw7dtYr4o0VZ=rFVKU=JAVvjOYzDiKs27cT_P0uYAK1ssm1w@mail.gmail.com>
 <CA+LW3i3+r54f51ZPR08J9CWuxc0K1nhSXwmTPBTr7hDcOfmaCw@mail.gmail.com>
 <CAOYw7dvNLxUbs5KNuc6Nr-vndfd845b8dh70sp1GMbEBgG51=Q@mail.gmail.com>
 <CAOYw7dv1YA2rtAc0Uvw+tEJgvo7GTXBpNCu1a2DcRgCOQPUW_w@mail.gmail.com>
 <CA+LW3i0NYygp4srSBXZv+542RUYFnytZkj9crhW0v98Ay7arFg@mail.gmail.com>
 <CAOYw7du2Tw0TMBqrzh99a8uf+ne5qtYAeqjKUiMRT2mMHEUvmg@mail.gmail.com>
 <CAEfz+Tzo6G4_92-KVaZH93F+4Ee-qu+efb3k9MoK31VaqK9AJw@mail.gmail.com>
 <CALGmxE+yZc6E4yeQPXp6pmFj07o5Sua0Etr1j9W817YHy_vJaw@mail.gmail.com>
 <p4tmp1$57e$1@blaine.gmane.org>
Message-ID: <CALGmxEJPL6SePGx0rgmtTGG6Zcw=Vuizc8M30qjoi9yrKQQOAg@mail.gmail.com>

>> Ned Deily is in charge of the Mac build (as well as current release
> manager).  Within the last week, he revised the official builds (now two, I
> believe) for 3.7.0b1, due in a day or so.  One will be a future oriented
> 64-bit build.  The PR and What's New have more.
>

What's New doesn't mention it, but Ned's annoucemtn does:

"""
Attention macOS users: with 3.7.0b1, we are providing a choice of
two binary installers.  The new variant provides a 64-bit-only
version for macOS 10.9 and later systems; this variant also now
includes its own built-in version of Tcl/Tk 8.6.  We welcome your
feedback.
"""

So that's a start -- thanks Ned!

He may not be reading this thread, but will read MacOS tracker issues with
> a specific proposal, data and a patch.  Comparisons should be against the
> current master or an installed 3.7.0b1.


I hope the folks on this thread that know what they are doing can test and
make suggestions :-)

-CHB



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180201/6644e3ed/attachment.html>

From elprans at gmail.com  Thu Feb  1 19:34:19 2018
From: elprans at gmail.com (Elvis Pranskevichus)
Date: Thu, 01 Feb 2018 19:34:19 -0500
Subject: [Python-Dev] Dataclasses and correct hashability
Message-ID: <3951276.IZdOgMHjlT@hammer.magicstack.net>

There appears to be a critical omission from the current dataclass 
implementation: it does not make hash=True fields immutable.  

Per Python spec: 

"the implementation of hashable collections requires that a key?s hash 
value is immutable (if the object?s hash value changes, it will be in 
the wrong hash bucket)"

Yet:

    import dataclasses

    @dataclasses.dataclass(hash=True)
    class A:
        foo: int = dataclasses.field(hash=True, compare=True)

    a = A(foo=1)

    s = set()
    s.add(a)   # s == {a}
    a.foo = 2

    print(a in s)
    print({a} == s}
    print(s == s)

    # prints False False True


This looks to me like a clearly wrong behavior.


                                    Elvis



From eric at trueblade.com  Thu Feb  1 20:17:00 2018
From: eric at trueblade.com (Eric V. Smith)
Date: Thu, 1 Feb 2018 20:17:00 -0500
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <3951276.IZdOgMHjlT@hammer.magicstack.net>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
Message-ID: <4b8e20d5-31db-69e3-2dbb-d2853d47c6c0@trueblade.com>

On 2/1/2018 7:34 PM, Elvis Pranskevichus wrote:
> There appears to be a critical omission from the current dataclass
> implementation: it does not make hash=True fields immutable.
> 
> Per Python spec:
> 
> "the implementation of hashable collections requires that a key?s hash
> value is immutable (if the object?s hash value changes, it will be in
> the wrong hash bucket)"
> 
> Yet:
> 
>      import dataclasses
> 
>      @dataclasses.dataclass(hash=True)
>      class A:
>          foo: int = dataclasses.field(hash=True, compare=True)
> 
>      a = A(foo=1)
> 
>      s = set()
>      s.add(a)   # s == {a}
>      a.foo = 2
> 
>      print(a in s)
>      print({a} == s}
>      print(s == s)
> 
>      # prints False False True
> 
> 
> This looks to me like a clearly wrong behavior.
> 
> 
>                                      Elvis

Data classes do not protect you from doing the wrong thing. This is the 
same as writing:

class A:
     def __init__(self, foo):
         self.foo = foo
     def __hash__(self):
         return hash((self.foo,))

You're allowed to override the parameters to dataclasses.dataclass for 
cases where you know what you're doing. Consenting adults, and all.

Eric.

From eric at trueblade.com  Thu Feb  1 20:21:03 2018
From: eric at trueblade.com (Eric V. Smith)
Date: Thu, 1 Feb 2018 20:21:03 -0500
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <4b8e20d5-31db-69e3-2dbb-d2853d47c6c0@trueblade.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <4b8e20d5-31db-69e3-2dbb-d2853d47c6c0@trueblade.com>
Message-ID: <441a5309-b4b6-adb4-27ca-3431f123f7dc@trueblade.com>

On 2/1/2018 8:17 PM, Eric V. Smith wrote:
> On 2/1/2018 7:34 PM, Elvis Pranskevichus wrote:
>> There appears to be a critical omission from the current dataclass
>> implementation: it does not make hash=True fields immutable.
>>
>> Per Python spec:
>>
>> "the implementation of hashable collections requires that a key?s hash
>> value is immutable (if the object?s hash value changes, it will be in
>> the wrong hash bucket)"
>>
>> Yet:
>>
>> ???? import dataclasses
>>
>> ???? @dataclasses.dataclass(hash=True)
>> ???? class A:
>> ???????? foo: int = dataclasses.field(hash=True, compare=True)
>>
>> ???? a = A(foo=1)
>>
>> ???? s = set()
>> ???? s.add(a)?? # s == {a}
>> ???? a.foo = 2
>>
>> ???? print(a in s)
>> ???? print({a} == s}
>> ???? print(s == s)
>>
>> ???? # prints False False True
>>
>>
>> This looks to me like a clearly wrong behavior.
>>
>>
>> ???????????????????????????????????? Elvis
> 
> Data classes do not protect you from doing the wrong thing. This is the 
> same as writing:
> 
> class A:
>  ??? def __init__(self, foo):
>  ??????? self.foo = foo
>  ??? def __hash__(self):
>  ??????? return hash((self.foo,))
> 
> You're allowed to override the parameters to dataclasses.dataclass for 
> cases where you know what you're doing. Consenting adults, and all.

I should add: This is why you shouldn't override the default (hash=None) 
unless you know what the consequences are. Can I ask why you want to 
specify hash=True?

Eric


From elprans at gmail.com  Thu Feb  1 20:29:57 2018
From: elprans at gmail.com (Elvis Pranskevichus)
Date: Thu, 01 Feb 2018 20:29:57 -0500
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <441a5309-b4b6-adb4-27ca-3431f123f7dc@trueblade.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <4b8e20d5-31db-69e3-2dbb-d2853d47c6c0@trueblade.com>
 <441a5309-b4b6-adb4-27ca-3431f123f7dc@trueblade.com>
Message-ID: <34493569.S535hZLbfa@klinga.prans.org>

On Thursday, February 1, 2018 8:21:03 PM EST Eric V. Smith wrote:
> I should add: This is why you shouldn't override the default
> (hash=None) unless you know what the consequences are. Can I ask
> why you want to specify hash=True?

hash=None and compare=True leads to the same result, which, I think is 
even worse.  

> You're allowed to override the parameters to dataclasses.dataclass
> for cases where you know what you're doing. Consenting adults,
> and all.

I don't agree with this.  You are comparing implicit dataclass 
behavior with an explicit shoot-in-the-foot __hash__() definition.

                            Elvis



From eric at trueblade.com  Thu Feb  1 20:37:41 2018
From: eric at trueblade.com (Eric V. Smith)
Date: Thu, 1 Feb 2018 20:37:41 -0500
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <34493569.S535hZLbfa@klinga.prans.org>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <4b8e20d5-31db-69e3-2dbb-d2853d47c6c0@trueblade.com>
 <441a5309-b4b6-adb4-27ca-3431f123f7dc@trueblade.com>
 <34493569.S535hZLbfa@klinga.prans.org>
Message-ID: <0a81521e-74f2-4883-6ab0-96e6fb03a419@trueblade.com>

On 2/1/2018 8:29 PM, Elvis Pranskevichus wrote:
> On Thursday, February 1, 2018 8:21:03 PM EST Eric V. Smith wrote:
>> I should add: This is why you shouldn't override the default
>> (hash=None) unless you know what the consequences are. Can I ask
>> why you want to specify hash=True?
> 
> hash=None and compare=True leads to the same result, which, I think is
> even worse.

Have you actually tried that?

 >>> @dataclass(hash=None)
... class A:
...   foo: int = field(hash=True, compare=True)
...
 >>> hash(A(2))
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'A'

I believe the default hash=None on the class decorator does right thing. 
Please provide a counter-example.

 >> You're allowed to override the parameters to dataclasses.dataclass
 >> for cases where you know what you're doing. Consenting adults,
 >> and all.
 >
 > I don't agree with this.  You are comparing implicit dataclass
 > behavior with an explicit shoot-in-the-foot __hash__() definition.

I don't recommend ever specifying the decorator hash= parameter unless 
you have an unusual use case, in which case it's on you to get it right. 
There was recently a long python-dev discussion on this issue. I need to 
update the PEP to reflect it, but the advice still stands: you almost 
always want to use the default hash=None.

Do you have a use case for specifying a hash function on a class with 
mutable instances? Maybe you want frozen=True?

Eric

From elprans at gmail.com  Thu Feb  1 20:49:44 2018
From: elprans at gmail.com (Elvis Pranskevichus)
Date: Thu, 01 Feb 2018 20:49:44 -0500
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <0a81521e-74f2-4883-6ab0-96e6fb03a419@trueblade.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <34493569.S535hZLbfa@klinga.prans.org>
 <0a81521e-74f2-4883-6ab0-96e6fb03a419@trueblade.com>
Message-ID: <1937305.WBmin8xbao@klinga.prans.org>

On Thursday, February 1, 2018 8:37:41 PM EST Eric V. Smith wrote:
> > hash=None and compare=True leads to the same result, which, I
> > think is even worse.
> 
> Have you actually tried that?

I meant this:

@dataclasses.dataclass(hash=True)
class A:
    foo: int = dataclasses.field(compare=True)

> I don't recommend ever specifying the decorator hash= parameter
> unless you have an unusual use case, in which case it's on you to
> get it right. 

In my experience this type of breakage is so subtle that people will 
happily write code lots of code like this without noticing.  My main 
objection here is that the dataclass does not go far enough to prevent 
obviously wrong behaviour.  Or it goes too far with the whole hash/
frozen distinction.

                             Elvis



From ncoghlan at gmail.com  Fri Feb  2 00:33:04 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 2 Feb 2018 15:33:04 +1000
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <1937305.WBmin8xbao@klinga.prans.org>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <34493569.S535hZLbfa@klinga.prans.org>
 <0a81521e-74f2-4883-6ab0-96e6fb03a419@trueblade.com>
 <1937305.WBmin8xbao@klinga.prans.org>
Message-ID: <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>

On 2 February 2018 at 11:49, Elvis Pranskevichus <elprans at gmail.com> wrote:
> In my experience this type of breakage is so subtle that people will
> happily write code lots of code like this without noticing.  My main
> objection here is that the dataclass does not go far enough to prevent
> obviously wrong behaviour.  Or it goes too far with the whole hash/
> frozen distinction.

For  3.7, I think we should seriously considered just straight up
disallowing the "hash=True, frozen=False" combination, and instead
require folks to provide their own hash function in that case.
"Accidentally hashable" (whether by identity or field hash) isn't a
thing that data classes should be allowing to happen.

If we did that, then the public "hash" parameter could potentially be
dropped entirely for the time being - the replacement for "hash=True"
would be a "def __hash__: ..." in the body of the class definition,
and the replacement for "hash=False" would be "__hash__ = None" in the
class body.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From tjreedy at udel.edu  Fri Feb  2 01:53:00 2018
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 2 Feb 2018 01:53:00 -0500
Subject: [Python-Dev] Is object the most base type? (bpo-20285)
Message-ID: <p511og$tf7$1@blaine.gmane.org>

 >>> object.__doc__
'The most base type'

I and several people on python-list thread "interactive help on the base 
object" (Dec   2013) thought this could be improved.  On
https://bugs.python.org/issue20285 and
https://github.com/python/cpython/pull/4759

After some research, I believe the following, which is wrote on the 
issue, explains the uneasiness many feel.

'Base' is actually two words.  As a noun (or verb), it comes from 
Ancient Greek ????? (b?sis), a foundation from which other things extend 
or derive.  As an adjective, it comes from Late Latin bassus (?low?).

In computer science and Python, the couplet 'base class' is being used, 
it seems to me and apparently others, as a noun-noun compound, meaning, 
'foundation class', not as an adjective-noun phrase meaning 'low class' 
(let along 'depraved class').  However, 'most base class' must be parsed 
as '(most base) class', with 'base' re-interpreted as the adjective 
meaning 'low' (or worse).  The switch in meaning of 'base' is similar in 
'baseball' versus  'most base ball'.
---

I have suggested
"The superclass for all Python classes."
"The starting base class of all types and classes other than itself."

I intended to pick the second, but Serhiy Storchake wants more opinions.

-- 
Terry Jan Reedy



From steve at pearwood.info  Fri Feb  2 02:25:34 2018
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 2 Feb 2018 18:25:34 +1100
Subject: [Python-Dev] Is object the most base type? (bpo-20285)
In-Reply-To: <p511og$tf7$1@blaine.gmane.org>
References: <p511og$tf7$1@blaine.gmane.org>
Message-ID: <20180202072534.GO26553@ando.pearwood.info>

On Fri, Feb 02, 2018 at 01:53:00AM -0500, Terry Reedy wrote:
> >>> object.__doc__
> 'The most base type'
[...]
> I have suggested
> "The superclass for all Python classes."
> "The starting base class of all types and classes other than itself."
> 
> I intended to pick the second, but Serhiy Storchake wants more opinions.

Yay, bike-shedding!

How about:

"the base class of the class heirarchy"

"the root of the class heirarchy"


Java used to say this about Object:

    "The root class of the Java class hierarchy. All non-primitive types 
    (including arrays) inherit either directly or indirectly from this
    class."

but now says:

    "Class Object is the root of the class hierarchy. Every class has 
    Object as a superclass. All objects, including arrays, implement the 
    methods of this class."

https://developer.android.com/reference/java/lang/Object.html


Ruby says this about Object and BasicObject:

    "Object is the default root of all Ruby objects. Object inherits 
    from BasicObject which allows creating alternate object 
    hierarchies."

    "BasicObject is the parent class of all classes in Ruby. It's an 
    explicit blank class."

https://ruby-doc.org/core-2.5.0/Object.html
https://ruby-doc.org/core-2.5.0/BasicObject.html


The root of Julia's object heirarchy is Any, but if it is documented 
anywhere, I can't find it. The interactive help just says:

    Any
        No documentation found.
    Summary:
        abstract Any <: Any




-- 
Steve

From vadmium+py at gmail.com  Fri Feb  2 06:23:13 2018
From: vadmium+py at gmail.com (Martin Panter)
Date: Fri, 2 Feb 2018 11:23:13 +0000
Subject: [Python-Dev] Is object the most base type? (bpo-20285)
In-Reply-To: <20180202072534.GO26553@ando.pearwood.info>
References: <p511og$tf7$1@blaine.gmane.org>
 <20180202072534.GO26553@ando.pearwood.info>
Message-ID: <CA+eR4cFKUjG_gMKeagNHv9jP4nA3tea-Tj_zOAf32GwRuwASHg@mail.gmail.com>

> On Fri, Feb 02, 2018 at 01:53:00AM -0500, Terry Reedy wrote:
>> >>> object.__doc__
>> 'The most base type'
> [...]
>> I have suggested
>> "The superclass for all Python classes."
>> "The starting base class of all types and classes other than itself."
>>
>> I intended to pick the second, but Serhiy Storchake wants more opinions.

On 2 February 2018 at 07:25, Steven D'Aprano <steve at pearwood.info> wrote:
> "the base class of the class heirarchy"
>
> "the root of the class heirarchy"

The RST documentation
<https://docs.python.org/release/3.6.4/library/functions.html#object>
says

Object is a base for all classes.

which is better than ?the most base type? IMO, although I would write
_the_ base rather than _a_ base; there is only one object class. In my
patch for the RST documentation in
<https://bugs.python.org/issue16508> I proposed

This is the ultimate base class of all other classes.

which I still think is okay (though other parts of my patch doesn?t
seem very good).

From elprans at gmail.com  Fri Feb  2 08:55:25 2018
From: elprans at gmail.com (Elvis Pranskevichus)
Date: Fri, 02 Feb 2018 08:55:25 -0500
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <1937305.WBmin8xbao@klinga.prans.org>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
Message-ID: <2657249.hN7IFKUuXt@klinga.prans.org>

On Friday, February 2, 2018 12:33:04 AM EST Nick Coghlan wrote:
> For  3.7, I think we should seriously considered just straight up
> disallowing the "hash=True, frozen=False" combination, and instead
> require folks to provide their own hash function in that case.
> "Accidentally hashable" (whether by identity or field hash) isn't a
> thing that data classes should be allowing to happen.
> 
> If we did that, then the public "hash" parameter could potentially
> be dropped entirely for the time being - the replacement for
> "hash=True" would be a "def __hash__: ..." in the body of the class
> definition, and the replacement for "hash=False" would be "__hash__
> = None" in the class body.

I think "frozen=True" should just imply hashability (by fields).  You 
can always do "__hash__ = None" to opt out.  I don't see the default 
hashability of an immutable object as a problem.  tuples and 
frozensets are hashable after all.

                               Elvis



From eric at trueblade.com  Fri Feb  2 10:08:43 2018
From: eric at trueblade.com (Eric V. Smith)
Date: Fri, 2 Feb 2018 10:08:43 -0500
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <34493569.S535hZLbfa@klinga.prans.org>
 <0a81521e-74f2-4883-6ab0-96e6fb03a419@trueblade.com>
 <1937305.WBmin8xbao@klinga.prans.org>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
Message-ID: <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>

On 2/2/2018 12:33 AM, Nick Coghlan wrote:
> For  3.7, I think we should seriously considered just straight up
> disallowing the "hash=True, frozen=False" combination, and instead
> require folks to provide their own hash function in that case.
> "Accidentally hashable" (whether by identity or field hash) isn't a
> thing that data classes should be allowing to happen.
> 
> If we did that, then the public "hash" parameter could potentially be
> dropped entirely for the time being - the replacement for "hash=True"
> would be a "def __hash__: ..." in the body of the class definition,
> and the replacement for "hash=False" would be "__hash__ = None" in the
> class body.

attrs has the same behavior (if you ignore how dataclasses handles the 
cases where __hash__ or __eq__ already exist in the class definition). 
Here's what attrs says about adding __hash__ via hash=True:

"Although not recommended, you can decide for yourself and force attrs 
to create one (e.g. if the class is immutable even though you didn?t 
freeze it programmatically) by passing True or not. Both of these cases 
are rather special and should be used carefully."

The problem with dropping hash=True is: how would you write __hash__ 
yourself? It seems like a bug magnet if you're adding fields to the 
class and forget to update __hash__, especially in the presence of 
per-field hash=False and eq=False settings. And you'd need to make sure 
it matches the generated __eq__ (if 2 objects are equal, they need to 
have the same hash value).

If we're going to start disallowing things, how about the per-field 
hash=True, eq=False case?

However, I don't feel very strongly about this. As I've said, I expect 
the use cases for hash=True to be very, very rare. And now that we allow 
overriding __hash__ in the class body without setting hash=False, there 
aren't a lot of uses for hash=False, either. But we would need to think 
through how you'd get the behavior of hash=False with multiple 
inheritance, if that's what you wanted. Again, a very, very rare case.

In all, I think we're better off documenting best practices and making 
them the default, like attrs does, and leave it to the programmer to 
follow them. I realize we're handing out footguns, the alternatives seem 
even more complex and are limiting.

Eric

From christian at python.org  Fri Feb  2 10:23:20 2018
From: christian at python.org (Christian Heimes)
Date: Fri, 2 Feb 2018 16:23:20 +0100
Subject: [Python-Dev] Deprecate crypt module and revert PR 3854
Message-ID: <11437ad6-450f-58a2-94db-b8fe8619157e@python.org>

Hi,

in PR 3854 [1] Serhiy added blowfish, extended DES and NT-Hash to
Python's crypt mdodule. I vetoed against addition of the APIs because
all these hashing algorithms are not state of the art. Their quality
ranges from old to horribly, horriblye broken beyond any repair.

Shortly after the PR has landed, I was made aware that glibc has
deprecated crypt(3) API [2] and favor of an external library called
libxcrypt [3] from OpenWall Linux. I have patched Python 3.7 [4] to
support libxcrypt.

In light of deprecation of crypt(3) glibc function and bad quality of
hashing algorithms, I'd like to raise the motion to revert 3854 and
deprecate the crypt module. The whole module should be rather moved into
3rd party library that wraps xcrypt.

Regards,
Christian

[1] https://github.com/python/cpython/pull/3854
[2]
https://www.phoronix.com/scan.php?page=news_item&px=Fedora-28-libxcrypt-Plans
[3] https://github.com/besser82/libxcrypt
[4] https://bugs.python.org/issue32635

From barry at python.org  Fri Feb  2 10:24:47 2018
From: barry at python.org (Barry Warsaw)
Date: Fri, 2 Feb 2018 10:24:47 -0500
Subject: [Python-Dev] Is object the most base type? (bpo-20285)
In-Reply-To: <20180202072534.GO26553@ando.pearwood.info>
References: <p511og$tf7$1@blaine.gmane.org>
 <20180202072534.GO26553@ando.pearwood.info>
Message-ID: <E8A32107-2975-4F79-973A-4E378B800410@python.org>

On Feb 2, 2018, at 02:25, Steven D'Aprano <steve at pearwood.info> wrote:
> 
> On Fri, Feb 02, 2018 at 01:53:00AM -0500, Terry Reedy wrote:
>>>>> object.__doc__
>> 'The most base type?

Clearly that?s a typo.  It should be: ?The most bass type? as in: "It all starts with the bass, the most important part of any band or class hierarchy.  Sure, the drums type is very important, and the bass type and drums type work hand-in-hand to provide you with a solid funky foundation on which to groove, er, code.  Don?t let those guitar types tell you anything else."

> "the base class of the class heirarchy"
> 
> "the root of the class heirarchy"

In all seriousness, either one of those works for me.  While the first one does repeat the word ?class?, it seems the most clear description of what this thing is.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180202/68b01b03/attachment.sig>

From elprans at gmail.com  Fri Feb  2 10:38:26 2018
From: elprans at gmail.com (Elvis Pranskevichus)
Date: Fri, 02 Feb 2018 10:38:26 -0500
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
Message-ID: <1906618.bnMMLoM6on@hammer.magicstack.net>

On Friday, February 2, 2018 10:08:43 AM EST Eric V. Smith wrote:
> However, I don't feel very strongly about this. As I've said, I expect
> the use cases for hash=True to be very, very rare. 

Why do you think that the requirement to make a dataclass hashable is a 
"very, very rare" requirement?  The moment you want to use a dataclass a 
a dict key, or put it in a set, you need it to be hashable.  

Just put yourself in the shoes of an average Python developer.  You try 
to put a dataclass in a set, you get a TypeError.  Your immediate 
reaction is to add "hash=True".  Things appear to work.  Then, you, or 
someone else, decides to mutate the dataclass object and then you are 
looking at a very frustrating debug session.

> In all, I think we're better off documenting best practices and making 
> them the default, like attrs does, and leave it to the programmer to 
> follow them. I realize we're handing out footguns

I don't think attrs doing the same thing is a valid justification.  This 
is a major footgun that is very easy to trigger, and there's really no 
precedent in standard data types.

> the alternatives seem even more complex and are limiting.

The alternative is simple and follows the design of other standard 
containers: immutable containers are hashable, mutable containers are 
not.  @dataclass(frozen=False) gives you a SimpleNamespace-like and 
@dataclass(frozen=True) gives you a namedtuple-like.  If you _really_ 
know what you are doing, then you can always declare an explicit 
__hash__.  

> The problem with dropping hash=True is: how would you write __hash__ 
> yourself?

Is "def __hash__(self): return hash((self.field1, self.field2))" that 
hard?  It is explicit, and you made a concious choice, i.e you 
understand how __hash__ works.  IMO, the danger of 
"@dataclass(hash=True)" far overweighs whatever convenience it might 
provide.

                               Elvis

From eric at trueblade.com  Fri Feb  2 10:51:11 2018
From: eric at trueblade.com (Eric V. Smith)
Date: Fri, 2 Feb 2018 10:51:11 -0500
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <1906618.bnMMLoM6on@hammer.magicstack.net>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
Message-ID: <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>

On 2/2/2018 10:38 AM, Elvis Pranskevichus wrote:
> On Friday, February 2, 2018 10:08:43 AM EST Eric V. Smith wrote:
>> However, I don't feel very strongly about this. As I've said, I expect
>> the use cases for hash=True to be very, very rare.
> 
> Why do you think that the requirement to make a dataclass hashable is a
> "very, very rare" requirement?  The moment you want to use a dataclass a
> a dict key, or put it in a set, you need it to be hashable.

I was specifically talking about the case of a non-frozen, hashable 
class. If you want to make a class frozen and hashable, then:

@dataclass(frozen=True)

will do just that.

The case you brought up initially is the non-frozen, hashable class. 
It's that case that I think is very rare. I'll ask again: what's your 
use case for wanting a non-frozen, hashable class? I'm genuinely interested.

You seem to think that hash=True means "make the class hashable". That's 
not true. It means "add a __hash__" to this class". There are other, 
better ways to make the class hashable, specifically frozen=True. You 
might want to read all of https://bugs.python.org/issue32513 for the 
background on the current behavior.

> Just put yourself in the shoes of an average Python developer.  You try
> to put a dataclass in a set, you get a TypeError.  Your immediate
> reaction is to add "hash=True".  Things appear to work.  Then, you, or
> someone else, decides to mutate the dataclass object and then you are
> looking at a very frustrating debug session.

I will be documented that the correct way to do this is frozen=True.

>> In all, I think we're better off documenting best practices and making
>> them the default, like attrs does, and leave it to the programmer to
>> follow them. I realize we're handing out footguns
> 
> I don't think attrs doing the same thing is a valid justification.  This
> is a major footgun that is very easy to trigger, and there's really no
> precedent in standard data types.
> 
>> the alternatives seem even more complex and are limiting.
> 
> The alternative is simple and follows the design of other standard
> containers: immutable containers are hashable, mutable containers are
> not.  @dataclass(frozen=False) gives you a SimpleNamespace-like and
> @dataclass(frozen=True) gives you a namedtuple-like.  If you _really_
> know what you are doing, then you can always declare an explicit
> __hash__.

I'm not sure what you're arguing for here. This is how dataclasses work.

>> The problem with dropping hash=True is: how would you write __hash__
>> yourself?
> 
> Is "def __hash__(self): return hash((self.field1, self.field2))" that
> hard?  It is explicit, and you made a concious choice, i.e you
> understand how __hash__ works.

I didn't say it was hard, I said it needed to be kept up to date as you 
add fields. That is, you'd have to duplicate the field list. dataclasses 
is trying to prevent you from repeating the field list anywhere.

> IMO, the danger of
> "@dataclass(hash=True)" far overweighs whatever convenience it might
> provide.

We'll just have to disagree about this. As I said, I don't feel very 
strongly about it, but I lean toward leaving it in and documenting it 
for what it is and does.

Eric

From p.f.moore at gmail.com  Fri Feb  2 10:51:37 2018
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 2 Feb 2018 15:51:37 +0000
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <1906618.bnMMLoM6on@hammer.magicstack.net>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
Message-ID: <CACac1F_-T2xCT3iF_d+3vCncem+ZRkXTziGV0YMP7JfZqbC9hA@mail.gmail.com>

On 2 February 2018 at 15:38, Elvis Pranskevichus <elprans at gmail.com> wrote:
> On Friday, February 2, 2018 10:08:43 AM EST Eric V. Smith wrote:
>> However, I don't feel very strongly about this. As I've said, I expect
>> the use cases for hash=True to be very, very rare.
>
> Why do you think that the requirement to make a dataclass hashable is a
> "very, very rare" requirement?  The moment you want to use a dataclass a
> a dict key, or put it in a set, you need it to be hashable.
>
> Just put yourself in the shoes of an average Python developer.  You try
> to put a dataclass in a set, you get a TypeError.  Your immediate
> reaction is to add "hash=True".  Things appear to work.  Then, you, or
> someone else, decides to mutate the dataclass object and then you are
> looking at a very frustrating debug session.

If I saw someone try to put a dataclass into a set, I'd point out that
dataclasses are *mutable*, and if they want immutable values they
should use "frozen=True". If it were me in that situation, that's what
I'd do as well. Adding hashability to a mutable object would *never*
be my immediate reaction.

To put it another way, using your words above, "The moment you want to
use a dataclass a a dict key, or put it in a set, you need it to be
*immutable*" (not hashable, unless you really know what you're doing).

Paul

From elprans at gmail.com  Fri Feb  2 10:56:46 2018
From: elprans at gmail.com (Elvis Pranskevichus)
Date: Fri, 02 Feb 2018 10:56:46 -0500
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
Message-ID: <3259286.YNdnAkMY4U@hammer.magicstack.net>

On Friday, February 2, 2018 10:51:11 AM EST Eric V. Smith wrote:
> I was specifically talking about the case of a non-frozen, hashable
> class. If you want to make a class frozen and hashable, then:
> 
> @dataclass(frozen=True)
> 
> will do just that.
> 
> The case you brought up initially is the non-frozen, hashable class.
> It's that case that I think is very rare. I'll ask again: what's your
> use case for wanting a non-frozen, hashable class? I'm genuinely
> interested.

My point is exactly that there is _no_ valid use case, so (hash=True, 
frozen=False) should not be a thing!  Why are you so insistent on adding 
a dangerous option which you admit is nearly useless?

                               Elvis

From yselivanov.ml at gmail.com  Fri Feb  2 11:14:54 2018
From: yselivanov.ml at gmail.com (Yury Selivanov)
Date: Fri, 2 Feb 2018 11:14:54 -0500
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CACac1F_-T2xCT3iF_d+3vCncem+ZRkXTziGV0YMP7JfZqbC9hA@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <CACac1F_-T2xCT3iF_d+3vCncem+ZRkXTziGV0YMP7JfZqbC9hA@mail.gmail.com>
Message-ID: <CA+St6D0n3pyB90hFQtHtkr7MXXT4fNA9phKuahwMaaEcammjiQ@mail.gmail.com>

On Fri, Feb 2, 2018 at 10:51 AM, Paul Moore <p.f.moore at gmail.com> wrote:
[..]
> To put it another way, using your words above, "The moment you want to
> use a dataclass a a dict key, or put it in a set, you need it to be
> *immutable*" (not hashable, unless you really know what you're doing).

Can someone clarify what is the actual use case of someone *knowingly*
making a mutable collection hashable?  Why can't that advanced user
write their own __hash__ implementation?  It's easy to do so.

For what it's worth I think this argument is being blindly used to
justify the current questionable design of "dataclass(hash=True)"
being the same as "dataclass(hash=True, frozen=False) case.  At least
a few other core developers are concerned with this, but all I see is
"attrs does the same thing".

Eric, in my opinion we shouldn't copy attrs.  It was designed as an
external package with its own backwards-compatibility story.  At some
point it was realized that "attrs(hash=True, frozen=False)" is an
anti-pattern, but it couldn't be removed at that point.  Hence the
warning in the documentation.  We can do better.

We are designing a new API that is going to be hugely popular.  Why
can't we ship it with dangerous options prohibited in 3.7 (it's easy
to do that!) and then enable them in 3.8 when there's an actual clear
use case?

Yury

From guido at python.org  Fri Feb  2 11:18:06 2018
From: guido at python.org (Guido van Rossum)
Date: Fri, 2 Feb 2018 08:18:06 -0800
Subject: [Python-Dev] Deprecate crypt module and revert PR 3854
In-Reply-To: <11437ad6-450f-58a2-94db-b8fe8619157e@python.org>
References: <11437ad6-450f-58a2-94db-b8fe8619157e@python.org>
Message-ID: <CAP7+vJLPiDkpq_mZA5L-8LK0U1D8peTX5B8GLHoaVsejFeTLxw@mail.gmail.com>

I'm all for nudging people in the direction of xcrypt. I assume we can't
just switch the C-level crypt with xcrypt and leave the Python API
unchanged?

However until a usable solution exist (either in the stdlib or as 3rd
party) I don't think we should deprecate anything (deprecating things
before the replacement is ready is stressful for everyone involved).

I'm also not sure I agree with removing support for old hashes. By all
means put in the docs that they are unsafe. But if someone has a database
full of old hashes it would be nice to be able to at least read/verify it,
right?

Was a release already made with blowfish, extended DES and NT-Hash? (And
what's so bad with blowfish? It's mentioned in the heading of the xcrypt
project too.)

On Fri, Feb 2, 2018 at 7:23 AM, Christian Heimes <christian at python.org>
wrote:

> Hi,
>
> in PR 3854 [1] Serhiy added blowfish, extended DES and NT-Hash to
> Python's crypt mdodule. I vetoed against addition of the APIs because
> all these hashing algorithms are not state of the art. Their quality
> ranges from old to horribly, horriblye broken beyond any repair.
>
> Shortly after the PR has landed, I was made aware that glibc has
> deprecated crypt(3) API [2] and favor of an external library called
> libxcrypt [3] from OpenWall Linux. I have patched Python 3.7 [4] to
> support libxcrypt.
>
> In light of deprecation of crypt(3) glibc function and bad quality of
> hashing algorithms, I'd like to raise the motion to revert 3854 and
> deprecate the crypt module. The whole module should be rather moved into
> 3rd party library that wraps xcrypt.
>
> Regards,
> Christian
>
> [1] https://github.com/python/cpython/pull/3854
> [2]
> https://www.phoronix.com/scan.php?page=news_item&px=Fedora-
> 28-libxcrypt-Plans
> [3] https://github.com/besser82/libxcrypt
> [4] https://bugs.python.org/issue32635
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180202/26055c07/attachment.html>

From elprans at gmail.com  Fri Feb  2 11:48:04 2018
From: elprans at gmail.com (Elvis Pranskevichus)
Date: Fri, 02 Feb 2018 11:48:04 -0500
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <f5ef0f33-d7d6-bed1-a754-516b8134ebc4@trueblade.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <3259286.YNdnAkMY4U@hammer.magicstack.net>
 <f5ef0f33-d7d6-bed1-a754-516b8134ebc4@trueblade.com>
Message-ID: <4737330.CapxtRJ2dz@hammer.magicstack.net>

> Because it's not the default, it will be documented as being an
> advanced use case, and it's useful in rare instances.
> 
> And as I've said a number of times, both here and in other
> discussions, I'm not arguing strenuously for this. I just think that,
> given that it's not the default and it's not recommended and is
> useful in advanced cases, I would prefer to leave it in. I understand
> that you disagree with me.

Is there a real world example of such an "advanced case"?  

Eric, have you read https://github.com/python-attrs/attrs/issues/136 ?  
Specifically this comment from Hynek [1]: 

"I never really thought about it, but yeah mutable objects shouldn?t 
have a __hash__ at all." 

It is clear from that thread that "hash=True" was an early design 
mistake, which was left in for compatibility reasons.  Why are we 
copying bad design to the standard library?

                              Elvis


[1] https://github.com/python-attrs/attrs/issues/
136#issuecomment-277425421

From eric at trueblade.com  Fri Feb  2 11:09:26 2018
From: eric at trueblade.com (Eric V. Smith)
Date: Fri, 2 Feb 2018 11:09:26 -0500
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <3259286.YNdnAkMY4U@hammer.magicstack.net>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <3259286.YNdnAkMY4U@hammer.magicstack.net>
Message-ID: <f5ef0f33-d7d6-bed1-a754-516b8134ebc4@trueblade.com>

On 2/2/2018 10:56 AM, Elvis Pranskevichus wrote:
> On Friday, February 2, 2018 10:51:11 AM EST Eric V. Smith wrote:
>> I was specifically talking about the case of a non-frozen, hashable
>> class. If you want to make a class frozen and hashable, then:
>>
>> @dataclass(frozen=True)
>>
>> will do just that.
>>
>> The case you brought up initially is the non-frozen, hashable class.
>> It's that case that I think is very rare. I'll ask again: what's your
>> use case for wanting a non-frozen, hashable class? I'm genuinely
>> interested.
> 
> My point is exactly that there is _no_ valid use case, so (hash=True,
> frozen=False) should not be a thing!  Why are you so insistent on adding
> a dangerous option which you admit is nearly useless?

Because it's not the default, it will be documented as being an advanced 
use case, and it's useful in rare instances.

And as I've said a number of times, both here and in other discussions, 
I'm not arguing strenuously for this. I just think that, given that it's 
not the default and it's not recommended and is useful in advanced 
cases, I would prefer to leave it in. I understand that you disagree 
with me.

Eric

From storchaka at gmail.com  Fri Feb  2 12:05:43 2018
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Fri, 2 Feb 2018 19:05:43 +0200
Subject: [Python-Dev] Deprecate crypt module and revert PR 3854
In-Reply-To: <CAP7+vJLPiDkpq_mZA5L-8LK0U1D8peTX5B8GLHoaVsejFeTLxw@mail.gmail.com>
References: <11437ad6-450f-58a2-94db-b8fe8619157e@python.org>
 <CAP7+vJLPiDkpq_mZA5L-8LK0U1D8peTX5B8GLHoaVsejFeTLxw@mail.gmail.com>
Message-ID: <p525la$e9$1@blaine.gmane.org>

02.02.18 18:18, Guido van Rossum ????:
> I'm all for nudging people in the direction of xcrypt. I assume we can't 
> just switch the C-level crypt with xcrypt and leave the Python API 
> unchanged?
> 
> However until a usable solution exist (either in the stdlib or as 3rd 
> party) I don't think we should deprecate anything (deprecating things 
> before the replacement is ready is stressful for everyone involved).
> 
> I'm also not sure I agree with removing support for old hashes. By all 
> means put in the docs that they are unsafe. But if someone has a 
> database full of old hashes it would be nice to be able to at least 
> read/verify it, right?
> 
> Was a release already made with blowfish, extended DES and NT-Hash? (And 
> what's so bad with blowfish? It's mentioned in the heading of the xcrypt 
> project too.)

To clarify, extended DES and NT-Hash were not added. They were removed 
from my PR after Christians request. Only the Blowfish method was added, 
and it is so strong as SHA-2 methods. It is the only method supported on 
OpenBSD.

This PR is not a single enhancement made in the crypt module recently. I 
also extended tests and added support for configuring SHA-2 methods. 
There is an open PR (not merged before 3.7b1 unfortunately) for using 
crypt_r() instead of crypt(): https://bugs.python.org/issue28503.

If deprecate the crypt module, should modules pwd, grp and spwd be 
deprecated too? The crypt module is needed for checking password hashes 
provided by spwd.


From status at bugs.python.org  Fri Feb  2 12:09:48 2018
From: status at bugs.python.org (Python tracker)
Date: Fri,  2 Feb 2018 18:09:48 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20180202170948.85B4D56864@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2018-01-26 - 2018-02-02)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    6411 (-34)
  closed 38089 (+101)
  total  44500 (+67)

Open issues with patches: 2492 


Issues opened (35)
==================

#32221: Converting ipv6 address to string representation using getname
https://bugs.python.org/issue32221  reopened by yselivanov

#32622: Implement loop.sendfile
https://bugs.python.org/issue32622  reopened by vstinner

#32683: isinstance is calling ob.__getattribute__ as a fallback instea
https://bugs.python.org/issue32683  opened by bup

#32684: asyncio.gather(..., return_exceptions=True) swallows cancellat
https://bugs.python.org/issue32684  opened by socketpair

#32689: shutil.move raises AttributeError if first argument is a pathl
https://bugs.python.org/issue32689  opened by craigh

#32691: "pdb -m <module>" sets __main__.__package__ incorrectly
https://bugs.python.org/issue32691  opened by ncoghlan

#32692: test_threading.test_set_and_clear fails in AppVeyor CI
https://bugs.python.org/issue32692  opened by xiang.zhang

#32694: Can no longer specify OpenSSL locations with CPPFLAGS / LDFLAG
https://bugs.python.org/issue32694  opened by yselivanov

#32695: tarfile.open() raises TypeError when using compresslevel param
https://bugs.python.org/issue32695  opened by bbayles

#32696: Fix pickling exceptions with multiple arguments
https://bugs.python.org/issue32696  opened by slallum

#32706: test_check_hostname() of test_ftplib started to fail randomly
https://bugs.python.org/issue32706  opened by vstinner

#32708: test_sendfile() hangs on AMD64 FreeBSD 10.x Shared 3.x buildbo
https://bugs.python.org/issue32708  opened by vstinner

#32710: test_asyncio leaked [4, 4, 3] memory blocks, sum=11 on AMD64 W
https://bugs.python.org/issue32710  opened by vstinner

#32713: tarfile.itn breaks if n is a negative float
https://bugs.python.org/issue32713  opened by j0ffrey

#32715: Make create_unix_server for SOCK_DGRAM work
https://bugs.python.org/issue32715  opened by holger+lp

#32716: setup.py register --repository is broken
https://bugs.python.org/issue32716  opened by shimizukawa

#32717: Document PEP 560
https://bugs.python.org/issue32717  opened by levkivskyi

#32718: Install PowerShell activation scripts for venv for all platfor
https://bugs.python.org/issue32718  opened by brett.cannon

#32719: fatal error raised when Ctrl-C print loop
https://bugs.python.org/issue32719  opened by xiang.zhang

#32720: Format mini-language integer definition is incorrect
https://bugs.python.org/issue32720  opened by ncoghlan

#32723: codecs.open silently ignores argument errors
https://bugs.python.org/issue32723  opened by xiang.zhang

#32725: Instance of _multiprocessing.PipeConnection-subtype crash on d
https://bugs.python.org/issue32725  opened by hakril

#32726: macOS installer and framework enhancements and changes for 3.7
https://bugs.python.org/issue32726  opened by ned.deily

#32728: Extend zipfile's compression level support  to LZMA
https://bugs.python.org/issue32728  opened by bbayles

#32729: socket error handling needed
https://bugs.python.org/issue32729  opened by rkdls

#32730: Allow py launcher to launch other registered Pythons
https://bugs.python.org/issue32730  opened by petsuter at gmail.com

#32731: getpass.getuser() raises an unspecified exceptions (ImportErro
https://bugs.python.org/issue32731  opened by gregory.p.smith

#32732: LoggingAdapter ignores extra kwargs of Logger#log()
https://bugs.python.org/issue32732  opened by mcoolive

#32734: Asyncio Lock safety issue (unlimited acquire)
https://bugs.python.org/issue32734  opened by bar.harel

#32739: collections.deque rotate(n=1) default value not documented
https://bugs.python.org/issue32739  opened by yuy

#32742: zipfile extractall needlessly re-wraps ZipInfo instances
https://bugs.python.org/issue32742  opened by peterbe

#32743: Typo in hamt.c comments
https://bugs.python.org/issue32743  opened by delimitry

#32745: ctypes string pointer fields should accept embedded null chara
https://bugs.python.org/issue32745  opened by theller

#32746: More misspellings, mostly in source code.
https://bugs.python.org/issue32746  opened by terry.reedy

#32749: Remove dbm.dumb behavior deprecated in 3.6
https://bugs.python.org/issue32749  opened by serhiy.storchaka



Most recent 15 issues with no replies (15)
==========================================

#32749: Remove dbm.dumb behavior deprecated in 3.6
https://bugs.python.org/issue32749

#32746: More misspellings, mostly in source code.
https://bugs.python.org/issue32746

#32728: Extend zipfile's compression level support  to LZMA
https://bugs.python.org/issue32728

#32718: Install PowerShell activation scripts for venv for all platfor
https://bugs.python.org/issue32718

#32717: Document PEP 560
https://bugs.python.org/issue32717

#32716: setup.py register --repository is broken
https://bugs.python.org/issue32716

#32713: tarfile.itn breaks if n is a negative float
https://bugs.python.org/issue32713

#32708: test_sendfile() hangs on AMD64 FreeBSD 10.x Shared 3.x buildbo
https://bugs.python.org/issue32708

#32696: Fix pickling exceptions with multiple arguments
https://bugs.python.org/issue32696

#32695: tarfile.open() raises TypeError when using compresslevel param
https://bugs.python.org/issue32695

#32692: test_threading.test_set_and_clear fails in AppVeyor CI
https://bugs.python.org/issue32692

#32682: test_zlib improve version parsing
https://bugs.python.org/issue32682

#32676: test_asyncio emits many warnings when run in debug mode
https://bugs.python.org/issue32676

#32671: redesign Windows  os.getlogin, and add os.getuser
https://bugs.python.org/issue32671

#32663: SMTPUTF8SimTests are not actually being run
https://bugs.python.org/issue32663



Most recent 15 issues waiting for review (15)
=============================================

#32749: Remove dbm.dumb behavior deprecated in 3.6
https://bugs.python.org/issue32749

#32746: More misspellings, mostly in source code.
https://bugs.python.org/issue32746

#32743: Typo in hamt.c comments
https://bugs.python.org/issue32743

#32742: zipfile extractall needlessly re-wraps ZipInfo instances
https://bugs.python.org/issue32742

#32739: collections.deque rotate(n=1) default value not documented
https://bugs.python.org/issue32739

#32734: Asyncio Lock safety issue (unlimited acquire)
https://bugs.python.org/issue32734

#32729: socket error handling needed
https://bugs.python.org/issue32729

#32726: macOS installer and framework enhancements and changes for 3.7
https://bugs.python.org/issue32726

#32720: Format mini-language integer definition is incorrect
https://bugs.python.org/issue32720

#32713: tarfile.itn breaks if n is a negative float
https://bugs.python.org/issue32713

#32706: test_check_hostname() of test_ftplib started to fail randomly
https://bugs.python.org/issue32706

#32695: tarfile.open() raises TypeError when using compresslevel param
https://bugs.python.org/issue32695

#32694: Can no longer specify OpenSSL locations with CPPFLAGS / LDFLAG
https://bugs.python.org/issue32694

#32691: "pdb -m <module>" sets __main__.__package__ incorrectly
https://bugs.python.org/issue32691

#32689: shutil.move raises AttributeError if first argument is a pathl
https://bugs.python.org/issue32689



Top 10 most discussed issues (10)
=================================

#32394: socket lib beahavior change in 3.6.4
https://bugs.python.org/issue32394  11 msgs

#32622: Implement loop.sendfile
https://bugs.python.org/issue32622  11 msgs

#32604: Expose the subinterpreters C-API in Python for testing use.
https://bugs.python.org/issue32604  10 msgs

#32591: Deprecate sys.set_coroutine_wrapper and replace it with more f
https://bugs.python.org/issue32591   8 msgs

#17305: IDNA2008 encoding missing
https://bugs.python.org/issue17305   7 msgs

#25988: collections.abc.Indexable
https://bugs.python.org/issue25988   7 msgs

#32694: Can no longer specify OpenSSL locations with CPPFLAGS / LDFLAG
https://bugs.python.org/issue32694   7 msgs

#26219: implement per-opcode cache in ceval
https://bugs.python.org/issue26219   6 msgs

#30693: tarfile add uses random order
https://bugs.python.org/issue30693   6 msgs

#31368: Add os.preadv() and os.pwritev()
https://bugs.python.org/issue31368   6 msgs



Issues closed (93)
==================

#14799: Tkinter ttk tests hang on linux
https://bugs.python.org/issue14799  closed by asvetlov

#16648: stdib should use new exception types from PEP 3151
https://bugs.python.org/issue16648  closed by asvetlov

#16705: Use concrete classes inherited from OSError instead of errno c
https://bugs.python.org/issue16705  closed by asvetlov

#17044: Implement PEP 422: Simple class initialisation hook
https://bugs.python.org/issue17044  closed by ncoghlan

#20984: 'Add/Remove Programs' dialog missing entries for 32-bit CPytho
https://bugs.python.org/issue20984  closed by steve.dower

#21417: Compression level for zipfile
https://bugs.python.org/issue21417  closed by gregory.p.smith

#22908: ZipExtFile in zipfile can be seekable
https://bugs.python.org/issue22908  closed by gregory.p.smith

#24325: Speedup types.coroutine()
https://bugs.python.org/issue24325  closed by yselivanov

#25942: Do not immediately SIGKILL subprocess child processes upon ^C
https://bugs.python.org/issue25942  closed by gregory.p.smith

#27505: Missing documentation for setting module __class__ attribute
https://bugs.python.org/issue27505  closed by ncoghlan

#27931: Email parse IndexError <""@wiarcom.com>
https://bugs.python.org/issue27931  closed by xiang.zhang

#28864: Add devnull file-like object
https://bugs.python.org/issue28864  closed by rhettinger

#28914: selectmodule build fails
https://bugs.python.org/issue28914  closed by vstinner

#29120: Move hash randomisation initialisation out of Python/random.c
https://bugs.python.org/issue29120  closed by ncoghlan

#29216: Space saving step for the LRU cache
https://bugs.python.org/issue29216  closed by rhettinger

#30117: test_lib2to3.test_parser.test_all_project_files() fails
https://bugs.python.org/issue30117  closed by benjamin.peterson

#30306: release arguments of contextmanager
https://bugs.python.org/issue30306  closed by ncoghlan

#30715: Test_winreg, test_dynamic_key hangs on my Win 10
https://bugs.python.org/issue30715  closed by terry.reedy

#31356: Add context manager to temporarily disable GC
https://bugs.python.org/issue31356  closed by rhettinger

#31429: TLS cipher suite compile time option for downstream
https://bugs.python.org/issue31429  closed by christian.heimes

#31853: Use super().method instead of socket.method in SSLSocket
https://bugs.python.org/issue31853  closed by christian.heimes

#31961: subprocess._execute_child doesn't accept a single PathLike arg
https://bugs.python.org/issue31961  closed by gregory.p.smith

#32045: Does json.dumps have a memory leak?
https://bugs.python.org/issue32045  closed by csabella

#32072: Issues with binary plists
https://bugs.python.org/issue32072  closed by serhiy.storchaka

#32102: Add "capture_output=True" option to subprocess.run
https://bugs.python.org/issue32102  closed by gregory.p.smith

#32137: Stack overflow in repr of deeply nested dicts
https://bugs.python.org/issue32137  closed by serhiy.storchaka

#32226: Implement PEP 560: Core support for typing module and generic 
https://bugs.python.org/issue32226  closed by levkivskyi

#32228: truncate() changes current stream position
https://bugs.python.org/issue32228  closed by pitrou

#32251: Add asyncio.BufferedProtocol
https://bugs.python.org/issue32251  closed by yselivanov

#32336: Save OrderedDict import in argparse
https://bugs.python.org/issue32336  closed by rhettinger

#32373: Add socket.getblocking() method
https://bugs.python.org/issue32373  closed by yselivanov

#32433: Provide optimized HMAC digest
https://bugs.python.org/issue32433  closed by christian.heimes

#32446: ResourceLoader.get_data() should accept a PathLike
https://bugs.python.org/issue32446  closed by barry

#32454: Add socket.close(fd) function
https://bugs.python.org/issue32454  closed by christian.heimes

#32513: dataclasses: make it easier to use user-supplied special metho
https://bugs.python.org/issue32513  closed by eric.smith

#32521: NIS module fails to build due to the removal of interfaces rel
https://bugs.python.org/issue32521  closed by christian.heimes

#32532: improve sys.settrace and sys.setprofile documentation
https://bugs.python.org/issue32532  closed by xiang.zhang

#32546: Unusual TypeError with dataclass decorator
https://bugs.python.org/issue32546  closed by eric.smith

#32550: STORE_ANNOTATION bytecode is unnecessary and can be removed.
https://bugs.python.org/issue32550  closed by rhettinger

#32565: Document the version of adding opcodes
https://bugs.python.org/issue32565  closed by serhiy.storchaka

#32571: Speed up and clean up getting optional attributes in C code
https://bugs.python.org/issue32571  closed by yselivanov

#32572: Add the ftplib option, overrides the IP address.
https://bugs.python.org/issue32572  closed by giampaolo.rodola

#32583: Crash during decoding using UTF-16/32 and custom error handler
https://bugs.python.org/issue32583  closed by xiang.zhang

#32605: Should we really hide unawaited coroutine warnings when an exc
https://bugs.python.org/issue32605  closed by yselivanov

#32611: Tkinter taskbar icon (Windows)
https://bugs.python.org/issue32611  closed by terry.reedy

#32624: Implement WriteTransport.is_protocol_paused()
https://bugs.python.org/issue32624  closed by asvetlov

#32630: Migrate decimal to use PEP 567 context variables
https://bugs.python.org/issue32630  closed by yselivanov

#32640: Python 2.7 str.join documentation is incorrect
https://bugs.python.org/issue32640  closed by Mariatta

#32645: test_asyncio: TLS tests fail on "x86 Windows7" buildbot
https://bugs.python.org/issue32645  closed by vstinner

#32649: complete C API doc debug and profile part with new PyTrace_OPC
https://bugs.python.org/issue32649  closed by ncoghlan

#32650: Debug support for native coroutines is broken
https://bugs.python.org/issue32650  closed by yselivanov

#32664: Connector "|"  missing between ImportError and LookupError
https://bugs.python.org/issue32664  closed by Mariatta

#32668: deepcopy() fails on ArgumentParser instances
https://bugs.python.org/issue32668  closed by terry.reedy

#32670: Enforce PEP 479???StopIteration and generators???in Python 3.7
https://bugs.python.org/issue32670  closed by yselivanov

#32674: minor documentation fix for '__import__'
https://bugs.python.org/issue32674  closed by Mariatta

#32675: dict.__contains__(unhashable) raises TypeError where False was
https://bugs.python.org/issue32675  closed by r.david.murray

#32677: Add.isascii() to str, bytes and bytearray
https://bugs.python.org/issue32677  closed by inada.naoki

#32678: Lazy import ast in inspect
https://bugs.python.org/issue32678  closed by inada.naoki

#32681: Fix uninitialized variable in os_dup2_impl
https://bugs.python.org/issue32681  closed by gregory.p.smith

#32685: Erroneous suggestion in print statement
https://bugs.python.org/issue32685  closed by ncoghlan

#32686: python3.6.4 build hangs on test_asyncio macOS 10.13
https://bugs.python.org/issue32686  closed by ned.deily

#32687: wrong meaning of arg in Py_tracefunc doc
https://bugs.python.org/issue32687  closed by xiang.zhang

#32688: weird comment out line of code
https://bugs.python.org/issue32688  closed by xiang.zhang

#32690: Return function locals() in order of creation?
https://bugs.python.org/issue32690  closed by rhettinger

#32693: os.path.ismount does not accept bytes objects
https://bugs.python.org/issue32693  closed by eryksun

#32697: Guarantee that Python preserves the order of kwonly parameters
https://bugs.python.org/issue32697  closed by larry

#32698: Improper gzip compression if output file extension is not "gz"
https://bugs.python.org/issue32698  closed by Delgan

#32699: pythonXY._pth : unclear how .pth files are handled
https://bugs.python.org/issue32699  closed by steve.dower

#32700: spam?
https://bugs.python.org/issue32700  closed by r.david.murray

#32701: Typo in quopri documentation
https://bugs.python.org/issue32701  closed by Mariatta

#32702: Minor markup typo
https://bugs.python.org/issue32702  closed by rhettinger

#32703: 'async with' somehow suppresses unawaited coroutine warnings
https://bugs.python.org/issue32703  closed by yselivanov

#32704: Tracking
https://bugs.python.org/issue32704  closed by eric.smith

#32705: Current Android does not have posix_spawn
https://bugs.python.org/issue32705  closed by gregory.p.smith

#32707: Python/hamt.c warnings
https://bugs.python.org/issue32707  closed by yselivanov

#32709: the iterable for itertools.groupby must be sorted
https://bugs.python.org/issue32709  closed by matrixise

#32711: Fix warnings for Python/ast_unparse.c
https://bugs.python.org/issue32711  closed by christian.heimes

#32712: Modifying a list/dict effects all variables sharing that addre
https://bugs.python.org/issue32712  closed by r.david.murray

#32714: remove resourceLoader related code since it's deprecated
https://bugs.python.org/issue32714  closed by jayyin11043

#32721: test_hashlib fails if _md5 is not built
https://bugs.python.org/issue32721  closed by benjamin.peterson

#32722: Mislabeling of a dict comprehension as a generator expression 
https://bugs.python.org/issue32722  closed by xiang.zhang

#32724: Fix references to commands in Doc/pdb.rst
https://bugs.python.org/issue32724  closed by asvetlov

#32727: smtplib's SMTP.send_message behaves differently with from_addr
https://bugs.python.org/issue32727  closed by r.david.murray

#32733: test_coroutines is failed when run with -We
https://bugs.python.org/issue32733  closed by njs

#32735: Typo in f-string example in docs
https://bugs.python.org/issue32735  closed by Mariatta

#32736: random.triangular yields unexpected distribution when args mix
https://bugs.python.org/issue32736  closed by rhettinger

#32737: str.format not producing strings for nan argument in accordanc
https://bugs.python.org/issue32737  closed by zach.ware

#32738: CDLL : Can't find module
https://bugs.python.org/issue32738  closed by christian.heimes

#32740: test_calendar and test_re fail with unknown locale: UTF-8 in _
https://bugs.python.org/issue32740  closed by vishalsingh

#32741: Add asyncio.TimerHandle.when() function
https://bugs.python.org/issue32741  closed by asvetlov

#32744: PEP 342 double colons typos in code
https://bugs.python.org/issue32744  closed by delimitry

#32747: remove trailing spaces in docstring
https://bugs.python.org/issue32747  closed by serhiy.storchaka

#32748: Improve _asyncio.TaskStepMethWrapper and TaskWakeupMethWrapper
https://bugs.python.org/issue32748  closed by asvetlov

From chris.barker at noaa.gov  Fri Feb  2 15:02:25 2018
From: chris.barker at noaa.gov (Chris Barker)
Date: Fri, 2 Feb 2018 12:02:25 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <1906618.bnMMLoM6on@hammer.magicstack.net>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
Message-ID: <CALGmxEJ0qyhUVywLKRQHbRZ=zuKV0RbvYO8VVWH_4GhXKUJX1g@mail.gmail.com>

On Fri, Feb 2, 2018 at 7:38 AM, Elvis Pranskevichus <elprans at gmail.com>
wrote:

> On Friday, February 2, 2018 10:08:43 AM EST Eric V. Smith wrote:
> > However, I don't feel very strongly about this. As I've said, I expect
> > the use cases for hash=True to be very, very rare.
>
> Why do you think that the requirement to make a dataclass hashable is a
> "very, very rare" requirement?


I think what's rare is wanting hashability without it being frozen.


> Just put yourself in the shoes of an average Python developer.  You try
> to put a dataclass in a set, you get a TypeError.  Your immediate
> reaction is to add "hash=True".  Things appear to work.


agreed, the easy and obvious way should be to make it frozen -- if it's
hard to make it hashable and not frozen, then that's good.

But it is nice to have the __hash__ generated more you.... so maybe a flag
for "unfrozen_hashable" -- really klunky, but if that is a rare need, then
there you go.

Or maybe:

If either hash or frozen is specified, it become both frozen and hashable.

If both are specified, then it does what the user is asking for.

-CHB
-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180202/3c338706/attachment.html>

From njs at pobox.com  Fri Feb  2 15:21:58 2018
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 2 Feb 2018 12:21:58 -0800
Subject: [Python-Dev] Deprecate crypt module and revert PR 3854
In-Reply-To: <11437ad6-450f-58a2-94db-b8fe8619157e@python.org>
References: <11437ad6-450f-58a2-94db-b8fe8619157e@python.org>
Message-ID: <CAPJVwBkuQ5o8V00-JzBWvvh45QtqiES_06=w6zQ4bNwtyiCmGw@mail.gmail.com>

On Feb 2, 2018 7:24 AM, "Christian Heimes" <christian at python.org> wrote:

Shortly after the PR has landed, I was made aware that glibc has
deprecated crypt(3) API [2] and favor of an external library called
libxcrypt [3] from OpenWall Linux. I have patched Python 3.7 [4] to
support libxcrypt.

In light of deprecation of crypt(3) glibc function and bad quality of
hashing algorithms, I'd like to raise the motion to revert 3854 and
deprecate the crypt module. The whole module should be rather moved into
3rd party library that wraps xcrypt.


If the crypt module can just as well use libxcrypt, and that's how the
distros are going to build it, then what's the advantage of deprecating it?
Won't it just continue to work indefinitely?

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180202/63e6a71e/attachment.html>

From solipsis at pitrou.net  Fri Feb  2 15:31:18 2018
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 2 Feb 2018 21:31:18 +0100
Subject: [Python-Dev] Deprecate crypt module and revert PR 3854
References: <11437ad6-450f-58a2-94db-b8fe8619157e@python.org>
Message-ID: <20180202213118.4143777d@fsol>

On Fri, 2 Feb 2018 16:23:20 +0100
Christian Heimes <christian at python.org> wrote:
> Hi,
> 
> in PR 3854 [1] Serhiy added blowfish, extended DES and NT-Hash to
> Python's crypt mdodule. I vetoed against addition of the APIs because
> all these hashing algorithms are not state of the art. Their quality
> ranges from old to horribly, horriblye broken beyond any repair.
> 
> Shortly after the PR has landed, I was made aware that glibc has
> deprecated crypt(3) API [2] and favor of an external library called
> libxcrypt [3] from OpenWall Linux. I have patched Python 3.7 [4] to
> support libxcrypt.
> 
> In light of deprecation of crypt(3) glibc function and bad quality of
> hashing algorithms, I'd like to raise the motion to revert 3854 and
> deprecate the crypt module.

Those are two separate proposals.

On the topic of reverting PR #3854, I don't see the point.  Is Blowfish
more fragile than the other algorithms?  If not, it sounds ok to add it.

On the topic of deprecating the crypt module, that doesn't sound like a
good idea right now.  People may need to generate crypt()-compatible
output for various reasons, such as being able to automate system
administration tasks.

Regards

Antoine.



From ja.py at farowl.co.uk  Fri Feb  2 15:10:44 2018
From: ja.py at farowl.co.uk (Jeff Allen)
Date: Fri, 2 Feb 2018 20:10:44 +0000
Subject: [Python-Dev] Is object the most base type? (bpo-20285)
In-Reply-To: <20180202072534.GO26553@ando.pearwood.info>
References: <p511og$tf7$1@blaine.gmane.org>
 <20180202072534.GO26553@ando.pearwood.info>
Message-ID: <e066945e-95c3-afd4-2fa4-ec1e66a706e1@farowl.co.uk>


On 02/02/2018 07:25, Steven D'Aprano wrote:
> How about:
> "the base class of the class heirarchy"
>
> "the root of the class heirarchy"
>
>
> Java ... now says:
>
>      "Class Object is the root of the class hierarchy. Every class has
>      Object as a superclass. All objects, including arrays, implement the
>      methods of this class."

Either for me, but I feel I should draw attention to the spelling. (Java 
is right.)

Ironically, the word derives from priesthood (hieratic), not from 
inheritance (heir).

Jeff Allen


From drsalists at gmail.com  Fri Feb  2 16:25:34 2018
From: drsalists at gmail.com (Dan Stromberg)
Date: Fri, 2 Feb 2018 13:25:34 -0800
Subject: [Python-Dev] Deprecate crypt module and revert PR 3854
In-Reply-To: <20180202213118.4143777d@fsol>
References: <11437ad6-450f-58a2-94db-b8fe8619157e@python.org>
 <20180202213118.4143777d@fsol>
Message-ID: <CAGGBd_rWfgaFtw4TcHA2ESX0n+T+oG=n1He31xeySQDhLCa4Gg@mail.gmail.com>

On Fri, Feb 2, 2018 at 12:31 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Fri, 2 Feb 2018 16:23:20 +0100
> Christian Heimes <christian at python.org> wrote:
>> Hi,
>>
>> in PR 3854 [1] Serhiy added blowfish, extended DES and NT-Hash to
>> Python's crypt mdodule. I vetoed against addition of the APIs because
>> all these hashing algorithms are not state of the art. Their quality
>> ranges from old to horribly, horriblye broken beyond any repair.
>>
>> Shortly after the PR has landed, I was made aware that glibc has
>> deprecated crypt(3) API [2] and favor of an external library called
>> libxcrypt [3] from OpenWall Linux. I have patched Python 3.7 [4] to
>> support libxcrypt.
>>
>> In light of deprecation of crypt(3) glibc function and bad quality of
>> hashing algorithms, I'd like to raise the motion to revert 3854 and
>> deprecate the crypt module.
>
> Those are two separate proposals.
>
> On the topic of reverting PR #3854, I don't see the point.  Is Blowfish
> more fragile than the other algorithms?  If not, it sounds ok to add it.

I'm no cryptographer, but I believe Blowfish's author recommends using
Twofish instead now.

> On the topic of deprecating the crypt module, that doesn't sound like a
> good idea right now.  People may need to generate crypt()-compatible
> output for various reasons, such as being able to automate system
> administration tasks.

Encryption algorithms continue to be needed even after the time at
which they should no longer be used in new code.  But they probably
should be documented as deprecated and warned about at runtime - after
there are suitable alternatives.

From ethan at stoneleaf.us  Fri Feb  2 17:08:31 2018
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 02 Feb 2018 14:08:31 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CA+St6D0n3pyB90hFQtHtkr7MXXT4fNA9phKuahwMaaEcammjiQ@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <CACac1F_-T2xCT3iF_d+3vCncem+ZRkXTziGV0YMP7JfZqbC9hA@mail.gmail.com>
 <CA+St6D0n3pyB90hFQtHtkr7MXXT4fNA9phKuahwMaaEcammjiQ@mail.gmail.com>
Message-ID: <5A74E15F.9060809@stoneleaf.us>

On 02/02/2018 08:14 AM, Yury Selivanov wrote:

> Eric, in my opinion we shouldn't copy attrs.  [...]

> We are designing a new API that is going to be hugely popular.  Why
> can't we ship it with dangerous options prohibited in 3.7 (it's easy
> to do that!) and then enable them in 3.8 when there's an actual clear
> use case?

+1

--
~Ethan~


From ethan at stoneleaf.us  Fri Feb  2 17:11:24 2018
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 02 Feb 2018 14:11:24 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <f5ef0f33-d7d6-bed1-a754-516b8134ebc4@trueblade.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <3259286.YNdnAkMY4U@hammer.magicstack.net>
 <f5ef0f33-d7d6-bed1-a754-516b8134ebc4@trueblade.com>
Message-ID: <5A74E20C.6060204@stoneleaf.us>

On 02/02/2018 08:09 AM, Eric V. Smith wrote:
> On 2/2/2018 10:56 AM, Elvis Pranskevichus wrote:

>> My point is exactly that there is _no_ valid use case, so (hash=True,
>> frozen=False) should not be a thing!  Why are you so insistent on adding
>> a dangerous option which you admit is nearly useless?
>
> Because it's not the default, it will be documented as being an advanced use case, and it's useful in rare instances.

Personally, I don't think advanced use-cases need to be supported by flags as they can be supported by just writing the 
__dunder__ methods.

--
~Ethan~

From mertz at gnosis.cx  Fri Feb  2 19:04:25 2018
From: mertz at gnosis.cx (David Mertz)
Date: Fri, 2 Feb 2018 16:04:25 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <5A74E20C.6060204@stoneleaf.us>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <3259286.YNdnAkMY4U@hammer.magicstack.net>
 <f5ef0f33-d7d6-bed1-a754-516b8134ebc4@trueblade.com>
 <5A74E20C.6060204@stoneleaf.us>
Message-ID: <CAEbHw4Z24iUuRnDR1VmsmCtJ=RuSjLFcLGDGiRbLhCUgHhLZHw@mail.gmail.com>

I agree with Ethan, Elvis, and a few others. I think 'hash=True,
frozen=False' should be disabled in 3.7.  It's an attractive nuisance.
Maybe not so attractive because its obscurity, but still with no clear
reason to exist.

If many users of of dataclass find themselves defining '__hash__' with
mutable dataclass, it's perfectly possible to allow the switch combination
later. But taking it out after previously allowing it?even if every use in
the wild is actually a bug in waiting?is harder.

On Feb 2, 2018 2:10 PM, "Ethan Furman" <ethan at stoneleaf.us> wrote:

> On 02/02/2018 08:09 AM, Eric V. Smith wrote:
>
>> On 2/2/2018 10:56 AM, Elvis Pranskevichus wrote:
>>
>
> My point is exactly that there is _no_ valid use case, so (hash=True,
>>> frozen=False) should not be a thing!  Why are you so insistent on adding
>>> a dangerous option which you admit is nearly useless?
>>>
>>
>> Because it's not the default, it will be documented as being an advanced
>> use case, and it's useful in rare instances.
>>
>
> Personally, I don't think advanced use-cases need to be supported by flags
> as they can be supported by just writing the __dunder__ methods.
>
> --
> ~Ethan~
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/mertz%
> 40gnosis.cx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180202/40ad8923/attachment.html>

From tjreedy at udel.edu  Fri Feb  2 19:28:24 2018
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 2 Feb 2018 19:28:24 -0500
Subject: [Python-Dev] Is object the most base type? (bpo-20285)
In-Reply-To: <p511og$tf7$1@blaine.gmane.org>
References: <p511og$tf7$1@blaine.gmane.org>
Message-ID: <p52vjc$3em$1@blaine.gmane.org>

On 2/2/2018 1:53 AM, Terry Reedy wrote:
>  >>> object.__doc__
> 'The most base type'
> 
> I and several people on python-list thread "interactive help on the base 
> object" (Dec?? 2013) thought this could be improved.? On
> https://bugs.python.org/issue20285 and
> https://github.com/python/cpython/pull/4759
> 
> After some research, I believe the following, which is wrote on the 
> issue, explains the uneasiness many feel.
> 
> 'Base' is actually two words.? As a noun (or verb), it comes from 
> Ancient Greek ????? (b?sis), a foundation from which other things extend 
> or derive.? As an adjective, it comes from Late Latin bassus (?low?).
> 
> In computer science and Python, the couplet 'base class' is being used, 
> it seems to me and apparently others, as a noun-noun compound, meaning, 
> 'foundation class', not as an adjective-noun phrase meaning 'low class' 
> (let along 'depraved class').? However, 'most base class' must be parsed 
> as '(most base) class', with 'base' re-interpreted as the adjective 
> meaning 'low' (or worse).? The switch in meaning of 'base' is similar in 
> 'baseball' versus? 'most base ball'.
> ---
> 
> I have suggested
> "The superclass for all Python classes."
> "The starting base class of all types and classes other than itself."

"a base for all classes." object entry in lib ref, function
"the ultimate base class of all other classes."  Martin Panter

object is neither a base or superclass of itself, so the first of each 
pair above is not quite right.

"the base class of the class heirarchy ['hierarchy']"  Steven D'Aprano
"the root of the class heirarchy"  ditto
+ quotes from Java and Ruby

Jeff Allen and Barry Warsaw like either, with BW slightly referring the 
first.

I was thinking about 'tree' instead of 'hierachy', but 'class tree' is 
wrong and 'directed acyclic graph of classes' a bit wordy.  While 
hierarchies are often thought of as trees, it is possible for someone to 
report to more than one person.

No one (yet) supported the status quo.

Martin also suggested a continuation:  "When called, it accepts no
arguments and returns a new unique and featureless object."

Even though object, unlike any other class, is primarily used as a base 
class rather than instance source, the latter should be included.  For 
other built-in classes, the summary line describes instances. I would 
leave out 'unique' as that could be confused with 'singleton'.

Provisional replacement, following the format of other class docstrings 
except for describing the class instead of instances, and intended to be 
helpful to beginners.

"The base class of the class hierarchy.

When called, it accepts no arguments and returns a new featureless 
instance that has no instance attributes and cannot be given any."

Passing arguments and adding attributes are two common mistakes.


-- 
Terry Jan Reedy



From asrp at email.com  Fri Feb  2 19:01:30 2018
From: asrp at email.com (asrp asrp)
Date: Sat, 3 Feb 2018 01:01:30 +0100
Subject: [Python-Dev] A minimal Python interpreter written in Python for
 experimenting with language changes
Message-ID: <trinity-aabb133a-c890-476f-a962-be6dacc697d6-1517616090604@3c-app-mailcom-lxa14>

Hello,

I don't know if this is the right place to post this. Please redirect as needed.

I've made a small Python interpreter in Python with runtime AST node semantics and edit-and-continue. I thought it could make prototyping language changes more easily and visualize usage before writing them in C.

Its here: https://github.com/asrp/python_terp

So, for example, redefining the for_stmt function in the right scope changes the behaviour of future for loops at runtime.

Although from discussion I've read in PEPs, actual implementation always look like a non-issue (which seems like magic to me) so maybe no-one here actually needs this.

(I really needed edit-and-continue for one of my projects but of course, running it in this extra interpreter is much too slow.)

asrp

From tritium-list at sdamon.com  Fri Feb  2 21:28:00 2018
From: tritium-list at sdamon.com (Alex Walters)
Date: Fri, 2 Feb 2018 21:28:00 -0500
Subject: [Python-Dev] A minimal Python interpreter written in Python for
 experimenting with language changes
In-Reply-To: <trinity-aabb133a-c890-476f-a962-be6dacc697d6-1517616090604@3c-app-mailcom-lxa14>
References: <trinity-aabb133a-c890-476f-a962-be6dacc697d6-1517616090604@3c-app-mailcom-lxa14>
Message-ID: <09cc01d39c96$9cb28910$d6179b30$@sdamon.com>

Are you aware of pypy?

> -----Original Message-----
> From: Python-Dev [mailto:python-dev-bounces+tritium-
> list=sdamon.com at python.org] On Behalf Of asrp asrp
> Sent: Friday, February 2, 2018 7:02 PM
> To: python-dev at python.org
> Subject: [Python-Dev] A minimal Python interpreter written in Python for
> experimenting with language changes
> 
> Hello,
> 
> I don't know if this is the right place to post this. Please redirect as
needed.
> 
> I've made a small Python interpreter in Python with runtime AST node
> semantics and edit-and-continue. I thought it could make prototyping
> language changes more easily and visualize usage before writing them in C.
> 
> Its here: https://github.com/asrp/python_terp
> 
> So, for example, redefining the for_stmt function in the right scope
changes
> the behaviour of future for loops at runtime.
> 
> Although from discussion I've read in PEPs, actual implementation always
> look like a non-issue (which seems like magic to me) so maybe no-one here
> actually needs this.
> 
> (I really needed edit-and-continue for one of my projects but of course,
> running it in this extra interpreter is much too slow.)
> 
> asrp
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium-
> list%40sdamon.com


From tjreedy at udel.edu  Sat Feb  3 00:50:11 2018
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 3 Feb 2018 00:50:11 -0500
Subject: [Python-Dev] A minimal Python interpreter written in Python for
 experimenting with language changes
In-Reply-To: <trinity-aabb133a-c890-476f-a962-be6dacc697d6-1517616090604@3c-app-mailcom-lxa14>
References: <trinity-aabb133a-c890-476f-a962-be6dacc697d6-1517616090604@3c-app-mailcom-lxa14>
Message-ID: <p53iem$dk6$1@blaine.gmane.org>

On 2/2/2018 7:01 PM, asrp asrp wrote:
> I don't know if this is the right place to post this. Please redirect as needed.

This list is for development *of* cpython.  Development *with* python in 
general belongs on python-list.


-- 
Terry Jan Reedy


From ncoghlan at gmail.com  Sat Feb  3 01:25:15 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 3 Feb 2018 16:25:15 +1000
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <34493569.S535hZLbfa@klinga.prans.org>
 <0a81521e-74f2-4883-6ab0-96e6fb03a419@trueblade.com>
 <1937305.WBmin8xbao@klinga.prans.org>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
Message-ID: <CADiSq7cRHy=zWNgOFUHiXpMYdR75D_co+skh53xCUCMjwYzo_w@mail.gmail.com>

On 3 Feb. 2018 1:09 am, "Eric V. Smith" <eric at trueblade.com> wrote:


The problem with dropping hash=True is: how would you write __hash__
yourself? It seems like a bug magnet if you're adding fields to the class
and forget to update __hash__, especially in the presence of per-field
hash=False and eq=False settings. And you'd need to make sure it matches
the generated __eq__ (if 2 objects are equal, they need to have the same
hash value).


I think anyone that does this needs to think *very* carefully about how
they do it, and offering both "hash=True" and "frozen=True" is an
attractive nuisance that means people will write "hash=True" when what they
wanted was "frozen=True".

In particular, having to work out how write a maintainable "__hash__" will
encourage folks to separate out the hashed fields as a separate frozen
subrecord or base class.

If this proves to be an intolerable burden then the short hand spelling
could be added back in 3.8, but once we ship it we're going to be stuck
with explaining the interactions indefinitely.

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180203/df11d81f/attachment.html>

From guido at python.org  Sat Feb  3 01:44:20 2018
From: guido at python.org (Guido van Rossum)
Date: Fri, 2 Feb 2018 22:44:20 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CADiSq7cRHy=zWNgOFUHiXpMYdR75D_co+skh53xCUCMjwYzo_w@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <34493569.S535hZLbfa@klinga.prans.org>
 <0a81521e-74f2-4883-6ab0-96e6fb03a419@trueblade.com>
 <1937305.WBmin8xbao@klinga.prans.org>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <CADiSq7cRHy=zWNgOFUHiXpMYdR75D_co+skh53xCUCMjwYzo_w@mail.gmail.com>
Message-ID: <CAP7+vJJWB8E5kLbPZ96WVNTgHobebpHh-0E0u=OYxNQ_C6ruig@mail.gmail.com>

It appears Eric and I are the only ones in favor of keeping the current
behavior. But I still am not convinced by all the worries about "attractive
nuisances" and all the other bad names this feature has been called. We
don't know that any of the doomsday scenarios will happen. In my
experience, usually once something is rolled out the set of issues that are
*actually* raised is entirely different from the things its designers were
worried about.

Please don't commit a change to roll this back without checking in with me;
I have some misgivings about the problem being raised here that I still
need to think through more carefully. In the meantime, please try to use
dataclasses with 3.7b1!

On Fri, Feb 2, 2018 at 10:25 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

>
>
> On 3 Feb. 2018 1:09 am, "Eric V. Smith" <eric at trueblade.com> wrote:
>
>
> The problem with dropping hash=True is: how would you write __hash__
> yourself? It seems like a bug magnet if you're adding fields to the class
> and forget to update __hash__, especially in the presence of per-field
> hash=False and eq=False settings. And you'd need to make sure it matches
> the generated __eq__ (if 2 objects are equal, they need to have the same
> hash value).
>
>
> I think anyone that does this needs to think *very* carefully about how
> they do it, and offering both "hash=True" and "frozen=True" is an
> attractive nuisance that means people will write "hash=True" when what they
> wanted was "frozen=True".
>
> In particular, having to work out how write a maintainable "__hash__" will
> encourage folks to separate out the hashed fields as a separate frozen
> subrecord or base class.
>
> If this proves to be an intolerable burden then the short hand spelling
> could be added back in 3.8, but once we ship it we're going to be stuck
> with explaining the interactions indefinitely.
>
> Cheers,
> Nick.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180202/4b167347/attachment-0001.html>

From steve at pearwood.info  Sat Feb  3 02:12:43 2018
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 3 Feb 2018 18:12:43 +1100
Subject: [Python-Dev] A minimal Python interpreter written in Python for
 experimenting with language changes
In-Reply-To: <p53iem$dk6$1@blaine.gmane.org>
References: <trinity-aabb133a-c890-476f-a962-be6dacc697d6-1517616090604@3c-app-mailcom-lxa14>
 <p53iem$dk6$1@blaine.gmane.org>
Message-ID: <20180203071242.GQ26553@ando.pearwood.info>

On Sat, Feb 03, 2018 at 12:50:11AM -0500, Terry Reedy wrote:
> On 2/2/2018 7:01 PM, asrp asrp wrote:
> >I don't know if this is the right place to post this. Please redirect as 
> >needed.
> 
> This list is for development *of* cpython.  Development *with* python in 
> general belongs on python-list.

This list is for development of Python the language, not just CPython 
the interpreter. It seems to me that announcing a new Python 
interpreter, especially one designed for the purpose of allowing rapid 
experimentation with the language, is on topic for this list.


-- 
Steve

From steve at pearwood.info  Sat Feb  3 02:27:46 2018
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 3 Feb 2018 18:27:46 +1100
Subject: [Python-Dev] A minimal Python interpreter written in Python for
 experimenting with language changes
In-Reply-To: <trinity-aabb133a-c890-476f-a962-be6dacc697d6-1517616090604@3c-app-mailcom-lxa14>
References: <trinity-aabb133a-c890-476f-a962-be6dacc697d6-1517616090604@3c-app-mailcom-lxa14>
Message-ID: <20180203072745.GR26553@ando.pearwood.info>

On Sat, Feb 03, 2018 at 01:01:30AM +0100, asrp asrp wrote:

> I've made a small Python interpreter in Python with runtime AST node 
> semantics and edit-and-continue. I thought it could make prototyping 
> language changes more easily and visualize usage before writing them 
> in C.

That sounds interesting.


> Its here: https://github.com/asrp/python_terp
> 
> So, for example, redefining the for_stmt function in the right scope 
> changes the behaviour of future for loops at runtime.

Can you give an example of how you would do that? I don't mean the 
mechanism used, I mean how would a developer implement a new syntactic 
feature. Suppose I wanted to add a new clause to for...else, let's say:

    for ... :
       block
    otherwise:
       # runs only if the for-loop was empty

How would do I do that?

(I've read the example on your github page, and it doesn't have much 
detail.)



-- 
Steve

From christian at python.org  Sat Feb  3 03:36:32 2018
From: christian at python.org (Christian Heimes)
Date: Sat, 3 Feb 2018 09:36:32 +0100
Subject: [Python-Dev] Deprecate crypt module and revert PR 3854
In-Reply-To: <p525la$e9$1@blaine.gmane.org>
References: <11437ad6-450f-58a2-94db-b8fe8619157e@python.org>
 <CAP7+vJLPiDkpq_mZA5L-8LK0U1D8peTX5B8GLHoaVsejFeTLxw@mail.gmail.com>
 <p525la$e9$1@blaine.gmane.org>
Message-ID: <3f008741-3bc8-d3aa-890d-a45688669183@python.org>

On 2018-02-02 18:05, Serhiy Storchaka wrote:
> 02.02.18 18:18, Guido van Rossum ????:
>> I'm all for nudging people in the direction of xcrypt. I assume we
>> can't just switch the C-level crypt with xcrypt and leave the Python
>> API unchanged?
>>
>> However until a usable solution exist (either in the stdlib or as 3rd
>> party) I don't think we should deprecate anything (deprecating things
>> before the replacement is ready is stressful for everyone involved).
>>
>> I'm also not sure I agree with removing support for old hashes. By all
>> means put in the docs that they are unsafe. But if someone has a
>> database full of old hashes it would be nice to be able to at least
>> read/verify it, right?
>>
>> Was a release already made with blowfish, extended DES and NT-Hash?
>> (And what's so bad with blowfish? It's mentioned in the heading of the
>> xcrypt project too.)
> 
> To clarify, extended DES and NT-Hash were not added. They were removed
> from my PR after Christians request. Only the Blowfish method was added,
> and it is so strong as SHA-2 methods. It is the only method supported on
> OpenBSD.

Ah, I misinterpreted the subject of the PR. The closed PR still mentions
extended DES and NT-Hash. I'm sorry and blame my travel fatigue. The
email was written at the airport after I had a conversion with somebody
about new Python 3.7 features.

> This PR is not a single enhancement made in the crypt module recently. I
> also extended tests and added support for configuring SHA-2 methods.
> There is an open PR (not merged before 3.7b1 unfortunately) for using
> crypt_r() instead of crypt(): https://bugs.python.org/issue28503.

In general I'm all for more tests and improvements of existing modules.
However in this case Python 3.7 is sending wrong signals. For example
additional of blowfish was prominently features on the largest German
newsletter for IT. Both blowfish and SSHA (salted sha) are legacy
password hashing algorithms. Glibc has moved them out of the main
library for a good reason. (*)

> If deprecate the crypt module, should modules pwd, grp and spwd be
> deprecated too? The crypt module is needed for checking password hashes
> provided by spwd.

The pwd and grp module are fine. The modules use proper libc APIs that
are internally backed by NSS (libc's Name Service Switch, not Mozilla's
Network Security Service). APIs such getpwnam are defined and
standardized since POSIX.1-2001. The pwd and grp automatically work with
any configured user and group provider, even LDAP, IdM or Active Directory.

Fun fact: Golang programs are usually statically compiled and don't even
use libc. However Go's os/user package requires CGO and libc because it
has to interface with libc and NSS to acquire user and group information.

The spwd module is a different story. It's a direct interface to
/etc/shadown using Linux-only APIs. The shadow DB API requires root
permission. I think it even circumvents system security policies and
identity provider.

tl;dr
pwd + grp == good, required
crypt + spwd == bad

Regards,
Christian

(*) Most Linux distros never had blowfish in libc anyway.

From tismer at stackless.com  Sat Feb  3 04:55:37 2018
From: tismer at stackless.com (Christian Tismer)
Date: Sat, 3 Feb 2018 10:55:37 +0100
Subject: [Python-Dev] A minimal Python interpreter written in Python for
 experimenting with language changes
In-Reply-To: <20180203071242.GQ26553@ando.pearwood.info>
References: <trinity-aabb133a-c890-476f-a962-be6dacc697d6-1517616090604@3c-app-mailcom-lxa14>
 <p53iem$dk6$1@blaine.gmane.org> <20180203071242.GQ26553@ando.pearwood.info>
Message-ID: <c173aa04-3b40-b9bf-a408-38b379d11c89@stackless.com>

On 03.02.18 08:12, Steven D'Aprano wrote:
> On Sat, Feb 03, 2018 at 12:50:11AM -0500, Terry Reedy wrote:
>> On 2/2/2018 7:01 PM, asrp asrp wrote:
>>> I don't know if this is the right place to post this. Please redirect as 
>>> needed.
>>
>> This list is for development *of* cpython.  Development *with* python in 
>> general belongs on python-list.
> 
> This list is for development of Python the language, not just CPython 
> the interpreter. It seems to me that announcing a new Python 
> interpreter, especially one designed for the purpose of allowing rapid 
> experimentation with the language, is on topic for this list.
> 
> 

Well spoken!

-- 
Christian Tismer-Sperling    :^)   tismer at stackless.com
Software Consulting          :     http://www.stackless.com/
Karl-Liebknecht-Str. 121     :     https://github.com/PySide
14482 Potsdam                :     GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 496 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180203/8c0064ba/attachment.sig>

From christian at python.org  Sat Feb  3 06:16:55 2018
From: christian at python.org (Christian Heimes)
Date: Sat, 3 Feb 2018 12:16:55 +0100
Subject: [Python-Dev] Deprecate crypt module and revert PR 3854
In-Reply-To: <20180202213118.4143777d@fsol>
References: <11437ad6-450f-58a2-94db-b8fe8619157e@python.org>
 <20180202213118.4143777d@fsol>
Message-ID: <p545j9$5fq$1@blaine.gmane.org>

On 2018-02-02 21:31, Antoine Pitrou wrote:
> On Fri, 2 Feb 2018 16:23:20 +0100
> Christian Heimes <christian at python.org> wrote:
>> Hi,
>>
>> in PR 3854 [1] Serhiy added blowfish, extended DES and NT-Hash to
>> Python's crypt mdodule. I vetoed against addition of the APIs because
>> all these hashing algorithms are not state of the art. Their quality
>> ranges from old to horribly, horriblye broken beyond any repair.
>>
>> Shortly after the PR has landed, I was made aware that glibc has
>> deprecated crypt(3) API [2] and favor of an external library called
>> libxcrypt [3] from OpenWall Linux. I have patched Python 3.7 [4] to
>> support libxcrypt.
>>
>> In light of deprecation of crypt(3) glibc function and bad quality of
>> hashing algorithms, I'd like to raise the motion to revert 3854 and
>> deprecate the crypt module.
> 
> Those are two separate proposals.
> 
> On the topic of reverting PR #3854, I don't see the point.  Is Blowfish
> more fragile than the other algorithms?  If not, it sounds ok to add it.

Blowfish password hashing algorithms (also known as bcrypt) hasn't been
state of the art of a long time. Here is an old blog post from 2012 that
explains some of the issues with Blowfish:

http://www.unlimitednovelty.com/2012/03/dont-use-bcrypt.html

> On the topic of deprecating the crypt module, that doesn't sound like a
> good idea right now.  People may need to generate crypt()-compatible
> output for various reasons, such as being able to automate system
> administration tasks.

IMO legacy support is not a good argument to keep the crypt module. The
passlib [1] package provides an excellent selection of legacy and modern
password derivation and hashing algorithms. As an admin you cannot rely
on the crypt module. libc's crypt() function usually provides a subset.
For example most Linux libc do not provide blowfish based hashing.

Christian

[1] https://pythonhosted.org/passlib/


From christian at python.org  Sat Feb  3 06:27:29 2018
From: christian at python.org (Christian Heimes)
Date: Sat, 3 Feb 2018 12:27:29 +0100
Subject: [Python-Dev] Deprecate crypt module and revert PR 3854
In-Reply-To: <CAPJVwBkuQ5o8V00-JzBWvvh45QtqiES_06=w6zQ4bNwtyiCmGw@mail.gmail.com>
References: <11437ad6-450f-58a2-94db-b8fe8619157e@python.org>
 <CAPJVwBkuQ5o8V00-JzBWvvh45QtqiES_06=w6zQ4bNwtyiCmGw@mail.gmail.com>
Message-ID: <p54673$i6j$1@blaine.gmane.org>

On 2018-02-02 21:21, Nathaniel Smith wrote:
> On Feb 2, 2018 7:24 AM, "Christian Heimes" <christian at python.org
> <mailto:christian at python.org>> wrote:
> 
>     Shortly after the PR has landed, I was made aware that glibc has
>     deprecated crypt(3) API [2] and favor of an external library called
>     libxcrypt [3] from OpenWall Linux. I have patched Python 3.7 [4] to
>     support libxcrypt.
> 
>     In light of deprecation of crypt(3) glibc function and bad quality of
>     hashing algorithms, I'd like to raise the motion to revert 3854 and
>     deprecate the crypt module. The whole module should be rather moved into
>     3rd party library that wraps xcrypt.
> 
> 
> If the crypt module can just as well use libxcrypt, and that's how the
> distros are going to build it, then what's the advantage of deprecating
> it? Won't it just continue to work indefinitely?

libxcrypt is now an external library that must be installed separately.
It bloats CPython's dependency tree and the size of Python container
images for a legacy feature.

For Python 4.0 I'd like to drop some of the old, dead batteries and
include include useful batteries.

Christian


From tismer at stackless.com  Sat Feb  3 06:29:19 2018
From: tismer at stackless.com (Christian Tismer)
Date: Sat, 3 Feb 2018 12:29:19 +0100
Subject: [Python-Dev] A minimal Python interpreter written in Python for
 experimenting with language changes
In-Reply-To: <trinity-aabb133a-c890-476f-a962-be6dacc697d6-1517616090604@3c-app-mailcom-lxa14>
References: <trinity-aabb133a-c890-476f-a962-be6dacc697d6-1517616090604@3c-app-mailcom-lxa14>
Message-ID: <9aa48a73-a02d-634c-dbae-c3c9144c0c06@stackless.com>

Hi user with no real name yet,

On 03.02.18 01:01, asrp asrp wrote:
> Hello,
> 
> I don't know if this is the right place to post this. Please redirect as needed.
> 
> I've made a small Python interpreter in Python with runtime AST node semantics and edit-and-continue. I thought it could make prototyping language changes more easily and visualize usage before writing them in C.
> 
> Its here: https://github.com/asrp/python_terp
> 
> So, for example, redefining the for_stmt function in the right scope changes the behaviour of future for loops at runtime.
> 
> Although from discussion I've read in PEPs, actual implementation always look like a non-issue (which seems like magic to me) so maybe no-one here actually needs this.
> 
> (I really needed edit-and-continue for one of my projects but of course, running it in this extra interpreter is much too slow.)
> 
> asrp

In the readme to python_terp you say:

"""
python_terp is intended to make language modification to Python easier
to preview changes more quickly and is not intended for full CPython
compatibility. However, a large subset of Python is already included. In
particular, enough to run the first stage of its parser.
"""

This needs clarification.
What do you mean by subset? A real subset or also things that
are different and will stay different?
To what extent are you planning to stay compatible, and where
do you plan to deviate?

The reason that I'm asking is that by compatible I mean the
compatibility of PyPy. If you can reach that, and be it just
by a subset, then it makes sense to speak of Python.

Cheers - Chris

-- 
Christian Tismer-Sperling    :^)   tismer at stackless.com
Software Consulting          :     http://www.stackless.com/
Karl-Liebknecht-Str. 121     :     https://github.com/PySide
14482 Potsdam                :     GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 496 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180203/bd6ec6cf/attachment.sig>

From christian at python.org  Sat Feb  3 07:07:17 2018
From: christian at python.org (Christian Heimes)
Date: Sat, 3 Feb 2018 13:07:17 +0100
Subject: [Python-Dev] Deprecate crypt module and revert PR 3854
In-Reply-To: <CAP7+vJLPiDkpq_mZA5L-8LK0U1D8peTX5B8GLHoaVsejFeTLxw@mail.gmail.com>
References: <11437ad6-450f-58a2-94db-b8fe8619157e@python.org>
 <CAP7+vJLPiDkpq_mZA5L-8LK0U1D8peTX5B8GLHoaVsejFeTLxw@mail.gmail.com>
Message-ID: <fdbee08d-1085-0aed-bc36-42e3a6e44954@python.org>

On 2018-02-02 17:18, Guido van Rossum wrote:
> I'm all for nudging people in the direction of xcrypt. I assume we can't
> just switch the C-level crypt with xcrypt and leave the Python API
> unchanged?
> 
> However until a usable solution exist (either in the stdlib or as 3rd
> party) I don't think we should deprecate anything (deprecating things
> before the replacement is ready is stressful for everyone involved).
> 
> I'm also not sure I agree with removing support for old hashes. By all
> means put in the docs that they are unsafe. But if someone has a
> database full of old hashes it would be nice to be able to at least
> read/verify it, right?
> 
> Was a release already made with blowfish, extended DES and NT-Hash? (And
> what's so bad with blowfish? It's mentioned in the heading of the xcrypt
> project too.)

I answered some of your questions in other replies and will answer the
remaining concerns on Monday.

You suggested a 3rd party module. I have cloned the crypt module with
Serhiy's improvements and turned it into a standalone module with a
ctypes interface, https://github.com/tiran/legacycrypt . I'll release
the package as soon as I find time to polish the documentation and give
Serhiy his will deserved credit for his work.

Christian

From asrp at email.com  Sat Feb  3 05:45:15 2018
From: asrp at email.com (asrp)
Date: Sat, 3 Feb 2018 11:45:15 +0100
Subject: [Python-Dev] A minimal Python interpreter written in Python for
 experimenting with language changes
In-Reply-To: <09cc01d39c96$9cb28910$d6179b30$@sdamon.com>
References: <trinity-aabb133a-c890-476f-a962-be6dacc697d6-1517616090604@3c-app-mailcom-lxa14>
 <09cc01d39c96$9cb28910$d6179b30$@sdamon.com>
Message-ID: <trinity-a323a69c-0a35-4e20-bb66-a6aa1df5c361-1517654715564@3c-app-mailcom-lxa05>

> Are you aware of pypy?
> 

Yes, but I do not know it well. It seems to be more focused on JIT compiling with many of its functions written for special cases and I did not find a way to make experimental changes easily.

I even tried (briefly) to have the main loop of this interpreter run in RPython but didn't find enough information on which subset of Python does RPython handle.

> Can you give an example of how you would do that? I don't mean the 
> mechanism used, I mean how would a developer implement a new syntactic 
> feature. Suppose I wanted to add a new clause to for...else, let's say:
>
>     for ... :
>        block
>     otherwise:
>        # runs only if the for-loop was empty
> 
> How would do I do that?
> 

Here's one way to do that. Sorry for the lack of doc for the moment.

To alter the language's grammar, you'd create a string with the new grammar, parse it and set the result to pyterp.parser. Then, depending on how the grammar was changed (which affects what AST is created), you'd change the corresponding function to handle the new semantics.

I didn't quite understand what you mean by "was empty". However, I noticed that "else" blocks after for are ignored in the current implement. Here's an example session adding "else".

If you tell me a bit more about the intended behaviour of "otherwise", I'd be happy to do an example with that clause.

    $ ipython -i test/python_repl.py 
    p>> simport simple_ast
    p>> for i in [1, 2]:
    ...     print i
    ... 
    1
    2
    p>> for i in [1, 2, 3]:
    ...     print i
    ... else:
    ...     print 100
    ... 
    1
    2
    3
    p>> ^D

In the above for loop, the else block is ignored! I press Control-D to exist the interpreter.

    In [1]: grammar = python_grammar.full_definition + python_grammar.extra

    In [2]: grammar += r"""
       ...: for_stmt = "for" {exprlist} "in" {testlist} ":" {suite} {(SAME_INDENT "else" ":" {suite}) | void=pass_stmt}
       ...: """

    In [3]: pyterp.parser = python.Interpreter(i3.parse("grammar", grammar))

    In [4]: pyterp.repl()

Now edit for_stmt in simple_ast.py

         except StopIteration:
+            evaluate(else_block)
             return

(If else_block was not already an (ignored) parameter, we'd have to change the `def for_stmt...` line too.) And run in the REPL we just restarted.

    p>> simple_ast.reload_module(simple_ast)
    p>> for i in [1, 2, 3]:
    ...     print i
    ... else:
    ...     print 100
    ... 
    1
    2
    3
    100
    p>> for i in [1, 2, 3]:
    ...     print i
    ...     break
    ... else:
    ...     print 100
    ... 
    1
    p>> 



Some notes:

1. I'm using the host Python interpreter to change the grammar here but that's not strictly necessary if we expose pyterp itself in the global scope.
2. I'm editing simple_ast.py and reloading because its less changes but redefining a new function and setting simple_ast.for_stmt (in other words, monkey-patching the change) should also work.

If we wanted an "otherwise" clause on top of an "else" clause, we probably want to make some provision for the runtime to distinguish between the two so I'd edit the grammar with something like

    for_stmt = "for" {exprlist} "in" {testlist} ":" {suite} {((SAME_INDENT "else" ":" {suite}) | void=pass_stmt)=else_block ((SAME_INDENT "otherwise" ":" {suite}) | void=pass_stmt)=otherwise_block}

(In this case, "otherwise" blocks have to come after the "else" block if both are present.)

asrp

From asrp at email.com  Sat Feb  3 09:17:47 2018
From: asrp at email.com (asrp)
Date: Sat, 3 Feb 2018 15:17:47 +0100
Subject: [Python-Dev] A minimal Python interpreter written in Python for
 experimenting with language changes
In-Reply-To: <9aa48a73-a02d-634c-dbae-c3c9144c0c06@stackless.com>
References: <trinity-aabb133a-c890-476f-a962-be6dacc697d6-1517616090604@3c-app-mailcom-lxa14>
 <9aa48a73-a02d-634c-dbae-c3c9144c0c06@stackless.com>
Message-ID: <trinity-40c82cbd-18db-40ef-a96a-23d798da7dd5-1517667467034@3c-app-mailcom-lxa12>

Hi,

(My other reply to Alex and Steven doesn't seems to appear, at least not on pipermail so I hope these don't come in the wrong order.)

> In the readme to python_terp you say:
> 
> """
> python_terp is intended to make language modification to Python easier
> to preview changes more quickly and is not intended for full CPython
> compatibility. However, a large subset of Python is already included. In
> particular, enough to run the first stage of its parser.
> """
> 
> This needs clarification.
> What do you mean by subset? A real subset or also things that
> are different and will stay different?
> To what extent are you planning to stay compatible, and where
> do you plan to deviate?
> 
> The reason that I'm asking is that by compatible I mean the
> compatibility of PyPy. If you can reach that, and be it just
> by a subset, then it makes sense to speak of Python.
> 

Good questions! There are two part to what I meant by that passage.

One is a reflection of the current state of the interpreter and its capabilities. I put an example instead listing all differences because its easier to update but admittedly much less precise.

The other concerns the intention for the future. My mind is not completely made but here's what I currently think it should/could be.

First, fidelity would be determined by how the interpreter is used (if its used). More (potentially) drastic changes in the beginning than later on. Hopefully the deviations from Python (if any) for this reason does not grow too much from this. But since its for testing language changes (to Python), I don't think it would stray too much, if any.

Second, I'd like to keep the whole thing small (of low complexity) as much as possible so in some cases, the less featureful choice will be kept. For example, in an earlier version, I had more functions parameters handling (defaults, *args, **kwargs, etc) but remove it because just setting up the initial local scope from the parameters passed and function definition took more than 100 lines. (Maybe I just went about this the wrong way...)

However, except for easier or simpler implementation of the interpreter or some of the initial AST nodes, I don't intend to make any additions not in the Python language. One example of this is simple_for and single_if statements which I explain a bit in this old post:

    http://blog.asrpo.com/bootstrap_chicken_or_egg

Although those are mainly intended to bootstrap the other statements like if_stmt and for_stmt and not used again later.

(Well, there might be some difference for class method lookup (late-binding instead of early-binding) that's needed for edit-and-continue. But the intention would be to list all of those differences.)

Feel free to ask again if I missed some aspect of your questions here.

asrp

> Sent: Saturday, February 03, 2018 at 12:29 PM
> From: "Christian Tismer" <tismer at stackless.com>
> To: "asrp asrp" <asrp at email.com>, python-dev at python.org
> Subject: Re: [Python-Dev] A minimal Python interpreter written in Python for experimenting with language changes
>
> Hi user with no real name yet,
> 
> On 03.02.18 01:01, asrp asrp wrote:
> > Hello,
> > 
> > I don't know if this is the right place to post this. Please redirect as needed.
> > 
> > I've made a small Python interpreter in Python with runtime AST node semantics and edit-and-continue. I thought it could make prototyping language changes more easily and visualize usage before writing them in C.
> > 
> > Its here: https://github.com/asrp/python_terp
> > 
> > So, for example, redefining the for_stmt function in the right scope changes the behaviour of future for loops at runtime.
> > 
> > Although from discussion I've read in PEPs, actual implementation always look like a non-issue (which seems like magic to me) so maybe no-one here actually needs this.
> > 
> > (I really needed edit-and-continue for one of my projects but of course, running it in this extra interpreter is much too slow.)
> > 
> > asrp
> 
> In the readme to python_terp you say:
> 
> """
> python_terp is intended to make language modification to Python easier
> to preview changes more quickly and is not intended for full CPython
> compatibility. However, a large subset of Python is already included. In
> particular, enough to run the first stage of its parser.
> """
> 
> This needs clarification.
> What do you mean by subset? A real subset or also things that
> are different and will stay different?
> To what extent are you planning to stay compatible, and where
> do you plan to deviate?
> 
> The reason that I'm asking is that by compatible I mean the
> compatibility of PyPy. If you can reach that, and be it just
> by a subset, then it makes sense to speak of Python.
> 
> Cheers - Chris
> 
> -- 
> Christian Tismer-Sperling    :^)   tismer at stackless.com
> Software Consulting          :     http://www.stackless.com/
> Karl-Liebknecht-Str. 121     :     https://github.com/PySide
> 14482 Potsdam                :     GPG key -> 0xFB7BEE0E
> phone +49 173 24 18 776  fax +49 (30) 700143-0023
> 
>

From asrp at email.com  Sat Feb  3 09:25:40 2018
From: asrp at email.com (asrp)
Date: Sat, 3 Feb 2018 15:25:40 +0100
Subject: [Python-Dev] A minimal Python interpreter written in Python for
 experimenting with language changes
References: <trinity-aabb133a-c890-476f-a962-be6dacc697d6-1517616090604@3c-app-mailcom-lxa14>
 <9aa48a73-a02d-634c-dbae-c3c9144c0c06@stackless.com>
 <trinity-40c82cbd-18db-40ef-a96a-23d798da7dd5-1517667467034@3c-app-mailcom-lxa12>
Message-ID: <trinity-fdbc146c-e858-4180-aa58-abd6179c43e5-1517667940097@3c-app-mailcom-lxa12>

Oh and I forgot to mention that since everything is mostly done at run time, there could just be different modules for (slightly) different languages. I don't know if that'd be problematic for maintenance though.
asrp

> Sent: Saturday, February 03, 2018 at 3:17 PM
> From: asrp <asrp at email.com>
> To: "Christian Tismer" <tismer at stackless.com>
> Cc: python-dev at python.org
> Subject: Re: [Python-Dev] A minimal Python interpreter written in Python for experimenting with language changes
>
> Hi,
> 
> (My other reply to Alex and Steven doesn't seems to appear, at least not on pipermail so I hope these don't come in the wrong order.)
> 
> > In the readme to python_terp you say:
> > 
> > """
> > python_terp is intended to make language modification to Python easier
> > to preview changes more quickly and is not intended for full CPython
> > compatibility. However, a large subset of Python is already included. In
> > particular, enough to run the first stage of its parser.
> > """
> > 
> > This needs clarification.
> > What do you mean by subset? A real subset or also things that
> > are different and will stay different?
> > To what extent are you planning to stay compatible, and where
> > do you plan to deviate?
> > 
> > The reason that I'm asking is that by compatible I mean the
> > compatibility of PyPy. If you can reach that, and be it just
> > by a subset, then it makes sense to speak of Python.
> > 
> 
> Good questions! There are two part to what I meant by that passage.
> 
> One is a reflection of the current state of the interpreter and its capabilities. I put an example instead listing all differences because its easier to update but admittedly much less precise.
> 
> The other concerns the intention for the future. My mind is not completely made but here's what I currently think it should/could be.
> 
> First, fidelity would be determined by how the interpreter is used (if its used). More (potentially) drastic changes in the beginning than later on. Hopefully the deviations from Python (if any) for this reason does not grow too much from this. But since its for testing language changes (to Python), I don't think it would stray too much, if any.
> 
> Second, I'd like to keep the whole thing small (of low complexity) as much as possible so in some cases, the less featureful choice will be kept. For example, in an earlier version, I had more functions parameters handling (defaults, *args, **kwargs, etc) but remove it because just setting up the initial local scope from the parameters passed and function definition took more than 100 lines. (Maybe I just went about this the wrong way...)
> 
> However, except for easier or simpler implementation of the interpreter or some of the initial AST nodes, I don't intend to make any additions not in the Python language. One example of this is simple_for and single_if statements which I explain a bit in this old post:
> 
>     http://blog.asrpo.com/bootstrap_chicken_or_egg
> 
> Although those are mainly intended to bootstrap the other statements like if_stmt and for_stmt and not used again later.
> 
> (Well, there might be some difference for class method lookup (late-binding instead of early-binding) that's needed for edit-and-continue. But the intention would be to list all of those differences.)
> 
> Feel free to ask again if I missed some aspect of your questions here.
> 
> asrp
> 
> > Sent: Saturday, February 03, 2018 at 12:29 PM
> > From: "Christian Tismer" <tismer at stackless.com>
> > To: "asrp asrp" <asrp at email.com>, python-dev at python.org
> > Subject: Re: [Python-Dev] A minimal Python interpreter written in Python for experimenting with language changes
> >
> > Hi user with no real name yet,
> > 
> > On 03.02.18 01:01, asrp asrp wrote:
> > > Hello,
> > > 
> > > I don't know if this is the right place to post this. Please redirect as needed.
> > > 
> > > I've made a small Python interpreter in Python with runtime AST node semantics and edit-and-continue. I thought it could make prototyping language changes more easily and visualize usage before writing them in C.
> > > 
> > > Its here: https://github.com/asrp/python_terp
> > > 
> > > So, for example, redefining the for_stmt function in the right scope changes the behaviour of future for loops at runtime.
> > > 
> > > Although from discussion I've read in PEPs, actual implementation always look like a non-issue (which seems like magic to me) so maybe no-one here actually needs this.
> > > 
> > > (I really needed edit-and-continue for one of my projects but of course, running it in this extra interpreter is much too slow.)
> > > 
> > > asrp
> > 
> > In the readme to python_terp you say:
> > 
> > """
> > python_terp is intended to make language modification to Python easier
> > to preview changes more quickly and is not intended for full CPython
> > compatibility. However, a large subset of Python is already included. In
> > particular, enough to run the first stage of its parser.
> > """
> > 
> > This needs clarification.
> > What do you mean by subset? A real subset or also things that
> > are different and will stay different?
> > To what extent are you planning to stay compatible, and where
> > do you plan to deviate?
> > 
> > The reason that I'm asking is that by compatible I mean the
> > compatibility of PyPy. If you can reach that, and be it just
> > by a subset, then it makes sense to speak of Python.
> > 
> > Cheers - Chris
> > 
> > -- 
> > Christian Tismer-Sperling    :^)   tismer at stackless.com
> > Software Consulting          :     http://www.stackless.com/
> > Karl-Liebknecht-Str. 121     :     https://github.com/PySide
> > 14482 Potsdam                :     GPG key -> 0xFB7BEE0E
> > phone +49 173 24 18 776  fax +49 (30) 700143-0023
> > 
> >

From guido at python.org  Sat Feb  3 11:13:26 2018
From: guido at python.org (Guido van Rossum)
Date: Sat, 3 Feb 2018 08:13:26 -0800
Subject: [Python-Dev] Deprecate crypt module and revert PR 3854
In-Reply-To: <fdbee08d-1085-0aed-bc36-42e3a6e44954@python.org>
References: <11437ad6-450f-58a2-94db-b8fe8619157e@python.org>
 <CAP7+vJLPiDkpq_mZA5L-8LK0U1D8peTX5B8GLHoaVsejFeTLxw@mail.gmail.com>
 <fdbee08d-1085-0aed-bc36-42e3a6e44954@python.org>
Message-ID: <CAP7+vJ+-181ZAW=AL7czq=U2m4kXJR84uZ7pDwdzrWBRHRi2rw@mail.gmail.com>

You can deprecate the crypt module, update the docs to explain it's
outdated and point to other 3rd party solutions. A few years from now we
may be able to delete it. (With an intermediate step of issuing a
non-silent deprecation warning.) Until then just leave it be. Possibly
that's what your desired outcome is anyway?

On Sat, Feb 3, 2018 at 4:07 AM, Christian Heimes <christian at python.org>
wrote:

> On 2018-02-02 17:18, Guido van Rossum wrote:
> > I'm all for nudging people in the direction of xcrypt. I assume we can't
> > just switch the C-level crypt with xcrypt and leave the Python API
> > unchanged?
> >
> > However until a usable solution exist (either in the stdlib or as 3rd
> > party) I don't think we should deprecate anything (deprecating things
> > before the replacement is ready is stressful for everyone involved).
> >
> > I'm also not sure I agree with removing support for old hashes. By all
> > means put in the docs that they are unsafe. But if someone has a
> > database full of old hashes it would be nice to be able to at least
> > read/verify it, right?
> >
> > Was a release already made with blowfish, extended DES and NT-Hash? (And
> > what's so bad with blowfish? It's mentioned in the heading of the xcrypt
> > project too.)
>
> I answered some of your questions in other replies and will answer the
> remaining concerns on Monday.
>
> You suggested a 3rd party module. I have cloned the crypt module with
> Serhiy's improvements and turned it into a standalone module with a
> ctypes interface, https://github.com/tiran/legacycrypt . I'll release
> the package as soon as I find time to polish the documentation and give
> Serhiy his will deserved credit for his work.
>
> Christian
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180203/d91829f4/attachment.html>

From greg at krypto.org  Sat Feb  3 14:25:38 2018
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 03 Feb 2018 19:25:38 +0000
Subject: [Python-Dev] "threading.Lock().locked()" is not documented
In-Reply-To: <06cf0afb-27ec-793e-61d1-3c0efb8da149@jcea.es>
References: <06cf0afb-27ec-793e-61d1-3c0efb8da149@jcea.es>
Message-ID: <CAGE7PNJ4MD0mpiyosUm7sBWaxGpCq2pMm8OV2ZYziLijGJ+AJw@mail.gmail.com>

On Wed, Jan 31, 2018 at 4:46 PM Jesus Cea <jcea at jcea.es> wrote:

> https://docs.python.org/3.6/library/threading.html doesn't document
> "threading.Lock().locked()", and it is something quite useful.
>
> In fact, it is used in "threading.py" itself. For instance, lines 109,
> 985, 1289.
>
> Is there any reason to not document it?.
>

No good reason.  It is a public method and part of the public API
regardless of being documented or not.  Always feel free to make a
documentation update PR.

The method does have a docstring.
  https://github.com/python/cpython/blob/master/Modules/_threadmodule.c#L193

-gps


>
> (I didn't investigate other objects in the module).
>
> --
> Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
> jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
> Twitter: @jcea                        _/_/    _/_/          _/_/_/_/_/
> jabber / xmpp:jcea at jabber.org  _/_/  _/_/    _/_/          _/_/  _/_/
> "Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
> "My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
> "El amor es poner tu felicidad en la felicidad de otro" - Leibniz
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180203/fd085e4b/attachment.html>

From greg at krypto.org  Sat Feb  3 15:05:07 2018
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 03 Feb 2018 20:05:07 +0000
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CADiSq7cRHy=zWNgOFUHiXpMYdR75D_co+skh53xCUCMjwYzo_w@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <34493569.S535hZLbfa@klinga.prans.org>
 <0a81521e-74f2-4883-6ab0-96e6fb03a419@trueblade.com>
 <1937305.WBmin8xbao@klinga.prans.org>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <CADiSq7cRHy=zWNgOFUHiXpMYdR75D_co+skh53xCUCMjwYzo_w@mail.gmail.com>
Message-ID: <CAGE7PNJ2J5g+NQ-hC6i13qVMLZCShbfCRw3c0jxK-EAOQLhLgw@mail.gmail.com>

On Fri, Feb 2, 2018 at 10:25 PM Nick Coghlan <ncoghlan at gmail.com> wrote:

>
>
> On 3 Feb. 2018 1:09 am, "Eric V. Smith" <eric at trueblade.com> wrote:
>
>
> The problem with dropping hash=True is: how would you write __hash__
> yourself? It seems like a bug magnet if you're adding fields to the class
> and forget to update __hash__, especially in the presence of per-field
> hash=False and eq=False settings. And you'd need to make sure it matches
> the generated __eq__ (if 2 objects are equal, they need to have the same
> hash value).
>
>
> I think anyone that does this needs to think *very* carefully about how
> they do it, and offering both "hash=True" and "frozen=True" is an
> attractive nuisance that means people will write "hash=True" when what they
> wanted was "frozen=True".
>

> In particular, having to work out how write a maintainable "__hash__" will
> encourage folks to separate out the hashed fields as a separate frozen
> subrecord or base class.
>
> If this proves to be an intolerable burden then the short hand spelling
> could be added back in 3.8, but once we ship it we're going to be stuck
> with explaining the interactions indefinitely.
>

+1 Nick put words to my chief concerns.

It is easy for an author see hash=True in existing code somewhere (cargo
culting) and assume it does what they want, or quickly glance at the the
API and see "hash=True" without actually taking the time to understand the
implications of that to see that the parameter named "frozen" is the one
they are supposed to want that _safely_ makes their dataclass properly
hashable, not the more attractive parameter named "hash" that enables
dangerous behavior.

Forcing people who need a __hash__ method to write it explicitly sounds
like a good thing to me.  I am not at all worried about someone forgetting
to add a new field to an implementation of the __hash__ method when adding
a new field, the fields and __hash__ method are all defined in the same
place in the code.

I expect someone with a common need for always having a __hash__ method
will produce a library on top of dataclasses that implements something like
our current hash=True behavior.  If that kind of thing turns out to be
widely used, we can reintroduce the feature in dataclasses in 3.8 or later,
informed by what we see practical uses actually doing.

In my practical experience, people writing Python code do not need to learn
and understand the intricacies of what it means to have a __hash__ method,
be hashable, or "frozen".  We intentionally warn people against writing
dunder methods other than __init__ in their code as they are often power
features with less obvious semantics than it may seem at first glance
making such code harder to maintain.

Even calling the parameter "hash=" and saying it adds a __hash__ method as
the PEP currently does seems to launder the danger, washing away the
"dunder smell" that adding a special considerations __hash__ method carries.

The PEP (and presumably forthcoming dataclasses module documentation) says
"This is a specialized use case and should be considered carefully" which I
agree with.  But any time we suggest that in an API, how about having the
API name make it clear that this is special and not to be done lightly?  I
guess i'm arguing against using "hash=" as the arg name in favor of
"danger_there_be_vorpal_rabbits_hash_me_maybe=" or something more usefully
similar if we're going to have it.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180203/2d57e832/attachment.html>

From benjamin at python.org  Sat Feb  3 16:08:29 2018
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 03 Feb 2018 13:08:29 -0800
Subject: [Python-Dev] "threading.Lock().locked()" is not documented
In-Reply-To: <CAGE7PNJ4MD0mpiyosUm7sBWaxGpCq2pMm8OV2ZYziLijGJ+AJw@mail.gmail.com>
References: <06cf0afb-27ec-793e-61d1-3c0efb8da149@jcea.es>
 <CAGE7PNJ4MD0mpiyosUm7sBWaxGpCq2pMm8OV2ZYziLijGJ+AJw@mail.gmail.com>
Message-ID: <1517692109.3300380.1258485616.38579E97@webmail.messagingengine.com>



On Sat, Feb 3, 2018, at 11:25, Gregory P. Smith wrote:
> On Wed, Jan 31, 2018 at 4:46 PM Jesus Cea <jcea at jcea.es> wrote:
> 
> > https://docs.python.org/3.6/library/threading.html doesn't document
> > "threading.Lock().locked()", and it is something quite useful.
> >
> > In fact, it is used in "threading.py" itself. For instance, lines 109,
> > 985, 1289.
> >
> > Is there any reason to not document it?.
> >
> 
> No good reason.  It is a public method and part of the public API
> regardless of being documented or not.

While I agree about .locked's present publicness, use of locked predicates in concurrent programs is usually serious code smell. So, the documentation should come with a warning note.

From ethan at stoneleaf.us  Sat Feb  3 17:02:18 2018
From: ethan at stoneleaf.us (Ethan Furman)
Date: Sat, 03 Feb 2018 14:02:18 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAP7+vJJWB8E5kLbPZ96WVNTgHobebpHh-0E0u=OYxNQ_C6ruig@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <34493569.S535hZLbfa@klinga.prans.org>
 <0a81521e-74f2-4883-6ab0-96e6fb03a419@trueblade.com>
 <1937305.WBmin8xbao@klinga.prans.org>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <CADiSq7cRHy=zWNgOFUHiXpMYdR75D_co+skh53xCUCMjwYzo_w@mail.gmail.com>
 <CAP7+vJJWB8E5kLbPZ96WVNTgHobebpHh-0E0u=OYxNQ_C6ruig@mail.gmail.com>
Message-ID: <5A76316A.40807@stoneleaf.us>

On 02/02/2018 10:44 PM, Guido van Rossum wrote:

> It appears Eric and I are the only ones in favor of keeping the current behavior. But I still am not convinced by all
> the worries about "attractive nuisances" and all the other bad names this feature has been called. We don't know that
> any of the doomsday scenarios will happen. In my experience, usually once something is rolled out the set of issues that
> are *actually* raised is entirely different from the things its designers were worried about.

This may all be true, but consider how many times we have asked, "How does attrs handle that?"

It would be wise to also ask, "What pitfalls have attrs discovered, and what would they do different if they could?"

--
~Ethan~

From tritium-list at sdamon.com  Sat Feb  3 17:40:30 2018
From: tritium-list at sdamon.com (Alex Walters)
Date: Sat, 3 Feb 2018 17:40:30 -0500
Subject: [Python-Dev] Is 4.0 a major breaking changes release?
Message-ID: <0a9e01d39d3f$ff2e9620$fd8bc260$@sdamon.com>

I am still working on porting code from 2.x to 3.x.  As of late on the lists
I've seen comments about making somewhat major changes in 4.0 - now I'm
concerned that I should pause my porting effort until that is released.  Is
python 4 going to be another python 3?


From nad at python.org  Sat Feb  3 17:46:21 2018
From: nad at python.org (Ned Deily)
Date: Sat, 3 Feb 2018 17:46:21 -0500
Subject: [Python-Dev] Is 4.0 a major breaking changes release?
In-Reply-To: <0a9e01d39d3f$ff2e9620$fd8bc260$@sdamon.com>
References: <0a9e01d39d3f$ff2e9620$fd8bc260$@sdamon.com>
Message-ID: <64701CC4-9E49-461A-AC3A-336967ECCF29@python.org>

On Feb 3, 2018, at 17:40, Alex Walters <tritium-list at sdamon.com> wrote:
> I am still working on porting code from 2.x to 3.x.  As of late on the lists
> I've seen comments about making somewhat major changes in 4.0 - now I'm
> concerned that I should pause my porting effort until that is released.  Is
> python 4 going to be another python 3?

At this point, Python 4 is just a convenient tag for really big changes.  There are no concrete plans or schedule for such a major undertaking.  Port away to Python 3.x!

--
  Ned Deily
  nad at python.org -- []


From p.f.moore at gmail.com  Sat Feb  3 17:48:59 2018
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 3 Feb 2018 22:48:59 +0000
Subject: [Python-Dev] Is 4.0 a major breaking changes release?
In-Reply-To: <0a9e01d39d3f$ff2e9620$fd8bc260$@sdamon.com>
References: <0a9e01d39d3f$ff2e9620$fd8bc260$@sdamon.com>
Message-ID: <CACac1F9GgHAuiObr97pCTQhNRbiQ66vUzWHJMCgnVNFBi9THUA@mail.gmail.com>

On 3 February 2018 at 22:40, Alex Walters <tritium-list at sdamon.com> wrote:
> I am still working on porting code from 2.x to 3.x.  As of late on the lists
> I've seen comments about making somewhat major changes in 4.0 - now I'm
> concerned that I should pause my porting effort until that is released.  Is
> python 4 going to be another python 3?

No. Guido has gone on record as saying this won't happen.

Paul

From zachary.ware+pydev at gmail.com  Sat Feb  3 17:50:02 2018
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Sat, 3 Feb 2018 16:50:02 -0600
Subject: [Python-Dev] Is 4.0 a major breaking changes release?
In-Reply-To: <0a9e01d39d3f$ff2e9620$fd8bc260$@sdamon.com>
References: <0a9e01d39d3f$ff2e9620$fd8bc260$@sdamon.com>
Message-ID: <CAKJDb-P8Z_tqXA-LQp=5yiv0Pvf=aVHV1tT2cOatCCkWSRiOXw@mail.gmail.com>

On Sat, Feb 3, 2018 at 4:40 PM, Alex Walters <tritium-list at sdamon.com> wrote:
> I am still working on porting code from 2.x to 3.x.  As of late on the lists
> I've seen comments about making somewhat major changes in 4.0 - now I'm
> concerned that I should pause my porting effort until that is released.  Is
> python 4 going to be another python 3?

Emphatically no.  Anyone suggesting big breaking changes in 4.0 is
exercising wishful thinking :)

There may be some cleanup in 4.0, but only removing things that have
been deprecated for a long time in 3.x but hadn't been removed to
maintain compatibility with 2.7.

-- 
Zach

From drsalists at gmail.com  Sat Feb  3 17:51:09 2018
From: drsalists at gmail.com (Dan Stromberg)
Date: Sat, 3 Feb 2018 14:51:09 -0800
Subject: [Python-Dev] Is 4.0 a major breaking changes release?
In-Reply-To: <0a9e01d39d3f$ff2e9620$fd8bc260$@sdamon.com>
References: <0a9e01d39d3f$ff2e9620$fd8bc260$@sdamon.com>
Message-ID: <CAGGBd_qcjOORg0qBnw6cZ8qW-BsY16DxPoXHOU8Ggss1M1fFzQ@mail.gmail.com>

On Sat, Feb 3, 2018 at 2:40 PM, Alex Walters <tritium-list at sdamon.com> wrote:
> I am still working on porting code from 2.x to 3.x.  As of late on the lists
> I've seen comments about making somewhat major changes in 4.0 - now I'm
> concerned that I should pause my porting effort until that is released.  Is
> python 4 going to be another python 3?

https://www.curiousefficiency.org/posts/2014/08/python-4000.html

Python 0.x to 1.x was a small change.
Python 1.x to 2.x was a small change.

I doubt there'll be anything as important as str -> (bytes, unicode)
to merit a breaking change.

From tjreedy at udel.edu  Sat Feb  3 20:48:08 2018
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 3 Feb 2018 20:48:08 -0500
Subject: [Python-Dev] Is 4.0 a major breaking changes release?
In-Reply-To: <CAKJDb-P8Z_tqXA-LQp=5yiv0Pvf=aVHV1tT2cOatCCkWSRiOXw@mail.gmail.com>
References: <0a9e01d39d3f$ff2e9620$fd8bc260$@sdamon.com>
 <CAKJDb-P8Z_tqXA-LQp=5yiv0Pvf=aVHV1tT2cOatCCkWSRiOXw@mail.gmail.com>
Message-ID: <p55oks$ei2$1@blaine.gmane.org>

On 2/3/2018 5:50 PM, Zachary Ware wrote:
> On Sat, Feb 3, 2018 at 4:40 PM, Alex Walters <tritium-list at sdamon.com> wrote:
>> I am still working on porting code from 2.x to 3.x.  As of late on the lists
>> I've seen comments about making somewhat major changes in 4.0 - now I'm
>> concerned that I should pause my porting effort until that is released.  Is
>> python 4 going to be another python 3?
> 
> Emphatically no.  Anyone suggesting big breaking changes in 4.0 is
> exercising wishful thinking :)
> 
> There may be some cleanup in 4.0, but only removing things that have
> been deprecated for a long time in 3.x but hadn't been removed to
> maintain compatibility with 2.7.

So do your porting with deprecation warnings on.


-- 
Terry Jan Reedy


From steve at holdenweb.com  Sat Feb  3 21:18:14 2018
From: steve at holdenweb.com (Steve Holden)
Date: Sun, 4 Feb 2018 02:18:14 +0000
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAP7+vJJWB8E5kLbPZ96WVNTgHobebpHh-0E0u=OYxNQ_C6ruig@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <34493569.S535hZLbfa@klinga.prans.org>
 <0a81521e-74f2-4883-6ab0-96e6fb03a419@trueblade.com>
 <1937305.WBmin8xbao@klinga.prans.org>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <CADiSq7cRHy=zWNgOFUHiXpMYdR75D_co+skh53xCUCMjwYzo_w@mail.gmail.com>
 <CAP7+vJJWB8E5kLbPZ96WVNTgHobebpHh-0E0u=OYxNQ_C6ruig@mail.gmail.com>
Message-ID: <CAMofdRBy1Pt687VdqzhBHO5FBjg2AqcEsTAWK9PPRGofC52NuA@mail.gmail.com>

As a Bear of Relatively Little Brain, I've grown up understanding, and
teaching, that mutable things aren't to be used as dict keys. I'm aware
that immutability isn't strictly the required condition, but it for most
people, that's the primary reason for using frozen sets and tuples, for
example, and immutability serves as a practical and comprehensible first
approximation. So I'm at a loss to understand why I am being offered a
feature that (especially during maintenance by a different developer) might
be prone to bizarre errors caused by a change in hash.

I realise that this won't happen very often, but the difficulty of the
debug task should surely merit at least some warning for us bears - you
know, the ones that take your work and use it to do mundane things with.

On a slightly tangential note, us bears are very glad that such questions
are taken seriously and discussed in such depth. Thank you all.

Steve Holden

On Sat, Feb 3, 2018 at 6:44 AM, Guido van Rossum <guido at python.org> wrote:

> It appears Eric and I are the only ones in favor of keeping the current
> behavior. But I still am not convinced by all the worries about "attractive
> nuisances" and all the other bad names this feature has been called. We
> don't know that any of the doomsday scenarios will happen. In my
> experience, usually once something is rolled out the set of issues that are
> *actually* raised is entirely different from the things its designers were
> worried about.
>
> Please don't commit a change to roll this back without checking in with
> me; I have some misgivings about the problem being raised here that I still
> need to think through more carefully. In the meantime, please try to use
> dataclasses with 3.7b1!
>
> On Fri, Feb 2, 2018 at 10:25 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>>
>>
>> On 3 Feb. 2018 1:09 am, "Eric V. Smith" <eric at trueblade.com> wrote:
>>
>>
>> The problem with dropping hash=True is: how would you write __hash__
>> yourself? It seems like a bug magnet if you're adding fields to the class
>> and forget to update __hash__, especially in the presence of per-field
>> hash=False and eq=False settings. And you'd need to make sure it matches
>> the generated __eq__ (if 2 objects are equal, they need to have the same
>> hash value).
>>
>>
>> I think anyone that does this needs to think *very* carefully about how
>> they do it, and offering both "hash=True" and "frozen=True" is an
>> attractive nuisance that means people will write "hash=True" when what they
>> wanted was "frozen=True".
>>
>> In particular, having to work out how write a maintainable "__hash__"
>> will encourage folks to separate out the hashed fields as a separate frozen
>> subrecord or base class.
>>
>> If this proves to be an intolerable burden then the short hand spelling
>> could be added back in 3.8, but once we ship it we're going to be stuck
>> with explaining the interactions indefinitely.
>>
>> Cheers,
>> Nick.
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180204/7db4e272/attachment-0001.html>

From steve at holdenweb.com  Sat Feb  3 21:23:52 2018
From: steve at holdenweb.com (Steve Holden)
Date: Sun, 4 Feb 2018 02:23:52 +0000
Subject: [Python-Dev] Is 4.0 a major breaking changes release?
In-Reply-To: <64701CC4-9E49-461A-AC3A-336967ECCF29@python.org>
References: <0a9e01d39d3f$ff2e9620$fd8bc260$@sdamon.com>
 <64701CC4-9E49-461A-AC3A-336967ECCF29@python.org>
Message-ID: <CAMofdRB6hQWA6caNg70k-BVTg5zdF=PPteop61RCLmK61VEqLw@mail.gmail.com>

On Sat, Feb 3, 2018 at 10:46 PM, Ned Deily <nad at python.org> wrote:

> On Feb 3, 2018, at 17:40, Alex Walters <tritium-list at sdamon.com> wrote:
> > I am still working on porting code from 2.x to 3.x.  As of late on the
> lists
> > I've seen comments about making somewhat major changes in 4.0 - now I'm
> > concerned that I should pause my porting effort until that is released.
> Is
> > python 4 going to be another python 3?
>
> At this point, Python 4 is just a convenient tag for really big changes.
> There are no concrete plans or schedule for such a major undertaking.  Port
> away to Python 3.x!
>
> ?"Py3K?" they said, "oh, no, we're just noodling around with a few ideas
..."? :-)

To be honest, and historically fair to Guido, he did warn for a long time
that we should expect breaking changes in an eventual wart-removal release.

It seemed to me the biggest disappointment was the team not having the
resources to devote to a mooted but never really achieved reorganisation of
the stdlib.

?Rectifying that omission would, I hope, be included as a priority in any
Python 4 design. Since people rely on the stdlib hugely, automated
translation of at least 98% of existing stdlib imports should be a goal.
But that's just  me.
?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180204/172a87e6/attachment.html>

From greg.ewing at canterbury.ac.nz  Sat Feb  3 23:51:12 2018
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 04 Feb 2018 17:51:12 +1300
Subject: [Python-Dev] Is 4.0 a major breaking changes release?
In-Reply-To: <CAMofdRB6hQWA6caNg70k-BVTg5zdF=PPteop61RCLmK61VEqLw@mail.gmail.com>
References: <0a9e01d39d3f$ff2e9620$fd8bc260$@sdamon.com>
 <64701CC4-9E49-461A-AC3A-336967ECCF29@python.org>
 <CAMofdRB6hQWA6caNg70k-BVTg5zdF=PPteop61RCLmK61VEqLw@mail.gmail.com>
Message-ID: <5A769140.6040302@canterbury.ac.nz>

> On Sat, Feb 3, 2018 at 10:46 PM, Ned Deily <nad at python.org 
> <mailto:nad at python.org>> wrote:
>>  I've seen comments about making somewhat major changes in 4.0 -
>     now I'm
>>  concerned that I should pause my porting effort until that is
>     released.  Is
>>  python 4 going to be another python 3?

Guido has repeatedly promised that there will never be another
upheaval as big as the 2-to-3 one, and that the change from
Python 3.9 to 4.0 won't be anything special. Hopefully we can
trust him on that.

-- 
Greg

From ronaldoussoren at mac.com  Sun Feb  4 04:44:34 2018
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sun, 04 Feb 2018 10:44:34 +0100
Subject: [Python-Dev] OS-X builds for 3.7.0
In-Reply-To: <CALGmxE+RxK+YMFax9J=5c7VRUhnfQCieayw-Q2KhxJRk3Fvghg@mail.gmail.com>
References: <CALGmxE+RxK+YMFax9J=5c7VRUhnfQCieayw-Q2KhxJRk3Fvghg@mail.gmail.com>
Message-ID: <91C9C795-B485-4E89-AA46-45B01B0D5ACD@mac.com>



> On 30 Jan 2018, at 18:42, Chris Barker <Chris.Barker at noaa.gov> wrote:
> 
> Ned,
> 
> It looks like you're still building OS-X the same way as in the past:
> 
> Intel 32+64 bit, 10.6 compatibility
> 
> Is that right?
> 
> Might it be time for an update?
> 
> Do we still need to support 32 bit?  From:
> 
> https://apple.stackexchange.com/questions/99640/how-old-are-macs-that-cannot-run-64-bit-applications <https://apple.stackexchange.com/questions/99640/how-old-are-macs-that-cannot-run-64-bit-applications>
> 
> There has not been a 32 bit-only Mac sold since 2006, and a out-of the box 32 bit OS since 2006 or 2007
> 
> I can't find out what the older OS version Apple supports, but I know my IT dept has been making me upgrade, so I"m going to guess 10.8 or newer?

A binary with a newer deployment target than 10.6 would be nice because AFAIK the installers are still build on a system running that old version of OSX. This results in binaries that cannot access newer system APIs like openat (and hence don?t support the ?dir_fd? parameter in a number of function in the os module.

> 
> And maybe we could even get rid of the "Framework" builds??

Why?  IMHO Framework builds are a nice way to get isolated side-by-side installations. Furthermore a number of Apple APIs (including the GUI libraries) don?t work unless you?re running from an application bundle, which the framework builds arranges for and normal unix builds don?t. 

Ronald

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180204/572437df/attachment.html>

From ronaldoussoren at mac.com  Sun Feb  4 06:02:17 2018
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sun, 04 Feb 2018 12:02:17 +0100
Subject: [Python-Dev] [python-committers] [RELEASE] Python 3.7.0b1 is
 now available for testing
In-Reply-To: <9425596C-A92F-4B10-A8B7-98F4E827E8D0@python.org>
References: <9425596C-A92F-4B10-A8B7-98F4E827E8D0@python.org>
Message-ID: <F56CE8A6-0946-4A54-9F97-E2B22F11F557@mac.com>



> On 1 Feb 2018, at 02:34, Ned Deily <nad at python.org> wrote:
> 
> [?]
> 
> Attention macOS users: with 3.7.0b1, we are providing a choice of
> two binary installers.  The new variant provides a 64-bit-only
> version for macOS 10.9 and later systems; this variant also now
> includes its own built-in version of Tcl/Tk 8.6.  We welcome your
> feedback.
> 

Why macOS 10.9 or later?  MacOS 10.10 introduced a number of useful APIs, in particular openat(2) and the like which are exposed using the ?dir_fd? parameter of functions in the posix module.

That said, macOS 10.9 seems to be a fairly common minimal platform requirement these days for developers not tracking Apple?s releases closely.

Ronald


From chris.barker at noaa.gov  Sun Feb  4 14:59:27 2018
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Sun, 4 Feb 2018 11:59:27 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
Message-ID: <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>

>> IMO, the danger of
>> "@dataclass(hash=True)" far overweighs whatever convenience it might
>> provide.

Is there any reason specifying has=True could set frozen=True unless
the user specifically sets frozen=False?

Or is that already the case?

I think the folks that are concerned about this issue are quite right
? most Python users equate immutable and hashable?so the dataclass API
should reflect that.

And this would still make it easy and clear to specify the unusual
(and arguably dangerous) case of:

hash=True, frozen=False

-CHB

>

From guido at python.org  Sun Feb  4 17:31:06 2018
From: guido at python.org (Guido van Rossum)
Date: Sun, 4 Feb 2018 14:31:06 -0800
Subject: [Python-Dev] Immutability vs. hashability
Message-ID: <CAP7+vJLQP29XTRVYSYGC7wS5RN9CF7CDu+CTBJtrotaxbNedZw@mail.gmail.com>

On Sun, Feb 4, 2018 at 11:59 AM, Chris Barker - NOAA Federal <
chris.barker at noaa.gov> wrote:

> I think the folks that are concerned about this issue are quite right
> ? most Python users equate immutable and hashable?so the dataclass API
> should reflect that.
>

Since they are *not* equivalent (consider a tuple containing a list) I'm
not at all convinced that any API in the core language should "reflect"
this misconception, depending on how you meant that.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180204/a5057caa/attachment.html>

From larry at hastings.org  Sun Feb  4 19:35:03 2018
From: larry at hastings.org (Larry Hastings)
Date: Sun, 4 Feb 2018 16:35:03 -0800
Subject: [Python-Dev] [RELEASED] Python 3.4.8 and Python 3.5.5 are now
 available
Message-ID: <7b3f1a47-bdb1-8601-3a68-2b6c5b53b1ac@hastings.org>


On behalf of the Python development community, I'm happy to announce the 
availability of Python 3.4.8 and Python 3.5.5.

Both Python 3.4 and 3.5 are in "security fixes only" mode.? Both 
versions only accept security fixes, not conventional bug fixes, and 
both releases are source-only.


You can find Python 3.4.8 here:

    https://www.python.org/downloads/release/python-348/


And you can find Python 3.5.5 here:

    https://www.python.org/downloads/release/python-355/



Happy Pythoning,


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180204/d8d060db/attachment.html>

From ncoghlan at gmail.com  Sun Feb  4 20:54:37 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 5 Feb 2018 11:54:37 +1000
Subject: [Python-Dev] Immutability vs. hashability
In-Reply-To: <CAP7+vJLQP29XTRVYSYGC7wS5RN9CF7CDu+CTBJtrotaxbNedZw@mail.gmail.com>
References: <CAP7+vJLQP29XTRVYSYGC7wS5RN9CF7CDu+CTBJtrotaxbNedZw@mail.gmail.com>
Message-ID: <CADiSq7fezK+tgHNrmw1VVR-STM5nc+pOAPUho9J3MCa_f3RPAA@mail.gmail.com>

On 5 February 2018 at 08:31, Guido van Rossum <guido at python.org> wrote:
> On Sun, Feb 4, 2018 at 11:59 AM, Chris Barker - NOAA Federal
> <chris.barker at noaa.gov> wrote:
>>
>> I think the folks that are concerned about this issue are quite right
>> ? most Python users equate immutable and hashable?so the dataclass API
>> should reflect that.
>
> Since they are *not* equivalent (consider a tuple containing a list) I'm not
> at all convinced that any API in the core language should "reflect" this
> misconception, depending on how you meant that.

Lists are themselves mutable, and hence inherently unhashable.

Tuples are themselves immutable, and hence hashable if their contents are.

I interpret Chris's comment as saying that data classes should behave
the same way that the builtin container types do:

* if the data class itself is mutable (frozen=False, comparable to
list, dict, set), then it is *not* hashable (unless you explicitly
implement __hash__)

* if the data class itself is immutable (frozen=True, comparable to
tuple or frozenset), then whether or not it is hashable depends on
whether or not the field values are hashable.

It's the ability to ask the interpreter to guess what you mean
"frozen=False, hash=True" that creates the likelihood of confusion.

Whereas if we leave out the "hash=True" option entirely, then the most
natural way to obtain a partially-mutable record, which has a fixed
comparison key and selectively mutable state, then the recommended way
of handling that would be through containment, where the mutable state
is moved out to a subrecord that gets excluded from hashes and
comparisons.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From jimjjewett at gmail.com  Sun Feb  4 23:42:01 2018
From: jimjjewett at gmail.com (Jim J. Jewett)
Date: Sun, 4 Feb 2018 23:42:01 -0500
Subject: [Python-Dev] unfrozen dataclasses and __hash__ (subsets are OK)
Message-ID: <CA+OGgf6==rn2Rz4_n-fY5JNqN0c=OsfOcn6LKMWNpcQBzPQE0w@mail.gmail.com>

I understand auto-generating the __hash__ (and __eq__) for a frozen
container; that is just convenient.

But why is there any desire to autogenerate a __hash__ for something
that isn't frozen?  Like a list or dict, the normal case would be for
it not to have a hash at all, and the author *should* write out any
explicit exceptions.

The objection to that seems to be that someone might forget to add
another field to the hash during later maintenance -- but so what?

__hash__ should reference a subset of the fields used for equality,
and strict subsets are OK.  It *should* ignore some fields if that
will provide the right balance between quick calculation and
sufficient dispersion.  If the record is complicated enough that
forgetting a field is a likely problem, then the hash is probably
already sufficiently complex without those new fields.

 -jJ

From guido at python.org  Mon Feb  5 00:18:25 2018
From: guido at python.org (Guido van Rossum)
Date: Sun, 4 Feb 2018 21:18:25 -0800
Subject: [Python-Dev] Immutability vs. hashability
In-Reply-To: <CADiSq7fezK+tgHNrmw1VVR-STM5nc+pOAPUho9J3MCa_f3RPAA@mail.gmail.com>
References: <CAP7+vJLQP29XTRVYSYGC7wS5RN9CF7CDu+CTBJtrotaxbNedZw@mail.gmail.com>
 <CADiSq7fezK+tgHNrmw1VVR-STM5nc+pOAPUho9J3MCa_f3RPAA@mail.gmail.com>
Message-ID: <CAP7+vJJ_aBVvgFd1pzA6=AaMCFRO3wQCJwjujmeryK5+4fd-SQ@mail.gmail.com>

That's a lot to read between the lines. I was unhappy that Chris took the
statement that immutability and hashability are equivalent, claimed that
most people think of it that way, and did not point out that it was false,
thereby making the impression that he wasn't aware of the difference.

The way I think of it generally is that immutability is a property of
types, while hashability is a property of values.

I don't want the original debate (about what to do with hash=True for
dataclasses) to be spread across multiple threads so I'll reply separately
there.

On Sun, Feb 4, 2018 at 5:54 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 5 February 2018 at 08:31, Guido van Rossum <guido at python.org> wrote:
> > On Sun, Feb 4, 2018 at 11:59 AM, Chris Barker - NOAA Federal
> > <chris.barker at noaa.gov> wrote:
> >>
> >> I think the folks that are concerned about this issue are quite right
> >> ? most Python users equate immutable and hashable?so the dataclass API
> >> should reflect that.
> >
> > Since they are *not* equivalent (consider a tuple containing a list) I'm
> not
> > at all convinced that any API in the core language should "reflect" this
> > misconception, depending on how you meant that.
>
> Lists are themselves mutable, and hence inherently unhashable.
>
> Tuples are themselves immutable, and hence hashable if their contents are.
>
> I interpret Chris's comment as saying that data classes should behave
> the same way that the builtin container types do:
>
> * if the data class itself is mutable (frozen=False, comparable to
> list, dict, set), then it is *not* hashable (unless you explicitly
> implement __hash__)
>
> * if the data class itself is immutable (frozen=True, comparable to
> tuple or frozenset), then whether or not it is hashable depends on
> whether or not the field values are hashable.
>
> It's the ability to ask the interpreter to guess what you mean
> "frozen=False, hash=True" that creates the likelihood of confusion.
>
> Whereas if we leave out the "hash=True" option entirely, then the most
> natural way to obtain a partially-mutable record, which has a fixed
> comparison key and selectively mutable state, then the recommended way
> of handling that would be through containment, where the mutable state
> is moved out to a subrecord that gets excluded from hashes and
> comparisons.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180204/6c41e98f/attachment.html>

From guido at python.org  Mon Feb  5 00:49:50 2018
From: guido at python.org (Guido van Rossum)
Date: Sun, 4 Feb 2018 21:49:50 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
Message-ID: <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>

Looks like this is turning into a major flamewar regardless of what I say.
:-(

I really don't want to lose the ability to add a hash function to a mutable
dataclass by flipping a flag in the decorator. I'll explain below. But I am
fine if this flag has a name that clearly signals it's an unsafe thing to
do.

I propose to replace the existing (as of 3.7.0b1) hash= keyword for the
@dataclass decorator with a simpler flag named unsafe_hash=. This would be
a simple bool (not a tri-state flag like the current hash=None|False|True).
The default would be False, and the behavior then would be to add a hash
function automatically only if it's safe (using the same rules as for
hash=None currently). With unsafe_hash=True, a hash function would always
be generated that takes all fields into account except those declared using
field(hash=False). If there's already a `def __hash__` in the function I
don't care what it does, maybe it should raise rather than quietly doing
nothing or quietly overwriting it.

Here's my use case.

A frozen class requires a lot of discipline, since you have to compute the
values of all fields before calling the constructor. A mutable class allows
other initialization patterns, e.g. manually setting some fields after the
instance has been constructed, or having a separate non-dunder init()
method. There may be good reasons for using these patterns, e.g. the object
may be part of a cycle (e.g. parent/child links in a tree). Or you may just
use one of these patterns because you're a pretty casual coder. Or you're
modeling something external.

My point is that once you have one of those patterns in place, changing
your code to avoid them may be difficult. And yet your code may treat the
objects as essentially immutable after the initialization phase (e.g. a
parse tree). So if you create a dataclass and start coding like that for a
while, and much later you need to put one of these into a set or use it as
a dict key, switching to frozen=True may not be a quick option. And writing
a __hash__ method by hand may feel like a lot of busywork. So this is where
[unsafe_]hash=True would come in handy.

I think naming the flag unsafe_hash should take away most objections, since
it will be clear that this is not a safe thing to do. People who don't
understand the danger are likely to copy a worse solution from
StackOverflow anyway. The docs can point to frozen=True and explain the
danger.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180204/a2278e1e/attachment.html>

From greg at krypto.org  Mon Feb  5 00:57:02 2018
From: greg at krypto.org (Gregory P. Smith)
Date: Mon, 05 Feb 2018 05:57:02 +0000
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
Message-ID: <CAGE7PNL8Ky1fuE6vbX7eDdHZXqmWcQRhQWBwGST60COiysRG8Q@mail.gmail.com>

+1 using unsafe_hash as a name addresses my concern. It's a good signal
that there are caveats worth considering.

-gps

On Sun, Feb 4, 2018, 9:50 PM Guido van Rossum <guido at python.org> wrote:

> Looks like this is turning into a major flamewar regardless of what I say.
> :-(
>
> I really don't want to lose the ability to add a hash function to a
> mutable dataclass by flipping a flag in the decorator. I'll explain below.
> But I am fine if this flag has a name that clearly signals it's an unsafe
> thing to do.
>
> I propose to replace the existing (as of 3.7.0b1) hash= keyword for the
> @dataclass decorator with a simpler flag named unsafe_hash=. This would be
> a simple bool (not a tri-state flag like the current hash=None|False|True).
> The default would be False, and the behavior then would be to add a hash
> function automatically only if it's safe (using the same rules as for
> hash=None currently). With unsafe_hash=True, a hash function would always
> be generated that takes all fields into account except those declared using
> field(hash=False). If there's already a `def __hash__` in the function I
> don't care what it does, maybe it should raise rather than quietly doing
> nothing or quietly overwriting it.
>
> Here's my use case.
>
> A frozen class requires a lot of discipline, since you have to compute the
> values of all fields before calling the constructor. A mutable class allows
> other initialization patterns, e.g. manually setting some fields after the
> instance has been constructed, or having a separate non-dunder init()
> method. There may be good reasons for using these patterns, e.g. the object
> may be part of a cycle (e.g. parent/child links in a tree). Or you may just
> use one of these patterns because you're a pretty casual coder. Or you're
> modeling something external.
>
> My point is that once you have one of those patterns in place, changing
> your code to avoid them may be difficult. And yet your code may treat the
> objects as essentially immutable after the initialization phase (e.g. a
> parse tree). So if you create a dataclass and start coding like that for a
> while, and much later you need to put one of these into a set or use it as
> a dict key, switching to frozen=True may not be a quick option. And writing
> a __hash__ method by hand may feel like a lot of busywork. So this is where
> [unsafe_]hash=True would come in handy.
>
> I think naming the flag unsafe_hash should take away most objections,
> since it will be clear that this is not a safe thing to do. People who
> don't understand the danger are likely to copy a worse solution from
> StackOverflow anyway. The docs can point to frozen=True and explain the
> danger.
>
> --
> --Guido van Rossum (python.org/~guido)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180205/0ae2e3eb/attachment-0001.html>

From chris.barker at noaa.gov  Mon Feb  5 01:09:52 2018
From: chris.barker at noaa.gov (Chris Barker)
Date: Mon, 5 Feb 2018 00:09:52 -0600
Subject: [Python-Dev] Immutability vs. hashability
In-Reply-To: <CADiSq7fezK+tgHNrmw1VVR-STM5nc+pOAPUho9J3MCa_f3RPAA@mail.gmail.com>
References: <CAP7+vJLQP29XTRVYSYGC7wS5RN9CF7CDu+CTBJtrotaxbNedZw@mail.gmail.com>
 <CADiSq7fezK+tgHNrmw1VVR-STM5nc+pOAPUho9J3MCa_f3RPAA@mail.gmail.com>
Message-ID: <CALGmxEJCx=4=qhRnNY0=7oV35D5mx_eYb6M7-0d1TocaAecTRA@mail.gmail.com>

On Sun, Feb 4, 2018 at 7:54 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 5 February 2018 at 08:31, Guido van Rossum <guido at python.org> wrote:
> > On Sun, Feb 4, 2018 at 11:59 AM, Chris Barker - NOAA Federal
> > <chris.barker at noaa.gov> wrote:
> >>
> >> I think the folks that are concerned about this issue are quite right
> >> ? most Python users equate immutable and hashable?so the dataclass API
> >> should reflect that.
> >
> > Since they are *not* equivalent (consider a tuple containing a list) I'm
> not
> > at all convinced that any API in the core language should "reflect" this
> > misconception, depending on how you meant that.
>
> Lists are themselves mutable, and hence inherently unhashable.
>
> Tuples are themselves immutable, and hence hashable if their contents are.
>
> I interpret Chris's comment as saying that data classes should behave
> the same way that the builtin container types do:
>

pretty much, yes,

But a bit more detail -- I'm commenting on the API, not the capability -
that is, since users often equate hashable and immutability, they will
expect that if they say hash=True, then will get an immutable, and if they
say frozen=True, they will get something hashable (as long as the fields
are hashable, just like a tuple.

That is, even though these concepts are independent, the defaults shouldn't
reflect that.

It's the ability to ask the interpreter to guess what you mean
> "frozen=False, hash=True" that creates the likelihood of confusion.
>

Actually, I think if the user does explicitly specify: "frozen=False,
hash=True", then that's what they should get, and it's a pretty fragile
beast, but apparently there's enough of a use case for folks to want it,
and I don't think it's a confusing API.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180205/4eefbcc6/attachment.html>

From chris.barker at noaa.gov  Mon Feb  5 01:19:10 2018
From: chris.barker at noaa.gov (Chris Barker)
Date: Mon, 5 Feb 2018 00:19:10 -0600
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAGE7PNL8Ky1fuE6vbX7eDdHZXqmWcQRhQWBwGST60COiysRG8Q@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <CAGE7PNL8Ky1fuE6vbX7eDdHZXqmWcQRhQWBwGST60COiysRG8Q@mail.gmail.com>
Message-ID: <CALGmxE+5s3pwzda2cyQkYC1+emq-jdtH_gF+HoO11i5vg08yuQ@mail.gmail.com>

On Sun, Feb 4, 2018 at 11:57 PM, Gregory P. Smith <greg at krypto.org> wrote:

> +1 using unsafe_hash as a name addresses my concern.
>
mine too -- anyone surprised by using this deserves what they get :-)

-CHB


On Sun, Feb 4, 2018, 9:50 PM Guido van Rossum <guido at python.org> wrote:
>
>> Looks like this is turning into a major flamewar regardless of what I
>> say. :-(
>>
>> I really don't want to lose the ability to add a hash function to a
>> mutable dataclass by flipping a flag in the decorator. I'll explain below.
>> But I am fine if this flag has a name that clearly signals it's an unsafe
>> thing to do.
>>
>> I propose to replace the existing (as of 3.7.0b1) hash= keyword for the
>> @dataclass decorator with a simpler flag named unsafe_hash=. This would be
>> a simple bool (not a tri-state flag like the current hash=None|False|True).
>> The default would be False, and the behavior then would be to add a hash
>> function automatically only if it's safe (using the same rules as for
>> hash=None currently). With unsafe_hash=True, a hash function would always
>> be generated that takes all fields into account except those declared using
>> field(hash=False). If there's already a `def __hash__` in the function I
>> don't care what it does, maybe it should raise rather than quietly doing
>> nothing or quietly overwriting it.
>>
>> Here's my use case.
>>
>> A frozen class requires a lot of discipline, since you have to compute
>> the values of all fields before calling the constructor. A mutable class
>> allows other initialization patterns, e.g. manually setting some fields
>> after the instance has been constructed, or having a separate non-dunder
>> init() method. There may be good reasons for using these patterns, e.g. the
>> object may be part of a cycle (e.g. parent/child links in a tree). Or you
>> may just use one of these patterns because you're a pretty casual coder. Or
>> you're modeling something external.
>>
>> My point is that once you have one of those patterns in place, changing
>> your code to avoid them may be difficult. And yet your code may treat the
>> objects as essentially immutable after the initialization phase (e.g. a
>> parse tree). So if you create a dataclass and start coding like that for a
>> while, and much later you need to put one of these into a set or use it as
>> a dict key, switching to frozen=True may not be a quick option. And writing
>> a __hash__ method by hand may feel like a lot of busywork. So this is where
>> [unsafe_]hash=True would come in handy.
>>
>> I think naming the flag unsafe_hash should take away most objections,
>> since it will be clear that this is not a safe thing to do. People who
>> don't understand the danger are likely to copy a worse solution from
>> StackOverflow anyway. The docs can point to frozen=True and explain the
>> danger.
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
>> greg%40krypto.org
>>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> chris.barker%40noaa.gov
>
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180205/1c24ccaf/attachment.html>

From v+python at g.nevcal.com  Mon Feb  5 02:28:03 2018
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Sun, 4 Feb 2018 23:28:03 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
Message-ID: <d46be51a-c33d-1bf2-4916-386c2761c076@g.nevcal.com>

On 2/4/2018 9:49 PM, Guido van Rossum wrote:
> A frozen class requires a lot of discipline, since you have to compute 
> the values of all fields before calling the constructor. A mutable 
> class allows other initialization patterns, e.g. manually setting some 
> fields after the instance has been constructed, or having a separate 
> non-dunder init() method. There may be good reasons for using these 
> patterns, e.g. the object may be part of a cycle (e.g. parent/child 
> links in a tree). Or you may just use one of these patterns because 
> you're a pretty casual coder. Or you're modeling something external.
>
> My point is that once you have one of those patterns in place, 
> changing your code to avoid them may be difficult. And yet your code 
> may treat the objects as essentially immutable after the 
> initialization phase (e.g. a parse tree). So if you create a dataclass 
> and start coding like that for a while, and much later you need to put 
> one of these into a set or use it as a dict key, switching to 
> frozen=True may not be a quick option. And writing a __hash__ method 
> by hand may feel like a lot of busywork. So this is where 
> [unsafe_]hash=True would come in handy.
>
> I think naming the flag unsafe_hash should take away most objections, 
> since it will be clear that this is not a safe thing to do. People who 
> don't understand the danger are likely to copy a worse solution from 
> StackOverflow anyway. The docs can point to frozen=True and explain 
> the danger.

This is an interesting use case. I haven't got the internals knowledge 
to know just how just different mutable and immutable classes and 
objects are under the hood. But this use case makes me wonder if, even 
at the cost of some performance that "normal" immutable classes and 
objects might obtain, if it would be possible to use the various 
undisciplined initialization patterns as desired, followed by as 
declaration "This OBJECT is now immutable" which would calculate its 
HASH value, and prevent future mutations of the object?

Yes, I'm aware that the decision for immutability has historically been 
done at the class level, not the object level, but in my ignorance of 
the internals, I wonder if that is necessary, for performance or more 
importantly, for other reasons.

And perhaps the implementation is internally almost like two classes, 
one mutable, and the other immutable, and the declaration would convert 
the object from one to the other.? But if I say more, I'd just be babbling.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180204/401cb7e5/attachment.html>

From njs at pobox.com  Mon Feb  5 03:11:35 2018
From: njs at pobox.com (Nathaniel Smith)
Date: Mon, 5 Feb 2018 00:11:35 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <d46be51a-c33d-1bf2-4916-386c2761c076@g.nevcal.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <d46be51a-c33d-1bf2-4916-386c2761c076@g.nevcal.com>
Message-ID: <CAPJVwBmGAvtRfUioSz7d79DVn6Wk4PrgJ8-m=aD_iHtrGOCuiw@mail.gmail.com>

On Sun, Feb 4, 2018 at 11:28 PM, Glenn Linderman <v+python at g.nevcal.com> wrote:
> This is an interesting use case. I haven't got the internals knowledge to
> know just how just different mutable and immutable classes and objects are
> under the hood. But this use case makes me wonder if, even at the cost of
> some performance that "normal" immutable classes and objects might obtain,
> if it would be possible to use the various undisciplined initialization
> patterns as desired, followed by as declaration "This OBJECT is now
> immutable" which would calculate its HASH value, and prevent future
> mutations of the object?

It would be technically possible to support something like

@dataclass(freezable=True)
class Foo:
    blah: int

foo = Foo()
# Initially, object is mutable, and hash(foo) raises an error
foo.blah = 1
assertRaises(hash, foo)

# This method is automatically generated for classes with freezable=True
foo.freeze()

# Now object is immutable, and hash(foo) is allowed
assertRaises(foo.__setattr__, "blah", 2)
hash(foo)

I don't know if it's worth the complexity, but I guess it would cover
at least some of the use cases Guido raised.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org

From v+python at g.nevcal.com  Mon Feb  5 03:54:49 2018
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 5 Feb 2018 00:54:49 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAPJVwBmGAvtRfUioSz7d79DVn6Wk4PrgJ8-m=aD_iHtrGOCuiw@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <d46be51a-c33d-1bf2-4916-386c2761c076@g.nevcal.com>
 <CAPJVwBmGAvtRfUioSz7d79DVn6Wk4PrgJ8-m=aD_iHtrGOCuiw@mail.gmail.com>
Message-ID: <beddd271-bd3d-c5e8-f7f4-02685dc2e09a@g.nevcal.com>

On 2/5/2018 12:11 AM, Nathaniel Smith wrote:
> On Sun, Feb 4, 2018 at 11:28 PM, Glenn Linderman <v+python at g.nevcal.com> wrote:
>> This is an interesting use case. I haven't got the internals knowledge to
>> know just how just different mutable and immutable classes and objects are
>> under the hood. But this use case makes me wonder if, even at the cost of
>> some performance that "normal" immutable classes and objects might obtain,
>> if it would be possible to use the various undisciplined initialization
>> patterns as desired, followed by as declaration "This OBJECT is now
>> immutable" which would calculate its HASH value, and prevent future
>> mutations of the object?
> It would be technically possible to support something like
>
> @dataclass(freezable=True)
> class Foo:
>      blah: int
>
> foo = Foo()
> # Initially, object is mutable, and hash(foo) raises an error
> foo.blah = 1
> assertRaises(hash, foo)
>
> # This method is automatically generated for classes with freezable=True
> foo.freeze()
>
> # Now object is immutable, and hash(foo) is allowed
> assertRaises(foo.__setattr__, "blah", 2)
> hash(foo)
>
> I don't know if it's worth the complexity, but I guess it would cover
> at least some of the use cases Guido raised.
>
> -n
>
Thanks, Nathaniel, for confirming that what I was suggesting is not 
impossible, even if it turns out to be undesirable for some reason, or 
unwanted by anyone else. But I have encountered a subset of the use 
cases Guido mentioned, and had to make a 2nd class to gather/hold the 
values of the eventual immutable class, before I could make it, because 
pieces of the data for the class values were obtained from different 
sources at different times. Once all collected, then the immutability 
could be obtained, the rest of the processing performed. Thrashes the 
allocator pretty well doing it that way, but the job got done.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180205/4feb9e3d/attachment.html>

From tjreedy at udel.edu  Mon Feb  5 04:25:28 2018
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 5 Feb 2018 04:25:28 -0500
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <d46be51a-c33d-1bf2-4916-386c2761c076@g.nevcal.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <d46be51a-c33d-1bf2-4916-386c2761c076@g.nevcal.com>
Message-ID: <p597qb$dp3$1@blaine.gmane.org>

On 2/5/2018 2:28 AM, Glenn Linderman wrote:

> This is an interesting use case. I haven't got the internals knowledge 
> to know just how just different mutable and immutable classes and 
> objects are under the hood.

I believe there is no internal difference.  An object is immutable if 
there is not way to mutate it with Python code that not poke into 
internals, such as one can do with ctypes or 3rd party extensions. 
Numbers and strings have no mutation methods, including no .__init__.

A tuple is a fixed sequence of objects and has no .__init__.  But if any 
object in a tuple is mutable, then the tuple is.  But the tuple does not 
know its status, and there is no 'is_mutable' function. However, 
tuple.__hash__ calls the .__hash__ method of each object and if that is 
missing for one, tuple.__hash raises.

 >>> hash((1, 'a', []))
Traceback (most recent call last):
   File "<pyshell#0>", line 1, in <module>
     hash((1, 'a', []))
TypeError: unhashable type: 'list'

The built-in immutable objects are mutated from their initial blank 
values in the C code of their .__new__ methods.  So they are only 
'immutable' once constructed.  Guido pointed out that users constructing 
objects in Python code might reasonably do so other than only with 
.__new__, but still want to treat the object as immutable once constructed.

In Lisp, for instance, lists are actually trees.  To be immutable, they 
can only be singly linked and must be constructed from leaf nodes to the 
root (or head).  Python programmers should be able to link in both 
directions and start from the root, and still treat the result as frozen 
and hashable.

> But this use case makes me wonder if, even 
> at the cost of some performance that "normal" immutable classes and 
> objects might obtain, if it would be possible to use the various 
> undisciplined initialization patterns as desired, followed by as 
> declaration "This OBJECT is now immutable" which would calculate its 
> HASH value, and prevent future mutations of the object?

Something like this has been proposed, at least for dicts, and rejected.

-- 
Terry Jan Reedy


From solipsis at pitrou.net  Mon Feb  5 08:48:37 2018
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 5 Feb 2018 14:48:37 +0100
Subject: [Python-Dev] Immutability vs. hashability
References: <CAP7+vJLQP29XTRVYSYGC7wS5RN9CF7CDu+CTBJtrotaxbNedZw@mail.gmail.com>
Message-ID: <20180205144837.79afa08b@fsol>

On Sun, 4 Feb 2018 14:31:06 -0800
Guido van Rossum <guido at python.org> wrote:
> On Sun, Feb 4, 2018 at 11:59 AM, Chris Barker - NOAA Federal <
> chris.barker at noaa.gov> wrote:  
> 
> > I think the folks that are concerned about this issue are quite right
> > ? most Python users equate immutable and hashable?so the dataclass API
> > should reflect that.
> >  
> 
> Since they are *not* equivalent (consider a tuple containing a list) I'm
> not at all convinced that any API in the core language should "reflect"
> this misconception, depending on how you meant that.

+1 from me.

Regards

Antoine.



From ncoghlan at gmail.com  Mon Feb  5 09:28:25 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 6 Feb 2018 00:28:25 +1000
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
Message-ID: <CADiSq7cHfaRbM2Xk0d3D+4yVfmWfSgTq2r46vVPF_QsEbYS+tg@mail.gmail.com>

On 5 February 2018 at 15:49, Guido van Rossum <guido at python.org> wrote:
> My point is that once you have one of those patterns in place, changing your
> code to avoid them may be difficult. And yet your code may treat the objects
> as essentially immutable after the initialization phase (e.g. a parse tree).
> So if you create a dataclass and start coding like that for a while, and
> much later you need to put one of these into a set or use it as a dict key,
> switching to frozen=True may not be a quick option. And writing a __hash__
> method by hand may feel like a lot of busywork. So this is where
> [unsafe_]hash=True would come in handy.
>
> I think naming the flag unsafe_hash should take away most objections, since
> it will be clear that this is not a safe thing to do. People who don't
> understand the danger are likely to copy a worse solution from StackOverflow
> anyway. The docs can point to frozen=True and explain the danger.

Aye, calling the flag unsafe_hash would convert me from -1 to -0.

The remaining -0 is because I think there's a different and more
robust way to tackle your example use case:

    # Mutable initialization phase
    >>> from dataclasses import dataclass
    >>> @dataclass
    ... class Example:
    ...     a: int
    ...     b: int
    ...
    >>> c = Example(None, None)
    >>> c
    Example(a=None, b=None)
    >>> c.a = 1
    >>> c.b = 2
    >>> c
    Example(a=1, b=2)


    # Frozen usage phase
    >>> @dataclass(frozen=True)
    ... class LockedExample(Example):
    ...     pass
    ...
    >>> c.__class__ = LockedExample
    >>> c.a = 1
    Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
     File "/home/ncoghlan/devel/cpython/Lib/dataclasses.py", line 448,
in _frozen_setattr
       raise FrozenInstanceError(f'cannot assign to field {name!r}')
    dataclasses.FrozenInstanceError: cannot assign to field 'a'
    >>> c.b = 2
    Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
     File "/home/ncoghlan/devel/cpython/Lib/dataclasses.py", line 448,
in _frozen_setattr
       raise FrozenInstanceError(f'cannot assign to field {name!r}')
    dataclasses.FrozenInstanceError: cannot assign to field 'b'
    >>> hash(c)
    3713081631934410656

The gist of that approach is to assume that there will be *somewhere*
in the code where it's possible to declare the construction of the
instance "complete", and flip the nominal class over to the frozen
subclass to make further mutation unlikely, even though the true
underlying type is still the mutable version.

That said, if we do provide "unsafe_hash", then the documentation for
that flag becomes a place where we can explicitly suggest using a
frozen subclass instead.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From kirillbalunov at gmail.com  Mon Feb  5 04:59:29 2018
From: kirillbalunov at gmail.com (Kirill Balunov)
Date: Mon, 5 Feb 2018 12:59:29 +0300
Subject: [Python-Dev] Dataclasses and correct hashability
Message-ID: <CABwCVQDRPo98rqxDXdEa8OXJJNJBp=1=3VgmV9fcc8EOo1-_zg@mail.gmail.com>

>
> On Sun, Feb 4, 2018, 9:50 PM Guido van Rossum <guido at python.org <https://mail.python.org/mailman/listinfo/python-dev>> wrote:
>
> Looks like this is turning into a major flamewar regardless of what I say.
> :-(
> I really don't want to lose the ability to add a hash function to a
> mutable dataclass by flipping a flag in the decorator. I'll explain below.
> But I am fine if this flag has a name that clearly signals it's an unsafe
> thing to do.
>
> I propose to replace the existing (as of 3.7.0b1) hash= keyword for the
> @dataclass decorator with a simpler flag named unsafe_hash=. This would be
> a simple bool (not a tri-state flag like the current hash=None|False|True).
> The default would be False, and the behavior then would be to add a hash
> function automatically only if it's safe (using the same rules as for
> hash=None currently). With unsafe_hash=True, a hash function would always
> be generated that takes all fields into account except those declared using
> field(hash=False). If there's already a `def __hash__` in the function I
> don't care what it does, maybe it should raise rather than quietly doing
> nothing or quietly overwriting it.
>
> Here's my use case.
>
>
May be it is better to provide a special purpose function
`make_unsafe_hash` in
dataclass module which will patch a dataclass, instead of to clutter
@dataclass
API with arguments which are rather special case.

This `unsafe_hash` argument will constantly raise questions among ordinary
users
like me, and will be possibly considered as a non-obvious design - there is
a
public API but it is somehow unsafe. On the other hand, with a function,
when
the user asks how to make a `frozen=False` dataclass hashable, you can
suggest
to use this `make_unsafe_hash` function with all its cautions in its docs
or to try to
implement __hash__ by yourself.

Also taking into account the Python approach for backward compatibility it
is
better to stick with function and if it will be usefull to add a
`unsafe_hash`
argument in Python 3.8. It is easier to add later than to deprecate in the
future.

With kind regards,
-gdg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180205/91492abd/attachment.html>

From guido at python.org  Mon Feb  5 12:46:05 2018
From: guido at python.org (Guido van Rossum)
Date: Mon, 5 Feb 2018 09:46:05 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CADiSq7cHfaRbM2Xk0d3D+4yVfmWfSgTq2r46vVPF_QsEbYS+tg@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <CADiSq7cHfaRbM2Xk0d3D+4yVfmWfSgTq2r46vVPF_QsEbYS+tg@mail.gmail.com>
Message-ID: <CAP7+vJK33Xw0bpxrXuU8ejpGmtQUBQWQYnBeeUeHvGYqEcAB_w@mail.gmail.com>

I'm sorry, but a solution that requires __class__ assignment is way too
fragile for my taste.

On Mon, Feb 5, 2018 at 6:28 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 5 February 2018 at 15:49, Guido van Rossum <guido at python.org> wrote:
> > My point is that once you have one of those patterns in place, changing
> your
> > code to avoid them may be difficult. And yet your code may treat the
> objects
> > as essentially immutable after the initialization phase (e.g. a parse
> tree).
> > So if you create a dataclass and start coding like that for a while, and
> > much later you need to put one of these into a set or use it as a dict
> key,
> > switching to frozen=True may not be a quick option. And writing a
> __hash__
> > method by hand may feel like a lot of busywork. So this is where
> > [unsafe_]hash=True would come in handy.
> >
> > I think naming the flag unsafe_hash should take away most objections,
> since
> > it will be clear that this is not a safe thing to do. People who don't
> > understand the danger are likely to copy a worse solution from
> StackOverflow
> > anyway. The docs can point to frozen=True and explain the danger.
>
> Aye, calling the flag unsafe_hash would convert me from -1 to -0.
>
> The remaining -0 is because I think there's a different and more
> robust way to tackle your example use case:
>
>     # Mutable initialization phase
>     >>> from dataclasses import dataclass
>     >>> @dataclass
>     ... class Example:
>     ...     a: int
>     ...     b: int
>     ...
>     >>> c = Example(None, None)
>     >>> c
>     Example(a=None, b=None)
>     >>> c.a = 1
>     >>> c.b = 2
>     >>> c
>     Example(a=1, b=2)
>
>
>     # Frozen usage phase
>     >>> @dataclass(frozen=True)
>     ... class LockedExample(Example):
>     ...     pass
>     ...
>     >>> c.__class__ = LockedExample
>     >>> c.a = 1
>     Traceback (most recent call last):
>      File "<stdin>", line 1, in <module>
>      File "/home/ncoghlan/devel/cpython/Lib/dataclasses.py", line 448,
> in _frozen_setattr
>        raise FrozenInstanceError(f'cannot assign to field {name!r}')
>     dataclasses.FrozenInstanceError: cannot assign to field 'a'
>     >>> c.b = 2
>     Traceback (most recent call last):
>      File "<stdin>", line 1, in <module>
>      File "/home/ncoghlan/devel/cpython/Lib/dataclasses.py", line 448,
> in _frozen_setattr
>        raise FrozenInstanceError(f'cannot assign to field {name!r}')
>     dataclasses.FrozenInstanceError: cannot assign to field 'b'
>     >>> hash(c)
>     3713081631934410656
>
> The gist of that approach is to assume that there will be *somewhere*
> in the code where it's possible to declare the construction of the
> instance "complete", and flip the nominal class over to the frozen
> subclass to make further mutation unlikely, even though the true
> underlying type is still the mutable version.
>
> That said, if we do provide "unsafe_hash", then the documentation for
> that flag becomes a place where we can explicitly suggest using a
> frozen subclass instead.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180205/b4050320/attachment.html>

From guido at python.org  Mon Feb  5 12:47:09 2018
From: guido at python.org (Guido van Rossum)
Date: Mon, 5 Feb 2018 09:47:09 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CABwCVQDRPo98rqxDXdEa8OXJJNJBp=1=3VgmV9fcc8EOo1-_zg@mail.gmail.com>
References: <CABwCVQDRPo98rqxDXdEa8OXJJNJBp=1=3VgmV9fcc8EOo1-_zg@mail.gmail.com>
Message-ID: <CAP7+vJK4s8tnUxC=7B+MEAqvwWm+d4+rWJyiy3v7V++UD65yjA@mail.gmail.com>

If there's going to be an API for it, it should be in the class, not
something that mutates the class afterwards.

On Mon, Feb 5, 2018 at 1:59 AM, Kirill Balunov <kirillbalunov at gmail.com>
wrote:

> On Sun, Feb 4, 2018, 9:50 PM Guido van Rossum <guido at python.org <https://mail.python.org/mailman/listinfo/python-dev>> wrote:
>>
>> Looks like this is turning into a major flamewar regardless of what I say.
>> :-(
>> I really don't want to lose the ability to add a hash function to a
>> mutable dataclass by flipping a flag in the decorator. I'll explain below.
>> But I am fine if this flag has a name that clearly signals it's an unsafe
>> thing to do.
>>
>> I propose to replace the existing (as of 3.7.0b1) hash= keyword for the
>> @dataclass decorator with a simpler flag named unsafe_hash=. This would be
>> a simple bool (not a tri-state flag like the current hash=None|False|True).
>> The default would be False, and the behavior then would be to add a hash
>> function automatically only if it's safe (using the same rules as for
>> hash=None currently). With unsafe_hash=True, a hash function would always
>> be generated that takes all fields into account except those declared using
>> field(hash=False). If there's already a `def __hash__` in the function I
>> don't care what it does, maybe it should raise rather than quietly doing
>> nothing or quietly overwriting it.
>>
>> Here's my use case.
>>
>>
> May be it is better to provide a special purpose function
> `make_unsafe_hash` in
> dataclass module which will patch a dataclass, instead of to clutter
> @dataclass
> API with arguments which are rather special case.
>
> This `unsafe_hash` argument will constantly raise questions among ordinary
> users
> like me, and will be possibly considered as a non-obvious design - there
> is a
> public API but it is somehow unsafe. On the other hand, with a function,
> when
> the user asks how to make a `frozen=False` dataclass hashable, you can
> suggest
> to use this `make_unsafe_hash` function with all its cautions in its docs
> or to try to
> implement __hash__ by yourself.
>
> Also taking into account the Python approach for backward compatibility it
> is
> better to stick with function and if it will be usefull to add a
> `unsafe_hash`
> argument in Python 3.8. It is easier to add later than to deprecate in the
> future.
>
> With kind regards,
> -gdg
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180205/708154cd/attachment.html>

From mertz at gnosis.cx  Mon Feb  5 13:50:21 2018
From: mertz at gnosis.cx (David Mertz)
Date: Mon, 5 Feb 2018 10:50:21 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CALGmxE+5s3pwzda2cyQkYC1+emq-jdtH_gF+HoO11i5vg08yuQ@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <CAGE7PNL8Ky1fuE6vbX7eDdHZXqmWcQRhQWBwGST60COiysRG8Q@mail.gmail.com>
 <CALGmxE+5s3pwzda2cyQkYC1+emq-jdtH_gF+HoO11i5vg08yuQ@mail.gmail.com>
Message-ID: <CAEbHw4aoY07dit-5vWVsDZHuzX6Z=6zigFbO9rA9bX3-R4LXoA@mail.gmail.com>

Absolutely I agree. 'unsafe_hash' as a name is clear warning to users.

On Feb 4, 2018 10:43 PM, "Chris Barker" <chris.barker at noaa.gov> wrote:



On Sun, Feb 4, 2018 at 11:57 PM, Gregory P. Smith <greg at krypto.org> wrote:

> +1 using unsafe_hash as a name addresses my concern.
>
mine too -- anyone surprised by using this deserves what they get :-)

-CHB


On Sun, Feb 4, 2018, 9:50 PM Guido van Rossum <guido at python.org> wrote:
>
>> Looks like this is turning into a major flamewar regardless of what I
>> say. :-(
>>
>> I really don't want to lose the ability to add a hash function to a
>> mutable dataclass by flipping a flag in the decorator. I'll explain below.
>> But I am fine if this flag has a name that clearly signals it's an unsafe
>> thing to do.
>>
>> I propose to replace the existing (as of 3.7.0b1) hash= keyword for the
>> @dataclass decorator with a simpler flag named unsafe_hash=. This would be
>> a simple bool (not a tri-state flag like the current hash=None|False|True).
>> The default would be False, and the behavior then would be to add a hash
>> function automatically only if it's safe (using the same rules as for
>> hash=None currently). With unsafe_hash=True, a hash function would always
>> be generated that takes all fields into account except those declared using
>> field(hash=False). If there's already a `def __hash__` in the function I
>> don't care what it does, maybe it should raise rather than quietly doing
>> nothing or quietly overwriting it.
>>
>> Here's my use case.
>>
>> A frozen class requires a lot of discipline, since you have to compute
>> the values of all fields before calling the constructor. A mutable class
>> allows other initialization patterns, e.g. manually setting some fields
>> after the instance has been constructed, or having a separate non-dunder
>> init() method. There may be good reasons for using these patterns, e.g. the
>> object may be part of a cycle (e.g. parent/child links in a tree). Or you
>> may just use one of these patterns because you're a pretty casual coder. Or
>> you're modeling something external.
>>
>> My point is that once you have one of those patterns in place, changing
>> your code to avoid them may be difficult. And yet your code may treat the
>> objects as essentially immutable after the initialization phase (e.g. a
>> parse tree). So if you create a dataclass and start coding like that for a
>> while, and much later you need to put one of these into a set or use it as
>> a dict key, switching to frozen=True may not be a quick option. And writing
>> a __hash__ method by hand may feel like a lot of busywork. So this is where
>> [unsafe_]hash=True would come in handy.
>>
>> I think naming the flag unsafe_hash should take away most objections,
>> since it will be clear that this is not a safe thing to do. People who
>> don't understand the danger are likely to copy a worse solution from
>> StackOverflow anyway. The docs can point to frozen=True and explain the
>> danger.
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/greg%
>> 40krypto.org
>>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/chris.
> barker%40noaa.gov
>
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/
mertz%40gnosis.cx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180205/7ff782cb/attachment.html>

From kirillbalunov at gmail.com  Mon Feb  5 14:09:47 2018
From: kirillbalunov at gmail.com (Kirill Balunov)
Date: Mon, 5 Feb 2018 22:09:47 +0300
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAP7+vJK4s8tnUxC=7B+MEAqvwWm+d4+rWJyiy3v7V++UD65yjA@mail.gmail.com>
References: <CABwCVQDRPo98rqxDXdEa8OXJJNJBp=1=3VgmV9fcc8EOo1-_zg@mail.gmail.com>
 <CAP7+vJK4s8tnUxC=7B+MEAqvwWm+d4+rWJyiy3v7V++UD65yjA@mail.gmail.com>
Message-ID: <CABwCVQAUU9O4WQk09py4mLr47Mv4tCGTuuXRyCey_xr5_KRq0A@mail.gmail.com>

2018-02-05 20:47 GMT+03:00 Guido van Rossum <guido at python.org>:

> If there's going to be an API for it, it should be in the class, not
> something that mutates the class afterwards.
>


I apologize and don't want to make unnecessary noise. But the already
selected design with decorator @dataclass implies that it will mutate
the freshly created class (which in its turn already limits some
possibilities), or I've missed something? If you meant that everything
should be defined in one place, then I basically understand your desire as
the least of two evils.

With kind regards,
-gdg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180205/20cfb643/attachment.html>

From guido at python.org  Mon Feb  5 14:12:36 2018
From: guido at python.org (Guido van Rossum)
Date: Mon, 5 Feb 2018 11:12:36 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CABwCVQAUU9O4WQk09py4mLr47Mv4tCGTuuXRyCey_xr5_KRq0A@mail.gmail.com>
References: <CABwCVQDRPo98rqxDXdEa8OXJJNJBp=1=3VgmV9fcc8EOo1-_zg@mail.gmail.com>
 <CAP7+vJK4s8tnUxC=7B+MEAqvwWm+d4+rWJyiy3v7V++UD65yjA@mail.gmail.com>
 <CABwCVQAUU9O4WQk09py4mLr47Mv4tCGTuuXRyCey_xr5_KRq0A@mail.gmail.com>
Message-ID: <CAP7+vJ+Rt4RtTCPX27_5HX4rzf4phRq9ceqc4ewnq3YRQZ5aeQ@mail.gmail.com>

Yes, that's what I meant -- "afterwards" meaning after the @dataclass
decorator is applied.

On Mon, Feb 5, 2018 at 11:09 AM, Kirill Balunov <kirillbalunov at gmail.com>
wrote:

>
> 2018-02-05 20:47 GMT+03:00 Guido van Rossum <guido at python.org>:
>
>> If there's going to be an API for it, it should be in the class, not
>> something that mutates the class afterwards.
>>
>
>
> I apologize and don't want to make unnecessary noise. But the already
> selected design with decorator @dataclass implies that it will mutate
> the freshly created class (which in its turn already limits some
> possibilities), or I've missed something? If you meant that everything
> should be defined in one place, then I basically understand your desire as
> the least of two evils.
>
> With kind regards,
> -gdg
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180205/c8b6b968/attachment.html>

From python-dev at mgmiller.net  Mon Feb  5 15:19:33 2018
From: python-dev at mgmiller.net (Mike Miller)
Date: Mon, 5 Feb 2018 12:19:33 -0800
Subject: [Python-Dev] [RELEASE] Python 3.7.0b1 is now available for
 testing
In-Reply-To: <9425596C-A92F-4B10-A8B7-98F4E827E8D0@python.org>
References: <9425596C-A92F-4B10-A8B7-98F4E827E8D0@python.org>
Message-ID: <8cdb5984-0148-1853-e9ac-9dfcbc6e76a4@mgmiller.net>


On 2018-01-31 17:34, Ned Deily wrote:
> Please see "What?s New In Python 3.7" for more information.
> Additional documentation for these features and for other changes
> will be provided during the beta phase.
> 
> https://docs.python.org/3.7/whatsnew/3.7.html


I see that the new classmethod fromisoformat in datetime arrived (thanks all) 
but it hasn't yet received a blurb in the what's new page.

https://docs.python.org/3.7/library/datetime.html#datetime.datetime.fromisoformat

-Mike

From paul at ganssle.io  Mon Feb  5 16:06:17 2018
From: paul at ganssle.io (Paul G)
Date: Mon, 5 Feb 2018 16:06:17 -0500
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAP7+vJ+Rt4RtTCPX27_5HX4rzf4phRq9ceqc4ewnq3YRQZ5aeQ@mail.gmail.com>
References: <CABwCVQDRPo98rqxDXdEa8OXJJNJBp=1=3VgmV9fcc8EOo1-_zg@mail.gmail.com>
 <CAP7+vJK4s8tnUxC=7B+MEAqvwWm+d4+rWJyiy3v7V++UD65yjA@mail.gmail.com>
 <CABwCVQAUU9O4WQk09py4mLr47Mv4tCGTuuXRyCey_xr5_KRq0A@mail.gmail.com>
 <CAP7+vJ+Rt4RtTCPX27_5HX4rzf4phRq9ceqc4ewnq3YRQZ5aeQ@mail.gmail.com>
Message-ID: <94a8ddd0-e54f-8b6c-713f-de6974e386e7@ganssle.io>

I don't think it matters so much whether you are stacking two decorators or a single decorator, but would an @add_unsafe_hash decorator be useful for anything *except* data classes? If not, then there's no point in having a *second* decorator that can *only* modify the first one - particularly considering @dataclass actually takes arguments.

On 02/05/2018 02:12 PM, Guido van Rossum wrote:
> Yes, that's what I meant -- "afterwards" meaning after the @dataclass
> decorator is applied.
> 
> On Mon, Feb 5, 2018 at 11:09 AM, Kirill Balunov <kirillbalunov at gmail.com>
> wrote:
> 
>>
>> 2018-02-05 20:47 GMT+03:00 Guido van Rossum <guido at python.org>:
>>
>>> If there's going to be an API for it, it should be in the class, not
>>> something that mutates the class afterwards.
>>>
>>
>>
>> I apologize and don't want to make unnecessary noise. But the already
>> selected design with decorator @dataclass implies that it will mutate
>> the freshly created class (which in its turn already limits some
>> possibilities), or I've missed something? If you meant that everything
>> should be defined in one place, then I basically understand your desire as
>> the least of two evils.
>>
>> With kind regards,
>> -gdg
>>
> 
> 
> 
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/paul%40ganssle.io
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180205/141c1146/attachment.sig>

From matt.priest at intel.com  Mon Feb  5 17:41:31 2018
From: matt.priest at intel.com (Priest, Matt)
Date: Mon, 5 Feb 2018 22:41:31 +0000
Subject: [Python-Dev] libxml2 installation/binding issue
Message-ID: <68B69E4C07B04A488BA8528DF70CA165844150BC@fmsmsx116.amr.corp.intel.com>

Hello,

I am not sure if this is the correct place to post an issue/question like this, but here goes...

I've successfully (?) installed Python 3.6.4 and libxml2, with the ultimate goal of installing GTK+ 3.22.0.
However, I'm running into this error:


python3
Python 3.6.4 (default, Feb  5 2018, 13:28:04)
[GCC 4.7.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import libxml2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib/python3.6/site-packages/libxml2.py", line 1, in <module>
    import libxml2mod
ImportError: /nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib/python3.6/site-packages/libxml2mod.so: undefined symbol: _PyVerify_fd


Here are the details on the version, cflags, and ldflags.
python3 --version ;
Python 3.6.4
python3-config --cflags
-I/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/include/python3.6m
-I/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/include/python3.6m
-Wno-unused-result
-Wsign-compare
-fPIC -DNDEBUG
-g
-fwrapv
-O3
-Wall
-Wstrict-prototypes

python3-config -ldflags;
-L/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib/python3.6/config-3.6m-x86_64-linux-gnu
-L/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib
-lpython3.6m
-lpthread
-ldl
-lutil
-lrt
-lm
-Xlinker
-export-dynamic

Anyhelp or hint would be appreciated...



Matt

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180205/df23412c/attachment.html>

From ethan at ethanhs.me  Mon Feb  5 17:55:21 2018
From: ethan at ethanhs.me (Ethan Smith)
Date: Mon, 5 Feb 2018 14:55:21 -0800
Subject: [Python-Dev] libxml2 installation/binding issue
In-Reply-To: <68B69E4C07B04A488BA8528DF70CA165844150BC@fmsmsx116.amr.corp.intel.com>
References: <68B69E4C07B04A488BA8528DF70CA165844150BC@fmsmsx116.amr.corp.intel.com>
Message-ID: <CAMzjp8G7UJyv=Q6=WAk+-n8YCRZaPe_-ZhW861rczKWwzerUvg@mail.gmail.com>

This list is for the discussion of development *of* Python. For discussion
of development *with* Python, you want python-list.

On Mon, Feb 5, 2018 at 2:41 PM, Priest, Matt <matt.priest at intel.com> wrote:

> Hello,
>
>
>
> I am not sure if this is the correct place to post an issue/question like
> this, but here goes?
>
>
>
> I?ve successfully (?) installed Python 3.6.4 and libxml2, with the
> ultimate goal of installing GTK+ 3.22.0.
>
> However, I?m running into this error:
>
>
>
>
>
> python3
>
> Python 3.6.4 (default, Feb  5 2018, 13:28:04)
>
> [GCC 4.7.2] on linux
>
> Type "help", "copyright", "credits" or "license" for more information.
>
> >>> import libxml2
>
> Traceback (most recent call last):
>
>   File "<stdin>", line 1, in <module>
>
>   File "/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/
> development/sfwr/lib/python3.6/site-packages/libxml2.py", line 1, in
> <module>
>
>     import libxml2mod
>
> ImportError: /nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/
> development/sfwr/lib/python3.6/site-packages/libxml2mod.so: undefined
> symbol: _PyVerify_fd
>
>
>
>
>
> Here are the details on the version, cflags, and ldflags.
>
> python3 --version ;
>
> Python 3.6.4
>
> python3-config --cflags
>
> -I/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/
> development/sfwr/include/python3.6m
>
> -I/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/
> development/sfwr/include/python3.6m
>
> -Wno-unused-result
>
> -Wsign-compare
>
> -fPIC -DNDEBUG
>
> -g
>
> -fwrapv
>
> -O3
>
> -Wall
>
> -Wstrict-prototypes
>
>
>
> python3-config ?ldflags;
>
> -L/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/
> development/sfwr/lib/python3.6/config-3.6m-x86_64-linux-gnu
>
> -L/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib
>
> -lpython3.6m
>
> -lpthread
>
> -ldl
>
> -lutil
>
> -lrt
>
> -lm
>
> -Xlinker
>
> -export-dynamic
>
>
>
> Anyhelp or hint would be appreciated?
>
>
>
>
>
>
>
> Matt
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> ethan%40ethanhs.me
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180205/25c8f347/attachment.html>

From ncoghlan at gmail.com  Mon Feb  5 17:56:40 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 6 Feb 2018 08:56:40 +1000
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAP7+vJK4s8tnUxC=7B+MEAqvwWm+d4+rWJyiy3v7V++UD65yjA@mail.gmail.com>
References: <CABwCVQDRPo98rqxDXdEa8OXJJNJBp=1=3VgmV9fcc8EOo1-_zg@mail.gmail.com>
 <CAP7+vJK4s8tnUxC=7B+MEAqvwWm+d4+rWJyiy3v7V++UD65yjA@mail.gmail.com>
Message-ID: <CADiSq7d8y1R=vr1AQPwbp1LM6GHBmkv8xg6254sM0B1LTqnOGQ@mail.gmail.com>

On 6 February 2018 at 03:47, Guido van Rossum <guido at python.org> wrote:
> If there's going to be an API for it, it should be in the class, not
> something that mutates the class afterwards.

Something I realised after posting the __class__ setting idea is that
you can actually use a comparable trick to inject an unsafe hash from
the frozen version into the mutable version:

    >>> from dataclasses import dataclass
    >>> @dataclass
    ... class Example:
    ...     a: int
    ...     b: int
    ...
    >>> c = Example(1, 2)
    >>> hash(c)
    Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
    TypeError: unhashable type: 'Example'

    >>> @dataclass(frozen=True)
    ... class LockedExample(Example):
    ...     pass
    ...
    >>> Example.__hash__ = LockedExample.__hash__
    >>> hash(c)
    3713081631934410656

So "unsafe_hash=True" would just be a shorthand spelling of that which
skips creating the full frozen version of the class (and with the
explicit parameter, we can better document the risks of making
something hashable without also freezing it post-creation).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From steve at pearwood.info  Mon Feb  5 18:37:43 2018
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 6 Feb 2018 10:37:43 +1100
Subject: [Python-Dev] Immutability vs. hashability
In-Reply-To: <CAP7+vJJ_aBVvgFd1pzA6=AaMCFRO3wQCJwjujmeryK5+4fd-SQ@mail.gmail.com>
References: <CAP7+vJLQP29XTRVYSYGC7wS5RN9CF7CDu+CTBJtrotaxbNedZw@mail.gmail.com>
 <CADiSq7fezK+tgHNrmw1VVR-STM5nc+pOAPUho9J3MCa_f3RPAA@mail.gmail.com>
 <CAP7+vJJ_aBVvgFd1pzA6=AaMCFRO3wQCJwjujmeryK5+4fd-SQ@mail.gmail.com>
Message-ID: <20180205233743.GW26553@ando.pearwood.info>

On Sun, Feb 04, 2018 at 09:18:25PM -0800, Guido van Rossum wrote:

> The way I think of it generally is that immutability is a property of
> types, while hashability is a property of values.

That's a great way to look at it, thanks.


-- 
Steve

From levkivskyi at gmail.com  Mon Feb  5 18:54:10 2018
From: levkivskyi at gmail.com (Ivan Levkivskyi)
Date: Mon, 5 Feb 2018 23:54:10 +0000
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CADiSq7d8y1R=vr1AQPwbp1LM6GHBmkv8xg6254sM0B1LTqnOGQ@mail.gmail.com>
References: <CABwCVQDRPo98rqxDXdEa8OXJJNJBp=1=3VgmV9fcc8EOo1-_zg@mail.gmail.com>
 <CAP7+vJK4s8tnUxC=7B+MEAqvwWm+d4+rWJyiy3v7V++UD65yjA@mail.gmail.com>
 <CADiSq7d8y1R=vr1AQPwbp1LM6GHBmkv8xg6254sM0B1LTqnOGQ@mail.gmail.com>
Message-ID: <CAOMjWknn3PRCPHiUJc_+10SXDU37y84--cUdX0KwqB=f1ay3gg@mail.gmail.com>

Just wanted to add my 5 cents here. I am a bit surprised how people are
scared by adding `__hash__` to mutable classes.
>From my experience it is quite normal, I was always thinking about `hash()`
as hashing a _value_,
and never as hashing _identity_, if I would need the latter, there is a
different function for this, `id()`.
Moreover, I often did this in situations where dataclasses would be useful:
a class with several fields,
necessary dunders, and few other methods (record-like classes).
My motivation was mostly speed-up by memorization.
To be fair my use cases were mostly about some tree-like strictures, but
this is probably a coincidence.

FWIW it is not a super-safe practice, but neither super-dangerous.

--
Ivan



On 5 February 2018 at 22:56, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 6 February 2018 at 03:47, Guido van Rossum <guido at python.org> wrote:
> > If there's going to be an API for it, it should be in the class, not
> > something that mutates the class afterwards.
>
> Something I realised after posting the __class__ setting idea is that
> you can actually use a comparable trick to inject an unsafe hash from
> the frozen version into the mutable version:
>
>     >>> from dataclasses import dataclass
>     >>> @dataclass
>     ... class Example:
>     ...     a: int
>     ...     b: int
>     ...
>     >>> c = Example(1, 2)
>     >>> hash(c)
>     Traceback (most recent call last):
>      File "<stdin>", line 1, in <module>
>     TypeError: unhashable type: 'Example'
>
>     >>> @dataclass(frozen=True)
>     ... class LockedExample(Example):
>     ...     pass
>     ...
>     >>> Example.__hash__ = LockedExample.__hash__
>     >>> hash(c)
>     3713081631934410656
>
> So "unsafe_hash=True" would just be a shorthand spelling of that which
> skips creating the full frozen version of the class (and with the
> explicit parameter, we can better document the risks of making
> something hashable without also freezing it post-creation).
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> levkivskyi%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180205/8d7ed728/attachment.html>

From steve at pearwood.info  Mon Feb  5 20:17:56 2018
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 6 Feb 2018 12:17:56 +1100
Subject: [Python-Dev] Immutability vs. hashability
In-Reply-To: <CALGmxEJCx=4=qhRnNY0=7oV35D5mx_eYb6M7-0d1TocaAecTRA@mail.gmail.com>
References: <CAP7+vJLQP29XTRVYSYGC7wS5RN9CF7CDu+CTBJtrotaxbNedZw@mail.gmail.com>
 <CADiSq7fezK+tgHNrmw1VVR-STM5nc+pOAPUho9J3MCa_f3RPAA@mail.gmail.com>
 <CALGmxEJCx=4=qhRnNY0=7oV35D5mx_eYb6M7-0d1TocaAecTRA@mail.gmail.com>
Message-ID: <20180206011756.GX26553@ando.pearwood.info>

On Mon, Feb 05, 2018 at 12:09:52AM -0600, Chris Barker wrote:

> But a bit more detail -- I'm commenting on the API, not the capability -
> that is, since users often equate hashable and immutability, they will
> expect that if they say hash=True, then will get an immutable, and if they
> say frozen=True, they will get something hashable (as long as the fields
> are hashable, just like a tuple.
> 
> That is, even though these concepts are independent, the defaults shouldn't
> reflect that.

I'm not happy about the concept of pandering to the least capable, most 
ignorant programmers by baking a miscomprehension into an important 
standard library API. The fact is that mutability and hashability ARE 
independent qualities, and the API ought to reflect reality, not 
ignorance. That's why there are two separate switches, frozen and hash, 
not just one "frozen_hashable" switch.

(Things would be different if we just outright banned mutable+hashable, 
but I don't think anyone wants that.)

Fortunately, I also believe that the number of programmers who would 
fail to draw the right conclusion from the existence of separate 
switches will actually be pretty small in practice. The fact that there 
are two separate switches is a pretty big clue that mutability and 
hashability can be controlled separately.

I believe that the proposed API is much simpler to understand than your 
revision. We have:

- frozen and hash both default to False;
- if you explicitly set one, the other uses the default.

This corresponds to a very common, Pythonic pattern that nearly 
everyone is familiar with:

    def spam(frozen=False, hash=False):
        ...

which is easy to understand and easy to explain. Versus your proposal:

- if you set neither, then frozen and hash both default to False;
- but if you explicitly set one, the other uses True, namely the 
  opposite of the standard default.

which corresponds to something harder to describe and much less common:

    def spam(frozen=None, hash=None):
        if frozen is hash is None:
            frozen = hash = False
        elif frozen is None:
            frozen = True
        elif hash is None:
            hash = True
        ...

"frozen and hash default to True, unless neither are set, in which case 
they default to False."


Let's look at the two possible scenarios you are worried about:

(1) I set frozen=True thinking that makes the class hashable so I can 
use it in a set or hash. The first time I actually do so, I get an 
explicit and obvious TypeError. Problem solved.[1]

(2) I set hash=True thinking that makes the class frozen. This scenario 
is more problematic, because there's no explicit and obvious error when 
I get it wrong. Instead, my program could silently do the wrong thing if 
my instances are quietly mutated.

The first error is self-correcting, and so I believe that the second is 
the only one we should worry about. There are two questions:

- how much should we worry? (how often will this happen?);

- what do we do about it?

I think the answers ought to be, not much and nothing. Or *at most*, 
raise a *warning* when hash=True is set without also explicitly setting 
frozen. But even that seems unnecessary to me.

I think that the intersection of events needed for this to be a real 
problem will be fairly small:

- people using DataClasses;
- who want a frozen, hashable class;
- and believe that the two are equivalent;
- and who weren't clued in by the existence of separate switches;
- and set hash=True without frozen=True;
- and don't write a unit test to confirm that their data is immutable;
- and accidentally mutate an instance which they thought was immutable;
- in such a way as to cause a silent failure.

I don't think this is a failure mode that we need to be concerned with. 
We can't protect everyone from everything.



[1] Yes, I'm glossing over the possible annoyance if not difficulty of 
actually solving the problem: somebody has to raise a bug report, 
someone has to fix the bug which in principle could involve a lot of 
disruption, there should be regression tests and maybe a new release of 
the application, etc. But this is par for the course for *any* bug -- 
there's no need to imagine that this specific bug is so terrible that 
the standard library needs to protect programmers from the possibility 
of ordinary, run-of-the-mill bugs.

-- 
Steve

From steve at pearwood.info  Mon Feb  5 22:40:13 2018
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 6 Feb 2018 14:40:13 +1100
Subject: [Python-Dev] A minimal Python interpreter written in Python for
 experimenting with language changes
In-Reply-To: <trinity-a323a69c-0a35-4e20-bb66-a6aa1df5c361-1517654715564@3c-app-mailcom-lxa05>
References: <trinity-aabb133a-c890-476f-a962-be6dacc697d6-1517616090604@3c-app-mailcom-lxa14>
 <09cc01d39c96$9cb28910$d6179b30$@sdamon.com>
 <trinity-a323a69c-0a35-4e20-bb66-a6aa1df5c361-1517654715564@3c-app-mailcom-lxa05>
Message-ID: <20180206034013.GZ26553@ando.pearwood.info>

On Sat, Feb 03, 2018 at 11:45:15AM +0100, asrp wrote:

> > Can you give an example of how you would do that? I don't mean the 
> > mechanism used, I mean how would a developer implement a new syntactic 
> > feature. Suppose I wanted to add a new clause to for...else, let's say:
> >
> >     for ... :
> >        block
> >     otherwise:
> >        # runs only if the for-loop was empty
> > 
> > How would do I do that?
[...]
> If you tell me a bit more about the intended behaviour of "otherwise", 
> I'd be happy to do an example with that clause.


Here's a faked session showing the sort of thing I am referring to. 
(Note that this is just an example, not a proposal for a new language 
feature.)

for x in [1, 2, 3]:
    print(x)
otherwise:
    print("nothing there")


prints 1, 2, 3.

for x in []:
    print(x)
otherwise:
    print("nothing there")

prints "nothing there". In other words, the otherwise block runs if, and 
only if, the loop iterable is empty and the for block does NOT run.

Can you do something like that?



-- 
Steve

From rymg19 at gmail.com  Mon Feb  5 23:01:37 2018
From: rymg19 at gmail.com (rymg19 at gmail.com)
Date: Mon, 5 Feb 2018 20:01:37 -0800
Subject: [Python-Dev] A minimal Python interpreter written in Python for
 experimenting with language changes
In-Reply-To: <<trinity-a323a69c-0a35-4e20-bb66-a6aa1df5c361-1517654715564@3c-app-mailcom-lxa05>>
 <<09cc01d39c96$9cb28910$d6179b30$@sdamon.com>>
References: <<trinity-aabb133a-c890-476f-a962-be6dacc697d6-1517616090604@3c-app-mailcom-lxa14>>
 <<09cc01d39c96$9cb28910$d6179b30$@sdamon.com>>
Message-ID: <CAO41-mNg9=CGwAuCdy--KsiuHB8vT=9xoxyM8e+r=0qrzzp9yA@mail.gmail.com>

<begin plug>

https://refi64.com/posts/the-magic-of-rpython.html

Note that this was written back when I used "like" and "really" in every
sentence, and when I used to think that Python copied tuples (don't ask).

On Feb 3, 2018 at 10:05 AM, <asrp <asrp at email.com>> wrote:

> Are you aware of pypy?
>

Yes, but I do not know it well. It seems to be more focused on JIT
compiling with many of its functions written for special cases and I
did not find a way to make experimental changes easily.

I even tried (briefly) to have the main loop of this interpreter run
in RPython but didn't find enough information on which subset of
Python does RPython handle.


> Can you give an example of how you would do that? I don't mean the
> mechanism used, I mean how would a developer implement a new syntactic
> feature. Suppose I wanted to add a new clause to for...else, let's say:
>
>     for ... :
>        block
>     otherwise:
>        # runs only if the for-loop was empty
>
> How would do I do that?
>

Here's one way to do that. Sorry for the lack of doc for the moment.

To alter the language's grammar, you'd create a string with the new
grammar, parse it and set the result to pyterp.parser. Then, depending
on how the grammar was changed (which affects what AST is created),
you'd change the corresponding function to handle the new semantics.

I didn't quite understand what you mean by "was empty". However, I
noticed that "else" blocks after for are ignored in the current
implement. Here's an example session adding "else".

If you tell me a bit more about the intended behaviour of "otherwise",
I'd be happy to do an example with that clause.

    $ ipython -i test/python_repl.py
    p>> simport simple_ast
    p>> for i in [1, 2]:
    ...     print i
    ...
    1
    2
    p>> for i in [1, 2, 3]:
    ...     print i
    ... else:
    ...     print 100
    ...
    1
    2
    3
    p>> ^D

In the above for loop, the else block is ignored! I press Control-D to
exist the interpreter.

    In [1]: grammar = python_grammar.full_definition + python_grammar.extra

    In [2]: grammar += r"""
       ...: for_stmt = "for" {exprlist} "in" {testlist} ":" {suite}
{(SAME_INDENT "else" ":" {suite}) | void=pass_stmt}
       ...: """

    In [3]: pyterp.parser = python.Interpreter(i3.parse("grammar", grammar))

    In [4]: pyterp.repl()

Now edit for_stmt in simple_ast.py

         except StopIteration:
+            evaluate(else_block)
             return

(If else_block was not already an (ignored) parameter, we'd have to
change the `def for_stmt...` line too.) And run in the REPL we just
restarted.

    p>> simple_ast.reload_module(simple_ast)
    p>> for i in [1, 2, 3]:
    ...     print i
    ... else:
    ...     print 100
    ...
    1
    2
    3
    100
    p>> for i in [1, 2, 3]:
    ...     print i
    ...     break
    ... else:
    ...     print 100
    ...
    1
    p>>



Some notes:

1. I'm using the host Python interpreter to change the grammar here
but that's not strictly necessary if we expose pyterp itself in the
global scope.
2. I'm editing simple_ast.py and reloading because its less changes
but redefining a new function and setting simple_ast.for_stmt (in
other words, monkey-patching the change) should also work.

If we wanted an "otherwise" clause on top of an "else" clause, we
probably want to make some provision for the runtime to distinguish
between the two so I'd edit the grammar with something like

    for_stmt = "for" {exprlist} "in" {testlist} ":" {suite}
{((SAME_INDENT "else" ":" {suite}) | void=pass_stmt)=else_block
((SAME_INDENT "otherwise" ":" {suite}) |
void=pass_stmt)=otherwise_block}

(In this case, "otherwise" blocks have to come after the "else" block
if both are present.)

asrp
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180205/051d7078/attachment.html>

From steve at holdenweb.com  Tue Feb  6 04:52:18 2018
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 6 Feb 2018 09:52:18 +0000
Subject: [Python-Dev] libxml2 installation/binding issue
In-Reply-To: <CAMzjp8G7UJyv=Q6=WAk+-n8YCRZaPe_-ZhW861rczKWwzerUvg@mail.gmail.com>
References: <68B69E4C07B04A488BA8528DF70CA165844150BC@fmsmsx116.amr.corp.intel.com>
 <CAMzjp8G7UJyv=Q6=WAk+-n8YCRZaPe_-ZhW861rczKWwzerUvg@mail.gmail.com>
Message-ID: <CAMofdRDotfGvdVT9Xde7iErSw2a91v_F7UeowRY+jeL8DY9mVQ@mail.gmail.com>

On Mon, Feb 5, 2018 at 10:55 PM, Ethan Smith <ethan at ethanhs.me> wrote:

> This list is for the discussion of development *of* Python. For
> discussion of development *with* Python, you want python-list.
>

 ... Whose web page can be found at
https://mail.python.org/mailman/listinfo/python-list

Steve Holden
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180206/224daa4a/attachment.html>

From steve at holdenweb.com  Tue Feb  6 05:00:52 2018
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 6 Feb 2018 10:00:52 +0000
Subject: [Python-Dev] A minimal Python interpreter written in Python for
 experimenting with language changes
In-Reply-To: <20180206034013.GZ26553@ando.pearwood.info>
References: <trinity-aabb133a-c890-476f-a962-be6dacc697d6-1517616090604@3c-app-mailcom-lxa14>
 <09cc01d39c96$9cb28910$d6179b30$@sdamon.com>
 <trinity-a323a69c-0a35-4e20-bb66-a6aa1df5c361-1517654715564@3c-app-mailcom-lxa05>
 <20180206034013.GZ26553@ando.pearwood.info>
Message-ID: <CAMofdRCdfv+-y8jGLd6vqjHCVXU0vLPKmLFhLrMgsY4P72amSg@mail.gmail.com>

On Tue, Feb 6, 2018 at 3:40 AM, Steven D'Aprano <steve at pearwood.info> wrote:

> On Sat, Feb 03, 2018 at 11:45:15AM +0100, asrp wrote:
>
> ?[...]
>


> Here's a faked session showing the sort of thing I am referring to.
> (Note that this is just an example, not a proposal for a new language
> feature.)
>
> for x in [1, 2, 3]:
>     print(x)
> otherwise:
>     print("nothing there")
>
>
> prints 1, 2, 3.
>

?I think you meant

    for x in ([1, 2, 3], ):

?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180206/03dc4881/attachment.html>

From lukasz at langa.pl  Tue Feb  6 10:32:35 2018
From: lukasz at langa.pl (Lukasz Langa)
Date: Tue, 6 Feb 2018 15:32:35 +0000
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAOMjWknn3PRCPHiUJc_+10SXDU37y84--cUdX0KwqB=f1ay3gg@mail.gmail.com>
References: <CABwCVQDRPo98rqxDXdEa8OXJJNJBp=1=3VgmV9fcc8EOo1-_zg@mail.gmail.com>
 <CAP7+vJK4s8tnUxC=7B+MEAqvwWm+d4+rWJyiy3v7V++UD65yjA@mail.gmail.com>
 <CADiSq7d8y1R=vr1AQPwbp1LM6GHBmkv8xg6254sM0B1LTqnOGQ@mail.gmail.com>
 <CAOMjWknn3PRCPHiUJc_+10SXDU37y84--cUdX0KwqB=f1ay3gg@mail.gmail.com>
Message-ID: <42A63D25-ADD8-4EBB-B0A9-3F56799718EA@langa.pl>

To add a counter-example that I'm painfully familiar with: the old Thrift for Python makes its (mutable) structures hashable. This is "useful" because you can memoize functions that take Thrift structures as arguments. You can key dictionaries with them. And so on.

Unfortunately, more often then not this same structure is passed around and inevitably gets mutated in place. Everything breaks. What should be memoized isn't (and in pathological cases, what shouldn't be memoized is), dictionary access *using the same object* raises unexpected key errors but iterating on the dictionary reveals the key.

It's clearly clowntown and in hindsight we shouldn't have enabled this behavior. But we can't go back since too much code relies on hashability now and it's "mostly" fine.

As a solution, the new asyncio-only Thrift implementation for Python uses C-level structures to make sure they are truly immutable in Python. We hash them and cache them like there's no tomorrow.

- ?


> On Feb 5, 2018, at 11:54 PM, Ivan Levkivskyi <levkivskyi at gmail.com> wrote:
> 
> Just wanted to add my 5 cents here. I am a bit surprised how people are scared by adding `__hash__` to mutable classes.
> From my experience it is quite normal, I was always thinking about `hash()` as hashing a _value_,
> and never as hashing _identity_, if I would need the latter, there is a different function for this, `id()`.
> Moreover, I often did this in situations where dataclasses would be useful: a class with several fields,
> necessary dunders, and few other methods (record-like classes).
> My motivation was mostly speed-up by memorization.
> To be fair my use cases were mostly about some tree-like strictures, but this is probably a coincidence.
> 
> FWIW it is not a super-safe practice, but neither super-dangerous.
> 
> --
> Ivan
> 
> 
> 
> On 5 February 2018 at 22:56, Nick Coghlan <ncoghlan at gmail.com <mailto:ncoghlan at gmail.com>> wrote:
> On 6 February 2018 at 03:47, Guido van Rossum <guido at python.org <mailto:guido at python.org>> wrote:
> > If there's going to be an API for it, it should be in the class, not
> > something that mutates the class afterwards.
> 
> Something I realised after posting the __class__ setting idea is that
> you can actually use a comparable trick to inject an unsafe hash from
> the frozen version into the mutable version:
> 
>     >>> from dataclasses import dataclass
>     >>> @dataclass
>     ... class Example:
>     ...     a: int
>     ...     b: int
>     ...
>     >>> c = Example(1, 2)
>     >>> hash(c)
>     Traceback (most recent call last):
>      File "<stdin>", line 1, in <module>
>     TypeError: unhashable type: 'Example'
> 
>     >>> @dataclass(frozen=True)
>     ... class LockedExample(Example):
>     ...     pass
>     ...
>     >>> Example.__hash__ = LockedExample.__hash__
>     >>> hash(c)
>     3713081631934410656
> 
> So "unsafe_hash=True" would just be a shorthand spelling of that which
> skips creating the full frozen version of the class (and with the
> explicit parameter, we can better document the risks of making
> something hashable without also freezing it post-creation).
> 
> Cheers,
> Nick.
> 
> --
> Nick Coghlan   |   ncoghlan at gmail.com <mailto:ncoghlan at gmail.com>   |   Brisbane, Australia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org <mailto:Python-Dev at python.org>
> https://mail.python.org/mailman/listinfo/python-dev <https://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/levkivskyi%40gmail.com <https://mail.python.org/mailman/options/python-dev/levkivskyi%40gmail.com>
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org <mailto:Python-Dev at python.org>
> https://mail.python.org/mailman/listinfo/python-dev <https://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/lukasz%40langa.pl <https://mail.python.org/mailman/options/python-dev/lukasz%40langa.pl>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180206/628783cd/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 874 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180206/628783cd/attachment.sig>

From steve at pearwood.info  Tue Feb  6 12:26:29 2018
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 7 Feb 2018 04:26:29 +1100
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAEbHw4aoY07dit-5vWVsDZHuzX6Z=6zigFbO9rA9bX3-R4LXoA@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <CAGE7PNL8Ky1fuE6vbX7eDdHZXqmWcQRhQWBwGST60COiysRG8Q@mail.gmail.com>
 <CALGmxE+5s3pwzda2cyQkYC1+emq-jdtH_gF+HoO11i5vg08yuQ@mail.gmail.com>
 <CAEbHw4aoY07dit-5vWVsDZHuzX6Z=6zigFbO9rA9bX3-R4LXoA@mail.gmail.com>
Message-ID: <20180206172628.GA26553@ando.pearwood.info>

On Mon, Feb 05, 2018 at 10:50:21AM -0800, David Mertz wrote:

> Absolutely I agree. 'unsafe_hash' as a name is clear warning to users.

(I don't mean to pick on David specifically, I had to reply to some 
message in this thread and I just picked his.)

I'm rather gobsmacked at the attitudes of many people here about hashing 
data classes. I thought *I* was the cynical pessimist who didn't have a 
high opinion of the quality of the average programmer, but according to 
this thread apparently I'm positively Pollyanna-esque for believing that 
most people will realise that if an API offers separate switches for 
hashable and frozen, you need to set *both* if you want both.

Greg Smith even says that writing dunders apart from __init__ is a code 
smell, and warns people not to write dunders. Seriously? I get that 
__hash__ is hard to write correctly, which is why we have a hash=True to 
do the hard work for us, but I can't help feeling that at the point 
we're saying "don't write dunders, any dunder, you'll only do it wrong" 
we have crossed over to the wrong side of the pessimist/optimist line.

But here we are: talking about naming a perfectly reasonable argument 
"unsafe_hash". Why are we trying to frighten people?

There is nothing unsafe about a DataClass with hash=True, frozen=True, 
but this scheme means that even people who know what they're doing will 
write unsafe_hash=True, frozen=True, as if hashability was some sort of 
hand grenade waiting to go off.

Perhaps we ought to deprecate __hash__ and start calling it 
__danger_danger_hash__ too? No, I don't think so.

In the past, we've (rightly!) rejected proposals to call things like 
eval "unsafe_eval", and that really is dangerously unsafe when used 
naively with untrusted, unsanitised data. Hashing mutable objects by 
accident might be annoyingly difficult and frustrating to debug, but 
code injection attacks can lead to identity theft and worse, serious 
consequences for real people.

I'm 100% in favour of programmer education, but I think this label is 
*miseducation*. We're suggesting that hashability is unsafe, regardless 
of whether the object is frozen or not.

I'd far prefer to get a runtime warning:

"Are you sure you want hash=True without frozen=True?"

(or words to that extent) rather than burden all uses of the hash 
parameter, good and bad, with the unsafe label.


-- 
Steve

From guido at python.org  Tue Feb  6 12:38:37 2018
From: guido at python.org (Guido van Rossum)
Date: Tue, 6 Feb 2018 09:38:37 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <20180206172628.GA26553@ando.pearwood.info>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <CAGE7PNL8Ky1fuE6vbX7eDdHZXqmWcQRhQWBwGST60COiysRG8Q@mail.gmail.com>
 <CALGmxE+5s3pwzda2cyQkYC1+emq-jdtH_gF+HoO11i5vg08yuQ@mail.gmail.com>
 <CAEbHw4aoY07dit-5vWVsDZHuzX6Z=6zigFbO9rA9bX3-R4LXoA@mail.gmail.com>
 <20180206172628.GA26553@ando.pearwood.info>
Message-ID: <CAP7+vJJHjXByhd8+TxRKHsvJ3gMcVhLgkePbm__ag7RzaKJOGg@mail.gmail.com>

Where do you get the impression that one would have to explicitly request
__hash__ if frozen=True is set? To the contrary, my proposal is for
@dataclass to automatically add a __hash__ method when frozen=True is set.
This is what the code currently released as 3.7.0b1 does if hash=None (the
default).

On Tue, Feb 6, 2018 at 9:26 AM, Steven D'Aprano <steve at pearwood.info> wrote:

> On Mon, Feb 05, 2018 at 10:50:21AM -0800, David Mertz wrote:
>
> > Absolutely I agree. 'unsafe_hash' as a name is clear warning to users.
>
> (I don't mean to pick on David specifically, I had to reply to some
> message in this thread and I just picked his.)
>
> I'm rather gobsmacked at the attitudes of many people here about hashing
> data classes. I thought *I* was the cynical pessimist who didn't have a
> high opinion of the quality of the average programmer, but according to
> this thread apparently I'm positively Pollyanna-esque for believing that
> most people will realise that if an API offers separate switches for
> hashable and frozen, you need to set *both* if you want both.
>
> Greg Smith even says that writing dunders apart from __init__ is a code
> smell, and warns people not to write dunders. Seriously? I get that
> __hash__ is hard to write correctly, which is why we have a hash=True to
> do the hard work for us, but I can't help feeling that at the point
> we're saying "don't write dunders, any dunder, you'll only do it wrong"
> we have crossed over to the wrong side of the pessimist/optimist line.
>
> But here we are: talking about naming a perfectly reasonable argument
> "unsafe_hash". Why are we trying to frighten people?
>
> There is nothing unsafe about a DataClass with hash=True, frozen=True,
> but this scheme means that even people who know what they're doing will
> write unsafe_hash=True, frozen=True, as if hashability was some sort of
> hand grenade waiting to go off.
>
> Perhaps we ought to deprecate __hash__ and start calling it
> __danger_danger_hash__ too? No, I don't think so.
>
> In the past, we've (rightly!) rejected proposals to call things like
> eval "unsafe_eval", and that really is dangerously unsafe when used
> naively with untrusted, unsanitised data. Hashing mutable objects by
> accident might be annoyingly difficult and frustrating to debug, but
> code injection attacks can lead to identity theft and worse, serious
> consequences for real people.
>
> I'm 100% in favour of programmer education, but I think this label is
> *miseducation*. We're suggesting that hashability is unsafe, regardless
> of whether the object is frozen or not.
>
> I'd far prefer to get a runtime warning:
>
> "Are you sure you want hash=True without frozen=True?"
>
> (or words to that extent) rather than burden all uses of the hash
> parameter, good and bad, with the unsafe label.
>
>
> --
> Steve
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180206/8329fe94/attachment.html>

From ethan at stoneleaf.us  Tue Feb  6 13:11:52 2018
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 06 Feb 2018 10:11:52 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAP7+vJJHjXByhd8+TxRKHsvJ3gMcVhLgkePbm__ag7RzaKJOGg@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <CAGE7PNL8Ky1fuE6vbX7eDdHZXqmWcQRhQWBwGST60COiysRG8Q@mail.gmail.com>
 <CALGmxE+5s3pwzda2cyQkYC1+emq-jdtH_gF+HoO11i5vg08yuQ@mail.gmail.com>
 <CAEbHw4aoY07dit-5vWVsDZHuzX6Z=6zigFbO9rA9bX3-R4LXoA@mail.gmail.com>
 <20180206172628.GA26553@ando.pearwood.info>
 <CAP7+vJJHjXByhd8+TxRKHsvJ3gMcVhLgkePbm__ag7RzaKJOGg@mail.gmail.com>
Message-ID: <5A79EFE8.1020909@stoneleaf.us>

On 02/06/2018 09:38 AM, Guido van Rossum wrote:

> Where do you get the impression that one would have to explicitly request __hash__ if frozen=True is set? To the
> contrary, my proposal is for @dataclass to automatically add a __hash__ method when frozen=True is set. This is what the
> code currently released as 3.7.0b1 does if hash=None (the default).

Which is my issue with the naming -- although, really, it's more with the parameter/argument:  in a hand-written class,

   __hash__ = None

means the object in is not hashable, but with the decorator:

   @dataclass(..., hash=None, ...)

it means something else.

My preference for "fixing" the issue:

1) make the default be a custom object (not None), so that `hash=None`
    means disable hashing

2) change the param name -- maybe to `add_hash` (I agree with D'Aprano
    that `unsafe_hash` can be misleading)

--
~Ethan~

From guido at python.org  Tue Feb  6 14:18:20 2018
From: guido at python.org (Guido van Rossum)
Date: Tue, 6 Feb 2018 11:18:20 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <5A79EFE8.1020909@stoneleaf.us>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <CAGE7PNL8Ky1fuE6vbX7eDdHZXqmWcQRhQWBwGST60COiysRG8Q@mail.gmail.com>
 <CALGmxE+5s3pwzda2cyQkYC1+emq-jdtH_gF+HoO11i5vg08yuQ@mail.gmail.com>
 <CAEbHw4aoY07dit-5vWVsDZHuzX6Z=6zigFbO9rA9bX3-R4LXoA@mail.gmail.com>
 <20180206172628.GA26553@ando.pearwood.info>
 <CAP7+vJJHjXByhd8+TxRKHsvJ3gMcVhLgkePbm__ag7RzaKJOGg@mail.gmail.com>
 <5A79EFE8.1020909@stoneleaf.us>
Message-ID: <CAP7+vJLXPfKLiTXRqxmh2hs6V_r8VhiDo7EgP1nXJO=_22Lsyw@mail.gmail.com>

We may be in violent agreement.

I propose *not* to add a way to *disable* hashing when the rest of the
flags to @dataclass() would indicate that it's safe to add a __hash__
method.

I propose that with @dataclass(unsafe_hash=False) (the default), a __hash__
method is added when the following conditions are true:

- frozen=True (not the default)
- compare=True (the default)
- no __hash__ method is defined in the class

If we instead use @dataclass(unsafe_hash=True), a __hash__ will be added
regardless of the other flags, but if a __hash__ method is present, an
exception is raised.

Other values (e.g. unsafe_hash=None) are illegal for this flag.

Note that the the hash= flag to the field() function is unchanged from
what's currently in PEP 557 or in the implementation in 3.7.0b1. In
particular, the generated __hash__ method will disregard fields declared
using field(hash=False). It will also disregard fields declared using
field(compare=False, hash=False|None).


On Tue, Feb 6, 2018 at 10:11 AM, Ethan Furman <ethan at stoneleaf.us> wrote:

> On 02/06/2018 09:38 AM, Guido van Rossum wrote:
>
> Where do you get the impression that one would have to explicitly request
>> __hash__ if frozen=True is set? To the
>> contrary, my proposal is for @dataclass to automatically add a __hash__
>> method when frozen=True is set. This is what the
>> code currently released as 3.7.0b1 does if hash=None (the default).
>>
>
> Which is my issue with the naming -- although, really, it's more with the
> parameter/argument:  in a hand-written class,
>
>   __hash__ = None
>
> means the object in is not hashable, but with the decorator:
>
>   @dataclass(..., hash=None, ...)
>
> it means something else.
>
> My preference for "fixing" the issue:
>
> 1) make the default be a custom object (not None), so that `hash=None`
>    means disable hashing
>
> 2) change the param name -- maybe to `add_hash` (I agree with D'Aprano
>    that `unsafe_hash` can be misleading)
>
> --
> ~Ethan~
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%
> 40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180206/d6563d3a/attachment.html>

From ethan at stoneleaf.us  Tue Feb  6 14:40:49 2018
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 06 Feb 2018 11:40:49 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAP7+vJLXPfKLiTXRqxmh2hs6V_r8VhiDo7EgP1nXJO=_22Lsyw@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <CAGE7PNL8Ky1fuE6vbX7eDdHZXqmWcQRhQWBwGST60COiysRG8Q@mail.gmail.com>
 <CALGmxE+5s3pwzda2cyQkYC1+emq-jdtH_gF+HoO11i5vg08yuQ@mail.gmail.com>
 <CAEbHw4aoY07dit-5vWVsDZHuzX6Z=6zigFbO9rA9bX3-R4LXoA@mail.gmail.com>
 <20180206172628.GA26553@ando.pearwood.info>
 <CAP7+vJJHjXByhd8+TxRKHsvJ3gMcVhLgkePbm__ag7RzaKJOGg@mail.gmail.com>
 <5A79EFE8.1020909@stoneleaf.us>
 <CAP7+vJLXPfKLiTXRqxmh2hs6V_r8VhiDo7EgP1nXJO=_22Lsyw@mail.gmail.com>
Message-ID: <5A7A04C1.8020709@stoneleaf.us>

On 02/06/2018 11:18 AM, Guido van Rossum wrote:

> We may be in violent agreement.
>
> I propose *not* to add a way to *disable* hashing when the rest of the flags to @dataclass() would indicate that it's
> safe to add a __hash__ method.

Okay.

> I propose that with @dataclass(unsafe_hash=False) (the default), a __hash__ method is added when the following
> conditions are true:
>
> - frozen=True (not the default)
> - compare=True (the default)
> - no __hash__ method is defined in the class
>
> If we instead use @dataclass(unsafe_hash=True), a __hash__ will be added regardless of the other flags, but if a
> __hash__ method is present, an exception is raised.
>
> Other values (e.g. unsafe_hash=None) are illegal for this flag.

Ah!  Excellent, that greatly allays my worries.

> Note that the the hash= flag to the field() function is unchanged from what's currently in PEP 557 or in the
> implementation in 3.7.0b1. In particular, the generated __hash__ method will disregard fields declared using
> field(hash=False). It will also disregard fields declared using field(compare=False, hash=False|None).

It sounds like `unsafe_hash=True` indicates a truly unsafe hash (that is, mutable data is involved in the hash 
calculation), but there still seems to be one possibility for an "unsafe_hash" to actually be safe -- that is, if only 
immutable fields are used in __eq__, then dataclass could safely generate a hash for us.

Do we have a way to know if the equality fields are hashable?  I suppose we could check each one for a for a non-None 
__hash__.  Then we could modify that first condition from

- frozen=True

to

- frozen=True or all(getattr(eq_fld, '__hash__', None) is not None for eq_field in equality_fields)

Thoughts?


On a different note, should the PEP be updated with the current signature?  It still talks about hash=None being the 
default.

--
~Ethan~

From mertz at gnosis.cx  Tue Feb  6 14:57:23 2018
From: mertz at gnosis.cx (David Mertz)
Date: Tue, 6 Feb 2018 11:57:23 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <5A79EFE8.1020909@stoneleaf.us>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <CAGE7PNL8Ky1fuE6vbX7eDdHZXqmWcQRhQWBwGST60COiysRG8Q@mail.gmail.com>
 <CALGmxE+5s3pwzda2cyQkYC1+emq-jdtH_gF+HoO11i5vg08yuQ@mail.gmail.com>
 <CAEbHw4aoY07dit-5vWVsDZHuzX6Z=6zigFbO9rA9bX3-R4LXoA@mail.gmail.com>
 <20180206172628.GA26553@ando.pearwood.info>
 <CAP7+vJJHjXByhd8+TxRKHsvJ3gMcVhLgkePbm__ag7RzaKJOGg@mail.gmail.com>
 <5A79EFE8.1020909@stoneleaf.us>
Message-ID: <CAEbHw4YcCP-K32ctYTifGuhhtScOF9or4_9dyQ=URE7LL9yhUA@mail.gmail.com>

Honestly, the name I would most want for the keyword argument is '_hash'.
That carries the semantics I desire.

On Feb 6, 2018 10:13 AM, "Ethan Furman" <ethan at stoneleaf.us> wrote:

> On 02/06/2018 09:38 AM, Guido van Rossum wrote:
>
> Where do you get the impression that one would have to explicitly request
>> __hash__ if frozen=True is set? To the
>> contrary, my proposal is for @dataclass to automatically add a __hash__
>> method when frozen=True is set. This is what the
>> code currently released as 3.7.0b1 does if hash=None (the default).
>>
>
> Which is my issue with the naming -- although, really, it's more with the
> parameter/argument:  in a hand-written class,
>
>   __hash__ = None
>
> means the object in is not hashable, but with the decorator:
>
>   @dataclass(..., hash=None, ...)
>
> it means something else.
>
> My preference for "fixing" the issue:
>
> 1) make the default be a custom object (not None), so that `hash=None`
>    means disable hashing
>
> 2) change the param name -- maybe to `add_hash` (I agree with D'Aprano
>    that `unsafe_hash` can be misleading)
>
> --
> ~Ethan~
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/mertz%
> 40gnosis.cx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180206/172290ce/attachment.html>

From eric at trueblade.com  Tue Feb  6 15:03:21 2018
From: eric at trueblade.com (Eric V. Smith)
Date: Tue, 6 Feb 2018 15:03:21 -0500
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <5A7A04C1.8020709@stoneleaf.us>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <CAGE7PNL8Ky1fuE6vbX7eDdHZXqmWcQRhQWBwGST60COiysRG8Q@mail.gmail.com>
 <CALGmxE+5s3pwzda2cyQkYC1+emq-jdtH_gF+HoO11i5vg08yuQ@mail.gmail.com>
 <CAEbHw4aoY07dit-5vWVsDZHuzX6Z=6zigFbO9rA9bX3-R4LXoA@mail.gmail.com>
 <20180206172628.GA26553@ando.pearwood.info>
 <CAP7+vJJHjXByhd8+TxRKHsvJ3gMcVhLgkePbm__ag7RzaKJOGg@mail.gmail.com>
 <5A79EFE8.1020909@stoneleaf.us>
 <CAP7+vJLXPfKLiTXRqxmh2hs6V_r8VhiDo7EgP1nXJO=_22Lsyw@mail.gmail.com>
 <5A7A04C1.8020709@stoneleaf.us>
Message-ID: <98c4f065-8c88-43af-6cf4-e61a066e0ca1@trueblade.com>

On 2/6/18 2:40 PM, Ethan Furman wrote:
> On a different note, should the PEP be updated with the current
> signature?  It still talks about hash=None being the default.

Once we've reached an agreement, I'll update the PEP. I don't think 
we're there quite yet.

Eric

From guido at python.org  Tue Feb  6 15:26:52 2018
From: guido at python.org (Guido van Rossum)
Date: Tue, 6 Feb 2018 12:26:52 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAEbHw4YcCP-K32ctYTifGuhhtScOF9or4_9dyQ=URE7LL9yhUA@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <CAGE7PNL8Ky1fuE6vbX7eDdHZXqmWcQRhQWBwGST60COiysRG8Q@mail.gmail.com>
 <CALGmxE+5s3pwzda2cyQkYC1+emq-jdtH_gF+HoO11i5vg08yuQ@mail.gmail.com>
 <CAEbHw4aoY07dit-5vWVsDZHuzX6Z=6zigFbO9rA9bX3-R4LXoA@mail.gmail.com>
 <20180206172628.GA26553@ando.pearwood.info>
 <CAP7+vJJHjXByhd8+TxRKHsvJ3gMcVhLgkePbm__ag7RzaKJOGg@mail.gmail.com>
 <5A79EFE8.1020909@stoneleaf.us>
 <CAEbHw4YcCP-K32ctYTifGuhhtScOF9or4_9dyQ=URE7LL9yhUA@mail.gmail.com>
Message-ID: <CAP7+vJK-wVOC1gc-ASyPx65UmsMvAjFwM9ueQS3R1sMhQK0+mw@mail.gmail.com>

That's much less self-descriptive and harder to search Google or
StackOverflow for. It's also easier to overlook. We really want to send the
signal that this is unsafe and requires serious consideration before it is
turned on.

On Tue, Feb 6, 2018 at 11:57 AM, David Mertz <mertz at gnosis.cx> wrote:

> Honestly, the name I would most want for the keyword argument is '_hash'.
> That carries the semantics I desire.
>
> On Feb 6, 2018 10:13 AM, "Ethan Furman" <ethan at stoneleaf.us> wrote:
>
>> On 02/06/2018 09:38 AM, Guido van Rossum wrote:
>>
>> Where do you get the impression that one would have to explicitly request
>>> __hash__ if frozen=True is set? To the
>>> contrary, my proposal is for @dataclass to automatically add a __hash__
>>> method when frozen=True is set. This is what the
>>> code currently released as 3.7.0b1 does if hash=None (the default).
>>>
>>
>> Which is my issue with the naming -- although, really, it's more with the
>> parameter/argument:  in a hand-written class,
>>
>>   __hash__ = None
>>
>> means the object in is not hashable, but with the decorator:
>>
>>   @dataclass(..., hash=None, ...)
>>
>> it means something else.
>>
>> My preference for "fixing" the issue:
>>
>> 1) make the default be a custom object (not None), so that `hash=None`
>>    means disable hashing
>>
>> 2) change the param name -- maybe to `add_hash` (I agree with D'Aprano
>>    that `unsafe_hash` can be misleading)
>>
>> --
>> ~Ethan~
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/mertz%40g
>> nosis.cx
>>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180206/17e498f9/attachment-0001.html>

From guido at python.org  Tue Feb  6 15:24:54 2018
From: guido at python.org (Guido van Rossum)
Date: Tue, 6 Feb 2018 12:24:54 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <5A7A04C1.8020709@stoneleaf.us>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <CAGE7PNL8Ky1fuE6vbX7eDdHZXqmWcQRhQWBwGST60COiysRG8Q@mail.gmail.com>
 <CALGmxE+5s3pwzda2cyQkYC1+emq-jdtH_gF+HoO11i5vg08yuQ@mail.gmail.com>
 <CAEbHw4aoY07dit-5vWVsDZHuzX6Z=6zigFbO9rA9bX3-R4LXoA@mail.gmail.com>
 <20180206172628.GA26553@ando.pearwood.info>
 <CAP7+vJJHjXByhd8+TxRKHsvJ3gMcVhLgkePbm__ag7RzaKJOGg@mail.gmail.com>
 <5A79EFE8.1020909@stoneleaf.us>
 <CAP7+vJLXPfKLiTXRqxmh2hs6V_r8VhiDo7EgP1nXJO=_22Lsyw@mail.gmail.com>
 <5A7A04C1.8020709@stoneleaf.us>
Message-ID: <CAP7+vJKb18G-YLRP2mZF3-ayGq=qbQr2S5kd3HHH0DcmZHHK2A@mail.gmail.com>

On Tue, Feb 6, 2018 at 11:40 AM, Ethan Furman <ethan at stoneleaf.us> wrote:

> It sounds like `unsafe_hash=True` indicates a truly unsafe hash (that is,
> mutable data is involved in the hash calculation), but there still seems to
> be one possibility for an "unsafe_hash" to actually be safe -- that is, if
> only immutable fields are used in __eq__, then dataclass could safely
> generate a hash for us.
>
> Do we have a way to know if the equality fields are hashable?  I suppose
> we could check each one for a for a non-None __hash__.  Then we could
> modify that first condition from
>
> - frozen=True
>
> to
>
> - frozen=True or all(getattr(eq_fld, '__hash__', None) is not None for
> eq_field in equality_fields)
>

There seems to be a misunderstanding underlying these questions. Even if
all fields have an immutable type (e.g. all ints, supporting __eq__ and
__hash__), if the containing class isn't frozen, they can be assigned to.
E.g.

@dataclass()
class Point:
    x: int
    y: int

p = Point(1, 1)
p.x = 2  # This is legal

The only way to make that assignment to p.x illegal is to make the *class*
frozen (using @dataclass(frozen=True)) -- nothing we can do about the
*field* will change this.

Of course if you use @dataclass(frozen=True, unsafe_hash=True) you may
still get a safe hash. :-)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180206/3b1133a7/attachment.html>

From ethan at stoneleaf.us  Tue Feb  6 15:44:52 2018
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 06 Feb 2018 12:44:52 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAP7+vJKb18G-YLRP2mZF3-ayGq=qbQr2S5kd3HHH0DcmZHHK2A@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <CAGE7PNL8Ky1fuE6vbX7eDdHZXqmWcQRhQWBwGST60COiysRG8Q@mail.gmail.com>
 <CALGmxE+5s3pwzda2cyQkYC1+emq-jdtH_gF+HoO11i5vg08yuQ@mail.gmail.com>
 <CAEbHw4aoY07dit-5vWVsDZHuzX6Z=6zigFbO9rA9bX3-R4LXoA@mail.gmail.com>
 <20180206172628.GA26553@ando.pearwood.info>
 <CAP7+vJJHjXByhd8+TxRKHsvJ3gMcVhLgkePbm__ag7RzaKJOGg@mail.gmail.com>
 <5A79EFE8.1020909@stoneleaf.us>
 <CAP7+vJLXPfKLiTXRqxmh2hs6V_r8VhiDo7EgP1nXJO=_22Lsyw@mail.gmail.com>
 <5A7A04C1.8020709@stoneleaf.us>
 <CAP7+vJKb18G-YLRP2mZF3-ayGq=qbQr2S5kd3HHH0DcmZHHK2A@mail.gmail.com>
Message-ID: <5A7A13C4.30201@stoneleaf.us>

On 02/06/2018 12:24 PM, Guido van Rossum wrote:
> On Tue, Feb 6, 2018 at 11:40 AM, Ethan Furman wrote:

>> It sounds like `unsafe_hash=True` indicates a truly unsafe hash (that is,
 >> mutable data is involved in the hash calculation), but there still seems
 >> to be one possibility for an "unsafe_hash" to actually be safe -- that is,
 >> if only immutable fields are used in __eq__, then dataclass could safely
 >> generate a hash for us.
>>
>> Do we have a way to know if the equality fields are hashable??  I suppose
 >> we could check each one for a for a non-None __hash__.?  Then we could
 >> modify that first condition from
>>
>> - frozen=True
>>
>> to
>>
>> - frozen=True or all(getattr(eq_fld, '__hash__', None) is not None for
 >>       eq_field in equality_fields)
>
> There seems to be a misunderstanding underlying these questions. Even if
 > all fields have an immutable type (e.g. all ints, supporting __eq__ and
 > __hash__), if the containing class isn't frozen, they can be assigned to.
 > E.g.
>
> @dataclass()
> class Point:
>   x: int
>   y: int
>
> p = Point(1, 1)
> p.x = 2  # This is legal
>
> The only way to make that assignment to p.x illegal is to make the *class*
 > frozen (using @dataclass(frozen=True)) -- nothing we can do about the *field*
 > will change this.

Oh, right.  When I was thinking this I thought a field could be frozen individually, didn't find the option at the field 
level when I checked the PEP, and then promptly forgot and suggested it anyway.

Although, couldn't we add a field-level frozen attribute (using property for the implementation), and check that all 
equality fields are properties as well as hashable?

--
~Ethan~

From guido at python.org  Tue Feb  6 16:06:12 2018
From: guido at python.org (Guido van Rossum)
Date: Tue, 6 Feb 2018 13:06:12 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <5A7A13C4.30201@stoneleaf.us>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <CAGE7PNL8Ky1fuE6vbX7eDdHZXqmWcQRhQWBwGST60COiysRG8Q@mail.gmail.com>
 <CALGmxE+5s3pwzda2cyQkYC1+emq-jdtH_gF+HoO11i5vg08yuQ@mail.gmail.com>
 <CAEbHw4aoY07dit-5vWVsDZHuzX6Z=6zigFbO9rA9bX3-R4LXoA@mail.gmail.com>
 <20180206172628.GA26553@ando.pearwood.info>
 <CAP7+vJJHjXByhd8+TxRKHsvJ3gMcVhLgkePbm__ag7RzaKJOGg@mail.gmail.com>
 <5A79EFE8.1020909@stoneleaf.us>
 <CAP7+vJLXPfKLiTXRqxmh2hs6V_r8VhiDo7EgP1nXJO=_22Lsyw@mail.gmail.com>
 <5A7A04C1.8020709@stoneleaf.us>
 <CAP7+vJKb18G-YLRP2mZF3-ayGq=qbQr2S5kd3HHH0DcmZHHK2A@mail.gmail.com>
 <5A7A13C4.30201@stoneleaf.us>
Message-ID: <CAP7+vJ+-UUHYAke6Bodm0Q03kbC2ghZKydsvH0Dbj4p+wSDZ=Q@mail.gmail.com>

On Tue, Feb 6, 2018 at 12:44 PM, Ethan Furman <ethan at stoneleaf.us> wrote:

> Although, couldn't we add a field-level frozen attribute (using property
> for the implementation), and check that all equality fields are properties
> as well as hashable?
>

That would be a totally unrelated feature request. Let's wait whether it's
actually needed a lot before designing it.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180206/73239ef4/attachment.html>

From asrp at email.com  Tue Feb  6 18:32:19 2018
From: asrp at email.com (asrp)
Date: Wed, 7 Feb 2018 00:32:19 +0100
Subject: [Python-Dev] A minimal Python interpreter written in Python for
 experimenting with language changes
References: <trinity-aabb133a-c890-476f-a962-be6dacc697d6-1517616090604@3c-app-mailcom-lxa14>
 <09cc01d39c96$9cb28910$d6179b30$@sdamon.com>
 <trinity-a323a69c-0a35-4e20-bb66-a6aa1df5c361-1517654715564@3c-app-mailcom-lxa05>
Message-ID: <trinity-198b98fe-bc02-4fb6-9880-00e29e02f5ad-1517959939147@3c-app-mailcom-lxa03>

> Message-ID: <20180206034013.GZ26553 at ando.pearwood.info>
> 
> On Sat, Feb 03, 2018 at 11:45:15AM +0100, asrp wrote:
> 
> > > Can you give an example of how you would do that? I don't mean the 
> > > mechanism used, I mean how would a developer implement a new syntactic 
> > > feature. Suppose I wanted to add a new clause to for...else, let's say:
> > >
> > >     for ... :
> > >        block
> > >     otherwise:
> > >        # runs only if the for-loop was empty
> > > 
> > > How would do I do that?
> [...]
> > If you tell me a bit more about the intended behaviour of "otherwise", 
> > I'd be happy to do an example with that clause.
> 
> 
> Here's a faked session showing the sort of thing I am referring to. 
> (Note that this is just an example, not a proposal for a new language 
> feature.)
> 
> for x in [1, 2, 3]:
>     print(x)
> otherwise:
>     print("nothing there")
> 
> 
> prints 1, 2, 3.
> 
> for x in []:
>     print(x)
> otherwise:
>     print("nothing there")
> 
> prints "nothing there". In other words, the otherwise block runs if, and 
> only if, the loop iterable is empty and the for block does NOT run.
> 
> Can you do something like that?
> 

Oh, I see. Yes, definitely. This time I'll change for_stmt in lib/simple_ast.py beforehand (but runtime reload also works).

def for_stmt(index_var, iterable, block, else_block, otherwise_block):
    iterator = iter(evaluate(iterable))
    try:
        assignment(index_var, iterator.next())
    except StopIteration:
        evaluate(otherwise_block)
        return
    while_true:
        __caller__['__continue__'] = __continue__
        __caller__['__break__'] = __break__
        evaluate(block)
        try:
            assignment(index_var, iterator.next())
        except StopIteration:
            evaluate(else_block)
            return

Then start the interpreter

$ ipython -i test/python_repl.py 
p>> simport simple_ast
p>> ^D
[...]
In [1]: grammar = python_grammar.full_definition + python_grammar.extra

In [2]: grammar += r"""
   ...: for_stmt = "for" {exprlist} "in" {testlist} ":" {suite} {((SAME_INDENT "else" ":" {suite}) | void=pass_stmt) ((SAME_INDENT "otherwise" ":" {suite}) | void=pass_stmt)}
   ...: """

In [3]: pyterp.parser = python.Interpreter(i3.parse("grammar", grammar))

In [4]: pyterp.repl()
p>> for x in [1, 2]:
...   print(x)
... otherwise:
...   print("Nothing there")
... 
1
2
p>> for x in []:
...   print(x)
... otherwise:
...   print("Nothing there")
... 
Nothing there
p>> for x in [1, 2]:
...   print(x)
... else:
...   print("Something there")
... otherwise:
...   print("Nothing there")
... 
1
2
Something there


I noticed since my last post that the else and otherwise blocks don't need to be named if they are positional in for_stmt. The grammar change here reflects that.

I've also posted an example with `until_stmt` and I talk a bit more about debugging changes made there.

    http://blog.asrpo.com/adding_new_statement

(Nothing here is a proposed language change, just demos.)

asrp

> 
> 
> -- 
> Steve
> 
>

From asrp at email.com  Tue Feb  6 20:13:40 2018
From: asrp at email.com (asrp)
Date: Wed, 7 Feb 2018 02:13:40 +0100
Subject: [Python-Dev] A minimal Python interpreter written in Python for
 experimenting with language changes
In-Reply-To: <CAO41-mNg9=CGwAuCdy--KsiuHB8vT=9xoxyM8e+r=0qrzzp9yA@mail.gmail.com>
References: <CAO41-mNg9=CGwAuCdy--KsiuHB8vT=9xoxyM8e+r=0qrzzp9yA@mail.gmail.com>
Message-ID: <trinity-16962b53-ab60-45d6-af47-8b6d610c705d-1517966020163@3c-app-mailcom-lxa03>

> <begin plug>
> 
> https://refi64.com/posts/the-magic-of-rpython.html
> 
> Note that this was written back when I used "like" and "really" in every
> sentence, and when I used to think that Python copied tuples (don't ask).
> 

Thanks! And thanks for making the RPython language a bit more explicit for the rest of us.

Reading your post reminded me that having nested lists of lists of ... eventually strings (not all at the same nesting level) already made it difficult. I don't remember if I stopped trying there or a bit more further along.

asrp

> On Feb 3, 2018 at 10:05 AM, <asrp <asrp at email.com>> wrote:
> 
> > Are you aware of pypy?
> >
> 
> Yes, but I do not know it well. It seems to be more focused on JIT
> compiling with many of its functions written for special cases and I
> did not find a way to make experimental changes easily.
> 
> I even tried (briefly) to have the main loop of this interpreter run
> in RPython but didn't find enough information on which subset of
> Python does RPython handle.
> 
> 
> > Can you give an example of how you would do that? I don't mean the
> > mechanism used, I mean how would a developer implement a new syntactic
> > feature. Suppose I wanted to add a new clause to for...else, let's say:
> >
> >     for ... :
> >        block
> >     otherwise:
> >        # runs only if the for-loop was empty
> >
> > How would do I do that?
> >
> 
> Here's one way to do that. Sorry for the lack of doc for the moment.
> 
> To alter the language's grammar, you'd create a string with the new
> grammar, parse it and set the result to pyterp.parser. Then, depending
> on how the grammar was changed (which affects what AST is created),
> you'd change the corresponding function to handle the new semantics.
> 
> I didn't quite understand what you mean by "was empty". However, I
> noticed that "else" blocks after for are ignored in the current
> implement. Here's an example session adding "else".
> 
> If you tell me a bit more about the intended behaviour of "otherwise",
> I'd be happy to do an example with that clause.
> 
>     $ ipython -i test/python_repl.py
>     p>> simport simple_ast
>     p>> for i in [1, 2]:
>     ...     print i
>     ...
>     1
>     2
>     p>> for i in [1, 2, 3]:
>     ...     print i
>     ... else:
>     ...     print 100
>     ...
>     1
>     2
>     3
>     p>> ^D
> 
> In the above for loop, the else block is ignored! I press Control-D to
> exist the interpreter.
> 
>     In [1]: grammar = python_grammar.full_definition + python_grammar.extra
> 
>     In [2]: grammar += r"""
>        ...: for_stmt = "for" {exprlist} "in" {testlist} ":" {suite}
> {(SAME_INDENT "else" ":" {suite}) | void=pass_stmt}
>        ...: """
> 
>     In [3]: pyterp.parser = python.Interpreter(i3.parse("grammar", grammar))
> 
>     In [4]: pyterp.repl()
> 
> Now edit for_stmt in simple_ast.py
> 
>          except StopIteration:
> +            evaluate(else_block)
>              return
> 
> (If else_block was not already an (ignored) parameter, we'd have to
> change the `def for_stmt...` line too.) And run in the REPL we just
> restarted.
> 
>     p>> simple_ast.reload_module(simple_ast)
>     p>> for i in [1, 2, 3]:
>     ...     print i
>     ... else:
>     ...     print 100
>     ...
>     1
>     2
>     3
>     100
>     p>> for i in [1, 2, 3]:
>     ...     print i
>     ...     break
>     ... else:
>     ...     print 100
>     ...
>     1
>     p>>
> 
> 
> 
> Some notes:
> 
> 1. I'm using the host Python interpreter to change the grammar here
> but that's not strictly necessary if we expose pyterp itself in the
> global scope.
> 2. I'm editing simple_ast.py and reloading because its less changes
> but redefining a new function and setting simple_ast.for_stmt (in
> other words, monkey-patching the change) should also work.
> 
> If we wanted an "otherwise" clause on top of an "else" clause, we
> probably want to make some provision for the runtime to distinguish
> between the two so I'd edit the grammar with something like
> 
>     for_stmt = "for" {exprlist} "in" {testlist} ":" {suite}
> {((SAME_INDENT "else" ":" {suite}) | void=pass_stmt)=else_block
> ((SAME_INDENT "otherwise" ":" {suite}) |
> void=pass_stmt)=otherwise_block}
> 
> (In this case, "otherwise" blocks have to come after the "else" block
> if both are present.)
> 
> asrp
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>

From eric at trueblade.com  Tue Feb  6 20:38:45 2018
From: eric at trueblade.com (Eric V. Smith)
Date: Tue, 6 Feb 2018 20:38:45 -0500
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
Message-ID: <880D1C2B-517D-4DE8-B59F-878ED2829C5C@trueblade.com>

Sorry for the late reply. Still recovering from a computer failure. 

My only concern with this approach is: what if you don?t want any __hash__ added? Say you want to use your base class?s hashing. I guess you could always ?del cls.__hash__? after the class is created, but it?s not elegant. 

That?s what we got from the tri-state option: never add (False), always add (True), or add if it?s safe (None).

--
Eric

> On Feb 5, 2018, at 12:49 AM, Guido van Rossum <guido at python.org> wrote:
> 
> Looks like this is turning into a major flamewar regardless of what I say. :-(
> 
> I really don't want to lose the ability to add a hash function to a mutable dataclass by flipping a flag in the decorator. I'll explain below. But I am fine if this flag has a name that clearly signals it's an unsafe thing to do.
> 
> I propose to replace the existing (as of 3.7.0b1) hash= keyword for the @dataclass decorator with a simpler flag named unsafe_hash=. This would be a simple bool (not a tri-state flag like the current hash=None|False|True). The default would be False, and the behavior then would be to add a hash function automatically only if it's safe (using the same rules as for hash=None currently). With unsafe_hash=True, a hash function would always be generated that takes all fields into account except those declared using field(hash=False). If there's already a `def __hash__` in the function I don't care what it does, maybe it should raise rather than quietly doing nothing or quietly overwriting it.
> 
> Here's my use case.
> 
> A frozen class requires a lot of discipline, since you have to compute the values of all fields before calling the constructor. A mutable class allows other initialization patterns, e.g. manually setting some fields after the instance has been constructed, or having a separate non-dunder init() method. There may be good reasons for using these patterns, e.g. the object may be part of a cycle (e.g. parent/child links in a tree). Or you may just use one of these patterns because you're a pretty casual coder. Or you're modeling something external.
> 
> My point is that once you have one of those patterns in place, changing your code to avoid them may be difficult. And yet your code may treat the objects as essentially immutable after the initialization phase (e.g. a parse tree). So if you create a dataclass and start coding like that for a while, and much later you need to put one of these into a set or use it as a dict key, switching to frozen=True may not be a quick option. And writing a __hash__ method by hand may feel like a lot of busywork. So this is where [unsafe_]hash=True would come in handy.
> 
> I think naming the flag unsafe_hash should take away most objections, since it will be clear that this is not a safe thing to do. People who don't understand the danger are likely to copy a worse solution from StackOverflow anyway. The docs can point to frozen=True and explain the danger.
> 
> -- 
> --Guido van Rossum (python.org/~guido)
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180206/363174ac/attachment.html>

From guido at python.org  Tue Feb  6 21:48:57 2018
From: guido at python.org (Guido van Rossum)
Date: Tue, 6 Feb 2018 18:48:57 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <880D1C2B-517D-4DE8-B59F-878ED2829C5C@trueblade.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <880D1C2B-517D-4DE8-B59F-878ED2829C5C@trueblade.com>
Message-ID: <CAP7+vJK9NALqT5-zARArLUtXekvyT55j=Vj75Cy160uMuV-ngQ@mail.gmail.com>

That seems a rare case (though I hadn't thought of it). I had thought of
the use case where you want a frozen type without a hash; that you can
presumably implement using

def __hash__(self): raise TypeError("not hashable")

We can do a similar thing to preserve the superclass __hash__ if it's rare
enough:

def __hash__(self): return super().__hash__()

If at all possible I'd like to kill the tri-state hash= flag -- the amount
of time spent creating and discussing the huge table in the bpo issue are
an indication of how much effort it would take other people to understand
it.

If either of those use cases becomes annoyingly common we'll have to think
of something else.

On Tue, Feb 6, 2018 at 5:38 PM, Eric V. Smith <eric at trueblade.com> wrote:

> Sorry for the late reply. Still recovering from a computer failure.
>
> My only concern with this approach is: what if you don?t want any __hash__
> added? Say you want to use your base class?s hashing. I guess you could
> always ?del cls.__hash__? after the class is created, but it?s not elegant.
>
> That?s what we got from the tri-state option: never add (False), always
> add (True), or add if it?s safe (None).
>
> --
> Eric
>
> On Feb 5, 2018, at 12:49 AM, Guido van Rossum <guido at python.org> wrote:
>
> Looks like this is turning into a major flamewar regardless of what I say.
> :-(
>
> I really don't want to lose the ability to add a hash function to a
> mutable dataclass by flipping a flag in the decorator. I'll explain below.
> But I am fine if this flag has a name that clearly signals it's an unsafe
> thing to do.
>
> I propose to replace the existing (as of 3.7.0b1) hash= keyword for the
> @dataclass decorator with a simpler flag named unsafe_hash=. This would be
> a simple bool (not a tri-state flag like the current hash=None|False|True).
> The default would be False, and the behavior then would be to add a hash
> function automatically only if it's safe (using the same rules as for
> hash=None currently). With unsafe_hash=True, a hash function would always
> be generated that takes all fields into account except those declared using
> field(hash=False). If there's already a `def __hash__` in the function I
> don't care what it does, maybe it should raise rather than quietly doing
> nothing or quietly overwriting it.
>
> Here's my use case.
>
> A frozen class requires a lot of discipline, since you have to compute the
> values of all fields before calling the constructor. A mutable class allows
> other initialization patterns, e.g. manually setting some fields after the
> instance has been constructed, or having a separate non-dunder init()
> method. There may be good reasons for using these patterns, e.g. the object
> may be part of a cycle (e.g. parent/child links in a tree). Or you may just
> use one of these patterns because you're a pretty casual coder. Or you're
> modeling something external.
>
> My point is that once you have one of those patterns in place, changing
> your code to avoid them may be difficult. And yet your code may treat the
> objects as essentially immutable after the initialization phase (e.g. a
> parse tree). So if you create a dataclass and start coding like that for a
> while, and much later you need to put one of these into a set or use it as
> a dict key, switching to frozen=True may not be a quick option. And writing
> a __hash__ method by hand may feel like a lot of busywork. So this is where
> [unsafe_]hash=True would come in handy.
>
> I think naming the flag unsafe_hash should take away most objections,
> since it will be clear that this is not a safe thing to do. People who
> don't understand the danger are likely to copy a worse solution from
> StackOverflow anyway. The docs can point to frozen=True and explain the
> danger.
>
> --
> --Guido van Rossum (python.org/~guido)
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> eric%2Ba-python-dev%40trueblade.com
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180206/73233332/attachment-0001.html>

From ethan at stoneleaf.us  Tue Feb  6 22:33:45 2018
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 06 Feb 2018 19:33:45 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAP7+vJK9NALqT5-zARArLUtXekvyT55j=Vj75Cy160uMuV-ngQ@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <880D1C2B-517D-4DE8-B59F-878ED2829C5C@trueblade.com>
 <CAP7+vJK9NALqT5-zARArLUtXekvyT55j=Vj75Cy160uMuV-ngQ@mail.gmail.com>
Message-ID: <5A7A7399.9000504@stoneleaf.us>

On 02/06/2018 06:48 PM, Guido van Rossum wrote:

> That seems a rare case (though I hadn't thought of it). I had thought of the use case where you want a frozen type
> without a hash; that you can presumably implement using
>
> def __hash__(self): raise TypeError("not hashable")
>
> We can do a similar thing to preserve the superclass __hash__ if it's rare enough:
>
> def __hash__(self): return super().__hash__()
>
> If at all possible I'd like to kill the tri-state hash= flag -- the amount of time spent creating and discussing the
> huge table in the bpo issue are an indication of how much effort it would take other people to understand it.

I think the biggest reason this has become so complicated is because we are refusing to use an Enum:

class Hashable(Enum):
     IF_SAFE = 1
     ADD = 2
     DEFER = 3
     NONE = 4

IF_SAFE is currently the False value.
ADD is currently the True value
DEFER means don't add one
NONE means set __hash__ to None

The only thing missing now is a setting to indicate that dataclass should do nothing if the class already has a __hash__ 
method -- possibly DEFER, although I think IF_SAFE can include "the class already has one, it's not safe to override it".

> If either of those use cases becomes annoyingly common we'll have to think of something else.

Or we could solve it now and not have to deal with backwards-compatibility issues in the future.

--
~Ethan~

From guido at python.org  Tue Feb  6 23:21:37 2018
From: guido at python.org (Guido van Rossum)
Date: Tue, 6 Feb 2018 20:21:37 -0800
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <5A7A7399.9000504@stoneleaf.us>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <880D1C2B-517D-4DE8-B59F-878ED2829C5C@trueblade.com>
 <CAP7+vJK9NALqT5-zARArLUtXekvyT55j=Vj75Cy160uMuV-ngQ@mail.gmail.com>
 <5A7A7399.9000504@stoneleaf.us>
Message-ID: <CAP7+vJLdPKMKSZraWcF5wtFZYRfETnwru-8_yBPh4sgUctMj+Q@mail.gmail.com>

I'm not a fan, sorry.

On Tue, Feb 6, 2018 at 7:33 PM, Ethan Furman <ethan at stoneleaf.us> wrote:

> On 02/06/2018 06:48 PM, Guido van Rossum wrote:
>
> That seems a rare case (though I hadn't thought of it). I had thought of
>> the use case where you want a frozen type
>> without a hash; that you can presumably implement using
>>
>> def __hash__(self): raise TypeError("not hashable")
>>
>> We can do a similar thing to preserve the superclass __hash__ if it's
>> rare enough:
>>
>> def __hash__(self): return super().__hash__()
>>
>> If at all possible I'd like to kill the tri-state hash= flag -- the
>> amount of time spent creating and discussing the
>> huge table in the bpo issue are an indication of how much effort it would
>> take other people to understand it.
>>
>
> I think the biggest reason this has become so complicated is because we
> are refusing to use an Enum:
>
> class Hashable(Enum):
>     IF_SAFE = 1
>     ADD = 2
>     DEFER = 3
>     NONE = 4
>
> IF_SAFE is currently the False value.
> ADD is currently the True value
> DEFER means don't add one
> NONE means set __hash__ to None
>
> The only thing missing now is a setting to indicate that dataclass should
> do nothing if the class already has a __hash__ method -- possibly DEFER,
> although I think IF_SAFE can include "the class already has one, it's not
> safe to override it".
>
> If either of those use cases becomes annoyingly common we'll have to think
>> of something else.
>>
>
> Or we could solve it now and not have to deal with backwards-compatibility
> issues in the future.
>
> --
> ~Ethan~
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%
> 40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180206/7986ff2b/attachment.html>

From ncoghlan at gmail.com  Wed Feb  7 03:03:14 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 7 Feb 2018 18:03:14 +1000
Subject: [Python-Dev] Dataclasses and correct hashability
In-Reply-To: <CAP7+vJK9NALqT5-zARArLUtXekvyT55j=Vj75Cy160uMuV-ngQ@mail.gmail.com>
References: <3951276.IZdOgMHjlT@hammer.magicstack.net>
 <CADiSq7ecZFDj0vjnDD2D8ye10SHo+xo-FvbiuBxcM20rvuD0aw@mail.gmail.com>
 <15d2818e-5c7b-ea5f-3a8f-e5edd482b5be@trueblade.com>
 <1906618.bnMMLoM6on@hammer.magicstack.net>
 <727029d5-3c0b-2242-001e-15a25253ca37@trueblade.com>
 <CALGmxELKPy3Y2y1TB2L14nhU0zjKw3EvKANFZ35Q5ppd6WtCMA@mail.gmail.com>
 <CAP7+vJL+G=UxqDZUtAma+z+PknFnNbYS9rbpnsgMrB9U-+00CQ@mail.gmail.com>
 <880D1C2B-517D-4DE8-B59F-878ED2829C5C@trueblade.com>
 <CAP7+vJK9NALqT5-zARArLUtXekvyT55j=Vj75Cy160uMuV-ngQ@mail.gmail.com>
Message-ID: <CADiSq7cxE3anddaD20H8nr3acWpQDY7+bbnQpOg1Q5jLwhL5iA@mail.gmail.com>

On 7 February 2018 at 12:48, Guido van Rossum <guido at python.org> wrote:
> That seems a rare case (though I hadn't thought of it). I had thought of the
> use case where you want a frozen type without a hash; that you can
> presumably implement using
>
> def __hash__(self): raise TypeError("not hashable")

Now that attributes in the class dict pre-empt the generated versions,
"__hash__ = None" in the class body will also turn off hash generation
without enabling hashing.

> We can do a similar thing to preserve the superclass __hash__ if it's rare
> enough:
>
> def __hash__(self): return super().__hash__()

Similarly for this variant, you can do "__hash__ =
BaseClassWithDesiredHashAlgorithm.__hash__". (That may be more
appropriate than dynamic lookup in the MRO if you're extending a
frozen class with a subclass that adds more fields, but you want to
keep using the base class hash definition for some reason)

> If at all possible I'd like to kill the tri-state hash= flag -- the amount
> of time spent creating and discussing the huge table in the bpo issue are an
> indication of how much effort it would take other people to understand it.

+1 - the nice thing about "unsafe_hash=True" is that you *only* need
it if adding the hash would be unsafe, and you haven't set __hash__
explicitly in the class body.

While "frozen=True, unsafe_hash=True" is redundant (since the hash is
safe when instances are frozen), it isn't a surprising defect waiting
to happen the way "frozen=False, hash=True" is in the current
interface.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From sjsumitj at gmail.com  Thu Feb  8 03:38:00 2018
From: sjsumitj at gmail.com (Sum J)
Date: Thu, 8 Feb 2018 14:08:00 +0530
Subject: [Python-Dev] How to set/update value in a xml file using requests
 in python
Message-ID: <CABmCpR6Y=DMjPY9TRB1fNH0p3ZH041zwsJLVX--o+_3A9Q-5uw@mail.gmail.com>

My xml file is located in local network:

http://192.168.43.109/DevMgmt/NetAppsDyn.xml

Below is a part content of above xml I want to update :

<nadyn:NetAppsDyn>

<nadyn:ProxyConfig>
    <dd:ResourceURI/>
    <dd:Port/>
    <dd:ProxySupport>off</dd:ProxySupport>
    </nadyn:ProxyConfig>

</nadyn:NetAppsDyn>

I want to set the value for 'ResourceUI' and 'Port' field in above xml.

I have used below code :

     import requests
     data = {
              'ResourceURI':'web-proxy.xxx.yy.com',
              'Port':8080
            }

    URL = 'http://192.168.75.165/DevMgmt/NetAppsDyn.xml'

    # content type
    head = {'Content-type': 'text/xml'}
    # sending get request
    gr= requests.get(url=URL)
    print gr

    # sending put request
    r = requests.put(url=URL, data=data,headers=head)
    print r.status_code
    # extracting response text
    output_xml = r.text
    print("The op xml is:%s" % output_xml)

Issue : The fields are not getting updated in xml using put request. I am
able to see the response for get (request) , but for put request it is
throwing errror code : 301 , resource has been moved permanently.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180208/2df09d61/attachment.html>

From drsalists at gmail.com  Thu Feb  8 11:45:01 2018
From: drsalists at gmail.com (Dan Stromberg)
Date: Thu, 8 Feb 2018 08:45:01 -0800
Subject: [Python-Dev] How to set/update value in a xml file using
 requests in python
In-Reply-To: <CABmCpR6Y=DMjPY9TRB1fNH0p3ZH041zwsJLVX--o+_3A9Q-5uw@mail.gmail.com>
References: <CABmCpR6Y=DMjPY9TRB1fNH0p3ZH041zwsJLVX--o+_3A9Q-5uw@mail.gmail.com>
Message-ID: <CAGGBd_qpaFrXpEOU6nPHW=11K1jdM87tLpiTjKw8QkwNs18d+Q@mail.gmail.com>

This is more relevant to python-list than python-dev. I've added
python-list to the To header.

Gmail doesn't appear to allow setting a reply-to for a single message,
so I've not set that; please, when replying, drop python-dev from the
to: header.

You'll likely want to set up some kind of REST endpoint to allow
updating your xml file.  Imagine if anyone could change anyone else's
xml files - it'd be pretty chaotic.

Perhaps flask would be appropriate?
http://flask.pocoo.org/docs/0.12/quickstart/

You likely will want a PUT or a POST HTTP verb:
https://stackoverflow.com/questions/107390/whats-the-difference-between-a-post-and-a-put-http-request

HTH.

On Thu, Feb 8, 2018 at 12:38 AM, Sum J <sjsumitj at gmail.com> wrote:
> My xml file is located in local network:
>
> http://192.168.43.109/DevMgmt/NetAppsDyn.xml
>
> Below is a part content of above xml I want to update :
>
> <nadyn:NetAppsDyn>
>
> <nadyn:ProxyConfig>
>     <dd:ResourceURI/>
>     <dd:Port/>
>     <dd:ProxySupport>off</dd:ProxySupport>
>     </nadyn:ProxyConfig>
>
> </nadyn:NetAppsDyn>
>
> I want to set the value for 'ResourceUI' and 'Port' field in above xml.
>
> I have used below code :
>
>      import requests
>      data = {
>               'ResourceURI':'web-proxy.xxx.yy.com',
>               'Port':8080
>             }
>
>     URL = 'http://192.168.75.165/DevMgmt/NetAppsDyn.xml'
>
>     # content type
>     head = {'Content-type': 'text/xml'}
>     # sending get request
>     gr= requests.get(url=URL)
>     print gr
>
>     # sending put request
>     r = requests.put(url=URL, data=data,headers=head)
>     print r.status_code
>     # extracting response text
>     output_xml = r.text
>     print("The op xml is:%s" % output_xml)
>
> Issue : The fields are not getting updated in xml using put request. I am
> able to see the response for get (request) , but for put request it is
> throwing errror code : 301 , resource has been moved permanently.
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/drsalists%40gmail.com
>

From status at bugs.python.org  Fri Feb  9 12:09:54 2018
From: status at bugs.python.org (Python tracker)
Date: Fri,  9 Feb 2018 18:09:54 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20180209170954.7A5BF11A9FA@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2018-02-02 - 2018-02-09)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    6439 (+28)
  closed 38121 (+32)
  total  44560 (+60)

Open issues with patches: 2497 


Issues opened (44)
==================

#31961: subprocess._execute_child doesn't accept a single PathLike arg
https://bugs.python.org/issue31961  reopened by serhiy.storchaka

#32521: NIS module fails to build due to the removal of interfaces rel
https://bugs.python.org/issue32521  reopened by cstratak

#32616: Significant performance problems with Python 2.7 built with cl
https://bugs.python.org/issue32616  reopened by inada.naoki

#32750: lib2to3 log_error method behavior is inconsitent with document
https://bugs.python.org/issue32750  opened by soupytwist

#32751: wait_for(future, ...) should wait for the future (even if a ti
https://bugs.python.org/issue32751  opened by njs

#32752: no information about accessing typing.Generic type arguments
https://bugs.python.org/issue32752  opened by Paul Pinterits

#32754: feature request: asyncio.gather/wait cancel children on first 
https://bugs.python.org/issue32754  opened by thehesiod

#32755: Several cookies with the same name get intermixed
https://bugs.python.org/issue32755  opened by ???????? ????????????????????

#32756: argparse: parse_known_args: raising exception on unknown arg f
https://bugs.python.org/issue32756  opened by actionless

#32757: Python 2.7 : Buffer Overflow vulnerability in exec() function
https://bugs.python.org/issue32757  opened by hadimene

#32758: Stack overflow when parse long expression to AST
https://bugs.python.org/issue32758  opened by serhiy.storchaka

#32759: multiprocessing.Array do not release shared memory
https://bugs.python.org/issue32759  opened by OO O

#32761: Create IDLE Modern Mac keyset
https://bugs.python.org/issue32761  opened by rhettinger

#32762: Choose protocol implementation on transport.set_protocol()
https://bugs.python.org/issue32762  opened by asvetlov

#32764: Popen doesn't work on Windows when args is a list
https://bugs.python.org/issue32764  opened by serhiy.storchaka

#32766: 4.7.7. Function Annotations
https://bugs.python.org/issue32766  opened by John Hossbach

#32767: Mutating a list while iterating:  clarify the docs
https://bugs.python.org/issue32767  opened by tim.peters

#32768: object.__new__ does not accept arguments if __bases__ is chang
https://bugs.python.org/issue32768  opened by VA

#32769: Add 'annotations' to the glossary
https://bugs.python.org/issue32769  opened by csabella

#32771: merge the underlying data stores of unicodedata and the str ty
https://bugs.python.org/issue32771  opened by benjamin.peterson

#32773: distutils should NOT preserve timestamps
https://bugs.python.org/issue32773  opened by jdemeyer

#32776: asyncio SIGCHLD scalability problems
https://bugs.python.org/issue32776  opened by holger+lp

#32779: urljoining an empty query string doesn't clear query string
https://bugs.python.org/issue32779  opened by Paul Fisher

#32780: ctypes: memoryview gives incorrect PEP3118 format strings for 
https://bugs.python.org/issue32780  opened by Eric.Wieser

#32781: lzh_tw is missing in locale.py
https://bugs.python.org/issue32781  opened by cypressyew

#32782: ctypes: memoryview gives incorrect PEP3118 itemsize for array 
https://bugs.python.org/issue32782  opened by Eric.Wieser

#32786: Didnot work strftime() when hangeul in format sting
https://bugs.python.org/issue32786  opened by leop0ld

#32787: Better error handling in ctypes
https://bugs.python.org/issue32787  opened by serhiy.storchaka

#32788: Better error handling in sqlite3
https://bugs.python.org/issue32788  opened by serhiy.storchaka

#32789: Note missing from logging.debug() docs
https://bugs.python.org/issue32789  opened by pgacv2

#32790: Keep trailing zeros in precision for string format option g
https://bugs.python.org/issue32790  opened by sk1d

#32792: ChainMap should preserve order of underlying mappings
https://bugs.python.org/issue32792  opened by rhettinger

#32793: smtplib: duplicated debug message
https://bugs.python.org/issue32793  opened by qingyunha

#32795: subprocess.check_output() with timeout does not exit if child 
https://bugs.python.org/issue32795  opened by Mike Lewis

#32797: Tracebacks from Cython modules no longer work
https://bugs.python.org/issue32797  opened by jdemeyer

#32798: mmap.flush() on Linux does not accept the "offset" and "size" 
https://bugs.python.org/issue32798  opened by byronhawkins

#32799: <class 'contextlib._GeneratorContextManager'> returned a resul
https://bugs.python.org/issue32799  opened by EliRibble

#32800: Replace deprecated link to new page on w3c site
https://bugs.python.org/issue32800  opened by sblondon

#32801: Lib/_strptime.py: utilize all()
https://bugs.python.org/issue32801  opened by dilyan.palauzov

#32803: smtplib: LMTP broken in the case of multiple RCPT
https://bugs.python.org/issue32803  opened by jasen betts

#32804: urllib.retrieve documentation doesn't mention context paramete
https://bugs.python.org/issue32804  opened by julian_o

#32805: Possible integer overflow when call PyDTrace_GC_DONE()
https://bugs.python.org/issue32805  opened by serhiy.storchaka

#32806: locally imported modules are unaccessible in lambdas in pdb
https://bugs.python.org/issue32806  opened by Yuri Kanivetsky

#32808: subprocess.check_output opens an unwanted command line window 
https://bugs.python.org/issue32808  opened by Christian Heigele



Most recent 15 issues with no replies (15)
==========================================

#32806: locally imported modules are unaccessible in lambdas in pdb
https://bugs.python.org/issue32806

#32805: Possible integer overflow when call PyDTrace_GC_DONE()
https://bugs.python.org/issue32805

#32801: Lib/_strptime.py: utilize all()
https://bugs.python.org/issue32801

#32800: Replace deprecated link to new page on w3c site
https://bugs.python.org/issue32800

#32793: smtplib: duplicated debug message
https://bugs.python.org/issue32793

#32788: Better error handling in sqlite3
https://bugs.python.org/issue32788

#32787: Better error handling in ctypes
https://bugs.python.org/issue32787

#32782: ctypes: memoryview gives incorrect PEP3118 itemsize for array 
https://bugs.python.org/issue32782

#32780: ctypes: memoryview gives incorrect PEP3118 format strings for 
https://bugs.python.org/issue32780

#32779: urljoining an empty query string doesn't clear query string
https://bugs.python.org/issue32779

#32769: Add 'annotations' to the glossary
https://bugs.python.org/issue32769

#32766: 4.7.7. Function Annotations
https://bugs.python.org/issue32766

#32762: Choose protocol implementation on transport.set_protocol()
https://bugs.python.org/issue32762

#32755: Several cookies with the same name get intermixed
https://bugs.python.org/issue32755

#32750: lib2to3 log_error method behavior is inconsitent with document
https://bugs.python.org/issue32750



Most recent 15 issues waiting for review (15)
=============================================

#32793: smtplib: duplicated debug message
https://bugs.python.org/issue32793

#32792: ChainMap should preserve order of underlying mappings
https://bugs.python.org/issue32792

#32788: Better error handling in sqlite3
https://bugs.python.org/issue32788

#32787: Better error handling in ctypes
https://bugs.python.org/issue32787

#32782: ctypes: memoryview gives incorrect PEP3118 itemsize for array 
https://bugs.python.org/issue32782

#32780: ctypes: memoryview gives incorrect PEP3118 format strings for 
https://bugs.python.org/issue32780

#32773: distutils should NOT preserve timestamps
https://bugs.python.org/issue32773

#32742: zipfile extractall needlessly re-wraps ZipInfo instances
https://bugs.python.org/issue32742

#32729: socket error handling needed
https://bugs.python.org/issue32729

#32728: Extend zipfile's compression level support  to LZMA
https://bugs.python.org/issue32728

#32726: macOS installer and framework enhancements and changes for 3.7
https://bugs.python.org/issue32726

#32720: Format mini-language integer definition is incorrect
https://bugs.python.org/issue32720

#32713: tarfile.itn breaks if n is a negative float
https://bugs.python.org/issue32713

#32706: test_check_hostname() of test_ftplib started to fail randomly
https://bugs.python.org/issue32706

#32695: tarfile.open() raises TypeError when using compresslevel param
https://bugs.python.org/issue32695



Top 10 most discussed issues (10)
=================================

#32394: socket lib beahavior change in 3.6.4
https://bugs.python.org/issue32394  20 msgs

#31961: subprocess._execute_child doesn't accept a single PathLike arg
https://bugs.python.org/issue31961  15 msgs

#32616: Significant performance problems with Python 2.7 built with cl
https://bugs.python.org/issue32616  13 msgs

#32773: distutils should NOT preserve timestamps
https://bugs.python.org/issue32773  10 msgs

#32647: Undefined references when compiling ctypes on binutils 2.29.1 
https://bugs.python.org/issue32647   8 msgs

#31356: Add context manager to temporarily disable GC
https://bugs.python.org/issue31356   7 msgs

#32720: Format mini-language integer definition is incorrect
https://bugs.python.org/issue32720   7 msgs

#32768: object.__new__ does not accept arguments if __bases__ is chang
https://bugs.python.org/issue32768   7 msgs

#32808: subprocess.check_output opens an unwanted command line window 
https://bugs.python.org/issue32808   7 msgs

#32751: wait_for(future, ...) should wait for the future (even if a ti
https://bugs.python.org/issue32751   6 msgs



Issues closed (34)
==================

#8722: Documentation for __getattr__
https://bugs.python.org/issue8722  closed by ncoghlan

#30300: asyncio.Controller
https://bugs.python.org/issue30300  closed by barry

#30693: tarfile add uses random order
https://bugs.python.org/issue30693  closed by serhiy.storchaka

#32303: Namespace packages have inconsistent __loader__ and __spec__.l
https://bugs.python.org/issue32303  closed by barry

#32305: Namespace packages have inconsistent __file__ and __spec__.ori
https://bugs.python.org/issue32305  closed by barry

#32368: Segfault when compiling many conditional expressions
https://bugs.python.org/issue32368  closed by snordhausen

#32585: Add ttk::spinbox to tkinter.ttk
https://bugs.python.org/issue32585  closed by serhiy.storchaka

#32614: Fix documentation examples of using re with escape sequences
https://bugs.python.org/issue32614  closed by terry.reedy

#32620: [3.5] Travis CI fails on Python 3.5 with "pyenv: version `3.5'
https://bugs.python.org/issue32620  closed by larry

#32691: "pdb -m <module>" sets __main__.__package__ incorrectly
https://bugs.python.org/issue32691  closed by ncoghlan

#32734: Asyncio Lock safety issue (unlimited acquire)
https://bugs.python.org/issue32734  closed by bar.harel

#32739: collections.deque rotate(n=1) default value not documented
https://bugs.python.org/issue32739  closed by rhettinger

#32743: Typo in hamt.c comments
https://bugs.python.org/issue32743  closed by delimitry

#32746: More misspellings, mostly in source code.
https://bugs.python.org/issue32746  closed by terry.reedy

#32749: Remove dbm.dumb behavior deprecated in 3.6
https://bugs.python.org/issue32749  closed by serhiy.storchaka

#32753: ssl.SSLError exceptions in test_poplib
https://bugs.python.org/issue32753  closed by christian.heimes

#32760: [Python Shell command issue]
https://bugs.python.org/issue32760  closed by matrixise

#32763: write() method in Transport should not buffer data
https://bugs.python.org/issue32763  closed by asvetlov

#32765: IDLE: Update configdialog docstrings to reflect extension inte
https://bugs.python.org/issue32765  closed by terry.reedy

#32770: collections.counter examples are misleading
https://bugs.python.org/issue32770  closed by rhettinger

#32772: lstrip not working when string has =e in it
https://bugs.python.org/issue32772  closed by serhiy.storchaka

#32774: distutils: cyclic reference in the documentation
https://bugs.python.org/issue32774  closed by matrixise

#32775: fnmatch.translate() can produce a pattern which emits a nested
https://bugs.python.org/issue32775  closed by serhiy.storchaka

#32777: subprocess: child_exec() uses _Py_set_inheritable() which is n
https://bugs.python.org/issue32777  closed by gregory.p.smith

#32778: Hi
https://bugs.python.org/issue32778  closed by steven.daprano

#32783: ln(2) isn't accurate in _math.c in cpython
https://bugs.python.org/issue32783  closed by mark.dickinson

#32784: Wrong argument name for csv.DictReader in documentation
https://bugs.python.org/issue32784  closed by Mariatta

#32785: Change the name of the 'f' argument of csv.DictReader and csv.
https://bugs.python.org/issue32785  closed by serhiy.storchaka

#32791: Add\Remove Programs entry is not created.
https://bugs.python.org/issue32791  closed by steve.dower

#32794: PyNumber_Float counterpart that doesn't accept strings
https://bugs.python.org/issue32794  closed by mark.dickinson

#32796: 3.8-dev definition not available
https://bugs.python.org/issue32796  closed by christian.heimes

#32802: Travis does not compile Python if there is one change in the D
https://bugs.python.org/issue32802  closed by Mariatta

#32807: Add 'str' property to PathLib
https://bugs.python.org/issue32807  closed by r.david.murray

#32809: Python 3.6 on Windows problem with source encoding header
https://bugs.python.org/issue32809  closed by eryksun

From larry at hastings.org  Sat Feb 10 21:15:09 2018
From: larry at hastings.org (Larry Hastings)
Date: Sat, 10 Feb 2018 18:15:09 -0800
Subject: [Python-Dev] [RELEASED] Python 3.4.8 and Python 3.5.5 are now
 available
In-Reply-To: <p5mkde$c0i$1@blaine.gmane.org>
References: <7b3f1a47-bdb1-8601-3a68-2b6c5b53b1ac@hastings.org>
 <p5mkde$c0i$1@blaine.gmane.org>
Message-ID: <81687a94-0ed0-1edb-c3f4-1ac6e6577d29@hastings.org>


Actually, it was updated on the server, but somehow the old version was 
sticking around in the CDN cache.? I "purged" it and it's fine now.? 
Weird that it would linger this long!

Cheers,


//arry/

On 02/10/2018 03:20 AM, Serhiy Storchaka wrote:
> 05.02.18 02:35, Larry Hastings ????:
>> On behalf of the Python development community, I'm happy to announce 
>> the availability of Python 3.4.8 and Python 3.5.5.
>>
>> Both Python 3.4 and 3.5 are in "security fixes only" mode.? Both 
>> versions only accept security fixes, not conventional bug fixes, and 
>> both releases are source-only.
>>
>>
>> You can find Python 3.4.8 here:
>>
>> ??? https://www.python.org/downloads/release/python-348/
>>
>>
>> And you can find Python 3.5.5 here:
>>
>> ??? https://www.python.org/downloads/release/python-355/
>
> Online documentation for 3.5 is not updated yet.
>
>
> https://docs.python.org/3.5/whatsnew/changelog.html#python-3-5-5
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180210/da1ebdce/attachment.html>

From storchaka at gmail.com  Sun Feb 11 06:49:02 2018
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sun, 11 Feb 2018 13:49:02 +0200
Subject: [Python-Dev] [RELEASED] Python 3.4.8 and Python 3.5.5 are now
 available
In-Reply-To: <81687a94-0ed0-1edb-c3f4-1ac6e6577d29@hastings.org>
References: <7b3f1a47-bdb1-8601-3a68-2b6c5b53b1ac@hastings.org>
 <p5mkde$c0i$1@blaine.gmane.org>
 <81687a94-0ed0-1edb-c3f4-1ac6e6577d29@hastings.org>
Message-ID: <p5paff$ho9$1@blaine.gmane.org>

11.02.18 04:15, Larry Hastings ????:
> 
> Actually, it was updated on the server, but somehow the old version was 
> sticking around in the CDN cache.? I "purged" it and it's fine now.  
> Weird that it would linger this long!

Great!


From chris.barker at noaa.gov  Mon Feb 12 13:51:06 2018
From: chris.barker at noaa.gov (Chris Barker)
Date: Mon, 12 Feb 2018 10:51:06 -0800
Subject: [Python-Dev] Immutability vs. hashability
In-Reply-To: <20180205233743.GW26553@ando.pearwood.info>
References: <CAP7+vJLQP29XTRVYSYGC7wS5RN9CF7CDu+CTBJtrotaxbNedZw@mail.gmail.com>
 <CADiSq7fezK+tgHNrmw1VVR-STM5nc+pOAPUho9J3MCa_f3RPAA@mail.gmail.com>
 <CAP7+vJJ_aBVvgFd1pzA6=AaMCFRO3wQCJwjujmeryK5+4fd-SQ@mail.gmail.com>
 <20180205233743.GW26553@ando.pearwood.info>
Message-ID: <CALGmxELsPu_XTXubDhtjvegKeTM_zSGkd6Pdxtmovk3ge1+duQ@mail.gmail.com>

On Mon, Feb 5, 2018 at 3:37 PM, Steven D'Aprano <steve at pearwood.info> wrote:

> On Sun, Feb 04, 2018 at 09:18:25PM -0800, Guido van Rossum wrote:
>
> > The way I think of it generally is that immutability is a property of
> > types, while hashability is a property of values.
>
> That's a great way to look at it, thanks.
>

hmm -- maybe we should get a ValueError then when you try to use a
non-hashable value?

In [*9*]: t = ([1,2,3],)


In [*10*]: set(t)

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-10-a2fddf4dafd8> in <module>()

----> 1 set(t)


TypeError: unhashable type: 'list'


Of course, in this case, the error is triggered by the type of the zeroth
element of the tuple, not by the  value of the tuple per se.

Which means that hashability really is a property of type -- but container
types require a specific way of thinking -- hashability is not determined
by the type of the container (or may not be), but by the types of it's
contents. Is that the value of the container?

So maybe: an object is hashable if it is a hashable type, and if it is a
container, if it's contents are hashable types.


With dataclasses as they stand -- it seems the values of the fields does
not affect hashability:

(this is the version 0.4 from PyPi -- disregard if it's out of date)

Unhashable by default:

In [*14*]: @dataclasses.dataclass()

    ...: *class* *NoHash*:

    ...:     x = 5

    ...:     l = [1,2,3]

    ...:


In [*15*]: set((nh,))

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-15-80dc23ed0b24> in <module>()

----> 1 set((nh,))


TypeError: unhashable type: 'NoHash'



OK, that's what we expect.



But then if it is hashable:

In [*19*]: @dataclasses.dataclass(hash=*True*)

    ...: *class* *Hash*:

    ...:     x = 5

    ...:     l = [1,2,3]

    ...:


In [*20*]: h = Hash()


In [*21*]: set((h,))

Out[*21*]: {Hash()}



All works, regardless of the values of the fields

I haven't looked at the code -- but it appears the hash has nothing to do
with the values of the fields:

In [*23*]: hash(h)

Out[*23*]: 3527539


In [*24*]: h.l.append(6)


In [*25*]: hash(h)

Out[*25*]: 3527539


In [*26*]: h.x = 7


In [*27*]: hash(h)

Out[*27*]: 3527539

and it looks like all instances hash the same:

In [*31*]: h2 = Hash()


In [*32*]: hash(h2)

Out[*32*]: 3527539


In [*33*]: hash(h)

Out[*33*]: 3527539

So I'm wondering how hashablility is useful at all?

But it sure looks like there's a lot of room for confusion and error, even
if it's a frozen dataclass.

This may a case where we need to really make sure the docs are good!

-CHB



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180212/364adddc/attachment.html>

From chris.barker at noaa.gov  Mon Feb 12 14:06:20 2018
From: chris.barker at noaa.gov (Chris Barker)
Date: Mon, 12 Feb 2018 11:06:20 -0800
Subject: [Python-Dev] Apology -- ignore previous message
Message-ID: <CALGmxEKf2oOpgnDfiqAEcPvciaHYpCmoDqU2QhJzc+taA7VD0g@mail.gmail.com>

Sorry -- too big a brain blip this morning.. my experiments were'n't
actually creating dataclasses properly.

Though it does show why the reliance on type hinting in troubling!!

IGNORE that message!

Sorry for the noise.

-CHB


On Mon, Feb 5, 2018 at 3:37 PM, Steven D'Aprano <steve at pearwood.info> wrote:

> On Sun, Feb 04, 2018 at 09:18:25PM -0800, Guido van Rossum wrote:
>
> > The way I think of it generally is that immutability is a property of
> > types, while hashability is a property of values.
>
> That's a great way to look at it, thanks.
>
>
> --
> Steve
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> chris.barker%40noaa.gov
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180212/f22cb4c9/attachment.html>

From chris.barker at noaa.gov  Mon Feb 12 14:12:37 2018
From: chris.barker at noaa.gov (Chris Barker)
Date: Mon, 12 Feb 2018 11:12:37 -0800
Subject: [Python-Dev] Immutability vs. hashability
In-Reply-To: <20180205233743.GW26553@ando.pearwood.info>
References: <CAP7+vJLQP29XTRVYSYGC7wS5RN9CF7CDu+CTBJtrotaxbNedZw@mail.gmail.com>
 <CADiSq7fezK+tgHNrmw1VVR-STM5nc+pOAPUho9J3MCa_f3RPAA@mail.gmail.com>
 <CAP7+vJJ_aBVvgFd1pzA6=AaMCFRO3wQCJwjujmeryK5+4fd-SQ@mail.gmail.com>
 <20180205233743.GW26553@ando.pearwood.info>
Message-ID: <CALGmxELPB=7QJKsCc9ejOemhDHXZSAuA2a3=a2HJsZ0qRVLn7Q@mail.gmail.com>

I don't seem to be getting my own messages back to reply to, but:

yes, of course, dataclasses won't hash if a field is mutable:


In [*58*]: @dataclasses.dataclass(hash=*True*, frozen=*True*)

    ...: *class* *Hash*:

    ...:     x: int  = 5

    ...:     l: list = dataclasses.field(default_factory=list)

    ...:

    ...:

    ...:

    ...:


In [*59*]: h = Hash()


In [*60*]: hash(h)

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-60-ad376ae4487e> in <module>()

----> 1 hash(h)


/Users/chris.barker/miniconda2/envs/py3/lib/python3.6/site-packages/dataclasses.py
in __hash__(self)


TypeError: unhashable type: 'list'


And the hash does depend on the values of the fields.So you can REALLY
ignore my previous note.

Again, sorry for the noise.

-CHB



On Mon, Feb 5, 2018 at 3:37 PM, Steven D'Aprano <steve at pearwood.info> wrote:

> On Sun, Feb 04, 2018 at 09:18:25PM -0800, Guido van Rossum wrote:
>
> > The way I think of it generally is that immutability is a property of
> > types, while hashability is a property of values.
>
> That's a great way to look at it, thanks.
>
>
> --
> Steve
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> chris.barker%40noaa.gov
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180212/855fe8ba/attachment-0001.html>

From chris.barker at noaa.gov  Mon Feb 12 14:22:56 2018
From: chris.barker at noaa.gov (Chris Barker)
Date: Mon, 12 Feb 2018 11:22:56 -0800
Subject: [Python-Dev] Immutability vs. hashability
In-Reply-To: <20180206011756.GX26553@ando.pearwood.info>
References: <CAP7+vJLQP29XTRVYSYGC7wS5RN9CF7CDu+CTBJtrotaxbNedZw@mail.gmail.com>
 <CADiSq7fezK+tgHNrmw1VVR-STM5nc+pOAPUho9J3MCa_f3RPAA@mail.gmail.com>
 <CALGmxEJCx=4=qhRnNY0=7oV35D5mx_eYb6M7-0d1TocaAecTRA@mail.gmail.com>
 <20180206011756.GX26553@ando.pearwood.info>
Message-ID: <CALGmxEJbYP-T2XjxSRf6negPKGgn3Hd-MEVZV9YMi9=vjr4N_Q@mail.gmail.com>

On Mon, Feb 5, 2018 at 5:17 PM, Steven D'Aprano <steve at pearwood.info> wrote:

> I'm not happy about the concept of pandering to the least capable, most
> ignorant programmers by baking a miscomprehension into an important
> standard library API.


I don't think this is "baking a miscomprehension", but rather adhering to
the principle of least surprize.


> (Things would be different if we just outright banned mutable+hashable,
> but I don't think anyone wants that.)
>

yeah, though I'm still not sure what the hashable mutable dataclass use
case is.

Fortunately, I also believe that the number of programmers who would
> fail to draw the right conclusion from the existence of separate
> switches will actually be pretty small in practice. The fact that there
> are two separate switches is a pretty big clue that mutability and
> hashability can be controlled separately.
>

well, yes, but as we all know, more people read code than write it -- so we
should be concerned about what a mid-level programmer will conclude when
s/he sees:

@dataclass(frozen=True)

or

@dataclass(hash=True)

It may not be clear that the other options is available, and what it's
default it.


> I believe that the proposed API is much simpler to understand than your
> revision. We have:
>
> - frozen and hash both default to False;
> - if you explicitly set one, the other uses the default.
>
> This corresponds to a very common, Pythonic pattern that nearly
> everyone is familiar with:
>
>     def spam(frozen=False, hash=False):
>         ...
>
> which is easy to understand and easy to explain. Versus your proposal:
>
> - if you set neither, then frozen and hash both default to False;
> - but if you explicitly set one, the other uses True, namely the
>   opposite of the standard default.
>
> which corresponds to something harder to describe and much less common:
>
>     def spam(frozen=None, hash=None):
>         if frozen is hash is None:
>             frozen = hash = False
>         elif frozen is None:
>             frozen = True
>         elif hash is None:
>             hash = True
>         ...
>
> "frozen and hash default to True, unless neither are set, in which case
> they default to False."
>

I agree that it's easier to document, but we all know no one reads
documentation anyway.

My  argument (which may not be correct) is based on the idea that the
hash=True, frozen=False is the rare, specialized use case -- the folks that
need that will figure it out.

(1) I set frozen=True thinking that makes the class hashable so I can
> use it in a set or hash. The first time I actually do so, I get an
> explicit and obvious TypeError. Problem solved.[1]
>
> (2) I set hash=True thinking that makes the class frozen. This scenario
> is more problematic, because there's no explicit and obvious error when
> I get it wrong. Instead, my program could silently do the wrong thing if
> my instances are quietly mutated.
>
> The first error is self-correcting, and so I believe that the second is
> the only one we should worry about.


fair enough.


> - how much should we worry? (how often will this happen?);
>

I probably wouldn't be worried at all, except that the word "frozen" is
used in the built-in frozenset, with this documentation:

"""
The frozenset type is immutable and hashable ? its contents cannot be
altered after it is created; it can therefore be used as a dictionary key
or as an element of another set.
"""

and frozenset pops up pretty high in google if you search "frozen python
frozen" (second to the topic of frozen binaries...)


> I don't think this is a failure mode that we need to be concerned with.
> We can't protect everyone from everything.
>

I suppose you're right -- I think were we disagree is how confusing my
proposal is -- I think the common less error prone use cases should be easy
and obvious to do, and the uncommon, more "expert" use-cases should be
possible and clearly documented when the code is read.

But it's not that big a deal -- I'm done -- thanks for specifically
addressing the issues I brought up.

-CHB



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180212/c2472958/attachment.html>

From status at bugs.python.org  Fri Feb 16 12:09:54 2018
From: status at bugs.python.org (Python tracker)
Date: Fri, 16 Feb 2018 18:09:54 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20180216170954.43D2B11A892@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2018-02-09 - 2018-02-16)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    6456 (+17)
  closed 38153 (+32)
  total  44609 (+49)

Open issues with patches: 2511 


Issues opened (35)
==================

#32810: Expose ags_gen and agt_gen in asynchronous generators
https://bugs.python.org/issue32810  opened by dabeaz

#32813: SSL shared_ciphers implementation wrong - returns configured b
https://bugs.python.org/issue32813  opened by noxxi

#32814: smtplib.send_message mishandles 8BITMIME RFC 6152
https://bugs.python.org/issue32814  opened by Segev Finer

#32818: multiprocessing segmentfault under Windows compatibility mode
https://bugs.python.org/issue32818  opened by Ma Lin

#32819: match_hostname() error reporting bug
https://bugs.python.org/issue32819  opened by christian.heimes

#32820: Add bits method to ipaddress
https://bugs.python.org/issue32820  opened by ewosborne

#32821: Add snippet on how to configure a "split" stream for console
https://bugs.python.org/issue32821  opened by mariocj89

#32822: finally block doesn't re-raise exception if return statement e
https://bugs.python.org/issue32822  opened by David Rebbe

#32823: Regression in test -j behavior and time in 3.7.0b1
https://bugs.python.org/issue32823  opened by terry.reedy

#32824: Docs: Using Python on a Macintosh has bad info per Apple site
https://bugs.python.org/issue32824  opened by griswolf

#32825: warn user of creation of multiple Tk instances
https://bugs.python.org/issue32825  opened by mps

#32829: Lib/ be more pythonic
https://bugs.python.org/issue32829  opened by dilyan.palauzov

#32831: IDLE: Add docstrings and tests for codecontext
https://bugs.python.org/issue32831  opened by csabella

#32832: doctest should support custom ps1/ps2 prompts
https://bugs.python.org/issue32832  opened by Sergey.Kirpichev

#32833: argparse doesn't recognise two option aliases as equal
https://bugs.python.org/issue32833  opened by Krzysztof Leszczy??ski

#32834: test_gdb fails with Posix locale in 3.7
https://bugs.python.org/issue32834  opened by serhiy.storchaka

#32835: Add documention mentioning that Cygwin isn't fully compatible
https://bugs.python.org/issue32835  opened by jayyin11043

#32836: Symbol table for comprehensions (list, dict, set) still includ
https://bugs.python.org/issue32836  opened by mjpieters

#32838: Fix Python versions in the table of magic numbers
https://bugs.python.org/issue32838  opened by serhiy.storchaka

#32839: Add after_info as a function to tkinter
https://bugs.python.org/issue32839  opened by csabella

#32840: Must install python 3.6.3 when 3.6.4 already installed
https://bugs.python.org/issue32840  opened by NaCl

#32841: Asyncio.Condition prevents cancellation
https://bugs.python.org/issue32841  opened by bar.harel

#32843: More revisions to test.support docs
https://bugs.python.org/issue32843  opened by csabella

#32844: subprocess may incorrectly redirect a low fd to stderr if anot
https://bugs.python.org/issue32844  opened by izbyshev

#32846: Deletion of large sets of strings is extra slow
https://bugs.python.org/issue32846  opened by terry.reedy

#32847: Add DirectoryNotEmptyError subclass of OSError
https://bugs.python.org/issue32847  opened by barry

#32849: Fatal Python error: Py_Initialize: can't initialize sys standa
https://bugs.python.org/issue32849  opened by rudolphf

#32850: Run gc_collect() before complaining about dangling threads
https://bugs.python.org/issue32850  opened by smurfix

#32852: trace changes sys.argv from list to tuple
https://bugs.python.org/issue32852  opened by altendky

#32853: struct's docstring implies alignment is always performed
https://bugs.python.org/issue32853  opened by Eli_B

#32854: Add ** Map Unpacking Support for namedtuple
https://bugs.python.org/issue32854  opened by John Crawford

#32855: Add documention stating supported Platforms
https://bugs.python.org/issue32855  opened by jayyin11043

#32856: Optimize the `for y in [x]` idiom in comprehensions
https://bugs.python.org/issue32856  opened by serhiy.storchaka

#32857: tkinter after_cancel does not behave correctly when called wit
https://bugs.python.org/issue32857  opened by csabella

#32858: Improve OpenSSL ECDH support
https://bugs.python.org/issue32858  opened by sruester



Most recent 15 issues with no replies (15)
==========================================

#32855: Add documention stating supported Platforms
https://bugs.python.org/issue32855

#32853: struct's docstring implies alignment is always performed
https://bugs.python.org/issue32853

#32852: trace changes sys.argv from list to tuple
https://bugs.python.org/issue32852

#32849: Fatal Python error: Py_Initialize: can't initialize sys standa
https://bugs.python.org/issue32849

#32847: Add DirectoryNotEmptyError subclass of OSError
https://bugs.python.org/issue32847

#32843: More revisions to test.support docs
https://bugs.python.org/issue32843

#32832: doctest should support custom ps1/ps2 prompts
https://bugs.python.org/issue32832

#32831: IDLE: Add docstrings and tests for codecontext
https://bugs.python.org/issue32831

#32825: warn user of creation of multiple Tk instances
https://bugs.python.org/issue32825

#32823: Regression in test -j behavior and time in 3.7.0b1
https://bugs.python.org/issue32823

#32821: Add snippet on how to configure a "split" stream for console
https://bugs.python.org/issue32821

#32819: match_hostname() error reporting bug
https://bugs.python.org/issue32819

#32805: Possible integer overflow when call PyDTrace_GC_DONE()
https://bugs.python.org/issue32805

#32793: smtplib: duplicated debug message
https://bugs.python.org/issue32793

#32788: Better error handling in sqlite3
https://bugs.python.org/issue32788



Most recent 15 issues waiting for review (15)
=============================================

#32858: Improve OpenSSL ECDH support
https://bugs.python.org/issue32858

#32856: Optimize the `for y in [x]` idiom in comprehensions
https://bugs.python.org/issue32856

#32852: trace changes sys.argv from list to tuple
https://bugs.python.org/issue32852

#32850: Run gc_collect() before complaining about dangling threads
https://bugs.python.org/issue32850

#32844: subprocess may incorrectly redirect a low fd to stderr if anot
https://bugs.python.org/issue32844

#32841: Asyncio.Condition prevents cancellation
https://bugs.python.org/issue32841

#32839: Add after_info as a function to tkinter
https://bugs.python.org/issue32839

#32838: Fix Python versions in the table of magic numbers
https://bugs.python.org/issue32838

#32836: Symbol table for comprehensions (list, dict, set) still includ
https://bugs.python.org/issue32836

#32831: IDLE: Add docstrings and tests for codecontext
https://bugs.python.org/issue32831

#32821: Add snippet on how to configure a "split" stream for console
https://bugs.python.org/issue32821

#32820: Add bits method to ipaddress
https://bugs.python.org/issue32820

#32819: match_hostname() error reporting bug
https://bugs.python.org/issue32819

#32801: Lib/_strptime.py: utilize all()
https://bugs.python.org/issue32801

#32798: mmap.flush() on Linux does not accept the "offset" and "size" 
https://bugs.python.org/issue32798



Top 10 most discussed issues (10)
=================================

#32820: Add bits method to ipaddress
https://bugs.python.org/issue32820  15 msgs

#32839: Add after_info as a function to tkinter
https://bugs.python.org/issue32839  12 msgs

#32841: Asyncio.Condition prevents cancellation
https://bugs.python.org/issue32841  10 msgs

#32813: SSL shared_ciphers implementation wrong - returns configured b
https://bugs.python.org/issue32813   9 msgs

#17050: argparse.REMAINDER doesn't work as first argument
https://bugs.python.org/issue17050   7 msgs

#32758: Stack overflow when parse long expression to AST
https://bugs.python.org/issue32758   7 msgs

#32801: Lib/_strptime.py: utilize all()
https://bugs.python.org/issue32801   6 msgs

#32846: Deletion of large sets of strings is extra slow
https://bugs.python.org/issue32846   5 msgs

#30945: loop.create_server does not detect if the interface is IPv6 en
https://bugs.python.org/issue30945   4 msgs

#31562: snakebite.net is not available
https://bugs.python.org/issue31562   4 msgs



Issues closed (31)
==================

#11015: Bring test.support docs up to date
https://bugs.python.org/issue11015  closed by terry.reedy

#20632: Define a new __key__ protocol
https://bugs.python.org/issue20632  closed by ncoghlan

#26743: Unable to import random with python2.7 on power pc based machi
https://bugs.python.org/issue26743  closed by mark.dickinson

#27846: Base64 expansion factor is 4 to 3, not 6 to 4
https://bugs.python.org/issue27846  closed by berker.peksag

#29248: os.readlink fails on Windows
https://bugs.python.org/issue29248  closed by berker.peksag

#29803: Remove some redandunt ops in unicodeobject.c
https://bugs.python.org/issue29803  closed by xiang.zhang

#30109: make reindent failed.
https://bugs.python.org/issue30109  closed by serhiy.storchaka

#30157: csv.Sniffer.sniff() regex error
https://bugs.python.org/issue30157  closed by serhiy.storchaka

#30688: support named Unicode escapes (\N{name}) in re
https://bugs.python.org/issue30688  closed by serhiy.storchaka

#31787: various refleaks when calling the __init__() method of an obje
https://bugs.python.org/issue31787  closed by inada.naoki

#32370: Wrong ANSI encoding used by subprocess for some locales
https://bugs.python.org/issue32370  closed by serhiy.storchaka

#32436: Implement PEP 567
https://bugs.python.org/issue32436  closed by yselivanov

#32720: Format mini-language integer definition is incorrect
https://bugs.python.org/issue32720  closed by Mariatta

#32781: lzh_tw is missing in locale.py
https://bugs.python.org/issue32781  closed by serhiy.storchaka

#32786: Didnot work strftime() when hangeul in format sting
https://bugs.python.org/issue32786  closed by terry.reedy

#32792: ChainMap should preserve order of underlying mappings
https://bugs.python.org/issue32792  closed by rhettinger

#32800: Replace deprecated link to new page on w3c site
https://bugs.python.org/issue32800  closed by terry.reedy

#32811: test_os.py fails when run in docker container on OSX host
https://bugs.python.org/issue32811  closed by ned.deily

#32812: edited code only runs after closing and re-opening Python.
https://bugs.python.org/issue32812  closed by ned.deily

#32815: Document text parameter to subprocess.Popen
https://bugs.python.org/issue32815  closed by gregory.p.smith

#32816: Python's json dumps/loads make integer keys of the dict str
https://bugs.python.org/issue32816  closed by serhiy.storchaka

#32817: MethodType ImportError, can't convert .py to .exe
https://bugs.python.org/issue32817  closed by r.david.murray

#32826: idle test fails at least on the 3.6 branch
https://bugs.python.org/issue32826  closed by terry.reedy

#32827: Fix incorrect usage of _PyUnicodeWriter_Prepare()
https://bugs.python.org/issue32827  closed by serhiy.storchaka

#32828: compress "True if bool(x) else False" expressions
https://bugs.python.org/issue32828  closed by r.david.murray

#32830: tkinter documentation suggests "from tkinter import *", contra
https://bugs.python.org/issue32830  closed by ned.deily

#32837: IDLE: require encoding argument for textview.view_file
https://bugs.python.org/issue32837  closed by terry.reedy

#32842: Fixing epoll timeout logics
https://bugs.python.org/issue32842  closed by YoSTEALTH

#32845: Mac: Local modules can shadow builtins (e.g. a local `math.py`
https://bugs.python.org/issue32845  closed by ned.deily

#32848: Valgrind error with python
https://bugs.python.org/issue32848  closed by r.david.murray

#32851: mistake
https://bugs.python.org/issue32851  closed by r.david.murray

From benlewisj at gmail.com  Fri Feb 16 18:38:44 2018
From: benlewisj at gmail.com (Ben Lewis)
Date: Sat, 17 Feb 2018 12:38:44 +1300
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
Message-ID: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>

Hello

I have been using dataclasses package in a pet project of mine. I'm sorry
if this issue has already been raised. I came across a situation where I
wanted to use the __post_init__ function to initialise some inherited
fields from a dataclass with frozen=True. The problem is that because it is
frozen, assigning to the field doesn't work.

There are two workarounds without changing the base class to frozen=False,
which could be in a library.

1. Use object.__setattr__, this is ugly and not very user or beginner
friendly.
2. Extract __post_init__ out into a factory function. Then it also loses
all the advantages of the __post_init__ and InitVar mechanism.

Both frozen and unfrozen dataclasses should be able to use the same
initialisation mechanism for consistency. Being consistent would ease of
converting an unfrozen dataclass to a frozen one if the only code that
actually modifies the instance is in __post_init__ function.

I think frozen classes should be able to be mutated during the
__post_init__ call. To implements this a frozen dataclass could have a flag
to says it's not yet fully initialised and the flag would be checked in the
frozen setattr/delattr methods. This flag could be located as a special
attribute on the instance or be in a weak reference dict.

Thanks
Ben Lewis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180217/a8c74809/attachment.html>

From guido at python.org  Sat Feb 17 00:40:01 2018
From: guido at python.org (Guido van Rossum)
Date: Fri, 16 Feb 2018 21:40:01 -0800
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
Message-ID: <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>

On Fri, Feb 16, 2018 at 3:38 PM, Ben Lewis <benlewisj at gmail.com> wrote:

> I have been using dataclasses package in a pet project of mine. I'm sorry
> if this issue has already been raised. I came across a situation where I
> wanted to use the __post_init__ function to initialise some inherited
> fields from a dataclass with frozen=True. The problem is that because it is
> frozen, assigning to the field doesn't work.
>
> There are two workarounds without changing the base class to frozen=False,
> which could be in a library.
>
> 1. Use object.__setattr__, this is ugly and not very user or beginner
> friendly.
> 2. Extract __post_init__ out into a factory function. Then it also loses
> all the advantages of the __post_init__ and InitVar mechanism.
>
> Both frozen and unfrozen dataclasses should be able to use the same
> initialisation mechanism for consistency. Being consistent would ease of
> converting an unfrozen dataclass to a frozen one if the only code that
> actually modifies the instance is in __post_init__ function.
>
> I think frozen classes should be able to be mutated during the
> __post_init__ call. To implements this a frozen dataclass could have a flag
> to says it's not yet fully initialised and the flag would be checked in the
> frozen setattr/delattr methods. This flag could be located as a special
> attribute on the instance or be in a weak reference dict.
>

That's a pretty tricky proposal, and one that's been debated on and off for
a long time in other contexts. And that flag would somehow have to be part
of every instance's state.

In general the right way to initialize an immutable/frozen object is not
through __init__ but through __new__ -- have you tried that?

Also, a small example that demonstrates your need would do wonders to help
us understand your use case better.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180216/e5c3a079/attachment.html>

From benlewisj at gmail.com  Sat Feb 17 02:43:48 2018
From: benlewisj at gmail.com (Ben Lewis)
Date: Sat, 17 Feb 2018 20:43:48 +1300
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
Message-ID: <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>

On Sat, Feb 17, 2018 at 6:40 PM, Guido van Rossum <guido at python.org> wrote:
>
>
> That's a pretty tricky proposal, and one that's been debated on and off
> for a long time in other contexts. And that flag would somehow have to be
> part of every instance's state.
>

> In general the right way to initialize an immutable/frozen object is not
> through __init__ but through __new__ -- have you tried that?
>

Constructing it throught __new__ doesn't actually work as it has no way to
alter the arguments that are passed into __init__, I think creating a
metaclass that overides __call__ is required to acheive the desired result.
Although a factory classmethod would acheive similar api.


>
> Also, a small example that demonstrates your need would do wonders to help
> us understand your use case better.
>
>

# unrelated object
class NamedObject:
    @property
    def name(self) -> str:
        return "some name"

// has may subclasses
@dataclass
class Item:
    name: str


@dataclass
class NamedObjectItem(Item):
    name: str = field(init=False)
    obj: NamedObject

    def __post_init__(self):
        self.name = self.obj.name

This works fine, until I decided them Item and therefore all subclasses
should be frozen as no instances are mutated
and if they are ever in the future then its a bug. But to do this the
following factory method needs to be added:

@classmethod
def create(cls, obj: NamedObject, *args, **kwargs):
    return cls(obj.name, obj, *args, **kwargs)

This doesn't look that bad but all fields(up to the last field used that
would have been used in __post_init__) needs to be declared in the
signature.

Thanks
Ben Lewis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180217/a63625fd/attachment.html>

From guido at python.org  Sat Feb 17 14:35:48 2018
From: guido at python.org (Guido van Rossum)
Date: Sat, 17 Feb 2018 11:35:48 -0800
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
Message-ID: <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>

Why can'y you make `name` on `NamedObjectItem` a property that returns `
self.obj.name`? Why store a duplicate copy of the name?

PS. I have to ponder why frozen dataclasses don't use `__new__`.

On Fri, Feb 16, 2018 at 11:43 PM, Ben Lewis <benlewisj at gmail.com> wrote:

> On Sat, Feb 17, 2018 at 6:40 PM, Guido van Rossum <guido at python.org>
> wrote:
>>
>>
>> That's a pretty tricky proposal, and one that's been debated on and off
>> for a long time in other contexts. And that flag would somehow have to be
>> part of every instance's state.
>>
>
>> In general the right way to initialize an immutable/frozen object is not
>> through __init__ but through __new__ -- have you tried that?
>>
>
> Constructing it throught __new__ doesn't actually work as it has no way to
> alter the arguments that are passed into __init__, I think creating a
> metaclass that overides __call__ is required to acheive the desired result.
> Although a factory classmethod would acheive similar api.
>
>
>>
>> Also, a small example that demonstrates your need would do wonders to
>> help us understand your use case better.
>>
>>
>
> # unrelated object
> class NamedObject:
>     @property
>     def name(self) -> str:
>         return "some name"
>
> // has may subclasses
> @dataclass
> class Item:
>     name: str
>
>
> @dataclass
> class NamedObjectItem(Item):
>     name: str = field(init=False)
>     obj: NamedObject
>
>     def __post_init__(self):
>         self.name = self.obj.name
>
> This works fine, until I decided them Item and therefore all subclasses
> should be frozen as no instances are mutated
> and if they are ever in the future then its a bug. But to do this the
> following factory method needs to be added:
>
> @classmethod
> def create(cls, obj: NamedObject, *args, **kwargs):
>     return cls(obj.name, obj, *args, **kwargs)
>
> This doesn't look that bad but all fields(up to the last field used that
> would have been used in __post_init__) needs to be declared in the
> signature.
>
> Thanks
> Ben Lewis
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180217/9eec6404/attachment.html>

From ramazanpolat at gmail.com  Sat Feb 17 18:14:28 2018
From: ramazanpolat at gmail.com (Ramazan POLAT)
Date: Sun, 18 Feb 2018 02:14:28 +0300
Subject: [Python-Dev] Replacement proposal for PEP 557 Data Classes
Message-ID: <CAEySH2dHa_REWs82j4b_axOug8G+ADZZpoLPrhF3mRzawmY9xQ@mail.gmail.com>

Hi, I have been working on a new way to use Python classes as enhanced dict
object(I called it Prodict).
My solution is (IMHO) more concise and brings more features.
So I wanted to make it a PEP until I saw PEP 557 Data Classes.
With my proposed enhancement, I can do everything Data Classes can do plus
some neat features.
Since 3.7b1 is out and already integrates data classes, how do I proceed?
Propose a replacement for PEP 557 or prepare a new PEP?

Although my approach doesn't use decorators, it can be easily done with
decorators as well.
You can check my solution here: https://github.com/ramazanpolat/prodict
____________
Ramazan Polat
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180218/71c68135/attachment.html>

From benlewisj at gmail.com  Sat Feb 17 20:59:57 2018
From: benlewisj at gmail.com (Ben Lewis)
Date: Sun, 18 Feb 2018 14:59:57 +1300
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
Message-ID: <CAD4jWF8-ihCodcEwX-WNMQ2Fwd7+QReXiv+iYmxsRMBMKEw=oA@mail.gmail.com>

>
> Why can'y you make `name` on `NamedObjectItem` a property that returns `
> self.obj.name`? Why store a duplicate copy of the name?
>

Agreed, it's probably a better design not to store a duplicate reference to
name. But when I tried that, the property clashed with the inherited field.
This caused the creation of the dataclass to fail as it thought that the
property was the default value for the field 'name'. Even if I set a
default for the obj field, it crashed as it tried to set the default value
for name to the read-only property.

Although I can think of situations where properties wouldn't be sufficent
as you only want to calculate the value once per instance on creation. My
thought is that most dataclasses would still be sensible and useful even if
all mutation ability was removed from them. Taking an example directly from
the PEP:

@dataclass
class C:
    i: int
    j: int = None
    database: InitVar[DatabaseType] = None

    def __post_init__(self, database):
        if self.j is None and database is not None:
            self.j = database.lookup('j')

Maybe I'm thinking of dataclasses wrong but this still make complete sense
and is useful even if its declared as frozen.

My thought is that initialisation logic and immutability is orthogonal to
each other. Possibly initialisation logic like this should occur before the
instance is created so it would work for immutable types as well.

A possible idea could be, instead of __post_init__, there is __pre_init__
which allows altering of fields before the instance is created. It would
take a dict as first argument which contain the field values passed into
the 'constructor' and default values would also be filled out.

@dataclass
class C:
    i: int
    j: int = None
    database: InitVar[DatabaseType]

    @classmethod
    def __pre_init__(cls, fields: Dict[str, Any], database: DatabaseType):
        if fields['j'] is None and database is not None:
            fields['j'] = database.lookup('j')

I personally see two problems with this idea:
1. This isn't as ergonomic as __post_init__ is as its modifing a dictionary
instead of its instance.
2. To implement this, it would require a metaclass.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180218/3bf260ee/attachment.html>

From guido at python.org  Sat Feb 17 23:10:57 2018
From: guido at python.org (Guido van Rossum)
Date: Sat, 17 Feb 2018 20:10:57 -0800
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <CAD4jWF8-ihCodcEwX-WNMQ2Fwd7+QReXiv+iYmxsRMBMKEw=oA@mail.gmail.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
 <CAD4jWF8-ihCodcEwX-WNMQ2Fwd7+QReXiv+iYmxsRMBMKEw=oA@mail.gmail.com>
Message-ID: <CAP7+vJLNzhEbFD-6LsE6KdD8tpN_+w7p7GDNgi_Mk+pbwsF0kQ@mail.gmail.com>

Agreed the __pre_init__ idea is no improvement. I think we're back where
you started -- just use `object.__setattr__` to set the attribute in
`__post_init__`. That's what the PEP says is used by the generated
`__init__`, so I think it is reasonable to copy that pattern. Presumably
the situation doesn't occur that frequently in real code (__post_init__
feels like a last resort hack anyway).

On Sat, Feb 17, 2018 at 5:59 PM, Ben Lewis <benlewisj at gmail.com> wrote:

> Why can'y you make `name` on `NamedObjectItem` a property that returns `
>> self.obj.name`? Why store a duplicate copy of the name?
>>
>
> Agreed, it's probably a better design not to store a duplicate reference
> to name. But when I tried that, the property clashed with the inherited
> field. This caused the creation of the dataclass to fail as it thought that
> the property was the default value for the field 'name'. Even if I set a
> default for the obj field, it crashed as it tried to set the default value
> for name to the read-only property.
>
> Although I can think of situations where properties wouldn't be sufficent
> as you only want to calculate the value once per instance on creation. My
> thought is that most dataclasses would still be sensible and useful even
> if all mutation ability was removed from them. Taking an example directly
> from the PEP:
>
> @dataclass
> class C:
>     i: int
>     j: int = None
>     database: InitVar[DatabaseType] = None
>
>     def __post_init__(self, database):
>         if self.j is None and database is not None:
>             self.j = database.lookup('j')
>
> Maybe I'm thinking of dataclasses wrong but this still make complete sense
> and is useful even if its declared as frozen.
>
> My thought is that initialisation logic and immutability is orthogonal to
> each other. Possibly initialisation logic like this should occur before the
> instance is created so it would work for immutable types as well.
>
> A possible idea could be, instead of __post_init__, there is __pre_init__
> which allows altering of fields before the instance is created. It would
> take a dict as first argument which contain the field values passed into
> the 'constructor' and default values would also be filled out.
>
> @dataclass
> class C:
>     i: int
>     j: int = None
>     database: InitVar[DatabaseType]
>
>     @classmethod
>     def __pre_init__(cls, fields: Dict[str, Any], database: DatabaseType):
>         if fields['j'] is None and database is not None:
>             fields['j'] = database.lookup('j')
>
> I personally see two problems with this idea:
> 1. This isn't as ergonomic as __post_init__ is as its modifing a
> dictionary instead of its instance.
> 2. To implement this, it would require a metaclass.
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180217/a913d259/attachment.html>

From ncoghlan at gmail.com  Sun Feb 18 19:26:59 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 19 Feb 2018 10:26:59 +1000
Subject: [Python-Dev] Replacement proposal for PEP 557 Data Classes
In-Reply-To: <CAEySH2dHa_REWs82j4b_axOug8G+ADZZpoLPrhF3mRzawmY9xQ@mail.gmail.com>
References: <CAEySH2dHa_REWs82j4b_axOug8G+ADZZpoLPrhF3mRzawmY9xQ@mail.gmail.com>
Message-ID: <CADiSq7eLf46OO40hprJwBNpvT_SJTht05Xm7CL=NNq5qnucoRA@mail.gmail.com>

On 18 February 2018 at 09:14, Ramazan POLAT <ramazanpolat at gmail.com> wrote:
> Hi, I have been working on a new way to use Python classes as enhanced dict
> object(I called it Prodict).
> My solution is (IMHO) more concise and brings more features.
> So I wanted to make it a PEP until I saw PEP 557 Data Classes.
> With my proposed enhancement, I can do everything Data Classes can do plus
> some neat features.
> Since 3.7b1 is out and already integrates data classes, how do I proceed?
> Propose a replacement for PEP 557 or prepare a new PEP?

The introduction of data classes was prompted by the popularity of the
3rd party attrs library, so the first step would be to achieve that
level of popularity.

However, with instances being dict instances, there are quite a few
attrs/dataclasses features that I expect you would find difficult to
replicate (hashability, immutable instances, ordered comparison
support), so your library strikes as being more an advanced
alternative to types.SimpleNamespace rather than an alternative to
data classes.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sun Feb 18 19:34:28 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 19 Feb 2018 10:34:28 +1000
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <CAP7+vJLNzhEbFD-6LsE6KdD8tpN_+w7p7GDNgi_Mk+pbwsF0kQ@mail.gmail.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
 <CAD4jWF8-ihCodcEwX-WNMQ2Fwd7+QReXiv+iYmxsRMBMKEw=oA@mail.gmail.com>
 <CAP7+vJLNzhEbFD-6LsE6KdD8tpN_+w7p7GDNgi_Mk+pbwsF0kQ@mail.gmail.com>
Message-ID: <CADiSq7c=f=jKHspyTEGE1s_MXEsJ3UaMJ5z=Q9Y4a89sD5PtGA@mail.gmail.com>

On 18 February 2018 at 14:10, Guido van Rossum <guido at python.org> wrote:
> Agreed the __pre_init__ idea is no improvement. I think we're back where you
> started -- just use `object.__setattr__` to set the attribute in
> `__post_init__`. That's what the PEP says is used by the generated
> `__init__`, so I think it is reasonable to copy that pattern. Presumably the
> situation doesn't occur that frequently in real code (__post_init__ feels
> like a last resort hack anyway).

FWIW, if someone really wanted to create a 3rd party context manager
to assist with this they can do:

    @contextmanager
    def mutable(obj):
        cls = obj.__class__
        obj.__class__ = object
        try:
            yield obj
        finally:
            obj.__class__ = cls

    @dataclass(frozen=True)
    class C:
       i: int
       j: int = None
       database: InitVar[DatabaseType] = None

       def __post_init__(self, database):
           if self.j is None and database is not None:
               with mutable(self):
                   self.j = database.lookup('j')

Using object.__setattr__ explicitly would be clearer, though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From guido at python.org  Sun Feb 18 21:30:16 2018
From: guido at python.org (Guido van Rossum)
Date: Sun, 18 Feb 2018 18:30:16 -0800
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <CADiSq7c=f=jKHspyTEGE1s_MXEsJ3UaMJ5z=Q9Y4a89sD5PtGA@mail.gmail.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
 <CAD4jWF8-ihCodcEwX-WNMQ2Fwd7+QReXiv+iYmxsRMBMKEw=oA@mail.gmail.com>
 <CAP7+vJLNzhEbFD-6LsE6KdD8tpN_+w7p7GDNgi_Mk+pbwsF0kQ@mail.gmail.com>
 <CADiSq7c=f=jKHspyTEGE1s_MXEsJ3UaMJ5z=Q9Y4a89sD5PtGA@mail.gmail.com>
Message-ID: <CAP7+vJKBZ+Z6e_ERUy+siq6USqk3fCa6+MVFx0CqbhQawmydBQ@mail.gmail.com>

The underlying issue here is that we don't want an extra state flag in the
object to indicate "this object is currently [im]mutable". Using __class__
assignment to signal this is clever way to add this state, though not
without risks.

On Sun, Feb 18, 2018 at 4:34 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 18 February 2018 at 14:10, Guido van Rossum <guido at python.org> wrote:
> > Agreed the __pre_init__ idea is no improvement. I think we're back where
> you
> > started -- just use `object.__setattr__` to set the attribute in
> > `__post_init__`. That's what the PEP says is used by the generated
> > `__init__`, so I think it is reasonable to copy that pattern. Presumably
> the
> > situation doesn't occur that frequently in real code (__post_init__ feels
> > like a last resort hack anyway).
>
> FWIW, if someone really wanted to create a 3rd party context manager
> to assist with this they can do:
>
>     @contextmanager
>     def mutable(obj):
>         cls = obj.__class__
>         obj.__class__ = object
>         try:
>             yield obj
>         finally:
>             obj.__class__ = cls
>
>     @dataclass(frozen=True)
>     class C:
>        i: int
>        j: int = None
>        database: InitVar[DatabaseType] = None
>
>        def __post_init__(self, database):
>            if self.j is None and database is not None:
>                with mutable(self):
>                    self.j = database.lookup('j')
>
> Using object.__setattr__ explicitly would be clearer, though.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180218/1ceeabbf/attachment.html>

From chris.barker at noaa.gov  Mon Feb 19 20:06:03 2018
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Mon, 19 Feb 2018 20:06:03 -0500
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <CAP7+vJKBZ+Z6e_ERUy+siq6USqk3fCa6+MVFx0CqbhQawmydBQ@mail.gmail.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
 <CAD4jWF8-ihCodcEwX-WNMQ2Fwd7+QReXiv+iYmxsRMBMKEw=oA@mail.gmail.com>
 <CAP7+vJLNzhEbFD-6LsE6KdD8tpN_+w7p7GDNgi_Mk+pbwsF0kQ@mail.gmail.com>
 <CADiSq7c=f=jKHspyTEGE1s_MXEsJ3UaMJ5z=Q9Y4a89sD5PtGA@mail.gmail.com>
 <CAP7+vJKBZ+Z6e_ERUy+siq6USqk3fCa6+MVFx0CqbhQawmydBQ@mail.gmail.com>
Message-ID: <CALGmxEJH-MERdNTc5XMP3TJUo=nAnwt_0RBNnx7eVHVjnxyGLw@mail.gmail.com>

If I have this right, on the discussion about frozen and hash, a use
case was brought up for taking a few steps to create an instance (and
thus wanting it not frozen) and then wanting it hashable.

Which pointed to the idea of a ? freeze this from now on? method.

This seems another use case ? maybe it would be helpful to be able to
freeze an instance after creation for multiple use-cases?

-CHB

From guido at python.org  Mon Feb 19 22:02:42 2018
From: guido at python.org (Guido van Rossum)
Date: Mon, 19 Feb 2018 19:02:42 -0800
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <CALGmxEJH-MERdNTc5XMP3TJUo=nAnwt_0RBNnx7eVHVjnxyGLw@mail.gmail.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
 <CAD4jWF8-ihCodcEwX-WNMQ2Fwd7+QReXiv+iYmxsRMBMKEw=oA@mail.gmail.com>
 <CAP7+vJLNzhEbFD-6LsE6KdD8tpN_+w7p7GDNgi_Mk+pbwsF0kQ@mail.gmail.com>
 <CADiSq7c=f=jKHspyTEGE1s_MXEsJ3UaMJ5z=Q9Y4a89sD5PtGA@mail.gmail.com>
 <CAP7+vJKBZ+Z6e_ERUy+siq6USqk3fCa6+MVFx0CqbhQawmydBQ@mail.gmail.com>
 <CALGmxEJH-MERdNTc5XMP3TJUo=nAnwt_0RBNnx7eVHVjnxyGLw@mail.gmail.com>
Message-ID: <CAP7+vJ+d_9HcE=KmNfVXZudTmDmtDSauTnU=RT+2b3FU1Hga3w@mail.gmail.com>

But how?

On Mon, Feb 19, 2018 at 5:06 PM, Chris Barker - NOAA Federal <
chris.barker at noaa.gov> wrote:

> If I have this right, on the discussion about frozen and hash, a use
> case was brought up for taking a few steps to create an instance (and
> thus wanting it not frozen) and then wanting it hashable.
>
> Which pointed to the idea of a ? freeze this from now on? method.
>
> This seems another use case ? maybe it would be helpful to be able to
> freeze an instance after creation for multiple use-cases?
>
> -CHB
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180219/a8b0cbd3/attachment.html>

From chris.barker at noaa.gov  Mon Feb 19 23:16:00 2018
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Mon, 19 Feb 2018 23:16:00 -0500
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <CAP7+vJ+d_9HcE=KmNfVXZudTmDmtDSauTnU=RT+2b3FU1Hga3w@mail.gmail.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
 <CAD4jWF8-ihCodcEwX-WNMQ2Fwd7+QReXiv+iYmxsRMBMKEw=oA@mail.gmail.com>
 <CAP7+vJLNzhEbFD-6LsE6KdD8tpN_+w7p7GDNgi_Mk+pbwsF0kQ@mail.gmail.com>
 <CADiSq7c=f=jKHspyTEGE1s_MXEsJ3UaMJ5z=Q9Y4a89sD5PtGA@mail.gmail.com>
 <CAP7+vJKBZ+Z6e_ERUy+siq6USqk3fCa6+MVFx0CqbhQawmydBQ@mail.gmail.com>
 <CALGmxEJH-MERdNTc5XMP3TJUo=nAnwt_0RBNnx7eVHVjnxyGLw@mail.gmail.com>
 <CAP7+vJ+d_9HcE=KmNfVXZudTmDmtDSauTnU=RT+2b3FU1Hga3w@mail.gmail.com>
Message-ID: <CALGmxEJkoq4roEY_YbGKdc+J22vgW_a_ROyOriR7jjg1em0Jbg@mail.gmail.com>

But how?


Well, I hadn?t thought that far ;-)

But it would make frozen an instance level property, rather than a
class-level one ? some instances would be frozen, some not.

Which would be kinda compatible with the idea of hashability being a
property of values, rather than type. Frozen-ness would also be a property
of values.

But it would require a flag, and a bunch of logic overhead, which could get
pretty ugly so maybe not worth it.

-CHB



On Mon, Feb 19, 2018 at 5:06 PM, Chris Barker - NOAA Federal <
chris.barker at noaa.gov> wrote:

> If I have this right, on the discussion about frozen and hash, a use
> case was brought up for taking a few steps to create an instance (and
> thus wanting it not frozen) and then wanting it hashable.
>
> Which pointed to the idea of a ? freeze this from now on? method.
>
> This seems another use case ? maybe it would be helpful to be able to
> freeze an instance after creation for multiple use-cases?
>
> -CHB
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180219/8053cddc/attachment.html>

From v+python at g.nevcal.com  Mon Feb 19 23:16:41 2018
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 19 Feb 2018 20:16:41 -0800
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <CAP7+vJ+d_9HcE=KmNfVXZudTmDmtDSauTnU=RT+2b3FU1Hga3w@mail.gmail.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
 <CAD4jWF8-ihCodcEwX-WNMQ2Fwd7+QReXiv+iYmxsRMBMKEw=oA@mail.gmail.com>
 <CAP7+vJLNzhEbFD-6LsE6KdD8tpN_+w7p7GDNgi_Mk+pbwsF0kQ@mail.gmail.com>
 <CADiSq7c=f=jKHspyTEGE1s_MXEsJ3UaMJ5z=Q9Y4a89sD5PtGA@mail.gmail.com>
 <CAP7+vJKBZ+Z6e_ERUy+siq6USqk3fCa6+MVFx0CqbhQawmydBQ@mail.gmail.com>
 <CALGmxEJH-MERdNTc5XMP3TJUo=nAnwt_0RBNnx7eVHVjnxyGLw@mail.gmail.com>
 <CAP7+vJ+d_9HcE=KmNfVXZudTmDmtDSauTnU=RT+2b3FU1Hga3w@mail.gmail.com>
Message-ID: <f26a13a0-d816-1e72-79d0-60439d4875da@g.nevcal.com>

On 2/19/2018 7:02 PM, Guido van Rossum wrote:
> But how?
>
> On Mon, Feb 19, 2018 at 5:06 PM, Chris Barker - NOAA Federal 
> <chris.barker at noaa.gov <mailto:chris.barker at noaa.gov>> wrote:
>
>     ... maybe it would be helpful to be able to
>     freeze an instance after creation for multiple use-cases?
>

And there's the crux of the issue... if the creator of Python can't 
figure out how to make mutable objects immutable by fiat after creation, 
then it is unlikely anyone else can!

Yet it seems there are use cases for that sort of ability.

Is it not possible to strip an object of its operations that mutate it? 
Probably it is, but, in doing so, it might strip the whole class of the 
operations that mutate it, rendering it impossible to the create 
additional mutable instances of the same class. In another message, 
Guido suggests that the solution of having a flag for each object, that 
presumably would be checked by each mutation operation, is not a 
wonderful idea:

On 2/18/2018 6:30 PM, Guido van Rossum wrote:
> The underlying issue here is that we don't want an extra state flag in 
> the object to indicate "this object is currently [im]mutable". Using 
> __class__ assignment to signal this is clever way to add this state, 
> though not without risks.

My only suggestion here is to add a new type of class to the language: 
one that would effectively be two nearly identical classes under the 
covers, having all the same interfaces and internal data structures.? 
Both internal classes would be subclasses of? the new class, for is-a 
purposes.? But they would be defined in such a way (if there is such a 
way) as to minimize the risks referred to just above (I don't know what 
those risks are), so that mutable instances could become immutable via 
__class__ assignment.

So just to play with this a bit in high-level syntax:

freezable_class Foo:? # or maybe ? ? class Foo( freezable ):
 ?? int bar;

All instances of Foo would first be created as __mutable__Foo.? Both 
__mutable__Foo and __immutable__Foo would be instance_of( Foo ).

Foo xyz

xyz.bar = 17? # works fine
xyz.bar += 1 # works fine
if xyz.bar == 18:
 ???? pass? # True
if instance_of( xyz, Foo ):
 ???? pass? # True
if instance_of( xyz, __mutable__Foo ):
pass?? # True

Foo would have an implicitly defined method (or defined on the parent 
freezable class) that would convert a class from __mutable__Foo? to? 
__immutable__Foo.? There would be no reverse conversion.

xyz.__immutable__()? # under the covers, assigns xyz.__class__ to 
__immutable__Foo

Mutating methods applied to __immutable__Foo instances would cause 
exceptions, because they wouldn't exist for the __immutable__Foo subclass.

if xyz.bar == 18:
 ???? pass? # True
if instance_of( xyz, __mutable__Foo ):
pass?? # False
else:
 ??? print("Can't change it any more.")

xyz.bar? = 19? # throws: xyz is now immutable

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180219/da5f5027/attachment.html>

From steve at pearwood.info  Tue Feb 20 00:11:47 2018
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 20 Feb 2018 16:11:47 +1100
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <CAP7+vJ+d_9HcE=KmNfVXZudTmDmtDSauTnU=RT+2b3FU1Hga3w@mail.gmail.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
 <CAD4jWF8-ihCodcEwX-WNMQ2Fwd7+QReXiv+iYmxsRMBMKEw=oA@mail.gmail.com>
 <CAP7+vJLNzhEbFD-6LsE6KdD8tpN_+w7p7GDNgi_Mk+pbwsF0kQ@mail.gmail.com>
 <CADiSq7c=f=jKHspyTEGE1s_MXEsJ3UaMJ5z=Q9Y4a89sD5PtGA@mail.gmail.com>
 <CAP7+vJKBZ+Z6e_ERUy+siq6USqk3fCa6+MVFx0CqbhQawmydBQ@mail.gmail.com>
 <CALGmxEJH-MERdNTc5XMP3TJUo=nAnwt_0RBNnx7eVHVjnxyGLw@mail.gmail.com>
 <CAP7+vJ+d_9HcE=KmNfVXZudTmDmtDSauTnU=RT+2b3FU1Hga3w@mail.gmail.com>
Message-ID: <20180220051146.GH10142@ando.pearwood.info>

On Mon, Feb 19, 2018 at 07:02:42PM -0800, Guido van Rossum wrote:

> On Mon, Feb 19, 2018 at 5:06 PM, Chris Barker - NOAA Federal <
> chris.barker at noaa.gov> wrote:
[...]
> > This seems another use case ? maybe it would be helpful to be able to
> > freeze an instance after creation for multiple use-cases?
>
> But how?

I haven't got all the details yet, but how badly do you hate the
technique of swapping out __class__ to get a change in behaviour?

https://www.safaribooksonline.com/library/view/python-cookbook/0596001673/ch05s19.html

or https://code.activestate.com/recipes/68429-ring-buffer/

(I believe that in the Objective C community, something very close to 
this is called "swizzling".)

So in principle, we could have a mutable class, and a immutable one, and 
when you flick the switch, the instance.__class__ changes from mutable 
to frozen.

If you don't hate this, we can think about the details needed to get 
it work in practice.



-- 
Steve

From ncoghlan at gmail.com  Tue Feb 20 08:03:25 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 20 Feb 2018 23:03:25 +1000
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <20180220051146.GH10142@ando.pearwood.info>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
 <CAD4jWF8-ihCodcEwX-WNMQ2Fwd7+QReXiv+iYmxsRMBMKEw=oA@mail.gmail.com>
 <CAP7+vJLNzhEbFD-6LsE6KdD8tpN_+w7p7GDNgi_Mk+pbwsF0kQ@mail.gmail.com>
 <CADiSq7c=f=jKHspyTEGE1s_MXEsJ3UaMJ5z=Q9Y4a89sD5PtGA@mail.gmail.com>
 <CAP7+vJKBZ+Z6e_ERUy+siq6USqk3fCa6+MVFx0CqbhQawmydBQ@mail.gmail.com>
 <CALGmxEJH-MERdNTc5XMP3TJUo=nAnwt_0RBNnx7eVHVjnxyGLw@mail.gmail.com>
 <CAP7+vJ+d_9HcE=KmNfVXZudTmDmtDSauTnU=RT+2b3FU1Hga3w@mail.gmail.com>
 <20180220051146.GH10142@ando.pearwood.info>
Message-ID: <CADiSq7fKum8KBsPYrSZV7zx5-W_3mXXXcFQeqXv-GYbF5n_D2g@mail.gmail.com>

On 20 February 2018 at 15:11, Steven D'Aprano <steve at pearwood.info> wrote:
> So in principle, we could have a mutable class, and a immutable one, and
> when you flick the switch, the instance.__class__ changes from mutable
> to frozen.
>
> If you don't hate this, we can think about the details needed to get
> it work in practice.

This doesn't technically require any additional changes, as you can
already do it with __post_init__:

    @dataclass
    class MyRecord:
        a: int
        b: str
        c: float

        def __post_init__(self):
            # self is still mutable here
            self.__class__ = _LockedMyRecord

    @dataclass(frozen=True)
    class _LockedMyRecord(MyRecord):
        pass

This is also the kind of runtime behaviour modification where hiding
how it's implemented is likely to create more problems than it solves,
as even though having "type(obj)" and "obj.__class__" refer to
different types is a formally supported state for instances, there are
also lots of APIs where it isn't well defined whether an attribute
lookup will use the nominal class or the true underlying type.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From guido at python.org  Tue Feb 20 11:45:07 2018
From: guido at python.org (Guido van Rossum)
Date: Tue, 20 Feb 2018 08:45:07 -0800
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <f26a13a0-d816-1e72-79d0-60439d4875da@g.nevcal.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
 <CAD4jWF8-ihCodcEwX-WNMQ2Fwd7+QReXiv+iYmxsRMBMKEw=oA@mail.gmail.com>
 <CAP7+vJLNzhEbFD-6LsE6KdD8tpN_+w7p7GDNgi_Mk+pbwsF0kQ@mail.gmail.com>
 <CADiSq7c=f=jKHspyTEGE1s_MXEsJ3UaMJ5z=Q9Y4a89sD5PtGA@mail.gmail.com>
 <CAP7+vJKBZ+Z6e_ERUy+siq6USqk3fCa6+MVFx0CqbhQawmydBQ@mail.gmail.com>
 <CALGmxEJH-MERdNTc5XMP3TJUo=nAnwt_0RBNnx7eVHVjnxyGLw@mail.gmail.com>
 <CAP7+vJ+d_9HcE=KmNfVXZudTmDmtDSauTnU=RT+2b3FU1Hga3w@mail.gmail.com>
 <f26a13a0-d816-1e72-79d0-60439d4875da@g.nevcal.com>
Message-ID: <CAP7+vJJ3Q=XdYB51A=tNjd4O=UDDo4iVSuLK+9YGBHfMn1o=VA@mail.gmail.com>

On Mon, Feb 19, 2018 at 8:16 PM, Glenn Linderman <v+python at g.nevcal.com>
wrote:

> On 2/19/2018 7:02 PM, Guido van Rossum wrote:
>
> But how?
>
> On Mon, Feb 19, 2018 at 5:06 PM, Chris Barker - NOAA Federal <
> chris.barker at noaa.gov> wrote:
>
>> ... maybe it would be helpful to be able to
>> freeze an instance after creation for multiple use-cases?
>>
>
> And there's the crux of the issue... if the creator of Python can't figure
> out how to make mutable objects immutable by fiat after creation, then it
> is unlikely anyone else can!
>

Um, the question was meant in the Socratic way. :-)

(Snipped the rest.)

TBH, I don't hate Nick's solution that assigns to __class__ in
__post_init__. You can probably come up with a class decorator that hides
construction of the frozen subclass. I do think that it should be used very
sparingly, as the existence of the subclass is hard to hide: instances of
MyRecord will show as instances of _LockedMyRecord when printed or
otherwise inspected, which could be confusing.

There's another very simple low-tech solution:

@dataclass
class MyRecord:
    a: int
    # etc.
    frozen: bool = False
    def freeze(self):
        self.frozen = True
    def __setattr__(self, name, value):
        if self.frozen: raise AttributeError
        super().__setattr__(name, value)
    def __hash__(self):
        assert self.frozen
        return super().__hash__()

It can easily be weaponized via an extra decorator, and it's simpler in
that it doesn't use a magical second class. I'm just not keen on adding the
extra attribute to all frozen instances created via
@dataclass(frozen=True), because it's extra space overhead.

(In fact you can conclude that I'm not keen on exposing *any* of these
"solutions" as a flag on the @dataclass() decorator. They all feel pretty
fragile to me.)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180220/7562ea34/attachment.html>

From chris.barker at noaa.gov  Tue Feb 20 15:50:09 2018
From: chris.barker at noaa.gov (Chris Barker)
Date: Tue, 20 Feb 2018 12:50:09 -0800
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <CAP7+vJJ3Q=XdYB51A=tNjd4O=UDDo4iVSuLK+9YGBHfMn1o=VA@mail.gmail.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
 <CAD4jWF8-ihCodcEwX-WNMQ2Fwd7+QReXiv+iYmxsRMBMKEw=oA@mail.gmail.com>
 <CAP7+vJLNzhEbFD-6LsE6KdD8tpN_+w7p7GDNgi_Mk+pbwsF0kQ@mail.gmail.com>
 <CADiSq7c=f=jKHspyTEGE1s_MXEsJ3UaMJ5z=Q9Y4a89sD5PtGA@mail.gmail.com>
 <CAP7+vJKBZ+Z6e_ERUy+siq6USqk3fCa6+MVFx0CqbhQawmydBQ@mail.gmail.com>
 <CALGmxEJH-MERdNTc5XMP3TJUo=nAnwt_0RBNnx7eVHVjnxyGLw@mail.gmail.com>
 <CAP7+vJ+d_9HcE=KmNfVXZudTmDmtDSauTnU=RT+2b3FU1Hga3w@mail.gmail.com>
 <f26a13a0-d816-1e72-79d0-60439d4875da@g.nevcal.com>
 <CAP7+vJJ3Q=XdYB51A=tNjd4O=UDDo4iVSuLK+9YGBHfMn1o=VA@mail.gmail.com>
Message-ID: <CALGmxEJ8vpy3HFN89pO55paEx-49YfvTnkOMMT0wJ7QzP_8gQw@mail.gmail.com>

On Tue, Feb 20, 2018 at 8:45 AM, Guido van Rossum <guido at python.org> wrote:

> TBH, I don't hate Nick's solution that assigns to __class__ in
> __post_init__. You can probably come up with a class decorator that hides
> construction of the frozen subclass. I do think that it should be used very
> sparingly, as the existence of the subclass is hard to hide: instances of
> MyRecord will show as instances of _LockedMyRecord when printed or
> otherwise inspected, which could be confusing.
>
> There's another very simple low-tech solution:
>
> @dataclass
> class MyRecord:
>     a: int
>     # etc.
>     frozen: bool = False
>     def freeze(self):
>         self.frozen = True
>     def __setattr__(self, name, value):
>         if self.frozen: raise AttributeError
>         super().__setattr__(name, value)
>     def __hash__(self):
>         assert self.frozen
>         return super().__hash__()
>

This sure seems far more straightforward and les magic than __class__
swapping.


> It can easily be weaponized via an extra decorator, and it's simpler in
> that it doesn't use a magical second class. I'm just not keen on adding the
> extra attribute to all frozen instances created via
> @dataclass(frozen=True), because it's extra space overhead.
>

Is that one attribute that big a deal? I suppose that simple dataclasses
with only two or so attributes would see a significant change, but if
you're really worried about space, wouldn't you use a namedtuple or simply
a tuple anyway?

(In fact you can conclude that I'm not keen on exposing *any* of these
> "solutions" as a flag on the @dataclass() decorator. They all feel pretty
> fragile to me.)
>

so provide a:

@freezable_dataclass

Additional decorator that folks that want the ability to freeze and
unfreeze instances can use?

and if it overrides __set__attr, it could be used an virtually any class to
make a frozen version, yes?

I can't say I have enough need to bother writing that myself, but if others
do -- it seems like a good solution.

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180220/d6ed8e50/attachment.html>

From eric at trueblade.com  Tue Feb 20 16:37:33 2018
From: eric at trueblade.com (Eric V. Smith)
Date: Tue, 20 Feb 2018 16:37:33 -0500
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
Message-ID: <b2dabc79-afaa-d3e5-015e-a3ef9b5ce0bd@trueblade.com>

On 2/17/2018 2:35 PM, Guido van Rossum wrote:
> PS. I have to ponder why frozen dataclasses don't use `__new__`.

As I'm sure everyone is now aware after the rest of this discussion: 
it's because the returned object isn't really immutable.

That said, I have threatened to create a decorator similar to 
typing.NamedTuple that has the @dataclass features (except maybe 
__post_init__) and returns a class that does inherit from tuple, in 
which case it really would use `__new__`. I'll save that for 3.8, if it 
ever happens. There's a lot there to think about, first. For now, 
typing.NamedTuple is the way to go if you want something based on 
namedtuple yet using type hints.

Eric

From guido at python.org  Tue Feb 20 16:47:06 2018
From: guido at python.org (Guido van Rossum)
Date: Tue, 20 Feb 2018 13:47:06 -0800
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <CALGmxEJ8vpy3HFN89pO55paEx-49YfvTnkOMMT0wJ7QzP_8gQw@mail.gmail.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
 <CAD4jWF8-ihCodcEwX-WNMQ2Fwd7+QReXiv+iYmxsRMBMKEw=oA@mail.gmail.com>
 <CAP7+vJLNzhEbFD-6LsE6KdD8tpN_+w7p7GDNgi_Mk+pbwsF0kQ@mail.gmail.com>
 <CADiSq7c=f=jKHspyTEGE1s_MXEsJ3UaMJ5z=Q9Y4a89sD5PtGA@mail.gmail.com>
 <CAP7+vJKBZ+Z6e_ERUy+siq6USqk3fCa6+MVFx0CqbhQawmydBQ@mail.gmail.com>
 <CALGmxEJH-MERdNTc5XMP3TJUo=nAnwt_0RBNnx7eVHVjnxyGLw@mail.gmail.com>
 <CAP7+vJ+d_9HcE=KmNfVXZudTmDmtDSauTnU=RT+2b3FU1Hga3w@mail.gmail.com>
 <f26a13a0-d816-1e72-79d0-60439d4875da@g.nevcal.com>
 <CAP7+vJJ3Q=XdYB51A=tNjd4O=UDDo4iVSuLK+9YGBHfMn1o=VA@mail.gmail.com>
 <CALGmxEJ8vpy3HFN89pO55paEx-49YfvTnkOMMT0wJ7QzP_8gQw@mail.gmail.com>
Message-ID: <CAP7+vJKMiJeRpFS9gjH7aN_OryJxjeNuSLvoG7Nu1gOBHSHkWg@mail.gmail.com>

On Tue, Feb 20, 2018 at 12:50 PM, Chris Barker <chris.barker at noaa.gov>
wrote:

> Is that one attribute that big a deal? I suppose that simple dataclasses
> with only two or so attributes would see a significant change, but if
> you're really worried about space, wouldn't you use a namedtuple or simply
> a tuple anyway?
>

It's not just the space I worry about, it's about stomping on the user's
namespace for instance variables. So far dataclasses works by adding things
to the class only -- I want to place the bar pretty high before we change
that.

Anyway, there are several solutions now that people can implement as
separate class decorators and distribute via PyPI (or copy into their own
codebase) so I don't think now's the time to consider adding this to
dataclasses. Maybe for  Python 3.8.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180220/c61901f4/attachment.html>

From guido at python.org  Tue Feb 20 17:38:37 2018
From: guido at python.org (Guido van Rossum)
Date: Tue, 20 Feb 2018 14:38:37 -0800
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <b2dabc79-afaa-d3e5-015e-a3ef9b5ce0bd@trueblade.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
 <b2dabc79-afaa-d3e5-015e-a3ef9b5ce0bd@trueblade.com>
Message-ID: <CAP7+vJK-=ywPCbhFWZNFNW0Tkce+m2sX8gbRCPYH+1H41HS6ug@mail.gmail.com>

On Tue, Feb 20, 2018 at 1:37 PM, Eric V. Smith <eric at trueblade.com> wrote:

> On 2/17/2018 2:35 PM, Guido van Rossum wrote:
>
>> PS. I have to ponder why frozen dataclasses don't use `__new__`.
>>
>
> As I'm sure everyone is now aware after the rest of this discussion: it's
> because the returned object isn't really immutable.
>
> That said, I have threatened to create a decorator similar to
> typing.NamedTuple that has the @dataclass features (except maybe
> __post_init__) and returns a class that does inherit from tuple, in which
> case it really would use `__new__`. I'll save that for 3.8, if it ever
> happens. There's a lot there to think about, first. For now,
> typing.NamedTuple is the way to go if you want something based on
> namedtuple yet using type hints.
>

But then the class would also inherit a bunch of misfeatures from tuple
(like being indexable and having a length). It would be nicer if it used
__slots__ instead. (Also, the problem with __slots__ is the same as the
problem with inheriting from tuple, and it should just be solved right,
somehow.)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180220/f15d80ea/attachment.html>

From eric at trueblade.com  Tue Feb 20 17:41:15 2018
From: eric at trueblade.com (Eric V. Smith)
Date: Tue, 20 Feb 2018 17:41:15 -0500
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <CAP7+vJK-=ywPCbhFWZNFNW0Tkce+m2sX8gbRCPYH+1H41HS6ug@mail.gmail.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
 <b2dabc79-afaa-d3e5-015e-a3ef9b5ce0bd@trueblade.com>
 <CAP7+vJK-=ywPCbhFWZNFNW0Tkce+m2sX8gbRCPYH+1H41HS6ug@mail.gmail.com>
Message-ID: <350F65E0-A3C4-40E9-8A7B-5C13E2690ED0@trueblade.com>


> On Feb 20, 2018, at 5:38 PM, Guido van Rossum <guido at python.org> wrote:
> 
>> On Tue, Feb 20, 2018 at 1:37 PM, Eric V. Smith <eric at trueblade.com> wrote:
>>> On 2/17/2018 2:35 PM, Guido van Rossum wrote:
>>> PS. I have to ponder why frozen dataclasses don't use `__new__`.
>> 
>> As I'm sure everyone is now aware after the rest of this discussion: it's because the returned object isn't really immutable.
>> 
>> That said, I have threatened to create a decorator similar to typing.NamedTuple that has the @dataclass features (except maybe __post_init__) and returns a class that does inherit from tuple, in which case it really would use `__new__`. I'll save that for 3.8, if it ever happens. There's a lot there to think about, first. For now, typing.NamedTuple is the way to go if you want something based on namedtuple yet using type hints.
> 
> But then the class would also inherit a bunch of misfeatures from tuple (like being indexable and having a length).

Right. That?s the ?lots to think about? part. It might just be unworkable. 

> It would be nicer if it used __slots__ instead. (Also, the problem with __slots__ is the same as the problem with inheriting from tuple, and it should just be solved right, somehow.)

Agreed. 

Eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180220/9d832240/attachment.html>

From raymond.hettinger at gmail.com  Tue Feb 20 18:24:02 2018
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Tue, 20 Feb 2018 15:24:02 -0800
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <CAP7+vJK-=ywPCbhFWZNFNW0Tkce+m2sX8gbRCPYH+1H41HS6ug@mail.gmail.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
 <b2dabc79-afaa-d3e5-015e-a3ef9b5ce0bd@trueblade.com>
 <CAP7+vJK-=ywPCbhFWZNFNW0Tkce+m2sX8gbRCPYH+1H41HS6ug@mail.gmail.com>
Message-ID: <F1050697-E94A-4E99-8646-DC98F850009D@gmail.com>



> On Feb 20, 2018, at 2:38 PM, Guido van Rossum <guido at python.org> wrote:
> 
> But then the class would also inherit a bunch of misfeatures from tuple (like being indexable and having a length). It would be nicer if it used __slots__ instead.


FWIW, George Sakkis made a tool like this about nine years ago.  https://code.activestate.com/recipes/576555-records  It would need to be modernized to include default arguments, types annotations and whatnot, but otherwise it has great performance and low API complexity.

> (Also, the problem with __slots__ is the same as the problem with inheriting from tuple, and it should just be solved right, somehow.)

Perhaps a new variant of __init_subclass__ would work.



Raymond



From greg.ewing at canterbury.ac.nz  Tue Feb 20 19:13:36 2018
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 21 Feb 2018 13:13:36 +1300
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <20180220051146.GH10142@ando.pearwood.info>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
 <CAD4jWF8-ihCodcEwX-WNMQ2Fwd7+QReXiv+iYmxsRMBMKEw=oA@mail.gmail.com>
 <CAP7+vJLNzhEbFD-6LsE6KdD8tpN_+w7p7GDNgi_Mk+pbwsF0kQ@mail.gmail.com>
 <CADiSq7c=f=jKHspyTEGE1s_MXEsJ3UaMJ5z=Q9Y4a89sD5PtGA@mail.gmail.com>
 <CAP7+vJKBZ+Z6e_ERUy+siq6USqk3fCa6+MVFx0CqbhQawmydBQ@mail.gmail.com>
 <CALGmxEJH-MERdNTc5XMP3TJUo=nAnwt_0RBNnx7eVHVjnxyGLw@mail.gmail.com>
 <CAP7+vJ+d_9HcE=KmNfVXZudTmDmtDSauTnU=RT+2b3FU1Hga3w@mail.gmail.com>
 <20180220051146.GH10142@ando.pearwood.info>
Message-ID: <5A8CB9B0.50905@canterbury.ac.nz>

Steven D'Aprano wrote:
> So in principle, we could have a mutable class, and a immutable one, and 
> when you flick the switch, the instance.__class__ changes from mutable 
> to frozen.

That seems unfriendly to subclasses as well. To extend a class
you now need to subclass both the mutable and immutable versions
of the base class, ensure they share the behaviour they should
have in common (possibly by using a third mixin class) and
arrange for the freezing switch to install the immutable subclass
instead of the base one.

All of which seems like a huge amount of hassle just to save
an instance attribute. And I don't even want to think what
multiple inheritance would do to all this.

-- 
Greg

From zhaoya881010 at 163.com  Tue Feb 20 21:28:55 2018
From: zhaoya881010 at 163.com (=?GBK?B?1dTRxw==?=)
Date: Wed, 21 Feb 2018 10:28:55 +0800 (CST)
Subject: [Python-Dev] pycapsule:PyObject * is NULL pointer
Message-ID: <71ad933d.852.161b6313c11.Coremail.zhaoya881010@163.com>

i have question:call,c-->python-->c. 
1.the c pointer void* abc="123" by pycapsule in the c code.
......
void* lpContext = "abc";
PyObject * lpPyContext = PyCapsule_New(lpContext, "Context",NULL);
......
PyTuple_SetItem(pArgs, 1, lpPyContext);
PyObject* pyResult = PyObject_CallObject(pFunc, pArgs);

2.c call python:

in the python code:
import ctypes
pycapsule = ctypes.windll.LoadLibrary("C:\Users\zhaoya16975\Documents\Visual Studio 2017\Projects\cpython\Release\pycapsule.dll")
def test( lpContext,lpRequest,lpAnswer):
    print lpContext
    pycapsule.hello()
    pycapsule.GetContext(lpContext)
.........
the lpContest is "<capsule object "Context" at 0x02747e00>"
but,i can't lpContext in the pycapsule.GetContest:
the capsule is null poniter,the GetContext no execute!!

void* FUNCTION_CALL_MODE GetContext(PyObject  *capsule) {
		printf(" GetContext......\n");
		//if (!PyCapsule_IsValid((PyObject *)capsule, "Context")) {
		//	printf(" the Capsule Context is no Valid\n");
		//	return NULL;
		//}
		//return PyCapsule_GetPointer((PyObject *)capsule, "Context");
		return NULL;
		
	}


FUNCTION_CALL_MODE is __stdcall i think no problem.because, if GetContext(PyObject  *capsule) -->GetContext() is ok. PyObject  *capsule from python to c unknown error occurred.
i hope c call python pass void* ,and python call c pass void* ,but the capsule call no success?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/68e45c95/attachment-0001.html>

From brett at python.org  Tue Feb 20 20:58:15 2018
From: brett at python.org (Brett Cannon)
Date: Wed, 21 Feb 2018 01:58:15 +0000
Subject: [Python-Dev] How is the GitHub workflow working for people?
Message-ID: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>

It's been a year and 10 days since we moved to GitHub, so I figured now is
as good a time as any to ask people if they are generally happy with the
workflow and if there is a particular sticking point to please bring it up
on the core-workflow mailing list so we can potentially address it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/0449c6fa/attachment.html>

From greg.ewing at canterbury.ac.nz  Tue Feb 20 18:49:09 2018
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 21 Feb 2018 12:49:09 +1300
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
In-Reply-To: <CALGmxEJH-MERdNTc5XMP3TJUo=nAnwt_0RBNnx7eVHVjnxyGLw@mail.gmail.com>
References: <CAD4jWF8B_9KvQ7OiNkg5L7A+MLtLVrRNo6=7v-9P1vZyEm_zYQ@mail.gmail.com>
 <CAP7+vJJNnY=ZAH+ggvbY5cROsV6bt-ktVtYxv5PoEAvGXO9=iQ@mail.gmail.com>
 <CAD4jWF_NaPH+P5D+SKoTNOG9iLqkWyrNSe08LkaEAw8K+YWu=Q@mail.gmail.com>
 <CAP7+vJ+2zt0DzzJUuSKGTSoiyUZ7hUZhVWd2bqX7Mdx=6U_9-w@mail.gmail.com>
 <CAD4jWF8-ihCodcEwX-WNMQ2Fwd7+QReXiv+iYmxsRMBMKEw=oA@mail.gmail.com>
 <CAP7+vJLNzhEbFD-6LsE6KdD8tpN_+w7p7GDNgi_Mk+pbwsF0kQ@mail.gmail.com>
 <CADiSq7c=f=jKHspyTEGE1s_MXEsJ3UaMJ5z=Q9Y4a89sD5PtGA@mail.gmail.com>
 <CAP7+vJKBZ+Z6e_ERUy+siq6USqk3fCa6+MVFx0CqbhQawmydBQ@mail.gmail.com>
 <CALGmxEJH-MERdNTc5XMP3TJUo=nAnwt_0RBNnx7eVHVjnxyGLw@mail.gmail.com>
Message-ID: <5A8CB3F5.4000701@canterbury.ac.nz>

Chris Barker - NOAA Federal wrote:
> If I have this right, on the discussion about frozen and hash, a use
> case was brought up for taking a few steps to create an instance (and
> thus wanting it not frozen) and then wanting it hashable.
> 
> Which pointed to the idea of a ? freeze this from now on? method.

The problem with a "freeze" method is that it doesn't play well
with subclasses. The object ends up frozen by the base class's
__new__ or __init__, preventing the subclass from doing its own
initialisation unless it uses some form of back-door access.
And if there is a back door, it might as well be used for all
initialisation, maing the freeze method unnecessary.

-- 
Greg

From python at mrabarnett.plus.com  Tue Feb 20 22:18:18 2018
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 21 Feb 2018 03:18:18 +0000
Subject: [Python-Dev] pycapsule:PyObject * is NULL pointer
In-Reply-To: <71ad933d.852.161b6313c11.Coremail.zhaoya881010@163.com>
References: <71ad933d.852.161b6313c11.Coremail.zhaoya881010@163.com>
Message-ID: <7ac2b1bb-cbdc-bffd-d12c-a0b1de655c86@mrabarnett.plus.com>

On 2018-02-21 02:28, ?? wrote:

This "python-dev" list is for development of the Python language itself. 
I think that this message should instead have been sent to "python-list".

> i have question:call,c-->python-->c.
> 1.the c pointer void* abc="123" by pycapsule in the c code.
> ......
> void* lpContext = "abc";
> PyObject * lpPyContext = PyCapsule_New(lpContext, "Context",NULL);
> ......

Is this C code calling the function 'test' in the Python code?

If yes, are you sure that lpPyContext should be in pArgs at index 1 and 
not at index 0?

The function 'test' expects argument 0 => lpContext, argument 1 => 
lpRequest, argument 2 => lpAnswer.

> PyTuple_SetItem(pArgs, 1, lpPyContext);
> PyObject* pyResult = PyObject_CallObject(pFunc, pArgs);
> 
> 2.c call python:
> 
> in the python code:
> import ctypes
> pycapsule = ctypes.windll.LoadLibrary("C:\Users\zhaoya16975\Documents\Visual Studio 2017\Projects\cpython\Release\pycapsule.dll")
> def test( lpContext,lpRequest,lpAnswer):
>      print lpContext
>      pycapsule.hello()
>      pycapsule.GetContext(lpContext)
> .........
> the lpContest is "<capsule object "Context" at 0x02747e00>"
> but,i can't lpContext in the pycapsule.GetContest:
> the capsule is null poniter,the GetContext no execute!!
> 
> void* FUNCTION_CALL_MODE GetContext(PyObject  *capsule) {
> 		printf(" GetContext......\n");
> 		//if (!PyCapsule_IsValid((PyObject *)capsule, "Context")) {
> 		//	printf(" the Capsule Context is no Valid\n");
> 		//	return NULL;
> 		//}
> 		//return PyCapsule_GetPointer((PyObject *)capsule, "Context");
> 		return NULL;
> 		
> 	}
> 
> FUNCTION_CALL_MODE is __stdcall i think no problem.because, if GetContext(PyObject  *capsule) -->GetContext() is ok. PyObject  *capsule from python to c unknown error occurred.
> 
> i hope c call python  pass void* ,and python call c pass void* ,but the capsule call no success?
> 

From zhaoya881010 at 163.com  Tue Feb 20 22:27:54 2018
From: zhaoya881010 at 163.com (=?GBK?B?1dTRxw==?=)
Date: Wed, 21 Feb 2018 11:27:54 +0800 (CST)
Subject: [Python-Dev] =?gbk?q?void*_from_c_to_python_and_to_c=A3=BF?=
Message-ID: <24e5c00e.b22.161b6673b0e.Coremail.zhaoya881010@163.com>

I have a demand:
c call python ,have a void* pointer  pass to python and python call c,the void* pointer need pass to c.
i don't know how to do?have any good idea ? thanks?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/11c84897/attachment.html>

From zhaoya881010 at 163.com  Tue Feb 20 23:34:23 2018
From: zhaoya881010 at 163.com (=?GBK?B?1dTRxw==?=)
Date: Wed, 21 Feb 2018 12:34:23 +0800 (CST)
Subject: [Python-Dev] pycapsule:PyObject * is NULL pointer
In-Reply-To: <7ac2b1bb-cbdc-bffd-d12c-a0b1de655c86@mrabarnett.plus.com>
References: <71ad933d.852.161b6313c11.Coremail.zhaoya881010@163.com>
 <7ac2b1bb-cbdc-bffd-d12c-a0b1de655c86@mrabarnett.plus.com>
Message-ID: <59a88db3.d0e.161b6a41a1d.Coremail.zhaoya881010@163.com>



hi @MRAB
i try again?it not ok. test files:
1.cpython.cpp ---c call python maintenance.py
2.maintenance.py  python maintenance.py call pycapsule.cpp(dll)
the code file,see attachment.








At 2018-02-21 11:18:18, "MRAB" <python at mrabarnett.plus.com> wrote:
>On 2018-02-21 02:28, ?? wrote:
>
>This "python-dev" list is for development of the Python language itself. 
>I think that this message should instead have been sent to "python-list".
>
>> i have question:call,c-->python-->c.
>> 1.the c pointer void* abc="123" by pycapsule in the c code.
>> ......
>> void* lpContext = "abc";
>> PyObject * lpPyContext = PyCapsule_New(lpContext, "Context",NULL);
>> ......
>
>Is this C code calling the function 'test' in the Python code?
>
>If yes, are you sure that lpPyContext should be in pArgs at index 1 and 
>not at index 0?
>
>The function 'test' expects argument 0 => lpContext, argument 1 => 
>lpRequest, argument 2 => lpAnswer.
>
>> PyTuple_SetItem(pArgs, 1, lpPyContext);
>> PyObject* pyResult = PyObject_CallObject(pFunc, pArgs);
>> 
>> 2.c call python:
>> 
>> in the python code:
>> import ctypes
>> pycapsule = ctypes.windll.LoadLibrary("C:\Users\zhaoya16975\Documents\Visual Studio 2017\Projects\cpython\Release\pycapsule.dll")
>> def test( lpContext,lpRequest,lpAnswer):
>>      print lpContext
>>      pycapsule.hello()
>>      pycapsule.GetContext(lpContext)
>> .........
>> the lpContest is "<capsule object "Context" at 0x02747e00>"
>> but,i can't lpContext in the pycapsule.GetContest:
>> the capsule is null poniter,the GetContext no execute!!
>> 
>> void* FUNCTION_CALL_MODE GetContext(PyObject  *capsule) {
>> 		printf(" GetContext......\n");
>> 		//if (!PyCapsule_IsValid((PyObject *)capsule, "Context")) {
>> 		//	printf(" the Capsule Context is no Valid\n");
>> 		//	return NULL;
>> 		//}
>> 		//return PyCapsule_GetPointer((PyObject *)capsule, "Context");
>> 		return NULL;
>> 		
>> 	}
>> 
>> FUNCTION_CALL_MODE is __stdcall i think no problem.because, if GetContext(PyObject  *capsule) -->GetContext() is ok. PyObject  *capsule from python to c unknown error occurred.
>> 
>> i hope c call python  pass void* ,and python call c pass void* ,but the capsule call no success?
>> 
>_______________________________________________
>Python-Dev mailing list
>Python-Dev at python.org
>https://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe: https://mail.python.org/mailman/options/python-dev/zhaoya881010%40163.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/097f7fd5/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: maintenance.py
Type: application/octet-stream
Size: 512 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/097f7fd5/attachment.obj>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: cpython.cpp
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/097f7fd5/attachment.ksh>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cpython.h
Type: application/octet-stream
Size: 208 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/097f7fd5/attachment-0001.obj>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pycapsule.cpp
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/097f7fd5/attachment-0001.ksh>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pycapsule.def
Type: application/octet-stream
Size: 160 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/097f7fd5/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pycapsule.h
Type: application/octet-stream
Size: 566 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/097f7fd5/attachment-0003.obj>

From ncoghlan at gmail.com  Wed Feb 21 01:42:12 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 21 Feb 2018 16:42:12 +1000
Subject: [Python-Dev] Making "Provisional" an explicit PEP state
Message-ID: <CADiSq7dNmt_KcgMi3yvv9wxL58zEd2NY8+csUMn7g1KzYBYhyw@mail.gmail.com>

Hi folks,

In the current PEP workflow, provisionally accepted PEPs are marked as
"Accepted", and remain in that state until they're declared stable and
moved to Final.

This makes it hard to identify which APIs and file formats have been
approved to have their reference implementations released (whether in
CPython or in other tools), so I filed
https://github.com/python/peps/issues/270 some time ago to propose
making Provisional an explicit state in its own right, distinct from
Accepted.

https://github.com/python/peps/pull/577 is a PR with a concrete
proposal along those lines:

* "Provisional" is a new state that parallels Accepted, with the same
basic state flows
* PEP 0 gains a new section, listing the provisionally accepted PEPs
separately from the accepted ones
* PEP 484 (type hinting), and the PyPA static build metadata
(pyproject.toml) PEPs are marked as provisionally accepted

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From tjreedy at udel.edu  Wed Feb 21 05:12:20 2018
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 21 Feb 2018 05:12:20 -0500
Subject: [Python-Dev] Update tcl/tk to 8.6.latest on Windows?
Message-ID: <p6jgi8$fjh$1@blaine.gmane.org>

3.7.0b1 Windows installer installs tcl/tk 8.6.6, which is a couple of 
years old.  Same for Windows repository builds.  I believe the MacOS 
installer shipped with something later.  Can we update Windows to the 
current 8.6.8?

-- 
Terry Jan Reedy


From storchaka at gmail.com  Wed Feb 21 05:22:05 2018
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Wed, 21 Feb 2018 12:22:05 +0200
Subject: [Python-Dev] Update tcl/tk to 8.6.latest on Windows?
In-Reply-To: <p6jgi8$fjh$1@blaine.gmane.org>
References: <p6jgi8$fjh$1@blaine.gmane.org>
Message-ID: <p6jh4g$pmt$1@blaine.gmane.org>

21.02.18 12:12, Terry Reedy ????:
> 3.7.0b1 Windows installer installs tcl/tk 8.6.6, which is a couple of 
> years old.? Same for Windows repository builds.? I believe the MacOS 
> installer shipped with something later.? Can we update Windows to the 
> current 8.6.8?

Yes, please open an issue for this. See also 
https://bugs.python.org/issue27647 and https://bugs.python.org/issue15663.


From yselivanov.ml at gmail.com  Wed Feb 21 12:30:22 2018
From: yselivanov.ml at gmail.com (Yury Selivanov)
Date: Wed, 21 Feb 2018 12:30:22 -0500
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
Message-ID: <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>

FWIW I'm extremely happy with the current workflow. The recent
improvements to @miss-islington (kudos to Mariatta!) allowing her to
auto-backport PRs and commit them is a big time saver.

I can only suggest a couple improvements:

1. Make our bots check the code style?fully enforce PEP 8, lint the
code, and detect trailing whitespace on all lines that a PR modifies.

2. AppVeyor and Travis are a bit slow at times.  Maybe it's possible
to ask them to slightly increase our quotas (again).  Although usually
this isn't a problem and CI is fast enough.

3. It would be great if our buildbots could update the PR at blame
when they detect a regression (I understand that this is a hard to
implement feature...)

Huge thanks to the core-workflow team!

Yury

On Tue, Feb 20, 2018 at 8:58 PM, Brett Cannon <brett at python.org> wrote:
> It's been a year and 10 days since we moved to GitHub, so I figured now is
> as good a time as any to ask people if they are generally happy with the
> workflow and if there is a particular sticking point to please bring it up
> on the core-workflow mailing list so we can potentially address it.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com
>

From brett at python.org  Wed Feb 21 12:53:10 2018
From: brett at python.org (Brett Cannon)
Date: Wed, 21 Feb 2018 17:53:10 +0000
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
Message-ID: <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>

On Wed, 21 Feb 2018 at 09:30 Yury Selivanov <yselivanov.ml at gmail.com> wrote:

> FWIW I'm extremely happy with the current workflow. The recent
> improvements to @miss-islington (kudos to Mariatta!) allowing her to
> auto-backport PRs and commit them is a big time saver.
>
> I can only suggest a couple improvements:
>
> 1. Make our bots check the code style?fully enforce PEP 8, lint the
> code, and detect trailing whitespace on all lines that a PR modifies.
>

Guido said "no" to this from the outset.


>
> 2. AppVeyor and Travis are a bit slow at times.  Maybe it's possible
> to ask them to slightly increase our quotas (again).  Although usually
> this isn't a problem and CI is fast enough.
>

We already have a huge amount of quota from Travis and AppVeyor is no
special quota (we're just like any other open source project). Our contact
at Circle CI left the company so that option has walked away.

IOW we either have to find another CI provider who happens to be faster or
we are going to have to build our own.


>
> 3. It would be great if our buildbots could update the PR at blame
> when they detect a regression (I understand that this is a hard to
> implement feature...)
>

Possibly. This is starting to dive into Bors/Zuul-level regression
management which is more infrastructure to run, so we would have to decide
we seriously want this maintenance overhead.

-Brett


>
> Huge thanks to the core-workflow team!
>
> Yury
>
> On Tue, Feb 20, 2018 at 8:58 PM, Brett Cannon <brett at python.org> wrote:
> > It's been a year and 10 days since we moved to GitHub, so I figured now
> is
> > as good a time as any to ask people if they are generally happy with the
> > workflow and if there is a particular sticking point to please bring it
> up
> > on the core-workflow mailing list so we can potentially address it.
> >
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> >
> https://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/d0f0bbcb/attachment.html>

From steve at holdenweb.com  Wed Feb 21 14:10:30 2018
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 21 Feb 2018 19:10:30 +0000
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
Message-ID: <CAMofdRC-vHX5etB5qRsUy93M=ifyo0kXXC4VNjavQda-P7+y-g@mail.gmail.com>

On Wed, Feb 21, 2018 at 5:53 PM, Brett Cannon <brett at python.org> wrote:

>
>
> On Wed, 21 Feb 2018 at 09:30 Yury Selivanov <yselivanov.ml at gmail.com>
> wrote:
>
>>
>
>
>> Huge thanks to the core-workflow team!
>>
>>
?Hear, hear!

regards
 Steve?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/6ecd1e50/attachment.html>

From solipsis at pitrou.net  Wed Feb 21 14:36:18 2018
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 21 Feb 2018 20:36:18 +0100
Subject: [Python-Dev] How is the GitHub workflow working for people?
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
Message-ID: <20180221203618.4261c906@fsol>

On Wed, 21 Feb 2018 01:58:15 +0000
Brett Cannon <brett at python.org> wrote:
> It's been a year and 10 days since we moved to GitHub, so I figured now is
> as good a time as any to ask people if they are generally happy with the
> workflow and if there is a particular sticking point to please bring it up
> on the core-workflow mailing list so we can potentially address it.

It works great for me.

Regards

Antoine.



From mikez302 at gmail.com  Wed Feb 21 14:55:39 2018
From: mikez302 at gmail.com (Elias Zamaria)
Date: Wed, 21 Feb 2018 11:55:39 -0800
Subject: [Python-Dev] PEP 467 (Minor API improvements for binary sequences)
 - any thoughts?
Message-ID: <CAFdOKfv+WMn6B0o4UtyPf1ziASUihELmQ_3K_WUNNfDsUva9+w@mail.gmail.com>

This is about some minor changes to the bytes, bytearray, and memoryview
classes. Here is the PEP: https://www.python.org/dev/peps/pep-0467/

The page in the bug tracker can be seen at
https://bugs.python.org/issue27923 and the pull request can be seen at
https://github.com/python/cpython/pull/3237.

I am waiting for this to be merged, or approved, or whatever is the next
step. Someone on the bug tracker mentioned restarting the discussion on the
mailing list, so that is what I'm trying to do here. Does anyone have any
thoughts?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/aa7228e8/attachment.html>

From ethan at stoneleaf.us  Wed Feb 21 15:21:46 2018
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 21 Feb 2018 12:21:46 -0800
Subject: [Python-Dev] PEP 467 (Minor API improvements for binary
 sequences) - any thoughts?
In-Reply-To: <CAFdOKfv+WMn6B0o4UtyPf1ziASUihELmQ_3K_WUNNfDsUva9+w@mail.gmail.com>
References: <CAFdOKfv+WMn6B0o4UtyPf1ziASUihELmQ_3K_WUNNfDsUva9+w@mail.gmail.com>
Message-ID: <5A8DD4DA.10306@stoneleaf.us>

On 02/21/2018 11:55 AM, Elias Zamaria wrote:

> This is about some minor changes to the bytes, bytearray, and memoryview classes. Here is the PEP:
> https://www.python.org/dev/peps/pep-0467/
>
> The page in the bug tracker can be seen at https://bugs.python.org/issue27923 and the pull request can be seen at
> https://github.com/python/cpython/pull/3237.
>
> I am waiting for this to be merged, or approved, or whatever is the next step. Someone on the bug tracker mentioned
> restarting the discussion on the mailing list, so that is what I'm trying to do here. Does anyone have any thoughts?

At this point the PEP itself has not been approved, and is undergoing changes.  I don't see anything happening with it 
right now while 3.7 is going through it's final stages to release.  Once 3.7.0 is published we can come back to this.

--
~Ethan~


From chris.barker at noaa.gov  Wed Feb 21 15:26:32 2018
From: chris.barker at noaa.gov (Chris Barker)
Date: Wed, 21 Feb 2018 12:26:32 -0800
Subject: [Python-Dev] PEP 467 (Minor API improvements for binary
 sequences) - any thoughts?
In-Reply-To: <CAFdOKfv+WMn6B0o4UtyPf1ziASUihELmQ_3K_WUNNfDsUva9+w@mail.gmail.com>
References: <CAFdOKfv+WMn6B0o4UtyPf1ziASUihELmQ_3K_WUNNfDsUva9+w@mail.gmail.com>
Message-ID: <CALGmxEKjG-=+dVY8sNPdszq84hOxUnG0Cp5MH6edJEz37v0_9g@mail.gmail.com>

On Wed, Feb 21, 2018 at 11:55 AM, Elias Zamaria <mikez302 at gmail.com> wrote:

> This is about some minor changes to the bytes, bytearray, and memoryview
> classes. Here is the PEP: https://www.python.org/dev/peps/pep-0467/
>


> I am waiting for this to be merged, or approved, or whatever is the next
> step. Someone on the bug tracker mentioned restarting the discussion on the
> mailing list, so that is what I'm trying to do here. Does anyone have any
> thoughts?
>

+1 all around.

One other thought:

: Addition of optimised iterator methods that produce bytes objects

Maybe it would make sense to have a "byte" type that holds a single byte.
It would be an integer that could only hold values from 0-255. Then the
regular iterator could simply return a bunch of single byte objects.

I can't say I've thought it through, but if a byte is a int with restricted
range, then it could act like an int in (almost?) every context, so there
would be no need for a separate iterator.

I also haven't thought through whether there is any real advantage to
having such a type -- but off the top of my head, making a distinction
between a bytes object that happens to be length-one and a single byte
could be handy. I sure do often wish for a character object.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/ddafc19a/attachment.html>

From steve at holdenweb.com  Wed Feb 21 15:39:17 2018
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 21 Feb 2018 20:39:17 +0000
Subject: [Python-Dev] PEP 467 (Minor API improvements for binary
 sequences) - any thoughts?
In-Reply-To: <CALGmxEKjG-=+dVY8sNPdszq84hOxUnG0Cp5MH6edJEz37v0_9g@mail.gmail.com>
References: <CAFdOKfv+WMn6B0o4UtyPf1ziASUihELmQ_3K_WUNNfDsUva9+w@mail.gmail.com>
 <CALGmxEKjG-=+dVY8sNPdszq84hOxUnG0Cp5MH6edJEz37v0_9g@mail.gmail.com>
Message-ID: <CAMofdRAWj4GuabDyD1z2hwy2_u01S0+Aoyu8DJ037n0LzDwSaw@mail.gmail.com>

I think the chances of a "byte" object are about as good as the chances of
a character object (though one can always implement such in C extensions,
that wouldn't build them into the syntax). The fact that characters are
single-byte strings is responsible for certain anomalies with (e.g.) the
__contains__ operator (list elements aren't lists, but string element are
strings), but overall the choices made lead to sensible, comprehensible
code.

regards
 Steve

Steve Holden

On Wed, Feb 21, 2018 at 8:26 PM, Chris Barker <chris.barker at noaa.gov> wrote:

> On Wed, Feb 21, 2018 at 11:55 AM, Elias Zamaria <mikez302 at gmail.com>
> wrote:
>
>> This is about some minor changes to the bytes, bytearray, and memoryview
>> classes. Here is the PEP: https://www.python.org/dev/peps/pep-0467/
>>
>
>
>> I am waiting for this to be merged, or approved, or whatever is the next
>> step. Someone on the bug tracker mentioned restarting the discussion on the
>> mailing list, so that is what I'm trying to do here. Does anyone have any
>> thoughts?
>>
>
> +1 all around.
>
> One other thought:
>
> : Addition of optimised iterator methods that produce bytes objects
>
> Maybe it would make sense to have a "byte" type that holds a single byte.
> It would be an integer that could only hold values from 0-255. Then the
> regular iterator could simply return a bunch of single byte objects.
>
> I can't say I've thought it through, but if a byte is a int with
> restricted range, then it could act like an int in (almost?) every context,
> so there would be no need for a separate iterator.
>
> I also haven't thought through whether there is any real advantage to
> having such a type -- but off the top of my head, making a distinction
> between a bytes object that happens to be length-one and a single byte
> could be handy. I sure do often wish for a character object.
>
> -CHB
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> steve%40holdenweb.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/e2bbb011/attachment-0001.html>

From guido at python.org  Wed Feb 21 16:22:44 2018
From: guido at python.org (Guido van Rossum)
Date: Wed, 21 Feb 2018 13:22:44 -0800
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
Message-ID: <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>

On Wed, Feb 21, 2018 at 9:53 AM, Brett Cannon <brett at python.org> wrote:

>
>
> On Wed, 21 Feb 2018 at 09:30 Yury Selivanov <yselivanov.ml at gmail.com>
> wrote:
>
>> FWIW I'm extremely happy with the current workflow. The recent
>> improvements to @miss-islington (kudos to Mariatta!) allowing her to
>> auto-backport PRs and commit them is a big time saver.
>>
>> I can only suggest a couple improvements:
>>
>> 1. Make our bots check the code style?fully enforce PEP 8, lint the
>> code, and detect trailing whitespace on all lines that a PR modifies.
>>
>
> Guido said "no" to this from the outset.
>

I'm willing to reconsider if there's a good enough tool. Ditto for C code
(or do we already do it for C?).

FWIW I'm personally hugely happy with the new workflow -- my only regret is
that we're not using GitHub for issue tracking yet.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/44a97708/attachment.html>

From tjreedy at udel.edu  Wed Feb 21 16:28:47 2018
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 21 Feb 2018 16:28:47 -0500
Subject: [Python-Dev] Update tcl/tk to 8.6.latest on Windows?
In-Reply-To: <p6jh4g$pmt$1@blaine.gmane.org>
References: <p6jgi8$fjh$1@blaine.gmane.org> <p6jh4g$pmt$1@blaine.gmane.org>
Message-ID: <p6ko6h$qo2$1@blaine.gmane.org>

On 2/21/2018 5:22 AM, Serhiy Storchaka wrote:
> 21.02.18 12:12, Terry Reedy ????:
>> 3.7.0b1 Windows installer installs tcl/tk 8.6.6, which is a couple of 
>> years old.? Same for Windows repository builds.? I believe the MacOS 
>> installer shipped with something later.? Can we update Windows to the 
>> current 8.6.8?
> 
> Yes, please open an issue for this. See also 
> https://bugs.python.org/issue27647 and https://bugs.python.org/issue15663.

https://bugs.python.org/issue32901

-- 
Terry Jan Reedy



From jokerjokerer at gmail.com  Wed Feb 21 16:28:08 2018
From: jokerjokerer at gmail.com (=?UTF-8?B?QmVybsOhdCBHw6Fib3I=?=)
Date: Wed, 21 Feb 2018 21:28:08 +0000
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
Message-ID: <CANy2qHcGn8WD1P=cUqiR=QC_A0SwWDCyBSws2z1a3gozq0j9og@mail.gmail.com>

Any reason in particular for not using github issues (or blockers in
achieving this)?

On 21 Feb 2018 21:25, "Guido van Rossum" <guido at python.org> wrote:

> On Wed, Feb 21, 2018 at 9:53 AM, Brett Cannon <brett at python.org> wrote:
>
>>
>>
>> On Wed, 21 Feb 2018 at 09:30 Yury Selivanov <yselivanov.ml at gmail.com>
>> wrote:
>>
>>> FWIW I'm extremely happy with the current workflow. The recent
>>> improvements to @miss-islington (kudos to Mariatta!) allowing her to
>>> auto-backport PRs and commit them is a big time saver.
>>>
>>> I can only suggest a couple improvements:
>>>
>>> 1. Make our bots check the code style?fully enforce PEP 8, lint the
>>> code, and detect trailing whitespace on all lines that a PR modifies.
>>>
>>
>> Guido said "no" to this from the outset.
>>
>
> I'm willing to reconsider if there's a good enough tool. Ditto for C code
> (or do we already do it for C?).
>
> FWIW I'm personally hugely happy with the new workflow -- my only regret
> is that we're not using GitHub for issue tracking yet.
>
> --
> --Guido van Rossum (python.org/~guido)
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> jokerjokerer%40gmail.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/37cd7b51/attachment.html>

From levkivskyi at gmail.com  Wed Feb 21 16:30:04 2018
From: levkivskyi at gmail.com (Ivan Levkivskyi)
Date: Wed, 21 Feb 2018 21:30:04 +0000
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
Message-ID: <CAOMjWk=quHnvPs1LmqFg-NDxEZ=SXWRDeXEKC=L-7PmQdjpMSg@mail.gmail.com>

> I'm personally hugely happy with the new workflow -- my only regret is
that we're not using GitHub for issue tracking yet.

I have the same feelings. I like the GitHub issue tracker, and it would be
great to migrate there (if possible).

--
Ivan



On 21 February 2018 at 21:22, Guido van Rossum <guido at python.org> wrote:

> On Wed, Feb 21, 2018 at 9:53 AM, Brett Cannon <brett at python.org> wrote:
>
>>
>>
>> On Wed, 21 Feb 2018 at 09:30 Yury Selivanov <yselivanov.ml at gmail.com>
>> wrote:
>>
>>> FWIW I'm extremely happy with the current workflow. The recent
>>> improvements to @miss-islington (kudos to Mariatta!) allowing her to
>>> auto-backport PRs and commit them is a big time saver.
>>>
>>> I can only suggest a couple improvements:
>>>
>>> 1. Make our bots check the code style?fully enforce PEP 8, lint the
>>> code, and detect trailing whitespace on all lines that a PR modifies.
>>>
>>
>> Guido said "no" to this from the outset.
>>
>
> I'm willing to reconsider if there's a good enough tool. Ditto for C code
> (or do we already do it for C?).
>
> FWIW I'm personally hugely happy with the new workflow -- my only regret
> is that we're not using GitHub for issue tracking yet.
>
> --
> --Guido van Rossum (python.org/~guido)
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> levkivskyi%40gmail.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/6dece241/attachment.html>

From guido at python.org  Wed Feb 21 16:38:39 2018
From: guido at python.org (Guido van Rossum)
Date: Wed, 21 Feb 2018 13:38:39 -0800
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CANy2qHcGn8WD1P=cUqiR=QC_A0SwWDCyBSws2z1a3gozq0j9og@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <CANy2qHcGn8WD1P=cUqiR=QC_A0SwWDCyBSws2z1a3gozq0j9og@mail.gmail.com>
Message-ID: <CAP7+vJ+1Wv-HtrwXc2=XeGBWcTaVJkLZwtW2LdG5+qqDq2=pHQ@mail.gmail.com>

On Wed, Feb 21, 2018 at 1:28 PM, Bern?t G?bor <jokerjokerer at gmail.com>
wrote:

> Any reason in particular for not using github issues (or blockers in
> achieving this)?
>

Many core devs did not want to change their workflow, and Brett did not
want to have to handle two migrations at once.

When a new serious volunteer steps up we might consider it (a PEP would
have to be written).

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/26aa43ff/attachment.html>

From tjreedy at udel.edu  Wed Feb 21 16:45:16 2018
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 21 Feb 2018 16:45:16 -0500
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
Message-ID: <p6kp5e$4e1$1@blaine.gmane.org>

On 2/20/2018 8:58 PM, Brett Cannon wrote:
> It's been a year and 10 days since we moved to GitHub, so I figured now 
> is as good a time as any to ask people if they are generally happy with 
> the workflow

***The nearly automatic backporting!!!  A responsible human *should* 
check backport PRs, as it is possible for a merge to 'work', but be 
wrong, but that is all that a human *needs* to do.

> and if there is a particular sticking point to please bring 
> it up on the core-workflow mailing list so we can potentially address it.

Neither Travis not *nix or Mac buildbots test tkinter code.  I will post 
on the c-w list, but I don't know if that covers buildbots.

-- 
Terry Jan Reedy


From rosuav at gmail.com  Wed Feb 21 16:49:34 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 22 Feb 2018 08:49:34 +1100
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAP7+vJ+1Wv-HtrwXc2=XeGBWcTaVJkLZwtW2LdG5+qqDq2=pHQ@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <CANy2qHcGn8WD1P=cUqiR=QC_A0SwWDCyBSws2z1a3gozq0j9og@mail.gmail.com>
 <CAP7+vJ+1Wv-HtrwXc2=XeGBWcTaVJkLZwtW2LdG5+qqDq2=pHQ@mail.gmail.com>
Message-ID: <CAPTjJmpdDp4=+oP39HNV6wj=ycz4RmRW0EtDbTphsFPye14ciQ@mail.gmail.com>

On Thu, Feb 22, 2018 at 8:38 AM, Guido van Rossum <guido at python.org> wrote:
> On Wed, Feb 21, 2018 at 1:28 PM, Bern?t G?bor <jokerjokerer at gmail.com>
> wrote:
>>
>> Any reason in particular for not using github issues (or blockers in
>> achieving this)?
>
>
> Many core devs did not want to change their workflow, and Brett did not want
> to have to handle two migrations at once.
>
> When a new serious volunteer steps up we might consider it (a PEP would have
> to be written).
>

Said PEP may also need to mention the possibility of exporting the
history of GitHub issues, in case CPython ever migrates away from
GitHub; I remember that concern being raised when the original
migration was discussed.

ChrisA

From barry at python.org  Wed Feb 21 17:19:54 2018
From: barry at python.org (Barry Warsaw)
Date: Wed, 21 Feb 2018 14:19:54 -0800
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
Message-ID: <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>

On Feb 21, 2018, at 13:22, Guido van Rossum <guido at python.org> wrote:
> 
> I'm willing to reconsider if there's a good enough tool. Ditto for C code (or do we already do it for C?).

For Python code, flake8 --possibly with our own custom plugins? is the way to go.  It would probably take some kind of ratchet or transition period before all of the stdlib were compliant.  We?d have to be careful of the inevitable raft of PRs to fix things, which may distract from actual bug fixes and improvements.  OTOH, that would be another external dependency pulled in for core Python development.

> FWIW I'm personally hugely happy with the new workflow -- my only regret is that we're not using GitHub for issue tracking yet.

I?m very happy with the workflow too, and Mariatta?s and others work has been hugely important in making things work so well.  I still miss my ?commit when CI completes? button, but oh well.

As for the bug tracker, I still do like Roundup, and we have a huge investment in it, not just in resources expended to make it rock, but also in all the history in it and everything that integrates with it.  I wouldn?t stop anybody who?s motivated to spearhead a move to GH issues, but I also don?t think that can be taken up lightly.  Just look at the vast amount of work Brett and others had to do to migrate code hosting.  It would be nice to have integrated reviews and issues (e.g. for auto-closing perhaps), but I also honestly don?t miss much with the current suite of tools.

Cheers,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/28d58355/attachment.sig>

From chris.barker at noaa.gov  Wed Feb 21 17:30:52 2018
From: chris.barker at noaa.gov (Chris Barker)
Date: Wed, 21 Feb 2018 14:30:52 -0800
Subject: [Python-Dev] PEP 467 (Minor API improvements for binary
 sequences) - any thoughts?
In-Reply-To: <5A8DD4DA.10306@stoneleaf.us>
References: <CAFdOKfv+WMn6B0o4UtyPf1ziASUihELmQ_3K_WUNNfDsUva9+w@mail.gmail.com>
 <5A8DD4DA.10306@stoneleaf.us>
Message-ID: <CALGmxEJJfOSuySF-y2f6DewrGmjsjdZ1zQ3CSuAEqfBOTpRmVw@mail.gmail.com>

On Wed, Feb 21, 2018 at 12:21 PM, Ethan Furman <ethan at stoneleaf.us> wrote:

> At this point the PEP itself has not been approved, and is undergoing
> changes.  I don't see anything happening with it right now while 3.7 is
> going through it's final stages to release.  Once 3.7.0 is published we can
> come back to this.
>

well, it was originally targeted for 3.5, so it did need kick to get going.

Probably too late for 3.7.0, but no reason not to get it moving if there is
support, and aren't objections.

Anyone know if it paused to controversy or just lack of momentum? From a
quick search, it looks like discussion simply petered out in Sept 2016.

-CHB




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/e1d6b59a/attachment-0001.html>

From guido at python.org  Wed Feb 21 17:35:51 2018
From: guido at python.org (Guido van Rossum)
Date: Wed, 21 Feb 2018 14:35:51 -0800
Subject: [Python-Dev] PEP 467 (Minor API improvements for binary
 sequences) - any thoughts?
In-Reply-To: <CALGmxEJJfOSuySF-y2f6DewrGmjsjdZ1zQ3CSuAEqfBOTpRmVw@mail.gmail.com>
References: <CAFdOKfv+WMn6B0o4UtyPf1ziASUihELmQ_3K_WUNNfDsUva9+w@mail.gmail.com>
 <5A8DD4DA.10306@stoneleaf.us>
 <CALGmxEJJfOSuySF-y2f6DewrGmjsjdZ1zQ3CSuAEqfBOTpRmVw@mail.gmail.com>
Message-ID: <CAP7+vJKqyni+TXWYVhs_4wcE=CKgL8NSvKfSCdDNJBDDwVwyoQ@mail.gmail.com>

It's too late for 3.7 period, but there's no reason it can't be considered
for 3.8.

On Wed, Feb 21, 2018 at 2:30 PM, Chris Barker <chris.barker at noaa.gov> wrote:

> On Wed, Feb 21, 2018 at 12:21 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
>
>> At this point the PEP itself has not been approved, and is undergoing
>> changes.  I don't see anything happening with it right now while 3.7 is
>> going through it's final stages to release.  Once 3.7.0 is published we can
>> come back to this.
>>
>
> well, it was originally targeted for 3.5, so it did need kick to get going.
>
> Probably too late for 3.7.0, but no reason not to get it moving if there
> is support, and aren't objections.
>
> Anyone know if it paused to controversy or just lack of momentum? From a
> quick search, it looks like discussion simply petered out in Sept 2016.
>
> -CHB
>
>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/21fd1541/attachment.html>

From chris.barker at noaa.gov  Wed Feb 21 17:36:10 2018
From: chris.barker at noaa.gov (Chris Barker)
Date: Wed, 21 Feb 2018 14:36:10 -0800
Subject: [Python-Dev] PEP 467 (Minor API improvements for binary
 sequences) - any thoughts?
In-Reply-To: <CAMofdRAWj4GuabDyD1z2hwy2_u01S0+Aoyu8DJ037n0LzDwSaw@mail.gmail.com>
References: <CAFdOKfv+WMn6B0o4UtyPf1ziASUihELmQ_3K_WUNNfDsUva9+w@mail.gmail.com>
 <CALGmxEKjG-=+dVY8sNPdszq84hOxUnG0Cp5MH6edJEz37v0_9g@mail.gmail.com>
 <CAMofdRAWj4GuabDyD1z2hwy2_u01S0+Aoyu8DJ037n0LzDwSaw@mail.gmail.com>
Message-ID: <CALGmxE+_kT07HNPPHeyMfhoqrN3DV26GTmLEHKpDaV_7CWaAEA@mail.gmail.com>

On Wed, Feb 21, 2018 at 12:39 PM, Steve Holden <steve at holdenweb.com> wrote:

> I think the chances of a "byte" object are about as good as the chances of
> a character object
>

probably right.


> (though one can always implement such in C extensions, that wouldn't build
> them into the syntax).
>

I think you could simply subclass, too (overriding __new__ and a couple
methods). But that would do exactly no good, unless you used your own
custom string and bytes objects, too. The whole point is that iterating
over a string (Or bytes) always returns an also-iterable object,
ad-infinitum.

This is the cause of the major remaining common "type error" in Python.
(the old integer division used to be the big one)


> The fact that characters are single-byte strings is responsible for
> certain anomalies with (e.g.) the __contains__ operator (list elements
> aren't lists, but string element are strings), but overall the choices made
> lead to sensible, comprehensible code.
>

I'm pretty convinced that the choice not to have a character type has had
basically zero benefits to sensible, comprehensible code, though it's not a
very big deal, either. not a big enough deal for the churn it would cause
to introduce it now, that's for sure.

so +1 for this PEP as is.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/a1f531b5/attachment.html>

From barry at python.org  Wed Feb 21 17:47:59 2018
From: barry at python.org (Barry Warsaw)
Date: Wed, 21 Feb 2018 14:47:59 -0800
Subject: [Python-Dev] Making "Provisional" an explicit PEP state
In-Reply-To: <CADiSq7dNmt_KcgMi3yvv9wxL58zEd2NY8+csUMn7g1KzYBYhyw@mail.gmail.com>
References: <CADiSq7dNmt_KcgMi3yvv9wxL58zEd2NY8+csUMn7g1KzYBYhyw@mail.gmail.com>
Message-ID: <04C0E483-A3AF-468A-8FE7-C2169EC79C64@python.org>

On Feb 20, 2018, at 22:42, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> In the current PEP workflow, provisionally accepted PEPs are marked as
> "Accepted", and remain in that state until they're declared stable and
> moved to Final.

I left a review on the PR, but the substance of the changes LGTM!

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/e1ada96c/attachment.sig>

From mcepl at cepl.eu  Wed Feb 21 17:29:56 2018
From: mcepl at cepl.eu (=?UTF-8?Q?Mat=C4=9Bj?= Cepl)
Date: Wed, 21 Feb 2018 23:29:56 +0100
Subject: [Python-Dev] How is the GitHub workflow working for people?
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <CANy2qHcGn8WD1P=cUqiR=QC_A0SwWDCyBSws2z1a3gozq0j9og@mail.gmail.com>
 <CAP7+vJ+1Wv-HtrwXc2=XeGBWcTaVJkLZwtW2LdG5+qqDq2=pHQ@mail.gmail.com>
 <CAPTjJmpdDp4=+oP39HNV6wj=ycz4RmRW0EtDbTphsFPye14ciQ@mail.gmail.com>
Message-ID: <slrnp8rsn4.el0.mcepl@mitmanek.ceplovi.cz>

On 2018-02-21, 21:49 GMT, Chris Angelico wrote:
> Said PEP may also need to mention the possibility of exporting 
> the history of GitHub issues, in case CPython ever migrates away from
> GitHub; I remember that concern being raised when the original
> migration was discussed.

There are tools for it (e.g., I wrote 
https://github.com/mcepl/github-issues-export to move issues to 
Bugzilla, yes I am weird) and it is not that difficult to write 
something just to get data from GitHub?s all embracing arms. Of 
course, I am not sure how it would work on really large number 
of bugs, and it would be necessary to do some postprocessing 
(changing issue numbers etc.).

Best,

Mat?j
-- 
https://matej.ceplovi.cz/blog/, Jabber: mcepl at ceplovi.cz
GPG Finger: 3C76 A027 CA45 AD70 98B5  BC1D 7920 5802 880B C9D8
 
God is not worried about my financial situation.


From brett at snarky.ca  Tue Feb 20 20:54:42 2018
From: brett at snarky.ca (Brett Cannon)
Date: Wed, 21 Feb 2018 01:54:42 +0000
Subject: [Python-Dev] Backfilling 'awaiting' labels
In-Reply-To: <CAP1=2W4ziRUDh5c0g60qTcsDZVgHVsV4ryZOmAOBw_5xw=Qc3Q@mail.gmail.com>
References: <CAP1=2W4ziRUDh5c0g60qTcsDZVgHVsV4ryZOmAOBw_5xw=Qc3Q@mail.gmail.com>
Message-ID: <CAP1=2W7j4Oj8s41o3h9Y9Sd1+iUwu2Bx-UT1y15kbkiB0D=QtA@mail.gmail.com>

FYI I am now done backfilling issues.

On Tue, 30 Jan 2018 at 19:35 Brett Cannon <brett at snarky.ca> wrote:

> I have written a script that will go through and backfill the 'awaiting'
> label on older pull requests based on the review state as it stands today.
> A comment will be left if an "awaiting changes" label is set explaining
> that we're backfilling and if you're ready for a change review then leave
> the magical comment to trigger Bedevere.
>
> My plan is to limit this to only 20 total comments within a day so at to
> not overwhelm any single person with notifications. I will also run this
> script manually so there's no guarantee this will even occur every day.
>
> Assuming that 20 comment/day limit seems reasonable to people I will
> probably do the inaugural run tomorrow which will add an 'awaiting label'
> to 158 issues (which should be more than half of the issues lacking an
> 'awaiting' label).
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/09dc5eaf/attachment.html>

From brett at python.org  Wed Feb 21 19:42:30 2018
From: brett at python.org (Brett Cannon)
Date: Thu, 22 Feb 2018 00:42:30 +0000
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
Message-ID: <CAP1=2W5+mHQhQ6Thz+fSe=jwX5T-acyMdGR=CAd2WNj3RXY0Wg@mail.gmail.com>

On Wed, 21 Feb 2018 at 14:21 Barry Warsaw <barry at python.org> wrote:

> On Feb 21, 2018, at 13:22, Guido van Rossum <guido at python.org> wrote:
> >
> > I'm willing to reconsider if there's a good enough tool. Ditto for C
> code (or do we already do it for C?).
>
> For Python code, flake8 --possibly with our own custom plugins? is the way
> to go.  It would probably take some kind of ratchet or transition period
> before all of the stdlib were compliant.  We?d have to be careful of the
> inevitable raft of PRs to fix things, which may distract from actual bug
> fixes and improvements.  OTOH, that would be another external dependency
> pulled in for core Python development.
>

As long as we pin the version and upgrade only consciously I don't think
it's a massive issue to have an external dependency.

As for C, maybe clang-format?


>
> > FWIW I'm personally hugely happy with the new workflow -- my only regret
> is that we're not using GitHub for issue tracking yet.
>
> I?m very happy with the workflow too, and Mariatta?s and others work has
> been hugely important in making things work so well.  I still miss my
> ?commit when CI completes? button, but oh well.
>
> As for the bug tracker, I still do like Roundup, and we have a huge
> investment in it, not just in resources expended to make it rock, but also
> in all the history in it and everything that integrates with it.  I
> wouldn?t stop anybody who?s motivated to spearhead a move to GH issues, but
> I also don?t think that can be taken up lightly.  Just look at the vast
> amount of work Brett and others had to do to migrate code hosting.  It
> would be nice to have integrated reviews and issues (e.g. for auto-closing
> perhaps), but I also honestly don?t miss much with the current suite of
> tools.
>

All I will say is I don't plan on doing this one myself :) .
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180222/0bf2870e/attachment.html>

From ncoghlan at gmail.com  Wed Feb 21 21:52:32 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 22 Feb 2018 12:52:32 +1000
Subject: [Python-Dev] PEP 467 (Minor API improvements for binary
 sequences) - any thoughts?
In-Reply-To: <CAP7+vJKqyni+TXWYVhs_4wcE=CKgL8NSvKfSCdDNJBDDwVwyoQ@mail.gmail.com>
References: <CAFdOKfv+WMn6B0o4UtyPf1ziASUihELmQ_3K_WUNNfDsUva9+w@mail.gmail.com>
 <5A8DD4DA.10306@stoneleaf.us>
 <CALGmxEJJfOSuySF-y2f6DewrGmjsjdZ1zQ3CSuAEqfBOTpRmVw@mail.gmail.com>
 <CAP7+vJKqyni+TXWYVhs_4wcE=CKgL8NSvKfSCdDNJBDDwVwyoQ@mail.gmail.com>
Message-ID: <CADiSq7dUCs1L+rQ-mBrX1XyLs6jtX07f+opjVsOVLT4-5SBiEQ@mail.gmail.com>

On 22 February 2018 at 08:35, Guido van Rossum <guido at python.org> wrote:
> It's too late for 3.7 period, but there's no reason it can't be considered
> for 3.8.

Something else the PEP needs is a new champion - my original interest
was to help lower barriers to Python 3 migration, but it's now more
about the general ergonmics of the bytes type, and I don't do enough
low level protocol work these days to have a strong opinion on that.

That new champion could be Elias, or else perhaps Ethan Furman (who
drove the last round of proposed updates to the PEP, which
unfortunately don't appear to have been submitted to the PEPs repo:
https://mail.python.org/pipermail/python-dev/2016-September/146043.html)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From drsalists at gmail.com  Wed Feb 21 22:03:12 2018
From: drsalists at gmail.com (Dan Stromberg)
Date: Wed, 21 Feb 2018 19:03:12 -0800
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
Message-ID: <CAGGBd_r+LkkYqSMwBvjq69hiqo5gky7hk5htnsDemzHKyMfcXg@mail.gmail.com>

On Wed, Feb 21, 2018 at 2:19 PM, Barry Warsaw <barry at python.org> wrote:
> On Feb 21, 2018, at 13:22, Guido van Rossum <guido at python.org> wrote:
>>
>> I'm willing to reconsider if there's a good enough tool. Ditto for C code (or do we already do it for C?).
>
> For Python code, flake8 --possibly with our own custom plugins? is the way to go.

Is flake8 that much better than pylint, that pylint wouldn't even be discussed?

pylint does warn about some relatively unimportant things out of the
box, but it can be configured to ignore (almost?) everything it
checks.

I've been editing Python code in vim with syntastic.  I have syntastic
set up to run pyflakes, pycodestyle and pydocstyle.  And I have a
macro that saves the buffer, shells out and runs "make" with the
default rule - and that default rule almost always does a
whole-program pylint.

It's possible to make pylint check just one file from syntastic, but I
don't do that because it's too slow on large files.

I've gotten the impression that pylint can detect some errors that
pyflakes misses.  Not sure about flake8.

I do like it that flake8 has a mccabe check.  pylint doesn't appear to
have that, and instead relies on things like "too many statements",
"too many branches" and "too many variables".

From mariatta.wijaya at gmail.com  Wed Feb 21 22:13:54 2018
From: mariatta.wijaya at gmail.com (Mariatta Wijaya)
Date: Wed, 21 Feb 2018 19:13:54 -0800
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAP1=2W5+mHQhQ6Thz+fSe=jwX5T-acyMdGR=CAd2WNj3RXY0Wg@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <CAP1=2W5+mHQhQ6Thz+fSe=jwX5T-acyMdGR=CAd2WNj3RXY0Wg@mail.gmail.com>
Message-ID: <CAGbohnY1cfyqo1jW7Hztov6JnJv-AS7=CZknZn0xAuTXa5ozaw@mail.gmail.com>

>
> It's been a year and 10 days since we moved to GitHub, so I figured now is
> as good a time as any to ask people if they are generally happy with the
> workflow


It's been great! Thanks!


> Said PEP may also need to mention the possibility of exporting
> > the history of GitHub issues, in case CPython ever migrates away from
> > GitHub; I remember that concern being raised when the original
> > migration was discussed.
> There are tools for it (e.g., I wrote
> https://github.com/mcepl/github-issues-export to move issues to
> Bugzilla, yes I am weird) and it is not that difficult to write
> something just to get data from GitHub?s all embracing arms. Of
> course, I am not sure how it would work on really large number
> of bugs, and it would be necessary to do some postprocessing
> (changing issue numbers etc.).


It's really more complicated than just copying over text from the bpo to
GitHub. Can we preserve the history and the existing discussions?
What about existing patches currently attached to issues? Can the issue
number be preserved?
Everywhere in our commit message we reference bpo-NNNN. What will happen to
those?
We'll need to come up with new workflow. How do we want to triage the
issues? And what does it mean to current bug tracker triage permission?

The recent
> improvements to @miss-islington (kudos to Mariatta!) allowing her to
> auto-backport PRs and commit them is a big time saver.


Thanks! I can't take all credit. The bots are easy to build and maintain
mainly because of Brett Cannon's gidgethub [1] library. Thanks Brett!
Also... is this a good time to advertise my PyCon tutorial [2] about
building GitHub bots?

 I still miss my ?commit when CI completes? button, but oh well.


It is in my todo list, to at least notify when all CI completed so we can
go back and merge it.
I'll have time to think about this after PyCon US.

if there is a particular sticking point to please bring it up


This has been brought up several times in different mailing lists: Please
clean up the commit message before merging.

1. Ensure bpo-NNNNN is included.
If the bpo-NNNNN is not included, the commit doesn't get linked to the
issue in the b.p.o.

2. Ensure that the GitHub PR number is replaced, from (#12345) to (GH-12345)
In the b.p.o,  (#12345) links to bugs.python.org/issue12345 instead of
linking to GitHub PR number 12345.

3. Don't reference bpo-NNNNN as #NNNNN in the commit message.
tl;dr it breaks the bots. See [3] and [4]

One improvement I've been thinking about is related to the way we add news
entry using "blurb add" on the command line.
I wish there's a web UI for doing it. A place where I can type in the news
entry, give it the GitHub PR number, the bpo number, and with a button
click the News.d file magically created and added to the PR. Any thoughts
about this?


Mariatta Wijaya

[1] https://gidgethub.readthedocs.io/en/stable/
[2] https://us.pycon.org/2018/schedule/presentation/41/
[3] https://github.com/python/bedevere/issues/95#issuecomment-366570410
[4] https://github.com/python/bedevere/issues/97



?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/3726add2/attachment.html>

From ncoghlan at gmail.com  Wed Feb 21 22:22:58 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 22 Feb 2018 13:22:58 +1000
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
Message-ID: <CADiSq7ehLTUnoN6OTDo08+VsKDkhe8rRE60fc519QnnJtj3ZJA@mail.gmail.com>

On 22 February 2018 at 08:19, Barry Warsaw <barry at python.org> wrote:
> As for the bug tracker, I still do like Roundup, and we have a huge investment in it, not just in resources expended to make it rock, but also in all the history in it and everything that integrates with it.  I wouldn?t stop anybody who?s motivated to spearhead a move to GH issues, but I also don?t think that can be taken up lightly.

There are also several additional challenges with migrating the issue
tracker that didn't impact the code migration (at least, not as
badly):

* Issue number preservation: because the previous issue tracker
migration was from SourceForge to self-hosted, the SourceForge issue
numbers could be injected directly into Roundup. As far as I know, we
wouldn't be able to do that for a migration into GitHub - all existing
issues would be arbitrarily renumbered. (The most viable solution to
this would probably be to continue running a redirector on
bugs.python.org that translated existing issue links to their GitHub
counterparts)
* External links to issues are far more common than external links
directly into source control, so a migration strategy for preserving
those would be highly desirable (the redirector above would also help
handle that)
* One of the requirements for the GitHub migration was to continue to
offer contribution workflows that don't require creation of an account
with a proprietary American service provider. That workflow is to post
a patch on Roundup and ask someone else to convert it into a GitHub
PR. As a result, migrating the issue tracker would also mean saying
we're dropping that workflow requirement, and deliberately excluding
everyone without a GitHub account entirely from the core development
process (they won't even be able to file issues any more).
* The Roundup tracker is the database-of-record for CLA signatures. We
don't have the ability to add arbitrary user metadata on GitHub, so if
the issue tracker migrates to GitHub, we'll need a different solution
for CLA tracking.
* Roundup's nosy list is integrated with the experts index in the
developer guide, so an equivalent for that would need to be worked out
for GitHub in order to help out triagers

Personally, the only things I really miss on Roundup vs GitHub issues
are usability tweaks like Markdown support in the comment editor, and
inline dropdowns for @-mentions of other users and #-mentions of other
issues, so if someone is motivated to work on issue tracking
enhancements, that seems like a more fruitful endeavour than trying to
migrate wholesale to a proprietary third party service.

Cheers,
Nick.

P.S. That said, one tracker that I think absolutely *would* be worth
migrating to GitHub is the meta-tracker at
http://psf.upfronthosting.co.za/roundup/meta. We haven't customised
that instance the way we have bugs.python.org, and consolidating it
and the source repo at http://hg.python.org/tracker/roundup/ into a
single https://github.com/python/bugs.python.org repo would better
align tracker development with development on other parts of the
infrastructure.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ethan at stoneleaf.us  Wed Feb 21 22:25:01 2018
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 21 Feb 2018 19:25:01 -0800
Subject: [Python-Dev] PEP 467 (Minor API improvements for binary
 sequences) - any thoughts?
In-Reply-To: <CADiSq7dUCs1L+rQ-mBrX1XyLs6jtX07f+opjVsOVLT4-5SBiEQ@mail.gmail.com>
References: <CAFdOKfv+WMn6B0o4UtyPf1ziASUihELmQ_3K_WUNNfDsUva9+w@mail.gmail.com>
 <5A8DD4DA.10306@stoneleaf.us>
 <CALGmxEJJfOSuySF-y2f6DewrGmjsjdZ1zQ3CSuAEqfBOTpRmVw@mail.gmail.com>
 <CAP7+vJKqyni+TXWYVhs_4wcE=CKgL8NSvKfSCdDNJBDDwVwyoQ@mail.gmail.com>
 <CADiSq7dUCs1L+rQ-mBrX1XyLs6jtX07f+opjVsOVLT4-5SBiEQ@mail.gmail.com>
Message-ID: <5A8E380D.6010407@stoneleaf.us>

On 02/21/2018 06:52 PM, Nick Coghlan wrote:
> On 22 February 2018 at 08:35, Guido van Rossum <guido at python.org> wrote:
>> It's too late for 3.7 period, but there's no reason it can't be considered
>> for 3.8.
>
> Something else the PEP needs is a new champion - my original interest
> was to help lower barriers to Python 3 migration, but it's now more
> about the general ergonmics of the bytes type, and I don't do enough
> low level protocol work these days to have a strong opinion on that.
>
> That new champion could be Elias, or else perhaps Ethan Furman (who
> drove the last round of proposed updates to the PEP, which
> unfortunately don't appear to have been submitted to the PEPs repo:
> https://mail.python.org/pipermail/python-dev/2016-September/146043.html)

The changes were still sitting in my ancient mercurial repo.  Copied into the git repo and pushed.

--
~Ethan~


From guido at python.org  Wed Feb 21 22:27:53 2018
From: guido at python.org (Guido van Rossum)
Date: Wed, 21 Feb 2018 19:27:53 -0800
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAGGBd_r+LkkYqSMwBvjq69hiqo5gky7hk5htnsDemzHKyMfcXg@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <CAGGBd_r+LkkYqSMwBvjq69hiqo5gky7hk5htnsDemzHKyMfcXg@mail.gmail.com>
Message-ID: <CAP7+vJKGg3Uq8YYbTwU9qcUggG-Mkkqaspk5_SJZEyXYin-vFQ@mail.gmail.com>

Where I work we have some teams using flake8 and some teams that use
pylint, and while pylint is more thorough, it is also slower and pickier,
and the general sense is to strongly prefer flake8.

I honestly expect that running either with close-to-default flags on stdlib
code would be a nightmare, and I wouldn't want *any* directives for either
one to appear in stdlib code, ever.

In some ideal future all code would just be reformatted before it's checked
in -- we're very far from that, and I used to be horrified by the very
idea, but in the Go world this is pretty much standard practice, and the
people at work who are using it are loving it. So I'm trying to have an
open mind about this. But there simply isn't a tool that does a good enough
job of this.

What I was thinking of was a much weaker option like tabnanny.py by Tim
Peters (still in the stdlib!), but I don't know whether this is feasible.

What we need now is not more opinions on which formatter or linter is best.
We need someone to actually do some work and estimate how much code would
be changed if we ran e.g. tabnanny.py (or something more advanced!) over
the entire stdlib, how much code would break (even the most conservative
formatter sometimes breaks code that wasn't expecting to be reformatted --
e.g. we used to have tests with significant trailing whitespace), and how
often the result would be just too ugly to look at. If you're not willing
to work on that, please don't respond to this thread.

--Guido

On Wed, Feb 21, 2018 at 7:03 PM, Dan Stromberg <drsalists at gmail.com> wrote:

> On Wed, Feb 21, 2018 at 2:19 PM, Barry Warsaw <barry at python.org> wrote:
> > On Feb 21, 2018, at 13:22, Guido van Rossum <guido at python.org> wrote:
> >>
> >> I'm willing to reconsider if there's a good enough tool. Ditto for C
> code (or do we already do it for C?).
> >
> > For Python code, flake8 --possibly with our own custom plugins? is the
> way to go.
>
> Is flake8 that much better than pylint, that pylint wouldn't even be
> discussed?
>
> pylint does warn about some relatively unimportant things out of the
> box, but it can be configured to ignore (almost?) everything it
> checks.
>
> I've been editing Python code in vim with syntastic.  I have syntastic
> set up to run pyflakes, pycodestyle and pydocstyle.  And I have a
> macro that saves the buffer, shells out and runs "make" with the
> default rule - and that default rule almost always does a
> whole-program pylint.
>
> It's possible to make pylint check just one file from syntastic, but I
> don't do that because it's too slow on large files.
>
> I've gotten the impression that pylint can detect some errors that
> pyflakes misses.  Not sure about flake8.
>
> I do like it that flake8 has a mccabe check.  pylint doesn't appear to
> have that, and instead relies on things like "too many statements",
> "too many branches" and "too many variables".
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180221/2556cf68/attachment-0001.html>

From ncoghlan at gmail.com  Wed Feb 21 22:34:49 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 22 Feb 2018 13:34:49 +1000
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAGGBd_r+LkkYqSMwBvjq69hiqo5gky7hk5htnsDemzHKyMfcXg@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <CAGGBd_r+LkkYqSMwBvjq69hiqo5gky7hk5htnsDemzHKyMfcXg@mail.gmail.com>
Message-ID: <CADiSq7emX-n1tmLidj+YM6h+XGQcDurQpq7g1uhQ+FdRKZizYA@mail.gmail.com>

On 22 February 2018 at 13:03, Dan Stromberg <drsalists at gmail.com> wrote:
> On Wed, Feb 21, 2018 at 2:19 PM, Barry Warsaw <barry at python.org> wrote:
>> On Feb 21, 2018, at 13:22, Guido van Rossum <guido at python.org> wrote:
>>>
>>> I'm willing to reconsider if there's a good enough tool. Ditto for C code (or do we already do it for C?).
>>
>> For Python code, flake8 --possibly with our own custom plugins? is the way to go.
>
> Is flake8 that much better than pylint, that pylint wouldn't even be discussed?
>
> pylint does warn about some relatively unimportant things out of the
> box, but it can be configured to ignore (almost?) everything it
> checks.

"pylint -E" is my own preferred IDE-level checker (since it's pretty
good about reserving the "Error" category for things that almost
certainly indicate bugs), but all of these tools share a common
problem when applied to CPython: we routinely break them :)

Part of that is permanent breakage in the test suite where we
deliberately test the compiler and interpreter's reaction to invalid
code, while the other part is transient breakage when we introduce new
syntax and other language level constructs.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Wed Feb 21 22:37:37 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 22 Feb 2018 13:37:37 +1000
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAP7+vJKGg3Uq8YYbTwU9qcUggG-Mkkqaspk5_SJZEyXYin-vFQ@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <CAGGBd_r+LkkYqSMwBvjq69hiqo5gky7hk5htnsDemzHKyMfcXg@mail.gmail.com>
 <CAP7+vJKGg3Uq8YYbTwU9qcUggG-Mkkqaspk5_SJZEyXYin-vFQ@mail.gmail.com>
Message-ID: <CADiSq7d=C_eX_RcNZ6KkP0LFxOfMPPKFosC0fCFjsvjvQu1n4w@mail.gmail.com>

On 22 February 2018 at 13:27, Guido van Rossum <guido at python.org> wrote:
> What we need now is not more opinions on which formatter or linter is best.
> We need someone to actually do some work and estimate how much code would be
> changed if we ran e.g. tabnanny.py (or something more advanced!) over the
> entire stdlib, how much code would break (even the most conservative
> formatter sometimes breaks code that wasn't expecting to be reformatted --
> e.g. we used to have tests with significant trailing whitespace), and how
> often the result would be just too ugly to look at.

I believe we still run Tools/scripts/reindent.py and
Tools/scripts/reindent-rst.py as pre-merge checks. (I actually thought
we add a reindent-c.py script as well, since Py3 switched all the C
code over to using spaces for indentation to match the Python
conventions, but it appears not).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From yselivanov.ml at gmail.com  Wed Feb 21 23:02:20 2018
From: yselivanov.ml at gmail.com (Yury Selivanov)
Date: Wed, 21 Feb 2018 23:02:20 -0500
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAP7+vJKGg3Uq8YYbTwU9qcUggG-Mkkqaspk5_SJZEyXYin-vFQ@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <CAGGBd_r+LkkYqSMwBvjq69hiqo5gky7hk5htnsDemzHKyMfcXg@mail.gmail.com>
 <CAP7+vJKGg3Uq8YYbTwU9qcUggG-Mkkqaspk5_SJZEyXYin-vFQ@mail.gmail.com>
Message-ID: <CA+St6D104HcUjwWf2csU_zvs7w7vrT9D1WkbYvd--c=4kehAFA@mail.gmail.com>

On Wed, Feb 21, 2018 at 10:27 PM, Guido van Rossum <guido at python.org> wrote:
[..]
> I honestly expect that running either with close-to-default flags on stdlib
> code would be a nightmare, and I wouldn't want *any* directives for either
> one to appear in stdlib code, ever.

It would be great to enable the linter on a per-module basis then.
For instance, I believe that all files in asyncio package pass flake8
with default flags (at least I'm doing my best to keep it that way).
Sometimes it takes an extra review round to fix the code style, having
the CI to enforce it would save time for everybody.

Something similar to "cpython/.github/CODEOWNERS" but for enabling
linters would work.

Yury

From benjamin at python.org  Wed Feb 21 23:16:47 2018
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 21 Feb 2018 20:16:47 -0800
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
Message-ID: <1519273007.487928.1279232208.7559B989@webmail.messagingengine.com>



On Wed, Feb 21, 2018, at 13:22, Guido van Rossum wrote:
> On Wed, Feb 21, 2018 at 9:53 AM, Brett Cannon <brett at python.org> wrote:
> 
> >
> >
> > On Wed, 21 Feb 2018 at 09:30 Yury Selivanov <yselivanov.ml at gmail.com>
> > wrote:
> >
> >> FWIW I'm extremely happy with the current workflow. The recent
> >> improvements to @miss-islington (kudos to Mariatta!) allowing her to
> >> auto-backport PRs and commit them is a big time saver.
> >>
> >> I can only suggest a couple improvements:
> >>
> >> 1. Make our bots check the code style?fully enforce PEP 8, lint the
> >> code, and detect trailing whitespace on all lines that a PR modifies.
> >>
> >
> > Guido said "no" to this from the outset.
> >
> 
> I'm willing to reconsider if there's a good enough tool. Ditto for C code
> (or do we already do it for C?).

The C/C++ world has very good tooling for formating in the form of clang-format. If someone made a clang-format configuration file close to the current PEP 7 guide and existing code, I would support running it over the entire codbase and enforcing formatting in CI. Massive reformatings of CPython C have actually already uneventfully happened: several years ago Antoine removed all tabs from C sources.

From arj.python at gmail.com  Thu Feb 22 00:29:48 2018
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Thu, 22 Feb 2018 09:29:48 +0400
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CADrxXXk_y9w7yY1KQC-6sWcEGfj9SMGwx-qL95dsib2P62GUvA@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <CADrxXXk_y9w7yY1KQC-6sWcEGfj9SMGwx-qL95dsib2P62GUvA@mail.gmail.com>
Message-ID: <CADrxXXkCRCOZVcaqHRTeXFqaST8zAPbycdWcut8R+FnKBNgh8Q@mail.gmail.com>

>FWIW I'm personally hugely happy with
>the new workflow -- my only regret is
>that we're not using GitHub for issue
>tracking yet.

Btw you can create your own issue labels. some pythonic names can be
considered

Abdur-Rahmaan Janhangeer
https://github.com/Abdur-rahmaanJ

On 22 Feb 2018 01:23, "Guido van Rossum" <guido at python.org> wrote:

...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180222/366a3679/attachment.html>

From raymond.hettinger at gmail.com  Thu Feb 22 01:56:08 2018
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Wed, 21 Feb 2018 22:56:08 -0800
Subject: [Python-Dev] Should the dataclass frozen property apply to
 subclasses?
Message-ID: <A27F9EB7-D3BA-4358-B23E-A4EFAD7088AB@gmail.com>

When working on the docs for dataclasses, something unexpected came up.  If a dataclass is specified to be frozen, that characteristic is inherited by subclasses which prevents them from assigning additional attributes:

    >>> @dataclass(frozen=True)
    class D:
            x: int = 10

    >>> class S(D):
            pass

    >>> s = S()
    >>> s.cached = True
    Traceback (most recent call last):
      File "<pyshell#49>", line 1, in <module>
        s.cached = True
      File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/dataclasses.py", line 448, in _frozen_setattr
        raise FrozenInstanceError(f'cannot assign to field {name!r}')
    dataclasses.FrozenInstanceError: cannot assign to field 'cached'

Other immutable classes in Python don't behave the same way:


    >>> class T(tuple):
            pass

    >>> t = T([10, 20, 30])
    >>> t.cached = True

    >>> class F(frozenset):
            pass

    >>> f = F([10, 20, 30])
    >>> f.cached = True

    >>> class B(bytes):
            pass

    >>> b = B()
    >>> b.cached = True


Raymond

From solipsis at pitrou.net  Thu Feb 22 05:12:39 2018
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 22 Feb 2018 11:12:39 +0100
Subject: [Python-Dev] How is the GitHub workflow working for people?
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
Message-ID: <20180222111239.4814f10e@fsol>

On Wed, 21 Feb 2018 14:19:54 -0800
Barry Warsaw <barry at python.org> wrote:
> On Feb 21, 2018, at 13:22, Guido van Rossum <guido at python.org> wrote:
> > 
> > I'm willing to reconsider if there's a good enough tool. Ditto for C code (or do we already do it for C?).  
> 
> For Python code, flake8 --possibly with our own custom plugins? is the way to go.  It would probably take some kind of ratchet or transition period before all of the stdlib were compliant.  We?d have to be careful of the inevitable raft of PRs to fix things, which may distract from actual bug fixes and improvements.  OTOH, that would be another external dependency pulled in for core Python development.

Everytime I contribute to a project which has automatic style checks in
CI, I find myself having to push pointless "cleanup" commits because
the CI is barking at me for some ridiculous reason (such as putting two
linefeeds instead of one between two Python functions).  Then I have to
wait some more until CI finishes, especially if builds take long or
build queues are clogged.

Overall it makes contributing more of a PITA than it needs be.  Do
automatic style *fixes* if you want, but please don't annoy me with
automatic style checks that ask me to do tedious grunt work on my spare
time.

Regards

Antoine.



From eric at trueblade.com  Thu Feb 22 05:55:07 2018
From: eric at trueblade.com (Eric V. Smith)
Date: Thu, 22 Feb 2018 05:55:07 -0500
Subject: [Python-Dev] Should the dataclass frozen property apply to
 subclasses?
In-Reply-To: <A27F9EB7-D3BA-4358-B23E-A4EFAD7088AB@gmail.com>
References: <A27F9EB7-D3BA-4358-B23E-A4EFAD7088AB@gmail.com>
Message-ID: <1a6d1aa6-60e1-58db-ef2d-672c7f6c6c49@trueblade.com>

On 2/22/2018 1:56 AM, Raymond Hettinger wrote:
> When working on the docs for dataclasses, something unexpected came up.  If a dataclass is specified to be frozen, that characteristic is inherited by subclasses which prevents them from assigning additional attributes:
> 
>      >>> @dataclass(frozen=True)
>      class D:
>              x: int = 10
> 
>      >>> class S(D):
>              pass
> 
>      >>> s = S()
>      >>> s.cached = True
>      Traceback (most recent call last):
>        File "<pyshell#49>", line 1, in <module>
>          s.cached = True
>        File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/dataclasses.py", line 448, in _frozen_setattr
>          raise FrozenInstanceError(f'cannot assign to field {name!r}')
>      dataclasses.FrozenInstanceError: cannot assign to field 'cached'

This is because "frozen-ness" is implemented by adding __setattr__ and 
__delattr__ methods in D, which get inherited by S.

> Other immutable classes in Python don't behave the same way:
> 
> 
>      >>> class T(tuple):
>              pass
> 
>      >>> t = T([10, 20, 30])
>      >>> t.cached = True
> 
>      >>> class F(frozenset):
>              pass
> 
>      >>> f = F([10, 20, 30])
>      >>> f.cached = True
> 
>      >>> class B(bytes):
>              pass
> 
>      >>> b = B()
>      >>> b.cached = True

The only way I can think of emulating this is checking in __setattr__ to 
see if the field name is a field of the frozen class, and only raising 
an error in that case.

A related issue is that dataclasses derived from frozen dataclasses are 
automatically "promoted" to being frozen.

 >>> @dataclass(frozen=True)
... class A:
...     i: int
...
 >>> @dataclass
... class B(A):
...     j: int
...
 >>> b = B(1, 2)
 >>> b.j = 3
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "C:\home\eric\local\python\cpython\lib\dataclasses.py", line 
452, in _frozen_setattr
     raise FrozenInstanceError(f'cannot assign to field {name!r}')
dataclasses.FrozenInstanceError: cannot assign to field 'j'

Maybe it should be an error to declare B as non-frozen?

Eric.


From levkivskyi at gmail.com  Thu Feb 22 06:04:49 2018
From: levkivskyi at gmail.com (Ivan Levkivskyi)
Date: Thu, 22 Feb 2018 11:04:49 +0000
Subject: [Python-Dev] Should the dataclass frozen property apply to
 subclasses?
In-Reply-To: <1a6d1aa6-60e1-58db-ef2d-672c7f6c6c49@trueblade.com>
References: <A27F9EB7-D3BA-4358-B23E-A4EFAD7088AB@gmail.com>
 <1a6d1aa6-60e1-58db-ef2d-672c7f6c6c49@trueblade.com>
Message-ID: <CAOMjWknPZPOmK3=RWstP_9pwTNND+VrrSTgU78o09qgzOMqzDg@mail.gmail.com>

On 22 February 2018 at 10:55, Eric V. Smith <eric at trueblade.com> wrote:

> On 2/22/2018 1:56 AM, Raymond Hettinger wrote:
>
>> Other immutable classes in Python don't behave the same way:
>
>
>>      >>> class T(tuple):
>>              pass
>>
>>      >>> t = T([10, 20, 30])
>>      >>> t.cached = True
>>
>>      >>> class F(frozenset):
>>              pass
>>
>>      >>> f = F([10, 20, 30])
>>      >>> f.cached = True
>>
>>      >>> class B(bytes):
>>              pass
>>
>>      >>> b = B()
>>      >>> b.cached = True
>>
>
> The only way I can think of emulating this is checking in __setattr__ to
> see if the field name is a field of the frozen class, and only raising an
> error in that case.
>

How about checking that the type of self is the type where decorator was
applied? For example (pseudocode):

def dataclass(cls, ...):
    def _set_attr(self, attr, value):
        if type(self) is not cls:
             use super()
        else:
            raise AttributeError
    cls.__setattr__ = _set_attr

It can be also more sophisticated, for example raising for all fields on
class where frozen=True was used, while only on frozen fields for
subclasses.

--
Ivan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180222/7f770ece/attachment.html>

From ethan at stoneleaf.us  Thu Feb 22 10:38:19 2018
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 22 Feb 2018 07:38:19 -0800
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <20180222111239.4814f10e@fsol>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <20180222111239.4814f10e@fsol>
Message-ID: <5A8EE3EB.5040705@stoneleaf.us>

On 02/22/2018 02:12 AM, Antoine Pitrou wrote:

> Overall it makes contributing more of a PITA than it needs be.  Do
> automatic style *fixes* if you want, but please don't annoy me with
> automatic style checks that ask me to do tedious grunt work on my spare
> time.

+1

--
~Ethan~


From steve.dower at python.org  Thu Feb 22 10:51:17 2018
From: steve.dower at python.org (Steve Dower)
Date: Thu, 22 Feb 2018 07:51:17 -0800
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <5A8EE3EB.5040705@stoneleaf.us>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <20180222111239.4814f10e@fsol> <5A8EE3EB.5040705@stoneleaf.us>
Message-ID: <mailman.62.1519314681.2787.python-dev@python.org>

It then becomes grunt work for reviewers, who also have to carefully balance encouraging new contributors against preventing the code base from getting worse.

I?d rather have a review bot that can detect problems in PRs and comment on them. We can choose to merge anyway and it won?t keep being noisy, but it also saves committers from potentially telling someone their contribution isn?t welcome because of their camelCase. (Maybe Mariatta?s tutorial will build this bot? Maybe I?ll go and learn how to do it myself :) )

(and here ends my contribution on this topic. Pretty sure we're firmly in core-workflow territory now.)

Top-posted from my Windows phone

From: Ethan Furman
Sent: Thursday, February 22, 2018 7:39
To: python-dev at python.org
Subject: Re: [Python-Dev] How is the GitHub workflow working for people?

On 02/22/2018 02:12 AM, Antoine Pitrou wrote:

> Overall it makes contributing more of a PITA than it needs be.  Do
> automatic style *fixes* if you want, but please don't annoy me with
> automatic style checks that ask me to do tedious grunt work on my spare
> time.

+1

--
~Ethan~

_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180222/94580276/attachment.html>

From solipsis at pitrou.net  Thu Feb 22 11:08:15 2018
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 22 Feb 2018 17:08:15 +0100
Subject: [Python-Dev] How is the GitHub workflow working for people?
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <20180222111239.4814f10e@fsol> <5A8EE3EB.5040705@stoneleaf.us>
 <3znJmZ4h2yzFqxh@mail.python.org>
Message-ID: <20180222170815.55773b4a@fsol>

On Thu, 22 Feb 2018 07:51:17 -0800
Steve Dower <steve.dower at python.org> wrote:
> It then becomes grunt work for reviewers, who also have to carefully balance encouraging new contributors against preventing the code base from getting worse.

That's a fair point I hadn't considered.  OTOH the style issues I
usually comment on as a reviewer aren't the kind that would be caught
by an automated style check (I tend to ask for comments or docstrings,
or be nitpicky about some variable or function name).  YMMV :-)

> I?d rather have a review bot that can detect problems in PRs and comment on them. We can choose to merge anyway and it won?t keep being noisy, but it also saves committers from potentially telling someone their contribution isn?t welcome because of their camelCase.

Yeah, that sounds like an interesting feature.

Regards

Antoine.



From p.f.moore at gmail.com  Thu Feb 22 11:30:22 2018
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 22 Feb 2018 16:30:22 +0000
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <20180222170815.55773b4a@fsol>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <20180222111239.4814f10e@fsol>
 <5A8EE3EB.5040705@stoneleaf.us> <3znJmZ4h2yzFqxh@mail.python.org>
 <20180222170815.55773b4a@fsol>
Message-ID: <CACac1F8JQ2Wmrs42Bp6usAqZoVn5cPoehYKHyyGJ21fn7_OYbg@mail.gmail.com>

On 22 February 2018 at 16:08, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Thu, 22 Feb 2018 07:51:17 -0800
> Steve Dower <steve.dower at python.org> wrote:
>> It then becomes grunt work for reviewers, who also have to carefully balance encouraging new contributors against preventing the code base from getting worse.
>
> That's a fair point I hadn't considered.  OTOH the style issues I
> usually comment on as a reviewer aren't the kind that would be caught
> by an automated style check (I tend to ask for comments or docstrings,
> or be nitpicky about some variable or function name).  YMMV :-)
>
>> I?d rather have a review bot that can detect problems in PRs and comment on them. We can choose to merge anyway and it won?t keep being noisy, but it also saves committers from potentially telling someone their contribution isn?t welcome because of their camelCase.
>
> Yeah, that sounds like an interesting feature.

My experience on pip is that automated style review is helpful for
avoiding debates over subjective details. But it does result in a
certain level of "tweak to satisfy the style checker" churn in PRs.
That can be frustrating when CI takes a long time to run.

Paul

From ericsnowcurrently at gmail.com  Thu Feb 22 12:09:51 2018
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Thu, 22 Feb 2018 10:09:51 -0700
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CACac1F8JQ2Wmrs42Bp6usAqZoVn5cPoehYKHyyGJ21fn7_OYbg@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <20180222111239.4814f10e@fsol>
 <5A8EE3EB.5040705@stoneleaf.us> <3znJmZ4h2yzFqxh@mail.python.org>
 <20180222170815.55773b4a@fsol>
 <CACac1F8JQ2Wmrs42Bp6usAqZoVn5cPoehYKHyyGJ21fn7_OYbg@mail.gmail.com>
Message-ID: <CALFfu7AOYpFSU+VmCTjaY-44iUTdGxUM01Tv2ph1gLRKRpLYuA@mail.gmail.com>

On Thu, Feb 22, 2018 at 9:30 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> My experience on pip is that automated style review is helpful for
> avoiding debates over subjective details.

This is the allure of Go's official linting tools.  Nobody is happy
with *all* the style choices but there isn't any room to fuss about it
so people don't.  Without that overhead, folks are more apt to lint
their changes.  (At least, that was my experience after several years
working on projects written in Go.)  One nice thing is that it frees
you up to argue about other things. :)

> But it does result in a
> certain level of "tweak to satisfy the style checker" churn in PRs.
> That can be frustrating when CI takes a long time to run.

I had exactly that experience on one particularly large Go project (on
GitHub, with slow CI, driven by bots).  To make matters worse, the
project had a dozen people actively working on it, meaning a high
potential that your PR would not apply cleanly if you took too long to
merge it.  So, coupled with slow CI, linting failures were
particularly onerous.  We all got in the habit of locally running the
linter frequently.  IIRC, I even set up VIM to run the linter whenever
I saved.

FWIW, I'm fine with a bot that leaves a message (or a review) if there
are linting issues.  Regardless, we should careful about adding any
extra overhead to our workflow, particularly since the move to GH has
been driven by the desire to reduce overhead.

-eric

From storchaka at gmail.com  Thu Feb 22 14:04:24 2018
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Thu, 22 Feb 2018 21:04:24 +0200
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
Message-ID: <p6n43o$qka$1@blaine.gmane.org>

Yet one discussion about reusing common subexpressions in comprehensions 
took place last week on the Python-ideas maillist (see topic "Temporary 
variables in comprehensions" [1]). The problem is that in comprehension 
like `[f(x) + g(f(x)) for x in range(10)]` the subexpression `f(x)` is 
evaluated twice. In normal loop you can introduce a temporary variable 
for `f(x)`. The OP wanted to add a special syntax for introducing 
temporary variables in comprehensions. This idea already was discussed 
multiple times in the past.

There are several ways of resolving this problem with existing syntax.

1. Inner generator expression:

     result = [y + g(y) for y in (f(x) for x in range(10))]

2. The same, but with extracting the inner generator expression as a 
variable:

     f_samples = (f(x) for x in range(10))
     result = [y+g(y) for y in f_samples]

3. Extracting the expression with repeated subexpressions as a function 
with local variables:

     def func(x):
         y = f(x)
         return y + g(y)
     result = [func(x) for x in range(10)]

4. Implementing the whole comprehension as a generator function:

     def gen():
         for x in range(10):
             y = f(x)
             yield y + g(y)
     result = list(gen())

5. Using a normal loop instead of a comprehension:

     result = []
     for x in range(10):
         y = f(x)
         result.append(y + g(y))

And maybe there are other ways.

Stephan Houben proposed an idiom which looks similar to new hypothetic 
syntax:

     result = [y + g(y) for x in range(10) for y in [f(x)]]

`for y in [expr]` in a comprehension means just assigning expr to y. I 
never seen this idiom before, but it can be a good replacement for a 
hypothetic syntax for assignment in comprehensions. It changes the 
original comprehension less than other approaches, just adds yet one 
element in a sequence of for-s and if-s. I think that after using it 
more widely it will become pretty idiomatic.

I have created a patch that optimizes this idiom, making it as fast as a 
normal assignment. [2] Yury suggested to ask Guido on the mailing list 
if he agrees that this language patten is worth optimizing/promoting.

[1] https://mail.python.org/pipermail/python-ideas/2018-February/048971.html
[2] https://bugs.python.org/issue32856


From jsbueno at python.org.br  Thu Feb 22 14:54:11 2018
From: jsbueno at python.org.br (Joao S. O. Bueno)
Date: Thu, 22 Feb 2018 16:54:11 -0300
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <p6n43o$qka$1@blaine.gmane.org>
References: <p6n43o$qka$1@blaine.gmane.org>
Message-ID: <CAH0mxTTBHa+28rWR_dC=1nwsaUt=LbBtY4Gs3HSNNVkoq3hkpA@mail.gmail.com>

This thing has bitten me in the past -

At the time I put together the "stackfull" package -

if allows stuff like:

from stackfull import push, pop
...
 [push(f(x)) + g(pop()) for x in range(10)]


It is painfully simple in its workings: it creates a plain old list in
the fame f_locals and uses
that as a stack in all stackfull.* operations.

Just posting because people involved in this thread might want to
experiment with that.
(it is on pypi)

   js
 -><-



On 22 February 2018 at 16:04, Serhiy Storchaka <storchaka at gmail.com> wrote:
> Yet one discussion about reusing common subexpressions in comprehensions
> took place last week on the Python-ideas maillist (see topic "Temporary
> variables in comprehensions" [1]). The problem is that in comprehension like
> `[f(x) + g(f(x)) for x in range(10)]` the subexpression `f(x)` is evaluated
> twice. In normal loop you can introduce a temporary variable for `f(x)`. The
> OP wanted to add a special syntax for introducing temporary variables in
> comprehensions. This idea already was discussed multiple times in the past.
>
> There are several ways of resolving this problem with existing syntax.
>
> 1. Inner generator expression:
>
>     result = [y + g(y) for y in (f(x) for x in range(10))]
>
> 2. The same, but with extracting the inner generator expression as a
> variable:
>
>     f_samples = (f(x) for x in range(10))
>     result = [y+g(y) for y in f_samples]
>
> 3. Extracting the expression with repeated subexpressions as a function with
> local variables:
>
>     def func(x):
>         y = f(x)
>         return y + g(y)
>     result = [func(x) for x in range(10)]
>
> 4. Implementing the whole comprehension as a generator function:
>
>     def gen():
>         for x in range(10):
>             y = f(x)
>             yield y + g(y)
>     result = list(gen())
>
> 5. Using a normal loop instead of a comprehension:
>
>     result = []
>     for x in range(10):
>         y = f(x)
>         result.append(y + g(y))
>
> And maybe there are other ways.
>
> Stephan Houben proposed an idiom which looks similar to new hypothetic
> syntax:
>
>     result = [y + g(y) for x in range(10) for y in [f(x)]]
>
> `for y in [expr]` in a comprehension means just assigning expr to y. I never
> seen this idiom before, but it can be a good replacement for a hypothetic
> syntax for assignment in comprehensions. It changes the original
> comprehension less than other approaches, just adds yet one element in a
> sequence of for-s and if-s. I think that after using it more widely it will
> become pretty idiomatic.
>
> I have created a patch that optimizes this idiom, making it as fast as a
> normal assignment. [2] Yury suggested to ask Guido on the mailing list if he
> agrees that this language patten is worth optimizing/promoting.
>
> [1] https://mail.python.org/pipermail/python-ideas/2018-February/048971.html
> [2] https://bugs.python.org/issue32856
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/jsbueno%40python.org.br

From ethan at stoneleaf.us  Thu Feb 22 15:03:34 2018
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 22 Feb 2018 12:03:34 -0800
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <CAH0mxTTBHa+28rWR_dC=1nwsaUt=LbBtY4Gs3HSNNVkoq3hkpA@mail.gmail.com>
References: <p6n43o$qka$1@blaine.gmane.org>
 <CAH0mxTTBHa+28rWR_dC=1nwsaUt=LbBtY4Gs3HSNNVkoq3hkpA@mail.gmail.com>
Message-ID: <5A8F2216.70608@stoneleaf.us>

On 02/22/2018 11:54 AM, Joao S. O. Bueno wrote:
 > On 22 February 2018 at 16:04, Serhiy Storchaka wrote:

 >> Stephan Houben proposed an idiom which looks similar to new hypothetic
 >> syntax:
 >>
 >>      result = [y + g(y) for x in range(10) for y in [f(x)]]
 >
 > This thing has bitten me in the past -

Do you recall how?  That would be useful information.

--
~Ethan~

From barry at python.org  Thu Feb 22 16:33:35 2018
From: barry at python.org (Barry Warsaw)
Date: Thu, 22 Feb 2018 13:33:35 -0800
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <p6n43o$qka$1@blaine.gmane.org>
References: <p6n43o$qka$1@blaine.gmane.org>
Message-ID: <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>

On Feb 22, 2018, at 11:04, Serhiy Storchaka <storchaka at gmail.com> wrote:
> 
> Stephan Houben proposed an idiom which looks similar to new hypothetic syntax:
> 
>    result = [y + g(y) for x in range(10) for y in [f(x)]]
> 
> `for y in [expr]` in a comprehension means just assigning expr to y. I never seen this idiom before, but it can be a good replacement for a hypothetic syntax for assignment in comprehensions. It changes the original comprehension less than other approaches, just adds yet one element in a sequence of for-s and if-s. I think that after using it more widely it will become pretty idiomatic.

My questions are 1) will this become idiomatic enough to be able to understand at a glance what is going on, rather than having to pause to reason about what that 1-element list-like syntax actually means, and 2) will this encourage even more complicated comprehensions that are less readable than just expanding the code into a for-loop?

for-loops-are-not-evil-ly y?rs,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180222/2220dfb0/attachment.sig>

From turnbull.stephen.fw at u.tsukuba.ac.jp  Thu Feb 22 21:31:11 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Fri, 23 Feb 2018 11:31:11 +0900
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>
References: <p6n43o$qka$1@blaine.gmane.org>
 <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>
Message-ID: <23183.31983.765318.187362@turnbull.sk.tsukuba.ac.jp>

Barry Warsaw writes:

 > My questions are 1) will this become idiomatic enough to be able to
 > understand at a glance what is going on,

Is it similar enough to

    def f(x=[0]):

which is sometimes seen as a way to produce a mutable default value
for function arguments, to be "idiomatic"?

 > rather than having to pause to reason about what that 1-element
 > list-like syntax actually means, and 2) will this encourage even
 > more complicated comprehensions that are less readable than just
 > expanding the code into a for-loop?

Of course it will encourage more complicated comprehensions, and we
know that complexity is less readable.  On the other hand, a for loop
with a temporary variable will take up at least 3 statements vs. a
one-statement comprehension.

I don't have an opinion about the equities there.  I myself will
likely use the [(y, f(y)) for x in xs for y in costly(x)] idiom very
occasionally, with emphasis on "very" (for almost all "costly"
functions I might use that's the Knuthian root of error).  But I don't
know how others feel about it.

Steve


From ncoghlan at gmail.com  Thu Feb 22 21:43:09 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 23 Feb 2018 12:43:09 +1000
Subject: [Python-Dev] Should the dataclass frozen property apply to
 subclasses?
In-Reply-To: <1a6d1aa6-60e1-58db-ef2d-672c7f6c6c49@trueblade.com>
References: <A27F9EB7-D3BA-4358-B23E-A4EFAD7088AB@gmail.com>
 <1a6d1aa6-60e1-58db-ef2d-672c7f6c6c49@trueblade.com>
Message-ID: <CADiSq7e+zHWcAf-8BDQdcnNduyzMSnghAicJ08=k34UqTY6=Tw@mail.gmail.com>

On 22 February 2018 at 20:55, Eric V. Smith <eric at trueblade.com> wrote:
> On 2/22/2018 1:56 AM, Raymond Hettinger wrote:
>>
>> When working on the docs for dataclasses, something unexpected came up.
>> If a dataclass is specified to be frozen, that characteristic is inherited
>> by subclasses which prevents them from assigning additional attributes:
>>
>>      >>> @dataclass(frozen=True)
>>      class D:
>>              x: int = 10
>>
>>      >>> class S(D):
>>              pass
>>
>>      >>> s = S()
>>      >>> s.cached = True
>>      Traceback (most recent call last):
>>        File "<pyshell#49>", line 1, in <module>
>>          s.cached = True
>>        File
>> "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/dataclasses.py",
>> line 448, in _frozen_setattr
>>          raise FrozenInstanceError(f'cannot assign to field {name!r}')
>>      dataclasses.FrozenInstanceError: cannot assign to field 'cached'
>
>
> This is because "frozen-ness" is implemented by adding __setattr__ and
> __delattr__ methods in D, which get inherited by S.
>
>> Other immutable classes in Python don't behave the same way:
>>
>>
>>      >>> class T(tuple):
>>              pass
>>
>>      >>> t = T([10, 20, 30])
>>      >>> t.cached = True
>>
>>      >>> class F(frozenset):
>>              pass
>>
>>      >>> f = F([10, 20, 30])
>>      >>> f.cached = True
>>
>>      >>> class B(bytes):
>>              pass
>>
>>      >>> b = B()
>>      >>> b.cached = True
>
>
> The only way I can think of emulating this is checking in __setattr__ to see
> if the field name is a field of the frozen class, and only raising an error
> in that case.

If you were going to do that then it would likely make more sense to
convert the frozen fields to data descriptors, so __setattr__ only
gets called for attempts to add new attributes.

Then for the `frozen=False` case, the decorator could force
__setattr__ and __delattr__ back to the default implementations from
object, rather than relying on the behaviour inherited from base
classes.

> A related issue is that dataclasses derived from frozen dataclasses are
> automatically "promoted" to being frozen.
>
>>>> @dataclass(frozen=True)
> ... class A:
> ...     i: int
> ...
>>>> @dataclass
> ... class B(A):
> ...     j: int
> ...
>>>> b = B(1, 2)
>>>> b.j = 3
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "C:\home\eric\local\python\cpython\lib\dataclasses.py", line 452, in
> _frozen_setattr
>     raise FrozenInstanceError(f'cannot assign to field {name!r}')
> dataclasses.FrozenInstanceError: cannot assign to field 'j'
>
> Maybe it should be an error to declare B as non-frozen?

It would be nice to avoid that, as a mutable subclass of a frozen base
class could be a nice way to model hashable-but-mutable types:

    >>> @dataclass(frozen=True) # This is the immutable/hashable bit
    ... class A:
    ...     i: int
    ...
    >>> @dataclass # This adds the mutable-but-comparable parts
    ... class B(A):
    ...     j: int
    ...     __hash__ = A.__hash__


However, disallowing this case for now *would* be a reasonable way to
postpone making a decision until 3.8 - in the meantime, folks would
still be able to experiment by overriding __setattr__ and __delattr__
after the dataclass decorator sets them.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From jimjjewett at gmail.com  Fri Feb 23 00:05:54 2018
From: jimjjewett at gmail.com (Jim J. Jewett)
Date: Fri, 23 Feb 2018 00:05:54 -0500
Subject: [Python-Dev] Dataclasses, frozen and __post_init__
Message-ID: <CA+OGgf5xdkr+a+5azo8QUXLy+utMLLDpDSX8Q-+0TGem97xP=A@mail.gmail.com>

On Mon, Feb 19, 2018 at 5:06 PM, Chris Barker - NOAA Federal <
chris.barker at noaa.gov> wrote:

> If I have this right, on the discussion about frozen and hash, a use
> case was brought up for taking a few steps to create an instance (and
> thus wanting it not frozen) and then wanting it hashable.

> Which pointed to the idea of a ? freeze this from now on? method.

> This seems another use case ? maybe it would be helpful to be able to
> freeze an instance after creation for multiple use-cases?

Yes, it would be helpful.  But in practice, I've just limited the hash
function to only the attributes that are available before I need to
stick the object in a dict.  In practice, that has always been more
than sufficient.

-jJ

From greg at krypto.org  Fri Feb 23 01:55:28 2018
From: greg at krypto.org (Gregory P. Smith)
Date: Fri, 23 Feb 2018 06:55:28 +0000
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAP7+vJKGg3Uq8YYbTwU9qcUggG-Mkkqaspk5_SJZEyXYin-vFQ@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <CAGGBd_r+LkkYqSMwBvjq69hiqo5gky7hk5htnsDemzHKyMfcXg@mail.gmail.com>
 <CAP7+vJKGg3Uq8YYbTwU9qcUggG-Mkkqaspk5_SJZEyXYin-vFQ@mail.gmail.com>
Message-ID: <CAGE7PNJ1J0yPBSGwLBMme4TRngJgqYNdd=M-i_mExVgxVVC2QA@mail.gmail.com>

How often do we find ourselves grumbling over .py file style in PRs on
github?  If the answer to that isn't very often, the rest of my response
below seems moot. :)

On Wed, Feb 21, 2018 at 7:30 PM Guido van Rossum <guido at python.org> wrote:

> Where I work we have some teams using flake8 and some teams that use
> pylint, and while pylint is more thorough, it is also slower and pickier,
> and the general sense is to strongly prefer flake8.
>
> I honestly expect that running either with close-to-default flags on
> stdlib code would be a nightmare, and I wouldn't want *any* directives for
> either one to appear in stdlib code, ever.
>
> In some ideal future all code would just be reformatted before it's
> checked in -- we're very far from that, and I used to be horrified by the
> very idea, but in the Go world this is pretty much standard practice, and
> the people at work who are using it are loving it. So I'm trying to have an
> open mind about this. But there simply isn't a tool that does a good enough
> job of this.
>

I don't know the specifics of your idea of "a good enough job of this" for
an auto-formatter is (we'd need to settle on that definition in a PEP)..
but there is for my definition: yapf <https://pypi.python.org/pypi/yapf>.

Many teams require it pre-check-in for their code at Google these days. We
also use it for auto-reformatting of surrounding lines of code during all
sorts of mass-refactorings.  IIRC Lukas said Instagram/Facebook adopted it
as standard practice as well.

Some teams go all the way to enforce a "yapf must suggest no changes to the
edited areas of .py files" pre-submit error on their projects.

As Eric (and I believe Lukas in the past) has mentioned: auto formatters
don't have to produce mythical(*) "perfect" style the way individuals might
choose - they need to be good enough to keep people from arguing about
style with one another.  That's the productivity and consistency win.

What we need now is not more opinions on which formatter or linter is best.
> We need someone to actually do some work and estimate how much code would
> be changed if we ran e.g. tabnanny.py (or something more advanced!) over
> the entire stdlib, how much code would break (even the most conservative
> formatter sometimes breaks code that wasn't expecting to be reformatted --
> e.g. we used to have tests with significant trailing whitespace), and how
> often the result would be just too ugly to look at. If you're not willing
> to work on that, please don't respond to this thread.
>

Indeed.  There are at *least* four Python style and gotcha checkers in wide
use for Python code out there today.  Prematurely picking one up front
rather than coming up for criteria of which limited set of things we would
want and could practically use on the stdlib + testsuite seems wrong.

I've added "run yapf on all of CPython's .py files" to my list of things to
explore (or encourage someone else to) some day...

-gps

(*) caveat: Given that Guido has obtained History status
<https://twitter.com/gvanrossum/status/966346442477510657> and is BDFL he
can define mythical perfect style
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180223/b1a45a17/attachment-0001.html>

From greg at krypto.org  Fri Feb 23 02:00:12 2018
From: greg at krypto.org (Gregory P. Smith)
Date: Fri, 23 Feb 2018 07:00:12 +0000
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
Message-ID: <CAGE7PNLoz6Ehg5v3Z12W0ZMxXA2ESw10ouJZJ=NWSZhvMRV+rw@mail.gmail.com>

On Tue, Feb 20, 2018 at 6:50 PM Brett Cannon <brett at python.org> wrote:

> It's been a year and 10 days since we moved to GitHub, so I figured now is
> as good a time as any to ask people if they are generally happy with the
> workflow and if there is a particular sticking point to please bring it up
> on the core-workflow mailing list so we can potentially address it.
>

+1 happy!  Especially with the amazing automation piled on top.

It makes it sooo much easier to deal with changes coming from people than
anything involving manual patch files and clients.  Even within github's
quite limited concept of a code review tool (from a Googler perspective).

I do feel like we need more CI resources during sprints.  But we always
need more everything resources during sprints so that is nothing new and
not related to github itself.

The move to our github workflow is a win for all Python users in the world.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180223/a1516801/attachment.html>

From stefan_ml at behnel.de  Fri Feb 23 04:12:25 2018
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 23 Feb 2018 10:12:25 +0100
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <23183.31983.765318.187362@turnbull.sk.tsukuba.ac.jp>
References: <p6n43o$qka$1@blaine.gmane.org>
 <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>
 <23183.31983.765318.187362@turnbull.sk.tsukuba.ac.jp>
Message-ID: <p6olpr$2qv$1@blaine.gmane.org>

Stephen J. Turnbull schrieb am 23.02.2018 um 03:31:
> Barry Warsaw writes:
>  > rather than having to pause to reason about what that 1-element
>  > list-like syntax actually means, and 2) will this encourage even
>  > more complicated comprehensions that are less readable than just
>  > expanding the code into a for-loop?
> 
> Of course it will encourage more complicated comprehensions, and we
> know that complexity is less readable.  On the other hand, a for loop
> with a temporary variable will take up at least 3 statements vs. a
> one-statement comprehension.

IMHO, any complex comprehension should be split across multiple lines,
definitely if it uses multiple for-loops, as in the discussed example. So
the "space win" of a complex comprehension that requires temporary values
over a multi-line for-statement is actually not big in these cases.

There are certainly cases where a comprehension still looks better, but I'm
all for not encouraging a hacky idiom to stuff more into a comprehension.
Comprehensions should be used to *improve* readabilty, not to reduce it.

Stefan


From storchaka at gmail.com  Fri Feb 23 04:27:59 2018
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Fri, 23 Feb 2018 11:27:59 +0200
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>
References: <p6n43o$qka$1@blaine.gmane.org>
 <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>
Message-ID: <p6omn0$6ha$1@blaine.gmane.org>

22.02.18 23:33, Barry Warsaw ????:
> On Feb 22, 2018, at 11:04, Serhiy Storchaka <storchaka at gmail.com> wrote:
>>
>> Stephan Houben proposed an idiom which looks similar to new hypothetic syntax:
>>
>>     result = [y + g(y) for x in range(10) for y in [f(x)]]
>>
>> `for y in [expr]` in a comprehension means just assigning expr to y. I never seen this idiom before, but it can be a good replacement for a hypothetic syntax for assignment in comprehensions. It changes the original comprehension less than other approaches, just adds yet one element in a sequence of for-s and if-s. I think that after using it more widely it will become pretty idiomatic.
> 
> My questions are 1) will this become idiomatic enough to be able to understand at a glance what is going on, rather than having to pause to reason about what that 1-element list-like syntax actually means, and 2) will this encourage even more complicated comprehensions that are less readable than just expanding the code into a for-loop?

I think everyone will have to pause when encounter this idiom the first 
time. Next time it will look more common. But the same is happened with 
other idioms like "lambda x=x:", "'...' % (x,)", "x = x or {}", etc. 
This is a correct Python syntax, and you don't need to know anything 
special, besides learned from the tutorial, for understanding it.

All other alternatives (except the first one, which looks to me less 
readable than iterating a 1-element list) can't be used as an 
expression. Then require several statements. At least four statements in 
the case of a for-loop.


From p.f.moore at gmail.com  Fri Feb 23 04:32:14 2018
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 23 Feb 2018 09:32:14 +0000
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <p6olpr$2qv$1@blaine.gmane.org>
References: <p6n43o$qka$1@blaine.gmane.org>
 <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>
 <23183.31983.765318.187362@turnbull.sk.tsukuba.ac.jp>
 <p6olpr$2qv$1@blaine.gmane.org>
Message-ID: <CACac1F8aMRtv9mSb559h295iEBDmAtr6LAtQHoeo-M2+i8Xy-g@mail.gmail.com>

On 23 February 2018 at 09:12, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Stephen J. Turnbull schrieb am 23.02.2018 um 03:31:
>> Barry Warsaw writes:
>>  > rather than having to pause to reason about what that 1-element
>>  > list-like syntax actually means, and 2) will this encourage even
>>  > more complicated comprehensions that are less readable than just
>>  > expanding the code into a for-loop?
>>
>> Of course it will encourage more complicated comprehensions, and we
>> know that complexity is less readable.  On the other hand, a for loop
>> with a temporary variable will take up at least 3 statements vs. a
>> one-statement comprehension.
>
> IMHO, any complex comprehension should be split across multiple lines,
> definitely if it uses multiple for-loops, as in the discussed example. So
> the "space win" of a complex comprehension that requires temporary values
> over a multi-line for-statement is actually not big in these cases.
>
> There are certainly cases where a comprehension still looks better, but I'm
> all for not encouraging a hacky idiom to stuff more into a comprehension.
> Comprehensions should be used to *improve* readabilty, not to reduce it.

In my view:

1. The proposal is for an optimisation, not a change to the language.
So anything bad that can be done after the change, can be done now.
2. I doubt many people avoid this construct at the moment because it's
slow, it's more likely they do so because they hadn't thought of it,
or because it harms readability.
3. Announcing that this construct is no longer slow might encourage
some extra people to use it (because they now know about it, and they
assume that the fact that we've optimised it implies we think it's a
good idea).
4. Ultimately, readability will be the main factor here. And
readability is subjective, so we sort of have to trust people to use
their common sense.

This could easily be a premature optimisation. But on the other hand,
it's a case of not making things unexpectedly slow, so I'm fine with
that.

If Serihy doesn't feel that the optimisation code is a major
maintenance burden, I'd say go for it. It's a minor quality of life
improvement for a niche case, let's not view (or promote) it as
anything more than that.

Paul

From ncoghlan at gmail.com  Fri Feb 23 06:13:24 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 23 Feb 2018 21:13:24 +1000
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CALFfu7AOYpFSU+VmCTjaY-44iUTdGxUM01Tv2ph1gLRKRpLYuA@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <20180222111239.4814f10e@fsol>
 <5A8EE3EB.5040705@stoneleaf.us> <3znJmZ4h2yzFqxh@mail.python.org>
 <20180222170815.55773b4a@fsol>
 <CACac1F8JQ2Wmrs42Bp6usAqZoVn5cPoehYKHyyGJ21fn7_OYbg@mail.gmail.com>
 <CALFfu7AOYpFSU+VmCTjaY-44iUTdGxUM01Tv2ph1gLRKRpLYuA@mail.gmail.com>
Message-ID: <CADiSq7dcDpct2frr_FujaX3ZmaqFboL3ck-98zbS8OTZ4t1APA@mail.gmail.com>

On 23 February 2018 at 03:09, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> So, coupled with slow CI, linting failures were
> particularly onerous.  We all got in the habit of locally running the
> linter frequently.  IIRC, I even set up VIM to run the linter whenever
> I saved.

For reindent.py, we used to have instructions for setting that up as a
local pre-commit hook in Mercurial. It may make sense to create
similar instructions for git (if we don't already have those
somewhere).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From wes.turner at gmail.com  Fri Feb 23 08:12:13 2018
From: wes.turner at gmail.com (Wes Turner)
Date: Fri, 23 Feb 2018 08:12:13 -0500
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CADiSq7dcDpct2frr_FujaX3ZmaqFboL3ck-98zbS8OTZ4t1APA@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <20180222111239.4814f10e@fsol>
 <5A8EE3EB.5040705@stoneleaf.us> <3znJmZ4h2yzFqxh@mail.python.org>
 <20180222170815.55773b4a@fsol>
 <CACac1F8JQ2Wmrs42Bp6usAqZoVn5cPoehYKHyyGJ21fn7_OYbg@mail.gmail.com>
 <CALFfu7AOYpFSU+VmCTjaY-44iUTdGxUM01Tv2ph1gLRKRpLYuA@mail.gmail.com>
 <CADiSq7dcDpct2frr_FujaX3ZmaqFboL3ck-98zbS8OTZ4t1APA@mail.gmail.com>
Message-ID: <CACfEFw_booXB-UPVh1pOFFoJHnWwGsGdzfKMJ0QOK64vbD-9PA@mail.gmail.com>

On Friday, February 23, 2018, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 23 February 2018 at 03:09, Eric Snow <ericsnowcurrently at gmail.com>
> wrote:
> > So, coupled with slow CI, linting failures were
> > particularly onerous.  We all got in the habit of locally running the
> > linter frequently.  IIRC, I even set up VIM to run the linter whenever
> > I saved.
>
> For reindent.py, we used to have instructions for setting that up as a
> local pre-commit hook in Mercurial. It may make sense to create
> similar instructions for git (if we don't already have those
> somewhere).


A pre-commit hook with flake8, clang-format, and the requisite CPython
style configs would be great.

https://git-scm.com/book/gr/v2/Customizing-Git-Git-Hooks

Batch and ad-hoc CLN commits to cleanup for style really muddle the git
blame history.


>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180223/db2e0696/attachment.html>

From eric at trueblade.com  Fri Feb 23 08:12:14 2018
From: eric at trueblade.com (Eric V. Smith)
Date: Fri, 23 Feb 2018 08:12:14 -0500
Subject: [Python-Dev] Should the dataclass frozen property apply to
 subclasses?
In-Reply-To: <CADiSq7e+zHWcAf-8BDQdcnNduyzMSnghAicJ08=k34UqTY6=Tw@mail.gmail.com>
References: <A27F9EB7-D3BA-4358-B23E-A4EFAD7088AB@gmail.com>
 <1a6d1aa6-60e1-58db-ef2d-672c7f6c6c49@trueblade.com>
 <CADiSq7e+zHWcAf-8BDQdcnNduyzMSnghAicJ08=k34UqTY6=Tw@mail.gmail.com>
Message-ID: <c460ed83-988f-ef23-195c-a4932fffa213@trueblade.com>

On 2/22/18 9:43 PM, Nick Coghlan wrote:
> On 22 February 2018 at 20:55, Eric V. Smith <eric at trueblade.com> wrote:
>> A related issue is that dataclasses derived from frozen dataclasses are
>> automatically "promoted" to being frozen.
>>
>>>>> @dataclass(frozen=True)
>> ... class A:
>> ...     i: int
>> ...
>>>>> @dataclass
>> ... class B(A):
>> ...     j: int
>> ...
>>>>> b = B(1, 2)
>>>>> b.j = 3
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>   File "C:\home\eric\local\python\cpython\lib\dataclasses.py", line 452, in
>> _frozen_setattr
>>     raise FrozenInstanceError(f'cannot assign to field {name!r}')
>> dataclasses.FrozenInstanceError: cannot assign to field 'j'
>>
>> Maybe it should be an error to declare B as non-frozen?
>
> It would be nice to avoid that, as a mutable subclass of a frozen base
> class could be a nice way to model hashable-but-mutable types:
>
>     >>> @dataclass(frozen=True) # This is the immutable/hashable bit
>     ... class A:
>     ...     i: int
>     ...
>     >>> @dataclass # This adds the mutable-but-comparable parts
>     ... class B(A):
>     ...     j: int
>     ...     __hash__ = A.__hash__
>
>
> However, disallowing this case for now *would* be a reasonable way to
> postpone making a decision until 3.8 - in the meantime, folks would
> still be able to experiment by overriding __setattr__ and __delattr__
> after the dataclass decorator sets them.

Yes, that's what I was thinking. If we can't come up with a good 
solution for this now, make it an error and think about it for 3.8.

I'm doing some experimenting on this and Raymond's case and I'll post 
some ideas this weekend.

Eric

From wes.turner at gmail.com  Fri Feb 23 08:18:54 2018
From: wes.turner at gmail.com (Wes Turner)
Date: Fri, 23 Feb 2018 08:18:54 -0500
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CADiSq7ehLTUnoN6OTDo08+VsKDkhe8rRE60fc519QnnJtj3ZJA@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <CADiSq7ehLTUnoN6OTDo08+VsKDkhe8rRE60fc519QnnJtj3ZJA@mail.gmail.com>
Message-ID: <CACfEFw-xCYyOWaCUzkUDTQHdEOBakKb0O5iA0ssriuiKQGf=TQ@mail.gmail.com>

On Wednesday, February 21, 2018, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 22 February 2018 at 08:19, Barry Warsaw <barry at python.org> wrote:
> > As for the bug tracker, I still do like Roundup, and we have a huge
> investment in it, not just in resources expended to make it rock, but also
> in all the history in it and everything that integrates with it.  I
> wouldn?t stop anybody who?s motivated to spearhead a move to GH issues, but
> I also don?t think that can be taken up lightly.
>
> [...]
>

> Personally, the only things I really miss on Roundup vs GitHub issues
> are usability tweaks like Markdown support in the comment editor, and
> inline dropdowns for @-mentions of other users and #-mentions of other
> issues, so if someone is motivated to work on issue tracking
> enhancements, that seems like a more fruitful endeavour than trying to
> migrate wholesale to a proprietary third party service.


Additional Roundup ENHancements:

- [ ] ENH: Issue mention 'trackbacks'
  Optionally quoting the full issue title inline instead of just #123 would
be great: "#123: Issue title"

- [ ] ENH: GitHub OAuth integration; to make it easy to login to Roundup
with GitHub credentials

  http://python-social-auth.readthedocs.io/en/latest/backends/github.html


>
> Cheers,
> Nick.
>
> P.S. That said, one tracker that I think absolutely *would* be worth
> migrating to GitHub is the meta-tracker at
> http://psf.upfronthosting.co.za/roundup/meta. We haven't customised
> that instance the way we have bugs.python.org, and consolidating it
> and the source repo at http://hg.python.org/tracker/roundup/ into a
> single https://github.com/python/bugs.python.org repo would better
> align tracker development with development on other parts of the
> infrastructure.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180223/04e84698/attachment.html>

From phd at phdru.name  Fri Feb 23 08:30:39 2018
From: phd at phdru.name (Oleg Broytman)
Date: Fri, 23 Feb 2018 14:30:39 +0100
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CACfEFw_booXB-UPVh1pOFFoJHnWwGsGdzfKMJ0QOK64vbD-9PA@mail.gmail.com>
References: <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <20180222111239.4814f10e@fsol> <5A8EE3EB.5040705@stoneleaf.us>
 <3znJmZ4h2yzFqxh@mail.python.org> <20180222170815.55773b4a@fsol>
 <CACac1F8JQ2Wmrs42Bp6usAqZoVn5cPoehYKHyyGJ21fn7_OYbg@mail.gmail.com>
 <CALFfu7AOYpFSU+VmCTjaY-44iUTdGxUM01Tv2ph1gLRKRpLYuA@mail.gmail.com>
 <CADiSq7dcDpct2frr_FujaX3ZmaqFboL3ck-98zbS8OTZ4t1APA@mail.gmail.com>
 <CACfEFw_booXB-UPVh1pOFFoJHnWwGsGdzfKMJ0QOK64vbD-9PA@mail.gmail.com>
Message-ID: <20180223133039.GA18211@phdru.name>

On Fri, Feb 23, 2018 at 08:12:13AM -0500, Wes Turner <wes.turner at gmail.com> wrote:
> A pre-commit hook with flake8

   The problem now is not how to configure git with flake8 (et al) but
how to configure flake8 (and other tools) to minimize the noise with the
current codebase.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From status at bugs.python.org  Fri Feb 23 12:09:57 2018
From: status at bugs.python.org (Python tracker)
Date: Fri, 23 Feb 2018 18:09:57 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20180223170957.695D811A935@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2018-02-16 - 2018-02-23)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    6485 (+29)
  closed 38190 (+37)
  total  44675 (+66)

Open issues with patches: 2530 


Issues opened (54)
==================

#31355: Remove Travis CI macOS job: rely on buildbots
https://bugs.python.org/issue31355  reopened by pitrou

#32008: Example suggest to use a TLSv1 socket
https://bugs.python.org/issue32008  reopened by christian.heimes

#32861: urllib.robotparser: incomplete __str__ methods
https://bugs.python.org/issue32861  opened by michael-lazar

#32862: os.dup2(fd, fd, inheritable=False) behaves inconsistently
https://bugs.python.org/issue32862  opened by izbyshev

#32865: os.pipe  creates inheritable FDs with a bad internal state on 
https://bugs.python.org/issue32865  opened by eryksun

#32866: zipimport loader.get_data() requires absolute zip file path
https://bugs.python.org/issue32866  opened by barry

#32867: argparse assertion failure with multiline metavars
https://bugs.python.org/issue32867  opened by MaT1g3R

#32871: Interrupt .communicate() on SIGTERM/INT
https://bugs.python.org/issue32871  opened by porton

#32872: backport of #32305 causes regressions in various packages
https://bugs.python.org/issue32872  opened by doko

#32873: Pickling of typing types
https://bugs.python.org/issue32873  opened by serhiy.storchaka

#32874: IDLE: Add tests for pyparse
https://bugs.python.org/issue32874  opened by csabella

#32875: Add __exit__() method to event loops
https://bugs.python.org/issue32875  opened by porton

#32876: HTMLParser raises exception on some inputs
https://bugs.python.org/issue32876  opened by hanno

#32877: Login to bugs.python.org with Google account NOT working
https://bugs.python.org/issue32877  opened by ruffsl

#32878: Document value of st_ino on Windows
https://bugs.python.org/issue32878  opened by gvanrossum

#32879: Race condition in multiprocessing Queue
https://bugs.python.org/issue32879  opened by TwistedSim

#32880: IDLE: Fix and update and cleanup pyparse
https://bugs.python.org/issue32880  opened by terry.reedy

#32881: pycapsule:PyObject * is NULL pointer
https://bugs.python.org/issue32881  opened by zhaoya

#32882: SSLContext.set_ecdh_curve() not accepting x25519
https://bugs.python.org/issue32882  opened by sruester

#32883: Key agreement parameters not accessible
https://bugs.python.org/issue32883  opened by sruester

#32884: Adding the ability for getpass to print asterisks when passowr
https://bugs.python.org/issue32884  opened by matanya.stroh

#32885: Tools/scripts/pathfix.py leaves bunch of ~ suffixed files arou
https://bugs.python.org/issue32885  opened by hroncok

#32886: new Boolean ABC in numbers module
https://bugs.python.org/issue32886  opened by smarie

#32887: os: Users of path_converter don't handle fd == -1 properly
https://bugs.python.org/issue32887  opened by izbyshev

#32888: Improve exception message in ast.literal_eval
https://bugs.python.org/issue32888  opened by Rosuav

#32890: os: Some functions may report bogus errors on Windows
https://bugs.python.org/issue32890  opened by izbyshev

#32891: Add 'Integer' as synonym for 'Integral' in numbers module.
https://bugs.python.org/issue32891  opened by terry.reedy

#32892: Remove specific constant AST types in favor of ast.Constant
https://bugs.python.org/issue32892  opened by serhiy.storchaka

#32893: ast.literal_eval() shouldn't accept booleans as numbers in AST
https://bugs.python.org/issue32893  opened by serhiy.storchaka

#32894: AST unparsing of infinity numbers
https://bugs.python.org/issue32894  opened by serhiy.storchaka

#32896: Error when subclassing a dataclass with a field that uses a de
https://bugs.python.org/issue32896  opened by John Didion

#32897: test_gdb failed on Fedora 27
https://bugs.python.org/issue32897  opened by amitg-b14

#32900: Teach pdb to step through asyncio et al.
https://bugs.python.org/issue32900  opened by smurfix

#32901: Update 3.7 and 3.8 Windows and macOS installer builds to tcl/t
https://bugs.python.org/issue32901  opened by terry.reedy

#32902: FileInput "inplace" redirects output of other threads
https://bugs.python.org/issue32902  opened by mj4int

#32903: os.chdir() may leak memory on Windows
https://bugs.python.org/issue32903  opened by izbyshev

#32904: os.chdir(), os.getcwd() may crash on Windows in presence of ra
https://bugs.python.org/issue32904  opened by izbyshev

#32907: pathlib: test_resolve_common fails on Windows
https://bugs.python.org/issue32907  opened by izbyshev

#32909: ApplePersistenceIgnoreState warning on macOS
https://bugs.python.org/issue32909  opened by cbrnr

#32910: venv: Deactivate.ps1 is not created when Activate.ps1 was used
https://bugs.python.org/issue32910  opened by godaygo

#32911: Doc strings no longer stored in body of AST
https://bugs.python.org/issue32911  opened by Mark.Shannon

#32912: Raise non-silent warning for invalid escape sequences
https://bugs.python.org/issue32912  opened by ebarry

#32913: Improve regular expression HOWTO
https://bugs.python.org/issue32913  opened by JoshuaRLi

#32914: python3-config --ldflags gives a CMP0004 error due to a whites
https://bugs.python.org/issue32914  opened by kekePower

#32915: Running Python 2 with -3 flag doesn't complain about cmp/__cmp
https://bugs.python.org/issue32915  opened by trey

#32916: IDLE: change 'str' to 'code' in idlelib.pyparse.PyParse and us
https://bugs.python.org/issue32916  opened by terry.reedy

#32917: ConfigParser writes a superfluous final blank line
https://bugs.python.org/issue32917  opened by tttnns

#32918: IDLE: make smart indent after comment line consistent
https://bugs.python.org/issue32918  opened by terry.reedy

#32919: csv.reader() to support QUOTE_ALL
https://bugs.python.org/issue32919  opened by Pavel Shpilev

#32920: Implement PEP 529 for os.getcwdb on Windows
https://bugs.python.org/issue32920  opened by eryksun

#32921: .pth files cannot contain folders with utf-8 names
https://bugs.python.org/issue32921  opened by einaren

#32922: dbm.open() encodes filename with default encoding rather than 
https://bugs.python.org/issue32922  opened by Josh Friend

#32923: Typo in documentation of unittest: whilst instead of while
https://bugs.python.org/issue32923  opened by ?????????????? ????????????????

#32924: Python 3.7 docs in docs.p.o points to GitHub's master branch
https://bugs.python.org/issue32924  opened by Mariatta



Most recent 15 issues with no replies (15)
==========================================

#32924: Python 3.7 docs in docs.p.o points to GitHub's master branch
https://bugs.python.org/issue32924

#32922: dbm.open() encodes filename with default encoding rather than 
https://bugs.python.org/issue32922

#32921: .pth files cannot contain folders with utf-8 names
https://bugs.python.org/issue32921

#32920: Implement PEP 529 for os.getcwdb on Windows
https://bugs.python.org/issue32920

#32919: csv.reader() to support QUOTE_ALL
https://bugs.python.org/issue32919

#32915: Running Python 2 with -3 flag doesn't complain about cmp/__cmp
https://bugs.python.org/issue32915

#32914: python3-config --ldflags gives a CMP0004 error due to a whites
https://bugs.python.org/issue32914

#32913: Improve regular expression HOWTO
https://bugs.python.org/issue32913

#32907: pathlib: test_resolve_common fails on Windows
https://bugs.python.org/issue32907

#32903: os.chdir() may leak memory on Windows
https://bugs.python.org/issue32903

#32902: FileInput "inplace" redirects output of other threads
https://bugs.python.org/issue32902

#32894: AST unparsing of infinity numbers
https://bugs.python.org/issue32894

#32890: os: Some functions may report bogus errors on Windows
https://bugs.python.org/issue32890

#32887: os: Users of path_converter don't handle fd == -1 properly
https://bugs.python.org/issue32887

#32878: Document value of st_ino on Windows
https://bugs.python.org/issue32878



Most recent 15 issues waiting for review (15)
=============================================

#32918: IDLE: make smart indent after comment line consistent
https://bugs.python.org/issue32918

#32917: ConfigParser writes a superfluous final blank line
https://bugs.python.org/issue32917

#32916: IDLE: change 'str' to 'code' in idlelib.pyparse.PyParse and us
https://bugs.python.org/issue32916

#32914: python3-config --ldflags gives a CMP0004 error due to a whites
https://bugs.python.org/issue32914

#32913: Improve regular expression HOWTO
https://bugs.python.org/issue32913

#32907: pathlib: test_resolve_common fails on Windows
https://bugs.python.org/issue32907

#32904: os.chdir(), os.getcwd() may crash on Windows in presence of ra
https://bugs.python.org/issue32904

#32903: os.chdir() may leak memory on Windows
https://bugs.python.org/issue32903

#32901: Update 3.7 and 3.8 Windows and macOS installer builds to tcl/t
https://bugs.python.org/issue32901

#32900: Teach pdb to step through asyncio et al.
https://bugs.python.org/issue32900

#32893: ast.literal_eval() shouldn't accept booleans as numbers in AST
https://bugs.python.org/issue32893

#32890: os: Some functions may report bogus errors on Windows
https://bugs.python.org/issue32890

#32888: Improve exception message in ast.literal_eval
https://bugs.python.org/issue32888

#32885: Tools/scripts/pathfix.py leaves bunch of ~ suffixed files arou
https://bugs.python.org/issue32885

#32882: SSLContext.set_ecdh_curve() not accepting x25519
https://bugs.python.org/issue32882



Top 10 most discussed issues (10)
=================================

#32880: IDLE: Fix and update and cleanup pyparse
https://bugs.python.org/issue32880  12 msgs

#32911: Doc strings no longer stored in body of AST
https://bugs.python.org/issue32911  12 msgs

#32604: Expose the subinterpreters C-API in Python for testing use.
https://bugs.python.org/issue32604  11 msgs

#32901: Update 3.7 and 3.8 Windows and macOS installer builds to tcl/t
https://bugs.python.org/issue32901  11 msgs

#32886: new Boolean ABC in numbers module
https://bugs.python.org/issue32886  10 msgs

#32846: Deletion of large sets of strings is extra slow
https://bugs.python.org/issue32846   8 msgs

#32849: Fatal Python error: Py_Initialize: can't initialize sys standa
https://bugs.python.org/issue32849   8 msgs

#32874: IDLE: Add tests for pyparse
https://bugs.python.org/issue32874   8 msgs

#32008: Example suggest to use a TLSv1 socket
https://bugs.python.org/issue32008   7 msgs

#32900: Teach pdb to step through asyncio et al.
https://bugs.python.org/issue32900   7 msgs



Issues closed (38)
==================

#18356: help(numpy) causes segfault on exit
https://bugs.python.org/issue18356  closed by csabella

#19412: Add docs for test.support.requires_docstrings decorator
https://bugs.python.org/issue19412  closed by csabella

#21060: Better error message for setup.py upload command without sdist
https://bugs.python.org/issue21060  closed by eric.araujo

#23462: All os.exec*e variants crash on Windows
https://bugs.python.org/issue23462  closed by steve.dower

#28339: "TypeError: Parameterized generics cannot be used with class o
https://bugs.python.org/issue28339  closed by levkivskyi

#28886: Move deprecated abc module decorators to separate section in d
https://bugs.python.org/issue28886  closed by inada.naoki

#29116: Make str and bytes error messages on concatenation conform wit
https://bugs.python.org/issue29116  closed by serhiy.storchaka

#30028: make test.support.temp_cwd() fork-safe
https://bugs.python.org/issue30028  closed by gregory.p.smith

#31333: Implement ABCMeta in C
https://bugs.python.org/issue31333  closed by levkivskyi

#31848: "aifc" module does not always initialize "Aifc_read._ssnd_chun
https://bugs.python.org/issue31848  closed by serhiy.storchaka

#31937: Add the term "dunder" to the glossary
https://bugs.python.org/issue31937  closed by brett.cannon

#31972: Inherited docstrings for pathlib classes are confusing
https://bugs.python.org/issue31972  closed by eric.araujo

#32016: Python 3.6.3 venv FAILURE
https://bugs.python.org/issue32016  closed by csabella

#32087: deprecated-removed directive generates overlapping msgids in .
https://bugs.python.org/issue32087  closed by inada.naoki

#32162: typing.Generic breaks __init_subclass__
https://bugs.python.org/issue32162  closed by levkivskyi

#32409: venv activate.bat is UTF-8 encoded but uses current console co
https://bugs.python.org/issue32409  closed by steve.dower

#32495: Adding Timer to multiprocessing
https://bugs.python.org/issue32495  closed by jcrotts

#32540: venv docs - doesn't match behavior
https://bugs.python.org/issue32540  closed by Mariatta

#32566: Not able to run Python 3.6 on Windows
https://bugs.python.org/issue32566  closed by steve.dower

#32682: test_zlib improve version parsing
https://bugs.python.org/issue32682  closed by pmpp

#32764: Popen doesn't work on Windows when args is a list
https://bugs.python.org/issue32764  closed by steve.dower

#32840: Must install python 3.6.3 when 3.6.4 already installed
https://bugs.python.org/issue32840  closed by steve.dower

#32852: trace changes sys.argv from list to tuple
https://bugs.python.org/issue32852  closed by serhiy.storchaka

#32858: Improve OpenSSL ECDH support
https://bugs.python.org/issue32858  closed by sruester

#32859: os.dup2() tests dup3() availability on each call
https://bugs.python.org/issue32859  closed by benjamin.peterson

#32860: Definition of iglob does not mention single star (unlike glob)
https://bugs.python.org/issue32860  closed by Mariatta

#32863: Missing support for Emojis in tkinter
https://bugs.python.org/issue32863  closed by serhiy.storchaka

#32864: Visual glitches when animating ScrolledText instances using pl
https://bugs.python.org/issue32864  closed by serhiy.storchaka

#32868: python 3 docs' iterator example has python 2 code
https://bugs.python.org/issue32868  closed by robotwizard

#32869: Incorrect dst buffer size for MultiByteToWideChar in _Py_fopen
https://bugs.python.org/issue32869  closed by izbyshev

#32870: Documentation typo (2.x only) for deque.remove
https://bugs.python.org/issue32870  closed by rhettinger

#32889: Valgrind suppressions need updating
https://bugs.python.org/issue32889  closed by benjamin.peterson

#32895: Make general function sum() use Numpy's sum when obviously pos
https://bugs.python.org/issue32895  closed by rhettinger

#32898: [BUILD] Using COUNT_ALLOCS breaks build
https://bugs.python.org/issue32898  closed by benjamin.peterson

#32899: Not documented: key in dict test may raise TypeError
https://bugs.python.org/issue32899  closed by rhettinger

#32905: IDLE: remove unused code in pyparse module
https://bugs.python.org/issue32905  closed by terry.reedy

#32906: AttributeError: 'NoneType' object has no attribute 'sendall'
https://bugs.python.org/issue32906  closed by steven.daprano

#32908: decimal ROUND_HALF_UP not according to spec for 9.95 to 10.0
https://bugs.python.org/issue32908  closed by mark.dickinson

From guido at python.org  Fri Feb 23 12:30:44 2018
From: guido at python.org (Guido van Rossum)
Date: Fri, 23 Feb 2018 09:30:44 -0800
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <p6n43o$qka$1@blaine.gmane.org>
References: <p6n43o$qka$1@blaine.gmane.org>
Message-ID: <CAP7+vJK+fDag27ua29_U2yMcirk0aecM4ND0TDd=e7v1RQWjOg@mail.gmail.com>

On Thu, Feb 22, 2018 at 11:04 AM, Serhiy Storchaka <storchaka at gmail.com>
wrote:

> Yet one discussion about reusing common subexpressions in comprehensions
> took place last week on the Python-ideas maillist (see topic "Temporary
> variables in comprehensions" [1]). The problem is that in comprehension
> like `[f(x) + g(f(x)) for x in range(10)]` the subexpression `f(x)` is
> evaluated twice. In normal loop you can introduce a temporary variable for
> `f(x)`. The OP wanted to add a special syntax for introducing temporary
> variables in comprehensions. This idea already was discussed multiple times
> in the past.
>
> There are several ways of resolving this problem with existing syntax.
>
> 1. Inner generator expression:
>
>     result = [y + g(y) for y in (f(x) for x in range(10))]
>
> 2. The same, but with extracting the inner generator expression as a
> variable:
>
>     f_samples = (f(x) for x in range(10))
>     result = [y+g(y) for y in f_samples]
>
> 3. Extracting the expression with repeated subexpressions as a function
> with local variables:
>
>     def func(x):
>         y = f(x)
>         return y + g(y)
>     result = [func(x) for x in range(10)]
>
> 4. Implementing the whole comprehension as a generator function:
>
>     def gen():
>         for x in range(10):
>             y = f(x)
>             yield y + g(y)
>     result = list(gen())
>
> 5. Using a normal loop instead of a comprehension:
>
>     result = []
>     for x in range(10):
>         y = f(x)
>         result.append(y + g(y))
>
> And maybe there are other ways.
>
> Stephan Houben proposed an idiom which looks similar to new hypothetic
> syntax:
>
>     result = [y + g(y) for x in range(10) for y in [f(x)]]
>
> `for y in [expr]` in a comprehension means just assigning expr to y. I
> never seen this idiom before, but it can be a good replacement for a
> hypothetic syntax for assignment in comprehensions. It changes the original
> comprehension less than other approaches, just adds yet one element in a
> sequence of for-s and if-s. I think that after using it more widely it will
> become pretty idiomatic.
>
> I have created a patch that optimizes this idiom, making it as fast as a
> normal assignment. [2] Yury suggested to ask Guido on the mailing list if
> he agrees that this language patten is worth optimizing/promoting.
>
> [1] https://mail.python.org/pipermail/python-ideas/2018-February
> /048971.html
> [2] https://bugs.python.org/issue32856
>

I'm not saying anything new here, but since you asked specifically for my
opinion: I don't care for the idiom; it's never occurred to me before, and
it smells of cleverness. If I saw it in a code review I would probably ask
for a regular for-loop to make the code more maintainable.

But if you say it's useful for some class of users and it would be more
useful if it was faster, I'm fine with the optimization. The optimization
is also clever, and here I appreciate cleverness!

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180223/a54748dc/attachment.html>

From chris.barker at noaa.gov  Fri Feb 23 12:41:50 2018
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Fri, 23 Feb 2018 12:41:50 -0500
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <23183.31983.765318.187362@turnbull.sk.tsukuba.ac.jp>
References: <p6n43o$qka$1@blaine.gmane.org>
 <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>
 <23183.31983.765318.187362@turnbull.sk.tsukuba.ac.jp>
Message-ID: <CALGmxEJ0+24hOf2QZ=hj2D1q_QF75evJV+uZycc3mq1YBLw5iQ@mail.gmail.com>

> Is it similar enough to
>
>    def f(x=[0]):

No, not at all ? it?s a very different use case.

When I first saw this on the original thread, I needed to stare at it
a good while, and then whip up some code to experiment with it to know
what it did.

And not because I don?t know what a single element list means, or what
it means to iterate over a single element list, or what two fors mean
in a comprehension.

I was confused by the ?x? in the second iterable. I guess I?m (still)
not really clear on the scope(s) inside a comprehension, and when the
elements get evaluated in a list.

I expected that the list would be created once, with the value x had
initially, rather than getting the-evaluated each time through the
outer loop.

So I think that it is a very confusing use of comprehensions, and
always will be. I?m still surprised it?s legal. Anyone know if this
being allowed was deliberate or just kind of fell out of the
implementation?

So no, I don?t think it should be promoted as idiomatic.

All that being said, it?s valid Python, so why not optimize it?

-CHB

From guido at python.org  Fri Feb 23 12:51:05 2018
From: guido at python.org (Guido van Rossum)
Date: Fri, 23 Feb 2018 09:51:05 -0800
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <CALGmxEJ0+24hOf2QZ=hj2D1q_QF75evJV+uZycc3mq1YBLw5iQ@mail.gmail.com>
References: <p6n43o$qka$1@blaine.gmane.org>
 <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>
 <23183.31983.765318.187362@turnbull.sk.tsukuba.ac.jp>
 <CALGmxEJ0+24hOf2QZ=hj2D1q_QF75evJV+uZycc3mq1YBLw5iQ@mail.gmail.com>
Message-ID: <CAP7+vJKQDsDW6aNsZvCp=_dxCk_1s6svJXKofNV-7u5rO-n5AQ@mail.gmail.com>

As to the validity or legality of this code, it's both, and working as
intended.

A list comprehension of the form

    [STUFF for VAR1 in SEQ1 for VAR2 in SEQ2 for VAR3 in SEQ3]

should be seen (informally) as

    for VAR1 in SEQ1:
        for VAR2 in SEQ2:
            for VAR3 in SEQ3:
                "put STUFF in the result"

(If there are `if COND` phrases too those get inserted into the nested set
of blocks where they occur in the sequence.)


On Fri, Feb 23, 2018 at 9:41 AM, Chris Barker - NOAA Federal <
chris.barker at noaa.gov> wrote:

> > Is it similar enough to
> >
> >    def f(x=[0]):
>
> No, not at all ? it?s a very different use case.
>
> When I first saw this on the original thread, I needed to stare at it
> a good while, and then whip up some code to experiment with it to know
> what it did.
>
> And not because I don?t know what a single element list means, or what
> it means to iterate over a single element list, or what two fors mean
> in a comprehension.
>
> I was confused by the ?x? in the second iterable. I guess I?m (still)
> not really clear on the scope(s) inside a comprehension, and when the
> elements get evaluated in a list.
>
> I expected that the list would be created once, with the value x had
> initially, rather than getting the-evaluated each time through the
> outer loop.
>
> So I think that it is a very confusing use of comprehensions, and
> always will be. I?m still surprised it?s legal. Anyone know if this
> being allowed was deliberate or just kind of fell out of the
> implementation?
>
> So no, I don?t think it should be promoted as idiomatic.
>
> All that being said, it?s valid Python, so why not optimize it?
>
> -CHB
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180223/1af3090c/attachment.html>

From chris.barker at noaa.gov  Fri Feb 23 13:16:03 2018
From: chris.barker at noaa.gov (Chris Barker)
Date: Fri, 23 Feb 2018 10:16:03 -0800
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <CAP7+vJKQDsDW6aNsZvCp=_dxCk_1s6svJXKofNV-7u5rO-n5AQ@mail.gmail.com>
References: <p6n43o$qka$1@blaine.gmane.org>
 <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>
 <23183.31983.765318.187362@turnbull.sk.tsukuba.ac.jp>
 <CALGmxEJ0+24hOf2QZ=hj2D1q_QF75evJV+uZycc3mq1YBLw5iQ@mail.gmail.com>
 <CAP7+vJKQDsDW6aNsZvCp=_dxCk_1s6svJXKofNV-7u5rO-n5AQ@mail.gmail.com>
Message-ID: <CALGmxE+Jr+-8cXXSuVmUA=0WRAUxqc6GAgTvexHpVD2a5pzomQ@mail.gmail.com>

On Fri, Feb 23, 2018 at 9:51 AM, Guido van Rossum <guido at python.org> wrote:

> As to the validity or legality of this code, it's both, and working as
> intended.
>
> A list comprehension of the form
>
>     [STUFF for VAR1 in SEQ1 for VAR2 in SEQ2 for VAR3 in SEQ3]
>
> should be seen (informally) as
>
>     for VAR1 in SEQ1:
>         for VAR2 in SEQ2:
>             for VAR3 in SEQ3:
>                 "put STUFF in the result"
>

Thanks -- right after posting, I realized that was the way to unpack it to
understand it. I think my confusion came from two things:

1) I usually don't care in which order the loops are ordered -- i.e., that
could be:

    for VAR3 in SEQ3:
        for VAR2 in SEQ2:
            for VAR1 in SEQ1:
                "put STUFF in the result"

As I usually don't care, I have to think about it (and maybe experiment to
be sure). (this is the old Fortran vs C order thing :-)

2) since it's a single expression, I wasn't sure of the evaluation order,
so maybe (in my head) it could have been (optimized) to be:

[STUFF for  VAR1 in Expression_that_evaluates_to_an_iterable1 for VAR2 in
Expression_that_evaluates_to_an_iterable2]

and that could translate to:

IT1 = Expression_that_evaluates_to_an_iterable1
IT2 = Expression_that_evaluates_to_an_iterable2
for VAR1 in IT1:
    for VAR2 in IT2:
        "put STUFF in the result"

In which case, VAR1 would not be available to
Expression_that_evaluates_to_an_iterable2.

Maybe that was very wrong headed -- but that's where my head went -- and
I'm not a Python newbie (maybe an oddity, though :-) )

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180223/d7833546/attachment-0001.html>

From guido at python.org  Fri Feb 23 13:45:20 2018
From: guido at python.org (Guido van Rossum)
Date: Fri, 23 Feb 2018 10:45:20 -0800
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <CALGmxE+Jr+-8cXXSuVmUA=0WRAUxqc6GAgTvexHpVD2a5pzomQ@mail.gmail.com>
References: <p6n43o$qka$1@blaine.gmane.org>
 <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>
 <23183.31983.765318.187362@turnbull.sk.tsukuba.ac.jp>
 <CALGmxEJ0+24hOf2QZ=hj2D1q_QF75evJV+uZycc3mq1YBLw5iQ@mail.gmail.com>
 <CAP7+vJKQDsDW6aNsZvCp=_dxCk_1s6svJXKofNV-7u5rO-n5AQ@mail.gmail.com>
 <CALGmxE+Jr+-8cXXSuVmUA=0WRAUxqc6GAgTvexHpVD2a5pzomQ@mail.gmail.com>
Message-ID: <CAP7+vJKQuqfWnsYOV68Qa3sFv7zy8A__L9LWT4gogzUqjGwkYA@mail.gmail.com>

There are useful things you can only do with comprehensions if the second
for-loop can use the variable in the first for-loop. E.g.

[(i, j) for i in range(10) for j in range(i)]

On Fri, Feb 23, 2018 at 10:16 AM, Chris Barker <chris.barker at noaa.gov>
wrote:

> On Fri, Feb 23, 2018 at 9:51 AM, Guido van Rossum <guido at python.org>
> wrote:
>
>> As to the validity or legality of this code, it's both, and working as
>> intended.
>>
>> A list comprehension of the form
>>
>>     [STUFF for VAR1 in SEQ1 for VAR2 in SEQ2 for VAR3 in SEQ3]
>>
>> should be seen (informally) as
>>
>>     for VAR1 in SEQ1:
>>         for VAR2 in SEQ2:
>>             for VAR3 in SEQ3:
>>                 "put STUFF in the result"
>>
>
> Thanks -- right after posting, I realized that was the way to unpack it to
> understand it. I think my confusion came from two things:
>
> 1) I usually don't care in which order the loops are ordered -- i.e., that
> could be:
>
>     for VAR3 in SEQ3:
>         for VAR2 in SEQ2:
>             for VAR1 in SEQ1:
>                 "put STUFF in the result"
>
> As I usually don't care, I have to think about it (and maybe experiment to
> be sure). (this is the old Fortran vs C order thing :-)
>
> 2) since it's a single expression, I wasn't sure of the evaluation order,
> so maybe (in my head) it could have been (optimized) to be:
>
> [STUFF for  VAR1 in Expression_that_evaluates_to_an_iterable1 for VAR2 in
> Expression_that_evaluates_to_an_iterable2]
>
> and that could translate to:
>
> IT1 = Expression_that_evaluates_to_an_iterable1
> IT2 = Expression_that_evaluates_to_an_iterable2
> for VAR1 in IT1:
>     for VAR2 in IT2:
>         "put STUFF in the result"
>
> In which case, VAR1 would not be available to Expression_that_evaluates_
> to_an_iterable2.
>
> Maybe that was very wrong headed -- but that's where my head went -- and
> I'm not a Python newbie (maybe an oddity, though :-) )
>
> -CHB
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE
> <https://maps.google.com/?q=7600+Sand+Point+Way+NE&entry=gmail&source=g>
>   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180223/df12cc00/attachment.html>

From chris.barker at noaa.gov  Fri Feb 23 14:23:04 2018
From: chris.barker at noaa.gov (Chris Barker)
Date: Fri, 23 Feb 2018 11:23:04 -0800
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <CAP7+vJKQuqfWnsYOV68Qa3sFv7zy8A__L9LWT4gogzUqjGwkYA@mail.gmail.com>
References: <p6n43o$qka$1@blaine.gmane.org>
 <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>
 <23183.31983.765318.187362@turnbull.sk.tsukuba.ac.jp>
 <CALGmxEJ0+24hOf2QZ=hj2D1q_QF75evJV+uZycc3mq1YBLw5iQ@mail.gmail.com>
 <CAP7+vJKQDsDW6aNsZvCp=_dxCk_1s6svJXKofNV-7u5rO-n5AQ@mail.gmail.com>
 <CALGmxE+Jr+-8cXXSuVmUA=0WRAUxqc6GAgTvexHpVD2a5pzomQ@mail.gmail.com>
 <CAP7+vJKQuqfWnsYOV68Qa3sFv7zy8A__L9LWT4gogzUqjGwkYA@mail.gmail.com>
Message-ID: <CALGmxE+kJ-JctOa2U3ALW_-LrevGfby+ndqmrNv2kCy_wAt3nA@mail.gmail.com>

On Fri, Feb 23, 2018 at 10:45 AM, Guido van Rossum <guido at python.org> wrote:

> There are useful things you can only do with comprehensions if the second
> for-loop can use the variable in the first for-loop. E.g.
>
> [(i, j) for i in range(10) for j in range(i)]
>

indeed -- and that is fairly common use-case in nested for loops -- so good
to preserve this.

But I still think the original:

[g(y) for x in range(5) for y in [f(x)]]

Is always going to be confusing to read. Though I do agree that it's not
too bad when you unpack it into for loops:

In [89]: for x in range(5):
    ...:     for y in [f(x)]:
    ...:         l.append(g(y))

BTW, would it be even a tiny bit more efficient to use a tuple in the inner
loop?

[g(y) for x in range(5) for y in (f(x),)]

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180223/b8ab47e2/attachment.html>

From stefan_ml at behnel.de  Fri Feb 23 15:50:39 2018
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 23 Feb 2018 21:50:39 +0100
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <CALGmxE+kJ-JctOa2U3ALW_-LrevGfby+ndqmrNv2kCy_wAt3nA@mail.gmail.com>
References: <p6n43o$qka$1@blaine.gmane.org>
 <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>
 <23183.31983.765318.187362@turnbull.sk.tsukuba.ac.jp>
 <CALGmxEJ0+24hOf2QZ=hj2D1q_QF75evJV+uZycc3mq1YBLw5iQ@mail.gmail.com>
 <CAP7+vJKQDsDW6aNsZvCp=_dxCk_1s6svJXKofNV-7u5rO-n5AQ@mail.gmail.com>
 <CALGmxE+Jr+-8cXXSuVmUA=0WRAUxqc6GAgTvexHpVD2a5pzomQ@mail.gmail.com>
 <CAP7+vJKQuqfWnsYOV68Qa3sFv7zy8A__L9LWT4gogzUqjGwkYA@mail.gmail.com>
 <CALGmxE+kJ-JctOa2U3ALW_-LrevGfby+ndqmrNv2kCy_wAt3nA@mail.gmail.com>
Message-ID: <p6pun0$7bi$1@blaine.gmane.org>

Chris Barker schrieb am 23.02.2018 um 20:23:
> BTW, would it be even a tiny bit more efficient to use a tuple in the inner
> loop?
> 
> [g(y) for x in range(5) for y in (f(x),)]

Serhiy's optimisation does not use a loop at all anymore and folds it into
a direct assignment "y=f(x)" instead.

But in general, yes, changing a list iterable into a tuple is an
improvement as tuples are more efficient to allocate. Haven't tried it in
CPython (*), but it might make a slight difference for very short
iterables, which are probably common. Although the execution of the loop
body will likely dominate the initial allocation by far.

Stefan



(*) I implemented this list->tuple transformation in Cython a while ago,
but seeing Serhiy's change now got me thinking that this could be further
improved into a stack allocated C array, to let the C compiler unroll the
loop at will. I'll probably try that at some point...

https://github.com/cython/cython/issues/2117


From turnbull.stephen.fw at u.tsukuba.ac.jp  Fri Feb 23 23:41:28 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Sat, 24 Feb 2018 13:41:28 +0900
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <CALGmxE+kJ-JctOa2U3ALW_-LrevGfby+ndqmrNv2kCy_wAt3nA@mail.gmail.com>
References: <p6n43o$qka$1@blaine.gmane.org>
 <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>
 <23183.31983.765318.187362@turnbull.sk.tsukuba.ac.jp>
 <CALGmxEJ0+24hOf2QZ=hj2D1q_QF75evJV+uZycc3mq1YBLw5iQ@mail.gmail.com>
 <CAP7+vJKQDsDW6aNsZvCp=_dxCk_1s6svJXKofNV-7u5rO-n5AQ@mail.gmail.com>
 <CALGmxE+Jr+-8cXXSuVmUA=0WRAUxqc6GAgTvexHpVD2a5pzomQ@mail.gmail.com>
 <CAP7+vJKQuqfWnsYOV68Qa3sFv7zy8A__L9LWT4gogzUqjGwkYA@mail.gmail.com>
 <CALGmxE+kJ-JctOa2U3ALW_-LrevGfby+ndqmrNv2kCy_wAt3nA@mail.gmail.com>
Message-ID: <23184.60664.613227.429793@turnbull.sk.tsukuba.ac.jp>

Chris Barker writes:

 > But I still think the original:
 > 
 > [g(y) for x in range(5) for y in [f(x)]]
 > 
 > Is always going to be confusing to read.

But the point I was making with "def f(x=[0]):" was this: you have a
situation where your desired semantics is "value of some type"[1], but
the language's syntax doesn't permit a value of that type there, while
"singleton sequence of that type" works fine.

In fact, "singleton as value" is baked into Python in the form of
str.__getitem__ and bytes.__getitem__.  So we now have four use cases
for singleton as value: two stringish actual types, and the two idioms
"mutable default argument" and "local variable in comprehension".  The
horse is long since out of the barn.

Steve

Footnotes: 
[1]  Both "value" and "type" are used rather loosely here.


From steve at pearwood.info  Sat Feb 24 00:20:16 2018
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 24 Feb 2018 16:20:16 +1100
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <CALGmxE+kJ-JctOa2U3ALW_-LrevGfby+ndqmrNv2kCy_wAt3nA@mail.gmail.com>
References: <p6n43o$qka$1@blaine.gmane.org>
 <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>
 <23183.31983.765318.187362@turnbull.sk.tsukuba.ac.jp>
 <CALGmxEJ0+24hOf2QZ=hj2D1q_QF75evJV+uZycc3mq1YBLw5iQ@mail.gmail.com>
 <CAP7+vJKQDsDW6aNsZvCp=_dxCk_1s6svJXKofNV-7u5rO-n5AQ@mail.gmail.com>
 <CALGmxE+Jr+-8cXXSuVmUA=0WRAUxqc6GAgTvexHpVD2a5pzomQ@mail.gmail.com>
 <CAP7+vJKQuqfWnsYOV68Qa3sFv7zy8A__L9LWT4gogzUqjGwkYA@mail.gmail.com>
 <CALGmxE+kJ-JctOa2U3ALW_-LrevGfby+ndqmrNv2kCy_wAt3nA@mail.gmail.com>
Message-ID: <20180224052016.GK10142@ando.pearwood.info>

On Fri, Feb 23, 2018 at 11:23:04AM -0800, Chris Barker wrote:

> But I still think the original:
> 
> [g(y) for x in range(5) for y in [f(x)]]
> 
> Is always going to be confusing to read. Though I do agree that it's not
> too bad when you unpack it into for loops:
> 
> In [89]: for x in range(5):
>     ...:     for y in [f(x)]:
>     ...:         l.append(g(y))


I think we should distinguish between: 

* actively confusing; and 
* merely not obvious at a glance.

I acknowledge that this idiom is not obvious at a glance, but I don't 
think this comes even close to actively confusing. As you say, once you 
mentally unpack the loops it becomes clear.

Given a potentially expensive DRY violation like:

    [(function(x), function(x)+1) for x in sequence]

there are at least five ways to solve it.

* Perhaps it doesn't need solving; if the DRY violation is trivial 
enough, and the cost low enough, who cares?

* Re-write as a for-loop instead of a comprehension;

* Use a helper function:

    def helper(x)
        tmp = function(x)
        return (tmp, tmp+1)

    [helper(x) for x in sequence]

* Chain the operations:

    [(a, a+1) for a in (function(x) for x in sequence)]

    [(a, a+1) for a in map(function, sequence)]

* Use a second loop to get an assignment:

    [(a, a+1) for x in sequence for a in [function(x)]]


I don't think we need to promote any one of the above as the One True 
idiom. They're all simple, more-or-less obvious or at least 
understandable, and deciding between them should be a matter of personal 
choice and in-house style guides.

 
> BTW, would it be even a tiny bit more efficient to use a tuple in the inner
> loop?
> 
> [g(y) for x in range(5) for y in (f(x),)]

The suggested patch will recognise both `y in (a,)` and `y in [a]` and 
treat them the same as a direct assignment `y=a`.

But if you're writing cross-interpreter code which might run on older 
versions of Python, or implementations which may not have this 
optimization, you might prefer to micro-optimize by using a tuple.


-- 
Steve

From mertz at gnosis.cx  Sat Feb 24 01:04:45 2018
From: mertz at gnosis.cx (David Mertz)
Date: Fri, 23 Feb 2018 22:04:45 -0800
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <CAEbHw4bajyXK2DHS_q09iAo_JcAGqpeZSBv_Wey-kru0sjQ10w@mail.gmail.com>
References: <p6n43o$qka$1@blaine.gmane.org>
 <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>
 <23183.31983.765318.187362@turnbull.sk.tsukuba.ac.jp>
 <CALGmxEJ0+24hOf2QZ=hj2D1q_QF75evJV+uZycc3mq1YBLw5iQ@mail.gmail.com>
 <CAP7+vJKQDsDW6aNsZvCp=_dxCk_1s6svJXKofNV-7u5rO-n5AQ@mail.gmail.com>
 <CALGmxE+Jr+-8cXXSuVmUA=0WRAUxqc6GAgTvexHpVD2a5pzomQ@mail.gmail.com>
 <CAP7+vJKQuqfWnsYOV68Qa3sFv7zy8A__L9LWT4gogzUqjGwkYA@mail.gmail.com>
 <CALGmxE+kJ-JctOa2U3ALW_-LrevGfby+ndqmrNv2kCy_wAt3nA@mail.gmail.com>
 <20180224052016.GK10142@ando.pearwood.info>
 <CAEbHw4bajyXK2DHS_q09iAo_JcAGqpeZSBv_Wey-kru0sjQ10w@mail.gmail.com>
Message-ID: <CAEbHw4aDghogj2yZ5SdbPeHWazo+-BBRj62h9gh-2SR0c-3Ugg@mail.gmail.com>

On Feb 23, 2018 9:26 PM, "Steven D'Aprano" <steve at pearwood.info> wrote:

Given a potentially expensive DRY violation like:

    [(function(x), function(x)+1) for x in sequence]

there are at least five ways to solve it.


A 6th way is to wrap the expensive function in @lru_cache() to make it
non-expensive.


    [(a, a+1) for x in sequence for a in [function(x)]]


It's funny to me how many people, even the BDFL, have said this is tricky
to reason about or recognize. I didn't think of it all by myself, but saw
it somewhere years ago. It seemed obvious once I saw it. Since then it's
something I do occasionally in my code without much need for thought.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180223/b7f47519/attachment.html>

From ncoghlan at gmail.com  Sat Feb 24 01:08:35 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 24 Feb 2018 16:08:35 +1000
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <p6pun0$7bi$1@blaine.gmane.org>
References: <p6n43o$qka$1@blaine.gmane.org>
 <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>
 <23183.31983.765318.187362@turnbull.sk.tsukuba.ac.jp>
 <CALGmxEJ0+24hOf2QZ=hj2D1q_QF75evJV+uZycc3mq1YBLw5iQ@mail.gmail.com>
 <CAP7+vJKQDsDW6aNsZvCp=_dxCk_1s6svJXKofNV-7u5rO-n5AQ@mail.gmail.com>
 <CALGmxE+Jr+-8cXXSuVmUA=0WRAUxqc6GAgTvexHpVD2a5pzomQ@mail.gmail.com>
 <CAP7+vJKQuqfWnsYOV68Qa3sFv7zy8A__L9LWT4gogzUqjGwkYA@mail.gmail.com>
 <CALGmxE+kJ-JctOa2U3ALW_-LrevGfby+ndqmrNv2kCy_wAt3nA@mail.gmail.com>
 <p6pun0$7bi$1@blaine.gmane.org>
Message-ID: <CADiSq7cbkZosZJ0HR924Ueihuzg0xJ1xXxXM_Lrtc4CN7WuSBA@mail.gmail.com>

On 24 February 2018 at 06:50, Stefan Behnel <stefan_ml at behnel.de> wrote:

> But in general, yes, changing a list iterable into a tuple is an
> improvement as tuples are more efficient to allocate. Haven't tried it in
> CPython (*), but it might make a slight difference for very short
> iterables, which are probably common.


CPython has included the list->tuple conversion for lists of literals for
quite some time, and Serhiy just posted a patch to extend that to all
inline lists where it's a safe change to make:
https://bugs.python.org/issue32925

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180224/76a8d650/attachment.html>

From mertz at gnosis.cx  Sat Feb 24 01:07:33 2018
From: mertz at gnosis.cx (David Mertz)
Date: Fri, 23 Feb 2018 22:07:33 -0800
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <CAEbHw4ZFe4UsXHDRePacqHgsG97wfLos_mtmYGhyEdyHNPHsQg@mail.gmail.com>
References: <p6n43o$qka$1@blaine.gmane.org>
 <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>
 <23183.31983.765318.187362@turnbull.sk.tsukuba.ac.jp>
 <CALGmxEJ0+24hOf2QZ=hj2D1q_QF75evJV+uZycc3mq1YBLw5iQ@mail.gmail.com>
 <CAP7+vJKQDsDW6aNsZvCp=_dxCk_1s6svJXKofNV-7u5rO-n5AQ@mail.gmail.com>
 <CALGmxE+Jr+-8cXXSuVmUA=0WRAUxqc6GAgTvexHpVD2a5pzomQ@mail.gmail.com>
 <CAP7+vJKQuqfWnsYOV68Qa3sFv7zy8A__L9LWT4gogzUqjGwkYA@mail.gmail.com>
 <CALGmxE+kJ-JctOa2U3ALW_-LrevGfby+ndqmrNv2kCy_wAt3nA@mail.gmail.com>
 <20180224052016.GK10142@ando.pearwood.info>
 <CAEbHw4bajyXK2DHS_q09iAo_JcAGqpeZSBv_Wey-kru0sjQ10w@mail.gmail.com>
 <CAEbHw4aDghogj2yZ5SdbPeHWazo+-BBRj62h9gh-2SR0c-3Ugg@mail.gmail.com>
 <CAEbHw4ZFe4UsXHDRePacqHgsG97wfLos_mtmYGhyEdyHNPHsQg@mail.gmail.com>
Message-ID: <CAEbHw4bCbjzaZYR-P_GQ60dPeafMqtY2Sk8CmXLUO5xYH2mdkg@mail.gmail.com>

FWIW, the nested loop over a single item is already in the language for 15
years or something. It's not that ugly, certainly not enough to need a new
'let' or 'where' keyword that basically does exactly the same thing with 3
fewer characters.

On Feb 23, 2018 10:04 PM, David Mertz <mertz at gnosis.cx> wrote:


On Feb 23, 2018 9:26 PM, "Steven D'Aprano" <steve at pearwood.info> wrote:

Given a potentially expensive DRY violation like:

    [(function(x), function(x)+1) for x in sequence]

there are at least five ways to solve it.


A 6th way is to wrap the expensive function in @lru_cache() to make it
non-expensive.


    [(a, a+1) for x in sequence for a in [function(x)]]


It's funny to me how many people, even the BDFL, have said this is tricky
to reason about or recognize. I didn't think of it all by myself, but saw
it somewhere years ago. It seemed obvious once I saw it. Since then it's
something I do occasionally in my code without much need for thought.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180223/701e87b3/attachment-0001.html>

From barry at python.org  Sat Feb 24 01:45:13 2018
From: barry at python.org (Barry Warsaw)
Date: Fri, 23 Feb 2018 22:45:13 -0800
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAGGBd_r+LkkYqSMwBvjq69hiqo5gky7hk5htnsDemzHKyMfcXg@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <CAGGBd_r+LkkYqSMwBvjq69hiqo5gky7hk5htnsDemzHKyMfcXg@mail.gmail.com>
Message-ID: <D1F7B9B8-8586-4335-8C18-854DF10D3ADE@python.org>

On Feb 21, 2018, at 19:03, Dan Stromberg <drsalists at gmail.com> wrote:
> 
> Is flake8 that much better than pylint, that pylint wouldn't even be discussed?

I honestly don?t use pylint all that much these days, but when I was evaluating them years ago, I found pylint to be too noisy about not-incorrect code.  I also liked how flake8 has a very nice plugin system, and I use that on my personal projects to enforce a consistent style.  It?s certainly possible that both tools have advanced significantly since I last made my personal decision.

I agree with Guido in general though, that we?d need to do some actual analysis on the stdlib and see what the results are.  I do think that a plugin system, or really good configurability, will help us tailor whatever tool we decide to use, so that it?s only warning about the things we care about and ignoring the rest, without requiring lots of comment pragmas in the code.

Cheers,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180223/9aaa248c/attachment.sig>

From barry at python.org  Sat Feb 24 01:50:45 2018
From: barry at python.org (Barry Warsaw)
Date: Fri, 23 Feb 2018 22:50:45 -0800
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <20180222111239.4814f10e@fsol>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <20180222111239.4814f10e@fsol>
Message-ID: <A93E4703-0041-43CC-A454-D834969BC35C@python.org>

On Feb 22, 2018, at 02:12, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Everytime I contribute to a project which has automatic style checks in
> CI, I find myself having to push pointless "cleanup" commits because
> the CI is barking at me for some ridiculous reason (such as putting two
> linefeeds instead of one between two Python functions).  Then I have to
> wait some more until CI finishes, especially if builds take long or
> build queues are clogged.
> 
> Overall it makes contributing more of a PITA than it needs be.  Do
> automatic style *fixes* if you want, but please don't annoy me with
> automatic style checks that ask me to do tedious grunt work on my spare
> time.

Look at it from the other side though.  If I?m reviewing someone?s PR, and the style is so inconsistent with the existing code base, then I?m distracted by constantly asking the contributor to clean up little stylistic things.  This wastes my time as a reviewer, where I?d rather be thinking about the substantive content of the change.  To have no style checks can leave you with a chaotic jumbled ness that is much harder to navigate around, understand, and contribute to by others, and once you?ve gone down that path, you just build up technical debt to eventually try to clean it up.

Cheers,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180223/ee61cef2/attachment.sig>

From barry at python.org  Sat Feb 24 01:53:40 2018
From: barry at python.org (Barry Warsaw)
Date: Fri, 23 Feb 2018 22:53:40 -0800
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <20180222170815.55773b4a@fsol>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <20180222111239.4814f10e@fsol> <5A8EE3EB.5040705@stoneleaf.us>
 <3znJmZ4h2yzFqxh@mail.python.org> <20180222170815.55773b4a@fsol>
Message-ID: <131790A1-46E2-40F3-B1AC-7E9F52F9D852@python.org>


> On Feb 22, 2018, at 08:08, Antoine Pitrou <solipsis at pitrou.net> wrote:
> 
> That's a fair point I hadn't considered.  OTOH the style issues I
> usually comment on as a reviewer aren't the kind that would be caught
> by an automated style check (I tend to ask for comments or docstrings,
> or be nitpicky about some variable or function name).  YMMV :-)

Those are aesthetic comments that are important, and are also difficult to automate.  What I really don?t want to comment on are things like whitespace (vertical, horizontal, trailing), import order or format, consistent indentation, etc.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180223/1213bd4d/attachment.sig>

From barry at python.org  Sat Feb 24 01:56:32 2018
From: barry at python.org (Barry Warsaw)
Date: Fri, 23 Feb 2018 22:56:32 -0800
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CALFfu7AOYpFSU+VmCTjaY-44iUTdGxUM01Tv2ph1gLRKRpLYuA@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <20180222111239.4814f10e@fsol> <5A8EE3EB.5040705@stoneleaf.us>
 <3znJmZ4h2yzFqxh@mail.python.org> <20180222170815.55773b4a@fsol>
 <CACac1F8JQ2Wmrs42Bp6usAqZoVn5cPoehYKHyyGJ21fn7_OYbg@mail.gmail.com>
 <CALFfu7AOYpFSU+VmCTjaY-44iUTdGxUM01Tv2ph1gLRKRpLYuA@mail.gmail.com>
Message-ID: <50484F14-78FF-43F7-9A6B-4D8A1DAF2587@python.org>

On Feb 22, 2018, at 09:09, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> 
> I had exactly that experience on one particularly large Go project (on
> GitHub, with slow CI, driven by bots).

One thing that?s nice to set up if you can is multiple, parallel, independent CI runs.  E.g. if your full test suite takes a long time to run, but you can run your code style tests pretty quickly (e.g. flake8 and mypy), then you can get more immediate feedback about your PR code style, and it?s not so painful if your test suite is slow.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180223/eb70386f/attachment.sig>

From ncoghlan at gmail.com  Sat Feb 24 07:09:38 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 24 Feb 2018 22:09:38 +1000
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <D1F7B9B8-8586-4335-8C18-854DF10D3ADE@python.org>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <CAGGBd_r+LkkYqSMwBvjq69hiqo5gky7hk5htnsDemzHKyMfcXg@mail.gmail.com>
 <D1F7B9B8-8586-4335-8C18-854DF10D3ADE@python.org>
Message-ID: <CADiSq7ci8sCEG0ks7Bk8_AmByxVB5C4UWMG2FB_Xzg-CAoz2Eg@mail.gmail.com>

On 24 February 2018 at 16:45, Barry Warsaw <barry at python.org> wrote:

> On Feb 21, 2018, at 19:03, Dan Stromberg <drsalists at gmail.com> wrote:
> >
> > Is flake8 that much better than pylint, that pylint wouldn't even be
> discussed?
>
> I honestly don?t use pylint all that much these days, but when I was
> evaluating them years ago, I found pylint to be too noisy about
> not-incorrect code.  I also liked how flake8 has a very nice plugin system,
> and I use that on my personal projects to enforce a consistent style.  It?s
> certainly possible that both tools have advanced significantly since I last
> made my personal decision.
>

When I recommend pylint, I try to make it clear that the command I
personally recommend is `pylint -E`, as that's the level that automates
checking for outright bugs (name errors, attribute errors, etc), as well as
effectively checking for clever dynamic variable and attribute injection
code that is likely to confuse human readers in addition to confusing
static analysers.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180224/111fcf35/attachment.html>

From python-dev at mgmiller.net  Sat Feb 24 12:33:54 2018
From: python-dev at mgmiller.net (Mike Miller)
Date: Sat, 24 Feb 2018 09:33:54 -0800
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <CAEbHw4bCbjzaZYR-P_GQ60dPeafMqtY2Sk8CmXLUO5xYH2mdkg@mail.gmail.com>
References: <p6n43o$qka$1@blaine.gmane.org>
 <244D7741-5A8F-446C-B668-E6D01A74679B@python.org>
 <23183.31983.765318.187362@turnbull.sk.tsukuba.ac.jp>
 <CALGmxEJ0+24hOf2QZ=hj2D1q_QF75evJV+uZycc3mq1YBLw5iQ@mail.gmail.com>
 <CAP7+vJKQDsDW6aNsZvCp=_dxCk_1s6svJXKofNV-7u5rO-n5AQ@mail.gmail.com>
 <CALGmxE+Jr+-8cXXSuVmUA=0WRAUxqc6GAgTvexHpVD2a5pzomQ@mail.gmail.com>
 <CAP7+vJKQuqfWnsYOV68Qa3sFv7zy8A__L9LWT4gogzUqjGwkYA@mail.gmail.com>
 <CALGmxE+kJ-JctOa2U3ALW_-LrevGfby+ndqmrNv2kCy_wAt3nA@mail.gmail.com>
 <20180224052016.GK10142@ando.pearwood.info>
 <CAEbHw4bajyXK2DHS_q09iAo_JcAGqpeZSBv_Wey-kru0sjQ10w@mail.gmail.com>
 <CAEbHw4aDghogj2yZ5SdbPeHWazo+-BBRj62h9gh-2SR0c-3Ugg@mail.gmail.com>
 <CAEbHw4ZFe4UsXHDRePacqHgsG97wfLos_mtmYGhyEdyHNPHsQg@mail.gmail.com>
 <CAEbHw4bCbjzaZYR-P_GQ60dPeafMqtY2Sk8CmXLUO5xYH2mdkg@mail.gmail.com>
Message-ID: <8e0e109c-8b6b-c39f-2536-e8044bfc9d9d@mgmiller.net>

I'm not sure, I found the "with f(x) as y" form previously mentioned the most 
readable despite it being a new use case, while not needing new keywords.

-Mike


On 2018-02-23 22:07, David Mertz wrote:
> FWIW, the nested loop over a single item is already in the language for 15 years 
> or something. It's not that ugly, certainly not enough to need a new 'let' or 
> 'where' keyword that basically does exactly the same thing with 3 fewer characters.
> 
> On Feb 23, 2018 10:04 PM, David Mertz <mertz at gnosis.cx <mailto:mertz at gnosis.cx>> 
> wrote:


From mariatta.wijaya at gmail.com  Sat Feb 24 22:35:54 2018
From: mariatta.wijaya at gmail.com (Mariatta Wijaya)
Date: Sat, 24 Feb 2018 19:35:54 -0800
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CADiSq7ci8sCEG0ks7Bk8_AmByxVB5C4UWMG2FB_Xzg-CAoz2Eg@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <CAGGBd_r+LkkYqSMwBvjq69hiqo5gky7hk5htnsDemzHKyMfcXg@mail.gmail.com>
 <D1F7B9B8-8586-4335-8C18-854DF10D3ADE@python.org>
 <CADiSq7ci8sCEG0ks7Bk8_AmByxVB5C4UWMG2FB_Xzg-CAoz2Eg@mail.gmail.com>
Message-ID: <CAGbohnYLFX5FD-w+KQ=2vCTzi_ghuPfXzCRMZ9c76HyL7Py74A@mail.gmail.com>

Can any of these said linters analyze only the diff in the PR, instead of
the entire CPython codebase?

Mariatta Wijaya

?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180224/9f5e8ce7/attachment.html>

From guido at python.org  Sun Feb 25 00:13:07 2018
From: guido at python.org (Guido van Rossum)
Date: Sat, 24 Feb 2018 21:13:07 -0800
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAGbohnYLFX5FD-w+KQ=2vCTzi_ghuPfXzCRMZ9c76HyL7Py74A@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <CAGGBd_r+LkkYqSMwBvjq69hiqo5gky7hk5htnsDemzHKyMfcXg@mail.gmail.com>
 <D1F7B9B8-8586-4335-8C18-854DF10D3ADE@python.org>
 <CADiSq7ci8sCEG0ks7Bk8_AmByxVB5C4UWMG2FB_Xzg-CAoz2Eg@mail.gmail.com>
 <CAGbohnYLFX5FD-w+KQ=2vCTzi_ghuPfXzCRMZ9c76HyL7Py74A@mail.gmail.com>
Message-ID: <CAP7+vJK_myg8sbs-G4BzqdH9ytyFHSyA05FTPOuZ=Sv1k8EhOA@mail.gmail.com>

It's easy to only analyze the files in the diff (these linters don't do
cross-file analysis anyways, typically) and it's possible to write a filter
that only keeps warnings about lines that are changed, but I don't know of
a standard solution for the latter (places where I worked where I've seen
this always had their own custom implementation).

On Sat, Feb 24, 2018 at 7:35 PM, Mariatta Wijaya <mariatta.wijaya at gmail.com>
wrote:

> Can any of these said linters analyze only the diff in the PR, instead of
> the entire CPython codebase?
>
> Mariatta Wijaya
>
> ?
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180224/ec1823eb/attachment.html>

From Nikolaus at rath.org  Sun Feb 25 07:02:26 2018
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Sun, 25 Feb 2018 12:02:26 +0000
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <p6n43o$qka$1@blaine.gmane.org> (Serhiy Storchaka's message of
 "Thu, 22 Feb 2018 21:04:24 +0200")
References: <p6n43o$qka$1@blaine.gmane.org>
Message-ID: <87o9kdcm5p.fsf@vostro.rath.org>

On Feb 22 2018, Serhiy Storchaka <storchaka at gmail.com> wrote:
> 1. Inner generator expression:
>
>     result = [y + g(y) for y in (f(x) for x in range(10))]
>
[...]
>
> And maybe there are other ways.

I think the syntax recently brough up by Nick is still the most
beautiful:

    result = [ (f(x) as y) + g(y) for x in range(10)]

..but I wonder if it is feasible to make the interpreter sufficiently
smart to evaluate the first summand before the second.

Best,
-Nikolaus

-- 
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             ?Time flies like an arrow, fruit flies like a Banana.?

From rosuav at gmail.com  Sun Feb 25 07:46:28 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 25 Feb 2018 23:46:28 +1100
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <87o9kdcm5p.fsf@vostro.rath.org>
References: <p6n43o$qka$1@blaine.gmane.org> <87o9kdcm5p.fsf@vostro.rath.org>
Message-ID: <CAPTjJmoM9PSL+QZBiX3YE2t9XrBF4HAxZtbMpaP9eEAJumeWSw@mail.gmail.com>

On Sun, Feb 25, 2018 at 11:02 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> On Feb 22 2018, Serhiy Storchaka <storchaka at gmail.com> wrote:
>> 1. Inner generator expression:
>>
>>     result = [y + g(y) for y in (f(x) for x in range(10))]
>>
> [...]
>>
>> And maybe there are other ways.
>
> I think the syntax recently brough up by Nick is still the most
> beautiful:
>
>     result = [ (f(x) as y) + g(y) for x in range(10)]
>
> ..but I wonder if it is feasible to make the interpreter sufficiently
> smart to evaluate the first summand before the second.

It already has to. The order of evaluation in Python is well defined,
mostly "left to right". But if you allow this in a comprehension, the
obvious next step will be "do we allow this in ANY expression?", and
the answer has to either be "yes" or "no, because {reasons}" for some
very good value of 'reasons'.

ChrisA

From Nikolaus at rath.org  Sun Feb 25 08:11:05 2018
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Sun, 25 Feb 2018 13:11:05 +0000
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <CAPTjJmoM9PSL+QZBiX3YE2t9XrBF4HAxZtbMpaP9eEAJumeWSw@mail.gmail.com>
 (Chris Angelico's message of "Sun, 25 Feb 2018 23:46:28 +1100")
References: <p6n43o$qka$1@blaine.gmane.org> <87o9kdcm5p.fsf@vostro.rath.org>
 <CAPTjJmoM9PSL+QZBiX3YE2t9XrBF4HAxZtbMpaP9eEAJumeWSw@mail.gmail.com>
Message-ID: <87inalciza.fsf@vostro.rath.org>

On Feb 25 2018, Chris Angelico <rosuav at gmail.com> wrote:
> On Sun, Feb 25, 2018 at 11:02 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>> On Feb 22 2018, Serhiy Storchaka <storchaka at gmail.com> wrote:
>>> 1. Inner generator expression:
>>>
>>>     result = [y + g(y) for y in (f(x) for x in range(10))]
>>>
>> [...]
>>>
>>> And maybe there are other ways.
>>
>> I think the syntax recently brough up by Nick is still the most
>> beautiful:
>>
>>     result = [ (f(x) as y) + g(y) for x in range(10)]
>>
>> ..but I wonder if it is feasible to make the interpreter sufficiently
>> smart to evaluate the first summand before the second.
>
> It already has to. The order of evaluation in Python is well defined,
> mostly "left to right".

Ah, then the problem is how to evaluate

    result = [ y + g(f(x) as y) for x in range(10)]

I don't think there'd be a good reason to allow one but not the other.

> But if you allow this in a comprehension, the
> obvious next step will be "do we allow this in ANY expression?"

Yes, of course. After all, IIRC Nick proposed it to simplify ternary
expressions.


Best,
-Nikolaus

-- 
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             ?Time flies like an arrow, fruit flies like a Banana.?

From desmoulinmichel at gmail.com  Sun Feb 25 09:32:36 2018
From: desmoulinmichel at gmail.com (Michel Desmoulin)
Date: Sun, 25 Feb 2018 15:32:36 +0100
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <87inalciza.fsf@vostro.rath.org>
References: <p6n43o$qka$1@blaine.gmane.org> <87o9kdcm5p.fsf@vostro.rath.org>
 <CAPTjJmoM9PSL+QZBiX3YE2t9XrBF4HAxZtbMpaP9eEAJumeWSw@mail.gmail.com>
 <87inalciza.fsf@vostro.rath.org>
Message-ID: <00bc90f6-8bf4-f6da-6c54-035094b5262f@gmail.com>



Le 25/02/2018 ? 14:11, Nikolaus Rath a ?crit?:
> On Feb 25 2018, Chris Angelico <rosuav at gmail.com> wrote:
>> On Sun, Feb 25, 2018 at 11:02 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>>> On Feb 22 2018, Serhiy Storchaka <storchaka at gmail.com> wrote:
>>>> 1. Inner generator expression:
>>>>
>>>>     result = [y + g(y) for y in (f(x) for x in range(10))]
>>>>
>>> [...]
>>>>


>>>> And maybe there are other ways.
>>>
>>> I think the syntax recently brough up by Nick is still the most
>>> beautiful:
>>>
>>>     result = [ (f(x) as y) + g(y) for x in range(10)]

Honestly I find this version the most readable while the double for loop
is completely weird to me, despite doing python for a living for years.

I really hope the later doesn't become a common idiom.

From storchaka at gmail.com  Sun Feb 25 09:36:16 2018
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sun, 25 Feb 2018 16:36:16 +0200
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <CAP7+vJK+fDag27ua29_U2yMcirk0aecM4ND0TDd=e7v1RQWjOg@mail.gmail.com>
References: <p6n43o$qka$1@blaine.gmane.org>
 <CAP7+vJK+fDag27ua29_U2yMcirk0aecM4ND0TDd=e7v1RQWjOg@mail.gmail.com>
Message-ID: <p6uhh3$g9e$1@blaine.gmane.org>

23.02.18 19:30, Guido van Rossum ????:
> I'm not saying anything new here, but since you asked specifically for 
> my opinion: I don't care for the idiom; it's never occurred to me 
> before, and it smells of cleverness. If I saw it in a code review I 
> would probably ask for a regular for-loop to make the code more 
> maintainable.
> 
> But if you say it's useful for some class of users and it would be more 
> useful if it was faster, I'm fine with the optimization. The 
> optimization is also clever, and here I appreciate cleverness!

Thank you. Given the contradictory relation of other core developers to 
this idiom, and small total effect of this optimization (since the 
problem solved by using this idiom is rarely occurred), I'm inclined to 
defer this optimization on to some time (months or years). Maybe 
something will be changed during this period: either this idiom will 
become more popular, or new arguments against using it will be found, or 
better solution will be found, or this optimization will become the part 
of more general optimization.


From rosuav at gmail.com  Sun Feb 25 10:08:49 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 26 Feb 2018 02:08:49 +1100
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <87inalciza.fsf@vostro.rath.org>
References: <p6n43o$qka$1@blaine.gmane.org> <87o9kdcm5p.fsf@vostro.rath.org>
 <CAPTjJmoM9PSL+QZBiX3YE2t9XrBF4HAxZtbMpaP9eEAJumeWSw@mail.gmail.com>
 <87inalciza.fsf@vostro.rath.org>
Message-ID: <CAPTjJmrT1B=rpSysLvHNr5ofvrc0N4JgT8NSV9YrzQ9sEBfRNA@mail.gmail.com>

On Mon, Feb 26, 2018 at 12:11 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> On Feb 25 2018, Chris Angelico <rosuav at gmail.com> wrote:
>> On Sun, Feb 25, 2018 at 11:02 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>>> On Feb 22 2018, Serhiy Storchaka <storchaka at gmail.com> wrote:
>>>> 1. Inner generator expression:
>>>>
>>>>     result = [y + g(y) for y in (f(x) for x in range(10))]
>>>>
>>> [...]
>>>>
>>>> And maybe there are other ways.
>>>
>>> I think the syntax recently brough up by Nick is still the most
>>> beautiful:
>>>
>>>     result = [ (f(x) as y) + g(y) for x in range(10)]
>>>
>>> ..but I wonder if it is feasible to make the interpreter sufficiently
>>> smart to evaluate the first summand before the second.
>>
>> It already has to. The order of evaluation in Python is well defined,
>> mostly "left to right".
>
> Ah, then the problem is how to evaluate
>
>     result = [ y + g(f(x) as y) for x in range(10)]
>

That ought to raise UnboundLocalError, since y is evaluated before g's
arguments. If you're reusing an expression, it isn't too much hassle
to demand that the *first* instance of that expression be the one with
the 'as' clause. Generally "first" means "leftmost", with rare
exceptions (eg it'd be "y if (expr as y) else y" with the assignment
in the middle), so that shouldn't bother most people.

>> But if you allow this in a comprehension, the
>> obvious next step will be "do we allow this in ANY expression?"
>
> Yes, of course. After all, IIRC Nick proposed it to simplify ternary
> expressions.

The trouble with allowing 'expr as name' in any context is that it's
pretty much guaranteed to create confusion in a 'with' statement.
Compare:

with open(fn) as f:
with (open(fn) as f):
with contextlib.closing(open(fn)) as f:
with (contextlib.closing(open(fn)) as f):

Do they all do the same thing? Can you see at a glance which one is
different, and *how* it's different?

And I'm sure there are other situations where it would be similarly
confusing, yet still potentially useful. Does this just get filed
under "consenting adults"?

Speaking as a C programmer who's quite happy to write code like "while
((var = func()) != sentinel)", I wouldn't object to this coming up in
Python; the "as name" syntax has the huge advantage over C's syntax in
that you can't accidentally leave off one equals sign and get the
wrong behaviour. But I know that a lot of people dislike this at a
more fundamental level.

If someone wants to push for this, I think it probably needs a PEP -
it's a point that comes up periodically. I don't think it's ever had a
PEP written about it, but it's a bit hard to search for; maybe someone
else knows off hand?

ChrisA

From guido at python.org  Sun Feb 25 12:19:07 2018
From: guido at python.org (Guido van Rossum)
Date: Sun, 25 Feb 2018 09:19:07 -0800
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <p6uhh3$g9e$1@blaine.gmane.org>
References: <p6n43o$qka$1@blaine.gmane.org>
 <CAP7+vJK+fDag27ua29_U2yMcirk0aecM4ND0TDd=e7v1RQWjOg@mail.gmail.com>
 <p6uhh3$g9e$1@blaine.gmane.org>
Message-ID: <CAP7+vJJseRh8T4YZfX4PXSeKPZWj9Lgnj55xKbRWYpg5SNFFTQ@mail.gmail.com>

On Sun, Feb 25, 2018 at 6:36 AM, Serhiy Storchaka <storchaka at gmail.com>
wrote:

> 23.02.18 19:30, Guido van Rossum ????:
>
>> I'm not saying anything new here, but since you asked specifically for my
>> opinion: I don't care for the idiom; it's never occurred to me before, and
>> it smells of cleverness. If I saw it in a code review I would probably ask
>> for a regular for-loop to make the code more maintainable.
>>
>> But if you say it's useful for some class of users and it would be more
>> useful if it was faster, I'm fine with the optimization. The optimization
>> is also clever, and here I appreciate cleverness!
>>
>
> Thank you. Given the contradictory relation of other core developers to
> this idiom, and small total effect of this optimization (since the problem
> solved by using this idiom is rarely occurred), I'm inclined to defer this
> optimization on to some time (months or years). Maybe something will be
> changed during this period: either this idiom will become more popular, or
> new arguments against using it will be found, or better solution will be
> found, or this optimization will become the part of more general
> optimization.
>

Yeah, it doesn't seem there's any hurry. Opinions on the idiom are
definitely, um, divided. :-)

FWIW I don't care much about the 'f(x) as y' solution either, and being new
syntax it has a much higher bar.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180225/e7c32b96/attachment.html>

From eric at trueblade.com  Sun Feb 25 13:49:27 2018
From: eric at trueblade.com (Eric V. Smith)
Date: Sun, 25 Feb 2018 13:49:27 -0500
Subject: [Python-Dev] [Python-checkins] Exhaustively test dataclass
 hashing when no hash= value is provided. This is in anticipation of
 changing how non-default hashing is handled. (GH-5834) (GH-5889)
In-Reply-To: <74f40a59-0c4a-3e76-8506-16b79394039c@udel.edu>
References: <3zqB222XPWzFrJk@mail.python.org>
 <74f40a59-0c4a-3e76-8506-16b79394039c@udel.edu>
Message-ID: <045c7e1b-df86-0128-5b3b-e5811ab7a25c@trueblade.com>

Thanks. Do you know if this gets backported to 3.7?

Eric.

On 2/25/2018 1:36 PM, Terry Reedy wrote:
> On 2/25/2018 11:56 AM, Eric V. Smith wrote:
> 
>> +??????? # specify any value in the deecorator).
> 
> I fixed this through the web interface.? The CI now knows that a comment 
> change does not need testing.? I could not request a review from 
> ericvsmith though.
> 
> tjr
> 
> 
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> https://mail.python.org/mailman/listinfo/python-checkins

From nad at python.org  Sun Feb 25 15:16:59 2018
From: nad at python.org (Ned Deily)
Date: Sun, 25 Feb 2018 15:16:59 -0500
Subject: [Python-Dev] 3.7.0b2 code cutoff soon!
Message-ID: <B7FB88EE-0F7C-47CB-BD65-1EE4AC326BA8@python.org>

Just a reminder that 3.7.0b2 is almost upon us. Please get your feature
fixes, bug fixes, and documentation updates in before 2018-02-26 ~23:59
Anywhere on Earth (UTC-12:00). That's a little over 1.5 days from now.
Also, as previously noted, for those of you who asked for and received
extensions for a few remaining 3.7.0 features, those extensions expire
as of the b2 cutoff so please plan accordingly.

Looking ahead, we need to start locking down 3.7.0 so that our
downstream users, that is, third-party package developers, Python
distributors, and end users, can test their code with confidence that
the actual release of 3.7.0 will hold no unpleasant surprises. So please
assume that the 3.7.0 ABI will be frozen as of beta 3, in 4 weeks on
2018-03-26, and that only doc updates and the kinds of bug fixes
appropriate for a maintenance release should be going into the 3.7
branch after 3.7.0b3 without further discussion.

Thanks again for all of your hard work towards making 3.7.0 yet another
great release!

--Ned

--
  Ned Deily
  nad at python.org -- []


From ncoghlan at gmail.com  Mon Feb 26 04:00:39 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 26 Feb 2018 19:00:39 +1000
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <CAPTjJmrT1B=rpSysLvHNr5ofvrc0N4JgT8NSV9YrzQ9sEBfRNA@mail.gmail.com>
References: <p6n43o$qka$1@blaine.gmane.org> <87o9kdcm5p.fsf@vostro.rath.org>
 <CAPTjJmoM9PSL+QZBiX3YE2t9XrBF4HAxZtbMpaP9eEAJumeWSw@mail.gmail.com>
 <87inalciza.fsf@vostro.rath.org>
 <CAPTjJmrT1B=rpSysLvHNr5ofvrc0N4JgT8NSV9YrzQ9sEBfRNA@mail.gmail.com>
Message-ID: <CADiSq7doqHqiLS4z5nQSTozBxiZJZS0CDygBGeHmsi-E681y9w@mail.gmail.com>

On 26 February 2018 at 01:08, Chris Angelico <rosuav at gmail.com> wrote:

> Speaking as a C programmer who's quite happy to write code like "while
> ((var = func()) != sentinel)", I wouldn't object to this coming up in
> Python; the "as name" syntax has the huge advantage over C's syntax in
> that you can't accidentally leave off one equals sign and get the
> wrong behaviour. But I know that a lot of people dislike this at a
> more fundamental level.
>
> If someone wants to push for this, I think it probably needs a PEP -
> it's a point that comes up periodically. I don't think it's ever had a
> PEP written about it, but it's a bit hard to search for; maybe someone
> else knows off hand?
>

PEP 3150 is the most closely related PEP we have at the moment (and that
only works for simple statements, since it relies on using a trailing block
to name the subexpressions).

The "(EXPR as NAME)" syntax comes up periodically, especially in the
context of while loops (where it would allow a direct translation of
C-style embedded assignment idioms).

In addition to the potential confusion for "with (EXPR as NAME):" vs "with
EXPR as NAME:" (and the similar ambiguity for "except" clauses), some other
major questions to be resolved are:

* are statement locals in a class namespace turned into attributes on the
resulting class? (it would be more useful if they weren't)
* are statement locals in a module namespace turned into attributes on the
resulting module? (it would be more useful if they weren't)
* are statement locals in a function/generator/coroutine namespace kept
alive until the entire call terminates? (it would be more useful if they
weren't)
* do currently defined statement locals appear in calls to locals() or in
frame.f_locals?
* can lexically nested scopes see names bound this way? (a lot of complex
name resolution problems disappear if they can't, plus you get a clearer
distinction between these and regular function locals)

To be interesting enough to potentially be worthy of syntax, I think name
bindings written this way would need to be truly statement local:

* reference is released at the end of the statement (whether simple or
compound)
* no ability to close over them (this goes hand in hand with eagerly
dropping the reference)
* we play name mangling games and/or use different opcodes to avoid
overwriting regular function locals and to avoid appearing in locals()

That's still only enough to get the concept into python-ideas territory
though (as per the discussion of "(EXPR as .NAME)" in
https://mail.python.org/pipermail/python-ideas/2018-February/049002.html) -
it's still a *long* way from being fully baked enough to make into a
concrete change proposal for 3.8+.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180226/9ed6216b/attachment.html>

From rosuav at gmail.com  Mon Feb 26 04:05:19 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 26 Feb 2018 20:05:19 +1100
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <CADiSq7doqHqiLS4z5nQSTozBxiZJZS0CDygBGeHmsi-E681y9w@mail.gmail.com>
References: <p6n43o$qka$1@blaine.gmane.org> <87o9kdcm5p.fsf@vostro.rath.org>
 <CAPTjJmoM9PSL+QZBiX3YE2t9XrBF4HAxZtbMpaP9eEAJumeWSw@mail.gmail.com>
 <87inalciza.fsf@vostro.rath.org>
 <CAPTjJmrT1B=rpSysLvHNr5ofvrc0N4JgT8NSV9YrzQ9sEBfRNA@mail.gmail.com>
 <CADiSq7doqHqiLS4z5nQSTozBxiZJZS0CDygBGeHmsi-E681y9w@mail.gmail.com>
Message-ID: <CAPTjJmqzzLU-e75YTb-Q+Xd1aCpbxa8yM3ohQV9o10nqadwpmA@mail.gmail.com>

On Mon, Feb 26, 2018 at 8:00 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 26 February 2018 at 01:08, Chris Angelico <rosuav at gmail.com> wrote:
>>
>> Speaking as a C programmer who's quite happy to write code like "while
>> ((var = func()) != sentinel)", I wouldn't object to this coming up in
>> Python; the "as name" syntax has the huge advantage over C's syntax in
>> that you can't accidentally leave off one equals sign and get the
>> wrong behaviour. But I know that a lot of people dislike this at a
>> more fundamental level.
>>
>> If someone wants to push for this, I think it probably needs a PEP -
>> it's a point that comes up periodically. I don't think it's ever had a
>> PEP written about it, but it's a bit hard to search for; maybe someone
>> else knows off hand?
>
>
> PEP 3150 is the most closely related PEP we have at the moment (and that
> only works for simple statements, since it relies on using a trailing block
> to name the subexpressions).
>
> The "(EXPR as NAME)" syntax comes up periodically, especially in the context
> of while loops (where it would allow a direct translation of C-style
> embedded assignment idioms).
>
> In addition to the potential confusion for "with (EXPR as NAME):" vs "with
> EXPR as NAME:" (and the similar ambiguity for "except" clauses), some other
> major questions to be resolved are:
>
> * are statement locals in a class namespace turned into attributes on the
> resulting class? (it would be more useful if they weren't)
> * are statement locals in a module namespace turned into attributes on the
> resulting module? (it would be more useful if they weren't)
> * are statement locals in a function/generator/coroutine namespace kept
> alive until the entire call terminates? (it would be more useful if they
> weren't)
> * do currently defined statement locals appear in calls to locals() or in
> frame.f_locals?
> * can lexically nested scopes see names bound this way? (a lot of complex
> name resolution problems disappear if they can't, plus you get a clearer
> distinction between these and regular function locals)
>
> To be interesting enough to potentially be worthy of syntax, I think name
> bindings written this way would need to be truly statement local:
>
> * reference is released at the end of the statement (whether simple or
> compound)
> * no ability to close over them (this goes hand in hand with eagerly
> dropping the reference)
> * we play name mangling games and/or use different opcodes to avoid
> overwriting regular function locals and to avoid appearing in locals()

Definitely possible. I wish I still had the POC patch that I put
together a while ago that created block-local variables from 'as'
bindings in except and with statements, but it got lost in a hard
drive crash (because, at the time, I didn't think it was useful for
anything more than "haha, isn't that cute"). Will see if I can
recreate it.

ChrisA

From turnbull.stephen.fw at u.tsukuba.ac.jp  Mon Feb 26 04:40:27 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Mon, 26 Feb 2018 18:40:27 +0900
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <00bc90f6-8bf4-f6da-6c54-035094b5262f@gmail.com>
References: <p6n43o$qka$1@blaine.gmane.org> <87o9kdcm5p.fsf@vostro.rath.org>
 <CAPTjJmoM9PSL+QZBiX3YE2t9XrBF4HAxZtbMpaP9eEAJumeWSw@mail.gmail.com>
 <87inalciza.fsf@vostro.rath.org>
 <00bc90f6-8bf4-f6da-6c54-035094b5262f@gmail.com>
Message-ID: <23187.54795.800451.72699@turnbull.sk.tsukuba.ac.jp>

Michel Desmoulin writes:

 > Le 25/02/2018 ? 14:11, Nikolaus Rath a ?crit?:

>>>     result = [ (f(x) as y) + g(y) for x in range(10)]

 > Honestly I find this version the most readable while the double for
 > loop is completely weird to me, despite doing python for a living
 > for years.

I find this one less readable because I don't expect name binding
syntax to return a value.  My brain is nonplussed by the "+". :-)

 > I really hope the later doesn't become a common idiom.

It already is common, for values of "common" = "some people have been
using it where it's useful, but it's not useful all that often".  I
suppose it's rare because it's a less-than-readable optimization,
which is frowned on in Python programming.

Somebody counted four or five ways to perform this optimization,
including this "double for" that allows the common subexpression
optimization to be made explicit in the comprehension.  We don't need
another, not at the cost of new syntax.

If we find that we really want a C-like assignment expression that
returns the assigned value[1], I don't have an objection to that.  But
I don't personally feel a need for it.

Footnotes: 
[1]  Yes, I know that technically it's a local name binding, not an
assignment.  But until the scope of "local" is resolved, it looks,
smells, and tastes like an assignment.


From rosuav at gmail.com  Mon Feb 26 06:48:01 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 26 Feb 2018 22:48:01 +1100
Subject: [Python-Dev] Code stability and a big thank you
Message-ID: <CAPTjJmpHoZJeB1tfGKL6r-msLjAcCs5269TCU2BCb75fVyyYjQ@mail.gmail.com>

As part of the discussions on local name binding, I dug up a patch
that I sent to python-ideas back in June 2015. That's two and a half
years ago, and was based on CPython 3.5. Here's what happened when I
attempted to apply it to the current master branch (3.8):

$ patch <~/Downloads/scope_hack.patch -p1
patching file Python/compile.c
Hunk #1 succeeded at 100 (offset 3 lines).
Hunk #2 succeeded at 134 (offset 3 lines).
Hunk #3 succeeded at 793 (offset 2 lines).
Hunk #4 succeeded at 2824 (offset 432 lines).
Hunk #5 succeeded at 2871 with fuzz 2 (offset 430 lines).
Hunk #6 succeeded at 3312 (offset 402 lines).
Hunk #7 succeeded at 3330 (offset 402 lines).
Hunk #8 succeeded at 5442 with fuzz 1 (offset 831 lines).
Hunk #9 succeeded at 5457 with fuzz 2 (offset 832 lines).
Hunk #10 succeeded at 5520 with fuzz 2 (offset 823 lines).
$

And the build ran fine, the test suite didn't gain any new failures
(there's no tests for the new functionality, but it didn't break
anything else), and my original demo program worked just fine.

Thank you to all the CPython core devs for maintaining code that is
THIS stable. Stuff happens but there's no unnecessary code churn.
Sure, GNU Patch had to do a bit of work to merge in the changes, but
it did it completely automatically. You folks are awesome!

ChrisA

From eric at trueblade.com  Mon Feb 26 08:40:55 2018
From: eric at trueblade.com (Eric V. Smith)
Date: Mon, 26 Feb 2018 08:40:55 -0500
Subject: [Python-Dev] Should the dataclass frozen property apply to
 subclasses?
In-Reply-To: <1a6d1aa6-60e1-58db-ef2d-672c7f6c6c49@trueblade.com>
References: <A27F9EB7-D3BA-4358-B23E-A4EFAD7088AB@gmail.com>
 <1a6d1aa6-60e1-58db-ef2d-672c7f6c6c49@trueblade.com>
Message-ID: <c683dc16-7ff4-c031-a20f-c9b3a6ec1c89@trueblade.com>

I've opened https://bugs.python.org/issue32953 to track this.

On 2/22/18 5:55 AM, Eric V. Smith wrote:
> On 2/22/2018 1:56 AM, Raymond Hettinger wrote:
>> When working on the docs for dataclasses, something unexpected came
>> up.  If a dataclass is specified to be frozen, that characteristic is
>> inherited by subclasses which prevents them from assigning additional
>> attributes:
>>
>>      >>> @dataclass(frozen=True)
>>      class D:
>>              x: int = 10
>>
>>      >>> class S(D):
>>              pass
>>
>>      >>> s = S()
>>      >>> s.cached = True
>>      Traceback (most recent call last):
>>        File "<pyshell#49>", line 1, in <module>
>>          s.cached = True
>>        File
>> "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/dataclasses.py",
>> line 448, in _frozen_setattr
>>          raise FrozenInstanceError(f'cannot assign to field {name!r}')
>>      dataclasses.FrozenInstanceError: cannot assign to field 'cached'
>
> This is because "frozen-ness" is implemented by adding __setattr__ and
> __delattr__ methods in D, which get inherited by S.
>

...

> A related issue is that dataclasses derived from frozen dataclasses are
> automatically "promoted" to being frozen.
>
>>>> @dataclass(frozen=True)
> ... class A:
> ...     i: int
> ...
>>>> @dataclass
> ... class B(A):
> ...     j: int
> ...
>>>> b = B(1, 2)
>>>> b.j = 3
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "C:\home\eric\local\python\cpython\lib\dataclasses.py", line 452,
> in _frozen_setattr
>     raise FrozenInstanceError(f'cannot assign to field {name!r}')
> dataclasses.FrozenInstanceError: cannot assign to field 'j'

This is tricky to fix.

Here's the problem with a inheriting a non-frozen dataclass from a 
frozen one. Consider class Y in this example:

@dataclass
class X:
     x: int

@dataclass
class Y(X):
     y: int

Y's __init__ looks like:

def __init__(self, x, y):
     self.x = x
     self.y = y

That is, all of the initializing for Y and its base classes which are 
dataclasses is done in Y's __init__. There are a number of reasons for 
this, including performance and not knowing how to call the base 
classes' __init__ methods.

If Y is frozen, then the __init__ currently looks like:

def __init__(self, x, y):
     object.__setattr__(self, 'x', x)
     object.__setattr__(self, 'y', y)

If X is frozen but Y isn't, then it should look like:

def __init__(self, x, y):
     object.__setattr__(self, 'x', x)
     self.y = y

But I currently can't generate the code that way, because I don't know 
if a base dataclass is frozen. That information is not saved on a dataclass.

I think the right thing to do is to record with each dataclass if it is 
frozen or not, so that derived classes can generate the correct code. 
Another option would be to always use object.__setattr__, but that will 
hurt performance in the common case.

As long as I'm saving if a dataclass is frozen, I should save all of the 
dataclass parameters on the class. Since it's per-class, it's not a lot 
of overhead.

I should probably separate the two issues here: 1) deriving a 
non-dataclass from a frozen dataclass and 2) deriving a non-frozen 
dataclass from a frozen dataclass, but since they're somewhat related 
I'm going to keep them together for the time being. #1 was Raymond's 
initial report.

Eric

From eric at trueblade.com  Mon Feb 26 13:35:11 2018
From: eric at trueblade.com (Eric V. Smith)
Date: Mon, 26 Feb 2018 13:35:11 -0500
Subject: [Python-Dev] Should the dataclass frozen property apply to
 subclasses?
In-Reply-To: <CADiSq7e+zHWcAf-8BDQdcnNduyzMSnghAicJ08=k34UqTY6=Tw@mail.gmail.com>
References: <A27F9EB7-D3BA-4358-B23E-A4EFAD7088AB@gmail.com>
 <1a6d1aa6-60e1-58db-ef2d-672c7f6c6c49@trueblade.com>
 <CADiSq7e+zHWcAf-8BDQdcnNduyzMSnghAicJ08=k34UqTY6=Tw@mail.gmail.com>
Message-ID: <3d26b4eb-02f9-74fe-e1ca-9e47a7bcc0ea@trueblade.com>

On 2/22/18 9:43 PM, Nick Coghlan wrote:
> On 22 February 2018 at 20:55, Eric V. Smith <eric at trueblade.com> wrote:
>> On 2/22/2018 1:56 AM, Raymond Hettinger wrote:
>>> When working on the docs for dataclasses, something unexpected came up.
>>> If a dataclass is specified to be frozen, that characteristic is inherited
>>> by subclasses which prevents them from assigning additional attributes:
>>>
>>>       >>> @dataclass(frozen=True)
>>>       class D:
>>>               x: int = 10
>>>
>>>       >>> class S(D):
>>>               pass
>>>
>>>       >>> s = S()
>>>       >>> s.cached = True
>>>       Traceback (most recent call last):
>>>         File "<pyshell#49>", line 1, in <module>
>>>           s.cached = True
>>>         File
>>> "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/dataclasses.py",
>>> line 448, in _frozen_setattr
>>>           raise FrozenInstanceError(f'cannot assign to field {name!r}')
>>>       dataclasses.FrozenInstanceError: cannot assign to field 'cached'
>>
>> This is because "frozen-ness" is implemented by adding __setattr__ and
>> __delattr__ methods in D, which get inherited by S.
>>
>>> Other immutable classes in Python don't behave the same way:
>>>
>>>
>>>       >>> class T(tuple):
>>>               pass
>>>
>>>       >>> t = T([10, 20, 30])
>>>       >>> t.cached = True
>>>
>>>       >>> class F(frozenset):
>>>               pass
>>>
>>>       >>> f = F([10, 20, 30])
>>>       >>> f.cached = True
>>>
>>>       >>> class B(bytes):
>>>               pass
>>>
>>>       >>> b = B()
>>>       >>> b.cached = True
>>
>> The only way I can think of emulating this is checking in __setattr__ to see
>> if the field name is a field of the frozen class, and only raising an error
>> in that case.
> If you were going to do that then it would likely make more sense to
> convert the frozen fields to data descriptors, so __setattr__ only
> gets called for attempts to add new attributes.
>
> Then for the `frozen=False` case, the decorator could force
> __setattr__ and __delattr__ back to the default implementations from
> object, rather than relying on the behaviour inherited from base
> classes.

I guess it depends on what we're trying to do. By using data 
descriptors, we'd be saying "the fields can't be modified, but other 
parts of the class can". For example, what should happen here:

@dataclass(frozen=True)
class F:
    i: int

f = F(10)
f.i = 0   # should be an error
f.j = 0   # should this be an error? It's not a field.

I was hoping to get all of this finished today, by a2, but it looks like 
that's not going to happen.
>> A related issue is that dataclasses derived from frozen dataclasses are
>> automatically "promoted" to being frozen.
>>
>>>>> @dataclass(frozen=True)
>> ... class A:
>> ...     i: int
>> ...
>>>>> @dataclass
>> ... class B(A):
>> ...     j: int
>> ...
>>>>> b = B(1, 2)
>>>>> b.j = 3
>> Traceback (most recent call last):
>>    File "<stdin>", line 1, in <module>
>>    File "C:\home\eric\local\python\cpython\lib\dataclasses.py", line 452, in
>> _frozen_setattr
>>      raise FrozenInstanceError(f'cannot assign to field {name!r}')
>> dataclasses.FrozenInstanceError: cannot assign to field 'j'
>>
>> Maybe it should be an error to declare B as non-frozen?
> It would be nice to avoid that, as a mutable subclass of a frozen base
> class could be a nice way to model hashable-but-mutable types:
>
>      >>> @dataclass(frozen=True) # This is the immutable/hashable bit
>      ... class A:
>      ...     i: int
>      ...
>      >>> @dataclass # This adds the mutable-but-comparable parts
>      ... class B(A):
>      ...     j: int
>      ...     __hash__ = A.__hash__
>
>
> However, disallowing this case for now *would* be a reasonable way to
> postpone making a decision until 3.8 - in the meantime, folks would
> still be able to experiment by overriding __setattr__ and __delattr__
> after the dataclass decorator sets them.
>

For today, I'm going to make it an error to derive a frozen dataclass 
from a non-frozen dataclass, and also an error to derive a non-frozen 
dataclass from a frozen dataclass. With any luck (and Guido and Ned 
willing), we can address this by a3.

Eric


From rob.cliffe at btinternet.com  Mon Feb 26 13:41:29 2018
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Mon, 26 Feb 2018 18:41:29 +0000
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <p6n43o$qka$1@blaine.gmane.org>
References: <p6n43o$qka$1@blaine.gmane.org>
Message-ID: <3b355686-b01c-468b-dbf8-78e1f676cb65@btinternet.com>



On 22/02/2018 19:04, Serhiy Storchaka wrote:
> Yet one discussion about reusing common subexpressions in 
> comprehensions took place last week on the Python-ideas maillist (see 
> topic "Temporary variables in comprehensions" [1]). The problem is 
> that in comprehension like `[f(x) + g(f(x)) for x in range(10)]` the 
> subexpression `f(x)` is evaluated twice. In normal loop you can 
> introduce a temporary variable for `f(x)`. The OP wanted to add a 
> special syntax for introducing temporary variables in comprehensions. 
> This idea already was discussed multiple times in the past.
>
> There are several ways of resolving this problem with existing syntax.
>
> [snip]
>
>
> Stephan Houben proposed an idiom which looks similar to new hypothetic 
> syntax:
>
> ??? result = [y + g(y) for x in range(10) for y in [f(x)]]
>
> `for y in [expr]` in a comprehension means just assigning expr to y. I 
> never seen this idiom before, but it can be a good replacement for a 
> hypothetic syntax for assignment in comprehensions. It changes the 
> original comprehension less than other approaches, just adds yet one 
> element in a sequence of for-s and if-s. I think that after using it 
> more widely it will become pretty idiomatic.
>
> I have created a patch that optimizes this idiom, making it as fast as 
> a normal assignment. [2] Yury suggested to ask Guido on the mailing 
> list if he agrees that this language patten is worth 
> optimizing/promoting.
>
Here's a thought: allow the syntax
 ??? for VAR = EXPR
to define a for-loop that is executed exactly once (both inside and 
outside comprehensions), i.e. pretty much a synonym for
 ??? for VAR in [ EXPR ]
 ??? for VAR in ( EXPR , )
especially if Serhiy's optimisation means that the list/tuple is not 
actually constructed in the latter.

Pros:
(1) Stephan Houben's example could be written as
 ??? ??? result = [y + g(y) for x in range(10) for y = f(x)]
 ??? which I find more readable.
(2) Code such as
 ??? ??? ??? for i in xrange(10):
 ??? could be changed on the fly to:
 ??? ??? ??? for i = 1:
 ??? I see this as especially useful in debugging, where you want to 
limit the program execution to a known problematic bit.
 ??? But it some contexts it could be good style.
(3) Preserves the compatibility between a list comprehension and its 
"expansion" into for-loops.
(4) Backward compatible, since it is currently illegal syntax
(5) No extra keyword needed
(6) It goes some way towards providing the functionality of
 ??? ??? ??? with VAR as EXPR
 ??? ??? that has been discussed multiple times.

Best wishes
Rob Cliffe

From ethan at stoneleaf.us  Mon Feb 26 13:51:12 2018
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 26 Feb 2018 10:51:12 -0800
Subject: [Python-Dev] Should the dataclass frozen property apply to
 subclasses?
In-Reply-To: <c683dc16-7ff4-c031-a20f-c9b3a6ec1c89@trueblade.com>
References: <A27F9EB7-D3BA-4358-B23E-A4EFAD7088AB@gmail.com>
 <1a6d1aa6-60e1-58db-ef2d-672c7f6c6c49@trueblade.com>
 <c683dc16-7ff4-c031-a20f-c9b3a6ec1c89@trueblade.com>
Message-ID: <5A945720.7030209@stoneleaf.us>

On 02/26/2018 05:40 AM, Eric V. Smith wrote:

> As long as I'm saving if a dataclass is frozen, I should save all of the dataclass parameters on the class. Since it's
> per-class, it's not a lot of overhead.

That's what Enum does.  A bit of pain, but makes so many other things easier.

--
~Ethan~


From guido at python.org  Mon Feb 26 14:08:42 2018
From: guido at python.org (Guido van Rossum)
Date: Mon, 26 Feb 2018 11:08:42 -0800
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <3b355686-b01c-468b-dbf8-78e1f676cb65@btinternet.com>
References: <p6n43o$qka$1@blaine.gmane.org>
 <3b355686-b01c-468b-dbf8-78e1f676cb65@btinternet.com>
Message-ID: <CAP7+vJLZAEStNVhopoas3AFKWsVmnMYC1Bc11dDZp-r59L0StA@mail.gmail.com>

I would like to remind all wannabe language designers that grammar design
is not just solving puzzles. It's also about keeping the overall feel of
the language readable. I'm getting the idea that none of the proposals
discussed so far (whether new syntax or clever use of existing syntax)
satisfy that constraint. Sometimes a for-loop is just better.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180226/d6b41ef7/attachment.html>

From ncoghlan at gmail.com  Mon Feb 26 16:27:45 2018
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 27 Feb 2018 07:27:45 +1000
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <CAP7+vJLZAEStNVhopoas3AFKWsVmnMYC1Bc11dDZp-r59L0StA@mail.gmail.com>
References: <p6n43o$qka$1@blaine.gmane.org>
 <3b355686-b01c-468b-dbf8-78e1f676cb65@btinternet.com>
 <CAP7+vJLZAEStNVhopoas3AFKWsVmnMYC1Bc11dDZp-r59L0StA@mail.gmail.com>
Message-ID: <CADiSq7dL=3fb926=yMbJjUJR7EQi8GVNJm4ffvZm0VLB6WZYtQ@mail.gmail.com>

On 27 February 2018 at 05:08, Guido van Rossum <guido at python.org> wrote:

> I would like to remind all wannabe language designers that grammar design
> is not just solving puzzles. It's also about keeping the overall feel of
> the language readable. I'm getting the idea that none of the proposals
> discussed so far (whether new syntax or clever use of existing syntax)
> satisfy that constraint. Sometimes a for-loop is just better.
>

+1

This is the main reason PEP 3150 (which adds a more limited form of
statement local named subexpressions) has spent more time Deferred than it
has ever being discussed as an active draft proposal: while naming
subexpressions is an occasionally attractive prospect, it's also an
addition that has significant potential to change the way various kinds of
code is typically written (even more so than something like type hints or
f-strings).

When even a PEP's author is thinking "I'm not sure this will actually be a
net improvement to the language", it's really not a good sign :)

Cheers,
Nick.

P.S. The comprehension-centric variants at least have the virtue of
precedent in Haskell's "let" clauses:
https://stackoverflow.com/questions/6067839/haskell-let-where-equivalent-within-list-comprehension/6067878#6067878

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180227/3bb01884/attachment.html>

From rob.cliffe at btinternet.com  Mon Feb 26 19:30:59 2018
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Tue, 27 Feb 2018 00:30:59 +0000
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <CAP7+vJLZAEStNVhopoas3AFKWsVmnMYC1Bc11dDZp-r59L0StA@mail.gmail.com>
References: <p6n43o$qka$1@blaine.gmane.org>
 <3b355686-b01c-468b-dbf8-78e1f676cb65@btinternet.com>
 <CAP7+vJLZAEStNVhopoas3AFKWsVmnMYC1Bc11dDZp-r59L0StA@mail.gmail.com>
Message-ID: <350a18a1-8472-fc62-0f17-621313e64498@btinternet.com>



On 26/02/2018 19:08, Guido van Rossum wrote:
> I would like to remind all wannabe language designers that grammar 
> design is not just solving puzzles. It's also about keeping the 
> overall feel of the language readable. I'm getting the idea that none 
> of the proposals discussed so far (whether new syntax or clever use of 
> existing syntax) satisfy that constraint. Sometimes a for-loop is just 
> better.
>
I don't know if you intended these remarks to include my proposal (to 
allow "for VAR = EXPR"), as your message was posted only 27 minutes 
after mine.
With respect, I honestly feel that this is a relatively small change 
that makes the language *more* readable.

Feel free, one and all, to tell me why I'm wrong.
Best wishes,
Rob Cliffe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180227/556f0546/attachment.html>

From guido at python.org  Mon Feb 26 19:51:10 2018
From: guido at python.org (Guido van Rossum)
Date: Mon, 26 Feb 2018 16:51:10 -0800
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <350a18a1-8472-fc62-0f17-621313e64498@btinternet.com>
References: <p6n43o$qka$1@blaine.gmane.org>
 <3b355686-b01c-468b-dbf8-78e1f676cb65@btinternet.com>
 <CAP7+vJLZAEStNVhopoas3AFKWsVmnMYC1Bc11dDZp-r59L0StA@mail.gmail.com>
 <350a18a1-8472-fc62-0f17-621313e64498@btinternet.com>
Message-ID: <CAP7+vJJru9ySsO9CoQ2R6apn6Ys-2oNbbp_qVu1GjjGjM+MoPw@mail.gmail.com>

On Mon, Feb 26, 2018 at 4:30 PM, Rob Cliffe via Python-Dev <
python-dev at python.org> wrote:


> On 26/02/2018 19:08, Guido van Rossum wrote:
>
> I would like to remind all wannabe language designers that grammar design
> is not just solving puzzles. It's also about keeping the overall feel of
> the language readable. I'm getting the idea that none of the proposals
> discussed so far (whether new syntax or clever use of existing syntax)
> satisfy that constraint. Sometimes a for-loop is just better.
>
> I don't know if you intended these remarks to include my proposal (to
> allow "for VAR = EXPR"), as your message was posted only 27 minutes after
> mine.
> With respect, I honestly feel that this is a relatively small change that
> makes the language *more* readable.
>
> Feel free, one and all, to tell me why I'm wrong.
> Best wishes,
> Rob Cliffe
>

I didn't want to single you out, but yes, I did include your proposal.

The reason is that for people who are not Python experts there's no obvious
reason why `for VAR = EXPR` should mean one thing and `for VAR in EXPR`
should mean another.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180226/3e28e243/attachment.html>

From g.rodola at gmail.com  Tue Feb 27 06:49:55 2018
From: g.rodola at gmail.com (Giampaolo Rodola')
Date: Tue, 27 Feb 2018 12:49:55 +0100
Subject: [Python-Dev] How is the GitHub workflow working for people?
In-Reply-To: <CAP7+vJK_myg8sbs-G4BzqdH9ytyFHSyA05FTPOuZ=Sv1k8EhOA@mail.gmail.com>
References: <CAP1=2W4SEuk4gHY3kXkhWcs8_e3cs-QEuYdMNV+3-aUgK419Aw@mail.gmail.com>
 <CA+St6D2C51PynrSKEdSgCjxG47fO2d+qe79a6Z7Lj6ioBcxnAQ@mail.gmail.com>
 <CAP1=2W7VWN0mTPDkOnfrhDsK0xMoObvNRuWc792a263BcTAHBQ@mail.gmail.com>
 <CAP7+vJ+xVuUn__55PGu3iqDWcR612VAC8bxbK31rQZMxaoE7Lw@mail.gmail.com>
 <3878742A-B840-4127-AD84-AF67ADED1E80@python.org>
 <CAGGBd_r+LkkYqSMwBvjq69hiqo5gky7hk5htnsDemzHKyMfcXg@mail.gmail.com>
 <D1F7B9B8-8586-4335-8C18-854DF10D3ADE@python.org>
 <CADiSq7ci8sCEG0ks7Bk8_AmByxVB5C4UWMG2FB_Xzg-CAoz2Eg@mail.gmail.com>
 <CAGbohnYLFX5FD-w+KQ=2vCTzi_ghuPfXzCRMZ9c76HyL7Py74A@mail.gmail.com>
 <CAP7+vJK_myg8sbs-G4BzqdH9ytyFHSyA05FTPOuZ=Sv1k8EhOA@mail.gmail.com>
Message-ID: <CAFYqXL-Tb0JRwd9GV7MZTuQ1crd3k2M4ZcRhMeA3eFDEZ=nOcQ@mail.gmail.com>

I personally use a GIT commit hook which runs flake8 against the *modified
py files only* and rejects the commit in case of non-compliance:
https://github.com/giampaolo/psutil/blob/master/.git-pre-commit
...I install it via make:
https://github.com/giampaolo/psutil/blob/ad4acae5489f86fc3bef645505b3873f156b4867/Makefile#L186
...and once the whole code base is in a good shape have Travis run flake8
against the whole code base on each commit.

When I join an existing project I try to enforce the same workflow and rely
on pep8ify (https://github.com/spulec/pep8ify) to adjust the existing code.
In order to not be too "strict" I may modify the GIT hook to emit a warning
instead of rejecting the commit.

On one hand I think it would be great to do the same for cPython. On the
other hand I see 2 big downsides:

- losing "git blame" history for A LOT of lines of code
- even if you're familiar with PEP8 it's sometimes hard to spot
non-compliant lines unless you integrate flake8 into your IDE (I do for
Sublime); putting such a constraint on contributing users is probably too
much and may discourage contributions.


On Sun, Feb 25, 2018 at 6:13 AM, Guido van Rossum <guido at python.org> wrote:

> It's easy to only analyze the files in the diff (these linters don't do
> cross-file analysis anyways, typically) and it's possible to write a filter
> that only keeps warnings about lines that are changed, but I don't know of
> a standard solution for the latter (places where I worked where I've seen
> this always had their own custom implementation).
>
> On Sat, Feb 24, 2018 at 7:35 PM, Mariatta Wijaya <
> mariatta.wijaya at gmail.com> wrote:
>
>> Can any of these said linters analyze only the diff in the PR, instead of
>> the entire CPython codebase?
>>
>> Mariatta Wijaya
>>
>> ?
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%
>> 40python.org
>>
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/g.
> rodola%40gmail.com
>
>


-- 
Giampaolo - http://grodola.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180227/97b6457c/attachment-0001.html>

From songofacandy at gmail.com  Tue Feb 27 08:37:24 2018
From: songofacandy at gmail.com (INADA Naoki)
Date: Tue, 27 Feb 2018 22:37:24 +0900
Subject: [Python-Dev] Backward incompatible change about docstring AST
Message-ID: <CAEfz+Twynf8O=sCj40p=GB3VhFLQTYJpHwGKkLJoN1g0OQww=Q@mail.gmail.com>

Hi, all.

There is design discussion which is deferred blocker of 3.7.
https://bugs.python.org/issue32911

## Background

An year ago, I moved docstring in AST from statements list to field of
module, class and functions.
https://bugs.python.org/issue29463

Without this change, AST-level constant folding was complicated because
"foo" can be docstring but "fo" + "o" can't be docstring.

This simplified some other edge cases.  For example, future import must
be on top of the module, but docstring can be before it.
Docstring is very special than other expressions/statement.

Of course, this change was backward incompatible.
Tools reading/writing docstring via AST will be broken by this change.
For example, it broke PyFlakes, and PyFlakes solved it already.

https://github.com/PyCQA/pyflakes/pull/273

Since AST doesn't guarantee backward compatibility, we can change
AST if it's reasonable.

Last week, Mark Shannon reported issue about this backward incompatibility.
As he said, this change losted lineno and column of docstring from AST.

https://bugs.python.org/issue32911#msg312567


## Design discussion

And as he said, there are three options:

https://bugs.python.org/issue32911#msg312625

> It seems to be that there are three reasonable choices:
> 1. Revert to 3.6 behaviour, with the addition of `docstring` attribute.
> 2. Change the docstring attribute to an AST node, possibly by modifying the grammar.
> 3. Do nothing.

1 is backward compatible about reading docstring.
But when writing, it's not DRY or SSOT.  There are two source of docstring.
For example: `ast.Module([ast.Str("spam")], docstring="egg")`

2 is considerable.  I tried to implement this idea by adding `DocString`
statement AST.
https://github.com/python/cpython/pull/5927/files

While it seems large change, most changes are reverting the AST changes.
So it's more closer to 3.6 codebase.  (especially, test_ast is very
close to 3.6)

In this PR, `ast.Module([ast.Str("spam")])` doesn't have docstring for
simplicity.  So it's backward incompatible for both of reading and
writing docstring too.
But it keeps lineno and column of docstring in AST.

3 is most conservative because 3.7b2 was cut now and there are some tools
supporting 3.7 already.


I prefer 2 or 3.  If we took 3, I don't want to do 2 in 3.8.  One
backward incompatible
change is better than two.

Any thoughts?

-- 
INADA Naoki  <songofacandy at gmail.com>

From storchaka at gmail.com  Tue Feb 27 09:32:22 2018
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 27 Feb 2018 16:32:22 +0200
Subject: [Python-Dev] Backward incompatible change about docstring AST
In-Reply-To: <CAEfz+Twynf8O=sCj40p=GB3VhFLQTYJpHwGKkLJoN1g0OQww=Q@mail.gmail.com>
References: <CAEfz+Twynf8O=sCj40p=GB3VhFLQTYJpHwGKkLJoN1g0OQww=Q@mail.gmail.com>
Message-ID: <p73q1n$18k$1@blaine.gmane.org>

27.02.18 15:37, INADA Naoki ????:
> Of course, this change was backward incompatible.
> Tools reading/writing docstring via AST will be broken by this change.
> For example, it broke PyFlakes, and PyFlakes solved it already.
> 
> https://github.com/PyCQA/pyflakes/pull/273

Other examples:

coveragepy: https://bitbucket.org/ned/coveragepy/commits/99176232199b
pytest: https://github.com/pytest-dev/pytest/pull/2870


> Last week, Mark Shannon reported issue about this backward incompatibility.
> As he said, this change losted lineno and column of docstring from AST.

While losing lineno and column is a loss, it is not so large. There are 
existing issues with docstring position.

1. CPython and PyPy set different position for multiline strings. PyPy 
sets the position of the start of string, but CPython sets the position 
of the end of the string. A program that utilizes the docstring position 
needs to handle both of these cases.

2. Usually the position of the docstring is used for determining the 
absolute position of some fragments in the docstring (for example 
doctests). But since the literal string can contain \n and escaped 
newlines, and this information is lost in AST, the position of the 
fragment can be determined accurately. This is just a best effort 
approximation.

3. You can determine an approximate position of the docstring by 
positions of preceding or following nodes.


From viktor.kovacs at gmail.com  Tue Feb 27 01:46:03 2018
From: viktor.kovacs at gmail.com (Viktor Kovacs)
Date: Tue, 27 Feb 2018 07:46:03 +0100
Subject: [Python-Dev] Disable built-in module import in embedded Python
Message-ID: <CAAZYCGkFREu4sqW2Bw=ZV=UP7LdF_2HHdW74b3q65wwWQy8Ytw@mail.gmail.com>

Hi All,

I have a question about embedding python in C++. I don't want to create a
duplication so I just leave here the existing stack overflow question:
https://stackoverflow.com/questions/48992030/disable-built-in-module-import-in-embedded-python

Do you have any idea how to achieve this?

Thanks,
Viktor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180227/57eea2f3/attachment.html>

From mikez302 at gmail.com  Tue Feb 27 02:34:28 2018
From: mikez302 at gmail.com (Elias Zamaria)
Date: Mon, 26 Feb 2018 23:34:28 -0800
Subject: [Python-Dev] PEP 467 (Minor API improvements for binary
 sequences) - any thoughts?
In-Reply-To: <CALGmxE+_kT07HNPPHeyMfhoqrN3DV26GTmLEHKpDaV_7CWaAEA@mail.gmail.com>
References: <CAFdOKfv+WMn6B0o4UtyPf1ziASUihELmQ_3K_WUNNfDsUva9+w@mail.gmail.com>
 <CALGmxEKjG-=+dVY8sNPdszq84hOxUnG0Cp5MH6edJEz37v0_9g@mail.gmail.com>
 <CAMofdRAWj4GuabDyD1z2hwy2_u01S0+Aoyu8DJ037n0LzDwSaw@mail.gmail.com>
 <CALGmxE+_kT07HNPPHeyMfhoqrN3DV26GTmLEHKpDaV_7CWaAEA@mail.gmail.com>
Message-ID: <CAFdOKftNojJ3GqojpW=ssSueY5LV9OZxFhuW5GhGvL_XaRQnBA@mail.gmail.com>

Nick, I'm trying to reply to your message, but I can't figure out how.

You mentioned that the PEP needs a "champion". What would that involve? How
much time and effort would it take? What kinds of decisions would I make?

The iterbytes thing in the PEP is something I was wishing for, while
working on a personal project. I stumbled upon this PEP and decided to try
to implement it myself, to learn about C and the Python internals, among
other reasons.

I don't know how I would feel working on something so general, of use to so
many people for lots of different purposes. Do I know enough about all of
the use cases and what everyone wants? I am not completely against it but
I'd need to think about it.

On Wed, Feb 21, 2018 at 2:36 PM, Chris Barker <chris.barker at noaa.gov> wrote:

>
>
> On Wed, Feb 21, 2018 at 12:39 PM, Steve Holden <steve at holdenweb.com>
> wrote:
>
>> I think the chances of a "byte" object are about as good as the chances
>> of a character object
>>
>
> probably right.
>
>
>> (though one can always implement such in C extensions, that wouldn't
>> build them into the syntax).
>>
>
> I think you could simply subclass, too (overriding __new__ and a couple
> methods). But that would do exactly no good, unless you used your own
> custom string and bytes objects, too. The whole point is that iterating
> over a string (Or bytes) always returns an also-iterable object,
> ad-infinitum.
>
> This is the cause of the major remaining common "type error" in Python.
> (the old integer division used to be the big one)
>
>
>> The fact that characters are single-byte strings is responsible for
>> certain anomalies with (e.g.) the __contains__ operator (list elements
>> aren't lists, but string element are strings), but overall the choices made
>> lead to sensible, comprehensible code.
>>
>
> I'm pretty convinced that the choice not to have a character type has had
> basically zero benefits to sensible, comprehensible code, though it's not
> a very big deal, either. not a big enough deal for the churn it would cause
> to introduce it now, that's for sure.
>
> so +1 for this PEP as is.
>
> -CHB
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE
> <https://maps.google.com/?q=7600+Sand+Point+Way+NE&entry=gmail&source=g>
>   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180226/3b160bfd/attachment.html>

From ethan at stoneleaf.us  Tue Feb 27 12:15:05 2018
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 27 Feb 2018 09:15:05 -0800
Subject: [Python-Dev] PEP 467 (Minor API improvements for binary
 sequences) - any thoughts?
In-Reply-To: <CAFdOKftNojJ3GqojpW=ssSueY5LV9OZxFhuW5GhGvL_XaRQnBA@mail.gmail.com>
References: <CAFdOKfv+WMn6B0o4UtyPf1ziASUihELmQ_3K_WUNNfDsUva9+w@mail.gmail.com>
 <CALGmxEKjG-=+dVY8sNPdszq84hOxUnG0Cp5MH6edJEz37v0_9g@mail.gmail.com>
 <CAMofdRAWj4GuabDyD1z2hwy2_u01S0+Aoyu8DJ037n0LzDwSaw@mail.gmail.com>
 <CALGmxE+_kT07HNPPHeyMfhoqrN3DV26GTmLEHKpDaV_7CWaAEA@mail.gmail.com>
 <CAFdOKftNojJ3GqojpW=ssSueY5LV9OZxFhuW5GhGvL_XaRQnBA@mail.gmail.com>
Message-ID: <5A959219.3040008@stoneleaf.us>

On 02/26/2018 11:34 PM, Elias Zamaria wrote:

> Nick, I'm trying to reply to your message, but I can't figure out how.
>
> You mentioned that the PEP needs a "champion". What would that involve? How much time and effort would it take? What
> kinds of decisions would I make?

Being a PEP "champion" involves collecting lots of data, sorting it, making decisions about API design, posting about 
those decisions along with the pros and cons, listening to more feedback, and continuing until there is general 
agreement about the PEP.  After that a request is made to (usually) Guido to accept or reject the PEP.  If accepted, 
then the code writing stage happens.

Writing code first is not bad as a working proof-of-concept is always handy.

> The iterbytes thing in the PEP is something I was wishing for, while working on a personal project. I stumbled upon this
> PEP and decided to try to implement it myself, to learn about C and the Python internals, among other reasons.

It's a good way to go about it!

> I don't know how I would feel working on something so general, of use to so many people for lots of different purposes.
> Do I know enough about all of the use cases and what everyone wants? I am not completely against it but I'd need to
> think about it.

Part of the PEP writing process is asking for and collecting use-cases;  if possible, looking at other code projects for 
use-cases is also useful.

Time needed can vary widely depending on the subject; if I recall correctly, PEP 409 only took a few days, while PEP 435 
took several weeks.  PEP 467 has already gone through a few iterations, so hopefully not too much more time is required.

If you would like to try you'll get plenty of help from the community -- at least from those willing to go through PEP 
467 again.  ;)

At the moment, though, we're concentrating on getting v3.7.0 as bug-free as possible, so feel free to research where the 
PEP is now and go through the last discussion, but you should wait for the 3.7.0 release before actively bringing the 
PEP discussion back to python-dev.

--
~Ethan~

From alexander.belopolsky at gmail.com  Tue Feb 27 13:41:19 2018
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Tue, 27 Feb 2018 13:41:19 -0500
Subject: [Python-Dev] The `for y in [x]` idiom in comprehensions
In-Reply-To: <CAP7+vJJru9ySsO9CoQ2R6apn6Ys-2oNbbp_qVu1GjjGjM+MoPw@mail.gmail.com>
References: <p6n43o$qka$1@blaine.gmane.org>
 <3b355686-b01c-468b-dbf8-78e1f676cb65@btinternet.com>
 <CAP7+vJLZAEStNVhopoas3AFKWsVmnMYC1Bc11dDZp-r59L0StA@mail.gmail.com>
 <350a18a1-8472-fc62-0f17-621313e64498@btinternet.com>
 <CAP7+vJJru9ySsO9CoQ2R6apn6Ys-2oNbbp_qVu1GjjGjM+MoPw@mail.gmail.com>
Message-ID: <CAP7h-xb+d175HTPS3EYpuG_yg_e5B0eta81HOKvn2s+BtrdtYQ@mail.gmail.com>

On Mon, Feb 26, 2018 at 7:51 PM, Guido van Rossum <guido at python.org> wrote:
..
> The reason is that for people who are not Python experts there's no obvious
> reason why `for VAR = EXPR` should mean one thing and `for VAR in EXPR`
> should mean another.

This would be particularly surprising for people exposed to Julia
where these two forms are equivalent:

julia> for x = [1,2] println(x); end
1
2

julia> for x in [1,2] println(x); end
1
2

From tjreedy at udel.edu  Tue Feb 27 17:31:30 2018
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 27 Feb 2018 17:31:30 -0500
Subject: [Python-Dev] Backward incompatible change about docstring AST
In-Reply-To: <p73q1n$18k$1@blaine.gmane.org>
References: <CAEfz+Twynf8O=sCj40p=GB3VhFLQTYJpHwGKkLJoN1g0OQww=Q@mail.gmail.com>
 <p73q1n$18k$1@blaine.gmane.org>
Message-ID: <p74m44$t77$1@blaine.gmane.org>

On 2/27/2018 9:32 AM, Serhiy Storchaka wrote:
> 27.02.18 15:37, INADA Naoki ????:
>> Of course, this change was backward incompatible.
>> Tools reading/writing docstring via AST will be broken by this change.
>> For example, it broke PyFlakes, and PyFlakes solved it already.
>>
>> https://github.com/PyCQA/pyflakes/pull/273
> 
> Other examples:
> 
> coveragepy: https://bitbucket.org/ned/coveragepy/commits/99176232199b
> pytest: https://github.com/pytest-dev/pytest/pull/2870
> 
> 
>> Last week, Mark Shannon reported issue about this backward 
>> incompatibility.
>> As he said, this change losted lineno and column of docstring from AST.
> 
> While losing lineno and column is a loss, it is not so large. There are 
> existing issues with docstring position.
> 
> 1. CPython and PyPy set different position for multiline strings. PyPy 
> sets the position of the start of string, but CPython sets the position 
> of the end of the string. A program that utilizes the docstring position 
> needs to handle both of these cases.
> 
> 2. Usually the position of the docstring is used for determining the 
> absolute position of some fragments in the docstring (for example 
> doctests). But since the literal string can contain \n and escaped 
> newlines, and this information is lost in AST, the position of the 
> fragment can be determined accurately.

You obviously meant 'cannot be determined accurately', because for 
reasons including the ones you gave, the relation between string 
literals in code and the resulting string objects is many-to-one, (as 
with int literals).

> This is just a best effort approximation.
> 
> 3. You can determine an approximate position of the docstring by 
> positions of preceding or following nodes.

-- 
Terry Jan Reedy



From storchaka at gmail.com  Tue Feb 27 17:48:25 2018
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Wed, 28 Feb 2018 00:48:25 +0200
Subject: [Python-Dev] Backward incompatible change about docstring AST
In-Reply-To: <p74m44$t77$1@blaine.gmane.org>
References: <CAEfz+Twynf8O=sCj40p=GB3VhFLQTYJpHwGKkLJoN1g0OQww=Q@mail.gmail.com>
 <p73q1n$18k$1@blaine.gmane.org> <p74m44$t77$1@blaine.gmane.org>
Message-ID: <p74n3r$8r6$1@blaine.gmane.org>

28.02.18 00:31, Terry Reedy ????:
>> 2. Usually the position of the docstring is used for determining the 
>> absolute position of some fragments in the docstring (for example 
>> doctests). But since the literal string can contain \n and escaped 
>> newlines, and this information is lost in AST, the position of the 
>> fragment can be determined accurately.
> 
> You obviously meant 'cannot be determined accurately', because for 
> reasons including the ones you gave, the relation between string 
> literals in code and the resulting string objects is many-to-one, (as 
> with int literals).

Yes, thank you for correction.

The reasons are that the relation between lines of source code and lines 
of the resulting string objects is not easy. The string literal 'a\nb\c' 
takes one line in the source code, but produces a three-line string 
object. On other side, the string literal

     '''\
     abc\
     '''

takes three lines in the source code, but produces a single-line string 
object. And it is not so rare that a docstring starts with an escaped 
newline.


From brett at python.org  Tue Feb 27 18:55:52 2018
From: brett at python.org (Brett Cannon)
Date: Tue, 27 Feb 2018 23:55:52 +0000
Subject: [Python-Dev] Backward incompatible change about docstring AST
In-Reply-To: <CAEfz+Twynf8O=sCj40p=GB3VhFLQTYJpHwGKkLJoN1g0OQww=Q@mail.gmail.com>
References: <CAEfz+Twynf8O=sCj40p=GB3VhFLQTYJpHwGKkLJoN1g0OQww=Q@mail.gmail.com>
Message-ID: <CAP1=2W4rOS_OzdSmQ0=HFz+oFgHFvZTLpx78vDH7icVL_5HDkg@mail.gmail.com>

On Tue, 27 Feb 2018 at 05:38 INADA Naoki <songofacandy at gmail.com> wrote:

> Hi, all.
>
> There is design discussion which is deferred blocker of 3.7.
> https://bugs.python.org/issue32911
>
> ## Background
>
> An year ago, I moved docstring in AST from statements list to field of
> module, class and functions.
> https://bugs.python.org/issue29463
>
> Without this change, AST-level constant folding was complicated because
> "foo" can be docstring but "fo" + "o" can't be docstring.
>
> This simplified some other edge cases.  For example, future import must
> be on top of the module, but docstring can be before it.
> Docstring is very special than other expressions/statement.
>
> Of course, this change was backward incompatible.
> Tools reading/writing docstring via AST will be broken by this change.
> For example, it broke PyFlakes, and PyFlakes solved it already.
>
> https://github.com/PyCQA/pyflakes/pull/273
>
> Since AST doesn't guarantee backward compatibility, we can change
> AST if it's reasonable.
>
> Last week, Mark Shannon reported issue about this backward incompatibility.
> As he said, this change losted lineno and column of docstring from AST.
>
> https://bugs.python.org/issue32911#msg312567
>
>
> ## Design discussion
>
> And as he said, there are three options:
>
> https://bugs.python.org/issue32911#msg312625
>
> > It seems to be that there are three reasonable choices:
> > 1. Revert to 3.6 behaviour, with the addition of `docstring` attribute.
> > 2. Change the docstring attribute to an AST node, possibly by modifying
> the grammar.
> > 3. Do nothing.
>

I'm +1 for #3, +0 for #2, -1 for #1.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180227/0eea1c81/attachment.html>

From eric at trueblade.com  Tue Feb 27 19:37:45 2018
From: eric at trueblade.com (Eric V. Smith)
Date: Tue, 27 Feb 2018 19:37:45 -0500
Subject: [Python-Dev] Should the dataclass frozen property apply to
 subclasses?
In-Reply-To: <A27F9EB7-D3BA-4358-B23E-A4EFAD7088AB@gmail.com>
References: <A27F9EB7-D3BA-4358-B23E-A4EFAD7088AB@gmail.com>
Message-ID: <2362d149-1b72-62e2-55b6-ce2d4143b04f@trueblade.com>

On 2/22/2018 1:56 AM, Raymond Hettinger wrote:
> When working on the docs for dataclasses, something unexpected came up.  If a dataclass is specified to be frozen, that characteristic is inherited by subclasses which prevents them from assigning additional attributes:
> 
>      >>> @dataclass(frozen=True)
>      class D:
>              x: int = 10
> 
>      >>> class S(D):
>              pass
> 
>      >>> s = S()
>      >>> s.cached = True
>      Traceback (most recent call last):
>        File "<pyshell#49>", line 1, in <module>
>          s.cached = True
>        File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/dataclasses.py", line 448, in _frozen_setattr
>          raise FrozenInstanceError(f'cannot assign to field {name!r}')
>      dataclasses.FrozenInstanceError: cannot assign to field 'cached'
> 
> Other immutable classes in Python don't behave the same way:
> 
> 
>      >>> class T(tuple):
>              pass
> 
>      >>> t = T([10, 20, 30])
>      >>> t.cached = True
> 
>      >>> class F(frozenset):
>              pass
> 
>      >>> f = F([10, 20, 30])
>      >>> f.cached = True
> 
>      >>> class B(bytes):
>              pass
> 
>      >>> b = B()
>      >>> b.cached = True
> 

I'll provide some background, then get in to the dataclasses design 
issues. Note that I'm using "field" here in the PEP 557 sense.

There are some questions to resolve:

1. What happens when a frozen dataclass inherits from a non-frozen 
dataclass?

2. What happens when a non-frozen dataclass inherits from a frozen 
dataclass?

3. What happens when a non-dataclass inherits from a frozen dataclass?

4. Can new non-field attributes be created for frozen dataclasses?


I think it's useful to look at what attrs does. Unsurprisingly, attrs 
works the way the dataclasses implementation in 3.7.0a1 works:

- If a frozen attrs class inherits from a non-frozen attrs class, the 
result is a frozen attrs class.

- If a non-frozen attrs class inherits from a frozen attrs class, the 
result is a frozen attrs class.

- For a frozen attrs class, you may not assign to any field, nor create 
new non-field instance attributes.

- If a non-attrs class derives from a frozen attrs class, then you 
cannot assign to or create any non-field instance attributes. This is 
because they override the class's __setattr__ to always raise. This is 
the case that Raymond initially brought up on this thread (but for 
dataclasses, of course).

As I said, this is also how 3.7.0a1 dataclasses also works. The only 
difference between this and 3.7.0.a2 is that I prohibited inheriting a 
non-frozen dataclass from a frozen one, and also prohibited the 
opposite: you can't inherit a frozen dataclass from a non-frozen 
dataclass. This was just a stop-gap measure to give us more wiggle room 
for future changes. But this does nothing to address Raymond's concern 
about non-dataclasses deriving from frozen dataclasses.

A last piece of background info on how dataclasses and attrs work: the 
most derived class implements all of the functionality. They never call 
in to the base class to do anything. The base classes just exist to 
provide the list of fields.

If frozen dataclasses only exist to protect fields that belong to the 
hash, then my suggestion is to change the implementation of frozen class 
to use properties instead of overwriting __setattr__ (Nick also 
suggested this). This would allow you to create non-field attributes. If 
the purpose is really to prevent any attributes from being added or 
modified, then I think __setattr__ should stay but we should change it 
to allow non-dataclass subclasses to add non-field instance attributes 
(addressing Raymond's concern).

I think we shouldn't allow non-field instance attributes to be added to 
a frozen dataclass, although if anyone has a strong feeling about it, 
I'd like to hear it.

So, given a frozen dataclass "C" with field names in "field_names", I 
propose changing __setattr__ to be:

def __setattr__(self, name, value):
     if type(self) is C or name in field_names:
         raise FrozenInstanceError(f'cannot assign to field {name!r}')
     super(cls, self).__setattr__(name, value)

In the current 3.7.0a2 implementation of frozen dataclasses, __setattr__ 
always raises. The change is the test and then call to 
super().__setattr__ if it's a derived class. The result is an exception 
if either self is an instance of C, or if self is an instance of a 
derived class, but the attribute being set is a field of C.

So going back to original questions above, my suggestions are:

1. What happens when a frozen dataclass inherits from a non-frozen 
dataclass? The result is a frozen dataclass, and all fields are 
non-writable. No non-fields can be added. This is a reversion to the 
3.7.0a1 behavior.

2. What happens when a non-frozen dataclass inherits from a frozen 
dataclass? The result is a frozen dataclass, and all fields are 
non-writable. No non-fields can be added. This is a reversion to the 
3.7.0a1 behavior. I'd also be okay with this case being an error, and 
you'd have to explicitly mark the derived class as frozen. This is the 
3.7.0a2 behavior.

3. What happens when a non-dataclass inherits from a frozen dataclass? 
The fields that are in the dataclass are non-writable, but new non-field 
attributes can be added. This is new behavior.

4. Can new non-field attributes be created for frozen dataclasses? No. 
This is existing behavior.


I'm hoping this change isn't so large that we can't get it in to 3.7.0a3 
next month.

Eric

From nad at python.org  Wed Feb 28 01:04:32 2018
From: nad at python.org (Ned Deily)
Date: Wed, 28 Feb 2018 01:04:32 -0500
Subject: [Python-Dev] [RELEASE] Python 3.7.0b2 is now available for testing
Message-ID: <66EE6E10-2815-4E53-841C-A8896BFA48A5@python.org>

On behalf of the Python development community and the Python 3.7 release
team, I'm happy to announce the availability of Python 3.7.0b2.  b2 is
the second of four planned beta releases of Python 3.7, the next major
release of Python, and marks the end of the feature development phase
for 3.7.  You can find Python 3.7.0b2 here:

https://www.python.org/downloads/release/python-370b2/

Among the new major new features in Python 3.7 are:

* PEP 538, Coercing the legacy C locale to a UTF-8 based locale
* PEP 539, A New C-API for Thread-Local Storage in CPython
* PEP 540, UTF-8 mode
* PEP 552, Deterministic pyc
* PEP 553, Built-in breakpoint()
* PEP 557, Data Classes
* PEP 560, Core support for typing module and generic types
* PEP 562, Module __getattr__ and __dir__
* PEP 563, Postponed Evaluation of Annotations
* PEP 564, Time functions with nanosecond resolution
* PEP 565, Show DeprecationWarning in __main__
* PEP 567, Context Variables

Please see "What?s New In Python 3.7" for more information.
Additional documentation for these features and for other changes
will be provided during the beta phase.

https://docs.python.org/3.7/whatsnew/3.7.html

Beta releases are intended to give you the opportunity to test new
features and bug fixes and to prepare their projects to support the
new feature release. We strongly encourage you to test your projects
with 3.7 during the beta phase and report issues found to
https://bugs.python.org as soon as possible.

While the release is feature complete entering the beta phase, it is
possible that features may be modified or, in rare cases, deleted up
until the start of the release candidate phase (2018-05-21). Our goal
is have no ABI changes after beta 3 and no code changes after rc1.
To achieve that, it will be extremely important to get as much exposure
for 3.7 as possible during the beta phase.

Attention macOS users: as of b1, there is a new installer variant for
macOS 10.9+ that includes a built-in version of Tcl/Tk 8.6. This
variant is expected to become the default version when 3.7.0 releases.
Check it out! We welcome your feedback.

Please keep in mind that this is a preview release and its use is
not recommended for production environments.

The next planned release of Python 3.7 will be 3.7.0b3, currently
scheduled for 2018-03-26. More information about the release schedule
can be found here:

https://www.python.org/dev/peps/pep-0537/


--
  Ned Deily
  nad at python.org -- []


From armin.rigo at gmail.com  Wed Feb 28 05:49:14 2018
From: armin.rigo at gmail.com (Armin Rigo)
Date: Wed, 28 Feb 2018 11:49:14 +0100
Subject: [Python-Dev] Backward incompatible change about docstring AST
In-Reply-To: <p73q1n$18k$1@blaine.gmane.org>
References: <CAEfz+Twynf8O=sCj40p=GB3VhFLQTYJpHwGKkLJoN1g0OQww=Q@mail.gmail.com>
 <p73q1n$18k$1@blaine.gmane.org>
Message-ID: <CAMSv6X2jVzdfsZAiRKM0cFj8VoM1STOWmcfkT5z6vp0sK94jMw@mail.gmail.com>

Hi,

On 27 February 2018 at 15:32, Serhiy Storchaka <storchaka at gmail.com> wrote:
> 1. CPython and PyPy set different position for multiline strings. PyPy sets
> the position of the start of string, but CPython sets the position of the
> end of the string. A program that utilizes the docstring position needs to
> handle both of these cases.

If that's true it's a PyPy bug.
https://bitbucket.org/pypy/pypy/issues/2767/docstring-position-in-the-ast


A bient?t,

Armin.

From storchaka at gmail.com  Wed Feb 28 06:04:57 2018
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Wed, 28 Feb 2018 13:04:57 +0200
Subject: [Python-Dev] Backward incompatible change about docstring AST
In-Reply-To: <CAMSv6X2jVzdfsZAiRKM0cFj8VoM1STOWmcfkT5z6vp0sK94jMw@mail.gmail.com>
References: <CAEfz+Twynf8O=sCj40p=GB3VhFLQTYJpHwGKkLJoN1g0OQww=Q@mail.gmail.com>
 <p73q1n$18k$1@blaine.gmane.org>
 <CAMSv6X2jVzdfsZAiRKM0cFj8VoM1STOWmcfkT5z6vp0sK94jMw@mail.gmail.com>
Message-ID: <p7628p$hdr$1@blaine.gmane.org>

28.02.18 12:49, Armin Rigo ????:
> On 27 February 2018 at 15:32, Serhiy Storchaka <storchaka at gmail.com> wrote:
>> 1. CPython and PyPy set different position for multiline strings. PyPy sets
>> the position of the start of string, but CPython sets the position of the
>> end of the string. A program that utilizes the docstring position needs to
>> handle both of these cases.
> 
> If that's true it's a PyPy bug.
> https://bitbucket.org/pypy/pypy/issues/2767/docstring-position-in-the-ast

This is rather a CPython bug (somewhere may be an open issue for this).