From ncoghlan at gmail.com  Thu Sep  1 07:19:03 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 1 Sep 2016 21:19:03 +1000
Subject: [Distutils] Accepting PEP 527 (reducing the variety of file types
 supported by PyPI)
Message-ID: <CADiSq7cV1R5yfHgKgHnN43OXtWXM4X1iz34hV9BJUgPavTho6A@mail.gmail.com>

Having reviewed the comments on the PEP 527 thread and the latest
draft of the PEP, I'm now accepting the PEP. This means that:

* supported sdist types will be reduced to .tar.gz and .zip
* projects will need to choose one or the other for future releases
* supported binary formats will be reduced to bdist_wheel and bdist_egg

The problem that Maurits van Rees pointed out with ez_setup.py
currently still specifically looking for a zip file and processing it
accordingly can be resolved by setuptools settling on zip as its sdist
publication format (unless/until ez_setup.py is changed to handle
tar.gz).

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From anntzer.lee at gmail.com  Thu Sep  1 23:30:16 2016
From: anntzer.lee at gmail.com (Antony Lee)
Date: Thu, 1 Sep 2016 20:30:16 -0700
Subject: [Distutils] setup_requires: the obvious option(?)
In-Reply-To: <CAGRr6BGyqEBDRMNsTMRRfMw7Ty=Uc_9R-pfXU-w3Y=Lk_=dKbA@mail.gmail.com>
References: <CAGRr6BG6RanD999ck=sYAPfO4grNku3pFUskE+kOECT-poCPsw@mail.gmail.com>
 <CAOTD34ZKZ2teQWoyZT4Jnp6usTU7n+-088=hYqD4FP5kNtPrvw@mail.gmail.com>
 <CAGRr6BGyqEBDRMNsTMRRfMw7Ty=Uc_9R-pfXU-w3Y=Lk_=dKbA@mail.gmail.com>
Message-ID: <CAGRr6BHayoGL-pBhBzRj78VU6u=YXznVmqdA3=hRzVXXr2yRsA@mail.gmail.com>

>
> Similarly, it wouldn't astonish me if we eventually see an emergent
> practice of people writing pyproject.toml.in files for complex
> projects, in order to move some particular forms of complexity away
> from build time and towards development time - this would be a similar
> practice to folks using autoconf to generate a project's C Makefile.


This actually formulates much better than I could have done the reasons why
I dislike PEP518: it's only going to lead to reinventing the wheel (AKA
autoconf, which is a pretty big wheel to reinvent).
Antony
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160901/03c24e02/attachment.html>

From sylvain.corlay at gmail.com  Fri Sep  2 04:58:27 2016
From: sylvain.corlay at gmail.com (Sylvain Corlay)
Date: Fri, 2 Sep 2016 10:58:27 +0200
Subject: [Distutils] What is the official position on distutils?
In-Reply-To: <CAP1=2W7L70oAim3Fbyw=Qi+D+OnpGn=B_Om22mBGFX2ho0peaA@mail.gmail.com>
References: <CAP1=2W4ZXcC5CO5oqKcrsEhYjRvrYqe7o1mnKnOJ5+sCkh9k8w@mail.gmail.com>
 <CACac1F8NXxGbKCPT8X1cviC1BJQ8MsPSZ4zPsPv6XCeEn3QyMQ@mail.gmail.com>
 <CAK=Phk4XYgn45_2vJqzGciZJKFx4aAdGzMqZQ7o3f3bPHQLH4A@mail.gmail.com>
 <CAP1=2W7L70oAim3Fbyw=Qi+D+OnpGn=B_Om22mBGFX2ho0peaA@mail.gmail.com>
Message-ID: <CAK=Phk5H49zTWCN8pc3f35R_2f269X94maMXk2ay3A-HT3p8gQ@mail.gmail.com>

Hi Brett,

On Mon, Aug 29, 2016 at 9:52 PM, Brett Cannon <brett at python.org> wrote:

>
>
>> As we discussed earlier, even though it is not a concern with C, checking
>> for availability of a compiler flag becomes crucial when building
>> extensions with C++ since new flavors of the language emerge every couple
>> of years now. It is important to be able to output meaningful error
>> messages when the compiler does not support C++[11/14/17] features if they
>> are needed for a given extension. This is a new aspect of the landscape in
>> this area.
>>
>> Finally, adding this method is a very straightforward change. `has_flag`
>> simply comes aside `has_function` as a method of ccompiler. I don't see a
>> more natural place for it. It would be a very weird design decision in my
>> opinion to not add it there, and instead to add it to distutils.ccompiler
>> by monkeypatching it in setuptools.
>>
>
> Honestly I have no comment on your feature, Sylvain, and I'm sorry your
> proposal happens to be the catalyst to this discussion. I'm just trying to
> get a general alignment from the PyPA group as the "distutils problem"
> comes up and has the same points made every time with no general decision
> on how to handle it long-term.
>
> -Brett
>
>

My point here was that I don't think that the proposed feature has much to
do with the concerns that were raised about distutils in general, unless it
is decided that incremental improvements to the library even backward
compatible will not be accepted anymore.

Sylvain
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160902/f3fe3271/attachment.html>

From p.f.moore at gmail.com  Fri Sep  2 05:28:58 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 2 Sep 2016 10:28:58 +0100
Subject: [Distutils] What is the official position on distutils?
In-Reply-To: <CAK=Phk5H49zTWCN8pc3f35R_2f269X94maMXk2ay3A-HT3p8gQ@mail.gmail.com>
References: <CAP1=2W4ZXcC5CO5oqKcrsEhYjRvrYqe7o1mnKnOJ5+sCkh9k8w@mail.gmail.com>
 <CACac1F8NXxGbKCPT8X1cviC1BJQ8MsPSZ4zPsPv6XCeEn3QyMQ@mail.gmail.com>
 <CAK=Phk4XYgn45_2vJqzGciZJKFx4aAdGzMqZQ7o3f3bPHQLH4A@mail.gmail.com>
 <CAP1=2W7L70oAim3Fbyw=Qi+D+OnpGn=B_Om22mBGFX2ho0peaA@mail.gmail.com>
 <CAK=Phk5H49zTWCN8pc3f35R_2f269X94maMXk2ay3A-HT3p8gQ@mail.gmail.com>
Message-ID: <CACac1F8F2OD=pNcbf4QiE==tDNQTROfdLYjJ+-hLyJ+=VdhOFA@mail.gmail.com>

On 2 September 2016 at 09:58, Sylvain Corlay <sylvain.corlay at gmail.com> wrote:
> My point here was that I don't think that the proposed feature has much to
> do with the concerns that were raised about distutils in general, unless it
> is decided that incremental improvements to the library even backward
> compatible will not be accepted anymore.

Agreed. I think your feature is only stalled for distutils by the lack
of people sufficiently comfortable with the code to apply it. The
suggestions to add it to setuptools are more in the way of practical
advice on how to make the feature available, rather than expressions
of a policy that "we don't make changes like that in the stdlib".

Paul

From donald at stufft.io  Fri Sep  2 12:28:31 2016
From: donald at stufft.io (Donald Stufft)
Date: Fri, 2 Sep 2016 12:28:31 -0400
Subject: [Distutils] Accepting PEP 527 (reducing the variety of file
 types supported by PyPI)
In-Reply-To: <CADiSq7cV1R5yfHgKgHnN43OXtWXM4X1iz34hV9BJUgPavTho6A@mail.gmail.com>
References: <CADiSq7cV1R5yfHgKgHnN43OXtWXM4X1iz34hV9BJUgPavTho6A@mail.gmail.com>
Message-ID: <9F6DF5DF-5C49-4274-BC15-8EF43D7BFE5D@stufft.io>

Thanks!

> On Sep 1, 2016, at 7:19 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> Having reviewed the comments on the PEP 527 thread and the latest
> draft of the PEP, I'm now accepting the PEP. This means that:
> 
> * supported sdist types will be reduced to .tar.gz and .zip
> * projects will need to choose one or the other for future releases
> * supported binary formats will be reduced to bdist_wheel and bdist_egg
> 
> The problem that Maurits van Rees pointed out with ez_setup.py
> currently still specifically looking for a zip file and processing it
> accordingly can be resolved by setuptools settling on zip as its sdist
> publication format (unless/until ez_setup.py is changed to handle
> tar.gz).
> 
> Regards,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


?
Donald Stufft




From donald at stufft.io  Fri Sep  2 17:05:43 2016
From: donald at stufft.io (Donald Stufft)
Date: Fri, 2 Sep 2016 17:05:43 -0400
Subject: [Distutils] When can we kill Python 2.6 support?
Message-ID: <40BEEAE9-F798-4972-889A-CDB22993A53F@stufft.io>

The packaging tools generally support 2.6+ and 3.(2|3)+ and that's sort of been
where they've been at for a while now. I would like to think about what we need
to be to start considering Python 2.6 as "too old" to support. In pip we
generally follow a usage based deprecation/removal of supported Pythons but we
don't have any real guidelines for when something is at a low enough usage to
consider it no longer supported and we instead just sort of wait until someone
makes a case that it's "low enough".

This issue tends to impact more than just pip, because once pip drops support
for something people tend to start dropping it across the entire ecosystem and
use pip's no longer supporting it as justification for doing so.

I would like to take a look at Python 2.6 and try and figure out if we're at a
point that we can deprecate and drop it, and if not what is such a point.

Looking at pure usage numbers for "modern" versions of pip (6, 7, and 8) for
downloading from PyPI I see the usage is ~3% of downloads are via Python 2.6.
The only thing lower than Python 2.6 that is still supported is Python 3.3.

Python 2.6 itself has been EOL since 2013-10-29 which is now just about 3 years
ago. It's SSL module is not generally secure and requires the use of additional
installed modules to get it to be so. I believe the only place to get a
Python 2.6 that is "supported" is through the Enterprise-y Linux Distributions
like RHEL/CentOS/etc.

Do we think that a ~3% usage of Python 2.6 and being end-of-life'd for ~3 years
is enough to start deprecating and dropping 2.6? If not what sort of threshold
do we think is enough? It'd be nice to get the albatross of Python 2.6 support
off from around our necks but I'm not sure how others feel. Obviously all of
the existing versions of all of the tooling will still be fully functional so
Python 2.6 users will simply need to not upgrade their tooling to continue to
work, *but* it also means that they will be left out of new packaging features
(and likewise, people can't rely on them if they still wish to support 2.6).

Thoughts?

?
Donald Stufft




From dmertz at continuum.io  Fri Sep  2 17:26:13 2016
From: dmertz at continuum.io (David Mertz)
Date: Fri, 2 Sep 2016 16:26:13 -0500
Subject: [Distutils] When can we kill Python 2.6 support?
In-Reply-To: <40BEEAE9-F798-4972-889A-CDB22993A53F@stufft.io>
References: <40BEEAE9-F798-4972-889A-CDB22993A53F@stufft.io>
Message-ID: <CAL2pxnRqk1CWhH+xZiwwO9GSYj7bcpSP6MhfR8ue1nkuU2z80g@mail.gmail.com>

Kill it with fire!

On Sep 2, 2016 2:06 PM, "Donald Stufft" <donald at stufft.io> wrote:

> The packaging tools generally support 2.6+ and 3.(2|3)+ and that's sort of
> been
> where they've been at for a while now. I would like to think about what we
> need
> to be to start considering Python 2.6 as "too old" to support. In pip we
> generally follow a usage based deprecation/removal of supported Pythons
> but we
> don't have any real guidelines for when something is at a low enough usage
> to
> consider it no longer supported and we instead just sort of wait until
> someone
> makes a case that it's "low enough".
>
> This issue tends to impact more than just pip, because once pip drops
> support
> for something people tend to start dropping it across the entire ecosystem
> and
> use pip's no longer supporting it as justification for doing so.
>
> I would like to take a look at Python 2.6 and try and figure out if we're
> at a
> point that we can deprecate and drop it, and if not what is such a point.
>
> Looking at pure usage numbers for "modern" versions of pip (6, 7, and 8)
> for
> downloading from PyPI I see the usage is ~3% of downloads are via Python
> 2.6.
> The only thing lower than Python 2.6 that is still supported is Python 3.3.
>
> Python 2.6 itself has been EOL since 2013-10-29 which is now just about 3
> years
> ago. It's SSL module is not generally secure and requires the use of
> additional
> installed modules to get it to be so. I believe the only place to get a
> Python 2.6 that is "supported" is through the Enterprise-y Linux
> Distributions
> like RHEL/CentOS/etc.
>
> Do we think that a ~3% usage of Python 2.6 and being end-of-life'd for ~3
> years
> is enough to start deprecating and dropping 2.6? If not what sort of
> threshold
> do we think is enough? It'd be nice to get the albatross of Python 2.6
> support
> off from around our necks but I'm not sure how others feel. Obviously all
> of
> the existing versions of all of the tooling will still be fully functional
> so
> Python 2.6 users will simply need to not upgrade their tooling to continue
> to
> work, *but* it also means that they will be left out of new packaging
> features
> (and likewise, people can't rely on them if they still wish to support
> 2.6).
>
> Thoughts?
>
> ?
> Donald Stufft
>
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160902/8400a826/attachment.html>

From tritium-list at sdamon.com  Fri Sep  2 17:47:18 2016
From: tritium-list at sdamon.com (tritium-list at sdamon.com)
Date: Fri, 2 Sep 2016 17:47:18 -0400
Subject: [Distutils] When can we kill Python 2.6 support?
In-Reply-To: <40BEEAE9-F798-4972-889A-CDB22993A53F@stufft.io>
References: <40BEEAE9-F798-4972-889A-CDB22993A53F@stufft.io>
Message-ID: <01af01d20563$93dc5f80$bb951e80$@hotmail.com>

Nick might have something better to say about this, but I don?t think catching enterprise-y linux distros like RHEL out of the blue is a good way to go, so even if we decide right now to drop 2.6 support, it shouldn?t actually ship with breaking changes for like... 3 months?  Maybe a little more or little less.

> -----Original Message-----
> From: Distutils-SIG [mailto:distutils-sig-bounces+tritium-
> list=sdamon.com at python.org] On Behalf Of Donald Stufft
> Sent: Friday, September 2, 2016 5:06 PM
> To: distutils sig <distutils-sig at python.org>
> Subject: [Distutils] When can we kill Python 2.6 support?
> 
> The packaging tools generally support 2.6+ and 3.(2|3)+ and that's sort of
> been
> where they've been at for a while now. I would like to think about what we
> need
> to be to start considering Python 2.6 as "too old" to support. In pip we
> generally follow a usage based deprecation/removal of supported Pythons
> but we
> don't have any real guidelines for when something is at a low enough usage
> to
> consider it no longer supported and we instead just sort of wait until
> someone
> makes a case that it's "low enough".
> 
> This issue tends to impact more than just pip, because once pip drops
> support
> for something people tend to start dropping it across the entire ecosystem
> and
> use pip's no longer supporting it as justification for doing so.
> 
> I would like to take a look at Python 2.6 and try and figure out if we're at a
> point that we can deprecate and drop it, and if not what is such a point.
> 
> Looking at pure usage numbers for "modern" versions of pip (6, 7, and 8) for
> downloading from PyPI I see the usage is ~3% of downloads are via Python
> 2.6.
> The only thing lower than Python 2.6 that is still supported is Python 3.3.
> 
> Python 2.6 itself has been EOL since 2013-10-29 which is now just about 3
> years
> ago. It's SSL module is not generally secure and requires the use of additional
> installed modules to get it to be so. I believe the only place to get a
> Python 2.6 that is "supported" is through the Enterprise-y Linux Distributions
> like RHEL/CentOS/etc.
> 
> Do we think that a ~3% usage of Python 2.6 and being end-of-life'd for ~3
> years
> is enough to start deprecating and dropping 2.6? If not what sort of threshold
> do we think is enough? It'd be nice to get the albatross of Python 2.6 support
> off from around our necks but I'm not sure how others feel. Obviously all of
> the existing versions of all of the tooling will still be fully functional so
> Python 2.6 users will simply need to not upgrade their tooling to continue to
> work, *but* it also means that they will be left out of new packaging features
> (and likewise, people can't rely on them if they still wish to support 2.6).
> 
> Thoughts?
> 
> ?
> Donald Stufft
> 
> 
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


From donald at stufft.io  Fri Sep  2 18:18:02 2016
From: donald at stufft.io (Donald Stufft)
Date: Fri, 2 Sep 2016 18:18:02 -0400
Subject: [Distutils] When can we kill Python 2.6 support?
In-Reply-To: <01af01d20563$93dc5f80$bb951e80$@hotmail.com>
References: <40BEEAE9-F798-4972-889A-CDB22993A53F@stufft.io>
 <01af01d20563$93dc5f80$bb951e80$@hotmail.com>
Message-ID: <CB3021B1-7A49-4A10-8195-15B864B83985@stufft.io>


> On Sep 2, 2016, at 5:47 PM, tritium-list at sdamon.com wrote:
> 
> Nick might have something better to say about this, but I don?t think catching enterprise-y linux distros like RHEL out of the blue is a good way to go, so even if we decide right now to drop 2.6 support, it shouldn?t actually ship with breaking changes for like... 3 months?  Maybe a little more or little less.


Right, I don?t specifically mean immediately ejecting support, but rather deprecate and then drop along whatever the normal timeline is for project (and stop worrying about 2.6 for new stuff).

?
Donald Stufft




From tritium-list at sdamon.com  Fri Sep  2 23:16:50 2016
From: tritium-list at sdamon.com (tritium-list at sdamon.com)
Date: Fri, 2 Sep 2016 23:16:50 -0400
Subject: [Distutils] When can we kill Python 2.6 support?
In-Reply-To: <CB3021B1-7A49-4A10-8195-15B864B83985@stufft.io>
References: <40BEEAE9-F798-4972-889A-CDB22993A53F@stufft.io>
 <01af01d20563$93dc5f80$bb951e80$@hotmail.com>
 <CB3021B1-7A49-4A10-8195-15B864B83985@stufft.io>
Message-ID: <01d401d20591$9d332fe0$d7998fa0$@hotmail.com>

Therein lies my suggestion.  Pick a date, announce, give 3 months, then stop testing on 2.6

> -----Original Message-----
> From: Donald Stufft [mailto:donald at stufft.io]
> Sent: Friday, September 2, 2016 6:18 PM
> To: tritium-list at sdamon.com
> Cc: distutils sig <distutils-sig at python.org>
> Subject: Re: [Distutils] When can we kill Python 2.6 support?
> 
> 
> > On Sep 2, 2016, at 5:47 PM, tritium-list at sdamon.com wrote:
> >
> > Nick might have something better to say about this, but I don?t think
> catching enterprise-y linux distros like RHEL out of the blue is a good way to
> go, so even if we decide right now to drop 2.6 support, it shouldn?t actually
> ship with breaking changes for like... 3 months?  Maybe a little more or little
> less.
> 
> 
> Right, I don?t specifically mean immediately ejecting support, but rather
> deprecate and then drop along whatever the normal timeline is for project
> (and stop worrying about 2.6 for new stuff).
> 
> ?
> Donald Stufft
> 
> 



From ncoghlan at gmail.com  Sat Sep  3 00:51:40 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 3 Sep 2016 14:51:40 +1000
Subject: [Distutils] setup_requires: the obvious option(?)
In-Reply-To: <CAGRr6BHayoGL-pBhBzRj78VU6u=YXznVmqdA3=hRzVXXr2yRsA@mail.gmail.com>
References: <CAGRr6BG6RanD999ck=sYAPfO4grNku3pFUskE+kOECT-poCPsw@mail.gmail.com>
 <CAOTD34ZKZ2teQWoyZT4Jnp6usTU7n+-088=hYqD4FP5kNtPrvw@mail.gmail.com>
 <CAGRr6BGyqEBDRMNsTMRRfMw7Ty=Uc_9R-pfXU-w3Y=Lk_=dKbA@mail.gmail.com>
 <CAGRr6BHayoGL-pBhBzRj78VU6u=YXznVmqdA3=hRzVXXr2yRsA@mail.gmail.com>
Message-ID: <CADiSq7fst=ZXAV-WBVM964Hpq4Qpn2gJxuOCKaXVsrE1A5xNPw@mail.gmail.com>

On 2 September 2016 at 13:30, Antony Lee <anntzer.lee at gmail.com> wrote:
>> Similarly, it wouldn't astonish me if we eventually see an emergent
>> practice of people writing pyproject.toml.in files for complex
>> projects, in order to move some particular forms of complexity away
>> from build time and towards development time - this would be a similar
>> practice to folks using autoconf to generate a project's C Makefile.
>
> This actually formulates much better than I could have done the reasons why
> I dislike PEP518: it's only going to lead to reinventing the wheel (AKA
> autoconf, which is a pretty big wheel to reinvent).

Unlike autoconf, we don't need to support building arbitrary C/C++
projects - rather, we just want people to have a backend independent
way to tell Python-centric toolchains how to invoke their *existing*
build system (whether that's autoconf/make, CMake, Scons, waf, Meson,
yotta, etc).

It's the current *lack* of that ability to readily integrate with
existing build tools (whether written in Python or not) that prompts
people to reinvent the world.

Defining and support PEP 518 means that a possible future workflow for
autoconf using projects would be something like:

    $ ./configure && make pyproject.toml && pip install -e .

In that kind of scenario, the "tell Python tools how to build our
Python bindings" file becomes just another output of a project's
existing build system, which is entirely feasible with a well
documented static format, but impractical with the current
underspecified and underdocumented setup.py based approach.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Sep  3 01:06:03 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 3 Sep 2016 15:06:03 +1000
Subject: [Distutils] What is the official position on distutils?
In-Reply-To: <CACac1F8F2OD=pNcbf4QiE==tDNQTROfdLYjJ+-hLyJ+=VdhOFA@mail.gmail.com>
References: <CAP1=2W4ZXcC5CO5oqKcrsEhYjRvrYqe7o1mnKnOJ5+sCkh9k8w@mail.gmail.com>
 <CACac1F8NXxGbKCPT8X1cviC1BJQ8MsPSZ4zPsPv6XCeEn3QyMQ@mail.gmail.com>
 <CAK=Phk4XYgn45_2vJqzGciZJKFx4aAdGzMqZQ7o3f3bPHQLH4A@mail.gmail.com>
 <CAP1=2W7L70oAim3Fbyw=Qi+D+OnpGn=B_Om22mBGFX2ho0peaA@mail.gmail.com>
 <CAK=Phk5H49zTWCN8pc3f35R_2f269X94maMXk2ay3A-HT3p8gQ@mail.gmail.com>
 <CACac1F8F2OD=pNcbf4QiE==tDNQTROfdLYjJ+-hLyJ+=VdhOFA@mail.gmail.com>
Message-ID: <CADiSq7e4GewDkAEJgY=P-H+RedzPxS_j1WDA8wkXPATj-GmOYg@mail.gmail.com>

On 2 September 2016 at 19:28, Paul Moore <p.f.moore at gmail.com> wrote:
> On 2 September 2016 at 09:58, Sylvain Corlay <sylvain.corlay at gmail.com> wrote:
>> My point here was that I don't think that the proposed feature has much to
>> do with the concerns that were raised about distutils in general, unless it
>> is decided that incremental improvements to the library even backward
>> compatible will not be accepted anymore.
>
> Agreed. I think your feature is only stalled for distutils by the lack
> of people sufficiently comfortable with the code to apply it. The
> suggestions to add it to setuptools are more in the way of practical
> advice on how to make the feature available, rather than expressions
> of a policy that "we don't make changes like that in the stdlib".

However, one of the other consequences of the status quo is that if
Jason's comfortable with a change for setuptools, there's very rarely
going to be anyone that will argue with him if he also considers it a
suitable addition to the next version of distutils :)

Since Jason's primary involvement in distutils-sig & PyPA is as the
lead setuptools maintainer, it's easy for folks to be unaware of the
fact that he's a distutils maintainer as well.

So perhaps that's what we should adopt as the official distutils-sig
policy? Any proposed distutils changes should *always* go into
setuptools, as that way they're available for all currently supported
Python versions, and then it's up to the setuptools project to
escalate changes or change proposals for stdlib inclusion when they
consider that an appropriate step.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Sep  3 02:03:00 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 3 Sep 2016 16:03:00 +1000
Subject: [Distutils] When can we kill Python 2.6 support?
In-Reply-To: <01af01d20563$93dc5f80$bb951e80$@hotmail.com>
References: <40BEEAE9-F798-4972-889A-CDB22993A53F@stufft.io>
 <01af01d20563$93dc5f80$bb951e80$@hotmail.com>
Message-ID: <CADiSq7e0tnq_qSZ4_th-_F7u4tTiKthU+rkOgcA=ffBBjBHJMQ@mail.gmail.com>

On 3 September 2016 at 07:47,  <tritium-list at sdamon.com> wrote:
> Nick might have something better to say about this, but I don?t think catching enterprise-y linux distros like RHEL out of the blue is a good way to go, so even if we decide right now to drop 2.6 support, it shouldn?t actually ship with breaking changes for like... 3 months?  Maybe a little more or little less.

It won't (or at least shouldn't) be out of the blue:
http://www.curiousefficiency.org/posts/2015/04/stop-supporting-python26.html
;)

>From a specifically Red Hat perspective, it's probably worth
mentioning that Red Hat doesn't commercially support pip in
conjunction with any version of Python 2.6 - we only provide a Red Hat
supported pip for the Software Collections runtimes, and the lowest
version of Python we provide as an SCL is 2.7.

My "Developing in Python on Red Hat Platforms" talk with Graham
Dumpleton in the DevNation/Red Hat Summit cross-over track this year
[1] was also basically telling people "Stop using the system Python to
run your own stuff, especially on RHEL 6", and advocating the use of
higher level alternatives like OpenShift v3, Software Collections, and
Ansible instead (depending on whether you were writing network
services, client applications, or system administration scripts).

So if pip 9.0 were to deprecate Python 2.6 support, I think that would
be an entirely reasonable decision for PyPA to make from an upstream
maintainability perspective - if folks genuinely want long term
infrastructure software maintenance for Python projects, they need to
be pestering commercial redistributors about funding that, rather than
assuming volunteers will always deliver it for free even when they're
running newer Python versions themselves.

The deprecation cycle would then mainly be about giving pip's
*community* users a heads up that CI testing against Python 2.6 was
being dropped, with Python 2.7+ likely becoming a requirement in a
future version.

Cheers,
Nick.

[1] http://www.slideshare.net/ncoghlan_dev/developing-in-python-on-red-hat-platforms-devnation-2016

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From brett at python.org  Sat Sep  3 14:05:38 2016
From: brett at python.org (Brett Cannon)
Date: Sat, 03 Sep 2016 18:05:38 +0000
Subject: [Distutils] What is the official position on distutils?
In-Reply-To: <CADiSq7e4GewDkAEJgY=P-H+RedzPxS_j1WDA8wkXPATj-GmOYg@mail.gmail.com>
References: <CAP1=2W4ZXcC5CO5oqKcrsEhYjRvrYqe7o1mnKnOJ5+sCkh9k8w@mail.gmail.com>
 <CACac1F8NXxGbKCPT8X1cviC1BJQ8MsPSZ4zPsPv6XCeEn3QyMQ@mail.gmail.com>
 <CAK=Phk4XYgn45_2vJqzGciZJKFx4aAdGzMqZQ7o3f3bPHQLH4A@mail.gmail.com>
 <CAP1=2W7L70oAim3Fbyw=Qi+D+OnpGn=B_Om22mBGFX2ho0peaA@mail.gmail.com>
 <CAK=Phk5H49zTWCN8pc3f35R_2f269X94maMXk2ay3A-HT3p8gQ@mail.gmail.com>
 <CACac1F8F2OD=pNcbf4QiE==tDNQTROfdLYjJ+-hLyJ+=VdhOFA@mail.gmail.com>
 <CADiSq7e4GewDkAEJgY=P-H+RedzPxS_j1WDA8wkXPATj-GmOYg@mail.gmail.com>
Message-ID: <CAP1=2W70CgLPqbtmiEKW8hVzDedsEzC4wrBLwN70Sr2LW-Gkow@mail.gmail.com>

On Fri, 2 Sep 2016 at 22:06 Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 2 September 2016 at 19:28, Paul Moore <p.f.moore at gmail.com> wrote:
> > On 2 September 2016 at 09:58, Sylvain Corlay <sylvain.corlay at gmail.com>
> wrote:
> >> My point here was that I don't think that the proposed feature has much
> to
> >> do with the concerns that were raised about distutils in general,
> unless it
> >> is decided that incremental improvements to the library even backward
> >> compatible will not be accepted anymore.
> >
> > Agreed. I think your feature is only stalled for distutils by the lack
> > of people sufficiently comfortable with the code to apply it. The
> > suggestions to add it to setuptools are more in the way of practical
> > advice on how to make the feature available, rather than expressions
> > of a policy that "we don't make changes like that in the stdlib".
>
> However, one of the other consequences of the status quo is that if
> Jason's comfortable with a change for setuptools, there's very rarely
> going to be anyone that will argue with him if he also considers it a
> suitable addition to the next version of distutils :)
>
> Since Jason's primary involvement in distutils-sig & PyPA is as the
> lead setuptools maintainer, it's easy for folks to be unaware of the
> fact that he's a distutils maintainer as well.
>
> So perhaps that's what we should adopt as the official distutils-sig
> policy? Any proposed distutils changes should *always* go into
> setuptools, as that way they're available for all currently supported
> Python versions, and then it's up to the setuptools project to
> escalate changes or change proposals for stdlib inclusion when they
> consider that an appropriate step.
>

If Jason is up for the responsibility that seems like a reasonable approach
to take. It also helps test out features in setuptools first before
upstreaming it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160903/3f2d3fd7/attachment.html>

From brett at python.org  Sat Sep  3 14:13:40 2016
From: brett at python.org (Brett Cannon)
Date: Sat, 03 Sep 2016 18:13:40 +0000
Subject: [Distutils] When can we kill Python 2.6 support?
In-Reply-To: <40BEEAE9-F798-4972-889A-CDB22993A53F@stufft.io>
References: <40BEEAE9-F798-4972-889A-CDB22993A53F@stufft.io>
Message-ID: <CAP1=2W6xVf7fdq-Uwe=4RKq4px8HrTZTepciLt7Wr=F36D2oCQ@mail.gmail.com>

I think the fact that Python 2.6 is past EOL means it's definitely up for
consideration. As for the 3% usage, as a trite comparison that's the amount
of scientists who deny climate change. So IMO that suggests 2.6 is not used
enough to burden PyPA with the maintenance and those who still want to use
it can take over maintaining 2.6 compatibility.

On Fri, 2 Sep 2016 at 14:06 Donald Stufft <donald at stufft.io> wrote:

> The packaging tools generally support 2.6+ and 3.(2|3)+ and that's sort of
> been
> where they've been at for a while now. I would like to think about what we
> need
> to be to start considering Python 2.6 as "too old" to support. In pip we
> generally follow a usage based deprecation/removal of supported Pythons
> but we
> don't have any real guidelines for when something is at a low enough usage
> to
> consider it no longer supported and we instead just sort of wait until
> someone
> makes a case that it's "low enough".
>
> This issue tends to impact more than just pip, because once pip drops
> support
> for something people tend to start dropping it across the entire ecosystem
> and
> use pip's no longer supporting it as justification for doing so.
>
> I would like to take a look at Python 2.6 and try and figure out if we're
> at a
> point that we can deprecate and drop it, and if not what is such a point.
>
> Looking at pure usage numbers for "modern" versions of pip (6, 7, and 8)
> for
> downloading from PyPI I see the usage is ~3% of downloads are via Python
> 2.6.
> The only thing lower than Python 2.6 that is still supported is Python 3.3.
>
> Python 2.6 itself has been EOL since 2013-10-29 which is now just about 3
> years
> ago. It's SSL module is not generally secure and requires the use of
> additional
> installed modules to get it to be so. I believe the only place to get a
> Python 2.6 that is "supported" is through the Enterprise-y Linux
> Distributions
> like RHEL/CentOS/etc.
>
> Do we think that a ~3% usage of Python 2.6 and being end-of-life'd for ~3
> years
> is enough to start deprecating and dropping 2.6? If not what sort of
> threshold
> do we think is enough? It'd be nice to get the albatross of Python 2.6
> support
> off from around our necks but I'm not sure how others feel. Obviously all
> of
> the existing versions of all of the tooling will still be fully functional
> so
> Python 2.6 users will simply need to not upgrade their tooling to continue
> to
> work, *but* it also means that they will be left out of new packaging
> features
> (and likewise, people can't rely on them if they still wish to support
> 2.6).
>
> Thoughts?
>
> ?
> Donald Stufft
>
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160903/80340730/attachment.html>

From dholth at gmail.com  Sat Sep  3 16:36:59 2016
From: dholth at gmail.com (Daniel Holth)
Date: Sat, 03 Sep 2016 20:36:59 +0000
Subject: [Distutils] When can we kill Python 2.6 support?
In-Reply-To: <CAP1=2W6xVf7fdq-Uwe=4RKq4px8HrTZTepciLt7Wr=F36D2oCQ@mail.gmail.com>
References: <40BEEAE9-F798-4972-889A-CDB22993A53F@stufft.io>
 <CAP1=2W6xVf7fdq-Uwe=4RKq4px8HrTZTepciLt7Wr=F36D2oCQ@mail.gmail.com>
Message-ID: <CAG8k2+5sffUvL+YBrEPjQzUS8URZGCvY62zUhbqCT_xBsNxxXg@mail.gmail.com>

It would be convenient to drop 2.6 in wheel too.

On Sat, Sep 3, 2016, 14:14 Brett Cannon <brett at python.org> wrote:

> I think the fact that Python 2.6 is past EOL means it's definitely up for
> consideration. As for the 3% usage, as a trite comparison that's the amount
> of scientists who deny climate change. So IMO that suggests 2.6 is not used
> enough to burden PyPA with the maintenance and those who still want to use
> it can take over maintaining 2.6 compatibility.
>
>
> On Fri, 2 Sep 2016 at 14:06 Donald Stufft <donald at stufft.io> wrote:
>
>> The packaging tools generally support 2.6+ and 3.(2|3)+ and that's sort
>> of been
>> where they've been at for a while now. I would like to think about what
>> we need
>> to be to start considering Python 2.6 as "too old" to support. In pip we
>> generally follow a usage based deprecation/removal of supported Pythons
>> but we
>> don't have any real guidelines for when something is at a low enough
>> usage to
>> consider it no longer supported and we instead just sort of wait until
>> someone
>> makes a case that it's "low enough".
>>
>> This issue tends to impact more than just pip, because once pip drops
>> support
>> for something people tend to start dropping it across the entire
>> ecosystem and
>> use pip's no longer supporting it as justification for doing so.
>>
>> I would like to take a look at Python 2.6 and try and figure out if we're
>> at a
>> point that we can deprecate and drop it, and if not what is such a point.
>>
>> Looking at pure usage numbers for "modern" versions of pip (6, 7, and 8)
>> for
>> downloading from PyPI I see the usage is ~3% of downloads are via Python
>> 2.6.
>> The only thing lower than Python 2.6 that is still supported is Python
>> 3.3.
>>
>> Python 2.6 itself has been EOL since 2013-10-29 which is now just about 3
>> years
>> ago. It's SSL module is not generally secure and requires the use of
>> additional
>> installed modules to get it to be so. I believe the only place to get a
>> Python 2.6 that is "supported" is through the Enterprise-y Linux
>> Distributions
>> like RHEL/CentOS/etc.
>>
>> Do we think that a ~3% usage of Python 2.6 and being end-of-life'd for ~3
>> years
>> is enough to start deprecating and dropping 2.6? If not what sort of
>> threshold
>> do we think is enough? It'd be nice to get the albatross of Python 2.6
>> support
>> off from around our necks but I'm not sure how others feel. Obviously all
>> of
>> the existing versions of all of the tooling will still be fully
>> functional so
>> Python 2.6 users will simply need to not upgrade their tooling to
>> continue to
>> work, *but* it also means that they will be left out of new packaging
>> features
>> (and likewise, people can't rely on them if they still wish to support
>> 2.6).
>>
>> Thoughts?
>>
>> ?
>> Donald Stufft
>>
>>
>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160903/dc34fe21/attachment-0001.html>

From sylvain.corlay at gmail.com  Sat Sep  3 16:44:12 2016
From: sylvain.corlay at gmail.com (Sylvain Corlay)
Date: Sat, 3 Sep 2016 22:44:12 +0200
Subject: [Distutils] What is the official position on distutils?
In-Reply-To: <CAP1=2W70CgLPqbtmiEKW8hVzDedsEzC4wrBLwN70Sr2LW-Gkow@mail.gmail.com>
References: <CAP1=2W4ZXcC5CO5oqKcrsEhYjRvrYqe7o1mnKnOJ5+sCkh9k8w@mail.gmail.com>
 <CACac1F8NXxGbKCPT8X1cviC1BJQ8MsPSZ4zPsPv6XCeEn3QyMQ@mail.gmail.com>
 <CAK=Phk4XYgn45_2vJqzGciZJKFx4aAdGzMqZQ7o3f3bPHQLH4A@mail.gmail.com>
 <CAP1=2W7L70oAim3Fbyw=Qi+D+OnpGn=B_Om22mBGFX2ho0peaA@mail.gmail.com>
 <CAK=Phk5H49zTWCN8pc3f35R_2f269X94maMXk2ay3A-HT3p8gQ@mail.gmail.com>
 <CACac1F8F2OD=pNcbf4QiE==tDNQTROfdLYjJ+-hLyJ+=VdhOFA@mail.gmail.com>
 <CADiSq7e4GewDkAEJgY=P-H+RedzPxS_j1WDA8wkXPATj-GmOYg@mail.gmail.com>
 <CAP1=2W70CgLPqbtmiEKW8hVzDedsEzC4wrBLwN70Sr2LW-Gkow@mail.gmail.com>
Message-ID: <CAK=Phk49Yu-7-TUrxk5ky2_NpnMKOmmj1o5QsRdpUUNbsE_U2Q@mail.gmail.com>

Hi Brett,

On Sat, Sep 3, 2016 at 8:05 PM, Brett Cannon <brett at python.org> wrote:
>
>
> If Jason is up for the responsibility that seems like a reasonable
> approach to take. It also helps test out features in setuptools first
> before upstreaming it.
>
>
How do you see `has_flag` get into setuptools? By monkey-patching
distutils' ccompiler to have it aside of `has_function` when setuptools is
imported?

I find really weird the idea of adding this in a convoluted fashion instead
of allowing incremental improvement of distutils.

Sylvain
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160903/3c2c70ca/attachment.html>

From ralf.gommers at gmail.com  Sat Sep  3 17:30:10 2016
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 4 Sep 2016 09:30:10 +1200
Subject: [Distutils] What is the official position on distutils?
In-Reply-To: <CADiSq7e4GewDkAEJgY=P-H+RedzPxS_j1WDA8wkXPATj-GmOYg@mail.gmail.com>
References: <CAP1=2W4ZXcC5CO5oqKcrsEhYjRvrYqe7o1mnKnOJ5+sCkh9k8w@mail.gmail.com>
 <CACac1F8NXxGbKCPT8X1cviC1BJQ8MsPSZ4zPsPv6XCeEn3QyMQ@mail.gmail.com>
 <CAK=Phk4XYgn45_2vJqzGciZJKFx4aAdGzMqZQ7o3f3bPHQLH4A@mail.gmail.com>
 <CAP1=2W7L70oAim3Fbyw=Qi+D+OnpGn=B_Om22mBGFX2ho0peaA@mail.gmail.com>
 <CAK=Phk5H49zTWCN8pc3f35R_2f269X94maMXk2ay3A-HT3p8gQ@mail.gmail.com>
 <CACac1F8F2OD=pNcbf4QiE==tDNQTROfdLYjJ+-hLyJ+=VdhOFA@mail.gmail.com>
 <CADiSq7e4GewDkAEJgY=P-H+RedzPxS_j1WDA8wkXPATj-GmOYg@mail.gmail.com>
Message-ID: <CABL7CQjrn_P7ayrxKWPQ8yL19qvC=zo-1zfQy4NyV8XkD_AWuQ@mail.gmail.com>

On Sat, Sep 3, 2016 at 5:06 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 2 September 2016 at 19:28, Paul Moore <p.f.moore at gmail.com> wrote:
> > On 2 September 2016 at 09:58, Sylvain Corlay <sylvain.corlay at gmail.com>
> wrote:
> >> My point here was that I don't think that the proposed feature has much
> to
> >> do with the concerns that were raised about distutils in general,
> unless it
> >> is decided that incremental improvements to the library even backward
> >> compatible will not be accepted anymore.
> >
> > Agreed. I think your feature is only stalled for distutils by the lack
> > of people sufficiently comfortable with the code to apply it. The
> > suggestions to add it to setuptools are more in the way of practical
> > advice on how to make the feature available, rather than expressions
> > of a policy that "we don't make changes like that in the stdlib".
>
> However, one of the other consequences of the status quo is that if
> Jason's comfortable with a change for setuptools, there's very rarely
> going to be anyone that will argue with him if he also considers it a
> suitable addition to the next version of distutils :)
>
> Since Jason's primary involvement in distutils-sig & PyPA is as the
> lead setuptools maintainer, it's easy for folks to be unaware of the
> fact that he's a distutils maintainer as well.
>
> So perhaps that's what we should adopt as the official distutils-sig
> policy? Any proposed distutils changes should *always* go into
> setuptools, as that way they're available for all currently supported
> Python versions,


and better maintained, and easier to fix if there's bugs, etc.


> and then it's up to the setuptools project to
> escalate changes or change proposals for stdlib inclusion when they
> consider that an appropriate step.
>

+1. clear and pragmatic policy.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160904/6009914f/attachment.html>

From ncoghlan at gmail.com  Sun Sep  4 05:17:59 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 4 Sep 2016 19:17:59 +1000
Subject: [Distutils] What is the official position on distutils?
In-Reply-To: <CAK=Phk49Yu-7-TUrxk5ky2_NpnMKOmmj1o5QsRdpUUNbsE_U2Q@mail.gmail.com>
References: <CAP1=2W4ZXcC5CO5oqKcrsEhYjRvrYqe7o1mnKnOJ5+sCkh9k8w@mail.gmail.com>
 <CACac1F8NXxGbKCPT8X1cviC1BJQ8MsPSZ4zPsPv6XCeEn3QyMQ@mail.gmail.com>
 <CAK=Phk4XYgn45_2vJqzGciZJKFx4aAdGzMqZQ7o3f3bPHQLH4A@mail.gmail.com>
 <CAP1=2W7L70oAim3Fbyw=Qi+D+OnpGn=B_Om22mBGFX2ho0peaA@mail.gmail.com>
 <CAK=Phk5H49zTWCN8pc3f35R_2f269X94maMXk2ay3A-HT3p8gQ@mail.gmail.com>
 <CACac1F8F2OD=pNcbf4QiE==tDNQTROfdLYjJ+-hLyJ+=VdhOFA@mail.gmail.com>
 <CADiSq7e4GewDkAEJgY=P-H+RedzPxS_j1WDA8wkXPATj-GmOYg@mail.gmail.com>
 <CAP1=2W70CgLPqbtmiEKW8hVzDedsEzC4wrBLwN70Sr2LW-Gkow@mail.gmail.com>
 <CAK=Phk49Yu-7-TUrxk5ky2_NpnMKOmmj1o5QsRdpUUNbsE_U2Q@mail.gmail.com>
Message-ID: <CADiSq7ejkgmdsizuopsnk4k2fZAe=V8Vv2nk=NDO7pb2k7AJkw@mail.gmail.com>

On 4 September 2016 at 06:44, Sylvain Corlay <sylvain.corlay at gmail.com> wrote:
> Hi Brett,
>
> On Sat, Sep 3, 2016 at 8:05 PM, Brett Cannon <brett at python.org> wrote:
>>
>>
>> If Jason is up for the responsibility that seems like a reasonable
>> approach to take. It also helps test out features in setuptools first before
>> upstreaming it.
>>
>
> How do you see `has_flag` get into setuptools? By monkey-patching distutils'
> ccompiler to have it aside of `has_function` when setuptools is imported?
>
> I find really weird the idea of adding this in a convoluted fashion instead
> of allowing incremental improvement of distutils.

The change to distutils would still be a plain patch to distutils, it
would just be accepted at the API design level in setuptools first.

The problem you're running into right now isn't a technical one - it's
that there isn't anyone that currently feels like they have sufficient
design authority over the distutils API to accept your proposal, hence
Brett starting this thread to address that underlying recurring
question, rather than the specifics of your change.

Jason *definitely* has that design authority over setuptools though,
and will be tasked with making any API additions available on older
versions of Python via setuptools regardless of what policy we adopt
for distutils maintenance, so if he's amenable to the idea, it makes
sense to me to invert the order we ask those questions: add it to
setuptools first, and then add things to distutils where we feel
they're sufficiently stable to not need the benefit of the faster
setuptools update cycle.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From steve.dower at python.org  Sun Sep  4 10:02:04 2016
From: steve.dower at python.org (Steve Dower)
Date: Sun, 4 Sep 2016 07:02:04 -0700
Subject: [Distutils] What is the official position on distutils?
In-Reply-To: <CADiSq7ejkgmdsizuopsnk4k2fZAe=V8Vv2nk=NDO7pb2k7AJkw@mail.gmail.com>
References: <CAP1=2W4ZXcC5CO5oqKcrsEhYjRvrYqe7o1mnKnOJ5+sCkh9k8w@mail.gmail.com>
 <CACac1F8NXxGbKCPT8X1cviC1BJQ8MsPSZ4zPsPv6XCeEn3QyMQ@mail.gmail.com>
 <CAK=Phk4XYgn45_2vJqzGciZJKFx4aAdGzMqZQ7o3f3bPHQLH4A@mail.gmail.com>
 <CAP1=2W7L70oAim3Fbyw=Qi+D+OnpGn=B_Om22mBGFX2ho0peaA@mail.gmail.com>
 <CAK=Phk5H49zTWCN8pc3f35R_2f269X94maMXk2ay3A-HT3p8gQ@mail.gmail.com>
 <CACac1F8F2OD=pNcbf4QiE==tDNQTROfdLYjJ+-hLyJ+=VdhOFA@mail.gmail.com>
 <CADiSq7e4GewDkAEJgY=P-H+RedzPxS_j1WDA8wkXPATj-GmOYg@mail.gmail.com>
 <CAP1=2W70CgLPqbtmiEKW8hVzDedsEzC4wrBLwN70Sr2LW-Gkow@mail.gmail.com>
 <CAK=Phk49Yu-7-TUrxk5ky2_NpnMKOmmj1o5QsRdpUUNbsE_U2Q@mail.gmail.com>
 <CADiSq7ejkgmdsizuopsnk4k2fZAe=V8Vv2nk=NDO7pb2k7AJkw@mail.gmail.com>
Message-ID: <E1bgY0i-00056Z-O6@se2-syd.hostedmail.net.au>

"add it to setuptools first, and then add things to distutils where we feel they're sufficiently stable to not need the benefit of the faster setuptools update cycle"

I nominate this as the official policy on API changes for distutils, with regular maintenance mode policies applying to anything else.

Top-posted from my Windows Phone

-----Original Message-----
From: "Nick Coghlan" <ncoghlan at gmail.com>
Sent: ?9/?4/?2016 2:19
To: "Sylvain Corlay" <sylvain.corlay at gmail.com>
Cc: "distutils sig" <distutils-sig at python.org>
Subject: Re: [Distutils] What is the official position on distutils?

On 4 September 2016 at 06:44, Sylvain Corlay <sylvain.corlay at gmail.com> wrote:
> Hi Brett,
>
> On Sat, Sep 3, 2016 at 8:05 PM, Brett Cannon <brett at python.org> wrote:
>>
>>
>> If Jason is up for the responsibility that seems like a reasonable
>> approach to take. It also helps test out features in setuptools first before
>> upstreaming it.
>>
>
> How do you see `has_flag` get into setuptools? By monkey-patching distutils'
> ccompiler to have it aside of `has_function` when setuptools is imported?
>
> I find really weird the idea of adding this in a convoluted fashion instead
> of allowing incremental improvement of distutils.

The change to distutils would still be a plain patch to distutils, it
would just be accepted at the API design level in setuptools first.

The problem you're running into right now isn't a technical one - it's
that there isn't anyone that currently feels like they have sufficient
design authority over the distutils API to accept your proposal, hence
Brett starting this thread to address that underlying recurring
question, rather than the specifics of your change.

Jason *definitely* has that design authority over setuptools though,
and will be tasked with making any API additions available on older
versions of Python via setuptools regardless of what policy we adopt
for distutils maintenance, so if he's amenable to the idea, it makes
sense to me to invert the order we ask those questions: add it to
setuptools first, and then add things to distutils where we feel
they're sufficiently stable to not need the benefit of the faster
setuptools update cycle.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG at python.org
https://mail.python.org/mailman/listinfo/distutils-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160904/d697f154/attachment-0001.html>

From sylvain.corlay at gmail.com  Sun Sep  4 10:27:15 2016
From: sylvain.corlay at gmail.com (Sylvain Corlay)
Date: Sun, 4 Sep 2016 16:27:15 +0200
Subject: [Distutils] What is the official position on distutils?
In-Reply-To: <E1bgY0i-00056Y-Hi@se2-syd.hostedmail.net.au>
References: <CAP1=2W4ZXcC5CO5oqKcrsEhYjRvrYqe7o1mnKnOJ5+sCkh9k8w@mail.gmail.com>
 <CACac1F8NXxGbKCPT8X1cviC1BJQ8MsPSZ4zPsPv6XCeEn3QyMQ@mail.gmail.com>
 <CAK=Phk4XYgn45_2vJqzGciZJKFx4aAdGzMqZQ7o3f3bPHQLH4A@mail.gmail.com>
 <CAP1=2W7L70oAim3Fbyw=Qi+D+OnpGn=B_Om22mBGFX2ho0peaA@mail.gmail.com>
 <CAK=Phk5H49zTWCN8pc3f35R_2f269X94maMXk2ay3A-HT3p8gQ@mail.gmail.com>
 <CACac1F8F2OD=pNcbf4QiE==tDNQTROfdLYjJ+-hLyJ+=VdhOFA@mail.gmail.com>
 <CADiSq7e4GewDkAEJgY=P-H+RedzPxS_j1WDA8wkXPATj-GmOYg@mail.gmail.com>
 <CAP1=2W70CgLPqbtmiEKW8hVzDedsEzC4wrBLwN70Sr2LW-Gkow@mail.gmail.com>
 <CAK=Phk49Yu-7-TUrxk5ky2_NpnMKOmmj1o5QsRdpUUNbsE_U2Q@mail.gmail.com>
 <CADiSq7ejkgmdsizuopsnk4k2fZAe=V8Vv2nk=NDO7pb2k7AJkw@mail.gmail.com>
 <E1bgY0i-00056Y-Hi@se2-syd.hostedmail.net.au>
Message-ID: <CAK=Phk6edeTBgn3RrosK0+Z+15f34S7i4CnMr54qaWfyNb0TKg@mail.gmail.com>

Hi Steve,

- I posted an on setuptools github https://github.com/pypa/
setuptools/issues/771 to ask the setuptools folks (and especially Jason)
whether they would want such a feature.

I still find it baffling that the preferred way to add distutils features
would be to monkey-patch it via setuptools first and then to add it into
distutils later, while my proposed approach was to add it to distutils and
monkey patch for earlier versions of python.

- Regarding the policy that you just nominated, does this apply to things
that have been in pip for a long time? In the initial email that triggered
this discussion, it was also question of adding the
`pip.location.distutils_scheme`
convenience function to distutils itself. Cf
https://bugs.python.org/issue26955 and pip's implementation at
https://github.com/pypa/pip/blob/281eb61b09d87765d7c2b92f6982b3fe76ccb0af/pip/locations.py#L124

Indeed, the problem with certain distutils commands like `install_headers`
is that the only way to retrieve the directory where things were installed
is to instantiate a `distutils.dist.Distribution`
object, which is not trivial to do. pip's distutils_scheme convenience
function does it. This function is *about* distutils, and IMO should really
be *in* distutils.

Thanks,

Sylvain

On Sun, Sep 4, 2016 at 4:02 PM, Steve Dower <steve.dower at python.org> wrote:

> "add it to setuptools first, and then add things to distutils where we
> feel they're sufficiently stable to not need the benefit of the faster
> setuptools update cycle"
>
> I nominate this as the official policy on API changes for distutils, with
> regular maintenance mode policies applying to anything else.
>
> Top-posted from my Windows Phone
> ------------------------------
> From: Nick Coghlan <ncoghlan at gmail.com>
> Sent: ?9/?4/?2016 2:19
> To: Sylvain Corlay <sylvain.corlay at gmail.com>
> Cc: distutils sig <distutils-sig at python.org>
> Subject: Re: [Distutils] What is the official position on distutils?
>
> On 4 September 2016 at 06:44, Sylvain Corlay <sylvain.corlay at gmail.com>
> wrote:
> > Hi Brett,
> >
> > On Sat, Sep 3, 2016 at 8:05 PM, Brett Cannon <brett at python.org> wrote:
> >>
> >>
> >> If Jason is up for the responsibility that seems like a reasonable
> >> approach to take. It also helps test out features in setuptools first
> before
> >> upstreaming it.
> >>
> >
> > How do you see `has_flag` get into setuptools? By monkey-patching
> distutils'
> > ccompiler to have it aside of `has_function` when setuptools is imported?
> >
> > I find really weird the idea of adding this in a convoluted fashion
> instead
> > of allowing incremental improvement of distutils.
>
> The change to distutils would still be a plain patch to distutils, it
> would just be accepted at the API design level in setuptools first.
>
> The problem you're running into right now isn't a technical one - it's
> that there isn't anyone that currently feels like they have sufficient
> design authority over the distutils API to accept your proposal, hence
> Brett starting this thread to address that underlying recurring
> question, rather than the specifics of your change.
>
> Jason *definitely* has that design authority over setuptools though,
> and will be tasked with making any API additions available on older
> versions of Python via setuptools regardless of what policy we adopt
> for distutils maintenance, so if he's amenable to the idea, it makes
> sense to me to invert the order we ask those questions: add it to
> setuptools first, and then add things to distutils where we feel
> they're sufficiently stable to not need the benefit of the faster
> setuptools update cycle.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160904/9c42a6bb/attachment.html>

From ncoghlan at gmail.com  Sun Sep  4 11:16:57 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 5 Sep 2016 01:16:57 +1000
Subject: [Distutils] What is the official position on distutils?
In-Reply-To: <CAK=Phk6edeTBgn3RrosK0+Z+15f34S7i4CnMr54qaWfyNb0TKg@mail.gmail.com>
References: <CAP1=2W4ZXcC5CO5oqKcrsEhYjRvrYqe7o1mnKnOJ5+sCkh9k8w@mail.gmail.com>
 <CACac1F8NXxGbKCPT8X1cviC1BJQ8MsPSZ4zPsPv6XCeEn3QyMQ@mail.gmail.com>
 <CAK=Phk4XYgn45_2vJqzGciZJKFx4aAdGzMqZQ7o3f3bPHQLH4A@mail.gmail.com>
 <CAP1=2W7L70oAim3Fbyw=Qi+D+OnpGn=B_Om22mBGFX2ho0peaA@mail.gmail.com>
 <CAK=Phk5H49zTWCN8pc3f35R_2f269X94maMXk2ay3A-HT3p8gQ@mail.gmail.com>
 <CACac1F8F2OD=pNcbf4QiE==tDNQTROfdLYjJ+-hLyJ+=VdhOFA@mail.gmail.com>
 <CADiSq7e4GewDkAEJgY=P-H+RedzPxS_j1WDA8wkXPATj-GmOYg@mail.gmail.com>
 <CAP1=2W70CgLPqbtmiEKW8hVzDedsEzC4wrBLwN70Sr2LW-Gkow@mail.gmail.com>
 <CAK=Phk49Yu-7-TUrxk5ky2_NpnMKOmmj1o5QsRdpUUNbsE_U2Q@mail.gmail.com>
 <CADiSq7ejkgmdsizuopsnk4k2fZAe=V8Vv2nk=NDO7pb2k7AJkw@mail.gmail.com>
 <E1bgY0i-00056Y-Hi@se2-syd.hostedmail.net.au>
 <CAK=Phk6edeTBgn3RrosK0+Z+15f34S7i4CnMr54qaWfyNb0TKg@mail.gmail.com>
Message-ID: <CADiSq7enssoKPuMHBWjcqm4OAYU3cKOzDYik8QbgXuAxiGx3fw@mail.gmail.com>

On 5 September 2016 at 00:27, Sylvain Corlay <sylvain.corlay at gmail.com> wrote:
> Hi Steve,
>
> - I posted an on setuptools github
> https://github.com/pypa/setuptools/issues/771 to ask the setuptools folks
> (and especially Jason) whether they would want such a feature.
>
> I still find it baffling that the preferred way to add distutils features
> would be to monkey-patch it via setuptools first and then to add it into
> distutils later, while my proposed approach was to add it to distutils and
> monkey patch for earlier versions of python.

Imagine a world where distutils wasn't rife with global state, and
hence setuptools didn't need to monkeypatch it, and the approach seems
more logical (e.g. this is roughly the way I've worked for larger
contextlib changes - the ContextStack precursor to ExitStack only
exists in contextlib2, since I didn't push that API to the stdlib
until I was happy with both the design and the name).

> - Regarding the policy that you just nominated, does this apply to things
> that have been in pip for a long time? In the initial email that triggered
> this discussion, it was also question of adding the
> `pip.location.distutils_scheme` convenience function to distutils itself. Cf
> https://bugs.python.org/issue26955 and pip's implementation at
> https://github.com/pypa/pip/blob/281eb61b09d87765d7c2b92f6982b3fe76ccb0af/pip/locations.py#L124
>
> Indeed, the problem with certain distutils commands like `install_headers`
> is that the only way to retrieve the directory where things were installed
> is to instantiate a `distutils.dist.Distribution`
> object, which is not trivial to do. pip's distutils_scheme convenience
> function does it. This function is *about* distutils, and IMO should really
> be *in* distutils.

Should it be in distutils, or should it be in the version independent
"packaging" helper library that pip uses?

It's important to note that one of our key goals in PyPA is to get to
a point where the API of a developer's publishing toolchain of choice
only depends on the *version of the publishing tools* they're using,
and not on the specific version of Python (as long as they're using a
supported version). Encouraging the expansion and increased use of the
distutils API runs counter to that goal in most cases (since it
encourages writing publication and distribution code that will only
work with newer versions of Python), but may sometimes make sense if
it's info that really is CPython centric, or if it's a feature
primarily aimed at tightly controlled environments that can
aggressively drop support for older versions.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From annaraven at gmail.com  Sun Sep  4 19:21:11 2016
From: annaraven at gmail.com (Anna Ravenscroft)
Date: Sun, 4 Sep 2016 16:21:11 -0700
Subject: [Distutils] What is the official position on distutils?
In-Reply-To: <E1bgY0i-00056Z-O6@se2-syd.hostedmail.net.au>
References: <CAP1=2W4ZXcC5CO5oqKcrsEhYjRvrYqe7o1mnKnOJ5+sCkh9k8w@mail.gmail.com>
 <CACac1F8NXxGbKCPT8X1cviC1BJQ8MsPSZ4zPsPv6XCeEn3QyMQ@mail.gmail.com>
 <CAK=Phk4XYgn45_2vJqzGciZJKFx4aAdGzMqZQ7o3f3bPHQLH4A@mail.gmail.com>
 <CAP1=2W7L70oAim3Fbyw=Qi+D+OnpGn=B_Om22mBGFX2ho0peaA@mail.gmail.com>
 <CAK=Phk5H49zTWCN8pc3f35R_2f269X94maMXk2ay3A-HT3p8gQ@mail.gmail.com>
 <CACac1F8F2OD=pNcbf4QiE==tDNQTROfdLYjJ+-hLyJ+=VdhOFA@mail.gmail.com>
 <CADiSq7e4GewDkAEJgY=P-H+RedzPxS_j1WDA8wkXPATj-GmOYg@mail.gmail.com>
 <CAP1=2W70CgLPqbtmiEKW8hVzDedsEzC4wrBLwN70Sr2LW-Gkow@mail.gmail.com>
 <CAK=Phk49Yu-7-TUrxk5ky2_NpnMKOmmj1o5QsRdpUUNbsE_U2Q@mail.gmail.com>
 <CADiSq7ejkgmdsizuopsnk4k2fZAe=V8Vv2nk=NDO7pb2k7AJkw@mail.gmail.com>
 <E1bgY0i-00056Z-O6@se2-syd.hostedmail.net.au>
Message-ID: <CA+nrBM_3yf26utHdXN5xAGRGF+Mso78mquRnXze8W_XJQMfZ1w@mail.gmail.com>

+1

On Sun, Sep 4, 2016 at 7:02 AM, Steve Dower <steve.dower at python.org> wrote:

> "add it to setuptools first, and then add things to distutils where we
> feel they're sufficiently stable to not need the benefit of the faster
> setuptools update cycle"
>
> I nominate this as the official policy on API changes for distutils, with
> regular maintenance mode policies applying to anything else.
>
> Top-posted from my Windows Phone
> ------------------------------
> From: Nick Coghlan <ncoghlan at gmail.com>
> Sent: ?9/?4/?2016 2:19
> To: Sylvain Corlay <sylvain.corlay at gmail.com>
> Cc: distutils sig <distutils-sig at python.org>
> Subject: Re: [Distutils] What is the official position on distutils?
>
> On 4 September 2016 at 06:44, Sylvain Corlay <sylvain.corlay at gmail.com>
> wrote:
> > Hi Brett,
> >
> > On Sat, Sep 3, 2016 at 8:05 PM, Brett Cannon <brett at python.org> wrote:
> >>
> >>
> >> If Jason is up for the responsibility that seems like a reasonable
> >> approach to take. It also helps test out features in setuptools first
> before
> >> upstreaming it.
> >>
> >
> > How do you see `has_flag` get into setuptools? By monkey-patching
> distutils'
> > ccompiler to have it aside of `has_function` when setuptools is imported?
> >
> > I find really weird the idea of adding this in a convoluted fashion
> instead
> > of allowing incremental improvement of distutils.
>
> The change to distutils would still be a plain patch to distutils, it
> would just be accepted at the API design level in setuptools first.
>
> The problem you're running into right now isn't a technical one - it's
> that there isn't anyone that currently feels like they have sufficient
> design authority over the distutils API to accept your proposal, hence
> Brett starting this thread to address that underlying recurring
> question, rather than the specifics of your change.
>
> Jason *definitely* has that design authority over setuptools though,
> and will be tasked with making any API additions available on older
> versions of Python via setuptools regardless of what policy we adopt
> for distutils maintenance, so if he's amenable to the idea, it makes
> sense to me to invert the order we ask those questions: add it to
> setuptools first, and then add things to distutils where we feel
> they're sufficiently stable to not need the benefit of the faster
> setuptools update cycle.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>


-- 
cordially,
Anna
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160904/56febe98/attachment.html>

From sylvain.corlay at gmail.com  Mon Sep  5 02:47:31 2016
From: sylvain.corlay at gmail.com (Sylvain Corlay)
Date: Mon, 5 Sep 2016 08:47:31 +0200
Subject: [Distutils] Fwd:  What is the official position on distutils?
In-Reply-To: <CAK=Phk6W71JWpyS4yrzhuXJV4xc-d2DcsNp1um3ne2C+hCDcVA@mail.gmail.com>
References: <CAP1=2W4ZXcC5CO5oqKcrsEhYjRvrYqe7o1mnKnOJ5+sCkh9k8w@mail.gmail.com>
 <CACac1F8NXxGbKCPT8X1cviC1BJQ8MsPSZ4zPsPv6XCeEn3QyMQ@mail.gmail.com>
 <CAK=Phk4XYgn45_2vJqzGciZJKFx4aAdGzMqZQ7o3f3bPHQLH4A@mail.gmail.com>
 <CAP1=2W7L70oAim3Fbyw=Qi+D+OnpGn=B_Om22mBGFX2ho0peaA@mail.gmail.com>
 <CAK=Phk5H49zTWCN8pc3f35R_2f269X94maMXk2ay3A-HT3p8gQ@mail.gmail.com>
 <CACac1F8F2OD=pNcbf4QiE==tDNQTROfdLYjJ+-hLyJ+=VdhOFA@mail.gmail.com>
 <CADiSq7e4GewDkAEJgY=P-H+RedzPxS_j1WDA8wkXPATj-GmOYg@mail.gmail.com>
 <CAP1=2W70CgLPqbtmiEKW8hVzDedsEzC4wrBLwN70Sr2LW-Gkow@mail.gmail.com>
 <CAK=Phk49Yu-7-TUrxk5ky2_NpnMKOmmj1o5QsRdpUUNbsE_U2Q@mail.gmail.com>
 <CADiSq7ejkgmdsizuopsnk4k2fZAe=V8Vv2nk=NDO7pb2k7AJkw@mail.gmail.com>
 <E1bgY0i-00056Y-Hi@se2-syd.hostedmail.net.au>
 <CAK=Phk6edeTBgn3RrosK0+Z+15f34S7i4CnMr54qaWfyNb0TKg@mail.gmail.com>
 <CADiSq7enssoKPuMHBWjcqm4OAYU3cKOzDYik8QbgXuAxiGx3fw@mail.gmail.com>
 <CAK=Phk6W71JWpyS4yrzhuXJV4xc-d2DcsNp1um3ne2C+hCDcVA@mail.gmail.com>
Message-ID: <CAK=Phk66646EX8qtnRzStmnLSs1485jtMPxRWdmJdfeC1frC7A@mail.gmail.com>

Hi Nick (re-sending to the list)

On Sun, Sep 4, 2016 at 5:16 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

>
> It's important to note that one of our key goals in PyPA is to get to
> a point where the API of a developer's publishing toolchain of choice
> only depends on the *version of the publishing tools* they're using,
> and not on the specific version of Python (as long as they're using a
> supported version). Encouraging the expansion and increased use of the
> distutils API runs counter to that goal in most cases (since it
> encourages writing publication and distribution code that will only
> work with newer versions of Python), but may sometimes make sense if
> it's info that really is CPython centric, or if it's a feature
> primarily aimed at tightly controlled environments that can
> aggressively drop support for older versions.


Today's state of things is that any project that is not a trivial
pure-python module often has use of a combination of all three libraries:
distutils, setuptools and pip in their setup.py and in  a convoluted way.
Newcomers to the toolchain look in similar projects for the right
incantations that will fulfill their needs. We are pretty far from being in
a state where good practices can even be defined.

As a package author, I generally make sure that my need is not fulfilled by
a proper usage of the standard library before adding a dependency to a
third-party library, but packaging is an area where this does not work.
Requiring pip only to know where distutils puts assets is pretty bad.

Interestingly, on the subject of building extensions (where the has_flag
method discussion applies), distutils.ccompiler is the only available tool
and happens to not be monkey-patched by setuptools at the moment. Although
setuptools re-defines buildext (and tries to get a version from cython
first!) So it seems that there no real "right place" within setuptools to
monkey-patch distutils' ccompiler.

Sylvain
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160905/10bcdfeb/attachment.html>

From ncoghlan at gmail.com  Mon Sep  5 05:58:44 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 5 Sep 2016 19:58:44 +1000
Subject: [Distutils] Fwd: What is the official position on distutils?
In-Reply-To: <CAK=Phk66646EX8qtnRzStmnLSs1485jtMPxRWdmJdfeC1frC7A@mail.gmail.com>
References: <CAP1=2W4ZXcC5CO5oqKcrsEhYjRvrYqe7o1mnKnOJ5+sCkh9k8w@mail.gmail.com>
 <CACac1F8NXxGbKCPT8X1cviC1BJQ8MsPSZ4zPsPv6XCeEn3QyMQ@mail.gmail.com>
 <CAK=Phk4XYgn45_2vJqzGciZJKFx4aAdGzMqZQ7o3f3bPHQLH4A@mail.gmail.com>
 <CAP1=2W7L70oAim3Fbyw=Qi+D+OnpGn=B_Om22mBGFX2ho0peaA@mail.gmail.com>
 <CAK=Phk5H49zTWCN8pc3f35R_2f269X94maMXk2ay3A-HT3p8gQ@mail.gmail.com>
 <CACac1F8F2OD=pNcbf4QiE==tDNQTROfdLYjJ+-hLyJ+=VdhOFA@mail.gmail.com>
 <CADiSq7e4GewDkAEJgY=P-H+RedzPxS_j1WDA8wkXPATj-GmOYg@mail.gmail.com>
 <CAP1=2W70CgLPqbtmiEKW8hVzDedsEzC4wrBLwN70Sr2LW-Gkow@mail.gmail.com>
 <CAK=Phk49Yu-7-TUrxk5ky2_NpnMKOmmj1o5QsRdpUUNbsE_U2Q@mail.gmail.com>
 <CADiSq7ejkgmdsizuopsnk4k2fZAe=V8Vv2nk=NDO7pb2k7AJkw@mail.gmail.com>
 <E1bgY0i-00056Y-Hi@se2-syd.hostedmail.net.au>
 <CAK=Phk6edeTBgn3RrosK0+Z+15f34S7i4CnMr54qaWfyNb0TKg@mail.gmail.com>
 <CADiSq7enssoKPuMHBWjcqm4OAYU3cKOzDYik8QbgXuAxiGx3fw@mail.gmail.com>
 <CAK=Phk6W71JWpyS4yrzhuXJV4xc-d2DcsNp1um3ne2C+hCDcVA@mail.gmail.com>
 <CAK=Phk66646EX8qtnRzStmnLSs1485jtMPxRWdmJdfeC1frC7A@mail.gmail.com>
Message-ID: <CADiSq7fQC85jBt7qU44kOMXJg8R0ajAG5Eg3sW1g1+7OmBicCg@mail.gmail.com>

On 5 September 2016 at 16:47, Sylvain Corlay <sylvain.corlay at gmail.com> wrote:
> Hi Nick (re-sending to the list)

(Sylvain's seen some of what I say below in off-list replies, but I
figured it made sense to go over my point of view again for the list's
benefit)

> On Sun, Sep 4, 2016 at 5:16 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>>
>> It's important to note that one of our key goals in PyPA is to get to
>> a point where the API of a developer's publishing toolchain of choice
>> only depends on the *version of the publishing tools* they're using,
>> and not on the specific version of Python (as long as they're using a
>> supported version). Encouraging the expansion and increased use of the
>> distutils API runs counter to that goal in most cases (since it
>> encourages writing publication and distribution code that will only
>> work with newer versions of Python), but may sometimes make sense if
>> it's info that really is CPython centric, or if it's a feature
>> primarily aimed at tightly controlled environments that can
>> aggressively drop support for older versions.
>
>
> Today's state of things is that any project that is not a trivial
> pure-python module often has use of a combination of all three libraries:
> distutils, setuptools and pip in their setup.py and in  a convoluted way.
> Newcomers to the toolchain look in similar projects for the right
> incantations that will fulfill their needs. We are pretty far from being in
> a state where good practices can even be defined.

One of the good practices that *is* defined is "Use setuptools rather
than distutils" (footnote [5] gives the rationale):
https://packaging.python.org/current/#packaging-tool-recommendations

That's one of the key reasons we want to ensure setuptools
consistently provides a superset of distutils functionality, and also
ensure that any new functionality is available on older versions of
Python, either by monkeypatching distutils from setuptools (when
absolutely necessary due to the way distutils works), or by providing
version independent equivalent APIs in setuptools itself (when it's
sufficient to give publishers access to the functionality from
setup.py, rather than implicitly altering the behaviour of distutils
itself).

> As a package author, I generally make sure that my need is not fulfilled by
> a proper usage of the standard library before adding a dependency to a
> third-party library, but packaging is an area where this does not work.

This is now by design, as PyPA's view is that coupling the lifecycle
of a language ecosystem's publishing toolchain to the reference
implementation of the language runtime is a bad idea, since publishing
tools need to be free to update in response to changes in supported
target environments, without necessarily dropping support for older
versions of the language runtime itself.

PEP 518 (declarative dependencies for setup.py), and the competing
PEPs 516 & 517 for pluggable build system invocation are designed to
better enable that, although Daniel Holth's work with enscons shows
that it's possible to bootstrap an existing external build system from
setup.py, even without formal support for that at the interoperability
specification level. The reason packaging in particular is insistent
on the use of documented interoperability standards is that part of
the goal is to drop the requirement that build systems for Python
projects must be written in Python - they may need or benefit from
offering Python bindings, but working with our data formats in other
languages should be fully supported.

> Requiring pip only to know where distutils puts assets is pretty bad.

This is an area that "just" requires funding and contributions -
distlib (pragmatic, not necessarily standards based) and packaging
(strictly based on agreed interoperability standards) both exist to
serve as pattern extraction libraries for pip, PyPI, setuptools,
distutils, sysconfig etc, and make the things they do internally more
readily available to other software.

However, pursuing this model effectively requires giving up on the
idea that distutils will ever be "fixed" - it won't. There are deep
structural flaws in the design of its API (which even Greg Ward, its
original creator, now considers a bad idea) that can't be addressed
without major compatibility breaks, so it makes more sense to enable
and pursue the "strangler application" model [1], where we facilitate
the adoption of alternative build systems by publishers in a way
that's transparent to software consumers, and then base future
recommendations for default tooling choices based on emergent
community behaviour. The focus for distutils becomes "don't break
currently working build processes", while setuptools and other PyPA
projects take up the challenge of driving improvements in the
capabilities made available to publishers.

At the end of that process (which is likely to take years), there
shouldn't be any software build activities left using plain distutils
- everything that hasn't switched to a different build system entirely
will have either switched explicitly to setuptools, or will have
setuptools injected into the build process by build environments the
same way pip already does.

[1] http://www.martinfowler.com/bliki/StranglerApplication.html

> Interestingly, on the subject of building extensions (where the has_flag
> method discussion applies), distutils.ccompiler is the only available tool
> and happens to not be monkey-patched by setuptools at the moment. Although
> setuptools re-defines buildext (and tries to get a version from cython
> first!) So it seems that there no real "right place" within setuptools to
> monkey-patch distutils' ccompiler.

If I understand the change correctly, the new feature would only be
used explicitly, and thus may not require monkeypatching at all.
Instead, it could just require importing buildext from setuptools
rather than distutils if someone wants the new behaviour on older
Python versions.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From chris.barker at noaa.gov  Tue Sep  6 18:34:58 2016
From: chris.barker at noaa.gov (Chris Barker)
Date: Tue, 6 Sep 2016 15:34:58 -0700
Subject: [Distutils] When can we kill Python 2.6 support?
In-Reply-To: <CAG8k2+5sffUvL+YBrEPjQzUS8URZGCvY62zUhbqCT_xBsNxxXg@mail.gmail.com>
References: <40BEEAE9-F798-4972-889A-CDB22993A53F@stufft.io>
 <CAP1=2W6xVf7fdq-Uwe=4RKq4px8HrTZTepciLt7Wr=F36D2oCQ@mail.gmail.com>
 <CAG8k2+5sffUvL+YBrEPjQzUS8URZGCvY62zUhbqCT_xBsNxxXg@mail.gmail.com>
Message-ID: <CALGmxEJHCegGv1gLNqFGOoz9hBUcFH7kA12m_YJc_2mNeTEH6w@mail.gmail.com>

> Looking at pure usage numbers for "modern" versions of pip (6, 7, and 8)
for
downloading from PyPI I see the usage is ~3% of downloads are via Python
2.6.

And a lot of those may be CI systems for packages that still support 2.6....

deprecate away!

-CHB




On Sat, Sep 3, 2016 at 1:36 PM, Daniel Holth <dholth at gmail.com> wrote:

> It would be convenient to drop 2.6 in wheel too.
>
> On Sat, Sep 3, 2016, 14:14 Brett Cannon <brett at python.org> wrote:
>
>> I think the fact that Python 2.6 is past EOL means it's definitely up for
>> consideration. As for the 3% usage, as a trite comparison that's the amount
>> of scientists who deny climate change. So IMO that suggests 2.6 is not used
>> enough to burden PyPA with the maintenance and those who still want to use
>> it can take over maintaining 2.6 compatibility.
>>
>>
>> On Fri, 2 Sep 2016 at 14:06 Donald Stufft <donald at stufft.io> wrote:
>>
>>> The packaging tools generally support 2.6+ and 3.(2|3)+ and that's sort
>>> of been
>>> where they've been at for a while now. I would like to think about what
>>> we need
>>> to be to start considering Python 2.6 as "too old" to support. In pip we
>>> generally follow a usage based deprecation/removal of supported Pythons
>>> but we
>>> don't have any real guidelines for when something is at a low enough
>>> usage to
>>> consider it no longer supported and we instead just sort of wait until
>>> someone
>>> makes a case that it's "low enough".
>>>
>>> This issue tends to impact more than just pip, because once pip drops
>>> support
>>> for something people tend to start dropping it across the entire
>>> ecosystem and
>>> use pip's no longer supporting it as justification for doing so.
>>>
>>> I would like to take a look at Python 2.6 and try and figure out if
>>> we're at a
>>> point that we can deprecate and drop it, and if not what is such a point.
>>>
>>> Looking at pure usage numbers for "modern" versions of pip (6, 7, and 8)
>>> for
>>> downloading from PyPI I see the usage is ~3% of downloads are via Python
>>> 2.6.
>>> The only thing lower than Python 2.6 that is still supported is Python
>>> 3.3.
>>>
>>> Python 2.6 itself has been EOL since 2013-10-29 which is now just about
>>> 3 years
>>> ago. It's SSL module is not generally secure and requires the use of
>>> additional
>>> installed modules to get it to be so. I believe the only place to get a
>>> Python 2.6 that is "supported" is through the Enterprise-y Linux
>>> Distributions
>>> like RHEL/CentOS/etc.
>>>
>>> Do we think that a ~3% usage of Python 2.6 and being end-of-life'd for
>>> ~3 years
>>> is enough to start deprecating and dropping 2.6? If not what sort of
>>> threshold
>>> do we think is enough? It'd be nice to get the albatross of Python 2.6
>>> support
>>> off from around our necks but I'm not sure how others feel. Obviously
>>> all of
>>> the existing versions of all of the tooling will still be fully
>>> functional so
>>> Python 2.6 users will simply need to not upgrade their tooling to
>>> continue to
>>> work, *but* it also means that they will be left out of new packaging
>>> features
>>> (and likewise, people can't rely on them if they still wish to support
>>> 2.6).
>>>
>>> Thoughts?
>>>
>>> ?
>>> Donald Stufft
>>>
>>>
>>>
>>> _______________________________________________
>>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>>> https://mail.python.org/mailman/listinfo/distutils-sig
>>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160906/b1b2bcd2/attachment.html>

From sylvain.corlay at gmail.com  Mon Sep  5 02:47:31 2016
From: sylvain.corlay at gmail.com (Sylvain Corlay)
Date: Mon, 5 Sep 2016 00:47:31 -0600
Subject: [Distutils]  Fwd:  What is the official position on distutils?
In-Reply-To: <CAK=Phk6W71JWpyS4yrzhuXJV4xc-d2DcsNp1um3ne2C+hCDcVA@mail.gmail.com>
References: <CAP1=2W4ZXcC5CO5oqKcrsEhYjRvrYqe7o1mnKnOJ5+sCkh9k8w@mail.gmail.com>
 <CACac1F8NXxGbKCPT8X1cviC1BJQ8MsPSZ4zPsPv6XCeEn3QyMQ@mail.gmail.com>
 <CAK=Phk4XYgn45_2vJqzGciZJKFx4aAdGzMqZQ7o3f3bPHQLH4A@mail.gmail.com>
 <CAP1=2W7L70oAim3Fbyw=Qi+D+OnpGn=B_Om22mBGFX2ho0peaA@mail.gmail.com>
 <CAK=Phk5H49zTWCN8pc3f35R_2f269X94maMXk2ay3A-HT3p8gQ@mail.gmail.com>
 <CACac1F8F2OD=pNcbf4QiE==tDNQTROfdLYjJ+-hLyJ+=VdhOFA@mail.gmail.com>
 <CADiSq7e4GewDkAEJgY=P-H+RedzPxS_j1WDA8wkXPATj-GmOYg@mail.gmail.com>
 <CAP1=2W70CgLPqbtmiEKW8hVzDedsEzC4wrBLwN70Sr2LW-Gkow@mail.gmail.com>
 <CAK=Phk49Yu-7-TUrxk5ky2_NpnMKOmmj1o5QsRdpUUNbsE_U2Q@mail.gmail.com>
 <CADiSq7ejkgmdsizuopsnk4k2fZAe=V8Vv2nk=NDO7pb2k7AJkw@mail.gmail.com>
 <E1bgY0i-00056Y-Hi@se2-syd.hostedmail.net.au>
 <CAK=Phk6edeTBgn3RrosK0+Z+15f34S7i4CnMr54qaWfyNb0TKg@mail.gmail.com>
 <CADiSq7enssoKPuMHBWjcqm4OAYU3cKOzDYik8QbgXuAxiGx3fw@mail.gmail.com>
 <CAK=Phk6W71JWpyS4yrzhuXJV4xc-d2DcsNp1um3ne2C+hCDcVA@mail.gmail.com>
Message-ID: <CAK=Phk66646EX8qtnRzStmnLSs1485jtMPxRWdmJdfeC1frC7A@mail.gmail.com>

Hi Nick (re-sending to the list)


On Sun, Sep 4, 2016 at 5:16 PM, Nick Coghlan <ncoghlan at gmail.com <mailto:ncoghlan at gmail.com> > wrote:



It's important to note that one of our key goals in PyPA is to get to
a point where the API of a developer's publishing toolchain of choice
only depends on the *version of the publishing tools* they're using,
and not on the specific version of Python (as long as they're using a
supported version). Encouraging the expansion and increased use of the
distutils API runs counter to that goal in most cases (since it
encourages writing publication and distribution code that will only
work with newer versions of Python), but may sometimes make sense if
it's info that really is CPython centric, or if it's a feature
primarily aimed at tightly controlled environments that can
aggressively drop support for older versions.

 
Today's state of things is that any project that is not a trivial pure-python module often has use of a combination of all three libraries: distutils, setuptools and pip in their setup.py and in  a convoluted way. Newcomers to the toolchain look in similar projects for the right incantations that will fulfill their needs. We are pretty far from being in a state where good practices can even be defined.

As a package author, I generally make sure that my need is not fulfilled by a proper usage of the standard library before adding a dependency to a third-party library, but packaging is an area where this does not work. Requiring pip only to know where distutils puts assets is pretty bad.

Interestingly, on the subject of building extensions (where the has_flag method discussion applies), distutils.ccompiler is the only available tool and happens to not be monkey-patched by setuptools at the moment. Although setuptools re-defines buildext (and tries to get a version from cython first!) So it seems that there no real "right place" within setuptools to monkey-patch distutils' ccompiler.

Sylvain

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160905/10bcdfeb/attachment-0002.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ATT00001.txt
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160905/10bcdfeb/attachment.txt>

From bussonniermatthias at gmail.com  Thu Sep  8 18:47:23 2016
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Thu, 8 Sep 2016 15:47:23 -0700
Subject: [Distutils] Pep 440 question/clarification Version specifiers no
 "Or" available
Message-ID: <CANJQusVKZBM1aokHC=7-fVqDQwbyzu1k+DLxZ6QY-ge7qJRQqg@mail.gmail.com>

Hi all,

In the line of recent discussions[1] about releasing some packages
only for some versions of Python.

I was reading pep 440 and in particular the "Version specifier section".

The following sentence caught my eye:

> The comma (",") is equivalent to a logical and operator: a candidate version must match all given version clauses in order to match the specifier as a whole.


Is there no way to have an "or" ? It seem to me that it might be
useful for package wanting to express compatibility with `2.6`, `2.7`
or `>3.3` for example, in the form `=~2.6 <or operator> =~3.3`.

I completely agree that the use case is _limited_ and likely rare, I
was just wondering if I'm missing something, if it was an oversight
and if not if adding a section detailing that "or" was not considered,
or that "or" was explicitly not included for X,Y,Z reason would be a
good thing.

Thanks,
-- 
Matthias



[1]:https://mail.python.org/pipermail/distutils-sig/2016-August/029604.html

From rui_sarmento at hotmail.com  Tue Sep  6 11:06:13 2016
From: rui_sarmento at hotmail.com (Rui Sarmento)
Date: Tue, 6 Sep 2016 16:06:13 +0100
Subject: [Distutils] Improved Stats Features in Python
Message-ID: <BLU436-SMTP3C562E7DB597D989CF778E3F90@phx.gbl>

Dear Sirs,

I've implemented some stats functions related to Factor Analysis in the 
statistics area. Specifically, the KMO test and the Bartlett test also. 
At this time I do not seem to find any module performing these tests. Is 
there any chance I could add these functions to a package in Python. 
What is the procedure to perform such contribution.

Thank you very much in advance for the suggestions.

Best Regards,

Rui


From donald at stufft.io  Thu Sep  8 22:21:24 2016
From: donald at stufft.io (Donald Stufft)
Date: Thu, 8 Sep 2016 22:21:24 -0400
Subject: [Distutils] Pep 440 question/clarification Version specifiers
 no "Or" available
In-Reply-To: <CANJQusVKZBM1aokHC=7-fVqDQwbyzu1k+DLxZ6QY-ge7qJRQqg@mail.gmail.com>
References: <CANJQusVKZBM1aokHC=7-fVqDQwbyzu1k+DLxZ6QY-ge7qJRQqg@mail.gmail.com>
Message-ID: <A3C9D392-3646-41E3-8068-9B284953106C@stufft.io>


> On Sep 8, 2016, at 6:47 PM, Matthias Bussonnier <bussonniermatthias at gmail.com> wrote:
> 
> Is there no way to have an "or" ? It seem to me that it might be
> useful for package wanting to express compatibility with `2.6`, `2.7`
> or `>3.3` for example, in the form `=~2.6 <or operator> =~3.3`.
> 
> I completely agree that the use case is _limited_ and likely rare, I
> was just wondering if I'm missing something, if it was an oversight
> and if not if adding a section detailing that "or" was not considered,
> or that "or" was explicitly not included for X,Y,Z reason would be a
> good thing.


There is currently no way to have an or. I don?t think there is any conceptual reason not to have it, except we were attempting to deviate from what was currently accepted as little as possible and the only character that was currently accepted was only ``,``. We did deviate in that the behavior of ``,`` changes somewhat, but the previous behavior was very tricky and hard to explain.

In the future I?d actually like to expand upon it but I think that once you add OR you kind of also need to add a grouping operator as well to handle more complex cases besides ones where you only need AND or OR. I think this would probably be a good place to just add a new AND syntax too, and make it something like:

`(~=2.6 or ~=3.3) and != 2.6.5`

That was more of a shake up then we wanted to do at the time though.

?
Donald Stufft




From robertc at robertcollins.net  Thu Sep  8 22:31:49 2016
From: robertc at robertcollins.net (Robert Collins)
Date: Fri, 9 Sep 2016 14:31:49 +1200
Subject: [Distutils] Pep 440 question/clarification Version specifiers
 no "Or" available
In-Reply-To: <CANJQusVKZBM1aokHC=7-fVqDQwbyzu1k+DLxZ6QY-ge7qJRQqg@mail.gmail.com>
References: <CANJQusVKZBM1aokHC=7-fVqDQwbyzu1k+DLxZ6QY-ge7qJRQqg@mail.gmail.com>
Message-ID: <CAJ3HoZ0Eg6pvAa+PzKZ4FRqm6DwwxcYbLnCp6Q17M6stebJuEQ@mail.gmail.com>

Thats right, there is no 'or' operator available today.

In addition to what Donald says, Or significantly complicates the
resolution job for the resolver: should 'A OR B' prefer A? If B is
already installed should it be uninstalled to permit choosing A (aka
is it a disjunction? What if A and B can coexist and one part of the
graph wants A and one part wants B but there is an A OR B higher up? -
if its a disjunction that becomes uninstallable.

tl;dr the ramifications of OR are much deeper than those of AND for
this problem space, and we don't even have AND solved properly - Issue
988.

-Rob


On 9 September 2016 at 10:47, Matthias Bussonnier
<bussonniermatthias at gmail.com> wrote:
> Hi all,
>
> In the line of recent discussions[1] about releasing some packages
> only for some versions of Python.
>
> I was reading pep 440 and in particular the "Version specifier section".
>
> The following sentence caught my eye:
>
>> The comma (",") is equivalent to a logical and operator: a candidate version must match all given version clauses in order to match the specifier as a whole.
>
>
> Is there no way to have an "or" ? It seem to me that it might be
> useful for package wanting to express compatibility with `2.6`, `2.7`
> or `>3.3` for example, in the form `=~2.6 <or operator> =~3.3`.
>
> I completely agree that the use case is _limited_ and likely rare, I
> was just wondering if I'm missing something, if it was an oversight
> and if not if adding a section detailing that "or" was not considered,
> or that "or" was explicitly not included for X,Y,Z reason would be a
> good thing.
>
> Thanks,
> --
> Matthias
>
>
>
> [1]:https://mail.python.org/pipermail/distutils-sig/2016-August/029604.html
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

From ncoghlan at gmail.com  Thu Sep  8 23:10:16 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 9 Sep 2016 13:10:16 +1000
Subject: [Distutils] Pep 440 question/clarification Version specifiers
 no "Or" available
In-Reply-To: <CAJ3HoZ0Eg6pvAa+PzKZ4FRqm6DwwxcYbLnCp6Q17M6stebJuEQ@mail.gmail.com>
References: <CANJQusVKZBM1aokHC=7-fVqDQwbyzu1k+DLxZ6QY-ge7qJRQqg@mail.gmail.com>
 <CAJ3HoZ0Eg6pvAa+PzKZ4FRqm6DwwxcYbLnCp6Q17M6stebJuEQ@mail.gmail.com>
Message-ID: <CADiSq7d=yWXzsjoU4JCBr6kU+zSZATBzC-FhivRDZJuJUccTmQ@mail.gmail.com>

On 9 Sep 2016 12:32 pm, "Robert Collins" <robertc at robertcollins.net> wrote:
>
> Thats right, there is no 'or' operator available today.
>
> In addition to what Donald says, Or significantly complicates the
> resolution job for the resolver: should 'A OR B' prefer A? If B is
> already installed should it be uninstalled to permit choosing A (aka
> is it a disjunction? What if A and B can coexist and one part of the
> graph wants A and one part wants B but there is an A OR B higher up? -
> if its a disjunction that becomes uninstallable.

Fedora started running into this kind of problem when rich dependencies
were added to RPM. Between that and the need for compatibility with older
RPM versions, it's been genuinely difficult for publishers to start relying
on the new features.

> tl;dr the ramifications of OR are much deeper than those of AND for
> this problem space, and we don't even have AND solved properly - Issue
> 988.

Disjoint environment markers provide most of the capabilities we actually
need (like installing one dependency on Py2 and a different one on Py3)
without ever forcing the resolver to make an arbitrary choice between them.

There are currently some bugs even in that, though - installing an sdist or
wheel directly instead of via a requirements file bypasses the environment
marker checks for its direct dependencies :(

(Steve Kowalik was trying to figure that one out at the PyCon AU sprints,
but I don't believe he was able to track down where the missing check
should go)

Cheers,
Nick.

>
> -Rob
>
>
> On 9 September 2016 at 10:47, Matthias Bussonnier
> <bussonniermatthias at gmail.com> wrote:
> > Hi all,
> >
> > In the line of recent discussions[1] about releasing some packages
> > only for some versions of Python.
> >
> > I was reading pep 440 and in particular the "Version specifier section".
> >
> > The following sentence caught my eye:
> >
> >> The comma (",") is equivalent to a logical and operator: a candidate
version must match all given version clauses in order to match the
specifier as a whole.
> >
> >
> > Is there no way to have an "or" ? It seem to me that it might be
> > useful for package wanting to express compatibility with `2.6`, `2.7`
> > or `>3.3` for example, in the form `=~2.6 <or operator> =~3.3`.
> >
> > I completely agree that the use case is _limited_ and likely rare, I
> > was just wondering if I'm missing something, if it was an oversight
> > and if not if adding a section detailing that "or" was not considered,
> > or that "or" was explicitly not included for X,Y,Z reason would be a
> > good thing.
> >
> > Thanks,
> > --
> > Matthias
> >
> >
> >
> > [1]:
https://mail.python.org/pipermail/distutils-sig/2016-August/029604.html
> > _______________________________________________
> > Distutils-SIG maillist  -  Distutils-SIG at python.org
> > https://mail.python.org/mailman/listinfo/distutils-sig
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160909/a919bd32/attachment-0001.html>

From robertc at robertcollins.net  Thu Sep  8 23:19:22 2016
From: robertc at robertcollins.net (Robert Collins)
Date: Fri, 9 Sep 2016 15:19:22 +1200
Subject: [Distutils] Pep 440 question/clarification Version specifiers
 no "Or" available
In-Reply-To: <CADiSq7d=yWXzsjoU4JCBr6kU+zSZATBzC-FhivRDZJuJUccTmQ@mail.gmail.com>
References: <CANJQusVKZBM1aokHC=7-fVqDQwbyzu1k+DLxZ6QY-ge7qJRQqg@mail.gmail.com>
 <CAJ3HoZ0Eg6pvAa+PzKZ4FRqm6DwwxcYbLnCp6Q17M6stebJuEQ@mail.gmail.com>
 <CADiSq7d=yWXzsjoU4JCBr6kU+zSZATBzC-FhivRDZJuJUccTmQ@mail.gmail.com>
Message-ID: <CAJ3HoZ09pLJx1tBLwV3=LD0r1CBzJN_qsFcSjifzgdmV9qhE4g@mail.gmail.com>

On 9 September 2016 at 15:10, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 9 Sep 2016 12:32 pm, "Robert Collins" <robertc at robertcollins.net> wrote:

>> tl;dr the ramifications of OR are much deeper than those of AND for
>> this problem space, and we don't even have AND solved properly - Issue
>> 988.
>
> Disjoint environment markers provide most of the capabilities we actually
> need (like installing one dependency on Py2 and a different one on Py3)
> without ever forcing the resolver to make an arbitrary choice between them.

Yup :) and we've got that pretty solidly adoptable now - chunk of work
that it was ;).

> There are currently some bugs even in that, though - installing an sdist or
> wheel directly instead of via a requirements file bypasses the environment
> marker checks for its direct dependencies :(
>
> (Steve Kowalik was trying to figure that one out at the PyCon AU sprints,
> but I don't believe he was able to track down where the missing check should
> go)

I didn't mention it before but I should - its my view we should be
really conservative about an OR in the dependency language, for the
reasons in my earlier email.

-Rob

From ncoghlan at gmail.com  Thu Sep  8 23:33:33 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 9 Sep 2016 13:33:33 +1000
Subject: [Distutils] Pep 440 question/clarification Version specifiers
 no "Or" available
In-Reply-To: <CAJ3HoZ09pLJx1tBLwV3=LD0r1CBzJN_qsFcSjifzgdmV9qhE4g@mail.gmail.com>
References: <CANJQusVKZBM1aokHC=7-fVqDQwbyzu1k+DLxZ6QY-ge7qJRQqg@mail.gmail.com>
 <CAJ3HoZ0Eg6pvAa+PzKZ4FRqm6DwwxcYbLnCp6Q17M6stebJuEQ@mail.gmail.com>
 <CADiSq7d=yWXzsjoU4JCBr6kU+zSZATBzC-FhivRDZJuJUccTmQ@mail.gmail.com>
 <CAJ3HoZ09pLJx1tBLwV3=LD0r1CBzJN_qsFcSjifzgdmV9qhE4g@mail.gmail.com>
Message-ID: <CADiSq7ficvX7f3rU3DS77HU=C8B1oK+Z00_tmAAvVAWvK6T7dg@mail.gmail.com>

On 9 September 2016 at 13:19, Robert Collins <robertc at robertcollins.net> wrote:
> On 9 September 2016 at 15:10, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> There are currently some bugs even in that, though - installing an sdist or
>> wheel directly instead of via a requirements file bypasses the environment
>> marker checks for its direct dependencies :(
>>
>> (Steve Kowalik was trying to figure that one out at the PyCon AU sprints,
>> but I don't believe he was able to track down where the missing check should
>> go)

Back on a real computer now, so the relevant bug report for that one:
https://github.com/pypa/pip/issues/3893

> I didn't mention it before but I should - its my view we should be
> really conservative about an OR in the dependency language, for the
> reasons in my earlier email.

Agreed, getting AND requirements, optional dependencies (extras), and
explicit conditional installation (environment markers) right is hard
enough without getting into the problems of full disjunctive
dependencies.

We could potentially do a better job of documenting this though - the
leap from "either A or B is fine" to having to be more explicit in
saying "in these cases, I expect A, in those cases, I expect B" isn't
always clear.

Cheers,
NIck.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From opensource at ronnypfannschmidt.de  Fri Sep  9 02:29:28 2016
From: opensource at ronnypfannschmidt.de (Ronny Pfannschmidt)
Date: Fri, 9 Sep 2016 08:29:28 +0200
Subject: [Distutils] Improved Stats Features in Python
In-Reply-To: <BLU436-SMTP3C562E7DB597D989CF778E3F90@phx.gbl>
References: <BLU436-SMTP3C562E7DB597D989CF778E3F90@phx.gbl>
Message-ID: <15c396aa-e631-09c1-c5c9-aba1beba27b5@ronnypfannschmidt.de>

Hello Rui,

this mailing list deal with tools you can use to publish 3rd party 
packages to something like the pypi package index,

if you want to add to the python stdlib, you need to get started with 
python-ideas, python-dev and very likely write a PEP that will have to 
get accepted.

if you just want to publish your own library, you just need to upload it 
to pypi and make it known.

-- Ronny

On 06.09.2016 17:06, Rui Sarmento wrote:
> Dear Sirs,
>
> I've implemented some stats functions related to Factor Analysis in 
> the statistics area. Specifically, the KMO test and the Bartlett test 
> also. At this time I do not seem to find any module performing these 
> tests. Is there any chance I could add these functions to a package in 
> Python. What is the procedure to perform such contribution.
>
> Thank you very much in advance for the suggestions.
>
> Best Regards,
>
> Rui
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


From opensource at ronnypfannschmidt.de  Fri Sep  9 02:46:41 2016
From: opensource at ronnypfannschmidt.de (Ronny Pfannschmidt)
Date: Fri, 9 Sep 2016 08:46:41 +0200
Subject: [Distutils] [proposal] module dependency specification and
 overriding for packages
Message-ID: <dbbf04f3-abd3-627c-c301-4e59b6ee9959@ronnypfannschmidt.de>

Hi,

while trying to vendor packages that depend on six, i noticed a 
problematic implication

it is impossible to do this without patching those modules, as such 
adding a fragile maintenance burden.

in order to elevate that i propose creating a mechanism that allows to 
physically or virtually map sets of interdependent libraries (that are 
prepared for it into a sub-package of the package vendor-ing it

this would elevate a few issues
a) removing the need to patch
b) making de-vendor-ing of exact versions trivial (just unpacked wheels 
and adding it to the sub-packages __path__)
c) easing updates, as the vendor-ing turns into a process of fetching 
the wheels and unpacking them into the package folder

i do have some implementation ideas, but i would like to hear the 
opinions of others before proposing an exact implementation

-- Ronny


From pombredanne at nexb.com  Fri Sep  9 03:37:26 2016
From: pombredanne at nexb.com (Philippe Ombredanne)
Date: Fri, 9 Sep 2016 09:37:26 +0200
Subject: [Distutils] [proposal] module dependency specification and
 overriding for packages
In-Reply-To: <dbbf04f3-abd3-627c-c301-4e59b6ee9959@ronnypfannschmidt.de>
References: <dbbf04f3-abd3-627c-c301-4e59b6ee9959@ronnypfannschmidt.de>
Message-ID: <CAOFm3uGXapQQdkQfBRQ6QNmaAyH8W6LAkiuYxhenzGB+8KZuDQ@mail.gmail.com>

On Fri, Sep 9, 2016 at 8:46 AM, Ronny Pfannschmidt
<opensource at ronnypfannschmidt.de> wrote:
> while trying to vendor packages that depend on six, i noticed a problematic
> implication
>
> it is impossible to do this without patching those modules, as such adding a
> fragile maintenance burden.


Ronny:
I grok the general issue, but would you have a concrete example?
-- 
Cordially
Philippe Ombredanne

From opensource at ronnypfannschmidt.de  Fri Sep  9 03:48:32 2016
From: opensource at ronnypfannschmidt.de (Ronny Pfannschmidt)
Date: Fri, 9 Sep 2016 09:48:32 +0200
Subject: [Distutils] [proposal] module dependency specification and
 overriding for packages
In-Reply-To: <CAOFm3uGXapQQdkQfBRQ6QNmaAyH8W6LAkiuYxhenzGB+8KZuDQ@mail.gmail.com>
References: <dbbf04f3-abd3-627c-c301-4e59b6ee9959@ronnypfannschmidt.de>
 <CAOFm3uGXapQQdkQfBRQ6QNmaAyH8W6LAkiuYxhenzGB+8KZuDQ@mail.gmail.com>
Message-ID: <f060611c-46dc-6e2e-fbd1-3057009dd1c3@ronnypfannschmidt.de>



On 09.09.2016 09:37, Philippe Ombredanne wrote:
> On Fri, Sep 9, 2016 at 8:46 AM, Ronny Pfannschmidt
> <opensource at ronnypfannschmidt.de> wrote:
>> while trying to vendor packages that depend on six, i noticed a problematic
>> implication
>>
>> it is impossible to do this without patching those modules, as such adding a
>> fragile maintenance burden.
>
> Ronny:
> I grok the general issue, but would you have a concrete example?
a basic example would be vendoring

https://github.com/kyrus/python-junit-xml and six into pytest

following that we would face 2 problems

a) python-junit-xml imports plain six instead of 
_pytest.vendored_packages.six meaning we have to patch it when including it
b) debian/fedora will want to remove six and python-junit-xml from their 
pytest-package
     instead linking to python-six-wheel-... and 
python-junit-xml-wheel... packages as part of their deduplication effort
meaning they will patch around in the pytest package (which tends to 
break the world)


From phil at riverbankcomputing.com  Fri Sep  9 04:06:43 2016
From: phil at riverbankcomputing.com (Phil Thompson)
Date: Fri, 9 Sep 2016 09:06:43 +0100
Subject: [Distutils] Compatibility Tags and the Stable API
Message-ID: <7BB98834-DB8F-4B87-A165-CD4E347F075C@riverbankcomputing.com>

I'm not sure how to interpret PEP 425 when it comes to packages containing C extension modules that use the limited Python API (PEP 384).

The Python v3.4 version of the limited API is used so should the compatibility tag be...

cp34-abi3-*

That would mean that the 'cp34' Python tag would have to be interpreted as a minimum (rather than exact) requirement.

Grateful for any clarification.

Phil

From dholth at gmail.com  Fri Sep  9 07:00:47 2016
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 09 Sep 2016 11:00:47 +0000
Subject: [Distutils] Compatibility Tags and the Stable API
In-Reply-To: <7BB98834-DB8F-4B87-A165-CD4E347F075C@riverbankcomputing.com>
References: <7BB98834-DB8F-4B87-A165-CD4E347F075C@riverbankcomputing.com>
Message-ID: <CAG8k2+5Eod4TD+xZf2-meAiVzTLyuiij0aPYu2F6ibfLG8UpKw@mail.gmail.com>

Correct. I have a pull request on wheel that does just that, and the next
version of pip will be ready for it.

On Fri, Sep 9, 2016, 04:14 Phil Thompson <phil at riverbankcomputing.com>
wrote:

> I'm not sure how to interpret PEP 425 when it comes to packages containing
> C extension modules that use the limited Python API (PEP 384).
>
> The Python v3.4 version of the limited API is used so should the
> compatibility tag be...
>
> cp34-abi3-*
>
> That would mean that the 'cp34' Python tag would have to be interpreted as
> a minimum (rather than exact) requirement.
>
> Grateful for any clarification.
>
> Phil
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160909/32e629dc/attachment.html>

From alex.gronholm at nextday.fi  Fri Sep  9 07:11:28 2016
From: alex.gronholm at nextday.fi (=?UTF-8?Q?Alex_Gr=c3=b6nholm?=)
Date: Fri, 9 Sep 2016 14:11:28 +0300
Subject: [Distutils] Compatibility Tags and the Stable API
In-Reply-To: <CAG8k2+5Eod4TD+xZf2-meAiVzTLyuiij0aPYu2F6ibfLG8UpKw@mail.gmail.com>
References: <7BB98834-DB8F-4B87-A165-CD4E347F075C@riverbankcomputing.com>
 <CAG8k2+5Eod4TD+xZf2-meAiVzTLyuiij0aPYu2F6ibfLG8UpKw@mail.gmail.com>
Message-ID: <74cfbca4-43ee-6686-d51a-669abec201cb@nextday.fi>

Am I understanding this correctly? Even though the limited API is 
supposed to work on all CPython versions supporting the numbered API, I 
cannot do the compiling using a newer version (say 3.5) or it won't work 
with older ones (3.3, 3.4)?


09.09.2016, 14:00, Daniel Holth kirjoitti:
>
> Correct. I have a pull request on wheel that does just that, and the 
> next version of pip will be ready for it.
>
>
> On Fri, Sep 9, 2016, 04:14 Phil Thompson <phil at riverbankcomputing.com 
> <mailto:phil at riverbankcomputing.com>> wrote:
>
>     I'm not sure how to interpret PEP 425 when it comes to packages
>     containing C extension modules that use the limited Python API
>     (PEP 384).
>
>     The Python v3.4 version of the limited API is used so should the
>     compatibility tag be...
>
>     cp34-abi3-*
>
>     That would mean that the 'cp34' Python tag would have to be
>     interpreted as a minimum (rather than exact) requirement.
>
>     Grateful for any clarification.
>
>     Phil
>     _______________________________________________
>     Distutils-SIG maillist  - Distutils-SIG at python.org
>     <mailto:Distutils-SIG at python.org>
>     https://mail.python.org/mailman/listinfo/distutils-sig
>
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160909/92599171/attachment.html>

From donald at stufft.io  Fri Sep  9 07:15:06 2016
From: donald at stufft.io (Donald Stufft)
Date: Fri, 9 Sep 2016 07:15:06 -0400
Subject: [Distutils] Compatibility Tags and the Stable API
In-Reply-To: <74cfbca4-43ee-6686-d51a-669abec201cb@nextday.fi>
References: <7BB98834-DB8F-4B87-A165-CD4E347F075C@riverbankcomputing.com>
 <CAG8k2+5Eod4TD+xZf2-meAiVzTLyuiij0aPYu2F6ibfLG8UpKw@mail.gmail.com>
 <74cfbca4-43ee-6686-d51a-669abec201cb@nextday.fi>
Message-ID: <005C090C-B7FA-4A49-8FD3-EB9F9CF3F21C@stufft.io>


> On Sep 9, 2016, at 7:11 AM, Alex Gr?nholm <alex.gronholm at nextday.fi> wrote:
> 
> Am I understanding this correctly? Even though the limited API is supposed to work on all CPython versions supporting the numbered API, I cannot do the compiling using a newer version (say 3.5) or it won't work with older ones (3.3, 3.4)?

The Py_Limited_API can be either set to an on/off value (in which case it is effectively CPython>=3.2) or you can set it to the hex value of a specific version (in which case it is CPython>=thatversion). So you can compile a Py_Limited_API for older CPythons just fine, you?ll just need to appropriately set the Py_Limited_API value. The reason for this is so CPython can *add* new functions to the stable ABI (but not remove or modify in an ABI incompatible way). So if you use a new to the ABI function, you?ll need a newer minimum.

?
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160909/aec5c64a/attachment.html>

From phil at riverbankcomputing.com  Fri Sep  9 07:28:16 2016
From: phil at riverbankcomputing.com (Phil Thompson)
Date: Fri, 9 Sep 2016 12:28:16 +0100
Subject: [Distutils] Compatibility Tags and the Stable API
In-Reply-To: <CAG8k2+5Eod4TD+xZf2-meAiVzTLyuiij0aPYu2F6ibfLG8UpKw@mail.gmail.com>
References: <7BB98834-DB8F-4B87-A165-CD4E347F075C@riverbankcomputing.com>
 <CAG8k2+5Eod4TD+xZf2-meAiVzTLyuiij0aPYu2F6ibfLG8UpKw@mail.gmail.com>
Message-ID: <D6DF2BF8-1D58-4CDC-AB38-B4F721D28CAD@riverbankcomputing.com>

On 9 Sep 2016, at 12:00 pm, Daniel Holth <dholth at gmail.com> wrote:
> 
> Correct. I have a pull request on wheel that does just that, and the next version of pip will be ready for it.

Is there a release date for that version of pip? There's no point in me making releases that can't be installed.

Phil

> On Fri, Sep 9, 2016, 04:14 Phil Thompson <phil at riverbankcomputing.com> wrote:
> I'm not sure how to interpret PEP 425 when it comes to packages containing C extension modules that use the limited Python API (PEP 384).
> 
> The Python v3.4 version of the limited API is used so should the compatibility tag be...
> 
> cp34-abi3-*
> 
> That would mean that the 'cp34' Python tag would have to be interpreted as a minimum (rather than exact) requirement.
> 
> Grateful for any clarification.
> 
> Phil
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


From ralf.gommers at gmail.com  Fri Sep  9 17:46:08 2016
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 10 Sep 2016 09:46:08 +1200
Subject: [Distutils] Improved Stats Features in Python
In-Reply-To: <15c396aa-e631-09c1-c5c9-aba1beba27b5@ronnypfannschmidt.de>
References: <BLU436-SMTP3C562E7DB597D989CF778E3F90@phx.gbl>
 <15c396aa-e631-09c1-c5c9-aba1beba27b5@ronnypfannschmidt.de>
Message-ID: <CABL7CQhwWNaxum_KG_yNgXqR3uAB8BxgYomsDyLFm2_LKvrNVw@mail.gmail.com>

On Fri, Sep 9, 2016 at 6:29 PM, Ronny Pfannschmidt <
opensource at ronnypfannschmidt.de> wrote:

> Hello Rui,
>
> this mailing list deal with tools you can use to publish 3rd party
> packages to something like the pypi package index,
>
> if you want to add to the python stdlib, you need to get started with
> python-ideas, python-dev and very likely write a PEP that will have to get
> accepted.
>
> if you just want to publish your own library, you just need to upload it
> to pypi and make it known.
>
> -- Ronny
>
> On 06.09.2016 17:06, Rui Sarmento wrote:
>
>> Dear Sirs,
>>
>> I've implemented some stats functions related to Factor Analysis in the
>> statistics area. Specifically, the KMO test and the Bartlett test also. At
>> this time I do not seem to find any module performing these tests. Is there
>> any chance I could add these functions to a package in Python. What is the
>> procedure to perform such contribution.
>>
>
Barlett is already implemented in SciPy:
http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.bartlett.html

KMO isn't available anywhere as far as I can tell; statsmodels would be the
best place if you would like to contribute your implementation there. See
http://statsmodels.sourceforge.net/devel/dev/ for how to go about that. I
wouldn't bother proposing that for stdlib inclusion, it's way too
specialized for that.

Cheers,
Ralf



>> Thank you very much in advance for the suggestions.
>>
>> Best Regards,
>>
>> Rui
>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160910/c306aece/attachment.html>

From ralf.gommers at gmail.com  Fri Sep  9 18:07:16 2016
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 10 Sep 2016 10:07:16 +1200
Subject: [Distutils] Improved Stats Features in Python
In-Reply-To: <BLU436-SMTP249A8D5E388C53E18F6F903E3FA0@phx.gbl>
References: <BLU436-SMTP3C562E7DB597D989CF778E3F90@phx.gbl>
 <15c396aa-e631-09c1-c5c9-aba1beba27b5@ronnypfannschmidt.de>
 <CABL7CQhwWNaxum_KG_yNgXqR3uAB8BxgYomsDyLFm2_LKvrNVw@mail.gmail.com>
 <BLU436-SMTP249A8D5E388C53E18F6F903E3FA0@phx.gbl>
Message-ID: <CABL7CQj2V9OtD32Qt_yw47OECBFAojNsddxfEo1C7LKbMWf0Yg@mail.gmail.com>

On Sat, Sep 10, 2016 at 10:01 AM, Rui Sarmento <rui_sarmento at hotmail.com>
wrote:

> Dear Ralf,
>
> Thank you for your suggestions. About the Bartlett test I'm aware that one
> of his tests (equal variance of samples) is already available. Nonetheless,
> I'm not talking about that particular test but about other Bartlett's test.
> The test I wish to contribute is directed to Factor Analysis and is related
> to the test for sphericity. I'll try to submit both to the statsmodel.
>

Ah okay, thanks - learned something new. For Bartlett's sphericity test
statsmodels is probably also the best place indeed.

Cheers,
Ralf



> Best Regards,
>
> Rui
> ?s 22:46 de 09-09-2016, Ralf Gommers escreveu:
>
>
>
> On Fri, Sep 9, 2016 at 6:29 PM, Ronny Pfannschmidt <
> opensource at ronnypfannschmidt.de> wrote:
>
>> Hello Rui,
>>
>> this mailing list deal with tools you can use to publish 3rd party
>> packages to something like the pypi package index,
>>
>> if you want to add to the python stdlib, you need to get started with
>> python-ideas, python-dev and very likely write a PEP that will have to get
>> accepted.
>>
>> if you just want to publish your own library, you just need to upload it
>> to pypi and make it known.
>>
>> -- Ronny
>>
>> On 06.09.2016 17:06, Rui Sarmento wrote:
>>
>>> Dear Sirs,
>>>
>>> I've implemented some stats functions related to Factor Analysis in the
>>> statistics area. Specifically, the KMO test and the Bartlett test also. At
>>> this time I do not seem to find any module performing these tests. Is there
>>> any chance I could add these functions to a package in Python. What is the
>>> procedure to perform such contribution.
>>>
>>
> Barlett is already implemented in SciPy: http://docs.scipy.org/doc/
> scipy/reference/generated/scipy.stats.bartlett.html
>
> KMO isn't available anywhere as far as I can tell; statsmodels would be
> the best place if you would like to contribute your implementation there.
> See http://statsmodels.sourceforge.net/devel/dev/ for how to go about
> that. I wouldn't bother proposing that for stdlib inclusion, it's way too
> specialized for that.
>
> Cheers,
> Ralf
>
>
>
>>> Thank you very much in advance for the suggestions.
>>>
>>> Best Regards,
>>>
>>> Rui
>>>
>>> _______________________________________________
>>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>>> https://mail.python.org/mailman/listinfo/distutils-sig
>>>
>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160910/1714ca36/attachment.html>

From rui_sarmento at hotmail.com  Fri Sep  9 18:01:55 2016
From: rui_sarmento at hotmail.com (Rui Sarmento)
Date: Fri, 9 Sep 2016 23:01:55 +0100
Subject: [Distutils] Improved Stats Features in Python
In-Reply-To: <CABL7CQhwWNaxum_KG_yNgXqR3uAB8BxgYomsDyLFm2_LKvrNVw@mail.gmail.com>
References: <BLU436-SMTP3C562E7DB597D989CF778E3F90@phx.gbl>
 <15c396aa-e631-09c1-c5c9-aba1beba27b5@ronnypfannschmidt.de>
 <CABL7CQhwWNaxum_KG_yNgXqR3uAB8BxgYomsDyLFm2_LKvrNVw@mail.gmail.com>
Message-ID: <BLU436-SMTP249A8D5E388C53E18F6F903E3FA0@phx.gbl>

Dear Ralf,

Thank you for your suggestions. About the Bartlett test I'm aware that 
one of his tests (equal variance of samples) is already available. 
Nonetheless, I'm not talking about that particular test but about other 
Bartlett's test. The test I wish to contribute is directed to Factor 
Analysis and is related to the test for sphericity. I'll try to submit 
both to the statsmodel.

Best Regards,

Rui

?s 22:46 de 09-09-2016, Ralf Gommers escreveu:
>
>
> On Fri, Sep 9, 2016 at 6:29 PM, Ronny Pfannschmidt 
> <opensource at ronnypfannschmidt.de 
> <mailto:opensource at ronnypfannschmidt.de>> wrote:
>
>     Hello Rui,
>
>     this mailing list deal with tools you can use to publish 3rd party
>     packages to something like the pypi package index,
>
>     if you want to add to the python stdlib, you need to get started
>     with python-ideas, python-dev and very likely write a PEP that
>     will have to get accepted.
>
>     if you just want to publish your own library, you just need to
>     upload it to pypi and make it known.
>
>     -- Ronny
>
>     On 06.09.2016 17:06, Rui Sarmento wrote:
>
>         Dear Sirs,
>
>         I've implemented some stats functions related to Factor
>         Analysis in the statistics area. Specifically, the KMO test
>         and the Bartlett test also. At this time I do not seem to find
>         any module performing these tests. Is there any chance I could
>         add these functions to a package in Python. What is the
>         procedure to perform such contribution.
>
>
> Barlett is already implemented in SciPy: 
> http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.bartlett.html
>
> KMO isn't available anywhere as far as I can tell; statsmodels would 
> be the best place if you would like to contribute your implementation 
> there. See http://statsmodels.sourceforge.net/devel/dev/ for how to go 
> about that. I wouldn't bother proposing that for stdlib inclusion, 
> it's way too specialized for that.
>
> Cheers,
> Ralf
>
>
>
>         Thank you very much in advance for the suggestions.
>
>         Best Regards,
>
>         Rui
>
>         _______________________________________________
>         Distutils-SIG maillist  - Distutils-SIG at python.org
>         <mailto:Distutils-SIG at python.org>
>         https://mail.python.org/mailman/listinfo/distutils-sig
>         <https://mail.python.org/mailman/listinfo/distutils-sig>
>
>
>     _______________________________________________
>     Distutils-SIG maillist  - Distutils-SIG at python.org
>     <mailto:Distutils-SIG at python.org>
>     https://mail.python.org/mailman/listinfo/distutils-sig
>     <https://mail.python.org/mailman/listinfo/distutils-sig>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160909/2b45ed19/attachment-0001.html>

From ncoghlan at gmail.com  Sat Sep 10 06:37:02 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 10 Sep 2016 20:37:02 +1000
Subject: [Distutils] [proposal] module dependency specification and
 overriding for packages
In-Reply-To: <dbbf04f3-abd3-627c-c301-4e59b6ee9959@ronnypfannschmidt.de>
References: <dbbf04f3-abd3-627c-c301-4e59b6ee9959@ronnypfannschmidt.de>
Message-ID: <CADiSq7eWJh5ov5tWnrtWFh9ZyAgKGun8K==cVUuJ7NqNRF58VQ@mail.gmail.com>

On 9 September 2016 at 16:46, Ronny Pfannschmidt
<opensource at ronnypfannschmidt.de> wrote:
> Hi,
>
> while trying to vendor packages that depend on six, i noticed a problematic
> implication
>
> it is impossible to do this without patching those modules, as such adding a
> fragile maintenance burden.

It's already possible to do this without patching by writing an
"install aliases" function that looks something like:

    import sys
    import myapp._vendor.six

    def install_module_aliases():
        sys.modules["six"] = myapp._vendor.six

Run that early in your app startup (before you import anything else),
and "import six" will get your vendored version from the module cache
rather than looking for the normal one.

For app level bundling, though, it's really better to find a way to
deploy a full venv if possible, rather than vendoring things (for
library level work, aim to avoid vendoring in general and use
dependency version specifiers instead).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From opensource at ronnypfannschmidt.de  Sat Sep 10 09:23:00 2016
From: opensource at ronnypfannschmidt.de (Ronny Pfannschmidt)
Date: Sat, 10 Sep 2016 15:23:00 +0200
Subject: [Distutils] [proposal] module dependency specification and
 overriding for packages
In-Reply-To: <CADiSq7eWJh5ov5tWnrtWFh9ZyAgKGun8K==cVUuJ7NqNRF58VQ@mail.gmail.com>
References: <dbbf04f3-abd3-627c-c301-4e59b6ee9959@ronnypfannschmidt.de>
 <CADiSq7eWJh5ov5tWnrtWFh9ZyAgKGun8K==cVUuJ7NqNRF58VQ@mail.gmail.com>
Message-ID: <1f430fab-5eeb-ac26-5010-d2f0c9a272c1@ronnypfannschmidt.de>

Hi,


On 10.09.2016 12:37, Nick Coghlan wrote:
> On 9 September 2016 at 16:46, Ronny Pfannschmidt
> <opensource at ronnypfannschmidt.de> wrote:
>> Hi,
>>
>> while trying to vendor packages that depend on six, i noticed a problematic
>> implication
>>
>> it is impossible to do this without patching those modules, as such adding a
>> fragile maintenance burden.
> It's already possible to do this without patching by writing an
> "install aliases" function that looks something like:
>
>      import sys
>      import myapp._vendor.six
>
>      def install_module_aliases():
>          sys.modules["six"] = myapp._vendor.six
that specifically breaks the very purpose the vendoring happened
which is to decouple from the global installed version

for example if we vendor six in that way in pytest,
we literally break testing anything that depends on a different version 
of six

the same goes for other libraries
> Run that early in your app startup (before you import anything else),
> and "import six" will get your vendored version from the module cache
> rather than looking for the normal one.
>
> For app level bundling, though, it's really better to find a way to
> deploy a full venv if possible, rather than vendoring things (for
> library level work, aim to avoid vendoring in general and use
> dependency version specifiers instead).
for my use-case (which is making a infrastructure package robust and 
having it reuse well tested code)
its basically the worst solution to just pin global dependencies, since 
testing code that depends on different versions will be impossible to test

its seems impossible to avoid vendoring when one wants to use 
library-specific versions of dependencies without literally poisoning 
pythonpath.

the python module system and sys.modules are the main crippling factors 
in that area,
and my understanding so far is, that python-dev will not ever remove 
that limitation.

(and its not a good idea to remove such a limitation without having 
actual working encapsulation)

-- Ronny
> Cheers,
> Nick.
>


From ncoghlan at gmail.com  Sat Sep 10 12:43:05 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 11 Sep 2016 02:43:05 +1000
Subject: [Distutils] [proposal] module dependency specification and
 overriding for packages
In-Reply-To: <1f430fab-5eeb-ac26-5010-d2f0c9a272c1@ronnypfannschmidt.de>
References: <dbbf04f3-abd3-627c-c301-4e59b6ee9959@ronnypfannschmidt.de>
 <CADiSq7eWJh5ov5tWnrtWFh9ZyAgKGun8K==cVUuJ7NqNRF58VQ@mail.gmail.com>
 <1f430fab-5eeb-ac26-5010-d2f0c9a272c1@ronnypfannschmidt.de>
Message-ID: <CADiSq7ccF9XEoeTr0=SAF6KSDOHfkoJkiNvEY2RQu5TiCSznmg@mail.gmail.com>

On 10 September 2016 at 23:23, Ronny Pfannschmidt
<opensource at ronnypfannschmidt.de> wrote:
> Hi,
>
>
> On 10.09.2016 12:37, Nick Coghlan wrote:
>>
>> On 9 September 2016 at 16:46, Ronny Pfannschmidt
>> <opensource at ronnypfannschmidt.de> wrote:
>>>
>>> Hi,
>>>
>>> while trying to vendor packages that depend on six, i noticed a
>>> problematic
>>> implication
>>>
>>> it is impossible to do this without patching those modules, as such
>>> adding a
>>> fragile maintenance burden.
>>
>> It's already possible to do this without patching by writing an
>> "install aliases" function that looks something like:
>>
>>      import sys
>>      import myapp._vendor.six
>>
>>      def install_module_aliases():
>>          sys.modules["six"] = myapp._vendor.six
>
> that specifically breaks the very purpose the vendoring happened
> which is to decouple from the global installed version

Right, I'd missed the context that you were wanting to do this in
py.test while leaving the global module namespace freely available for
the code under test.

> for example if we vendor six in that way in pytest,
> we literally break testing anything that depends on a different version of
> six

As long as the software under test is running in the same process as
the pytest infrastructure, you're always going to have that problem of
potential side effects via the interpreter's global state.

The most you can achieve is to put tight scope control on your global
modifications, such that you can do:

  - modify sys.modules to point to the vendored libraries
  - force load everything that needs to see the vendored libraries
  - revert the changes to sys.modules

This is a pretty standard problem for testing infrastructure that runs
in the same process as the testing target.

> the same goes for other libraries
>>
>> Run that early in your app startup (before you import anything else),
>> and "import six" will get your vendored version from the module cache
>> rather than looking for the normal one.
>>
>> For app level bundling, though, it's really better to find a way to
>> deploy a full venv if possible, rather than vendoring things (for
>> library level work, aim to avoid vendoring in general and use
>> dependency version specifiers instead).
>
> for my use-case (which is making a infrastructure package robust and having
> it reuse well tested code)
> its basically the worst solution to just pin global dependencies, since
> testing code that depends on different versions will be impossible to test
>
> its seems impossible to avoid vendoring when one wants to use
> library-specific versions of dependencies without literally poisoning
> pythonpath.

Aye, py.test is in a tough place here, where most of the conventional
porting guidelines don't work for you, and the approach Michael Foord
took for unittest (i.e. separate Python 2 and 3 source trees) isn't
going to be something you want to do either.

pip does successfully vendor a number of other projects, but they tend
to be ones that were themselves designed to use vendoring to minimise
bootstrapping problems.

> the python module system and sys.modules are the main crippling factors in
> that area,
> and my understanding so far is, that python-dev will not ever remove that
> limitation.

Right, even pkg_resources.requires() still restricts you to one
version of a given package per process, it just lets multiple versions
co-exist in the same filesystem tree.

> (and its not a good idea to remove such a limitation without having actual
> working encapsulation)

Exactly - allowing multiple versions of the same library into the
process namespace is problematic enough that we consider it a bug when
the interpreter allows that to happen inadvertently (not always a
fixable bug, but a bug nonetheless). We don't have the internal
sandboxing needed to make that a supportable behaviour, and beyond a
few ideas around ironing the various problems with the subinterpreter
support, no particular desire to add that either (since you're pretty
much reinventing operating system process encapsulation at that point,
and the never ending security arms race that goes with it).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ralf.gommers at gmail.com  Sat Sep 10 17:56:09 2016
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 11 Sep 2016 09:56:09 +1200
Subject: [Distutils] Improved Stats Features in Python
In-Reply-To: <BLU436-SMTP108FC6403D53F032BEFC03CE3FD0@phx.gbl>
References: <BLU436-SMTP3C562E7DB597D989CF778E3F90@phx.gbl>
 <15c396aa-e631-09c1-c5c9-aba1beba27b5@ronnypfannschmidt.de>
 <CABL7CQhwWNaxum_KG_yNgXqR3uAB8BxgYomsDyLFm2_LKvrNVw@mail.gmail.com>
 <BLU436-SMTP249A8D5E388C53E18F6F903E3FA0@phx.gbl>
 <CABL7CQj2V9OtD32Qt_yw47OECBFAojNsddxfEo1C7LKbMWf0Yg@mail.gmail.com>
 <BLU436-SMTP108FC6403D53F032BEFC03CE3FD0@phx.gbl>
Message-ID: <CABL7CQg8Y_BovU+jeJORW35VNMc7ZPTBOV0tAm5fu5kxh5tPzw@mail.gmail.com>

On Sun, Sep 11, 2016 at 6:24 AM, Rui Sarmento <rui_sarmento at hotmail.com>
wrote:

> Dear Ralf,
>
> No problem, its always nice to discover something new. In fact I've seen
> the statsmodel page you sent, talking about submitting with git. I'm not
> familiar with these procedures. Is there a How-To you would suggest me to
> read? It is the first time I submit to these repositories. My goal is to
> submit two functions, one for Bartlett and another for KMO.
>

Did you see that this page expands at the bottom? This is pretty much a
walkthrough of how you go about submitting a PR:
http://statsmodels.sourceforge.net/devel/dev/git_notes.html. It also has
links to a couple of other helpful tutorials.

Cheers,
Ralf


Cheers,
>
> Rui
>
> ?s 23:07 de 09-09-2016, Ralf Gommers escreveu:
>
>
>
> On Sat, Sep 10, 2016 at 10:01 AM, Rui Sarmento <rui_sarmento at hotmail.com>
> wrote:
>
>> Dear Ralf,
>>
>> Thank you for your suggestions. About the Bartlett test I'm aware that
>> one of his tests (equal variance of samples) is already available.
>> Nonetheless, I'm not talking about that particular test but about other
>> Bartlett's test. The test I wish to contribute is directed to Factor
>> Analysis and is related to the test for sphericity. I'll try to submit both
>> to the statsmodel.
>>
>
> Ah okay, thanks - learned something new. For Bartlett's sphericity test
> statsmodels is probably also the best place indeed.
>
> Cheers,
> Ralf
>
>
>
>> Best Regards,
>>
>> Rui
>> ?s 22:46 de 09-09-2016, Ralf Gommers escreveu:
>>
>>
>>
>> On Fri, Sep 9, 2016 at 6:29 PM, Ronny Pfannschmidt <
>> opensource at ronnypfannschmidt.de> wrote:
>>
>>> Hello Rui,
>>>
>>> this mailing list deal with tools you can use to publish 3rd party
>>> packages to something like the pypi package index,
>>>
>>> if you want to add to the python stdlib, you need to get started with
>>> python-ideas, python-dev and very likely write a PEP that will have to get
>>> accepted.
>>>
>>> if you just want to publish your own library, you just need to upload it
>>> to pypi and make it known.
>>>
>>> -- Ronny
>>>
>>> On 06.09.2016 17:06, Rui Sarmento wrote:
>>>
>>>> Dear Sirs,
>>>>
>>>> I've implemented some stats functions related to Factor Analysis in the
>>>> statistics area. Specifically, the KMO test and the Bartlett test also. At
>>>> this time I do not seem to find any module performing these tests. Is there
>>>> any chance I could add these functions to a package in Python. What is the
>>>> procedure to perform such contribution.
>>>>
>>>
>> Barlett is already implemented in SciPy: http://docs.scipy.org/doc/scip
>> y/reference/generated/scipy.stats.bartlett.html
>>
>> KMO isn't available anywhere as far as I can tell; statsmodels would be
>> the best place if you would like to contribute your implementation there.
>> See http://statsmodels.sourceforge.net/devel/dev/ for how to go about
>> that. I wouldn't bother proposing that for stdlib inclusion, it's way too
>> specialized for that.
>>
>> Cheers,
>> Ralf
>>
>>
>>
>>>> Thank you very much in advance for the suggestions.
>>>>
>>>> Best Regards,
>>>>
>>>> Rui
>>>>
>>>> _______________________________________________
>>>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>>>> https://mail.python.org/mailman/listinfo/distutils-sig
>>>>
>>>
>>> _______________________________________________
>>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>>> https://mail.python.org/mailman/listinfo/distutils-sig
>>>
>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160911/3da13788/attachment.html>

From ralf.gommers at gmail.com  Sat Sep 10 19:39:59 2016
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 11 Sep 2016 11:39:59 +1200
Subject: [Distutils] Improved Stats Features in Python
In-Reply-To: <BLU436-SMTP1416FC1CA9D95BA27792EF9E3FD0@phx.gbl>
References: <BLU436-SMTP3C562E7DB597D989CF778E3F90@phx.gbl>
 <15c396aa-e631-09c1-c5c9-aba1beba27b5@ronnypfannschmidt.de>
 <CABL7CQhwWNaxum_KG_yNgXqR3uAB8BxgYomsDyLFm2_LKvrNVw@mail.gmail.com>
 <BLU436-SMTP249A8D5E388C53E18F6F903E3FA0@phx.gbl>
 <CABL7CQj2V9OtD32Qt_yw47OECBFAojNsddxfEo1C7LKbMWf0Yg@mail.gmail.com>
 <BLU436-SMTP108FC6403D53F032BEFC03CE3FD0@phx.gbl>
 <CABL7CQg8Y_BovU+jeJORW35VNMc7ZPTBOV0tAm5fu5kxh5tPzw@mail.gmail.com>
 <BLU436-SMTP1416FC1CA9D95BA27792EF9E3FD0@phx.gbl>
Message-ID: <CABL7CQiQ4cV-KAXori16=1FmOa93243Wi8W_QKBTdMV-Tz6QsA@mail.gmail.com>

On Sun, Sep 11, 2016 at 11:26 AM, Rui Sarmento <rui_sarmento at hotmail.com>
wrote:

> Hi Ralf,
>
> Yes, in fact I was trying to submit in git but I have some doubts. I will
> explore it more tomorrow (it is late here).
>
> For example I still have doubts with this:
>
> "If you are adding new functionality, you need to add it to the
> documentation by editing (or creating) the appropriate file in docs/source
> ."
>
> What exactly is "the appropriate file"?
>
> and also "Open the docs/source/release/versionX.X.rst file that has the
> version number of the next release and add your changes to the appropriate
> section", I see that the last version in the repository is version0.8.rst
> but I'm not sure this is the file I should edit...
>
> Maybe tomorrow with a good night sleep I'll figure it out.
>

Okay, I'd say just don't worry too much about those details if it's not
clear. Here's what you do:
1. Send an email to the statsmodels mailing list saying you want to add KMO
and Bartlett's spericity, ask if that's welcomed and in what file to put
that.
2. Add your functions in that file, and add tests for that (for function in
fname.py, tests go in tests/test_fname.py).
3. Commit that and put it up on your own GitHub account.
4. From there, send a pull request to statsmodels. Or if it's really not in
good enough shape, Cc me (@rgommers) and I'll give you a few pointers.

Let's take the discussion off this list, this is very off-topic.

Cheers,
Ralf



> Cheers,
>
> Rui
> ?s 22:56 de 10-09-2016, Ralf Gommers escreveu:
>
>
>
> On Sun, Sep 11, 2016 at 6:24 AM, Rui Sarmento <rui_sarmento at hotmail.com>
> wrote:
>
>> Dear Ralf,
>>
>> No problem, its always nice to discover something new. In fact I've seen
>> the statsmodel page you sent, talking about submitting with git. I'm not
>> familiar with these procedures. Is there a How-To you would suggest me to
>> read? It is the first time I submit to these repositories. My goal is to
>> submit two functions, one for Bartlett and another for KMO.
>>
>
> Did you see that this page expands at the bottom? This is pretty much a
> walkthrough of how you go about submitting a PR: http://statsmodels.
> sourceforge.net/devel/dev/git_notes.html. It also has links to a couple
> of other helpful tutorials.
>
> Cheers,
> Ralf
>
>
> Cheers,
>>
>> Rui
>>
>> ?s 23:07 de 09-09-2016, Ralf Gommers escreveu:
>>
>>
>>
>> On Sat, Sep 10, 2016 at 10:01 AM, Rui Sarmento <rui_sarmento at hotmail.com>
>> wrote:
>>
>>> Dear Ralf,
>>>
>>> Thank you for your suggestions. About the Bartlett test I'm aware that
>>> one of his tests (equal variance of samples) is already available.
>>> Nonetheless, I'm not talking about that particular test but about other
>>> Bartlett's test. The test I wish to contribute is directed to Factor
>>> Analysis and is related to the test for sphericity. I'll try to submit both
>>> to the statsmodel.
>>>
>>
>> Ah okay, thanks - learned something new. For Bartlett's sphericity test
>> statsmodels is probably also the best place indeed.
>>
>> Cheers,
>> Ralf
>>
>>
>>
>>> Best Regards,
>>>
>>> Rui
>>> ?s 22:46 de 09-09-2016, Ralf Gommers escreveu:
>>>
>>>
>>>
>>> On Fri, Sep 9, 2016 at 6:29 PM, Ronny Pfannschmidt <
>>> opensource at ronnypfannschmidt.de> wrote:
>>>
>>>> Hello Rui,
>>>>
>>>> this mailing list deal with tools you can use to publish 3rd party
>>>> packages to something like the pypi package index,
>>>>
>>>> if you want to add to the python stdlib, you need to get started with
>>>> python-ideas, python-dev and very likely write a PEP that will have to get
>>>> accepted.
>>>>
>>>> if you just want to publish your own library, you just need to upload
>>>> it to pypi and make it known.
>>>>
>>>> -- Ronny
>>>>
>>>> On 06.09.2016 17:06, Rui Sarmento wrote:
>>>>
>>>>> Dear Sirs,
>>>>>
>>>>> I've implemented some stats functions related to Factor Analysis in
>>>>> the statistics area. Specifically, the KMO test and the Bartlett test also.
>>>>> At this time I do not seem to find any module performing these tests. Is
>>>>> there any chance I could add these functions to a package in Python. What
>>>>> is the procedure to perform such contribution.
>>>>>
>>>>
>>> Barlett is already implemented in SciPy: http://docs.scipy.org/doc/scip
>>> y/reference/generated/scipy.stats.bartlett.html
>>>
>>> KMO isn't available anywhere as far as I can tell; statsmodels would be
>>> the best place if you would like to contribute your implementation there.
>>> See http://statsmodels.sourceforge.net/devel/dev/ for how to go about
>>> that. I wouldn't bother proposing that for stdlib inclusion, it's way too
>>> specialized for that.
>>>
>>> Cheers,
>>> Ralf
>>>
>>>
>>>
>>>>> Thank you very much in advance for the suggestions.
>>>>>
>>>>> Best Regards,
>>>>>
>>>>> Rui
>>>>>
>>>>> _______________________________________________
>>>>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>>>>> https://mail.python.org/mailman/listinfo/distutils-sig
>>>>>
>>>>
>>>> _______________________________________________
>>>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>>>> https://mail.python.org/mailman/listinfo/distutils-sig
>>>>
>>>
>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160911/3016018d/attachment-0001.html>

From robertc at robertcollins.net  Sun Sep 11 19:44:03 2016
From: robertc at robertcollins.net (Robert Collins)
Date: Mon, 12 Sep 2016 11:44:03 +1200
Subject: [Distutils] [proposal] module dependency specification and
 overriding for packages
In-Reply-To: <f060611c-46dc-6e2e-fbd1-3057009dd1c3@ronnypfannschmidt.de>
References: <dbbf04f3-abd3-627c-c301-4e59b6ee9959@ronnypfannschmidt.de>
 <CAOFm3uGXapQQdkQfBRQ6QNmaAyH8W6LAkiuYxhenzGB+8KZuDQ@mail.gmail.com>
 <f060611c-46dc-6e2e-fbd1-3057009dd1c3@ronnypfannschmidt.de>
Message-ID: <CAJ3HoZ3iY7AarM81GV9k6-4_cV5PbcS1YUiYjQi0RZojM6yf=g@mail.gmail.com>

So I realise this isn't the specific thing you're dealing with, but
you could use https://pypi.python.org/pypi/junitxml which is mature,
has no external dependencies, and as a LGPL module should be license
compatible with py.test.

-Rob

On 9 September 2016 at 19:48, Ronny Pfannschmidt
<opensource at ronnypfannschmidt.de> wrote:
>
>
> On 09.09.2016 09:37, Philippe Ombredanne wrote:
>>
>> On Fri, Sep 9, 2016 at 8:46 AM, Ronny Pfannschmidt
>> <opensource at ronnypfannschmidt.de> wrote:
>>>
>>> while trying to vendor packages that depend on six, i noticed a
>>> problematic
>>> implication
>>>
>>> it is impossible to do this without patching those modules, as such
>>> adding a
>>> fragile maintenance burden.
>>
>>
>> Ronny:
>> I grok the general issue, but would you have a concrete example?
>
> a basic example would be vendoring
>
> https://github.com/kyrus/python-junit-xml and six into pytest
>
> following that we would face 2 problems
>
> a) python-junit-xml imports plain six instead of
> _pytest.vendored_packages.six meaning we have to patch it when including it
> b) debian/fedora will want to remove six and python-junit-xml from their
> pytest-package
>     instead linking to python-six-wheel-... and python-junit-xml-wheel...
> packages as part of their deduplication effort
> meaning they will patch around in the pytest package (which tends to break
> the world)
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

From patter001 at gmail.com  Sun Sep 11 20:01:22 2016
From: patter001 at gmail.com (KP)
Date: Sun, 11 Sep 2016 20:01:22 -0400
Subject: [Distutils] namespace_package
In-Reply-To: <C4E622FF-08B6-492A-99E3-D245B60E12DD@gmail.com>
References: <CANaFf5Nb-7fRQLV_x2bFarAOKNqegEz0_w2TEbudd7HAkoRT2g@mail.gmail.com>
 <20151201063802.GB8007@platonas>
 <CANaFf5MdrJTWb5mvX-VwMphwM2XSQCNbJhD7vteHBvij3C-9jw@mail.gmail.com>
 <CANaFf5N1isRAWXNaNNmp=cH72P44rkshTXADNEhFMVf66O6JFg@mail.gmail.com>
 <CANaFf5OwM2Q10knb_c1o4gnLZQPUsHEbhunR8Qcc6E0jN0y8SQ@mail.gmail.com>
 <F9AFCE8D-CACC-4E39-AA34-0DF63FB9C7A9@gmail.com>
 <CANaFf5MJJe4nV0oSvMgCAgHxzLei5ktgYGqZtoS4CgF6KkZYUw@mail.gmail.com>
 <7DCE46B4-F363-46EA-9489-4C60D0ED67AF@gmail.com>
 <87wpswosfy.fsf@metapensiero.it>
 <CANaFf5MfkfVPe-2AyGi+mzt3v-ng481p83hDABHkmr2pY_3aag@mail.gmail.com>
 <C4E622FF-08B6-492A-99E3-D245B60E12DD@gmail.com>
Message-ID: <CANaFf5NTOGXpXbUrpS+ZEAVrYh6yuxVaeWVWPQFk_TRtAKG7qA@mail.gmail.com>

I apologize for bringing up this old thread again, but I was working on
another namespace package, and going through this posted BKM again. First
off I noticed I did not give a good Thank You for posting the fix, my
apologies there.

I noticed the original issue (https://github.com/pypa/pip/issues/3)  was
still open, and was wondering, what is the downfall for the new .pth file
that was posted. Any reason that the proposed fix could not be permanently
added to setuptools?



On Fri, Dec 4, 2015 at 12:31 AM, Zvezdan Petkovic <zvezdanpetkovic at gmail.com
> wrote:

> Hi Kevin,
>
> Sorry for not being able to respond to this earlier. I have time only in
> the evening.
>
>
> On Dec 3, 2015, at 3:57 AM, KP <patter001 at gmail.com> wrote:
>
> Yes, the https://github.com/pypa/pip/issues/3 definitely sounds like my
> issue. Seems there is some concern over the nspkg.pth there as well. It
> seems taht the nspkg.pth is a fine idea to replace the install of the
> __init__.py, but that it just doesn?t work to fully extend the locations in
> which a namespace can reside.
>
>
>
> I now understand what the issue is and can help perhaps with some advice
> below.
>
>
> Either way, there a few other workarounds posted there as well, I will
> check them out to see if any of them are more palatable than the one I
> posted here.
>
>
>
> You need to make foo.bar-1.0.0-py2.7-nspkg.pth look like this:
>
> import sys, types, os, pkgutil; p = os.path.join(sys._getframe(1).f_locals['sitedir'],
> *('foo',)); ie = os.path.exists(os.path.join(p,'__init__.py')); m = not
> ie and sys.modules.setdefault('foo',types.ModuleType('foo')); mp = (m
> or []) and m.__dict__.setdefault('__path__',[]); (p not in mp)
> and mp.append(p); mp[:] = m and pkgutil.extend_path(mp, 'foo') or mp
>
> Notice the difference from the default in use of pkgutil.extend_path()
> that helps with editable packages.
>
> How to make this happen automatically is explained below.
>
>
> Thanks Lele for sending that link!
>
> -Kevin
>
>
>
> I also want to thank Lele for pointing out the possible cause of your
> issues.
>
> Now, I?d like to separate two things here:
>
> 1. Defining namespaces correctly ? that should be done as we talked before
> in the standard way.
>
> 2. Issue with the tooling ? in this case pip is the tool that causes
> issues.
>
> Let?s talk about it more. These are the reasons I didn?t recognize Kevin?s
> issue. :-)
>
> - The namespaces work great when defined correctly (#1 above) with
> zc.buildout and other tools.
>   There are good open-source build systems out there that take care of
> namespaces properly.
>
> - When one makes a good custom build system, it can fix the pip
> deficiencies too.
>   For example, the build system specifically used in a company, team, ?
>
> So, my advice is to:
>
> 1. Make a custom distutils Distribution extension, for example,
> MyDistribution (or some better name)
>
>     (a) make custom install_egg_info class that overrides either:
>
>           -  two templates for pth files (for newer versions of
> setuptools) or
>           - install_namespaces method (for older versions of setuptools)
>
>     (b) override get_command_class method in the MyDistribution class with
> your own install_egg_info
>
> 2. Build this package and install it.
>
> 3. Use distclass=MyDistribution in your foo package setup.py (import it
> from your package).
>
> Using this approach everything will work for every of your packages
> automatically.
>
> I attached your original code from foo_test.zip and added to it a minimal
> mydist package + a sample __init__.py file that I think you should use in
> your namespace packages (if you want).
>
> Kevin, you can try this by doing the following:
>
> - create a virtual environment
> - source bin/activate
> - go to mydist package and run ?python setup.py sdist?
> - pip install /path/to/mydist-1.0.0.tar.gz
> - go to your foo_bar package and run ?python setup.py sdist?
> - pip install /path/to/foo.bar-1.0.0.tar.gz
> - pip install -e /path/to/foo_more
> - start the python interpreter in your virtual environment
> - confirm that it works
>
> >>> import foo.bar
> foo.bar imported
> >>> import foo.more
> foo.more imported
>
> I know that this is not a fix for pip, but it is a generic solution that I
> hope helps you.
>
> Perhaps I should make a little better example (better names, better
> __init__.py, ?) and post it as a little project on GitHub that people can
> refer to.
>
> All the best,
>
> Zvezdan
>
>
>
>
>
>
> On Thu, Dec 3, 2015 at 3:19 AM, Lele Gaifax <lele at metapensiero.it> wrote:
>
>> Zvezdan Petkovic <zvezdanpetkovic at gmail.com> writes:
>>
>> > Hi KP,
>> >
>> >> On Dec 2, 2015, at 7:00 PM, KP <patter001 at gmail.com> wrote:
>> >>
>> >> >It looks like you are trying to find a workaround for the problem
>> that perhaps is not a problem at all if you use the standard approach
>> properly.
>> >>
>> >> I'm definitely _trying_ to use a standard approach...That is why I am
>> here posting. Put simply this seems like a valid use case:
>> >>
>> >> >pip install foo.bar
>> >> >pip install -e svn+http://<path-to-foo.more>
>> >>
>> >> Even if both tools have the namespace_package foo, the "foo.more" will
>> not properly import.
>> >>
>> >> How is this going against standard approaches?
>> >
>> > I don?t know.
>> > Without seeing the source code for these packages (which Marius already
>> asked for) everything is hypothetical.
>> > All I know is that I?m using namespaces successfully for over a decade
>> now and never had the need to work around them.
>> >
>> > Show us the code.  We may be able to help better if we can try and
>> reproduce.
>> > Otherwise, it?s hard.
>>
>> It seems that KP case is either https://github.com/pypa/pip/issues/3160
>> or
>> https://github.com/pypa/pip/issues/3, isn't it? Both come with sample
>> code
>> that demonstrate the problem.
>>
>> ciao, lele.
>> --
>> nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
>> real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
>> lele at metapensiero.it  |                 -- Fortunato Depero, 1929.
>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160911/a23c2925/attachment-0001.html>

From rui_sarmento at hotmail.com  Sat Sep 10 14:24:39 2016
From: rui_sarmento at hotmail.com (Rui Sarmento)
Date: Sat, 10 Sep 2016 19:24:39 +0100
Subject: [Distutils] Improved Stats Features in Python
In-Reply-To: <CABL7CQj2V9OtD32Qt_yw47OECBFAojNsddxfEo1C7LKbMWf0Yg@mail.gmail.com>
References: <BLU436-SMTP3C562E7DB597D989CF778E3F90@phx.gbl>
 <15c396aa-e631-09c1-c5c9-aba1beba27b5@ronnypfannschmidt.de>
 <CABL7CQhwWNaxum_KG_yNgXqR3uAB8BxgYomsDyLFm2_LKvrNVw@mail.gmail.com>
 <BLU436-SMTP249A8D5E388C53E18F6F903E3FA0@phx.gbl>
 <CABL7CQj2V9OtD32Qt_yw47OECBFAojNsddxfEo1C7LKbMWf0Yg@mail.gmail.com>
Message-ID: <BLU436-SMTP108FC6403D53F032BEFC03CE3FD0@phx.gbl>

Dear Ralf,

No problem, its always nice to discover something new. In fact I've seen 
the statsmodel page you sent, talking about submitting with git. I'm not 
familiar with these procedures. Is there a How-To you would suggest me 
to read? It is the first time I submit to these repositories. My goal is 
to submit two functions, one for Bartlett and another for KMO.

Cheers,

Rui


?s 23:07 de 09-09-2016, Ralf Gommers escreveu:
>
>
> On Sat, Sep 10, 2016 at 10:01 AM, Rui Sarmento 
> <rui_sarmento at hotmail.com <mailto:rui_sarmento at hotmail.com>> wrote:
>
>     Dear Ralf,
>
>     Thank you for your suggestions. About the Bartlett test I'm aware
>     that one of his tests (equal variance of samples) is already
>     available. Nonetheless, I'm not talking about that particular test
>     but about other Bartlett's test. The test I wish to contribute is
>     directed to Factor Analysis and is related to the test for
>     sphericity. I'll try to submit both to the statsmodel.
>
>
> Ah okay, thanks - learned something new. For Bartlett's sphericity 
> test statsmodels is probably also the best place indeed.
>
> Cheers,
> Ralf
>
>
>     Best Regards,
>
>     Rui
>
>     ?s 22:46 de 09-09-2016, Ralf Gommers escreveu:
>>
>>
>>     On Fri, Sep 9, 2016 at 6:29 PM, Ronny Pfannschmidt
>>     <opensource at ronnypfannschmidt.de
>>     <mailto:opensource at ronnypfannschmidt.de>> wrote:
>>
>>         Hello Rui,
>>
>>         this mailing list deal with tools you can use to publish 3rd
>>         party packages to something like the pypi package index,
>>
>>         if you want to add to the python stdlib, you need to get
>>         started with python-ideas, python-dev and very likely write a
>>         PEP that will have to get accepted.
>>
>>         if you just want to publish your own library, you just need
>>         to upload it to pypi and make it known.
>>
>>         -- Ronny
>>
>>         On 06.09.2016 17:06, Rui Sarmento wrote:
>>
>>             Dear Sirs,
>>
>>             I've implemented some stats functions related to Factor
>>             Analysis in the statistics area. Specifically, the KMO
>>             test and the Bartlett test also. At this time I do not
>>             seem to find any module performing these tests. Is there
>>             any chance I could add these functions to a package in
>>             Python. What is the procedure to perform such contribution.
>>
>>
>>     Barlett is already implemented in SciPy:
>>     http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.bartlett.html
>>     <http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.bartlett.html>
>>
>>     KMO isn't available anywhere as far as I can tell; statsmodels
>>     would be the best place if you would like to contribute your
>>     implementation there. See
>>     http://statsmodels.sourceforge.net/devel/dev/
>>     <http://statsmodels.sourceforge.net/devel/dev/> for how to go
>>     about that. I wouldn't bother proposing that for stdlib
>>     inclusion, it's way too specialized for that.
>>
>>     Cheers,
>>     Ralf
>>
>>
>>
>>             Thank you very much in advance for the suggestions.
>>
>>             Best Regards,
>>
>>             Rui
>>
>>             _______________________________________________
>>             Distutils-SIG maillist  - Distutils-SIG at python.org
>>             <mailto:Distutils-SIG at python.org>
>>             https://mail.python.org/mailman/listinfo/distutils-sig
>>             <https://mail.python.org/mailman/listinfo/distutils-sig>
>>
>>
>>         _______________________________________________
>>         Distutils-SIG maillist  - Distutils-SIG at python.org
>>         <mailto:Distutils-SIG at python.org>
>>         https://mail.python.org/mailman/listinfo/distutils-sig
>>         <https://mail.python.org/mailman/listinfo/distutils-sig>
>>
>>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160910/67ddea28/attachment.html>

From rui_sarmento at hotmail.com  Sat Sep 10 19:26:15 2016
From: rui_sarmento at hotmail.com (Rui Sarmento)
Date: Sun, 11 Sep 2016 00:26:15 +0100
Subject: [Distutils] Improved Stats Features in Python
In-Reply-To: <CABL7CQg8Y_BovU+jeJORW35VNMc7ZPTBOV0tAm5fu5kxh5tPzw@mail.gmail.com>
References: <BLU436-SMTP3C562E7DB597D989CF778E3F90@phx.gbl>
 <15c396aa-e631-09c1-c5c9-aba1beba27b5@ronnypfannschmidt.de>
 <CABL7CQhwWNaxum_KG_yNgXqR3uAB8BxgYomsDyLFm2_LKvrNVw@mail.gmail.com>
 <BLU436-SMTP249A8D5E388C53E18F6F903E3FA0@phx.gbl>
 <CABL7CQj2V9OtD32Qt_yw47OECBFAojNsddxfEo1C7LKbMWf0Yg@mail.gmail.com>
 <BLU436-SMTP108FC6403D53F032BEFC03CE3FD0@phx.gbl>
 <CABL7CQg8Y_BovU+jeJORW35VNMc7ZPTBOV0tAm5fu5kxh5tPzw@mail.gmail.com>
Message-ID: <BLU436-SMTP1416FC1CA9D95BA27792EF9E3FD0@phx.gbl>

Hi Ralf,

Yes, in fact I was trying to submit in git but I have some doubts. I 
will explore it more tomorrow (it is late here).

For example I still have doubts with this:

"If you are adding new functionality, you need to add it to the 
documentation by editing (or creating) the appropriate file in 
|docs/source|."

What exactly is "the appropriate file"?

and also "Open the |docs/source/release/versionX.X.rst| file that has 
the version number of the next release and add your changes to the 
appropriate section", I see that the last version in the repository is 
version0.8.rst but I'm not sure this is the file I should edit...

Maybe tomorrow with a good night sleep I'll figure it out.

Cheers,

Rui

?s 22:56 de 10-09-2016, Ralf Gommers escreveu:
>
>
> On Sun, Sep 11, 2016 at 6:24 AM, Rui Sarmento 
> <rui_sarmento at hotmail.com <mailto:rui_sarmento at hotmail.com>> wrote:
>
>     Dear Ralf,
>
>     No problem, its always nice to discover something new. In fact
>     I've seen the statsmodel page you sent, talking about submitting
>     with git. I'm not familiar with these procedures. Is there a
>     How-To you would suggest me to read? It is the first time I submit
>     to these repositories. My goal is to submit two functions, one for
>     Bartlett and another for KMO.
>
>
> Did you see that this page expands at the bottom? This is pretty much 
> a walkthrough of how you go about submitting a PR: 
> http://statsmodels.sourceforge.net/devel/dev/git_notes.html. It also 
> has links to a couple of other helpful tutorials.
>
> Cheers,
> Ralf
>
>
>     Cheers,
>
>     Rui
>
>
>     ?s 23:07 de 09-09-2016, Ralf Gommers escreveu:
>>
>>
>>     On Sat, Sep 10, 2016 at 10:01 AM, Rui Sarmento
>>     <rui_sarmento at hotmail.com <mailto:rui_sarmento at hotmail.com>> wrote:
>>
>>         Dear Ralf,
>>
>>         Thank you for your suggestions. About the Bartlett test I'm
>>         aware that one of his tests (equal variance of samples) is
>>         already available. Nonetheless, I'm not talking about that
>>         particular test but about other Bartlett's test. The test I
>>         wish to contribute is directed to Factor Analysis and is
>>         related to the test for sphericity. I'll try to submit both
>>         to the statsmodel.
>>
>>
>>     Ah okay, thanks - learned something new. For Bartlett's
>>     sphericity test statsmodels is probably also the best place indeed.
>>
>>     Cheers,
>>     Ralf
>>
>>
>>         Best Regards,
>>
>>         Rui
>>
>>         ?s 22:46 de 09-09-2016, Ralf Gommers escreveu:
>>>
>>>
>>>         On Fri, Sep 9, 2016 at 6:29 PM, Ronny Pfannschmidt
>>>         <opensource at ronnypfannschmidt.de
>>>         <mailto:opensource at ronnypfannschmidt.de>> wrote:
>>>
>>>             Hello Rui,
>>>
>>>             this mailing list deal with tools you can use to publish
>>>             3rd party packages to something like the pypi package index,
>>>
>>>             if you want to add to the python stdlib, you need to get
>>>             started with python-ideas, python-dev and very likely
>>>             write a PEP that will have to get accepted.
>>>
>>>             if you just want to publish your own library, you just
>>>             need to upload it to pypi and make it known.
>>>
>>>             -- Ronny
>>>
>>>             On 06.09.2016 17:06, Rui Sarmento wrote:
>>>
>>>                 Dear Sirs,
>>>
>>>                 I've implemented some stats functions related to
>>>                 Factor Analysis in the statistics area.
>>>                 Specifically, the KMO test and the Bartlett test
>>>                 also. At this time I do not seem to find any module
>>>                 performing these tests. Is there any chance I could
>>>                 add these functions to a package in Python. What is
>>>                 the procedure to perform such contribution.
>>>
>>>
>>>         Barlett is already implemented in SciPy:
>>>         http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.bartlett.html
>>>         <http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.bartlett.html>
>>>
>>>         KMO isn't available anywhere as far as I can tell;
>>>         statsmodels would be the best place if you would like to
>>>         contribute your implementation there. See
>>>         http://statsmodels.sourceforge.net/devel/dev/
>>>         <http://statsmodels.sourceforge.net/devel/dev/> for how to
>>>         go about that. I wouldn't bother proposing that for stdlib
>>>         inclusion, it's way too specialized for that.
>>>
>>>         Cheers,
>>>         Ralf
>>>
>>>
>>>
>>>                 Thank you very much in advance for the suggestions.
>>>
>>>                 Best Regards,
>>>
>>>                 Rui
>>>
>>>                 _______________________________________________
>>>                 Distutils-SIG maillist  - Distutils-SIG at python.org
>>>                 <mailto:Distutils-SIG at python.org>
>>>                 https://mail.python.org/mailman/listinfo/distutils-sig
>>>                 <https://mail.python.org/mailman/listinfo/distutils-sig>
>>>
>>>
>>>             _______________________________________________
>>>             Distutils-SIG maillist  - Distutils-SIG at python.org
>>>             <mailto:Distutils-SIG at python.org>
>>>             https://mail.python.org/mailman/listinfo/distutils-sig
>>>             <https://mail.python.org/mailman/listinfo/distutils-sig>
>>>
>>>
>>
>>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160911/bbf046e5/attachment-0001.html>

From Ryan.Mills at texascapitalbank.com  Tue Sep 13 10:48:11 2016
From: Ryan.Mills at texascapitalbank.com (Mills, Ryan)
Date: Tue, 13 Sep 2016 14:48:11 +0000
Subject: [Distutils] Module Installation Issues
Message-ID: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>

I just recently downloaded Python 3.5 and cannot seem to install any packages like Numpy, etc.  I have tried all the instructions on the website and keep getting errors:

For example, when I type "python -m pip install Numpy" it returns  a Syntax Error. I am completely new to Python so I must be missing something here - I haven't altered any files since installing it the other day. Do I use the Python IDLE Shell? Are there other packages I need to install first?  Any help would be greatly appreciated.

-Ryan

Ryan Mills
Quantitative Risk Analyst, Banking Officer

[cid:image001.jpg at 01CA3228.EF8AEBA0]

Capital Analytics & Stress Testing
2000 McKinney Avenue, Suite 700
Dallas, TX 75201
214.932.6653 direct
20.6653 internal
ryan.mills at texascapitalbank.com<mailto:%0bryan.mills at texascapitalbank.com>

[cid:image001.jpg at 01CA3228.EF8AEBA0]

[TCB_horiz_Log_rgb]
[cid:image004.jpg at 01D20DA2.E1988D60]


If you are not the addressee and have received this email in error, please notify me immediately. This email is confidential and may contain privileged or proprietary information that is unlawful for you to read, copy, distribute, disclose or otherwise use in any way.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160913/f50ac0cc/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 6260 bytes
Desc: image001.jpg
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160913/f50ac0cc/attachment-0003.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.jpg
Type: image/jpeg
Size: 32315 bytes
Desc: image004.jpg
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160913/f50ac0cc/attachment-0004.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 2346 bytes
Desc: image003.jpg
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160913/f50ac0cc/attachment-0005.jpg>

From p.f.moore at gmail.com  Tue Sep 13 13:00:26 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 13 Sep 2016 18:00:26 +0100
Subject: [Distutils] Module Installation Issues
In-Reply-To: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
Message-ID: <CACac1F-_jsttQPHv8uXC=rvrjsbFpji5NqMGZfTpYZ66rGTD0A@mail.gmail.com>

On 13 September 2016 at 15:48, Mills, Ryan <Ryan.Mills at texascapitalbank.com>
wrote:

> Do I use the Python IDLE Shell?


No, pip is a command line utility so you should go to the command prompt
and run "py -m pip install numpy" (I assume you're on Windows - on Unix
you'd need to run the appropriate python command).

Paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160913/a413a0f5/attachment.html>

From steve.dower at python.org  Tue Sep 13 15:39:22 2016
From: steve.dower at python.org (Steve Dower)
Date: Tue, 13 Sep 2016 12:39:22 -0700
Subject: [Distutils] Module Installation Issues
In-Reply-To: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
Message-ID: <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>

It would help if you could post the full error output (sanitizing paths if needed). But you may just need to upgrade pip (python -m install -U pip).

Knowing exactly where the syntax error is coming from will help us figure out which package has the problem. There are at least three involved here.

Cheers,
Steve

Top-posted from my Windows Phone

-----Original Message-----
From: "Mills, Ryan" <Ryan.Mills at texascapitalbank.com>
Sent: ?9/?13/?2016 9:56
To: "distutils-sig at python.org" <distutils-sig at python.org>
Subject: [Distutils] Module Installation Issues

I just recently downloaded Python 3.5 and cannot seem to install any packages like Numpy, etc.  I have tried all the instructions on the website and keep getting errors:
 
For example, when I type ?python ?m pip install Numpy? it returns  a Syntax Error. I am completely new to Python so I must be missing something here ? I haven?t altered any files since installing it the other day. Do I use the Python IDLE Shell? Are there other packages I need to install first?  Any help would be greatly appreciated.  
 
-Ryan
 
Ryan Mills
Quantitative Risk Analyst, Banking Officer

Capital Analytics & Stress Testing
2000 McKinney Avenue, Suite 700
Dallas, TX 75201
214.932.6653 direct
20.6653 internal
ryan.mills at texascapitalbank.com



 
If you are not the addressee and have received this email in error, please notify me immediately. This email is confidential and may contain privileged or proprietary information that is unlawful for you to read, copy, distribute, disclose or otherwise use in any way.

 
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160913/c33e01a3/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 6260 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160913/c33e01a3/attachment-0002.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 2346 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160913/c33e01a3/attachment-0003.jpg>

From matthew.brett at gmail.com  Tue Sep 13 15:58:43 2016
From: matthew.brett at gmail.com (Matthew Brett)
Date: Tue, 13 Sep 2016 12:58:43 -0700
Subject: [Distutils] Code for inserting paths into system configuration?
Message-ID: <CAH6Pt5pa5=G0etFzM-arP2HMwFJfJkdr_h-rKviHsLEwBwq-2g@mail.gmail.com>

Hi,

I was reflecting on the detail of putting pip user bin directories on
the user's path, and was then thinking of making a tiny pip package
like this:

pip install pip_user_config
python -m pip_user_config

where the latter would cleverly insert the user bin directory into the
user's configuration.

Does anyone know of any code to do something like that?

Best,

Matthew

From thomas at kluyver.me.uk  Tue Sep 13 16:12:10 2016
From: thomas at kluyver.me.uk (Thomas Kluyver)
Date: Tue, 13 Sep 2016 21:12:10 +0100
Subject: [Distutils] Module Installation Issues
In-Reply-To: <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
Message-ID: <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>

On Tue, Sep 13, 2016, at 08:39 PM, Steve Dower wrote:
> It would help if you could post the full error output (sanitizing
> paths if needed). But you may just need to upgrade pip (python -m
> install -U pip).

I think Ryan may have typed that command at a Python prompt rather than
a system command prompt. Unfortunately the distinction often isn't
clear in examples, because the experienced developers writing the
instructions are used to guessing which commands are Python and which
are system commands.

One thing I'd quite like to see Python grow is a standard function to
install packages from inside Python. In R, the install.packages()
function is the default way to get third party packages, and I think
staying in one interactive prompt does make things easier for new users
to understand.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160913/5a041013/attachment.html>

From steve.dower at python.org  Tue Sep 13 16:24:12 2016
From: steve.dower at python.org (Steve Dower)
Date: Tue, 13 Sep 2016 13:24:12 -0700
Subject: [Distutils] Module Installation Issues
In-Reply-To: <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
Message-ID: <8b70e550-0165-115b-f426-cb9f500ec8ec@python.org>

On 13Sep2016 1312, Thomas Kluyver wrote:
> On Tue, Sep 13, 2016, at 08:39 PM, Steve Dower wrote:
>> It would help if you could post the full error output (sanitizing
>> paths if needed). But you may just need to upgrade pip (python -m
>> install -U pip).
>
> I think Ryan may have typed that command at a Python prompt rather than
> a system command prompt. Unfortunately the distinction often isn't clear
> in examples, because the experienced developers writing the instructions
> are used to guessing which commands are Python and which are system
> commands.
>
> One thing I'd quite like to see Python grow is a standard function to
> install packages from inside Python. In R, the install.packages()
> function is the default way to get third party packages, and I think
> staying in one interactive prompt does make things easier for new users
> to understand.

Ah, I suspect you're right.

I have considered adding a "Python x.y Shell" start menu item that would 
configure PATH properly for commands like "python" and "pip". 
Instinctively, it seems this would be more useful than a direct shortcut 
to the executable, but at the same time I don't want to start competing 
with all the other app-specific shells out there.

Cheers,
Steve



From sandro at e-den.it  Tue Sep 13 16:07:10 2016
From: sandro at e-den.it (Alessandro Dentella)
Date: Tue, 13 Sep 2016 22:07:10 +0200
Subject: [Distutils] Buoldout2 e setup.py: install_requires ignored
Message-ID: <20160913200710.GA18019@localhost>

Hi,


I just migrated from buildout 1.7 to buildout 2.5. In this migration I
stopped using a recipe that created a virtualenv as a part of
buildout and I now use an external (basic) virtualenv to calll
bootrap, so I can't compare the two configuration in a strict way.

My problem is that now it seems that install_requires declared in
setup.py are simply ignored, while all packages declared in the eggs
directory are used/installed along with all the packages they depends
on.

I can't find what's wrong with my configuration, I report below a
minimal setup that shows the problem.

thanks in advance for any hints

sandro
*:-)

sandro at bluff:/tmp/new$ cat buildout.cfg
[buildout]
parts = django
eggs = django
       ipython
       django_extensions
project-name = mytest

[django]
recipe = djangorecipe
settings = settings
eggs = ${buildout:eggs}

----------------------------------------------------
sandro at bluff:/tmp/new$ cat setup.py
from setuptools import setup, find_packages

setup(
    name = "example with buildout2",
    version = "0.1",
    author = "Sandro",
    author_email = "sandro at e-den.it",
    description = ("Sperem ghe la vaga ben"),
    license = "BSD",
    url = "http://hg.trepalchi.it/",
    #packages=['an_example_pypi_project', 'tests'],
    long_description="Bla bla",
    classifiers=[
        "Development Status :: 3 - Alpha",
        "Topic :: Utilities",
        "License :: OSI Approved :: BSD License",
    ],
    install_requires=[
        'pyquery',
        ]
)


The steps I do::

  mkvirtualenv -p /usr/bin/python3 base3
  /home/sandro/.virtualenvs/base3/bin/python3 bootstrap.py
  ./bin/buildout

And I obtain bin/django where pyquery **is missing**

$ cat bin/django
#!/home/sandro/.virtualenvs/base3/bin/python3

import sys
sys.path[0:0] = [
  '/home/sandro/.buildout/eggs/Django-1.10.1-py3.5.egg',
  '/home/sandro/.buildout/eggs/ipython-5.0.0-py3.5.egg',
  '/home/sandro/.buildout/eggs/djangorecipe-2.2.1-py3.5.egg',
  '/home/sandro/.buildout/eggs/zc.recipe.egg-2.0.3-py3.5.egg',
  '/home/sandro/.buildout/eggs/zc.buildout-2.5.3-py3.5.egg',
  '/home/sandro/.buildout/eggs/pexpect-4.2.1-py3.5.egg',
  '/home/sandro/.buildout/eggs/Pygments-2.1.3-py3.5.egg',
  '/home/sandro/.buildout/eggs/prompt_toolkit-1.0.7-py3.5.egg',
  '/home/sandro/.buildout/eggs/traitlets-4.2.2-py3.5.egg',
  '/home/sandro/.buildout/eggs/simplegeneric-0.8.1-py3.5.egg',
  '/home/sandro/.buildout/eggs/pickleshare-0.7.4-py3.5.egg',
  '/home/sandro/.buildout/eggs/decorator-4.0.10-py3.5.egg',
  '/home/sandro/.buildout/eggs/setuptools-27.1.2-py3.5.egg',
  '/home/sandro/.buildout/eggs/ptyprocess-0.5.1-py3.5.egg',
  '/home/sandro/.buildout/eggs/wcwidth-0.1.7-py3.5.egg',
  '/home/sandro/.buildout/eggs/six-1.10.0-py3.5.egg',
  '/home/sandro/.buildout/eggs/ipython_genutils-0.1.0-py3.5.egg',
  '/tmp/new',
  ]

import djangorecipe.binscripts

if __name__ == '__main__':
    sys.exit(djangorecipe.binscripts.manage('project.settings'))



--
Sandro Dentella  *:-)
http://trepalchi.it                Il portale degli artisti

http://www.reteisi.org             Soluzioni libere per le scuole
http://sqlkit.argolinux.org        SQLkit home page - PyGTK/python/sqlalchemy

From leorochael at gmail.com  Tue Sep 13 16:48:33 2016
From: leorochael at gmail.com (Leonardo Rochael Almeida)
Date: Tue, 13 Sep 2016 17:48:33 -0300
Subject: [Distutils] Buoldout2 e setup.py: install_requires ignored
In-Reply-To: <20160913200710.GA18019@localhost>
References: <20160913200710.GA18019@localhost>
Message-ID: <CAMxX_oNpGXr-TXGa3MwznRvMwn6TR+O_LOrYBEHCpiQxvuwGxw@mail.gmail.com>

Hi Sandro,

I don't know what your previous setup did, but in your current setup, your
`buildout.cfg` is not configured to take your `setup.py` into account in
any way.

In the `eggs` setting, only `django`, `ipython` and `django-settings` are
mentioned. To take your `setup.py` into account it should mention your
package by name as well (by the way, please consider using a name that
doesn't contain spaces, instead of "example with buildout2".

E.g. assuming you rename your package to "mypackage" in `setup.py`, and
this `setup.py` is in the root of your buildout, the following
`buildout.cfg` should install your package and its `install_requires`:

[buildout]
parts = django
develop = .
eggs =
       mypackage
       django
       ipython
       django_extensions
project-name = mytest

[django]
recipe = djangorecipe
settings = settings
eggs = ${buildout:eggs}



On 13 September 2016 at 17:07, Alessandro Dentella <sandro at e-den.it> wrote:

> Hi,
>
>
> I just migrated from buildout 1.7 to buildout 2.5. In this migration I
> stopped using a recipe that created a virtualenv as a part of
> buildout and I now use an external (basic) virtualenv to calll
> bootrap, so I can't compare the two configuration in a strict way.
>
> My problem is that now it seems that install_requires declared in
> setup.py are simply ignored, while all packages declared in the eggs
> directory are used/installed along with all the packages they depends
> on.
>
> I can't find what's wrong with my configuration, I report below a
> minimal setup that shows the problem.
>
> thanks in advance for any hints
>
> sandro
> *:-)
>
> sandro at bluff:/tmp/new$ cat buildout.cfg
> [buildout]
> parts = django
> eggs = django
>        ipython
>        django_extensions
> project-name = mytest
>
> [django]
> recipe = djangorecipe
> settings = settings
> eggs = ${buildout:eggs}
>
> ----------------------------------------------------
> sandro at bluff:/tmp/new$ cat setup.py
> from setuptools import setup, find_packages
>
> setup(
>     name = "example with buildout2",
>     version = "0.1",
>     author = "Sandro",
>     author_email = "sandro at e-den.it",
>     description = ("Sperem ghe la vaga ben"),
>     license = "BSD",
>     url = "http://hg.trepalchi.it/",
>     #packages=['an_example_pypi_project', 'tests'],
>     long_description="Bla bla",
>     classifiers=[
>         "Development Status :: 3 - Alpha",
>         "Topic :: Utilities",
>         "License :: OSI Approved :: BSD License",
>     ],
>     install_requires=[
>         'pyquery',
>         ]
> )
>
>
> The steps I do::
>
>   mkvirtualenv -p /usr/bin/python3 base3
>   /home/sandro/.virtualenvs/base3/bin/python3 bootstrap.py
>   ./bin/buildout
>
> And I obtain bin/django where pyquery **is missing**
>
> $ cat bin/django
> #!/home/sandro/.virtualenvs/base3/bin/python3
>
> import sys
> sys.path[0:0] = [
>   '/home/sandro/.buildout/eggs/Django-1.10.1-py3.5.egg',
>   '/home/sandro/.buildout/eggs/ipython-5.0.0-py3.5.egg',
>   '/home/sandro/.buildout/eggs/djangorecipe-2.2.1-py3.5.egg',
>   '/home/sandro/.buildout/eggs/zc.recipe.egg-2.0.3-py3.5.egg',
>   '/home/sandro/.buildout/eggs/zc.buildout-2.5.3-py3.5.egg',
>   '/home/sandro/.buildout/eggs/pexpect-4.2.1-py3.5.egg',
>   '/home/sandro/.buildout/eggs/Pygments-2.1.3-py3.5.egg',
>   '/home/sandro/.buildout/eggs/prompt_toolkit-1.0.7-py3.5.egg',
>   '/home/sandro/.buildout/eggs/traitlets-4.2.2-py3.5.egg',
>   '/home/sandro/.buildout/eggs/simplegeneric-0.8.1-py3.5.egg',
>   '/home/sandro/.buildout/eggs/pickleshare-0.7.4-py3.5.egg',
>   '/home/sandro/.buildout/eggs/decorator-4.0.10-py3.5.egg',
>   '/home/sandro/.buildout/eggs/setuptools-27.1.2-py3.5.egg',
>   '/home/sandro/.buildout/eggs/ptyprocess-0.5.1-py3.5.egg',
>   '/home/sandro/.buildout/eggs/wcwidth-0.1.7-py3.5.egg',
>   '/home/sandro/.buildout/eggs/six-1.10.0-py3.5.egg',
>   '/home/sandro/.buildout/eggs/ipython_genutils-0.1.0-py3.5.egg',
>   '/tmp/new',
>   ]
>
> import djangorecipe.binscripts
>
> if __name__ == '__main__':
>     sys.exit(djangorecipe.binscripts.manage('project.settings'))
>
>
>
> --
> Sandro Dentella  *:-)
> http://trepalchi.it                Il portale degli artisti
>
> http://www.reteisi.org             Soluzioni libere per le scuole
> http://sqlkit.argolinux.org        SQLkit home page -
> PyGTK/python/sqlalchemy
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160913/f49d16ee/attachment-0001.html>

From jim at jimfulton.info  Tue Sep 13 16:44:24 2016
From: jim at jimfulton.info (Jim Fulton)
Date: Tue, 13 Sep 2016 16:44:24 -0400
Subject: [Distutils] Buoldout2 e setup.py: install_requires ignored
In-Reply-To: <20160913200710.GA18019@localhost>
References: <20160913200710.GA18019@localhost>
Message-ID: <CAPDm-FijW_THytfnUvfH9gPrDNS5oxrQSY3wRHLya9sAEf2AqQ@mail.gmail.com>

On Tue, Sep 13, 2016 at 4:07 PM, Alessandro Dentella <sandro at e-den.it> wrote:
> Hi,
>
>
> I just migrated from buildout 1.7 to buildout 2.5. In this migration I
> stopped using a recipe that created a virtualenv as a part of
> buildout and I now use an external (basic) virtualenv to calll
> bootrap, so I can't compare the two configuration in a strict way.
>
> My problem is that now it seems that install_requires declared in
> setup.py are simply ignored, while all packages declared in the eggs
> directory are used/installed along with all the packages they depends
> on.
>
> I can't find what's wrong with my configuration, I report below a
> minimal setup that shows the problem.
>
> thanks in advance for any hints
>
> sandro
> *:-)
>
> sandro at bluff:/tmp/new$ cat buildout.cfg
> [buildout]
> parts = django
> eggs = django
>        ipython
>        django_extensions
> project-name = mytest

You're missing:

   develop .

to have buildout create and include a develop egg using the current
directory with it's dependencies.

Jim

-- 
Jim Fulton
http://jimfulton.info

From glyph at twistedmatrix.com  Tue Sep 13 17:00:05 2016
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Tue, 13 Sep 2016 14:00:05 -0700
Subject: [Distutils] Code for inserting paths into system configuration?
In-Reply-To: <CAH6Pt5pa5=G0etFzM-arP2HMwFJfJkdr_h-rKviHsLEwBwq-2g@mail.gmail.com>
References: <CAH6Pt5pa5=G0etFzM-arP2HMwFJfJkdr_h-rKviHsLEwBwq-2g@mail.gmail.com>
Message-ID: <40DB323D-75B3-44A3-BC07-B38C10C2F29B@twistedmatrix.com>


> On Sep 13, 2016, at 12:58 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> 
> Hi,
> 
> I was reflecting on the detail of putting pip user bin directories on
> the user's path, and was then thinking of making a tiny pip package
> like this:
> 
> pip install pip_user_config
> python -m pip_user_config
> 
> where the latter would cleverly insert the user bin directory into the
> user's configuration.
> 
> Does anyone know of any code to do something like that?

This is shell- and platform-specific.  This would be a great idea!  But it would be a challenging integration project.

-glyph



From fred at fdrake.net  Tue Sep 13 17:18:35 2016
From: fred at fdrake.net (Fred Drake)
Date: Tue, 13 Sep 2016 17:18:35 -0400
Subject: [Distutils] Buoldout2 e setup.py: install_requires ignored
In-Reply-To: <CAPDm-FijW_THytfnUvfH9gPrDNS5oxrQSY3wRHLya9sAEf2AqQ@mail.gmail.com>
References: <20160913200710.GA18019@localhost>
 <CAPDm-FijW_THytfnUvfH9gPrDNS5oxrQSY3wRHLya9sAEf2AqQ@mail.gmail.com>
Message-ID: <CAFT4OTHDftCUZBky5=pi5zutyH63gO07g4FboB7s9w6O3EuoHw@mail.gmail.com>

On Tue, Sep 13, 2016 at 4:44 PM, Jim Fulton <jim at jimfulton.info> wrote:
> You're missing:
>
>    develop .
>
> to have buildout create and include a develop egg using the current
> directory with it's dependencies.

Ahem.

    develop = .



  -Fred

-- 
Fred L. Drake, Jr.    <fred at fdrake.net>
"A storm broke loose in my mind."  --Albert Einstein

From jim at jimfulton.info  Tue Sep 13 17:25:41 2016
From: jim at jimfulton.info (Jim Fulton)
Date: Tue, 13 Sep 2016 17:25:41 -0400
Subject: [Distutils] Buoldout2 e setup.py: install_requires ignored
In-Reply-To: <CAFT4OTHDftCUZBky5=pi5zutyH63gO07g4FboB7s9w6O3EuoHw@mail.gmail.com>
References: <20160913200710.GA18019@localhost>
 <CAPDm-FijW_THytfnUvfH9gPrDNS5oxrQSY3wRHLya9sAEf2AqQ@mail.gmail.com>
 <CAFT4OTHDftCUZBky5=pi5zutyH63gO07g4FboB7s9w6O3EuoHw@mail.gmail.com>
Message-ID: <CAPDm-FhHoadoP-Nk_xb84i4xbowTRwv=Xp_WRaf+dyRy21L=ww@mail.gmail.com>

Gaaa, I should have stayed mum.  Missed referring to the develop egg
too, as Leonardo pointed out.

Thanks.

Jim

On Tue, Sep 13, 2016 at 5:18 PM, Fred Drake <fred at fdrake.net> wrote:
> On Tue, Sep 13, 2016 at 4:44 PM, Jim Fulton <jim at jimfulton.info> wrote:
>> You're missing:
>>
>>    develop .
>>
>> to have buildout create and include a develop egg using the current
>> directory with it's dependencies.
>
> Ahem.
>
>     develop = .
>
>
>
>   -Fred
>
> --
> Fred L. Drake, Jr.    <fred at fdrake.net>
> "A storm broke loose in my mind."  --Albert Einstein



-- 
Jim Fulton
http://jimfulton.info

From chris.barker at noaa.gov  Tue Sep 13 17:56:13 2016
From: chris.barker at noaa.gov (Chris Barker)
Date: Tue, 13 Sep 2016 14:56:13 -0700
Subject: [Distutils] Module Installation Issues
In-Reply-To: <8b70e550-0165-115b-f426-cb9f500ec8ec@python.org>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <8b70e550-0165-115b-f426-cb9f500ec8ec@python.org>
Message-ID: <CALGmxEJ02jhmRZRHq-CAn7+PgPDWMrX=S9Ujfx0OcK5p-8eAMA@mail.gmail.com>

>
> I think Ryan may have typed that command at a Python prompt rather than
>> a system command prompt. Unfortunately the distinction often isn't clear
>> in examples, because the experienced developers writing the instructions
>> are used to guessing which commands are Python and which are system
>> commands.
>>
>
indeed -- not a rare error for newbies.


> One thing I'd quite like to see Python grow is a standard function to
>> install packages from inside Python.
>
>
indeed:

import pip

pip.install('package_name')

would make lots of sense, except:

1) It is going to have to be a different command than the shell command, so
the above confusion would still be an issue.

2) are there any technicals reasons this would fail -- i.e. sys.path would
have to be re-calculated at least -- any other re-initializaton that would
have to be done?

3) permissions could be an issue, you don't necessarily want python to be
running with the permissions to write to it's install dir....

In short -- I like the idea, but I'm not sure it'll really help the newbie
confusion.

-CHB








> In R, the install.packages()
>> function is the default way to get third party packages, and I think
>> staying in one interactive prompt does make things easier for new users
>> to understand.
>>
>
> Ah, I suspect you're right.
>
> I have considered adding a "Python x.y Shell" start menu item that would
> configure PATH properly for commands like "python" and "pip".
> Instinctively, it seems this would be more useful than a direct shortcut to
> the executable, but at the same time I don't want to start competing with
> all the other app-specific shells out there.
>
> Cheers,
> Steve
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160913/2fb26fc1/attachment-0001.html>

From p.f.moore at gmail.com  Tue Sep 13 18:00:25 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 13 Sep 2016 23:00:25 +0100
Subject: [Distutils] Module Installation Issues
In-Reply-To: <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
Message-ID: <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>

On 13 September 2016 at 21:12, Thomas Kluyver <thomas at kluyver.me.uk> wrote:
> One thing I'd quite like to see Python grow is a standard function to
> install packages from inside Python.

That's not too hard in principle - pip.main(['install', package]) is
basically all you'd need (modulo various design details, and wrapping
it in a nice user friendly function). The bigger problems are:

1. pip is an independent module (albeit shipped with Python) and it's
not clear to what extent Python would want a builtin function that
depends on a non-stdlib module. But that's not a technical issue.
2. Python's module system isn't really designed around installing new
modules while a process is running. There's caching involved, so the
effects can be unpredictable. It's difficult to know whether the
benefits (avoiding confused users who tried to install from a Python
prompt) would outweigh the costs (confused users having installed a
module but not being able to see it because the module system didn't
notice things had changed).

I'm not honestly sure how big the "installing while a process is
running" issue would be - I did a few simple experiments and couldn't
immediately trigger weirdness, but I believe it can happen. And things
get significantly worse if we allow upgrades from the Python prompt
rather than just installs (e.g., if you have already imported
something from the old version and then upgrade).

Overall, it'd probably be a nice thing to have, but it's nowhere near
as simple as it seems at first glance.
Paul

From leorochael at gmail.com  Tue Sep 13 18:21:08 2016
From: leorochael at gmail.com (Leonardo Rochael Almeida)
Date: Tue, 13 Sep 2016 19:21:08 -0300
Subject: [Distutils] Module Installation Issues
In-Reply-To: <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
Message-ID: <CAMxX_oPPtO6nbG6X+ejoCRzG8+Xvi4SS_3VCDyY3oDMHOx4nkA@mail.gmail.com>

On 13 September 2016 at 19:00, Paul Moore <p.f.moore at gmail.com> wrote:

> [...]
>
> I'm not honestly sure how big the "installing while a process is
> running" issue would be - I did a few simple experiments and couldn't
> immediately trigger weirdness, but I believe it can happen. And things
> get significantly worse if we allow upgrades from the Python prompt
> rather than just installs (e.g., if you have already imported
> something from the old version and then upgrade).
>

The only weirdness on install I can imagine would happen is if the
installation creates `.pth` files, as those wouldn't be re-read by the
currently running Python.

The only package I remember having these was PIL, which can't be installed
by pip anyway. Pillow, the PIL fork, doesn't use .pth files.

(easy_install creates and manages .pth files, but if you're using
setuptools APIs to install packages from inside a running process then 1.
you already have a system to automatically make the packages available in
your namespace and 2. you know, or should know, what you're getting into).

Overall, it'd probably be a nice thing to have, but it's nowhere near
> as simple as it seems at first glance.
> Paul


Considering that installing a package and importing a package are two very
different things, I don't see why we couldn't handle the simple case of
installing the package for the first time with a nice API. Just document
the usual caveat emptors.

Cheers,

Leo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160913/87f29423/attachment.html>

From steve.dower at python.org  Tue Sep 13 18:41:45 2016
From: steve.dower at python.org (Steve Dower)
Date: Tue, 13 Sep 2016 15:41:45 -0700
Subject: [Distutils] Module Installation Issues
In-Reply-To: <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
Message-ID: <731fe5e5-0f7a-65ad-a502-52b32dedf2ee@python.org>

On 13Sep2016 1500, Paul Moore wrote:
> On 13 September 2016 at 21:12, Thomas Kluyver <thomas at kluyver.me.uk> wrote:
>> One thing I'd quite like to see Python grow is a standard function to
>> install packages from inside Python.
>
> That's not too hard in principle - pip.main(['install', package]) is
> basically all you'd need (modulo various design details, and wrapping
> it in a nice user friendly function). The bigger problems are:
>
> 1. pip is an independent module (albeit shipped with Python) and it's
> not clear to what extent Python would want a builtin function that
> depends on a non-stdlib module. But that's not a technical issue.
> 2. Python's module system isn't really designed around installing new
> modules while a process is running. There's caching involved, so the
> effects can be unpredictable. It's difficult to know whether the
> benefits (avoiding confused users who tried to install from a Python
> prompt) would outweigh the costs (confused users having installed a
> module but not being able to see it because the module system didn't
> notice things had changed).
>
> I'm not honestly sure how big the "installing while a process is
> running" issue would be - I did a few simple experiments and couldn't
> immediately trigger weirdness, but I believe it can happen. And things
> get significantly worse if we allow upgrades from the Python prompt
> rather than just installs (e.g., if you have already imported
> something from the old version and then upgrade).
>
> Overall, it'd probably be a nice thing to have, but it's nowhere near
> as simple as it seems at first glance.
> Paul

I think it's one of these things where we should suck it up and let the 
90% case work fine, then display a big fat warning if anything weird may 
have happened and let users sort it out themselves.

A simple builtin that tries to run "sys.executable -m pip {args}" in a 
subprocess and displays "success", "success but you should probably 
restart Python now", "failed because you don't have pip", "failed and 
have all the output" is going to make lots of people happy, even if it 
upsets those who want it to be perfect in every scenario.

Cheers,
Steve


From fungi at yuggoth.org  Tue Sep 13 18:21:10 2016
From: fungi at yuggoth.org (Jeremy Stanley)
Date: Tue, 13 Sep 2016 22:21:10 +0000
Subject: [Distutils] Module Installation Issues
In-Reply-To: <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
Message-ID: <20160913222109.GK24597@yuggoth.org>

On 2016-09-13 23:00:25 +0100 (+0100), Paul Moore wrote:
[...]
> And things get significantly worse if we allow upgrades from the
> Python prompt rather than just installs (e.g., if you have already
> imported something from the old version and then upgrade).
[...]

If you need it, and of course only if your software is actually
written to handle it (contextually), there is a function to handle
that:

    https://docs.python.org/2/library/functions.html#reload
    https://docs.python.org/3/library/importlib.html#importlib.reload

I've used it in some very specific situations where you need to
change out parts of a running daemon without restarting the process,
but you really need a lot of extra care to get out of contexts
utilizing the modules being reloaded if you want the replacements to
actually be used.
-- 
Jeremy Stanley

From donald at stufft.io  Tue Sep 13 18:55:49 2016
From: donald at stufft.io (Donald Stufft)
Date: Tue, 13 Sep 2016 18:55:49 -0400
Subject: [Distutils] Module Installation Issues
In-Reply-To: <731fe5e5-0f7a-65ad-a502-52b32dedf2ee@python.org>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
 <731fe5e5-0f7a-65ad-a502-52b32dedf2ee@python.org>
Message-ID: <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>


> On Sep 13, 2016, at 6:41 PM, Steve Dower <steve.dower at python.org> wrote:
> 
> I think it's one of these things where we should suck it up and let the 90% case work fine, then display a big fat warning if anything weird may have happened and let users sort it out themselves.


I am unsure. One of the really egregious and hard to debug weirdness is going to be something like:


>>> import foo.bar  # foo, and foo.bar are in sys.modules
>>> pip.install(?thing?)  # This implicitly upgrades foo
>>> import foo.widget  # the old foo is in sys.modules, but the new foo.widget.

The 90% case works when it?s *only* pure python and there?s no upgrading/downgrading involved, however you can?t control whether there are going to be upgrades/downgrades or not when dependencies are in play because of ==, >=, >, <, <=, etc.

Another problem, particularly on Windows, is going to be stuff like:

>>> import someclibrary
>>> pip.install(?thing?)  # Implicitly upgrades someclibrary

In this case, this would explode because Python will have the someclibrary.dll locked and the upgrade will attempt to remove it.

My fears here are that people are going to get really confused when they make state modifications to their Python environment and they don?t see those modifications reflected (because of sys.modules caching or what have you). You can possibly get around that with reload(), but we already know the problems that reload() has. Of course, that doesn?t stop Python from offering reload(), but I think the difference is that reload() is targeted towards people with weird edge cases who can understand the caveats while the hypothetical ``pip.install()`` is targeted towards people who are beginners who are still having problems figuring out the difference between the REPL and their shell, much less the vagaries of Python?s global state and attempting to modify that in a now running Python program.

Perhaps a better idea would be to add some smarts to the REPL (but not to Python itself) that would detect something like:

>>> pip install

And print a better error message that gives a better indication about what?s gone wrong besides a SyntaxError?


?
Donald Stufft




From matthew.brett at gmail.com  Tue Sep 13 18:59:36 2016
From: matthew.brett at gmail.com (Matthew Brett)
Date: Tue, 13 Sep 2016 15:59:36 -0700
Subject: [Distutils] Module Installation Issues
In-Reply-To: <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
 <731fe5e5-0f7a-65ad-a502-52b32dedf2ee@python.org>
 <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
Message-ID: <CAH6Pt5pmr-W_mvmn-VC6WcAxMm6QWsVVw4heu=HpiaPkjHejjg@mail.gmail.com>

Hi,

On Tue, Sep 13, 2016 at 3:55 PM, Donald Stufft <donald at stufft.io> wrote:
>
>> On Sep 13, 2016, at 6:41 PM, Steve Dower <steve.dower at python.org> wrote:
>>
>> I think it's one of these things where we should suck it up and let the 90% case work fine, then display a big fat warning if anything weird may have happened and let users sort it out themselves.
>
>
> I am unsure. One of the really egregious and hard to debug weirdness is going to be something like:
>
>
>>>> import foo.bar  # foo, and foo.bar are in sys.modules
>>>> pip.install(?thing?)  # This implicitly upgrades foo
>>>> import foo.widget  # the old foo is in sys.modules, but the new foo.widget.
>
> The 90% case works when it?s *only* pure python and there?s no upgrading/downgrading involved, however you can?t control whether there are going to be upgrades/downgrades or not when dependencies are in play because of ==, >=, >, <, <=, etc.
>
> Another problem, particularly on Windows, is going to be stuff like:
>
>>>> import someclibrary
>>>> pip.install(?thing?)  # Implicitly upgrades someclibrary
>
> In this case, this would explode because Python will have the someclibrary.dll locked and the upgrade will attempt to remove it.
>
> My fears here are that people are going to get really confused when they make state modifications to their Python environment and they don?t see those modifications reflected (because of sys.modules caching or what have you). You can possibly get around that with reload(), but we already know the problems that reload() has. Of course, that doesn?t stop Python from offering reload(), but I think the difference is that reload() is targeted towards people with weird edge cases who can understand the caveats while the hypothetical ``pip.install()`` is targeted towards people who are beginners who are still having problems figuring out the difference between the REPL and their shell, much less the vagaries of Python?s global state and attempting to modify that in a now running Python program.
>
> Perhaps a better idea would be to add some smarts to the REPL (but not to Python itself) that would detect something like:
>
>>>> pip install
>
> And print a better error message that gives a better indication about what?s gone wrong besides a SyntaxError?

I was thinking the same thing.  If it could also catch accidental

>>> python3 -m pip install

commands, that would be even better.

Cheers,

Matthew

From steve.dower at python.org  Tue Sep 13 19:06:55 2016
From: steve.dower at python.org (Steve Dower)
Date: Tue, 13 Sep 2016 16:06:55 -0700
Subject: [Distutils] Module Installation Issues
In-Reply-To: <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
 <731fe5e5-0f7a-65ad-a502-52b32dedf2ee@python.org>
 <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
Message-ID: <0b50104e-754c-df74-00df-806d6a91cb5e@python.org>

On 13Sep2016 1555, Donald Stufft wrote:
>
>> On Sep 13, 2016, at 6:41 PM, Steve Dower <steve.dower at python.org> wrote:
>>
>> I think it's one of these things where we should suck it up and let the 90% case work fine, then display a big fat warning if anything weird may have happened and let users sort it out themselves.
>
>
> I am unsure. One of the really egregious and hard to debug weirdness is going to be something like:
>
>
>>>> import foo.bar  # foo, and foo.bar are in sys.modules
>>>> pip.install(?thing?)  # This implicitly upgrades foo
>>>> import foo.widget  # the old foo is in sys.modules, but the new foo.widget.
>

Except my version of this goes more like:

 >>> import foo.bar  # foo, and foo.bar are in sys.modules
 >>> pip.install("thing")  # This implicitly upgrades foo
WARNING: Packages were updated. You need to restart Python now.
 >>> import foo.widget  # because I happily ignore messages starting 
with WARNING

Doesn't matter to me whether it's "safe" to keep going or not - if any 
files are overwritten at all then show the warning. (This is exactly the 
argument I was expecting when I said "those who want it to be perfect in 
every scenario" ;) )

Cheers,
Steve


From steve.dower at python.org  Tue Sep 13 19:09:36 2016
From: steve.dower at python.org (Steve Dower)
Date: Tue, 13 Sep 2016 16:09:36 -0700
Subject: [Distutils] Module Installation Issues
In-Reply-To: <CAH6Pt5pmr-W_mvmn-VC6WcAxMm6QWsVVw4heu=HpiaPkjHejjg@mail.gmail.com>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
 <731fe5e5-0f7a-65ad-a502-52b32dedf2ee@python.org>
 <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
 <CAH6Pt5pmr-W_mvmn-VC6WcAxMm6QWsVVw4heu=HpiaPkjHejjg@mail.gmail.com>
Message-ID: <ccda915a-412f-c30d-1a5c-7f4e02b71be3@python.org>

On 13Sep2016 1559, Matthew Brett wrote:
>> Perhaps a better idea would be to add some smarts to the REPL (but not to Python itself) that would detect something like:
>>
>>>>> pip install
>>
>> And print a better error message that gives a better indication about what?s gone wrong besides a SyntaxError?
>
> I was thinking the same thing.  If it could also catch accidental
>
>>>> python3 -m pip install
>
> commands, that would be even better.

I feel like these would deserve text adventure style errors:

 >>> pip install spam
I don't see a pip here

 >>> python
You're already running a Python!

 >>> pip uninstall eggs
You ask nicely, but pip refuses to uninstall from inside this shell.

Cheers,
Steve

From bussonniermatthias at gmail.com  Tue Sep 13 20:55:55 2016
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Tue, 13 Sep 2016 17:55:55 -0700
Subject: [Distutils] Module Installation Issues
In-Reply-To: <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
 <731fe5e5-0f7a-65ad-a502-52b32dedf2ee@python.org>
 <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
Message-ID: <CANJQusUZZX_YoyC7ZMLuzCgUGeLrGtewJ5ywf7voQb+hyu9HBA@mail.gmail.com>

On Tue, Sep 13, 2016 at 3:55 PM, Donald Stufft <donald at stufft.io> wrote:

> Perhaps a better idea would be to add some smarts to the REPL (but not to Python itself) that would detect something like:
>
>>>> pip install
>
> And print a better error message that gives a better indication about what?s gone wrong besides a SyntaxError?

I've that for IPython:

https://pypi.python.org/pypi/pip_magic

It need to be loaded explicitly, but I'm[1] considering making it
enabled by default.

We're more and more encountering users that are confused with
installing form within the shell; More and more users are in a Python
REPL[2] long before even knowing what a shell command line is and how
to get one. Worse they can choose environment in a dropdown, so having
to tell them they need to activate the env to install things make no
sens for them.

I think a 'pip install', or whatever else that could hook into the
thing to install package (like something that conda could set to
theirown thing) would be great. A message like "You might need to
restart your Python interpreter for changes to be taken into account"
seem enough to C.Y.A and would get us 90% there.

-- 
M

[1] I think other dev agree with me.
[2] in a notebook most of the time

From ncoghlan at gmail.com  Wed Sep 14 00:13:43 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 14 Sep 2016 14:13:43 +1000
Subject: [Distutils] Code for inserting paths into system configuration?
In-Reply-To: <40DB323D-75B3-44A3-BC07-B38C10C2F29B@twistedmatrix.com>
References: <CAH6Pt5pa5=G0etFzM-arP2HMwFJfJkdr_h-rKviHsLEwBwq-2g@mail.gmail.com>
 <40DB323D-75B3-44A3-BC07-B38C10C2F29B@twistedmatrix.com>
Message-ID: <CADiSq7fBYMCdCzD8m+hTuUuvjx-tNtej1WAp=En9H9ocOe-qRQ@mail.gmail.com>

On 14 September 2016 at 07:00, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
> On Sep 13, 2016, at 12:58 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
>> I was reflecting on the detail of putting pip user bin directories on
>> the user's path, and was then thinking of making a tiny pip package
>> like this:
>>
>> pip install pip_user_config
>> python -m pip_user_config
>>
>> where the latter would cleverly insert the user bin directory into the
>> user's configuration.
>>
>> Does anyone know of any code to do something like that?
>
> This is shell- and platform-specific.  This would be a great idea!  But it would be a challenging integration project.

I know the Anaconda installer (the Continuum Analytics one) does this
(since I've reported a bug against it for messing up user level access
to the system SQLite installation), but I don't know if the code for
that is publicly available anywhere (conda itself is open source, but
I'm not sure about the build scripts for their installers).

That said, as far as I know, just covering the discrepancy on Debian
and Ubuntu between "the system pip is patched to use --user by
default" and "~/.local/bin is not on PATH by default when using the
default shell" would cover most of the cases where this comes up.

Even short of automating the config change though, there could be
significant value in having a "check_install_config" utility that you
ran as:

    pip install [--user] check_install_config
    python -m check_install_config

and answered some essential fault diagnosis questions like:

- which Python version is being used?
- is there an active virtual environment?
- is it a venv environment or a virtualenv one?
- where will package installations go by default?
- where will they go with --user specified?
- where will script installations go in those two cases?
- are both of the binary install directories on the user's executable path?

Cheers,
NIck.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Wed Sep 14 00:24:39 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 14 Sep 2016 14:24:39 +1000
Subject: [Distutils] Code for inserting paths into system configuration?
In-Reply-To: <CADiSq7fBYMCdCzD8m+hTuUuvjx-tNtej1WAp=En9H9ocOe-qRQ@mail.gmail.com>
References: <CAH6Pt5pa5=G0etFzM-arP2HMwFJfJkdr_h-rKviHsLEwBwq-2g@mail.gmail.com>
 <40DB323D-75B3-44A3-BC07-B38C10C2F29B@twistedmatrix.com>
 <CADiSq7fBYMCdCzD8m+hTuUuvjx-tNtej1WAp=En9H9ocOe-qRQ@mail.gmail.com>
Message-ID: <CADiSq7dgKNoxhHVB3Ak-DHJ7F0f0PPOypfZOqOr12mNPS_b6nw@mail.gmail.com>

On 14 September 2016 at 14:13, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 14 September 2016 at 07:00, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
>> On Sep 13, 2016, at 12:58 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
>>> I was reflecting on the detail of putting pip user bin directories on
>>> the user's path, and was then thinking of making a tiny pip package
>>> like this:
>>>
>>> pip install pip_user_config
>>> python -m pip_user_config
>>>
>>> where the latter would cleverly insert the user bin directory into the
>>> user's configuration.
>>>
>>> Does anyone know of any code to do something like that?
>>
>> This is shell- and platform-specific.  This would be a great idea!  But it would be a challenging integration project.
>
> I know the Anaconda installer (the Continuum Analytics one) does this
> (since I've reported a bug against it for messing up user level access
> to the system SQLite installation)

Thinking about that a bit more, I realised I was misremembering - the
bug was in the way "conda env" activation scripts worked, not in the
initial installation of Anaconda.

So those activation scripts and the equivalent ones for virtualenv and
venv are worth a look for understanding how to configure a currently
active shell session (which is shell dependent), while venv management
tools like vex and pew show how to launch a new subshell with a
reconfigured environment (which is easier to make shell independent
and portable across platforms).

What we don't currently have is anything which makes it easy to set up
a shell session where you're still using the system Python, but "pip
install" is targeting your user site-packages directory (regardless of
the system-wide default behaviour - this is the default on Debian &
Ubuntu for example, but not yet the default on Fedora et al), and you
can run scripts from "~/.local/bin" (this is the default on Fedora,
but not on Debian and Ubuntu, and not necessarily on Fedora
derivatives either)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Wed Sep 14 01:01:46 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 14 Sep 2016 15:01:46 +1000
Subject: [Distutils] Module Installation Issues
In-Reply-To: <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
 <731fe5e5-0f7a-65ad-a502-52b32dedf2ee@python.org>
 <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
Message-ID: <CADiSq7fBM3cru+x6gugPkqS=Oedp21U6vj1xzhKMAat21dAiog@mail.gmail.com>

On 14 September 2016 at 08:55, Donald Stufft <donald at stufft.io> wrote:
> Another problem, particularly on Windows, is going to be stuff like:
>
>>>> import someclibrary
>>>> pip.install(?thing?)  # Implicitly upgrades someclibrary
>
> In this case, this would explode because Python will have the someclibrary.dll locked and the upgrade will attempt to remove it.
>
> My fears here are that people are going to get really confused when they make state modifications to their Python environment and they don?t see those modifications reflected (because of sys.modules caching or what have you). You can possibly get around that with reload(), but we already know the problems that reload() has. Of course, that doesn?t stop Python from offering reload(), but I think the difference is that reload() is targeted towards people with weird edge cases who can understand the caveats while the hypothetical ``pip.install()`` is targeted towards people who are beginners who are still having problems figuring out the difference between the REPL and their shell, much less the vagaries of Python?s global state and attempting to modify that in a now running Python program.

While I'm normally not a fan, I think this is a case where a widely
ignored warning is the most sensible option - if people see *any* kind
of weirdness after doing a live package install, their first reaction
should be to restart their Python environment and see if the problem
goes away. In a lot of cases things will work fine (since a lot of
modules are well-behaved in this regard), and for those that aren't, a
restart is usually a sufficient remedial step.

To further reduce the support footprint, we can explicitly limit this
API to supporting installation using the *default* configuration
settings only - absolutely no configurability whatsoever other than an
optional version constraint on the package being installed.

Since this is a new API, we could also make it only do --user installs
when outside a virtual environment.

That said, to fend off requests for increased complexity in the
beginner focused `pip.install()` API, it may be necessary to finally
define a semi-supported `pip._custom_install()` API that exposes more
of the bells and whistles supported by `pip install`. The underscore
prefix would clearly indicate the officially unstable nature of the
API, and beginners are unlikely to reach for "pip._custom_install()"
when "pip.install()" exists.

> Perhaps a better idea would be to add some smarts to the REPL (but not to Python itself) that would detect something like:
>
>>>> pip install
>
> And print a better error message that gives a better indication about what?s gone wrong besides a SyntaxError?

I filed http://bugs.python.org/issue28140 for that - we should be able
to hook into the same system that gives special error messages for
"print message" and "exec code" in Python 3. (The related special
casing is actually in the SyntaxError constructor, so this would
affect non-interactive code as well, but I think that's fine for this
case).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From p.f.moore at gmail.com  Wed Sep 14 04:02:15 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 14 Sep 2016 09:02:15 +0100
Subject: [Distutils] Module Installation Issues
In-Reply-To: <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
 <731fe5e5-0f7a-65ad-a502-52b32dedf2ee@python.org>
 <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
Message-ID: <CACac1F-WYGam2+_A5QNoORaKrs=YkLdqeuMLB-HEwcsXQENCvQ@mail.gmail.com>

On 13 September 2016 at 23:55, Donald Stufft <donald at stufft.io> wrote:
> The 90% case works when it?s *only* pure python and there?s no upgrading/downgrading involved, however you can?t control whether there are going to be upgrades/downgrades or not when dependencies are in play because of ==, >=, >, <, <=, etc.

I would be OK with an invocation that only ever installed and never
did an upgrade, as that would avoid the worst situations. Of course,
(a) AFAIK pip doesn't have such a method (other than --no-deps) at the
moment (dependencies are always upgraded if needed) and (b) there is
an ongoing proposal to change "pip install" to always upgrade, and
this is the exactly the opposite of what we'd need from the Python
prompt.

I understood Steve's proposal to be that we detect cases which do an
upgrade rather than a pure install, and in that case report "you
should restart your Python session". That's a common[1] pattern on
Windows, so should be familiar to Windows users, who I suspect are
more likely to hit this issue.

Paul

[1] Common, but not particularly loved - it's the source of the jokes
about "your mouse has moved, do you want to restart Windows for this
change to take effect?"

From sandro at e-den.it  Wed Sep 14 03:05:55 2016
From: sandro at e-den.it (Alessandro Dentella)
Date: Wed, 14 Sep 2016 09:05:55 +0200
Subject: [Distutils] Buoldout2 e setup.py: install_requires ignored
In-Reply-To: <CAMxX_oNpGXr-TXGa3MwznRvMwn6TR+O_LOrYBEHCpiQxvuwGxw@mail.gmail.com>
References: <20160913200710.GA18019@localhost>
 <CAMxX_oNpGXr-TXGa3MwznRvMwn6TR+O_LOrYBEHCpiQxvuwGxw@mail.gmail.com>
Message-ID: <20160914070555.GA30796@localhost>


Hi Leonardo,

On Tue, Sep 13, 2016 at 05:48:33PM -0300, Leonardo Rochael Almeida wrote:
> Hi Sandro,
>
> I don't know what your previous setup did, but in your current setup, your
> `buildout.cfg` is not configured to take your `setup.py` into account in
> any way.
>
> In the `eggs` setting, only `django`, `ipython` and `django-settings` are
> mentioned. To take your `setup.py` into account it should mention your
> package by name as well (by the way, please consider using a name that


thanks that was it. In fact I had it in the old configuration as
well. I somehow managed to drop it when starting anew.


Everything works well now.

Thanks to Fred and Jim as well

sandro
*:-)

--
Sandro Dentella  *:-)
http://trepalchi.it                Il portale degli artisti

http://www.reteisi.org             Soluzioni libere per le scuole
http://sqlkit.argolinux.org        SQLkit home page - PyGTK/python/sqlalchemy

From ncoghlan at gmail.com  Wed Sep 14 07:38:56 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 14 Sep 2016 21:38:56 +1000
Subject: [Distutils] Module Installation Issues
In-Reply-To: <CACac1F-WYGam2+_A5QNoORaKrs=YkLdqeuMLB-HEwcsXQENCvQ@mail.gmail.com>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
 <731fe5e5-0f7a-65ad-a502-52b32dedf2ee@python.org>
 <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
 <CACac1F-WYGam2+_A5QNoORaKrs=YkLdqeuMLB-HEwcsXQENCvQ@mail.gmail.com>
Message-ID: <CADiSq7dF2KbupZVYDg98Tod0ttbzKzcK9f3V=h5o3LtnzerhRw@mail.gmail.com>

On 14 September 2016 at 18:02, Paul Moore <p.f.moore at gmail.com> wrote:
> On 13 September 2016 at 23:55, Donald Stufft <donald at stufft.io> wrote:
>> The 90% case works when it?s *only* pure python and there?s no upgrading/downgrading involved, however you can?t control whether there are going to be upgrades/downgrades or not when dependencies are in play because of ==, >=, >, <, <=, etc.
>
> I would be OK with an invocation that only ever installed and never
> did an upgrade, as that would avoid the worst situations. Of course,
> (a) AFAIK pip doesn't have such a method (other than --no-deps) at the
> moment (dependencies are always upgraded if needed) and (b) there is
> an ongoing proposal to change "pip install" to always upgrade, and
> this is the exactly the opposite of what we'd need from the Python
> prompt.
>
> I understood Steve's proposal to be that we detect cases which do an
> upgrade rather than a pure install, and in that case report "you
> should restart your Python session". That's a common[1] pattern on
> Windows, so should be familiar to Windows users, who I suspect are
> more likely to hit this issue.
>
> Paul
>
> [1] Common, but not particularly loved - it's the source of the jokes
> about "your mouse has moved, do you want to restart Windows for this
> change to take effect?"

Agreed, but if we're able to limit the warning to cases where:

1. The requested package wasn't already present; and
2. Installing it upgraded one or more existing packages

that should eliminate a lot of the warning noise.

We could even enhance the filtering a little further by having pip
look at the top level module names in the built wheel(s) and only emit
the warning if at least one of those is already present in
sys.modules. That way, doing:

    >>> pip.install("i_need_this")
    >>> import i_need_this

in a clean REPL wouldn't emit a warning even if something was upgraded
as part of the install step.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From donald at stufft.io  Wed Sep 14 07:43:20 2016
From: donald at stufft.io (Donald Stufft)
Date: Wed, 14 Sep 2016 07:43:20 -0400
Subject: [Distutils] Module Installation Issues
In-Reply-To: <0b50104e-754c-df74-00df-806d6a91cb5e@python.org>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
 <731fe5e5-0f7a-65ad-a502-52b32dedf2ee@python.org>
 <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
 <0b50104e-754c-df74-00df-806d6a91cb5e@python.org>
Message-ID: <387D6727-6A73-4434-8EA8-930BDE31019B@stufft.io>


> On Sep 13, 2016, at 7:06 PM, Steve Dower <steve.dower at python.org> wrote:
> 
> This is exactly the argument I was expecting when I said "those who want it to be perfect in every scenario" 

FWIW I don?t have a strong feeling on whether we should add it or not. I haven?t been a beginner in a long time and I am not very good at putting myself in their shoes. I legitimately don?t know what would be more or less confusing here, possible weird state (with or without a warning that they might be getting into weird state territory) or an error message telling them to use a real shell and not the Python REPL.

?
Donald Stufft




From ncoghlan at gmail.com  Wed Sep 14 08:31:27 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 14 Sep 2016 22:31:27 +1000
Subject: [Distutils] Module Installation Issues
In-Reply-To: <387D6727-6A73-4434-8EA8-930BDE31019B@stufft.io>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
 <731fe5e5-0f7a-65ad-a502-52b32dedf2ee@python.org>
 <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
 <0b50104e-754c-df74-00df-806d6a91cb5e@python.org>
 <387D6727-6A73-4434-8EA8-930BDE31019B@stufft.io>
Message-ID: <CADiSq7cRmx694W=3S6-N0ic60fYaf02ftkW-2gW+3KaQtH5ONA@mail.gmail.com>

On 14 September 2016 at 21:43, Donald Stufft <donald at stufft.io> wrote:
>
>> On Sep 13, 2016, at 7:06 PM, Steve Dower <steve.dower at python.org> wrote:
>>
>> This is exactly the argument I was expecting when I said "those who want it to be perfect in every scenario"
>
> FWIW I don?t have a strong feeling on whether we should add it or not. I haven?t been a beginner in a long time and I am not very good at putting myself in their shoes. I legitimately don?t know what would be more or less confusing here, possible weird state (with or without a warning that they might be getting into weird state territory) or an error message telling them to use a real shell and not the Python REPL.

I think they're both confusing, but one of them we can be more certain
the user will have the ability to resolve, regardless of their
computing background.

Specifically, someone already familiar with the difference between a
system shell and the Python REPL isn't likely to make the mistake in
question in the first place - "$ pip install name" is clearly a shell
command *if* you know what the "$" indicates, while the unqualified
"pip install name" used in Warehouse for cross-platform consistency
still looks more like a system command than it does a Python
expression (it would be a potentially valid function call in some
other languages though, most notably MATLAB in the context of this
discussion).

For someone that *doesn't* know the difference though, the only thing
we can be sure of is that they know how to run a Python REPL, and
hence presumably how to stop and restart that REPL. It's also the case
that for through-the-browser development, the user may not have system
shell access at all, even if they do already know the difference
between the two environments.

So if we're able to emit a warning that says "NOTE: Updated package
installed for already loaded module '<name>'. Restart Python to load
the new version.", then novice users are likely to take us at our word
and restart Python. Intermediate users may say "Hah, I know
imp.reload(), so I can ignore that warning!" and try to selective
reload things. Experienced users more familiar with what a debugging
nightmare selective reloading can be with a deep module dependency
tree and usage of the "from X import Y" early binding syntax may then
end up going back to just restarting their REPL session after
installing new things :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From marius at gedmin.as  Wed Sep 14 08:27:49 2016
From: marius at gedmin.as (Marius Gedminas)
Date: Wed, 14 Sep 2016 15:27:49 +0300
Subject: [Distutils] Module Installation Issues
In-Reply-To: <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
 <731fe5e5-0f7a-65ad-a502-52b32dedf2ee@python.org>
 <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
Message-ID: <20160914122749.GA24084@platonas>

On Tue, Sep 13, 2016 at 06:55:49PM -0400, Donald Stufft wrote:
> >>> pip install
> 
> And print a better error message that gives a better indication about what?s gone wrong besides a SyntaxError?

This way you could also make

  >>> exit

work like people expect it to work, without the ().

Marius Gedminas
-- 
If your company is not involved in something called "ISO 9000" you probably
have no idea what it is.  If your company _is_ involved in ISO 9000 then you
definitely have no idea what it is.
                                (Scott Adams - The Dilbert principle)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160914/3b367204/attachment.sig>

From ncoghlan at gmail.com  Wed Sep 14 11:11:12 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 15 Sep 2016 01:11:12 +1000
Subject: [Distutils] Module Installation Issues
In-Reply-To: <20160914122749.GA24084@platonas>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
 <731fe5e5-0f7a-65ad-a502-52b32dedf2ee@python.org>
 <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
 <20160914122749.GA24084@platonas>
Message-ID: <CADiSq7d-f+cmtYr4bfi+eUkHbW3uLOHcJ30rL=z45vGN7hmb_g@mail.gmail.com>

On 14 September 2016 at 22:27, Marius Gedminas <marius at gedmin.as> wrote:
> On Tue, Sep 13, 2016 at 06:55:49PM -0400, Donald Stufft wrote:
>> >>> pip install
>>
>> And print a better error message that gives a better indication about what?s gone wrong besides a SyntaxError?
>
> This way you could also make
>
>   >>> exit
>
> work like people expect it to work, without the ().

That's never going to happen, as aside from some additional builtins,
the default REPL will always work the same way as non-interactive
code.

Cheers,
Nick.

P.S. It's also entirely unrelated to the current discussion, as it's
not a SyntaxError, it's a valid reference to the exit builtin injected
by the site module.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From p.f.moore at gmail.com  Wed Sep 14 11:31:17 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 14 Sep 2016 16:31:17 +0100
Subject: [Distutils] Module Installation Issues
In-Reply-To: <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
 <731fe5e5-0f7a-65ad-a502-52b32dedf2ee@python.org>
 <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
Message-ID: <CACac1F_GHi5qD0G=4obD1pbiVU1fK7RM_XN=7y9aXrrKK_G5zQ@mail.gmail.com>

On 13 September 2016 at 23:55, Donald Stufft <donald at stufft.io> wrote:
> Perhaps a better idea would be to add some smarts to the REPL (but not to Python itself) that would detect something like:
>
>>>> pip install
>
> And print a better error message that gives a better indication about what?s gone wrong besides a SyntaxError?

This is pretty easy to do:

def excepthook(typ, value, traceback):
    # Tidy this up to suit
    if typ is SyntaxError and value.text.startswith("pip"):
        print("You may want to run this from a shell prompt")
        return
    sys.__excepthook__(typ, value, traceback)

sys.excepthook = excepthook

Of course, it doesn't solve the problem of a user who doesn't know
what a "shell prompt" is...

Paul

From thomas at kluyver.me.uk  Wed Sep 14 12:00:11 2016
From: thomas at kluyver.me.uk (Thomas Kluyver)
Date: Wed, 14 Sep 2016 17:00:11 +0100
Subject: [Distutils] Module Installation Issues
In-Reply-To: <CACac1F_GHi5qD0G=4obD1pbiVU1fK7RM_XN=7y9aXrrKK_G5zQ@mail.gmail.com>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
 <731fe5e5-0f7a-65ad-a502-52b32dedf2ee@python.org>
 <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
 <CACac1F_GHi5qD0G=4obD1pbiVU1fK7RM_XN=7y9aXrrKK_G5zQ@mail.gmail.com>
Message-ID: <1473868811.587867.725647641.4A844B4B@webmail.messagingengine.com>

On Wed, Sep 14, 2016, at 04:31 PM, Paul Moore wrote:
> Of course, it doesn't solve the problem of a user who doesn't know
> what a "shell prompt" is...

Especially since the interactive Python interpreter is also a shell
prompt. The title of an IDLE window on my system is "Python 3.5.2 Shell"
;-). I'd suggest the terminology "system command prompt" to refer to...
that thing that you should type 'pip' into.

Even if users don't know exactly what it means, indicating that they're
running something in the wrong place gives them a better chance of
working out what to do.

Thomas

From ncoghlan at gmail.com  Wed Sep 14 23:42:44 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 15 Sep 2016 13:42:44 +1000
Subject: [Distutils] Module Installation Issues
In-Reply-To: <CACac1F_GHi5qD0G=4obD1pbiVU1fK7RM_XN=7y9aXrrKK_G5zQ@mail.gmail.com>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <E1bjtYS-0003Bc-It@se2-syd.hostedmail.net.au>
 <1473797530.1546222.724728337.698E3D0B@webmail.messagingengine.com>
 <CACac1F_J9-feMB=DyxcP6F+EwYGECTqJEcFwWnRx7wKiRCpd8A@mail.gmail.com>
 <731fe5e5-0f7a-65ad-a502-52b32dedf2ee@python.org>
 <0EEA49EC-99A2-44D9-B55E-0F6C74E5204D@stufft.io>
 <CACac1F_GHi5qD0G=4obD1pbiVU1fK7RM_XN=7y9aXrrKK_G5zQ@mail.gmail.com>
Message-ID: <CADiSq7crWSKAEYxWenUC5RdG-m+WOF8aq+31B-oEQ1zNiL08OA@mail.gmail.com>

On 15 September 2016 at 01:31, Paul Moore <p.f.moore at gmail.com> wrote:
> On 13 September 2016 at 23:55, Donald Stufft <donald at stufft.io> wrote:
>> Perhaps a better idea would be to add some smarts to the REPL (but not to Python itself) that would detect something like:
>>
>>>>> pip install
>>
>> And print a better error message that gives a better indication about what?s gone wrong besides a SyntaxError?
>
> This is pretty easy to do:
>
> def excepthook(typ, value, traceback):
>     # Tidy this up to suit
>     if typ is SyntaxError and value.text.startswith("pip"):
>         print("You may want to run this from a shell prompt")
>         return
>     sys.__excepthook__(typ, value, traceback)
>
> sys.excepthook = excepthook

Good point, now noted at http://bugs.python.org/issue28140#msg276506

> Of course, it doesn't solve the problem of a user who doesn't know
> what a "shell prompt" is...

As Thomas suggested, even though *we* use "shell" specifically for the
system shell, and "REPL" for the interactive prompt, something like
"system command prompt" is likely the better phrase, since the REPL
does get referred to as a "Python shell" fairly often, including by
IDLE.

Plugging "system command prompt" into Google doesn't give helpful
results, though.

An unqualified "command prompt" is better, as searching for that gives
generic Windows command line access instructions, while qualifying it
with "command prompt mac os x" gives Mac instructions and "command
prompt linux" gives decent Linux instructions.

As far as the error message goes, what I'd suggest is:

1. detect "pip install" specifically (anywhere in the line), not just "pip"
2. report what triggered the custom recommendation in addition to the
recommendation itself

For example:

    >>> python ?m pip install Numpy
    SyntaxError: 'pip install' is a command line operation
    Try running this at a command prompt rather than from Python

Once the pip bundled via ensurepip gains "pip.install()" support
(assuming we end up doing that), then the custom message can change
to:

    >>> python ?m pip install Numpy
    SyntaxError: 'pip install' is a command line operation
    Try running this at a command prompt rather than from Python, or else try:

        import pip
        pip.install("NumPy")

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From wes.turner at gmail.com  Thu Sep 15 08:58:22 2016
From: wes.turner at gmail.com (Wes Turner)
Date: Thu, 15 Sep 2016 07:58:22 -0500
Subject: [Distutils] Module Installation Issues
In-Reply-To: <CACfEFw9-ZZ0Vfp7_Fqr2usKrc5G-u6hryBigzAFpcNc3T3vaRg@mail.gmail.com>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <CACfEFw9-ZZ0Vfp7_Fqr2usKrc5G-u6hryBigzAFpcNc3T3vaRg@mail.gmail.com>
Message-ID: <CACfEFw83-1EfPTqbtH0W8Pz4r=Qxn2y-2rZajrpCUjr2RMTCAg@mail.gmail.com>

Useful banking/finance Python resources:

- https://pypi.python.org/pypi/ofxparse
- https://wrdrd.com/docs/consulting/investing#quantopian
- Places that have some Python in their stack:
  - JP Morgan
  - Bank of America Merrill Lynch
  - Quantopian
   - qgrid, zipline, pyfolio, ...


On Thursday, September 15, 2016, Wes Turner <wes.turner at gmail.com> wrote:

>
>
> On Tuesday, September 13, 2016, Mills, Ryan <Ryan.Mills at texascapitalbank.
> com <javascript:_e(%7B%7D,'cvml','Ryan.Mills at texascapitalbank.com');>>
> wrote:
>
>> I just recently downloaded Python 3.5 and cannot seem to install any
>> packages like Numpy, etc.  I have tried all the instructions on the website
>> and keep getting errors:
>>
>>
>>
>> For example, when I type ?python ?m pip install Numpy? it returns  a
>> Syntax Error. I am completely new to Python so I must be missing something
>> here ? I haven?t altered any files since installing it the other day. Do I
>> use the Python IDLE Shell? Are there other packages I need to install
>> first?  Any help would be greatly appreciated.
>>
>
> Shell commands can/could/should be clearly indicated with a '$' prefix:
>
>     $ python -m pip install numpy
>
> Python expressions are sometimes prefixed with '>>>':
>
>     >>> import numpy as np
>
> TBH, as a beginner, it's probably way easier to start with Anaconda (conda
> packages) and pip (python packages) (because there are a number of
> libraries required to build numpy and then whatever else):
>
> - https://docs.continuum.io/anaconda/
> - $ conda install pip; python -m pip install
> - $ which python; python -m site
>
> The software carpentry lessons are a good place to start from:
>
> - http://software-carpentry.org/lessons/
>   - http://swcarpentry.github.io/python-novice-inflammation/
>
> Also great resources:
>
> - http://www.scipy-lectures.org
> - https://github.com/jrjohansson/scientific-python-lectures
> (IPython/Jupyter nb)
>
> - https://westurner.org/wiki/awesome-python-testing#python
>   - https://westurner.org/tools/#python
>   - http://learnxinyminutes.com/docs/python/
>   - http://docs.python.org/2/library/unittest.html
>
>   - https://westurner.org/tools/#numpy
>
> - IDLE is pretty cool.
> - IPython is great.
>
> - Jupyter Notebook is a reverse shell.
> - Spyder ($ conda install spyder)
> - [commercial IDE preference]
>
>
> - If you write tests from the start, there's less run/check/run/check
> manual testing and more test coverage.
>
>
>
>>
>> -Ryan
>>
>>
>>
>> Ryan Mills
>>
>> *Quantitative Risk Analyst, Banking Officer*
>>
>> [image: cid:image001.jpg at 01CA3228.EF8AEBA0]
>>
>> Capital Analytics & Stress Testing
>> 2000 McKinney Avenue, Suite 700
>>
>> Dallas, TX 75201
>>
>> 214.932.6653 direct
>>
>> 20.6653 internal
>> ryan.mills at texascapitalbank.com
>>
>> [image: cid:image001.jpg at 01CA3228.EF8AEBA0]
>>
>> [image: TCB_horiz_Log_rgb]
>>
>>
>>
>> If you are not the addressee and have received this email in error,
>> please notify me immediately. This email is confidential and may contain
>> privileged or proprietary information that is unlawful for you to read,
>> copy, distribute, disclose or otherwise use in any way.
>>
>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160915/8b21895e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.jpg
Type: image/jpeg
Size: 32315 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160915/8b21895e/attachment-0003.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 6260 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160915/8b21895e/attachment-0004.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 2346 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160915/8b21895e/attachment-0005.jpg>

From wes.turner at gmail.com  Thu Sep 15 08:53:44 2016
From: wes.turner at gmail.com (Wes Turner)
Date: Thu, 15 Sep 2016 07:53:44 -0500
Subject: [Distutils] Module Installation Issues
In-Reply-To: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
Message-ID: <CACfEFw9-ZZ0Vfp7_Fqr2usKrc5G-u6hryBigzAFpcNc3T3vaRg@mail.gmail.com>

On Tuesday, September 13, 2016, Mills, Ryan <Ryan.Mills at texascapitalbank.com>
wrote:

> I just recently downloaded Python 3.5 and cannot seem to install any
> packages like Numpy, etc.  I have tried all the instructions on the website
> and keep getting errors:
>
>
>
> For example, when I type ?python ?m pip install Numpy? it returns  a
> Syntax Error. I am completely new to Python so I must be missing something
> here ? I haven?t altered any files since installing it the other day. Do I
> use the Python IDLE Shell? Are there other packages I need to install
> first?  Any help would be greatly appreciated.
>

Shell commands can/could/should be clearly indicated with a '$' prefix:

    $ python -m pip install numpy

Python expressions are sometimes prefixed with '>>>':

    >>> import numpy as np

TBH, as a beginner, it's probably way easier to start with Anaconda (conda
packages) and pip (python packages) (because there are a number of
libraries required to build numpy and then whatever else):

- https://docs.continuum.io/anaconda/
- $ conda install pip; python -m pip install
- $ which python; python -m site

The software carpentry lessons are a good place to start from:

- http://software-carpentry.org/lessons/
  - http://swcarpentry.github.io/python-novice-inflammation/

Also great resources:

- http://www.scipy-lectures.org
- https://github.com/jrjohansson/scientific-python-lectures
(IPython/Jupyter nb)

- https://westurner.org/wiki/awesome-python-testing#python
  - https://westurner.org/tools/#python
  - http://learnxinyminutes.com/docs/python/
  - http://docs.python.org/2/library/unittest.html

  - https://westurner.org/tools/#numpy

- IDLE is pretty cool.
- IPython is great.

- Jupyter Notebook is a reverse shell.
- Spyder ($ conda install spyder)
- [commercial IDE preference]


- If you write tests from the start, there's less run/check/run/check
manual testing and more test coverage.



>
> -Ryan
>
>
>
> Ryan Mills
>
> *Quantitative Risk Analyst, Banking Officer*
>
> [image: cid:image001.jpg at 01CA3228.EF8AEBA0]
>
> Capital Analytics & Stress Testing
> 2000 McKinney Avenue, Suite 700
>
> Dallas, TX 75201
>
> 214.932.6653 direct
>
> 20.6653 internal
> ryan.mills at texascapitalbank.com
> <javascript:_e(%7B%7D,'cvml','%5Cx0bryan.mills at texascapitalbank.com');>
>
> [image: cid:image001.jpg at 01CA3228.EF8AEBA0]
>
> [image: TCB_horiz_Log_rgb]
>
>
>
> If you are not the addressee and have received this email in error, please
> notify me immediately. This email is confidential and may contain
> privileged or proprietary information that is unlawful for you to read,
> copy, distribute, disclose or otherwise use in any way.
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160915/8bd17a7e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 2346 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160915/8bd17a7e/attachment-0003.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.jpg
Type: image/jpeg
Size: 32315 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160915/8bd17a7e/attachment-0004.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 6260 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160915/8bd17a7e/attachment-0005.jpg>

From thomas at kluyver.me.uk  Thu Sep 15 09:06:16 2016
From: thomas at kluyver.me.uk (Thomas Kluyver)
Date: Thu, 15 Sep 2016 14:06:16 +0100
Subject: [Distutils] Module Installation Issues
In-Reply-To: <CACfEFw9-ZZ0Vfp7_Fqr2usKrc5G-u6hryBigzAFpcNc3T3vaRg@mail.gmail.com>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <CACfEFw9-ZZ0Vfp7_Fqr2usKrc5G-u6hryBigzAFpcNc3T3vaRg@mail.gmail.com>
Message-ID: <1473944776.2424257.726646049.66476727@webmail.messagingengine.com>

On Thu, Sep 15, 2016, at 01:53 PM, Wes Turner wrote:
> Shell commands can/could/should be clearly indicated with a '$'
> prefix:
>
>     $ python -m pip install numpy

That's a common convention, but:

- It comes from POSIX platforms where the default prompt ends in $. The
  default prompt in Windows ends with >, so $ is not clear.
- On all platforms, new users may well not recognise that convention.
- Users may think they have to type the $ as part of the command,
  leading to even more confusion.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160915/52e5103d/attachment.html>

From wes.turner at gmail.com  Thu Sep 15 15:38:57 2016
From: wes.turner at gmail.com (Wes Turner)
Date: Thu, 15 Sep 2016 14:38:57 -0500
Subject: [Distutils] Module Installation Issues
In-Reply-To: <1473944776.2424257.726646049.66476727@webmail.messagingengine.com>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <CACfEFw9-ZZ0Vfp7_Fqr2usKrc5G-u6hryBigzAFpcNc3T3vaRg@mail.gmail.com>
 <1473944776.2424257.726646049.66476727@webmail.messagingengine.com>
Message-ID: <CACfEFw9MFiLxEOt1AycUYBpr8kU46rUiDBzryEDbMTap7QbM6A@mail.gmail.com>

You can install bash on windows. (GoW, Cygwin, Ming), but Docker (and,
soon, runC OCP 1.0) is probably your best bet for maximum compatibility
with most {python, open source} tutorials. Installing the code as non-root
is a good idea (e.g. with a .zip or a .tar.gz or a Python .whl wheel)


I think the bash $ and # PS1 should be fairly easy to differentiate from
unprefixed expressions. IDK if those are the POSIX defaults.

These list {Linux,Mac} when Windows is not supported:
https://docs.continuum.io/anaconda/pkg-docs

.rst/ReStructuredText: code directive

.. code:: bash

    # Check out the switches for ifconfig/ipconfig /h.


.. code:: python

   """PowerShell is open source now"""


.md/Markdown: fenced code block

```bash
bash --version
help declare -r
```

```python
python -v
```

On Thursday, September 15, 2016, Thomas Kluyver <thomas at kluyver.me.uk>
wrote:

> On Thu, Sep 15, 2016, at 01:53 PM, Wes Turner wrote:
>
> Shell commands can/could/should be clearly indicated with a '$' prefix:
>
>     $ python -m pip install numpy
>
>
> That's a common convention, but:
>
> - It comes from POSIX platforms where the default prompt ends in $. The
> default prompt in Windows ends with >, so $ is not clear.
> - On all platforms, new users may well not recognise that convention.
> - Users may think they have to type the $ as part of the command, leading
> to even more confusion.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160915/e2b3b54a/attachment.html>

From ncoghlan at gmail.com  Fri Sep 16 02:33:44 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 16 Sep 2016 16:33:44 +1000
Subject: [Distutils] Module Installation Issues
In-Reply-To: <CACfEFw9MFiLxEOt1AycUYBpr8kU46rUiDBzryEDbMTap7QbM6A@mail.gmail.com>
References: <808BFDDCC0AFE64FA610908F2EB075AC8A129571@RDPIEXCHMB2.tcbna.net>
 <CACfEFw9-ZZ0Vfp7_Fqr2usKrc5G-u6hryBigzAFpcNc3T3vaRg@mail.gmail.com>
 <1473944776.2424257.726646049.66476727@webmail.messagingengine.com>
 <CACfEFw9MFiLxEOt1AycUYBpr8kU46rUiDBzryEDbMTap7QbM6A@mail.gmail.com>
Message-ID: <CADiSq7d8oeRPPPC0bzr66i-0HV2ERCSweunuEr_v3vH=TWhBoQ@mail.gmail.com>

On 16 September 2016 at 05:38, Wes Turner <wes.turner at gmail.com> wrote:
> You can install bash on windows. (GoW, Cygwin, Ming), but Docker (and, soon,
> runC OCP 1.0) is probably your best bet for maximum compatibility with most
> {python, open source} tutorials.

Wes, none of these are beginner friendly recommendations - our target
audience here is folks that *don't yet know what the phrase "command
prompt" means*.

So while there are a myriad of ways to get access to a *nix prompt on
Windows, they're all targeted at folks that are already experienced
developers, and they're also the wrong answer when the goal is to help
people to more effectively use the computer they already have, not the
computer we happen to think they *should* have.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Sep 17 06:29:52 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 17 Sep 2016 20:29:52 +1000
Subject: [Distutils] Current Python packaging status (from my point of view)
Message-ID: <CADiSq7cWUKd=EGTDc7w=gxTx-afnQbt_aTCvMUTkDEQEbjbUjQ@mail.gmail.com>

Hi folks,

Prompted by a few posts I read recently about the current state of the
Python packaging ecosystem, I figured it made sense to put together an
article summarising my own perspective on the current state of things:
http://www.curiousefficiency.org/posts/2016/09/python-packaging-ecosystem.html

It's pretty long, so the short version tailored specifically for the
distutils-sig audience would be:

* restating the point that pip & conda solve different problems, so
while there's some overlap in their core capabilities, neither is a
substitute for the other
* we've actually managed to put some pretty hard problems behind us in
the last few years, so my thanks to everyone that's played a part in
that process
* some of the thorniest problems that still remain really do require
proper funding of the core ecosystem infrastructure (most notably
PyPI), which means that either commercial redistributors need to step
up and handle the problem on behalf of their customers, or else the
PSF needs to figure out alternative sources of funding (with "let's do
both!" really being my preferred outcome on that front)

Cheers,
Nick.

P.S. For the Twitter users amongst you, feel free to pass along my
link to the article:
https://twitter.com/ncoghlan_dev/status/777087670837088256

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From jason at jasonantman.com  Sun Sep 18 07:42:54 2016
From: jason at jasonantman.com (Jason Antman)
Date: Sun, 18 Sep 2016 07:42:54 -0400
Subject: [Distutils] Publicly Queryable Statistics
Message-ID: <CAFt4V4kKm3cD_xBz=rKVV9BuGq1hCPUQNUm-5bMMb8WkWoRZSQ@mail.gmail.com>

Donald,

Thanks so much for all of your work on this.

I know it's an old thread, but I've built a tool that generates some
statistics from this information. It queries data for a given list of
projects, or for all of a specific user's projects, caches the data locally
on disk, and generates both a HTML report with a bunch of graphs, as well
as download badges. For my own use, I'm running it cron'ed from my desktop
once a night, and uploading the reports and badges to a public Amazon S3
bucket.

The project is: https://pypi.org/project/pypi-download-stats/

Example output:
http://jantman-personal-public.s3-website-us-east-1.amazonaws.com/pypi-stats/awslimitchecker/index.html

It's a bit rough around the edges, and currently doesn't have any unit
tests - my hope is that this will be an interim solution until Warehouse
has built-in stats, but I'd be happy to polish it up a bit as time allows
if anyone finds it useful.

Side note for Donald: It appears that the dataset currently contains data
for 2016-01-22 to 2016-03-06 and 2016-05-22 to current. Is there any plan
or possibility of backfilling either the 2016-03-07 to 2016-05-21 gap, or
the older data?

-Jason Antman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160918/797e9fd2/attachment.html>

From donald at stufft.io  Sun Sep 18 16:51:56 2016
From: donald at stufft.io (Donald Stufft)
Date: Sun, 18 Sep 2016 16:51:56 -0400
Subject: [Distutils] Publicly Queryable Statistics
In-Reply-To: <CAFt4V4kKm3cD_xBz=rKVV9BuGq1hCPUQNUm-5bMMb8WkWoRZSQ@mail.gmail.com>
References: <CAFt4V4kKm3cD_xBz=rKVV9BuGq1hCPUQNUm-5bMMb8WkWoRZSQ@mail.gmail.com>
Message-ID: <F6E02471-ABED-42F4-985C-75D60FF28766@stufft.io>


> On Sep 18, 2016, at 7:42 AM, Jason Antman <jason at jasonantman.com> wrote:
> 
> Donald,
> 
> Thanks so much for all of your work on this.
> 
> I know it's an old thread, but I've built a tool that generates some statistics from this information. It queries data for a given list of projects, or for all of a specific user's projects, caches the data locally on disk, and generates both a HTML report with a bunch of graphs, as well as download badges. For my own use, I'm running it cron'ed from my desktop once a night, and uploading the reports and badges to a public Amazon S3 bucket.
> 
> The project is: https://pypi.org/project/pypi-download-stats/ <https://pypi.org/project/pypi-download-stats/>
> 
> Example output: http://jantman-personal-public.s3-website-us-east-1.amazonaws.com/pypi-stats/awslimitchecker/index.html <http://jantman-personal-public.s3-website-us-east-1.amazonaws.com/pypi-stats/awslimitchecker/index.html>
> 
> It's a bit rough around the edges, and currently doesn't have any unit tests - my hope is that this will be an interim solution until Warehouse has built-in stats, but I'd be happy to polish it up a bit as time allows if anyone finds it useful.

Awesome, this is exactly the kind of thing I was hoping to enable by making these all public :)

> 
> Side note for Donald: It appears that the dataset currently contains data for 2016-01-22 to 2016-03-06 and 2016-05-22 to current. Is there any plan or possibility of backfilling either the 2016-03-07 to 2016-05-21 gap, or the older data?

Yes there is. I started backfilling then got distracted and quit (hence the gap). Filling in the gap is easy, it?s just a matter of downloading the files and running a script. Getting even older data requires more effort to first munge the existing log files into the correct format, and then run them through the same script as above. When it?s all said and done we should be able to go back to like, Jan of 2014 I think.


?
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160918/e788e606/attachment.html>

From barry at python.org  Tue Sep 20 18:43:15 2016
From: barry at python.org (Barry Warsaw)
Date: Tue, 20 Sep 2016 18:43:15 -0400
Subject: [Distutils] When can we kill Python 2.6 support?
References: <40BEEAE9-F798-4972-889A-CDB22993A53F@stufft.io>
Message-ID: <20160920184315.17f9db3a@anarchist>

On Sep 02, 2016, at 05:05 PM, Donald Stufft wrote:

>Do we think that a ~3% usage of Python 2.6 and being end-of-life'd for ~3
>years is enough to start deprecating and dropping 2.6?

Yes!

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160920/c7240562/attachment.sig>

From phil at riverbankcomputing.com  Tue Sep 27 13:08:19 2016
From: phil at riverbankcomputing.com (Phil Thompson)
Date: Tue, 27 Sep 2016 18:08:19 +0100
Subject: [Distutils] Compatibility Tags and the Stable API
In-Reply-To: <D6DF2BF8-1D58-4CDC-AB38-B4F721D28CAD@riverbankcomputing.com>
References: <7BB98834-DB8F-4B87-A165-CD4E347F075C@riverbankcomputing.com>
 <CAG8k2+5Eod4TD+xZf2-meAiVzTLyuiij0aPYu2F6ibfLG8UpKw@mail.gmail.com>
 <D6DF2BF8-1D58-4CDC-AB38-B4F721D28CAD@riverbankcomputing.com>
Message-ID: <81901F01-5C09-4355-A326-74E710BA03D0@riverbankcomputing.com>

Apologies for pestering...

Is there any more information about this?

I have cp34-abi3 wheels ready for release (ie. Python 3.4 or later) but they can't be installed with pip3.5. I could release them as cp35-abi3 but that will become a pain with the release of Python v3.6.

Thanks,
Phil


> On 9 Sep 2016, at 12:28 pm, Phil Thompson <phil at riverbankcomputing.com> wrote:
> 
> On 9 Sep 2016, at 12:00 pm, Daniel Holth <dholth at gmail.com> wrote:
>> 
>> Correct. I have a pull request on wheel that does just that, and the next version of pip will be ready for it.
> 
> Is there a release date for that version of pip? There's no point in me making releases that can't be installed.
> 
> Phil
> 
>> On Fri, Sep 9, 2016, 04:14 Phil Thompson <phil at riverbankcomputing.com> wrote:
>> I'm not sure how to interpret PEP 425 when it comes to packages containing C extension modules that use the limited Python API (PEP 384).
>> 
>> The Python v3.4 version of the limited API is used so should the compatibility tag be...
>> 
>> cp34-abi3-*
>> 
>> That would mean that the 'cp34' Python tag would have to be interpreted as a minimum (rather than exact) requirement.
>> 
>> Grateful for any clarification.
>> 
>> Phil


From dholth at gmail.com  Tue Sep 27 14:01:31 2016
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 27 Sep 2016 18:01:31 +0000
Subject: [Distutils] Compatibility Tags and the Stable API
In-Reply-To: <81901F01-5C09-4355-A326-74E710BA03D0@riverbankcomputing.com>
References: <7BB98834-DB8F-4B87-A165-CD4E347F075C@riverbankcomputing.com>
 <CAG8k2+5Eod4TD+xZf2-meAiVzTLyuiij0aPYu2F6ibfLG8UpKw@mail.gmail.com>
 <D6DF2BF8-1D58-4CDC-AB38-B4F721D28CAD@riverbankcomputing.com>
 <81901F01-5C09-4355-A326-74E710BA03D0@riverbankcomputing.com>
Message-ID: <CAG8k2+4mQm8eM69m2P9OE-GRhpREGtT86BdRV229nWGMMzq=Gw@mail.gmail.com>

You could manually change the tag to cp34.cp35.cp36-abi3 . The current
version pip running on those 3 versions of Python should recognize that tag.

On Tue, Sep 27, 2016 at 1:16 PM Phil Thompson <phil at riverbankcomputing.com>
wrote:

> Apologies for pestering...
>
> Is there any more information about this?
>
> I have cp34-abi3 wheels ready for release (ie. Python 3.4 or later) but
> they can't be installed with pip3.5. I could release them as cp35-abi3 but
> that will become a pain with the release of Python v3.6.
>
> Thanks,
> Phil
>
>
> > On 9 Sep 2016, at 12:28 pm, Phil Thompson <phil at riverbankcomputing.com>
> wrote:
> >
> > On 9 Sep 2016, at 12:00 pm, Daniel Holth <dholth at gmail.com> wrote:
> >>
> >> Correct. I have a pull request on wheel that does just that, and the
> next version of pip will be ready for it.
> >
> > Is there a release date for that version of pip? There's no point in me
> making releases that can't be installed.
> >
> > Phil
> >
> >> On Fri, Sep 9, 2016, 04:14 Phil Thompson <phil at riverbankcomputing.com>
> wrote:
> >> I'm not sure how to interpret PEP 425 when it comes to packages
> containing C extension modules that use the limited Python API (PEP 384).
> >>
> >> The Python v3.4 version of the limited API is used so should the
> compatibility tag be...
> >>
> >> cp34-abi3-*
> >>
> >> That would mean that the 'cp34' Python tag would have to be interpreted
> as a minimum (rather than exact) requirement.
> >>
> >> Grateful for any clarification.
> >>
> >> Phil
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160927/325ee80d/attachment.html>

From vinay_sajip at yahoo.co.uk  Fri Sep 30 05:51:36 2016
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 30 Sep 2016 09:51:36 +0000 (UTC)
Subject: [Distutils] ANN: distlib 0.2.4 released on PyPI
References: <1592008463.2275511.1475229096733.ref@mail.yahoo.com>
Message-ID: <1592008463.2275511.1475229096733@mail.yahoo.com>

I've just released version 0.2.4 of distlib on PyPI [1]. For newcomers,
distlib is a library of packaging functionality which is intended to be
usable as the basis for third-party packaging tools.

The main changes in this release are as follows:

* Updated to not fail during import if SSL is not available.
* Changed project name comparisons to follow PEP 503.
* Changed manifest and resources logic to work correctly under (upcoming) Python 3.6.
* Updated Windows launchers with fixes to bugs related to argument-passing in shebang lines.

A more detailed change log is available at [2].

Please try it out, and if you find any problems or have any suggestions for
improvements, please give some feedback using the issue tracker! [3]

Regards,

Vinay Sajip

[1] https://pypi.python.org/pypi/distlib/0.2.4
[2] https://goo.gl/M3kQzR
[3] https://bitbucket.org/pypa/distlib/issues/new