From qwcode at gmail.com  Tue Oct  1 00:06:34 2013
From: qwcode at gmail.com (Marcus Smith)
Date: Mon, 30 Sep 2013 15:06:34 -0700
Subject: [Distutils] unparseable sdist filenames
In-Reply-To: <CALeMXf5r+1hO=2uN8q6mkf-u1Do5nKGnPRxB+QMkHJwJWm7gNA@mail.gmail.com>
References: <CAPYWazp26RhA4626MVV0KNDVBLXfn8u7V0mFWTxjhcqG=i80hw@mail.gmail.com>
 <CALeMXf6-K35-zHQAx1mrKiHQyMjPOsC7Oor9HSQm4zotaUeFPg@mail.gmail.com>
 <CAPYWazqbDnpjZ0WpUoPDH=42ne=y552ieJGxwKK3hGA8-wz8mw@mail.gmail.com>
 <CALeMXf5r+1hO=2uN8q6mkf-u1Do5nKGnPRxB+QMkHJwJWm7gNA@mail.gmail.com>
Message-ID: <CAPYWazrLEKGGr33z=73zaW25Gjk4-no3T6XWWpiVzD6x8-7eKg@mail.gmail.com>

>
>
> > how will context decide between the version being "dev" or "xdist-dev"?
>
> By whether you asked to install "pytest-xdist" or "pytest", and by
> whether "dev" or "xdist-dev" match your version requirements.  In
> practice this tends to only be a problem if you are:
>
> 1. Installing the two packages during the same run, and
> 2. Aren't using version specifiers.
>

#2 is a big deal.  the pip issue (#1192) that motivated this was a #2.
local find-links with non-versioned requirements.
so, suppose you have "pytest-xdist-dev.tar.gz" in a find-links location.
whether you're trying to install "pytest" or "pytest-xdist" doesn't help
the installer determine how to parse that archive.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130930/a77d9d21/attachment-0001.html>

From pje at telecommunity.com  Tue Oct  1 02:15:00 2013
From: pje at telecommunity.com (PJ Eby)
Date: Mon, 30 Sep 2013 20:15:00 -0400
Subject: [Distutils] unparseable sdist filenames
In-Reply-To: <CAPYWazrLEKGGr33z=73zaW25Gjk4-no3T6XWWpiVzD6x8-7eKg@mail.gmail.com>
References: <CAPYWazp26RhA4626MVV0KNDVBLXfn8u7V0mFWTxjhcqG=i80hw@mail.gmail.com>
 <CALeMXf6-K35-zHQAx1mrKiHQyMjPOsC7Oor9HSQm4zotaUeFPg@mail.gmail.com>
 <CAPYWazqbDnpjZ0WpUoPDH=42ne=y552ieJGxwKK3hGA8-wz8mw@mail.gmail.com>
 <CALeMXf5r+1hO=2uN8q6mkf-u1Do5nKGnPRxB+QMkHJwJWm7gNA@mail.gmail.com>
 <CAPYWazrLEKGGr33z=73zaW25Gjk4-no3T6XWWpiVzD6x8-7eKg@mail.gmail.com>
Message-ID: <CALeMXf4==ecLSHO9y2SVxRYTLko2LOo9wf0-q-GVYnDacgLxxQ@mail.gmail.com>

On Mon, Sep 30, 2013 at 6:06 PM, Marcus Smith <qwcode at gmail.com> wrote:
>>
>> > how will context decide between the version being "dev" or "xdist-dev"?
>>
>> By whether you asked to install "pytest-xdist" or "pytest", and by
>> whether "dev" or "xdist-dev" match your version requirements.  In
>> practice this tends to only be a problem if you are:
>>
>> 1. Installing the two packages during the same run, and
>> 2. Aren't using version specifiers.
>
>
> #2 is a big deal.  the pip issue (#1192) that motivated this was a #2. local
> find-links with non-versioned requirements.

I should probably add a #3 to that list: it's also rarely a problem
because usually there's at least one *numbered* version of the desired
package available, which will always prioritize newer than an
arbitrary alphabetic version like "xdist", even if you haven't
requested a specific version.

IOW, you need to not only not request a version and come across the
other package in the same run, but you *also* need for there not to be
any valid versions of your intended target present.


> so, suppose you have "pytest-xdist-dev.tar.gz" in a find-links location.
> whether you're trying to install "pytest" or "pytest-xdist" doesn't help the
> installer determine how to parse that archive.

In easy_install's case, it will take the one with the highest version
number, which means it'll try to install pytest-xdist-dev.tar.gz, on
the theory that 'xdist-dev' is a newer version than 'dev'.  This is
kind of silly, but it was the Simplest Possible Thing That Could Work,
and I figured I'd change it when there was enough feedback/field
experience to decide what to change it to.  ;-)

Probably the "right" thing to do would be to warn or decline on
ambiguous filenames in the future, since it is quite possible to
detect if a filename has more than one possible interpretation.
Another possibility would be to treat them analagously to
stable/unstable versions, prioritize them below regular matches, etc.

From p.f.moore at gmail.com  Tue Oct  1 18:59:05 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 1 Oct 2013 17:59:05 +0100
Subject: [Distutils] pkg_resources get_distribution - zipfile support?
Message-ID: <CACac1F-8s4d3BQTkD4WnQ6hMuNFci=_nUfoki6ZN0RBOJ78Yng@mail.gmail.com>

I'm having trouble using pkg_resources.get_distribution, and I'm not
sure if it's because what I'm trying to do is unsupported (likely) or
if I'm doing something wrong (also possible :-()

I have a wheel file - essentially a zip file containing a
distribution, plus a dist-info directory. If I put that zipfile on
sys.path, I can import from it with no issues. However, if I try to do
pkg_resources.get_distribution to find the distribution contained in
that wheel/zipfile I get nothing back.

Is this a bug, or an unsupported feature, or should I do something
particular to make it work?

Thanks,
Paul

From dholth at gmail.com  Tue Oct  1 19:51:44 2013
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 1 Oct 2013 13:51:44 -0400
Subject: [Distutils] pkg_resources get_distribution - zipfile support?
In-Reply-To: <CACac1F-8s4d3BQTkD4WnQ6hMuNFci=_nUfoki6ZN0RBOJ78Yng@mail.gmail.com>
References: <CACac1F-8s4d3BQTkD4WnQ6hMuNFci=_nUfoki6ZN0RBOJ78Yng@mail.gmail.com>
Message-ID: <CAG8k2+4kiPA6=f0s4tGQYAEoZx4w=ATTEYKwpD7=hq4osigoKg@mail.gmail.com>

On Tue, Oct 1, 2013 at 12:59 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> I'm having trouble using pkg_resources.get_distribution, and I'm not
> sure if it's because what I'm trying to do is unsupported (likely) or
> if I'm doing something wrong (also possible :-()
>
> I have a wheel file - essentially a zip file containing a
> distribution, plus a dist-info directory. If I put that zipfile on
> sys.path, I can import from it with no issues. However, if I try to do
> pkg_resources.get_distribution to find the distribution contained in
> that wheel/zipfile I get nothing back.
>
> Is this a bug, or an unsupported feature, or should I do something
> particular to make it work?
>
> Thanks,
> Paul
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

pkg_resources only finds distributions inside .egg and inside sys.path
entries that are filesystem directories. You might be able to manually
add a new Distribution instance to working_set or start looking for a
place to add the feature here:

https://bitbucket.org/pypa/setuptools/src/27df3c725f9696ba730456f3f444cc2fb5271d4b/pkg_resources.py?at=default#cl-2560

_distributionImpl = {
    '.egg': Distribution,
    '.egg-info': Distribution,
    '.dist-info': DistInfoDistribution,
    }

https://bitbucket.org/pypa/setuptools/src/27df3c725f9696ba730456f3f444cc2fb5271d4b/pkg_resources.py?at=default#cl-2219

(Distribution.from_location())

From p.f.moore at gmail.com  Tue Oct  1 20:11:25 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 1 Oct 2013 19:11:25 +0100
Subject: [Distutils] pkg_resources get_distribution - zipfile support?
In-Reply-To: <CAG8k2+4kiPA6=f0s4tGQYAEoZx4w=ATTEYKwpD7=hq4osigoKg@mail.gmail.com>
References: <CACac1F-8s4d3BQTkD4WnQ6hMuNFci=_nUfoki6ZN0RBOJ78Yng@mail.gmail.com>
 <CAG8k2+4kiPA6=f0s4tGQYAEoZx4w=ATTEYKwpD7=hq4osigoKg@mail.gmail.com>
Message-ID: <CACac1F_OZ3n2sSRGXB6HJ1AOoAgynBPkWbTObPz6Hh=K3_rq+A@mail.gmail.com>

On 1 October 2013 18:51, Daniel Holth <dholth at gmail.com> wrote:
> pkg_resources only finds distributions inside .egg and inside sys.path
> entries that are filesystem directories. You might be able to manually
> add a new Distribution instance to working_set or start looking for a
> place to add the feature here:

OK, I thought that might be the case.

Context - you can't run pip/setuptools from uninstalled wheels to do
pip.main(['install', '--use-wheel', '<wheelfile>']) because
--use-wheel checks for a suitable version of setuptools using
find_distribution. So the check fails. This is for virtualenv bundling
wheels rather than sdists for faster creation.

Looks like I'll have to work out a patch for pip to allow disabling
that check somehow, or modifying it to use setuptools.__version__
rather than find_distribution.

Paul

From pje at telecommunity.com  Tue Oct  1 22:32:26 2013
From: pje at telecommunity.com (PJ Eby)
Date: Tue, 1 Oct 2013 16:32:26 -0400
Subject: [Distutils] pkg_resources get_distribution - zipfile support?
In-Reply-To: <CAG8k2+4kiPA6=f0s4tGQYAEoZx4w=ATTEYKwpD7=hq4osigoKg@mail.gmail.com>
References: <CACac1F-8s4d3BQTkD4WnQ6hMuNFci=_nUfoki6ZN0RBOJ78Yng@mail.gmail.com>
 <CAG8k2+4kiPA6=f0s4tGQYAEoZx4w=ATTEYKwpD7=hq4osigoKg@mail.gmail.com>
Message-ID: <CALeMXf5h6LPp9DgNLnBAG9G5Nujaj3Za4owWc=V1exFNKxKATg@mail.gmail.com>

On Tue, Oct 1, 2013 at 1:51 PM, Daniel Holth <dholth at gmail.com> wrote:
> pkg_resources only finds distributions inside .egg and inside sys.path
> entries that are filesystem directories.

Actually it looks in zipfiles (or subdirectory paths thereof) or
filesystem directories, and can spot zipfile subdirectories named
'.egg' inside zipfiles (or subdirectories thereof).

It will also look in any other sort of sys.path entry, if you register
a handler for the importer type.

> You might be able to manually
> add a new Distribution instance to working_set or start looking for a
> place to add the feature here:
>
> https://bitbucket.org/pypa/setuptools/src/27df3c725f9696ba730456f3f444cc2fb5271d4b/pkg_resources.py?at=default#cl-2560
>
> _distributionImpl = {
>     '.egg': Distribution,
>     '.egg-info': Distribution,
>     '.dist-info': DistInfoDistribution,
>     }
>
> https://bitbucket.org/pypa/setuptools/src/27df3c725f9696ba730456f3f444cc2fb5271d4b/pkg_resources.py?at=default#cl-2219
>
> (Distribution.from_location())

Actually, you would add the feature *here*:

https://bitbucket.org/pypa/setuptools/src/27df3c725f9696ba730456f3f444cc2fb5271d4b/pkg_resources.py?at=default#cl-1810

That is, in the find_in_zip() function, which is used to identify
distributions contained in zip files (or subdirectories thereof) on
sys.path.

(This feature needs to get added at some point anyway, for
pkg_resources to support mountable wheels.)

From p.f.moore at gmail.com  Tue Oct  1 23:11:44 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 1 Oct 2013 22:11:44 +0100
Subject: [Distutils] pkg_resources get_distribution - zipfile support?
In-Reply-To: <CALeMXf5h6LPp9DgNLnBAG9G5Nujaj3Za4owWc=V1exFNKxKATg@mail.gmail.com>
References: <CACac1F-8s4d3BQTkD4WnQ6hMuNFci=_nUfoki6ZN0RBOJ78Yng@mail.gmail.com>
 <CAG8k2+4kiPA6=f0s4tGQYAEoZx4w=ATTEYKwpD7=hq4osigoKg@mail.gmail.com>
 <CALeMXf5h6LPp9DgNLnBAG9G5Nujaj3Za4owWc=V1exFNKxKATg@mail.gmail.com>
Message-ID: <CACac1F_KWV6S5q2xjQWx1T326gEL2b18vAe9rKXSM0Wf3FgQeQ@mail.gmail.com>

On 1 October 2013 21:32, PJ Eby <pje at telecommunity.com> wrote:
> On Tue, Oct 1, 2013 at 1:51 PM, Daniel Holth <dholth at gmail.com> wrote:
>> pkg_resources only finds distributions inside .egg and inside sys.path
>> entries that are filesystem directories.
>
> Actually it looks in zipfiles (or subdirectory paths thereof) or
> filesystem directories, and can spot zipfile subdirectories named
> '.egg' inside zipfiles (or subdirectories thereof).

But not dist-info? I thought setuptools supported dist-info formats
these days. Is this somewhere that got missed? Or is it more subtle
than that?
Paul

From ncoghlan at gmail.com  Wed Oct  2 01:10:44 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 2 Oct 2013 09:10:44 +1000
Subject: [Distutils] pkg_resources get_distribution - zipfile support?
In-Reply-To: <CACac1F_KWV6S5q2xjQWx1T326gEL2b18vAe9rKXSM0Wf3FgQeQ@mail.gmail.com>
References: <CACac1F-8s4d3BQTkD4WnQ6hMuNFci=_nUfoki6ZN0RBOJ78Yng@mail.gmail.com>
 <CAG8k2+4kiPA6=f0s4tGQYAEoZx4w=ATTEYKwpD7=hq4osigoKg@mail.gmail.com>
 <CALeMXf5h6LPp9DgNLnBAG9G5Nujaj3Za4owWc=V1exFNKxKATg@mail.gmail.com>
 <CACac1F_KWV6S5q2xjQWx1T326gEL2b18vAe9rKXSM0Wf3FgQeQ@mail.gmail.com>
Message-ID: <CADiSq7eM+POoV4EpbJx1BTsw1v7iwesMrKndm32R-RphgHSZ_Q@mail.gmail.com>

On 2 Oct 2013 07:12, "Paul Moore" <p.f.moore at gmail.com> wrote:
>
> On 1 October 2013 21:32, PJ Eby <pje at telecommunity.com> wrote:
> > On Tue, Oct 1, 2013 at 1:51 PM, Daniel Holth <dholth at gmail.com> wrote:
> >> pkg_resources only finds distributions inside .egg and inside sys.path
> >> entries that are filesystem directories.
> >
> > Actually it looks in zipfiles (or subdirectory paths thereof) or
> > filesystem directories, and can spot zipfile subdirectories named
> > '.egg' inside zipfiles (or subdirectories thereof).
>
> But not dist-info? I thought setuptools supported dist-info formats
> these days. Is this somewhere that got missed? Or is it more subtle
> than that?

I believe it's not recognising "*.whl" as interesting, so it isn't even
looking inside the nested wheel to check for dist-info.

Cheers,
Nick.

> Paul
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131002/42e587b1/attachment-0001.html>

From pje at telecommunity.com  Wed Oct  2 01:52:20 2013
From: pje at telecommunity.com (PJ Eby)
Date: Tue, 1 Oct 2013 19:52:20 -0400
Subject: [Distutils] pkg_resources get_distribution - zipfile support?
In-Reply-To: <CACac1F_KWV6S5q2xjQWx1T326gEL2b18vAe9rKXSM0Wf3FgQeQ@mail.gmail.com>
References: <CACac1F-8s4d3BQTkD4WnQ6hMuNFci=_nUfoki6ZN0RBOJ78Yng@mail.gmail.com>
 <CAG8k2+4kiPA6=f0s4tGQYAEoZx4w=ATTEYKwpD7=hq4osigoKg@mail.gmail.com>
 <CALeMXf5h6LPp9DgNLnBAG9G5Nujaj3Za4owWc=V1exFNKxKATg@mail.gmail.com>
 <CACac1F_KWV6S5q2xjQWx1T326gEL2b18vAe9rKXSM0Wf3FgQeQ@mail.gmail.com>
Message-ID: <CALeMXf7zmxGB3AAzJr5K_XySFn3eqKnTUSi_vL5L-FPOeU4q=A@mail.gmail.com>

On Tue, Oct 1, 2013 at 5:11 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 1 October 2013 21:32, PJ Eby <pje at telecommunity.com> wrote:
>> On Tue, Oct 1, 2013 at 1:51 PM, Daniel Holth <dholth at gmail.com> wrote:
>>> pkg_resources only finds distributions inside .egg and inside sys.path
>>> entries that are filesystem directories.
>>
>> Actually it looks in zipfiles (or subdirectory paths thereof) or
>> filesystem directories, and can spot zipfile subdirectories named
>> '.egg' inside zipfiles (or subdirectories thereof).
>
> But not dist-info? I thought setuptools supported dist-info formats
> these days. Is this somewhere that got missed? Or is it more subtle
> than that?

The person or persons who implemented dist-info either didn't know
where to put in the support, or didn't consider it a priority to also
support zipped dist-info.

A cursory glance at the DistinfoDistribution class, however, suggests
that it should work fine as long as it's paired with an appropriate
metadata provider.  Something like:

    class WheelMetadata(EggMetadata):
        def _setup_prefix(self):
            # like EggProvider._setup_prefix, except w/".whl" and
            # ".dist-info" instead of ".egg" and "EGG-INFO"
            # ... or refactor EggProvider to parameterize these as
            # class attrs and override here

and then make find_in_zip look more like:

def find_in_zip(importer, path_item, only=False):
    for (meta_factory, fn, dist_factory) in [
        (EggMetadata, 'PKG-INFO', Distribution),
        (WheelMetadata, 'METADATA', DistInfoDistribution),
    ]:
        metadata = meta_factory(importer)
        if metadata.has_metadata(fn):
            yield dist_factory.from_filename(path_item, metadata=metadata)
    if only:
        return  # don't yield nested distros
    for subitem in metadata.resource_listdir('/'):
        if subitem.endswith('.egg') or subitem.endswith('.whl'):
            subpath = os.path.join(path_item, subitem)
            for dist in find_in_zip(zipimport.zipimporter(subpath), subpath):
                yield dist

So it's not a huge deal to implement, just a bit of tedium.  Under the
hood, there's little difference between wheels and eggs besides naming
conventions.

From maciej.blizinski at gmail.com  Tue Oct  1 18:12:01 2013
From: maciej.blizinski at gmail.com (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=)
Date: Tue, 1 Oct 2013 17:12:01 +0100
Subject: [Distutils] File names with spaces
Message-ID: <CALtRa-5s9UD5qTr7nSNqr55fi83VeSi-Dtt14_vsQ+pL8f38Mw@mail.gmail.com>

Hello setuptools developers,

I'm attempting to package the newest setuptools package on Solaris 9
and 10. One of the limitations of the Solaris package manager (the old
one, pkgadd/pkgrm), is that it is unable to handle file names with
spaces. Would you mind renaming "script template.py" to something like
"script_template.py"?

Regards,
Maciej

From maciej at opencsw.org  Tue Oct  1 19:04:14 2013
From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=)
Date: Tue, 1 Oct 2013 18:04:14 +0100
Subject: [Distutils] File names with spaces
Message-ID: <CALtRa-6CD1Q5j92mGYW=PT6wbaXFAseHdCaYQGdc0Z9GUQ2_Dw@mail.gmail.com>

Hello setuptools developers,

I'm attempting to package the newest setuptools version (1.1.6)
on Solaris 9 and 10. One of the limitations of the Solaris package manager
(the old one, pkgadd/pkgrm), is that it is unable to handle file names with
spaces. Would you mind renaming "script template.py" to something like
"script_template.py"?

Maciej

From pje at telecommunity.com  Wed Oct  2 04:32:37 2013
From: pje at telecommunity.com (PJ Eby)
Date: Tue, 1 Oct 2013 22:32:37 -0400
Subject: [Distutils] pkg_resources get_distribution - zipfile support?
In-Reply-To: <CADiSq7eM+POoV4EpbJx1BTsw1v7iwesMrKndm32R-RphgHSZ_Q@mail.gmail.com>
References: <CACac1F-8s4d3BQTkD4WnQ6hMuNFci=_nUfoki6ZN0RBOJ78Yng@mail.gmail.com>
 <CAG8k2+4kiPA6=f0s4tGQYAEoZx4w=ATTEYKwpD7=hq4osigoKg@mail.gmail.com>
 <CALeMXf5h6LPp9DgNLnBAG9G5Nujaj3Za4owWc=V1exFNKxKATg@mail.gmail.com>
 <CACac1F_KWV6S5q2xjQWx1T326gEL2b18vAe9rKXSM0Wf3FgQeQ@mail.gmail.com>
 <CADiSq7eM+POoV4EpbJx1BTsw1v7iwesMrKndm32R-RphgHSZ_Q@mail.gmail.com>
Message-ID: <CALeMXf7j69QDC2sZQzw1+7UoL_a=a4oArPALrXrOpsK347c2uw@mail.gmail.com>

On Tue, Oct 1, 2013 at 7:10 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
> On 2 Oct 2013 07:12, "Paul Moore" <p.f.moore at gmail.com> wrote:
>>
>> On 1 October 2013 21:32, PJ Eby <pje at telecommunity.com> wrote:
>> > On Tue, Oct 1, 2013 at 1:51 PM, Daniel Holth <dholth at gmail.com> wrote:
>> >> pkg_resources only finds distributions inside .egg and inside sys.path
>> >> entries that are filesystem directories.
>> >
>> > Actually it looks in zipfiles (or subdirectory paths thereof) or
>> > filesystem directories, and can spot zipfile subdirectories named
>> > '.egg' inside zipfiles (or subdirectories thereof).
>>
>> But not dist-info? I thought setuptools supported dist-info formats
>> these days. Is this somewhere that got missed? Or is it more subtle
>> than that?
>
> I believe it's not recognising "*.whl" as interesting, so it isn't even
> looking inside the nested wheel to check for dist-info.

That would only be relevant if the .whl weren't on sys.path.  Since
it's on sys.path, it's processed by importer, not by filename.  (It's
just that there's no .dist-info detection in the zipimporter handler.)
 The .whl extension is only relevant for discovery of nested .whl's
(wheels within wheels...  or within eggs or plain zipfiles...), or
.whl's in directories.

There isn't any good terminology for describing this at the moment,
which makes all this sound much more complicated than it actually is.
;-)

Making up some new terminology, suppose we have foo.egg, bar.egg-info,
baz.dist-info, and spam.whl in site-packages.  Then the bar and baz
distributions are "mounted" at the '.../site-packages' location
string, but the foo and spam distributions are merely "discoverable"
at that same location string.  (They are *mounted* at
'.../site-packages/foo.egg' and '.../site-packages/spam.whl',
respectively.)

That is, to be mounted at a given location string means "to be
importable if that location string is on sys.path", and to be
discoverable at a given location means "to be available for dynamic
dependency resolution (e.g. via require()) if that location string is
on sys.path".

Determining what is mounted or discoverable at a given sys.path
location is the job of the find_distributions() API.  If the 'only'
flag is true, it yields only mounted distributions at the given
location string.  If false (the default), it yields both mounted and
discoverable distributions.

Behind the scenes, this is implemented by finding a handler for the
importer that the PEP 302 import protocol would use to look for
modules at the given location string, and then delegating the
operation to that handler.  The handler then has to look at the
location string and figure out what distributions are mounted and/or
discoverable there.

To find mounted distributions, the directory handler (find_on_path())
checks whether the directory string itself ends in '.egg' (and could
theoretically do the same for .whl), and also looks for contained
.dist-info and .egg-info files or directories.  To find mountable
distributions, it checks for files or directories ending in '.egg'
(and could theoretically do the same for .whl).

The zipfile handler (find_in_zip()) doesn't actually bother checking
for an .egg extension; instead it checks for an EGG-INFO/PKG-INFO and
assumes it'll be able to figure things out from that.  And it checks
for nested .eggs if it's looking for discoverables.

So, what it's missing to support Paul's use case is a check for
.dist-info/METADATA, analagous to the EGG-INFO/PKG-INFO check.  It
should be relatively simple to add, if somebody wants to do that.  (It
can even be done from code outside pkg_resources, as there is a
'register_finder()' API that can be called to register a replacement
handler.)

In some ways, I'm finding the code structure irritating now, because
the one abstraction I *didn't* build into the original framework was a
concept that there would ever be competing formats to .egg and
.egg-info, so implementing .dist-info and .whl requires annoying
repetitions of code at the moment.  But it's probably not worth
refactoring to make this cleaner, because the odds that there will be
a *third* file format needing to be supported any time soon are
hopefully quite small.  ;-)

From donald at stufft.io  Thu Oct  3 04:57:47 2013
From: donald at stufft.io (Donald Stufft)
Date: Wed, 2 Oct 2013 22:57:47 -0400
Subject: [Distutils] Warehouse - Simple API
Message-ID: <EE9E9386-7737-41A0-85C6-B96FD9EE518C@stufft.io>

I've recently deployed the current Warehouse code. So far the
simple API has been implemented and it would be great if
people could point their installers to it and report back to me
if they run into any problems.

The current PyPI simple api is not documented at all so I've had
to reverse engineer it going from what I know, pip, etc. Ideally
if i've missed anything it will be found out with testing from y'all.

You can try the warehouse code by setting your installers to use
the url https://preview-pypi.python.org/simple/ . This url *will* be
removed eventually so do not depend on it. It is backed by the
same database as PyPI so all of the same packages should
exist and be installable. It is not powered by the CDN at the
moment.

I welcome any reports of problems either by response to this thread,
privately to me, or on the issue tracker at Github [1]

[1] https://github.com/dstufft/warehouse/issues

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131002/5b5d2392/attachment.sig>

From donald at stufft.io  Thu Oct  3 16:11:41 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 3 Oct 2013 10:11:41 -0400
Subject: [Distutils] Test Windows Installers
Message-ID: <4FA3D010-0F79-4808-B8C5-87A9EEC9E145@stufft.io>

Anyone who has a windows machine mind testing some installers for me?

These should install pip and setuptools:

https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py27.msi

https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py32.msi

https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py33.msi

Let me know?

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131003/2c27346a/attachment.sig>

From vladimir.v.diaz at gmail.com  Thu Oct  3 18:06:06 2013
From: vladimir.v.diaz at gmail.com (Vladimir Diaz)
Date: Thu, 3 Oct 2013 12:06:06 -0400
Subject: [Distutils] Test Windows Installers
In-Reply-To: <4FA3D010-0F79-4808-B8C5-87A9EEC9E145@stufft.io>
References: <4FA3D010-0F79-4808-B8C5-87A9EEC9E145@stufft.io>
Message-ID: <CAJYWieg19_bnAyXrm_rz-T2t-8x_vs1YdYptkQHfUCyos1576A@mail.gmail.com>

Successfully tested on Windows 7 (64-bit) running Python 2.7.5 (32-bit).  I
tested pip installing a distribution and importing setuptools.

This appears intentional but I'll just mention it just in case:
I have to specify the full path (i.e., C:\Python27\Scripts\pip) on the
command-line to execute pip.
"python -m pip" works though.


On Thu, Oct 3, 2013 at 10:11 AM, Donald Stufft <donald at stufft.io> wrote:

> Anyone who has a windows machine mind testing some installers for me?
>
> These should install pip and setuptools:
>
> https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py27.msi
>
> https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py32.msi
>
> https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py33.msi
>
> Let me know?
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
> DCFA
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131003/d28642b4/attachment.html>

From ncoghlan at gmail.com  Thu Oct  3 22:56:32 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 4 Oct 2013 06:56:32 +1000
Subject: [Distutils] Test Windows Installers
In-Reply-To: <CAJYWieg19_bnAyXrm_rz-T2t-8x_vs1YdYptkQHfUCyos1576A@mail.gmail.com>
References: <4FA3D010-0F79-4808-B8C5-87A9EEC9E145@stufft.io>
 <CAJYWieg19_bnAyXrm_rz-T2t-8x_vs1YdYptkQHfUCyos1576A@mail.gmail.com>
Message-ID: <CADiSq7eDmpTX8FC5wkPG+bWL+28EQq=6nv0zsZWv6wh+j02sMw@mail.gmail.com>

On 4 Oct 2013 02:13, "Vladimir Diaz" <vladimir.v.diaz at gmail.com> wrote:
>
> Successfully tested on Windows 7 (64-bit) running Python 2.7.5 (32-bit).
I tested pip installing a distribution and importing setuptools.
>
> This appears intentional but I'll just mention it just in case:
> I have to specify the full path (i.e., C:\Python27\Scripts\pip) on the
command-line to execute pip.
> "python -m pip" works though.

Yeah, the CPython installers currently don't add the scripts directory to
PATH. 3.4 is likely to change that.

Thanks for checking!

Cheers,
Nick.

>
>
> On Thu, Oct 3, 2013 at 10:11 AM, Donald Stufft <donald at stufft.io> wrote:
>>
>> Anyone who has a windows machine mind testing some installers for me?
>>
>> These should install pip and setuptools:
>>
>> https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py27.msi
>>
>> https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py32.msi
>>
>> https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py33.msi
>>
>> Let me know?
>>
>> -----------------
>> Donald Stufft
>> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
DCFA
>>
>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131004/38e102ac/attachment.html>

From Steve.Dower at microsoft.com  Thu Oct  3 23:59:34 2013
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Thu, 3 Oct 2013 21:59:34 +0000
Subject: [Distutils] Test Windows Installers
In-Reply-To: <4FA3D010-0F79-4808-B8C5-87A9EEC9E145@stufft.io>
References: <4FA3D010-0F79-4808-B8C5-87A9EEC9E145@stufft.io>
Message-ID: <142ec53f82354db48d8b0391dbab51ae@BY2PR03MB206.namprd03.prod.outlook.com>

I've done basic testing (install, pip install, pip list, pip uninstall, repair, uninstall) against:

WinXP SP3 x86
Vista SP2 x86/x64
Win7 SP1 x86/x64
Win8 RTM x86/x64
Win8.1 RTM x86/x64

With Python 2.7.5, 3.2.3 and 3.3.2, x86 and x64 for each. (It helps to have a bit of practice with large test matrices, convenient access to clean VMs for lots of operating systems, and a couple of internal automation tools :) )

One issue I noticed is that if you've previously installed pip from source and it's in easy-install.pth, the version from the MSI won't be used when it is installed. The same applies to setuptools. (I used setuptools 0.9.8 and pip 1.3.1 for testing this.)

In general there don't seem to be any other problems with it. I've noted some possible issues below, but since I don't know how much control you have over these, please don't take it personally if I'm pointing out things that cannot be changed.

- the default value for specifying a manual path ("D:\PythonX") should probably be "C:\PythonXY" where XY is the version the installer is targeting.

- 64-bit versions of Python installed for all users are not detected. Are you planning a 64-bit version of this installer? (Specifying the path manually worked fine.)

- the RemoveFile table is incorrect for Python 3.x - there are no references to __pycache__ folders, only to .pyc and .pyo files in the same folder as their .py counterpart. As a result, uninstall is not clean for Python 3.2 and 3.3. In particular, Python 3.3 will show this (instead of "No module named pip"):

C:\ >C:\Python33\python.exe -m pip
Traceback (most recent call last):
  File "C:\Python33\lib\runpy.py", line 140, in _run_module_as_main
    mod_name, loader, code, fname = _get_module_details(mod_name)
  File "C:\Python33\lib\runpy.py", line 105, in _get_module_details
    if loader.is_package(mod_name):
AttributeError: 'NamespaceLoader' object has no attribute 'is_package'

- choosing between 'All Users' and 'Just For Me' in the pip installer didn't seem to affect the install location.

- uninstalling Python before pip worked fine. (No problem here, just letting you know that I tried it :) )
- selecting both install options (Python from registry/custom path) and specifying the same path worked fine

Caveats:
- all machines were clean OS installs + Python from python.org. I didn't try installing Python from other sources
- I only tested upgrading pip with the installer on Windows 8, but I'm confident it will behave the same on all other platforms

All up, it looks great, and it's going to make things much easier for Windows users. Thanks for doing this.

Cheers,
Steve

-----Original Message-----
From: Distutils-SIG [mailto:distutils-sig-bounces+steve.dower=microsoft.com at python.org] On Behalf Of Donald Stufft
Sent: Thursday, 3 October 2013 0712
To: DistUtils mailing list
Subject: [Distutils] Test Windows Installers

Anyone who has a windows machine mind testing some installers for me?

These should install pip and setuptools:

https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py27.msi

https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py32.msi

https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py33.msi

Let me know?

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


From donald at stufft.io  Fri Oct  4 00:10:55 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 3 Oct 2013 18:10:55 -0400
Subject: [Distutils] Test Windows Installers
In-Reply-To: <142ec53f82354db48d8b0391dbab51ae@BY2PR03MB206.namprd03.prod.outlook.com>
References: <4FA3D010-0F79-4808-B8C5-87A9EEC9E145@stufft.io>
 <142ec53f82354db48d8b0391dbab51ae@BY2PR03MB206.namprd03.prod.outlook.com>
Message-ID: <13B2F48E-B802-4635-AC9A-A7C41594B3E2@stufft.io>


On Oct 3, 2013, at 5:59 PM, Steve Dower <Steve.Dower at microsoft.com> wrote:

> I've done basic testing (install, pip install, pip list, pip uninstall, repair, uninstall) against:
> 
> WinXP SP3 x86
> Vista SP2 x86/x64
> Win7 SP1 x86/x64
> Win8 RTM x86/x64
> Win8.1 RTM x86/x64
> 
> With Python 2.7.5, 3.2.3 and 3.3.2, x86 and x64 for each. (It helps to have a bit of practice with large test matrices, convenient access to clean VMs for lots of operating systems, and a couple of internal automation tools :) )
> 
> One issue I noticed is that if you've previously installed pip from source and it's in easy-install.pth, the version from the MSI won't be used when it is installed. The same applies to setuptools. (I used setuptools 0.9.8 and pip 1.3.1 for testing this.)

What do you mean for this? That if you already have pip 1.3.1 installed and you install from the MSI when it's over the installed version is still 1.3.1?

> 
> In general there don't seem to be any other problems with it. I've noted some possible issues below, but since I don't know how much control you have over these, please don't take it personally if I'm pointing out things that cannot be changed.
> 
> - the default value for specifying a manual path ("D:\PythonX") should probably be "C:\PythonXY" where XY is the version the installer is targeting.

Hmm, I'll see what I can do about this, I'm reusing routines from distutils to handle this but I may have some level of control over that.

> 
> - 64-bit versions of Python installed for all users are not detected. Are you planning a 64-bit version of this installer? (Specifying the path manually worked fine.)

Yea that was pointed out to me, I'll probably make 64bit installers since I don't think a 32bit installer can detect the 64bit versions.

> 
> - the RemoveFile table is incorrect for Python 3.x - there are no references to __pycache__ folders, only to .pyc and .pyo files in the same folder as their .py counterpart. As a result, uninstall is not clean for Python 3.2 and 3.3. In particular, Python 3.3 will show this (instead of "No module named pip"):

This is going to be an issue with distutils again, I'll see if I can control it but it'd probably be a good idea to get this fixed in distutils too.

> 
> C:\ >C:\Python33\python.exe -m pip
> Traceback (most recent call last):
>  File "C:\Python33\lib\runpy.py", line 140, in _run_module_as_main
>    mod_name, loader, code, fname = _get_module_details(mod_name)
>  File "C:\Python33\lib\runpy.py", line 105, in _get_module_details
>    if loader.is_package(mod_name):
> AttributeError: 'NamespaceLoader' object has no attribute 'is_package'
> 
> - choosing between 'All Users' and 'Just For Me' in the pip installer didn't seem to affect the install location.

AFAIK distutils added that, no idea what it does or means. I'll see if I can remove it.

> 
> - uninstalling Python before pip worked fine. (No problem here, just letting you know that I tried it :) )
> - selecting both install options (Python from registry/custom path) and specifying the same path worked fine
> 
> Caveats:
> - all machines were clean OS installs + Python from python.org. I didn't try installing Python from other sources
> - I only tested upgrading pip with the installer on Windows 8, but I'm confident it will behave the same on all other platforms
> 
> All up, it looks great, and it's going to make things much easier for Windows users. Thanks for doing this.
> 
> Cheers,
> Steve
> 
> -----Original Message-----
> From: Distutils-SIG [mailto:distutils-sig-bounces+steve.dower=microsoft.com at python.org] On Behalf Of Donald Stufft
> Sent: Thursday, 3 October 2013 0712
> To: DistUtils mailing list
> Subject: [Distutils] Test Windows Installers
> 
> Anyone who has a windows machine mind testing some installers for me?
> 
> These should install pip and setuptools:
> 
> https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py27.msi
> 
> https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py32.msi
> 
> https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py33.msi
> 
> Let me know?
> 
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> 


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131003/8ba1028d/attachment-0001.sig>

From donald at stufft.io  Fri Oct  4 00:21:20 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 3 Oct 2013 18:21:20 -0400
Subject: [Distutils] Test Windows Installers
In-Reply-To: <b5b5ec040da44836883a32f4fce33e4e@BY2PR03MB206.namprd03.prod.outlook.com>
References: <4FA3D010-0F79-4808-B8C5-87A9EEC9E145@stufft.io>
 <142ec53f82354db48d8b0391dbab51ae@BY2PR03MB206.namprd03.prod.outlook.com>
 <13B2F48E-B802-4635-AC9A-A7C41594B3E2@stufft.io>
 <b5b5ec040da44836883a32f4fce33e4e@BY2PR03MB206.namprd03.prod.outlook.com>
Message-ID: <50178BDE-3057-4A46-A9FA-0E5DBCBCED9F@stufft.io>


On Oct 3, 2013, at 6:20 PM, Steve Dower <Steve.Dower at microsoft.com> wrote:

> Donald Stufft wrote:
>> On Oct 3, 2013, at 5:59 PM, Steve Dower <Steve.Dower at microsoft.com> wrote:
>> 
>>> I've done basic testing (install, pip install, pip list, pip uninstall,
>> repair, uninstall) against:
>>> 
>>> WinXP SP3 x86
>>> Vista SP2 x86/x64
>>> Win7 SP1 x86/x64
>>> Win8 RTM x86/x64
>>> Win8.1 RTM x86/x64
>>> 
>>> With Python 2.7.5, 3.2.3 and 3.3.2, x86 and x64 for each. (It helps to
>>> have a bit of practice with large test matrices, convenient access to
>>> clean VMs for lots of operating systems, and a couple of internal
>>> automation tools :) )
>>> 
>>> One issue I noticed is that if you've previously installed pip from
>>> source and it's in easy-install.pth, the version from the MSI won't be
>>> used when it is installed. The same applies to setuptools. (I used
>>> setuptools 0.9.8 and pip 1.3.1 for testing this.)
>> 
>> What do you mean for this? That if you already have pip 1.3.1 installed and you
>> install from the MSI when it's over the installed version is still 1.3.1?
> 
> Both versions are installed, but `python -m pip` is going to select 1.3.1. (I didn't actually try Scripts\pip.exe, but since that's using pkg_resources I assume it will get the right one.)
> 
> I did the initial installs with the package setup.py files which produced a .egg file for setuptools and a .egg folder for pip. I'm not actually sure whether there's anything you could do about this from an installer?

Ahhh, Ok that makes sense. Yea I'm not sure what I can do about that. I'll have to think it over and see if I can come up with anything.

> 
> 
> Cheers,
> Steve
> 
>>> 
>>> In general there don't seem to be any other problems with it. I've noted some
>> possible issues below, but since I don't know how much control you have over
>> these, please don't take it personally if I'm pointing out things that cannot be
>> changed.
>>> 
>>> - the default value for specifying a manual path ("D:\PythonX") should
>> probably be "C:\PythonXY" where XY is the version the installer is targeting.
>> 
>> Hmm, I'll see what I can do about this, I'm reusing routines from distutils to
>> handle this but I may have some level of control over that.
>> 
>>> 
>>> - 64-bit versions of Python installed for all users are not detected.
>>> Are you planning a 64-bit version of this installer? (Specifying the
>>> path manually worked fine.)
>> 
>> Yea that was pointed out to me, I'll probably make 64bit installers since I
>> don't think a 32bit installer can detect the 64bit versions.
>> 
>>> 
>>> - the RemoveFile table is incorrect for Python 3.x - there are no references
>> to __pycache__ folders, only to .pyc and .pyo files in the same folder as their
>> .py counterpart. As a result, uninstall is not clean for Python 3.2 and 3.3. In
>> particular, Python 3.3 will show this (instead of "No module named pip"):
>> 
>> This is going to be an issue with distutils again, I'll see if I can control it
>> but it'd probably be a good idea to get this fixed in distutils too.
>> 
>>> 
>>> C:\ >C:\Python33\python.exe -m pip
>>> Traceback (most recent call last):
>>> File "C:\Python33\lib\runpy.py", line 140, in _run_module_as_main
>>> mod_name, loader, code, fname = _get_module_details(mod_name) File
>>> "C:\Python33\lib\runpy.py", line 105, in _get_module_details
>>> if loader.is_package(mod_name):
>>> AttributeError: 'NamespaceLoader' object has no attribute 'is_package'
>>> 
>>> - choosing between 'All Users' and 'Just For Me' in the pip installer didn't
>> seem to affect the install location.
>> 
>> AFAIK distutils added that, no idea what it does or means. I'll see if I can
>> remove it.
>> 
>>> 
>>> - uninstalling Python before pip worked fine. (No problem here, just
>>> letting you know that I tried it :) )
>>> - selecting both install options (Python from registry/custom path)
>>> and specifying the same path worked fine
>>> 
>>> Caveats:
>>> - all machines were clean OS installs + Python from python.org. I
>>> didn't try installing Python from other sources
>>> - I only tested upgrading pip with the installer on Windows 8, but I'm
>>> confident it will behave the same on all other platforms
>>> 
>>> All up, it looks great, and it's going to make things much easier for Windows
>> users. Thanks for doing this.
>>> 
>>> Cheers,
>>> Steve
>>> 
>>> -----Original Message-----
>>> From: Distutils-SIG
>>> [mailto:distutils-sig-bounces+steve.dower=microsoft.com at python.org] On
>>> Behalf Of Donald Stufft
>>> Sent: Thursday, 3 October 2013 0712
>>> To: DistUtils mailing list
>>> Subject: [Distutils] Test Windows Installers
>>> 
>>> Anyone who has a windows machine mind testing some installers for me?
>>> 
>>> These should install pip and setuptools:
>>> 
>>> https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py27.
>>> msi
>>> 
>>> https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py32.
>>> msi
>>> 
>>> https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py33.
>>> msi
>>> 
>>> Let me know?
>>> 
>>> -----------------
>>> Donald Stufft
>>> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9
>>> 3372 DCFA
>>> 
>> 
>> 
>> -----------------
>> Donald Stufft
>> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>> 


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131003/d09472cf/attachment.sig>

From Steve.Dower at microsoft.com  Fri Oct  4 00:20:12 2013
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Thu, 3 Oct 2013 22:20:12 +0000
Subject: [Distutils] Test Windows Installers
In-Reply-To: <13B2F48E-B802-4635-AC9A-A7C41594B3E2@stufft.io>
References: <4FA3D010-0F79-4808-B8C5-87A9EEC9E145@stufft.io>
 <142ec53f82354db48d8b0391dbab51ae@BY2PR03MB206.namprd03.prod.outlook.com>
 <13B2F48E-B802-4635-AC9A-A7C41594B3E2@stufft.io>
Message-ID: <b5b5ec040da44836883a32f4fce33e4e@BY2PR03MB206.namprd03.prod.outlook.com>

Donald Stufft wrote:
> On Oct 3, 2013, at 5:59 PM, Steve Dower <Steve.Dower at microsoft.com> wrote:
> 
>> I've done basic testing (install, pip install, pip list, pip uninstall,
> repair, uninstall) against:
>>
>> WinXP SP3 x86
>> Vista SP2 x86/x64
>> Win7 SP1 x86/x64
>> Win8 RTM x86/x64
>> Win8.1 RTM x86/x64
>>
>> With Python 2.7.5, 3.2.3 and 3.3.2, x86 and x64 for each. (It helps to
>> have a bit of practice with large test matrices, convenient access to
>> clean VMs for lots of operating systems, and a couple of internal
>> automation tools :) )
>>
>> One issue I noticed is that if you've previously installed pip from
>> source and it's in easy-install.pth, the version from the MSI won't be
>> used when it is installed. The same applies to setuptools. (I used
>> setuptools 0.9.8 and pip 1.3.1 for testing this.)
> 
> What do you mean for this? That if you already have pip 1.3.1 installed and you
> install from the MSI when it's over the installed version is still 1.3.1?

Both versions are installed, but `python -m pip` is going to select 1.3.1. (I didn't actually try Scripts\pip.exe, but since that's using pkg_resources I assume it will get the right one.)

I did the initial installs with the package setup.py files which produced a .egg file for setuptools and a .egg folder for pip. I'm not actually sure whether there's anything you could do about this from an installer...


Cheers,
Steve

>>
>> In general there don't seem to be any other problems with it. I've noted some
> possible issues below, but since I don't know how much control you have over
> these, please don't take it personally if I'm pointing out things that cannot be
> changed.
>>
>> - the default value for specifying a manual path ("D:\PythonX") should
> probably be "C:\PythonXY" where XY is the version the installer is targeting.
> 
> Hmm, I'll see what I can do about this, I'm reusing routines from distutils to
> handle this but I may have some level of control over that.
>
>>
>> - 64-bit versions of Python installed for all users are not detected.
>> Are you planning a 64-bit version of this installer? (Specifying the
>> path manually worked fine.)
> 
> Yea that was pointed out to me, I'll probably make 64bit installers since I
> don't think a 32bit installer can detect the 64bit versions.
> 
>>
>> - the RemoveFile table is incorrect for Python 3.x - there are no references
> to __pycache__ folders, only to .pyc and .pyo files in the same folder as their
> .py counterpart. As a result, uninstall is not clean for Python 3.2 and 3.3. In
> particular, Python 3.3 will show this (instead of "No module named pip"):
> 
> This is going to be an issue with distutils again, I'll see if I can control it
> but it'd probably be a good idea to get this fixed in distutils too.
> 
>>
>> C:\ >C:\Python33\python.exe -m pip
>> Traceback (most recent call last):
>> File "C:\Python33\lib\runpy.py", line 140, in _run_module_as_main
>> mod_name, loader, code, fname = _get_module_details(mod_name) File
>> "C:\Python33\lib\runpy.py", line 105, in _get_module_details
>> if loader.is_package(mod_name):
>> AttributeError: 'NamespaceLoader' object has no attribute 'is_package'
>>
>> - choosing between 'All Users' and 'Just For Me' in the pip installer didn't
> seem to affect the install location.
> 
> AFAIK distutils added that, no idea what it does or means. I'll see if I can
> remove it.
> 
>>
>> - uninstalling Python before pip worked fine. (No problem here, just
>> letting you know that I tried it :) )
>> - selecting both install options (Python from registry/custom path)
>> and specifying the same path worked fine
>>
>> Caveats:
>> - all machines were clean OS installs + Python from python.org. I
>> didn't try installing Python from other sources
>> - I only tested upgrading pip with the installer on Windows 8, but I'm
>> confident it will behave the same on all other platforms
>>
>> All up, it looks great, and it's going to make things much easier for Windows
> users. Thanks for doing this.
>>
>> Cheers,
>> Steve
>>
>> -----Original Message-----
>> From: Distutils-SIG
>> [mailto:distutils-sig-bounces+steve.dower=microsoft.com at python.org] On
>> Behalf Of Donald Stufft
>> Sent: Thursday, 3 October 2013 0712
>> To: DistUtils mailing list
>> Subject: [Distutils] Test Windows Installers
>>
>> Anyone who has a windows machine mind testing some installers for me?
>>
>> These should install pip and setuptools:
>>
>> https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py27.
>> msi
>>
>> https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py32.
>> msi
>>
>> https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py33.
>> msi
>>
>> Let me know?
>>
>> -----------------
>> Donald Stufft
>> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9
>> 3372 DCFA
>>
> 
> 
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>

From donald at stufft.io  Fri Oct  4 02:12:26 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 3 Oct 2013 20:12:26 -0400
Subject: [Distutils] Test Windows Installers
In-Reply-To: <50178BDE-3057-4A46-A9FA-0E5DBCBCED9F@stufft.io>
References: <4FA3D010-0F79-4808-B8C5-87A9EEC9E145@stufft.io>
 <142ec53f82354db48d8b0391dbab51ae@BY2PR03MB206.namprd03.prod.outlook.com>
 <13B2F48E-B802-4635-AC9A-A7C41594B3E2@stufft.io>
 <b5b5ec040da44836883a32f4fce33e4e@BY2PR03MB206.namprd03.prod.outlook.com>
 <50178BDE-3057-4A46-A9FA-0E5DBCBCED9F@stufft.io>
Message-ID: <39992151-C646-4DCD-8460-F7ADB4D86C94@stufft.io>

Bleh, and of course I find an issue, the generated scripts have a hardcoded shebang, will need to figure out how to regenerate those during the install process I guess.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131003/25acf245/attachment.sig>

From mail at timgolden.me.uk  Thu Oct  3 16:20:35 2013
From: mail at timgolden.me.uk (Tim Golden)
Date: Thu, 03 Oct 2013 15:20:35 +0100
Subject: [Distutils] Test Windows Installers
In-Reply-To: <4FA3D010-0F79-4808-B8C5-87A9EEC9E145@stufft.io>
References: <4FA3D010-0F79-4808-B8C5-87A9EEC9E145@stufft.io>
Message-ID: <524D7D33.20302@timgolden.me.uk>

On 03/10/2013 15:11, Donald Stufft wrote:
> Anyone who has a windows machine mind testing some installers for me?
> 
> These should install pip and setuptools:
> 
> https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py27.msi
> 
> https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py32.msi
> 
> https://dl.dropboxusercontent.com/u/45265381/MSI/pip/1.4/pip-1.4-py33.msi

They all install and appear to have dropped the right files in the right
place. Haven't actually tried installing anything yet!

TJG


From mail at timgolden.me.uk  Fri Oct  4 18:06:20 2013
From: mail at timgolden.me.uk (Tim Golden)
Date: Fri, 04 Oct 2013 17:06:20 +0100
Subject: [Distutils] Test Windows Installers
In-Reply-To: <524D7D33.20302@timgolden.me.uk>
References: <4FA3D010-0F79-4808-B8C5-87A9EEC9E145@stufft.io>
 <524D7D33.20302@timgolden.me.uk>
Message-ID: <524EE77C.3050704@timgolden.me.uk>

On 03/10/2013 15:20, Tim Golden wrote:
> They all install and appear to have dropped the right files in the right
> place. Haven't actually tried installing anything yet!

Sorry... that obviously got caught in moderation; I sent it a day ago.
Fairly useless now given Steve's extensive testing & analysis!

TJG

From donald at stufft.io  Tue Oct  8 15:19:37 2013
From: donald at stufft.io (Donald Stufft)
Date: Tue, 8 Oct 2013 09:19:37 -0400
Subject: [Distutils] URL Structure of Packages URLs
Message-ID: <70E49064-C95C-4B81-ACE5-13F76688803B@stufft.io>

I was wondering if anyone was aware of anything that relies on
the current structure of package urls, namely:

    /packages/source|any|etc/D/Django-1.0.tar.gz?

I would like to change this but before I work up a concrete plan for
people to comment on and discuss I'm trying to figure out what, if
anything, depends on that current structure.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131008/dad0a95b/attachment.sig>

From marius at pov.lt  Tue Oct  8 16:05:54 2013
From: marius at pov.lt (Marius Gedminas)
Date: Tue, 8 Oct 2013 17:05:54 +0300
Subject: [Distutils] URL Structure of Packages URLs
In-Reply-To: <70E49064-C95C-4B81-ACE5-13F76688803B@stufft.io>
References: <70E49064-C95C-4B81-ACE5-13F76688803B@stufft.io>
Message-ID: <20131008140553.GA20713@fridge.pov.lt>

On Tue, Oct 08, 2013 at 09:19:37AM -0400, Donald Stufft wrote:
> I was wondering if anyone was aware of anything that relies on
> the current structure of package urls, namely:
> 
>     /packages/source|any|etc/D/Django-1.0.tar.gz?
> 
> I would like to change this but before I work up a concrete plan for
> people to comment on and discuss I'm trying to figure out what, if
> anything, depends on that current structure.

I've told Launchpad that it can look for new releases at URLs like
http://pypi.python.org/packages/source/g/gtimelog/gtimelog-*.tar.gz
(for a few projects, not just GTimeLog).

I'm not sure how Launchpad crawls those pages looking for releases; all
the documentation I could find in 5 minutes was this blog post:
http://blog.launchpad.net/cool-new-stuff/automatically-import-files-to-launchpad-using-product-release-finder


Some Debian packages have debian/watch files using similar URL patterns
to watch for new upstream releases showing up on PyPI.  This mechanism
is documented at https://wiki.debian.org/debian/watch/.  Here's
python-requests:

    $ apt-get source python-requests
    $ cat requests-1.1.0/debian/watch
    version=3
    http://pypi.python.org/packages/source/r/requests/requests-(.*)\.tar\.gz


HTH,
Marius Gedminas
-- 
Some people have told me they don't think a fat penguin really embodies the
grace of Linux, which just tells me they have never seen a angry penguin
charging at them in excess of 100mph. They'd be a lot more careful about what
they say if they had.
                -- Linus Torvalds, announcing Linux v2.0
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131008/a9f7cfe9/attachment.sig>

From brian at vanguardistas.net  Tue Oct  8 16:31:40 2013
From: brian at vanguardistas.net (Brian Sutherland)
Date: Tue, 8 Oct 2013 16:31:40 +0200
Subject: [Distutils] URL Structure of Packages URLs
In-Reply-To: <20131008140553.GA20713@fridge.pov.lt>
References: <70E49064-C95C-4B81-ACE5-13F76688803B@stufft.io>
 <20131008140553.GA20713@fridge.pov.lt>
Message-ID: <20131008143140.GA36092@Brians-MacBook-Air.local>

On Tue, Oct 08, 2013 at 05:05:54PM +0300, Marius Gedminas wrote:
> On Tue, Oct 08, 2013 at 09:19:37AM -0400, Donald Stufft wrote:
> > I was wondering if anyone was aware of anything that relies on
> > the current structure of package urls, namely:
> > 
> >     /packages/source|any|etc/D/Django-1.0.tar.gz?
> > 
> > I would like to change this but before I work up a concrete plan for
> > people to comment on and discuss I'm trying to figure out what, if
> > anything, depends on that current structure.
> 
> Some Debian packages have debian/watch files using similar URL patterns

Perhaps "Some" is an understatement ;)

    http://qa.debian.org/developer.php?login=python-modules-team at lists.alioth.debian.org

lists over 500 packages, almost all of which have watch files (last 2
columns). There are probably quite a few more Debian packages with watch
files not maintained by the Python Modules Team.

-- 
Brian Sutherland

From donald at stufft.io  Tue Oct  8 16:44:06 2013
From: donald at stufft.io (Donald Stufft)
Date: Tue, 8 Oct 2013 10:44:06 -0400
Subject: [Distutils] URL Structure of Packages URLs
In-Reply-To: <20131008140553.GA20713@fridge.pov.lt>
References: <70E49064-C95C-4B81-ACE5-13F76688803B@stufft.io>
 <20131008140553.GA20713@fridge.pov.lt>
Message-ID: <60EC66B9-7661-4F3E-BECC-6A6520BE57DD@stufft.io>

Hrm, I'm assuming these require a file listing at

/packages/source/D/ too.

Of course these files should probably be using the simple API and
not the packages url directly :/

On Oct 8, 2013, at 10:05 AM, Marius Gedminas <marius at pov.lt> wrote:

> On Tue, Oct 08, 2013 at 09:19:37AM -0400, Donald Stufft wrote:
>> I was wondering if anyone was aware of anything that relies on
>> the current structure of package urls, namely:
>> 
>>    /packages/source|any|etc/D/Django-1.0.tar.gz?
>> 
>> I would like to change this but before I work up a concrete plan for
>> people to comment on and discuss I'm trying to figure out what, if
>> anything, depends on that current structure.
> 
> I've told Launchpad that it can look for new releases at URLs like
> http://pypi.python.org/packages/source/g/gtimelog/gtimelog-*.tar.gz
> (for a few projects, not just GTimeLog).
> 
> I'm not sure how Launchpad crawls those pages looking for releases; all
> the documentation I could find in 5 minutes was this blog post:
> http://blog.launchpad.net/cool-new-stuff/automatically-import-files-to-launchpad-using-product-release-finder
> 
> 
> Some Debian packages have debian/watch files using similar URL patterns
> to watch for new upstream releases showing up on PyPI.  This mechanism
> is documented at https://wiki.debian.org/debian/watch/.  Here's
> python-requests:
> 
>    $ apt-get source python-requests
>    $ cat requests-1.1.0/debian/watch
>    version=3
>    http://pypi.python.org/packages/source/r/requests/requests-(.*)\.tar\.gz
> 
> 
> HTH,
> Marius Gedminas
> -- 
> Some people have told me they don't think a fat penguin really embodies the
> grace of Linux, which just tells me they have never seen a angry penguin
> charging at them in excess of 100mph. They'd be a lot more careful about what
> they say if they had.
>                -- Linus Torvalds, announcing Linux v2.0
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131008/574359c5/attachment.sig>

From holger at merlinux.eu  Wed Oct  9 17:46:50 2013
From: holger at merlinux.eu (holger krekel)
Date: Wed, 9 Oct 2013 15:46:50 +0000
Subject: [Distutils] URL Structure of Packages URLs
In-Reply-To: <60EC66B9-7661-4F3E-BECC-6A6520BE57DD@stufft.io>
References: <70E49064-C95C-4B81-ACE5-13F76688803B@stufft.io>
 <20131008140553.GA20713@fridge.pov.lt>
 <60EC66B9-7661-4F3E-BECC-6A6520BE57DD@stufft.io>
Message-ID: <20131009154650.GZ14010@merlinux.eu>

On Tue, Oct 08, 2013 at 10:44 -0400, Donald Stufft wrote:
> Hrm, I'm assuming these require a file listing at
> 
> /packages/source/D/ too.
> 
> Of course these files should probably be using the simple API and
> not the packages url directly :/

Indeed.  Why not watch a project's simple index instead?  It might give
false updates if a project is in non-pypi hosting mode and a new
description is registered (this could change links in the simple page).
But those should be rare i'd think.

Watching such distribution-type specific URLs is unreliable,
for example when a pypi maintainer change their distribution format
(e.g. to wheel which is kind of realistic in the mid-term).  So a
change will need to happen in suchs cases, anyway.

cheers,
holger

> On Oct 8, 2013, at 10:05 AM, Marius Gedminas <marius at pov.lt> wrote:
> 
> > On Tue, Oct 08, 2013 at 09:19:37AM -0400, Donald Stufft wrote:
> >> I was wondering if anyone was aware of anything that relies on
> >> the current structure of package urls, namely:
> >> 
> >>    /packages/source|any|etc/D/Django-1.0.tar.gz?
> >> 
> >> I would like to change this but before I work up a concrete plan for
> >> people to comment on and discuss I'm trying to figure out what, if
> >> anything, depends on that current structure.
> > 
> > I've told Launchpad that it can look for new releases at URLs like
> > http://pypi.python.org/packages/source/g/gtimelog/gtimelog-*.tar.gz
> > (for a few projects, not just GTimeLog).
> > 
> > I'm not sure how Launchpad crawls those pages looking for releases; all
> > the documentation I could find in 5 minutes was this blog post:
> > http://blog.launchpad.net/cool-new-stuff/automatically-import-files-to-launchpad-using-product-release-finder
> > 
> > 
> > Some Debian packages have debian/watch files using similar URL patterns
> > to watch for new upstream releases showing up on PyPI.  This mechanism
> > is documented at https://wiki.debian.org/debian/watch/.  Here's
> > python-requests:
> > 
> >    $ apt-get source python-requests
> >    $ cat requests-1.1.0/debian/watch
> >    version=3
> >    http://pypi.python.org/packages/source/r/requests/requests-(.*)\.tar\.gz
> > 
> > 
> > HTH,
> > Marius Gedminas
> > -- 
> > Some people have told me they don't think a fat penguin really embodies the
> > grace of Linux, which just tells me they have never seen a angry penguin
> > charging at them in excess of 100mph. They'd be a lot more careful about what
> > they say if they had.
> >                -- Linus Torvalds, announcing Linux v2.0
> > _______________________________________________
> > Distutils-SIG maillist  -  Distutils-SIG at python.org
> > https://mail.python.org/mailman/listinfo/distutils-sig
> 
> 
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> 



> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


From barry at python.org  Wed Oct  9 20:02:43 2013
From: barry at python.org (Barry Warsaw)
Date: Wed, 9 Oct 2013 14:02:43 -0400
Subject: [Distutils] URL Structure of Packages URLs
References: <70E49064-C95C-4B81-ACE5-13F76688803B@stufft.io>
 <20131008140553.GA20713@fridge.pov.lt>
 <60EC66B9-7661-4F3E-BECC-6A6520BE57DD@stufft.io>
Message-ID: <20131009140243.76b45d85@anarchist>

On Oct 08, 2013, at 10:44 AM, Donald Stufft wrote:

>Hrm, I'm assuming these require a file listing at
>
>/packages/source/D/ too.
>
>Of course these files should probably be using the simple API and
>not the packages url directly :/

Maybe that URL should be given on the PyPI page for the package?

The current watch patterns are used overwhelmingly probably because they are
easily discovered.

E.g. Copy the URL from the PyPI page, and "patternize" it.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131009/e5636e74/attachment.sig>

From tseaver at palladion.com  Wed Oct  9 20:52:32 2013
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 09 Oct 2013 14:52:32 -0400
Subject: [Distutils] URL Structure of Packages URLs
In-Reply-To: <70E49064-C95C-4B81-ACE5-13F76688803B@stufft.io>
References: <70E49064-C95C-4B81-ACE5-13F76688803B@stufft.io>
Message-ID: <l348l7$vdi$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/08/2013 09:19 AM, Donald Stufft wrote:
> I was wondering if anyone was aware of anything that relies on the
> current structure of package urls, namely:
> 
> /packages/source|any|etc/D/Django-1.0.tar.gz?
> 
> I would like to change this but before I work up a concrete plan for 
> people to comment on and discuss I'm trying to figure out what, if 
> anything, depends on that current structure.

Any such change should leave behind "rewrite rules" which 301 to the new,
"blest" URLs.  #dontbreaktheweb and all that.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlJVpfAACgkQ+gerLs4ltQ6rhwCffZISAc+VDiqX5NQ/3ET9inJp
daAAmgNsHb/0vIVjSTMLrxEGT9S4yodT
=2TOU
-----END PGP SIGNATURE-----


From barry at python.org  Sat Oct 12 23:42:26 2013
From: barry at python.org (Barry Warsaw)
Date: Sat, 12 Oct 2013 17:42:26 -0400
Subject: [Distutils] URL Structure of Packages URLs
In-Reply-To: <CAAWk_Dxs=r3pRdn9ATsTqUd1tJGQiD+f2oXF4uuNsym-Z09NDg@mail.gmail.com>
References: <70E49064-C95C-4B81-ACE5-13F76688803B@stufft.io>
 <20131008140553.GA20713@fridge.pov.lt>
 <60EC66B9-7661-4F3E-BECC-6A6520BE57DD@stufft.io>
 <20131009140243.76b45d85@anarchist>
 <CAAWk_Dxs=r3pRdn9ATsTqUd1tJGQiD+f2oXF4uuNsym-Z09NDg@mail.gmail.com>
Message-ID: <20131012174226.7fa3b965@anarchist>

On Oct 12, 2013, at 08:50 PM, Floris Bruynooghe wrote:

>Sounds like this could be addressed with a lintian warning and a long
>transition time (~2 years) so leaving proper redirects around would be
>advisable.

Sure, but my point was that people will use the most easily discoverable
pattern available, which currently is the download link from the package's
PyPI page.  If we want to make some other link more commonly used, it should
be more easily discoverable.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131012/c9b23290/attachment.sig>

From donald at stufft.io  Sun Oct 13 00:15:53 2013
From: donald at stufft.io (Donald Stufft)
Date: Sat, 12 Oct 2013 18:15:53 -0400
Subject: [Distutils] URL Structure of Packages URLs
In-Reply-To: <20131012174226.7fa3b965@anarchist>
References: <70E49064-C95C-4B81-ACE5-13F76688803B@stufft.io>
 <20131008140553.GA20713@fridge.pov.lt>
 <60EC66B9-7661-4F3E-BECC-6A6520BE57DD@stufft.io>
 <20131009140243.76b45d85@anarchist>
 <CAAWk_Dxs=r3pRdn9ATsTqUd1tJGQiD+f2oXF4uuNsym-Z09NDg@mail.gmail.com>
 <20131012174226.7fa3b965@anarchist>
Message-ID: <6238CA7D-6DDE-437B-B2CF-ABFCDDDC9874@stufft.io>


On Oct 12, 2013, at 5:42 PM, Barry Warsaw <barry at python.org> wrote:

> On Oct 12, 2013, at 08:50 PM, Floris Bruynooghe wrote:
> 
>> Sounds like this could be addressed with a lintian warning and a long
>> transition time (~2 years) so leaving proper redirects around would be
>> advisable.
> 
> Sure, but my point was that people will use the most easily discoverable
> pattern available, which currently is the download link from the package's
> PyPI page.  If we want to make some other link more commonly used, it should
> be more easily discoverable.
> 
> -Barry
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

Just to be clear, I don't fault folks for using the /packages/ urls. I was just trying
to get some sort of idea if anyone actually used that url structure or not. I also
don't plan on breaking anything for people who do.

That being said, do you know if the Debian case requires something to be served
at /packages/source/D/Django/ in order to locate a file at /packages/source/D/Django/Django-*.tar.gz?

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131012/3d629e54/attachment.sig>

From donald at stufft.io  Sun Oct 13 02:26:12 2013
From: donald at stufft.io (Donald Stufft)
Date: Sat, 12 Oct 2013 20:26:12 -0400
Subject: [Distutils] Warehouse - Simple API
In-Reply-To: <CALeMXf4ogETyqOYy=L0vZQzRom_Pv0hVtbki3ax0RCwkHNbs6g@mail.gmail.com>
References: <EE9E9386-7737-41A0-85C6-B96FD9EE518C@stufft.io>
 <CALeMXf4ogETyqOYy=L0vZQzRom_Pv0hVtbki3ax0RCwkHNbs6g@mail.gmail.com>
Message-ID: <A3547CEA-A632-4EB9-AA11-A0F870FE34D5@stufft.io>


On Oct 12, 2013, at 8:23 PM, PJ Eby <pje at telecommunity.com> wrote:

> On Wed, Oct 2, 2013 at 10:57 PM, Donald Stufft <donald at stufft.io> wrote:
>> I've recently deployed the current Warehouse code. So far the
>> simple API has been implemented and it would be great if
>> people could point their installers to it and report back to me
>> if they run into any problems.
>> 
>> The current PyPI simple api is not documented at all so I've had
>> to reverse engineer it going from what I know, pip, etc. Ideally
>> if i've missed anything it will be found out with testing from y'all.
> 
> FYI, compare:
> 
> https://preview-pypi.python.org/simple/setuptools/
> 
> and:
> 
> https://pypi.python.org/simple/setuptools/
> 
> The former is missing the #egg URLs, which are required to install SVN versions.
> 
> In general, extracted URLs don't seem to be working on the preview site.

Hmm, I thought I had that working. Thanks i'll get it taken care of.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131012/957fbdc9/attachment.sig>

From pje at telecommunity.com  Sun Oct 13 02:23:24 2013
From: pje at telecommunity.com (PJ Eby)
Date: Sat, 12 Oct 2013 20:23:24 -0400
Subject: [Distutils] Warehouse - Simple API
In-Reply-To: <EE9E9386-7737-41A0-85C6-B96FD9EE518C@stufft.io>
References: <EE9E9386-7737-41A0-85C6-B96FD9EE518C@stufft.io>
Message-ID: <CALeMXf4ogETyqOYy=L0vZQzRom_Pv0hVtbki3ax0RCwkHNbs6g@mail.gmail.com>

On Wed, Oct 2, 2013 at 10:57 PM, Donald Stufft <donald at stufft.io> wrote:
> I've recently deployed the current Warehouse code. So far the
> simple API has been implemented and it would be great if
> people could point their installers to it and report back to me
> if they run into any problems.
>
> The current PyPI simple api is not documented at all so I've had
> to reverse engineer it going from what I know, pip, etc. Ideally
> if i've missed anything it will be found out with testing from y'all.

FYI, compare:

https://preview-pypi.python.org/simple/setuptools/

and:

https://pypi.python.org/simple/setuptools/

The former is missing the #egg URLs, which are required to install SVN versions.

In general, extracted URLs don't seem to be working on the preview site.

From thatcher.peskens at dotcloud.com  Fri Oct 11 20:31:46 2013
From: thatcher.peskens at dotcloud.com (Thatcher Peskens)
Date: Fri, 11 Oct 2013 11:31:46 -0700
Subject: [Distutils] dependency_links format changed?
Message-ID: <F3921BCF-3DAD-4B8F-9A74-A443F756CD63@dotcloud.com>

Hi all,

I've been looking for information on this topic, but cannot really find anything about it.. It looks like setuptools has changed something that breaks a bunch of my installs (last deployed months ago) that were working before.

I have some dependencies on a python package A* which is not available in PyPi, which itself has dependencies not available on PyPi. 

A has a setup.py containing:

install_requires=[
        'django-inline-edit',
	]

and 

dependency_links = [
    	'http://github.com/gabrielgrant/django-inline-edit/tarball/master#egg=django-inline-edit',
]


Now... This doesn't work anymore, where it used to work before.

I have found that if I specify the install_requires with an explicit version e.g. django-inline-edit==dev AND the dependency_link with 
#egg=django-inline-edit-dev it does work.


My previous experience shows that it was working without the explicit version before, and what I find on the internet also seem to suggest it should work.  Has this been changed on purpose, or should it be reported as a bug?


Thanks in advance,
Thatcher



*A -> https://github.com/gabrielgrant/django-easy-pages

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131011/ded66577/attachment.html>

From jmdana at gmail.com  Sat Oct 12 01:26:07 2013
From: jmdana at gmail.com (J.M. Dana)
Date: Sat, 12 Oct 2013 00:26:07 +0100 (BST)
Subject: [Distutils] Scripts output directory
Message-ID: <alpine.LNX.2.02.1310120010360.22489@triton.Home>


Hello,

I'd like to get (programmatically) the directory where the scripts have
been copied after doing "install".

I know that "install_scripts" has a method called "get_outputs" that
retrieves a list with all the destinations. However, when "easy_install"
is called as part of "install", "install_scripts.get_outputs()" returns
the location within the egg. I'd like to know how to retrieve the
outputs for the final step in which those are moved to the final
location.

Any thoughts?

Thanks in advance,
Jose.

From yasar11732 at gmail.com  Sat Oct 12 12:08:20 2013
From: yasar11732 at gmail.com (=?ISO-8859-9?Q?Ya=FEar_Arabac=FD?=)
Date: Sat, 12 Oct 2013 13:08:20 +0300
Subject: [Distutils] How to compile a simple Python embeded program using
	distutils (windows)
Message-ID: <CAFEUn8YFp0krydGV9F1ickxtV7R7ne=PxiAhjmeO_6EQrNxXvg@mail.gmail.com>

Hi,

I want to compile a simple python embedded program using distutils. My
codes are attached.

Codes create python_launcher.exe but windows gives me not a valid
win32 application error.

I can compile same code using visual c++ 2008 gui, so there is not any
problem on my compiler. I think problem is the way I use distutils.

My motivation is to create an executable file that will be distributed
with python27.dll so that end users can just click and run the program
without having to setup python interpreter on their computer.

This question can be found on
http://stackoverflow.com/questions/19332836/how-to-compile-simple-python-embeded-program-using-distutils
if you prefer answering on stackoverflow.

Additionally, can I use distutils to find paths like
`C:\Python27\include`, so that my codes can be portable?


-- 
http://ysar.net/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: make_launcher.py
Type: application/octet-stream
Size: 743 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131012/c511a8c1/attachment.obj>

From floris.bruynooghe at gmail.com  Sat Oct 12 21:50:33 2013
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Sat, 12 Oct 2013 20:50:33 +0100
Subject: [Distutils] URL Structure of Packages URLs
In-Reply-To: <20131009140243.76b45d85@anarchist>
References: <70E49064-C95C-4B81-ACE5-13F76688803B@stufft.io>
 <20131008140553.GA20713@fridge.pov.lt>
 <60EC66B9-7661-4F3E-BECC-6A6520BE57DD@stufft.io>
 <20131009140243.76b45d85@anarchist>
Message-ID: <CAAWk_Dxs=r3pRdn9ATsTqUd1tJGQiD+f2oXF4uuNsym-Z09NDg@mail.gmail.com>

On 9 Oct 2013 21:03, "Barry Warsaw" <barry at python.org> wrote:
>
> On Oct 08, 2013, at 10:44 AM, Donald Stufft wrote:
>
> >Hrm, I'm assuming these require a file listing at
> >
> >/packages/source/D/ too.
> >
> >Of course these files should probably be using the simple API and
> >not the packages url directly :/
>
> Maybe that URL should be given on the PyPI page for the package?
>
> The current watch patterns are used overwhelmingly probably because they
are
> easily discovered.
>
> E.g. Copy the URL from the PyPI page, and "patternize" it.

Sounds like this could be addressed with a lintian warning and a long
transition time (~2 years) so leaving proper redirects around would be
advisable.

Regards,
Floris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131012/ff62cdba/attachment.html>

From jmdana at gmail.com  Sun Oct 13 03:50:55 2013
From: jmdana at gmail.com (J.M. Dana)
Date: Sun, 13 Oct 2013 02:50:55 +0100
Subject: [Distutils] Scripts output directory
Message-ID: <20131013015055.GB3645@triton.Home>


Hello,

I'd like to get (programmatically) the directory where the scripts have
been copied after doing "install".

I know that "install_scripts" has a method called "get_outputs" that
retrieves a list with all the destinations. However, when "easy_install"
is called as part of "install", "install_scripts.get_outputs()" returns
the location within the egg. I'd like to know how to retrieve the
outputs for the final step in which those are moved to the final
location.

Any thoughts?

Thanks in advance,
Jose.

From marius at pov.lt  Mon Oct 14 14:47:02 2013
From: marius at pov.lt (Marius Gedminas)
Date: Mon, 14 Oct 2013 15:47:02 +0300
Subject: [Distutils] dependency_links format changed?
In-Reply-To: <F3921BCF-3DAD-4B8F-9A74-A443F756CD63@dotcloud.com>
References: <F3921BCF-3DAD-4B8F-9A74-A443F756CD63@dotcloud.com>
Message-ID: <20131014124702.GA1943@fridge.pov.lt>

On Fri, Oct 11, 2013 at 11:31:46AM -0700, Thatcher Peskens wrote:
> I've been looking for information on this topic, but cannot really
> find anything about it.. It looks like setuptools has changed
> something that breaks a bunch of my installs (last deployed months
> ago) that were working before.

I don't believe anything major changed in setuptools itself, but pip 1.4
changed behavior and stopped installing unreleased package versions by
default.

> I have some dependencies on a python package A* which is not available
> in PyPi, which itself has dependencies not available on PyPi. 
> 
> A has a setup.py containing:
> 
> install_requires=[
>         'django-inline-edit',
> 	]
> 
> and 
> 
> dependency_links = [
>     	'http://github.com/gabrielgrant/django-inline-edit/tarball/master#egg=django-inline-edit',
> ]
> 
> 
> Now... This doesn't work anymore, where it used to work before.
> 
> I have found that if I specify the install_requires with an explicit
> version e.g. django-inline-edit==dev AND the dependency_link with
> #egg=django-inline-edit-dev it does work.

Sounds like the pip change.

> My previous experience shows that it was working without the explicit
> version before, and what I find on the internet also seem to suggest
> it should work.  Has this been changed on purpose, or should it be
> reported as a bug?

It was on purpose.

Marius Gedminas
-- 
Gates' Law: Every 18 months, the speed of software halves.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131014/217f5963/attachment.sig>

From agroszer.ll at gmail.com  Mon Oct 14 19:33:03 2013
From: agroszer.ll at gmail.com (Adam GROSZER)
Date: Mon, 14 Oct 2013 19:33:03 +0200
Subject: [Distutils] zc.buildout bootstrap.py v1 died /
	python-distribute.org down
Message-ID: <525C2ACF.8080309@gmail.com>

Hi,

python-distribute.org seems to be down, that breaks zc.buildout v1 
bootstrap.py when used with "-d" to use distribute.

-- 
Best regards,
  Adam GROSZER
--
Quote of the day:
"Some days you're the dog, and some days you're the hydrant."

From aclark at aclark.net  Tue Oct 15 00:14:15 2013
From: aclark at aclark.net (Alex Clark)
Date: Mon, 14 Oct 2013 22:14:15 +0000 (UTC)
Subject: [Distutils]
	=?utf-8?q?zc=2Ebuildout_bootstrap=2Epy_v1_died_/=09py?=
	=?utf-8?q?thon-distribute=2Eorg_down?=
References: <525C2ACF.8080309@gmail.com>
Message-ID: <loom.20131015T001328-582@post.gmane.org>

Adam GROSZER <agroszer.ll <at> gmail.com> writes:


> python-distribute.org seems to be down, that breaks zc.buildout v1 
> bootstrap.py when used with "-d" to use distribute.


Distribute is dead, long live Setuptools (don't use -d anymore)




From hv at tbz-pariv.de  Mon Oct 14 11:40:36 2013
From: hv at tbz-pariv.de (=?ISO-8859-15?Q?Thomas_G=FCttler?=)
Date: Mon, 14 Oct 2013 11:40:36 +0200
Subject: [Distutils] common command after repo update.
Message-ID: <525BBC14.9050102@tbz-pariv.de>

Hi,

from http://pythonhosted.org/setuptools/setuptools.html#dynamic-discovery-of-services-and-plugins

{{{
about setup.py develop:

... For example, if you recommend that people use setup.py develop when tracking your in-development code, you should 
let them know that this needs to be run after every update or commit.

}}}

It would be very nice, if you could agree on a common name, and suggest it here.

I know, you can't force it. Please give a good advice here.

Should I open a new ticket?

   Thomas G?ttler

From hv at tbz-pariv.de  Mon Oct 14 12:54:26 2013
From: hv at tbz-pariv.de (=?ISO-8859-15?Q?Thomas_G=FCttler?=)
Date: Mon, 14 Oct 2013 12:54:26 +0200
Subject: [Distutils] develop install: how to update entry point file
Message-ID: <525BCD62.5000208@tbz-pariv.de>

Hi,


what needs to be done, to get an update of the entry_point file,
if you have installed a package as "develop".

Example:

pip install -e ...#egg=mylib
git update # new code contains new entry point
... entrypoint not available.

   Thomas

From donald at stufft.io  Tue Oct 15 02:10:58 2013
From: donald at stufft.io (Donald Stufft)
Date: Mon, 14 Oct 2013 20:10:58 -0400
Subject: [Distutils] develop install: how to update entry point file
In-Reply-To: <525BCD62.5000208@tbz-pariv.de>
References: <525BCD62.5000208@tbz-pariv.de>
Message-ID: <DB631A36-A535-430C-9CDA-A39B5E4059D0@stufft.io>

Run pip install -e again, or possible just setup.py egg_info

On Oct 14, 2013, at 6:54 AM, Thomas G?ttler <hv at tbz-pariv.de> wrote:

> Hi,
> 
> 
> what needs to be done, to get an update of the entry_point file,
> if you have installed a package as "develop".
> 
> Example:
> 
> pip install -e ...#egg=mylib
> git update # new code contains new entry point
> ... entrypoint not available.
> 
>  Thomas
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131014/80338bdc/attachment.sig>

From marius at pov.lt  Tue Oct 15 08:17:50 2013
From: marius at pov.lt (Marius Gedminas)
Date: Tue, 15 Oct 2013 09:17:50 +0300
Subject: [Distutils] zc.buildout bootstrap.py v1 died
 /?python-distribute.org down
In-Reply-To: <loom.20131015T001328-582@post.gmane.org>
References: <525C2ACF.8080309@gmail.com>
 <loom.20131015T001328-582@post.gmane.org>
Message-ID: <20131015061749.GB18055@fridge.pov.lt>

On Mon, Oct 14, 2013 at 10:14:15PM +0000, Alex Clark wrote:
> Adam GROSZER <agroszer.ll <at> gmail.com> writes:
> > python-distribute.org seems to be down, that breaks zc.buildout v1 
> > bootstrap.py when used with "-d" to use distribute.
> 
> Distribute is dead, long live Setuptools (don't use -d anymore)

Are you saying that python-distribute.org was shut down on purpose and
will not come back up?

Marius Gedminas
-- 
We have enough youth, how about a fountain of SMART?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131015/05457bb1/attachment-0001.sig>

From agroszer.ll at gmail.com  Tue Oct 15 09:43:25 2013
From: agroszer.ll at gmail.com (Adam GROSZER)
Date: Tue, 15 Oct 2013 09:43:25 +0200
Subject: [Distutils] zc.buildout bootstrap.py v1 died /
 python-distribute.org down
In-Reply-To: <loom.20131015T001328-582@post.gmane.org>
References: <525C2ACF.8080309@gmail.com>
 <loom.20131015T001328-582@post.gmane.org>
Message-ID: <525CF21D.6010209@gmail.com>

On 10/15/2013 12:14 AM, Alex Clark wrote:
> Adam GROSZER <agroszer.ll <at> gmail.com> writes:
>
>
>> python-distribute.org seems to be down, that breaks zc.buildout v1
>> bootstrap.py when used with "-d" to use distribute.
>
>
> Distribute is dead, long live Setuptools (don't use -d anymore)

Well easier said than done.
This is windows, 64bit

On top of that http://downloads.buildout.org/1/bootstrap.py references 
that URL.

-- 
Best regards,
  Adam GROSZER
--
Quote of the day:
Cowardice asks, Is it safe? Expediency asks, Is it politic? Vanity asks, 
Is it popular? But conscience asks, Is it right?
- William Morley Punshon

From ncoghlan at gmail.com  Tue Oct 15 12:13:35 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 15 Oct 2013 20:13:35 +1000
Subject: [Distutils] zc.buildout bootstrap.py v1 died
 /?python-distribute.org down
In-Reply-To: <20131015061749.GB18055@fridge.pov.lt>
References: <525C2ACF.8080309@gmail.com>
 <loom.20131015T001328-582@post.gmane.org>
 <20131015061749.GB18055@fridge.pov.lt>
Message-ID: <CADiSq7f+_xkm8THSy3keMCFxWVJnYxZx0bjYuA8c+0=kk8yhGQ@mail.gmail.com>

On 15 Oct 2013 16:18, "Marius Gedminas" <marius at pov.lt> wrote:
>
> On Mon, Oct 14, 2013 at 10:14:15PM +0000, Alex Clark wrote:
> > Adam GROSZER <agroszer.ll <at> gmail.com> writes:
> > > python-distribute.org seems to be down, that breaks zc.buildout v1
> > > bootstrap.py when used with "-d" to use distribute.
> >
> > Distribute is dead, long live Setuptools (don't use -d anymore)
>
> Are you saying that python-distribute.org was shut down on purpose and
> will not come back up?

"On purpose" is too strong. It's more that there's nobody to investigate
any failures, as distribute has been merged back into setuptools, ending
distribute's existence as a separate project.

Cheers,
Nick.

>
> Marius Gedminas
> --
> We have enough youth, how about a fountain of SMART?
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131015/eaf53cd9/attachment.html>

From sdouche at gmail.com  Tue Oct 15 15:19:47 2013
From: sdouche at gmail.com (Sebastien Douche)
Date: Tue, 15 Oct 2013 15:19:47 +0200
Subject: [Distutils] zc.buildout bootstrap.py v1 died
 /?python-distribute.org down
In-Reply-To: <20131015061749.GB18055@fridge.pov.lt>
References: <525C2ACF.8080309@gmail.com>
 <loom.20131015T001328-582@post.gmane.org>
 <20131015061749.GB18055@fridge.pov.lt>
Message-ID: <CAAGHeXFW5cCVGdjbh7iAK6Nb_Vq24T+Nuu=ZogpRs3s9dx1MEw@mail.gmail.com>

On Tue, Oct 15, 2013 at 8:17 AM, Marius Gedminas <marius at pov.lt> wrote:
> Are you saying that python-distribute.org was shut down on purpose and
> will not come back up?

On IRC, Tarek said he forgot to renew the domain. It's done.



-- 
Sebastien Douche <sdouche at gmail.com>
Twitter: @sdouche / G+: +sdouche

From Martin.Fiers at intec.ugent.be  Tue Oct 15 14:07:05 2013
From: Martin.Fiers at intec.ugent.be (Martin Fiers)
Date: Tue, 15 Oct 2013 14:07:05 +0200
Subject: [Distutils] Egg name computation
Message-ID: <525D2FE9.6050904@intec.ugent.be>

Hello,

What's the preferred approach to having different eggs (for different 
platforms) in one folder?

I noticed that fortunately easy_install recognizes the platform, so I 
can get away by having good filenames, such as

mytest-2.1.2c1-py2.7-win32
mytest-2.1.2c1-py2.7-win-amd64

The problem is that the egg-generation does not include the platform 
name (so it is not compatible with the easy_install behavior, which is 
weird). When I look in bdist_egg.py, I can see this:

             # Compute filename of the output egg
             basename = Distribution(
                 None, None, ei_cmd.egg_name, ei_cmd.egg_version,
                 get_python_version(),
                 self.distribution.has_ext_modules() and self.plat_name
             ).egg_name()


So the platform argument now is
self.distribution.has_ext_modules() and self.plat_name
Shouldn't it just be
self.plat_name
?

Would there be a workaround? I can think of giving the full name to 
setup, without specifying a version:

setup(name='mytest-2.1.2c1-py2.7-win32', version = '', ...)


Thanks very much!
Martin

P.S. On an unrelated note, wouldn't it be more pythonic, and easier to 
read, to include keyword arguments in the basename = Distribution(...) line?

From aclark at aclark.net  Tue Oct 15 15:24:03 2013
From: aclark at aclark.net (Alex Clark)
Date: Tue, 15 Oct 2013 13:24:03 +0000 (UTC)
Subject: [Distutils] zc.buildout bootstrap.py v1 died /
	python-distribute.org down
References: <525C2ACF.8080309@gmail.com>
 <loom.20131015T001328-582@post.gmane.org> <525CF21D.6010209@gmail.com>
Message-ID: <loom.20131015T152347-716@post.gmane.org>

Adam GROSZER <agroszer.ll <at> gmail.com> writes:

>
> On top of that http://downloads.buildout.org/1/bootstrap.py references 
> that URL.
> 

Please open a ticket here: https://github.com/buildout/buildout/issues




From jim at zope.com  Tue Oct 15 15:38:05 2013
From: jim at zope.com (Jim Fulton)
Date: Tue, 15 Oct 2013 09:38:05 -0400
Subject: [Distutils] zc.buildout bootstrap.py v1 died /
 python-distribute.org down
In-Reply-To: <525CF21D.6010209@gmail.com>
References: <525C2ACF.8080309@gmail.com>
 <loom.20131015T001328-582@post.gmane.org>
 <525CF21D.6010209@gmail.com>
Message-ID: <CAPDm-FiTRxqRaSyL1W1vkAh3Ho8016NdJ9Bb4ywQpP-QWhiqig@mail.gmail.com>

On Tue, Oct 15, 2013 at 3:43 AM, Adam GROSZER <agroszer.ll at gmail.com> wrote:
> On 10/15/2013 12:14 AM, Alex Clark wrote:
>>
>> Adam GROSZER <agroszer.ll <at> gmail.com> writes:
>>
>>
>>> python-distribute.org seems to be down, that breaks zc.buildout v1
>>> bootstrap.py when used with "-d" to use distribute.
>>
>>
>>
>> Distribute is dead, long live Setuptools (don't use -d anymore)
>
>
> Well easier said than done.
> This is windows, 64bit

Does setuptools not work with 64-bit windows?

> On top of that http://downloads.buildout.org/1/bootstrap.py references that
> URL.

Is there a different URL that should be used? Or do you suggest dropping
distribute support from the buildout 1 bootstrap script?

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From agroszer.ll at gmail.com  Tue Oct 15 16:27:28 2013
From: agroszer.ll at gmail.com (Adam GROSZER)
Date: Tue, 15 Oct 2013 16:27:28 +0200
Subject: [Distutils] zc.buildout bootstrap.py v1 died /
 python-distribute.org down
In-Reply-To: <CAPDm-FiTRxqRaSyL1W1vkAh3Ho8016NdJ9Bb4ywQpP-QWhiqig@mail.gmail.com>
References: <525C2ACF.8080309@gmail.com>
 <loom.20131015T001328-582@post.gmane.org> <525CF21D.6010209@gmail.com>
 <CAPDm-FiTRxqRaSyL1W1vkAh3Ho8016NdJ9Bb4ywQpP-QWhiqig@mail.gmail.com>
Message-ID: <525D50D0.6090500@gmail.com>

On 10/15/2013 03:38 PM, Jim Fulton wrote:
> On Tue, Oct 15, 2013 at 3:43 AM, Adam GROSZER <agroszer.ll at gmail.com> wrote:
>> On 10/15/2013 12:14 AM, Alex Clark wrote:
>>>
>>> Adam GROSZER <agroszer.ll <at> gmail.com> writes:
>>>
>>>
>>>> python-distribute.org seems to be down, that breaks zc.buildout v1
>>>> bootstrap.py when used with "-d" to use distribute.
>>>
>>>
>>>
>>> Distribute is dead, long live Setuptools (don't use -d anymore)
>>
>>
>> Well easier said than done.
>> This is windows, 64bit
>
> Does setuptools not work with 64-bit windows?

0.6c11 does not work

>
>> On top of that http://downloads.buildout.org/1/bootstrap.py references that
>> URL.
>
> Is there a different URL that should be used? Or do you suggest dropping
> distribute support from the buildout 1 bootstrap script?
>
> Jim
>

This breaks current projects.
We don't have time right now to update to buildout 2.x.
A workaround is to patch the v1 bootstrap with:

distribute_source = 
'https://bitbucket.org/pypa/setuptools/raw/f657df1f1ed46596d236376649c99a470662b4ba/distribute_setup.py'

Maybe that should get into http://downloads.buildout.org/1/bootstrap.py

-- 
Best regards,
  Adam GROSZER
--
Quote of the day:
Private ownership is an invitation to disaster

From p.f.moore at gmail.com  Tue Oct 15 16:31:51 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 15 Oct 2013 15:31:51 +0100
Subject: [Distutils] PEP 453 - virtualenv change to Scripts/bin directory
Message-ID: <CACac1F_O9Cc3D_8VX8e-GJUjUZTz7GfT79g-4rZhkKG1+cUsqg@mail.gmail.com>

As part of PEP 453, there is a proposal to change the layout of python
environments on Windows by renaming the "Scripts" directory to "bin"
(to match other platforms). The PEP is targeted at Python 3.4 (core
and venv module) but for consistency virtualenv should also change to
match (only for virtualenvs based on Python 3.4+, obviously).

This change could potentially affect tools that build on virtualenv
(tox, virtualenv-wrapper, for example).

Coping with the change should be relatively easy - where code
currently uses "Scripts" on Windows, they can simply check to see
which of "Scripts" and "bin" exists and use that.

Could anyone who has code like this please check if this change will
cause them problems and let us know?

Thanks,
Paul

From jim at zope.com  Tue Oct 15 16:35:31 2013
From: jim at zope.com (Jim Fulton)
Date: Tue, 15 Oct 2013 10:35:31 -0400
Subject: [Distutils] zc.buildout bootstrap.py v1 died /
 python-distribute.org down
In-Reply-To: <525D50D0.6090500@gmail.com>
References: <525C2ACF.8080309@gmail.com>
 <loom.20131015T001328-582@post.gmane.org>
 <525CF21D.6010209@gmail.com>
 <CAPDm-FiTRxqRaSyL1W1vkAh3Ho8016NdJ9Bb4ywQpP-QWhiqig@mail.gmail.com>
 <525D50D0.6090500@gmail.com>
Message-ID: <CAPDm-Fif1Bmm+0_C9gX470wcfieA-xixXck3=ocPnfjCMYN8Ug@mail.gmail.com>

On Tue, Oct 15, 2013 at 10:27 AM, Adam GROSZER <agroszer.ll at gmail.com> wrote:
> On 10/15/2013 03:38 PM, Jim Fulton wrote:
>>
>> On Tue, Oct 15, 2013 at 3:43 AM, Adam GROSZER <agroszer.ll at gmail.com>
>> wrote:
>>>
>>> On 10/15/2013 12:14 AM, Alex Clark wrote:
>>>>
>>>>
>>>> Adam GROSZER <agroszer.ll <at> gmail.com> writes:
>>>>
>>>>
>>>>> python-distribute.org seems to be down, that breaks zc.buildout v1
>>>>> bootstrap.py when used with "-d" to use distribute.
>>>>
>>>>
>>>>
>>>>
>>>> Distribute is dead, long live Setuptools (don't use -d anymore)
>>>
>>>
>>>
>>> Well easier said than done.
>>> This is windows, 64bit
>>
>>
>> Does setuptools not work with 64-bit windows?
>
>
> 0.6c11 does not work

There are quite a few 0.6c12 releases that were never uploaded to
PyPI.  Perhaps one of
those works.  I'd be happy to provide the eggs I have if someone wants
to try them out.

Maybe PJE knows of they work on 64-bit windows.


>
>
>>
>>> On top of that http://downloads.buildout.org/1/bootstrap.py references
>>> that
>>> URL.
>>
>>
>> Is there a different URL that should be used? Or do you suggest dropping
>> distribute support from the buildout 1 bootstrap script?
>>
>> Jim
>>
>
> This breaks current projects.
> We don't have time right now to update to buildout 2.x.
> A workaround is to patch the v1 bootstrap with:
>
> distribute_source =
> 'https://bitbucket.org/pypa/setuptools/raw/f657df1f1ed46596d236376649c99a470662b4ba/distribute_setup.py'
>
> Maybe that should get into http://downloads.buildout.org/1/bootstrap.py

OK, I'll work on that.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From aclark at aclark.net  Tue Oct 15 16:46:28 2013
From: aclark at aclark.net (Alex Clark)
Date: Tue, 15 Oct 2013 14:46:28 +0000 (UTC)
Subject: [Distutils] zc.buildout bootstrap.py v1 died /
	python-distribute.org down
References: <525C2ACF.8080309@gmail.com>
 <loom.20131015T001328-582@post.gmane.org> <525CF21D.6010209@gmail.com>
 <CAPDm-FiTRxqRaSyL1W1vkAh3Ho8016NdJ9Bb4ywQpP-QWhiqig@mail.gmail.com>
 <525D50D0.6090500@gmail.com>
Message-ID: <loom.20131015T163959-53@post.gmane.org>

Adam GROSZER <agroszer.ll <at> gmail.com> writes:



> 
> 0.6c11 does not work

Why do you care about 0.6c11 when the latest release is 1.1.6? (Exes are
here: http://www.lfd.uci.edu/~gohlke/pythonlibs/ and if jaraco or PJE are
watching, you should ask cgohlke if you can upload those to PyPI like we do
with Pillow:
https://pypi.python.org/pypi?name=Pillow&version=2.2.1&:action=files)



From nad at acm.org  Tue Oct 15 20:25:44 2013
From: nad at acm.org (Ned Deily)
Date: Tue, 15 Oct 2013 11:25:44 -0700
Subject: [Distutils] PEP 453 - virtualenv change to Scripts/bin directory
References: <CACac1F_O9Cc3D_8VX8e-GJUjUZTz7GfT79g-4rZhkKG1+cUsqg@mail.gmail.com>
Message-ID: <nad-F5A2BE.11254315102013@news.gmane.org>

In article 
<CACac1F_O9Cc3D_8VX8e-GJUjUZTz7GfT79g-4rZhkKG1+cUsqg at mail.gmail.com>,
 Paul Moore <p.f.moore at gmail.com> wrote:
> As part of PEP 453, there is a proposal to change the layout of python
> environments on Windows by renaming the "Scripts" directory to "bin"
> (to match other platforms). The PEP is targeted at Python 3.4 (core
> and venv module) but for consistency virtualenv should also change to
> match (only for virtualenvs based on Python 3.4+, obviously).
> 
> This change could potentially affect tools that build on virtualenv
> (tox, virtualenv-wrapper, for example).
> 
> Coping with the change should be relatively easy - where code
> currently uses "Scripts" on Windows, they can simply check to see
> which of "Scripts" and "bin" exists and use that.
> 
> Could anyone who has code like this please check if this change will
> cause them problems and let us know?

It might be worthwhile reviewing the discussion that took place on python-dev 
last year when Van Lindberg proposed changing the Windows file layout, 
including the scripts path.

https://mail.python.org/pipermail/python-dev/2012-March/117552.html

-- 
 Ned Deily,
 nad at acm.org


From p.f.moore at gmail.com  Tue Oct 15 21:31:51 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 15 Oct 2013 20:31:51 +0100
Subject: [Distutils] PEP 453 - virtualenv change to Scripts/bin directory
In-Reply-To: <nad-F5A2BE.11254315102013@news.gmane.org>
References: <CACac1F_O9Cc3D_8VX8e-GJUjUZTz7GfT79g-4rZhkKG1+cUsqg@mail.gmail.com>
 <nad-F5A2BE.11254315102013@news.gmane.org>
Message-ID: <CACac1F8BES=nXMp9pqY=Vm+s1bj3O2D=1mJgZjRqbuZBu+AHqw@mail.gmail.com>

On 15 October 2013 19:25, Ned Deily <nad at acm.org> wrote:
> It might be worthwhile reviewing the discussion that took place on python-dev
> last year when Van Lindberg proposed changing the Windows file layout,
> including the scripts path.
>
> https://mail.python.org/pipermail/python-dev/2012-March/117552.html

Thanks, I thought this had come up before. As the PEP 453 (bundling
PIP) proposal was originally made by Nick, maybe he would like to
check that any issues covered there are addressed. Personally, I'm
going to focus on the implied virtualenv change, on the assumption
taht PEP 453 is accepted with the Scripts->bin change included (which
is not to say that I won't read that thread, just that I don't
personally have any means to change the PEP itself).

Paul

From p.f.moore at gmail.com  Tue Oct 15 22:20:59 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 15 Oct 2013 21:20:59 +0100
Subject: [Distutils] PEP 453 - virtualenv change to Scripts/bin directory
In-Reply-To: <CACac1F8BES=nXMp9pqY=Vm+s1bj3O2D=1mJgZjRqbuZBu+AHqw@mail.gmail.com>
References: <CACac1F_O9Cc3D_8VX8e-GJUjUZTz7GfT79g-4rZhkKG1+cUsqg@mail.gmail.com>
 <nad-F5A2BE.11254315102013@news.gmane.org>
 <CACac1F8BES=nXMp9pqY=Vm+s1bj3O2D=1mJgZjRqbuZBu+AHqw@mail.gmail.com>
Message-ID: <CACac1F8ynyipnboQSijV=Vr7=ciJ5bZnt601tYBFhOM6+_xzUg@mail.gmail.com>

On 15 October 2013 20:31, Paul Moore <p.f.moore at gmail.com> wrote:
> On 15 October 2013 19:25, Ned Deily <nad at acm.org> wrote:
>> It might be worthwhile reviewing the discussion that took place on python-dev
>> last year when Van Lindberg proposed changing the Windows file layout,
>> including the scripts path.
>>
>> https://mail.python.org/pipermail/python-dev/2012-March/117552.html
>
> Thanks, I thought this had come up before. As the PEP 453 (bundling
> PIP) proposal was originally made by Nick, maybe he would like to
> check that any issues covered there are addressed. Personally, I'm
> going to focus on the implied virtualenv change, on the assumption
> taht PEP 453 is accepted with the Scripts->bin change included (which
> is not to say that I won't read that thread, just that I don't
> personally have any means to change the PEP itself).

I've read a reasonable chunk of that thread. The things that struck me were:

1. It's very confused between a number of proposals, making it
extremely hard to identify which arguments apply to a simple
Scripts->bin renaming (both at the time, and now in hindsight).
2. As far as I can see, on the subject of Scripts->bin, the thread
from python-dev was essentially neutral, nobody could see a benefit
that justified the cost. (PEP 453 doesn't say much on the benefits,
either, FWIW).

By the way, PEP 453 is actually misleading in one respect, distutils
(and setuptools) installs scripts to PythonXY\Scripts (which is
created on first use, not at install time), it's only the
core-supplied scripts that go into PythonXY\Tools\Scripts.

Personally, I'd like to see a better justification than "to improve
consistency". I'd also like the point that the *existing* Scripts
directory (managed by distutils/setuptools installs) will have the
contents of Tools/Scripts dumped into it made explicit and discussed.
(For example, there's a diff.py in Tools/Scripts. Will the new bin
directory be put at the start or end of PATH? If I have .py in my
PATHEXT that matters to me, as diff.py could end up overriding my
existing diff.exe). And I'd like to see the actual patch that
implements the Scripts->bin change - I suspect that it would benefit
from careful review.

Overall, I'm -0 on the proposal, The arguments either way are weak,
and I don't see why backward compatibility is being ignored quite so
calmly. It is entirely peripheral to the PEP as far as I can see, and
not worth including in there.

But once again, that's separate from my point here, which is that I
need to collect information on whether virtualenv users will be
impacted by the corresponding change in virtualenv.

I suggest that virtualenv comments remain on distutils-sig, but
comments on the PEP proposal to rename Scripts be redirected to
python-dev which is where the PEP discussion is taking place. I
*haven't* cross-posted to python-dev, because that would only start a
2-list thread, but can people redirect appropriately any responses?

Paul

From ncoghlan at gmail.com  Tue Oct 15 23:33:17 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 16 Oct 2013 07:33:17 +1000
Subject: [Distutils] PEP 453 - virtualenv change to Scripts/bin directory
In-Reply-To: <CACac1F8ynyipnboQSijV=Vr7=ciJ5bZnt601tYBFhOM6+_xzUg@mail.gmail.com>
References: <CACac1F_O9Cc3D_8VX8e-GJUjUZTz7GfT79g-4rZhkKG1+cUsqg@mail.gmail.com>
 <nad-F5A2BE.11254315102013@news.gmane.org>
 <CACac1F8BES=nXMp9pqY=Vm+s1bj3O2D=1mJgZjRqbuZBu+AHqw@mail.gmail.com>
 <CACac1F8ynyipnboQSijV=Vr7=ciJ5bZnt601tYBFhOM6+_xzUg@mail.gmail.com>
Message-ID: <CADiSq7dT654aCjNmB54Qq602B95esKnPLXwMz-hYnC5b-kai2w@mail.gmail.com>

On 16 Oct 2013 06:20, "Paul Moore" <p.f.moore at gmail.com> wrote:
>
> On 15 October 2013 20:31, Paul Moore <p.f.moore at gmail.com> wrote:
> > On 15 October 2013 19:25, Ned Deily <nad at acm.org> wrote:
> >> It might be worthwhile reviewing the discussion that took place on
python-dev
> >> last year when Van Lindberg proposed changing the Windows file layout,
> >> including the scripts path.
> >>
> >> https://mail.python.org/pipermail/python-dev/2012-March/117552.html
> >
> > Thanks, I thought this had come up before. As the PEP 453 (bundling
> > PIP) proposal was originally made by Nick, maybe he would like to
> > check that any issues covered there are addressed. Personally, I'm
> > going to focus on the implied virtualenv change, on the assumption
> > taht PEP 453 is accepted with the Scripts->bin change included (which
> > is not to say that I won't read that thread, just that I don't
> > personally have any means to change the PEP itself).
>
> I've read a reasonable chunk of that thread. The things that struck me
were:
>
> 1. It's very confused between a number of proposals, making it
> extremely hard to identify which arguments apply to a simple
> Scripts->bin renaming (both at the time, and now in hindsight).
> 2. As far as I can see, on the subject of Scripts->bin, the thread
> from python-dev was essentially neutral, nobody could see a benefit
> that justified the cost. (PEP 453 doesn't say much on the benefits,
> either, FWIW).
>
> By the way, PEP 453 is actually misleading in one respect, distutils
> (and setuptools) installs scripts to PythonXY\Scripts (which is
> created on first use, not at install time), it's only the
> core-supplied scripts that go into PythonXY\Tools\Scripts.
>
> Personally, I'd like to see a better justification than "to improve
> consistency". I'd also like the point that the *existing* Scripts
> directory (managed by distutils/setuptools installs) will have the
> contents of Tools/Scripts dumped into it made explicit and discussed.
> (For example, there's a diff.py in Tools/Scripts. Will the new bin
> directory be put at the start or end of PATH? If I have .py in my
> PATHEXT that matters to me, as diff.py could end up overriding my
> existing diff.exe). And I'd like to see the actual patch that
> implements the Scripts->bin change - I suspect that it would benefit
> from careful review.
>
> Overall, I'm -0 on the proposal, The arguments either way are weak,
> and I don't see why backward compatibility is being ignored quite so
> calmly. It is entirely peripheral to the PEP as far as I can see, and
> not worth including in there.

It's only there because MvL didn't really want to add the existing Scripts
directory to PATH.

As for compatibility, the view is that the filesystem layout is an
implementation detail, with sysconfig as the stable API with backwards
compatibility guarantees.

Cheers,
Nick.

>
> But once again, that's separate from my point here, which is that I
> need to collect information on whether virtualenv users will be
> impacted by the corresponding change in virtualenv.
>
> I suggest that virtualenv comments remain on distutils-sig, but
> comments on the PEP proposal to rename Scripts be redirected to
> python-dev which is where the PEP discussion is taking place. I
> *haven't* cross-posted to python-dev, because that would only start a
> 2-list thread, but can people redirect appropriately any responses?
>
> Paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131016/ce790fad/attachment-0001.html>

From p.f.moore at gmail.com  Wed Oct 16 00:17:14 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 15 Oct 2013 23:17:14 +0100
Subject: [Distutils] PEP 453 - virtualenv change to Scripts/bin directory
In-Reply-To: <CADiSq7dT654aCjNmB54Qq602B95esKnPLXwMz-hYnC5b-kai2w@mail.gmail.com>
References: <CACac1F_O9Cc3D_8VX8e-GJUjUZTz7GfT79g-4rZhkKG1+cUsqg@mail.gmail.com>
 <nad-F5A2BE.11254315102013@news.gmane.org>
 <CACac1F8BES=nXMp9pqY=Vm+s1bj3O2D=1mJgZjRqbuZBu+AHqw@mail.gmail.com>
 <CACac1F8ynyipnboQSijV=Vr7=ciJ5bZnt601tYBFhOM6+_xzUg@mail.gmail.com>
 <CADiSq7dT654aCjNmB54Qq602B95esKnPLXwMz-hYnC5b-kai2w@mail.gmail.com>
Message-ID: <CACac1F_rOK-8T7rm-1aFknTiVF2ap9Yb9EEdRfDH=wnnSHgZFg@mail.gmail.com>

On 15 October 2013 22:33, Nick Coghlan <ncoghlan at gmail.com> wrote:
> It's only there because MvL didn't really want to add the existing Scripts
> directory to PATH.

Sorry to bang on about this, but do you mean PythonXY\Scripts (which
doesn't exist at install time and hence I can see why adding it to
PATH could be an issue) or PythonXY\Tools\Scripts (which isn't used by
pip/distutils/setuptools, so adding it wouldn't put pip.exe on PATH
anyway)?

There are two parts to this proposal:

1. Use bin instead of Scripts (which means changing sysconfig and
possibly distutils or 3rd party tools that hard code the name rather
than using sysconfig).
2. Expose Tools/Scripts by putting the code that's currently there into bin.

I'm -0 on the first of these, and -1 on the second. I see the scripts
in Tools/Scripts as a mixture of examples and optional utilities. If I
follow the makefile, they aren't installed by default on Unix, they
are part of the scriptinstall target. In the Windows installer they
are an optional install ("Utility Scripts") that currently are
installed but not on PATH by default.

Anyway, I'm the only one arguing here and I've said my piece, so I'll
leave it now.

As for virtualenv:

1. I'll implement the change once PEP 453 is accepted and there's a
patch for CPython implementing it (so that I can confirm any
subtleties of behaviour) assuming no-one wants to do it before then.
2. I'll assume that if there's no response on this thread, that
virtualenv users won't have issues with this change.

Paul

From ncoghlan at gmail.com  Wed Oct 16 00:41:00 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 16 Oct 2013 08:41:00 +1000
Subject: [Distutils] PEP 453 - virtualenv change to Scripts/bin directory
In-Reply-To: <CACac1F_rOK-8T7rm-1aFknTiVF2ap9Yb9EEdRfDH=wnnSHgZFg@mail.gmail.com>
References: <CACac1F_O9Cc3D_8VX8e-GJUjUZTz7GfT79g-4rZhkKG1+cUsqg@mail.gmail.com>
 <nad-F5A2BE.11254315102013@news.gmane.org>
 <CACac1F8BES=nXMp9pqY=Vm+s1bj3O2D=1mJgZjRqbuZBu+AHqw@mail.gmail.com>
 <CACac1F8ynyipnboQSijV=Vr7=ciJ5bZnt601tYBFhOM6+_xzUg@mail.gmail.com>
 <CADiSq7dT654aCjNmB54Qq602B95esKnPLXwMz-hYnC5b-kai2w@mail.gmail.com>
 <CACac1F_rOK-8T7rm-1aFknTiVF2ap9Yb9EEdRfDH=wnnSHgZFg@mail.gmail.com>
Message-ID: <CADiSq7dMpkknC+m1cNLhLHjZKx6ZxHiSSgUOMCTiiu_Xd=Oh3w@mail.gmail.com>

On 16 Oct 2013 08:17, "Paul Moore" <p.f.moore at gmail.com> wrote:
>
> On 15 October 2013 22:33, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > It's only there because MvL didn't really want to add the existing
Scripts
> > directory to PATH.
>
> Sorry to bang on about this, but do you mean PythonXY\Scripts (which
> doesn't exist at install time and hence I can see why adding it to
> PATH could be an issue) or PythonXY\Tools\Scripts (which isn't used by
> pip/distutils/setuptools, so adding it wouldn't put pip.exe on PATH
> anyway)?
>
> There are two parts to this proposal:
>
> 1. Use bin instead of Scripts (which means changing sysconfig and
> possibly distutils or 3rd party tools that hard code the name rather
> than using sysconfig).

The intended change is just this bit.

> 2. Expose Tools/Scripts by putting the code that's currently there into
bin.

This part is just a mistake in the PEP, which I will fix (that stuff
doesn't generally get installed at all under *nix, and I didn't realise it
used a different directory than pip does on Windows)

It was previously pointed out that this part was ambiguous, but I forgot to
go back and figure out the difference between the two. Thanks for spelling
it out for me :)

Cheers,
Nick.

> I'm -0 on the first of these, and -1 on the second. I see the scripts
> in Tools/Scripts as a mixture of examples and optional utilities. If I
> follow the makefile, they aren't installed by default on Unix, they
> are part of the scriptinstall target. In the Windows installer they
> are an optional install ("Utility Scripts") that currently are
> installed but not on PATH by default.
>
> Anyway, I'm the only one arguing here and I've said my piece, so I'll
> leave it now.
>
> As for virtualenv:
>
> 1. I'll implement the change once PEP 453 is accepted and there's a
> patch for CPython implementing it (so that I can confirm any
> subtleties of behaviour) assuming no-one wants to do it before then.
> 2. I'll assume that if there's no response on this thread, that
> virtualenv users won't have issues with this change.
>
> Paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131016/2b2365fd/attachment.html>

From ncoghlan at gmail.com  Wed Oct 16 00:44:04 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 16 Oct 2013 08:44:04 +1000
Subject: [Distutils] PEP 453 - virtualenv change to Scripts/bin directory
In-Reply-To: <CACac1F-bgDQkjP104jGJvEdJkhGK97PSfY2LLk371sn-E3XQKw@mail.gmail.com>
References: <CACac1F_O9Cc3D_8VX8e-GJUjUZTz7GfT79g-4rZhkKG1+cUsqg@mail.gmail.com>
 <nad-F5A2BE.11254315102013@news.gmane.org>
 <CACac1F8BES=nXMp9pqY=Vm+s1bj3O2D=1mJgZjRqbuZBu+AHqw@mail.gmail.com>
 <CACac1F8ynyipnboQSijV=Vr7=ciJ5bZnt601tYBFhOM6+_xzUg@mail.gmail.com>
 <CADiSq7dT654aCjNmB54Qq602B95esKnPLXwMz-hYnC5b-kai2w@mail.gmail.com>
 <CACac1F_rOK-8T7rm-1aFknTiVF2ap9Yb9EEdRfDH=wnnSHgZFg@mail.gmail.com>
 <CACac1F-bgDQkjP104jGJvEdJkhGK97PSfY2LLk371sn-E3XQKw@mail.gmail.com>
Message-ID: <CADiSq7dq5siF3DJ-ZAy=Uno_yzGML++fjZpHMQ82SRh41F8z=A@mail.gmail.com>

On 16 Oct 2013 08:39, "Paul Moore" <p.f.moore at gmail.com> wrote:
>
> On 15 October 2013 23:17, Paul Moore <p.f.moore at gmail.com> wrote:
> > 1. Use bin instead of Scripts (which means changing sysconfig and
> > possibly distutils or 3rd party tools that hard code the name rather
> > than using sysconfig).
>
> By the way, I just did a scan. The following from the cpython repo
> will need to change:
>
> 1. The sysconfig schemes
> 2. distutils/commands/install.py (hard coded INSTALL_SCHEMES variable)
> 3. PC\bdist_wininst\install.c (SCHEME variables hard code Scripts as
> the target for the SCRIPTS section)
> 4. venv (already noted) code and tests
> 5. Documentation
> 6. The installer scripts in Tools\msi\msi.py
>
> Also, as a result of (3) existing wininst installers will not install
> correctly on Python 3.4+ (they'll put their scripts in "Scripts"
> still). So projects will need to distribute separate installers for
> Python <= 3.3 and 3.4+ if they contain scripts. (This problem goes
> away in the long term if everyone switches to wheels, of course).

The installer compatibility issue is enough to convince me to drop the
directory name change from PEP 453.

I'll redefine that part of the PEP in terms of sysconfig.

Cheers,
Nick.

>
> I don't know if bdist_msi has similar issues. That code is not
> something I understand. There's no mention of the Scripts directory so
> I suspect item (2) will take care of building bdist_msi installers.
> But I don't know if they will have the runtime issue that wininst has
> of installing to whatever scripts directory the build was done on,
> rather than the one appropriate for the Python being installed to.
>
> Paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131016/6797b988/attachment.html>

From p.f.moore at gmail.com  Wed Oct 16 00:39:19 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 15 Oct 2013 23:39:19 +0100
Subject: [Distutils] PEP 453 - virtualenv change to Scripts/bin directory
In-Reply-To: <CACac1F_rOK-8T7rm-1aFknTiVF2ap9Yb9EEdRfDH=wnnSHgZFg@mail.gmail.com>
References: <CACac1F_O9Cc3D_8VX8e-GJUjUZTz7GfT79g-4rZhkKG1+cUsqg@mail.gmail.com>
 <nad-F5A2BE.11254315102013@news.gmane.org>
 <CACac1F8BES=nXMp9pqY=Vm+s1bj3O2D=1mJgZjRqbuZBu+AHqw@mail.gmail.com>
 <CACac1F8ynyipnboQSijV=Vr7=ciJ5bZnt601tYBFhOM6+_xzUg@mail.gmail.com>
 <CADiSq7dT654aCjNmB54Qq602B95esKnPLXwMz-hYnC5b-kai2w@mail.gmail.com>
 <CACac1F_rOK-8T7rm-1aFknTiVF2ap9Yb9EEdRfDH=wnnSHgZFg@mail.gmail.com>
Message-ID: <CACac1F-bgDQkjP104jGJvEdJkhGK97PSfY2LLk371sn-E3XQKw@mail.gmail.com>

On 15 October 2013 23:17, Paul Moore <p.f.moore at gmail.com> wrote:
> 1. Use bin instead of Scripts (which means changing sysconfig and
> possibly distutils or 3rd party tools that hard code the name rather
> than using sysconfig).

By the way, I just did a scan. The following from the cpython repo
will need to change:

1. The sysconfig schemes
2. distutils/commands/install.py (hard coded INSTALL_SCHEMES variable)
3. PC\bdist_wininst\install.c (SCHEME variables hard code Scripts as
the target for the SCRIPTS section)
4. venv (already noted) code and tests
5. Documentation
6. The installer scripts in Tools\msi\msi.py

Also, as a result of (3) existing wininst installers will not install
correctly on Python 3.4+ (they'll put their scripts in "Scripts"
still). So projects will need to distribute separate installers for
Python <= 3.3 and 3.4+ if they contain scripts. (This problem goes
away in the long term if everyone switches to wheels, of course).

I don't know if bdist_msi has similar issues. That code is not
something I understand. There's no mention of the Scripts directory so
I suspect item (2) will take care of building bdist_msi installers.
But I don't know if they will have the runtime issue that wininst has
of installing to whatever scripts directory the build was done on,
rather than the one appropriate for the Python being installed to.

Paul

From agroszer.ll at gmail.com  Wed Oct 16 08:05:35 2013
From: agroszer.ll at gmail.com (Adam GROSZER)
Date: Wed, 16 Oct 2013 08:05:35 +0200
Subject: [Distutils] zc.buildout bootstrap.py v1 died /
 python-distribute.org down
In-Reply-To: <CAPDm-Fif1Bmm+0_C9gX470wcfieA-xixXck3=ocPnfjCMYN8Ug@mail.gmail.com>
References: <525C2ACF.8080309@gmail.com>
 <loom.20131015T001328-582@post.gmane.org> <525CF21D.6010209@gmail.com>
 <CAPDm-FiTRxqRaSyL1W1vkAh3Ho8016NdJ9Bb4ywQpP-QWhiqig@mail.gmail.com>
 <525D50D0.6090500@gmail.com>
 <CAPDm-Fif1Bmm+0_C9gX470wcfieA-xixXck3=ocPnfjCMYN8Ug@mail.gmail.com>
Message-ID: <525E2CAF.4080409@gmail.com>

On 10/15/2013 04:35 PM, Jim Fulton wrote:
> On Tue, Oct 15, 2013 at 10:27 AM, Adam GROSZER <agroszer.ll at gmail.com> wrote:
>> On 10/15/2013 03:38 PM, Jim Fulton wrote:
>>>
>>> On Tue, Oct 15, 2013 at 3:43 AM, Adam GROSZER <agroszer.ll at gmail.com>
>>> wrote:
>>>>
>>>> On 10/15/2013 12:14 AM, Alex Clark wrote:
>>>>>
>>>>>
>>>>> Adam GROSZER <agroszer.ll <at> gmail.com> writes:
>>>>>
>>>>>
>>>>>> python-distribute.org seems to be down, that breaks zc.buildout v1
>>>>>> bootstrap.py when used with "-d" to use distribute.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Distribute is dead, long live Setuptools (don't use -d anymore)
>>>>
>>>>
>>>>
>>>> Well easier said than done.
>>>> This is windows, 64bit
>>>
>>>
>>> Does setuptools not work with 64-bit windows?
>>
>>
>> 0.6c11 does not work
>
> There are quite a few 0.6c12 releases that were never uploaded to
> PyPI.  Perhaps one of
> those works.  I'd be happy to provide the eggs I have if someone wants
> to try them out.
>
> Maybe PJE knows of they work on 64-bit windows.
>
>
>>
>>
>>>
>>>> On top of that http://downloads.buildout.org/1/bootstrap.py references
>>>> that
>>>> URL.
>>>
>>>
>>> Is there a different URL that should be used? Or do you suggest dropping
>>> distribute support from the buildout 1 bootstrap script?
>>>
>>> Jim
>>>
>>
>> This breaks current projects.
>> We don't have time right now to update to buildout 2.x.
>> A workaround is to patch the v1 bootstrap with:
>>
>> distribute_source =
>> 'https://bitbucket.org/pypa/setuptools/raw/f657df1f1ed46596d236376649c99a470662b4ba/distribute_setup.py'
>>
>> Maybe that should get into http://downloads.buildout.org/1/bootstrap.py
>
> OK, I'll work on that.
>
> Jim
>

Jim, your recent changes to http://downloads.buildout.org/1/bootstrap.py 
pass on our buildbot.

-- 
Best regards,
  Adam GROSZER
--
Quote of the day:
[As an atheist] I never noticed that the very strength of the 
pessimists' case poses us a problem. If the universe is so bad, how on 
earth did human beings ever come to attribute it to the activity of a 
wise and good Creator?
- C.S. Lewis

From p.f.moore at gmail.com  Wed Oct 16 08:33:59 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 16 Oct 2013 07:33:59 +0100
Subject: [Distutils] PEP 453 - virtualenv change to Scripts/bin directory
In-Reply-To: <CADiSq7dq5siF3DJ-ZAy=Uno_yzGML++fjZpHMQ82SRh41F8z=A@mail.gmail.com>
References: <CACac1F_O9Cc3D_8VX8e-GJUjUZTz7GfT79g-4rZhkKG1+cUsqg@mail.gmail.com>
 <nad-F5A2BE.11254315102013@news.gmane.org>
 <CACac1F8BES=nXMp9pqY=Vm+s1bj3O2D=1mJgZjRqbuZBu+AHqw@mail.gmail.com>
 <CACac1F8ynyipnboQSijV=Vr7=ciJ5bZnt601tYBFhOM6+_xzUg@mail.gmail.com>
 <CADiSq7dT654aCjNmB54Qq602B95esKnPLXwMz-hYnC5b-kai2w@mail.gmail.com>
 <CACac1F_rOK-8T7rm-1aFknTiVF2ap9Yb9EEdRfDH=wnnSHgZFg@mail.gmail.com>
 <CACac1F-bgDQkjP104jGJvEdJkhGK97PSfY2LLk371sn-E3XQKw@mail.gmail.com>
 <CADiSq7dq5siF3DJ-ZAy=Uno_yzGML++fjZpHMQ82SRh41F8z=A@mail.gmail.com>
Message-ID: <CACac1F9ZAFxtHQJiHQF1BpZYCCgUwz84OBP4X94GijnAPwFcrw@mail.gmail.com>

On 15 October 2013 23:44, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Also, as a result of (3) existing wininst installers will not install
>> correctly on Python 3.4+ (they'll put their scripts in "Scripts"
>> still). So projects will need to distribute separate installers for
>> Python <= 3.3 and 3.4+ if they contain scripts. (This problem goes
>> away in the long term if everyone switches to wheels, of course).
>
> The installer compatibility issue is enough to convince me to drop the
> directory name change from PEP 453.

Cool. Thanks for your patience over this, and my apologies that it
took me so long to pin down the key part of my discomfort with the
proposal.
Paul

From donald at stufft.io  Thu Oct 17 05:01:55 2013
From: donald at stufft.io (Donald Stufft)
Date: Wed, 16 Oct 2013 23:01:55 -0400
Subject: [Distutils] Deprecate and Block requires/provides
Message-ID: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>

As many are aware distutils has the "requires" field and such. These
fields are designed to list something you *import* not something you
install from PyPI. I believe these fields to be generally useless, supported
by the fact distutils2 deprecated them, and outright user hostile. Users
do not tend to realize that these aren't PyPI distributions and attempt to
use them for that case. This generally "works" in that it will show up on
PyPI but it will fail as soon as a distribution contains a character like -.

So what I would like to do is "remove" these fields. This would consist
of modifying PyPI to return an error code if they are included and hiding
the existing data in the UX. It might at a future time also consist of removing
the data from the DB all together.

What do people think?

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131016/d4250563/attachment.sig>

From vinay_sajip at yahoo.co.uk  Thu Oct 17 11:35:23 2013
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 17 Oct 2013 10:35:23 +0100 (BST)
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
Message-ID: <1382002523.77642.YahooMailBasic@web171406.mail.ir2.yahoo.com>


>  What do people think?
 
+1

Perhaps "Obsoletes", too, if that's used anywhere (Superseded by "Obsoleted-By").

Regards,

Vinay Sajip

From ncoghlan at gmail.com  Thu Oct 17 12:04:36 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 17 Oct 2013 20:04:36 +1000
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
Message-ID: <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>

On 17 Oct 2013 13:02, "Donald Stufft" <donald at stufft.io> wrote:
>
> As many are aware distutils has the "requires" field and such. These
> fields are designed to list something you *import* not something you
> install from PyPI. I believe these fields to be generally useless,
supported
> by the fact distutils2 deprecated them, and outright user hostile. Users
> do not tend to realize that these aren't PyPI distributions and attempt to
> use them for that case. This generally "works" in that it will show up on
> PyPI but it will fail as soon as a distribution contains a character like
-.
>
> So what I would like to do is "remove" these fields. This would consist
> of modifying PyPI to return an error code if they are included and hiding
> the existing data in the UX. It might at a future time also consist of
removing
> the data from the DB all together.
>
> What do people think?

Don't break things when they don't pose an immediate security threat (and
sometimes not even then) :)

For this case, I like the idea of adding a note on projects where these
fields are displayed saying something like "These metadata fields are
obsolete and ignored by installation tools. Use setuptools metadata to
declare dependencies on other PyPI projects." (with an appropriate
hyperlink from "setuptools metadata").

And then maybe another caremad page listing projects that still use the
obsolete fields :)

Cheers,
Nick.

>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
DCFA
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/3eab8be2/attachment.html>

From fuzzyman at gmail.com  Thu Oct 17 12:28:11 2013
From: fuzzyman at gmail.com (Michael Foord)
Date: Thu, 17 Oct 2013 11:28:11 +0100
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
Message-ID: <CAKCKLWxKh49y4Y5aQq--Uy+r5HaEgM_B4-9rB1ReYA-9B3tbRA@mail.gmail.com>

On 17 October 2013 04:01, Donald Stufft <donald at stufft.io> wrote:

> As many are aware distutils has the "requires" field and such. These
> fields are designed to list something you *import* not something you
> install from PyPI. I believe these fields to be generally useless,
> supported
> by the fact distutils2 deprecated them, and outright user hostile. Users
> do not tend to realize that these aren't PyPI distributions and attempt to
> use them for that case. This generally "works" in that it will show up on
> PyPI but it will fail as soon as a distribution contains a character like
> -.
>
> So what I would like to do is "remove" these fields. This would consist
> of modifying PyPI to return an error code if they are included and hiding
> the existing data in the UX. It might at a future time also consist of
> removing
> the data from the DB all together.
>
> What do people think?
>
>

+1 to hiding them in the PyPI UI.

-1 to breaking setup.py upload / register !

Michael


> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
> DCFA
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>


-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/2a99783c/attachment.html>

From donald at stufft.io  Thu Oct 17 12:56:40 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 17 Oct 2013 06:56:40 -0400
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>
Message-ID: <F33C6AA5-383B-4060-8643-CC6F561F3F5F@stufft.io>

Arguably it's already broken. I've had to explain to a number of people that it won't cause their dependencies to install. I think its way more user friendly to tell them up front then to confuse them when it doesn't work or when it appears to work but they get an error from a - 

Packaging is confusing enough without leaving foot guns littered through it. 

> On Oct 17, 2013, at 6:04 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> Don't break things when they don't pose an immediate security threat (and sometimes not even then) :)

From jim at zope.com  Thu Oct 17 13:14:19 2013
From: jim at zope.com (Jim Fulton)
Date: Thu, 17 Oct 2013 07:14:19 -0400
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
Message-ID: <CAPDm-Fi-ER_rnryYiV0cdYFW3ExiP3xd2RLaG-qKkUtNc96MWQ@mail.gmail.com>

On Wed, Oct 16, 2013 at 11:01 PM, Donald Stufft <donald at stufft.io> wrote:
> So what I would like to do is "remove" these fields. This would consist
> of modifying PyPI to return an error code if they are included and hiding
> the existing data in the UX. It might at a future time also consist of removing
> the data from the DB all together.
>
> What do people think?

IIUC, you're proposing to error on uploads of distributions with these
fields set. This wouldn't effect distributions already uploaded and wouldn't
cause (new) breakage for consumers of those distributions.  The only
breakage would be for authors uploading the buggy distributions. These
are the people who could actually address the breakage and who would benefit
from the breakage by finding out that they have a bug in their distributions.
This seems an appropriate form of breakage to me, so +1.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From donald at stufft.io  Thu Oct 17 13:19:25 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 17 Oct 2013 07:19:25 -0400
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CAPDm-Fi-ER_rnryYiV0cdYFW3ExiP3xd2RLaG-qKkUtNc96MWQ@mail.gmail.com>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CAPDm-Fi-ER_rnryYiV0cdYFW3ExiP3xd2RLaG-qKkUtNc96MWQ@mail.gmail.com>
Message-ID: <CE360445-BA75-4181-BFED-01F27FC9A9A3@stufft.io>


On Oct 17, 2013, at 7:14 AM, Jim Fulton <jim at zope.com> wrote:

> On Wed, Oct 16, 2013 at 11:01 PM, Donald Stufft <donald at stufft.io> wrote:
>> So what I would like to do is "remove" these fields. This would consist
>> of modifying PyPI to return an error code if they are included and hiding
>> the existing data in the UX. It might at a future time also consist of removing
>> the data from the DB all together.
>> 
>> What do people think?
> 
> IIUC, you're proposing to error on uploads of distributions with these
> fields set. This wouldn't effect distributions already uploaded and wouldn't
> cause (new) breakage for consumers of those distributions.  The only
> breakage would be for authors uploading the buggy distributions. These
> are the people who could actually address the breakage and who would benefit
> from the breakage by finding out that they have a bug in their distributions.
> This seems an appropriate form of breakage to me, so +1.
> 
> Jim
> 
> -- 
> Jim Fulton
> http://www.linkedin.com/in/jimfulton

This is correct, the only place it would break is for someone uploading
something that used this, which is a signal to them to go remove those
fields. There is a good chance they are sing those fields thinking they
do something they don't, and even if they understand those fields the
fields themselves are more or less useless so losing them isn't a big
issue IMO.

For some numbers, 6% of the projects hosted on PyPI have *ever*
uploaded a release using the requires field. (This does not mean they
are actively using it, just that at least once they did use it). I think that's
a pretty low number of affected users, especially when they can immediately
fix it and I can provide an error message telling them what to do.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/c32e3efe/attachment-0001.sig>

From donald at stufft.io  Thu Oct 17 14:12:08 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 17 Oct 2013 08:12:08 -0400
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CE360445-BA75-4181-BFED-01F27FC9A9A3@stufft.io>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CAPDm-Fi-ER_rnryYiV0cdYFW3ExiP3xd2RLaG-qKkUtNc96MWQ@mail.gmail.com>
 <CE360445-BA75-4181-BFED-01F27FC9A9A3@stufft.io>
Message-ID: <1593C975-1292-487B-8E0F-71E8FA980949@stufft.io>


On Oct 17, 2013, at 7:19 AM, Donald Stufft <donald at stufft.io> wrote:

> For some numbers, 6% of the projects hosted on PyPI have *ever*
> uploaded a release using the requires field. (This does not mean they
> are actively using it, just that at least once they did use it). I think that's
> a pretty low number of affected users, especially when they can immediately
> fix it and I can provide an error message telling them what to do.

More numbers (total number of uses):

requires - 17492
provides - 3876
obsoletes - 176
requires_dist - 120 (Would Break Wheels)
provides_dist - 0
obsoletes_dist - 0
requires_external - 9 (all the same project)

I think we should deprecate/remove requires, provides, obsoletes, 
obsoletes_dist, and requires_externals.

The first 3 were already deprecated with PEP345 so it would just
be a matter of removing PyPI support for them, the other two are
 not/barely used and don't jive with PEP426. Since they are basically
unused I think it would make sense to just kill them now to simplify
DB schema and UX so we don't need to compensate for the case
where they might exist.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/e19419d0/attachment.sig>

From ncoghlan at gmail.com  Thu Oct 17 14:49:57 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 17 Oct 2013 22:49:57 +1000
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CAPDm-Fi-ER_rnryYiV0cdYFW3ExiP3xd2RLaG-qKkUtNc96MWQ@mail.gmail.com>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CAPDm-Fi-ER_rnryYiV0cdYFW3ExiP3xd2RLaG-qKkUtNc96MWQ@mail.gmail.com>
Message-ID: <CADiSq7eNnDc=CL59Z-ACkaSWnJ7OWM1OECWxwQ6QW4M_ssWSJQ@mail.gmail.com>

On 17 October 2013 21:14, Jim Fulton <jim at zope.com> wrote:
> On Wed, Oct 16, 2013 at 11:01 PM, Donald Stufft <donald at stufft.io> wrote:
>> So what I would like to do is "remove" these fields. This would consist
>> of modifying PyPI to return an error code if they are included and hiding
>> the existing data in the UX. It might at a future time also consist of removing
>> the data from the DB all together.
>>
>> What do people think?
>
> IIUC, you're proposing to error on uploads of distributions with these
> fields set. This wouldn't effect distributions already uploaded and wouldn't
> cause (new) breakage for consumers of those distributions.  The only
> breakage would be for authors uploading the buggy distributions. These
> are the people who could actually address the breakage and who would benefit
> from the breakage by finding out that they have a bug in their distributions.
> This seems an appropriate form of breakage to me, so +1.

Except we can't tell if it's a human or a script doing the upload, and
if it's the latter, we just broke somebody's automated release process
without anything even remotely approaching adequate justification.

PyPI is a trusted service provider: we can nudge people towards better
ways of doing things, but something has to pose an imminent threat to
the integrity of the service itself for us to consider breaking
backwards compatibility.

That's why I suggested following the same approach as Donald did for
reducing the amount of external scanning going on: creating a separate
list of all the projects still using the deprecated metadata, and
encouraging users of those projects to get in touch with the
maintainers.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From donald at stufft.io  Thu Oct 17 15:22:02 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 17 Oct 2013 09:22:02 -0400
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CADiSq7eNnDc=CL59Z-ACkaSWnJ7OWM1OECWxwQ6QW4M_ssWSJQ@mail.gmail.com>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CAPDm-Fi-ER_rnryYiV0cdYFW3ExiP3xd2RLaG-qKkUtNc96MWQ@mail.gmail.com>
 <CADiSq7eNnDc=CL59Z-ACkaSWnJ7OWM1OECWxwQ6QW4M_ssWSJQ@mail.gmail.com>
Message-ID: <F7116169-DF4B-45E5-983E-CD8C7EE86E27@stufft.io>


On Oct 17, 2013, at 8:49 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> Except we can't tell if it's a human or a script doing the upload, and
> if it's the latter, we just broke somebody's automated release process
> without anything even remotely approaching adequate justification.

Even if it breaks it, it's for a small number of users, 931 total projects in
2928 releases in the last year have submitted something using the
``requires`` field. Looking through the data for these it appears that most
of them are NOT using it correctly.

I'd be willing to bet money that the people who are using it are just trying
to get dependency information to show up on PyPI, which they can do
now using Wheel + Twine.

It's an attractive nuisance // foot gun, it confuses uses, and by continuing
to allow it we are making packaging more confusing for users.

> 
> PyPI is a trusted service provider: we can nudge people towards better
> ways of doing things, but something has to pose an imminent threat to
> the integrity of the service itself for us to consider breaking
> backwards compatibility.

I'm not sure I agree with this. PyPI isn't versioned so unless we deprecate
and remove things it's going to just get cruftier and cruftier. For instance
An API might end up with 3 or more different concepts of "obsoletes"
because we were unwilling to remove older metadata that is no longer
in use. That is actively making the entire system harder to use.

That's not to say we should just willy nilly remove stuff, but this seems like
a prime candidate for removal as it is a common pain point that i've seen
new users trying to package things stumble on.

> 
> That's why I suggested following the same approach as Donald did for
> reducing the amount of external scanning going on: creating a separate
> list of all the projects still using the deprecated metadata, and
> encouraging users of those projects to get in touch with the
> maintainers.
> 


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/b7fbed64/attachment.sig>

From ncoghlan at gmail.com  Thu Oct 17 15:53:03 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 17 Oct 2013 23:53:03 +1000
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <F7116169-DF4B-45E5-983E-CD8C7EE86E27@stufft.io>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CAPDm-Fi-ER_rnryYiV0cdYFW3ExiP3xd2RLaG-qKkUtNc96MWQ@mail.gmail.com>
 <CADiSq7eNnDc=CL59Z-ACkaSWnJ7OWM1OECWxwQ6QW4M_ssWSJQ@mail.gmail.com>
 <F7116169-DF4B-45E5-983E-CD8C7EE86E27@stufft.io>
Message-ID: <CADiSq7cL4VL-ikppSke223rsLXBsZfL8=dGQTz2X3uSfrLcx7A@mail.gmail.com>

On 17 October 2013 23:22, Donald Stufft <donald at stufft.io> wrote:
>
> On Oct 17, 2013, at 8:49 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>> Except we can't tell if it's a human or a script doing the upload, and
>> if it's the latter, we just broke somebody's automated release process
>> without anything even remotely approaching adequate justification.
>
> Even if it breaks it, it's for a small number of users, 931 total projects in
> 2928 releases in the last year have submitted something using the
> ``requires`` field. Looking through the data for these it appears that most
> of them are NOT using it correctly.

"You're not allowed to upload to PyPI any more until you fix this
thing that we were previously OK with" is a lousy way to educate them.

> I'd be willing to bet money that the people who are using it are just trying
> to get dependency information to show up on PyPI, which they can do
> now using Wheel + Twine.
>
> It's an attractive nuisance // foot gun, it confuses uses, and by continuing
> to allow it we are making packaging more confusing for users.

I'm fine with turning it off for projects that don't already use this
data. I'm *not* fine with breaking something that was working for
people.

>> PyPI is a trusted service provider: we can nudge people towards better
>> ways of doing things, but something has to pose an imminent threat to
>> the integrity of the service itself for us to consider breaking
>> backwards compatibility.
>
> I'm not sure I agree with this. PyPI isn't versioned so unless we deprecate
> and remove things it's going to just get cruftier and cruftier. For instance
> An API might end up with 3 or more different concepts of "obsoletes"
> because we were unwilling to remove older metadata that is no longer
> in use. That is actively making the entire system harder to use.

I'm not opposed to deprecating and removing it. I'm opposed to just
turning it off cold for projects where it was previously working.

> That's not to say we should just willy nilly remove stuff, but this seems like
> a prime candidate for removal as it is a common pain point that i've seen
> new users trying to package things stumble on.

Yes, but breaking these users' uploads is not the way to do that.
That's an incredibly user hostile step to take, and the problem
doesn't even come close to justifying breaking even a
not-really-working process (since you can work around the current
breakage by manually installing the dependencies - you can't work
around an inability to upload without making changes to your code).

If you really wanted to push them towards fixing their releases, you
could always have PyPI send them an email saying something like:

"Hi, an automated scan of PyPI has shown that you're currently
uploading metadata that uses the obsolete module dependency fields
defined by distutils. These fields aren't actually checked by
automated installation tools like pip, so if you're attempting to
indicate that your project depends on other PyPI projects, this
mechanism doesn't actually work to achieve that.

At some point in the future, PyPI may disallow use of these fields
entirely, so if you'd like to declare your dependencies in a way that
automated tools can process, you may want to look at using the
dependency declaration features in setuptools rather than using plain
distutils".

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From donald at stufft.io  Thu Oct 17 15:59:58 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 17 Oct 2013 09:59:58 -0400
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CADiSq7cL4VL-ikppSke223rsLXBsZfL8=dGQTz2X3uSfrLcx7A@mail.gmail.com>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CAPDm-Fi-ER_rnryYiV0cdYFW3ExiP3xd2RLaG-qKkUtNc96MWQ@mail.gmail.com>
 <CADiSq7eNnDc=CL59Z-ACkaSWnJ7OWM1OECWxwQ6QW4M_ssWSJQ@mail.gmail.com>
 <F7116169-DF4B-45E5-983E-CD8C7EE86E27@stufft.io>
 <CADiSq7cL4VL-ikppSke223rsLXBsZfL8=dGQTz2X3uSfrLcx7A@mail.gmail.com>
Message-ID: <2EA1E6C1-95BA-4813-B8D5-F3A9DFFB0B9E@stufft.io>


On Oct 17, 2013, at 9:53 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 17 October 2013 23:22, Donald Stufft <donald at stufft.io> wrote:
>> 
>> On Oct 17, 2013, at 8:49 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> 
>>> Except we can't tell if it's a human or a script doing the upload, and
>>> if it's the latter, we just broke somebody's automated release process
>>> without anything even remotely approaching adequate justification.
>> 
>> Even if it breaks it, it's for a small number of users, 931 total projects in
>> 2928 releases in the last year have submitted something using the
>> ``requires`` field. Looking through the data for these it appears that most
>> of them are NOT using it correctly.
> 
> "You're not allowed to upload to PyPI any more until you fix this
> thing that we were previously OK with" is a lousy way to educate them.
> 
>> I'd be willing to bet money that the people who are using it are just trying
>> to get dependency information to show up on PyPI, which they can do
>> now using Wheel + Twine.
>> 
>> It's an attractive nuisance // foot gun, it confuses uses, and by continuing
>> to allow it we are making packaging more confusing for users.
> 
> I'm fine with turning it off for projects that don't already use this
> data. I'm *not* fine with breaking something that was working for
> people.
> 
>>> PyPI is a trusted service provider: we can nudge people towards better
>>> ways of doing things, but something has to pose an imminent threat to
>>> the integrity of the service itself for us to consider breaking
>>> backwards compatibility.
>> 
>> I'm not sure I agree with this. PyPI isn't versioned so unless we deprecate
>> and remove things it's going to just get cruftier and cruftier. For instance
>> An API might end up with 3 or more different concepts of "obsoletes"
>> because we were unwilling to remove older metadata that is no longer
>> in use. That is actively making the entire system harder to use.
> 
> I'm not opposed to deprecating and removing it. I'm opposed to just
> turning it off cold for projects where it was previously working.

It's been deprecated since 2010 (and the PEP that deprecated it was created
in 2005).

> 
>> That's not to say we should just willy nilly remove stuff, but this seems like
>> a prime candidate for removal as it is a common pain point that i've seen
>> new users trying to package things stumble on.
> 
> Yes, but breaking these users' uploads is not the way to do that.
> That's an incredibly user hostile step to take, and the problem
> doesn't even come close to justifying breaking even a
> not-really-working process (since you can work around the current
> breakage by manually installing the dependencies - you can't work
> around an inability to upload without making changes to your code).
> 
> If you really wanted to push them towards fixing their releases, you
> could always have PyPI send them an email saying something like:
> 
> "Hi, an automated scan of PyPI has shown that you're currently
> uploading metadata that uses the obsolete module dependency fields
> defined by distutils. These fields aren't actually checked by
> automated installation tools like pip, so if you're attempting to
> indicate that your project depends on other PyPI projects, this
> mechanism doesn't actually work to achieve that.
> 
> At some point in the future, PyPI may disallow use of these fields
> entirely, so if you'd like to declare your dependencies in a way that
> automated tools can process, you may want to look at using the
> dependency declaration features in setuptools rather than using plain
> distutils".

This could be part of the deprecation process, but unless there's a plan
in place to deprecate them I don't know how useful this email would be.
It's a reactive warning to something that confuses people while they are
creating a package. In other words by the time the email goes out they've
already been confused.

Perhaps this needs a PEP to specify a warning period with emails and
then ultimately removal.

> 
> Cheers,
> Nick.
> 
> 
> -- 
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/f5debf46/attachment-0001.sig>

From erik.m.bray at gmail.com  Thu Oct 17 16:24:00 2013
From: erik.m.bray at gmail.com (Erik Bray)
Date: Thu, 17 Oct 2013 10:24:00 -0400
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CADiSq7cL4VL-ikppSke223rsLXBsZfL8=dGQTz2X3uSfrLcx7A@mail.gmail.com>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CAPDm-Fi-ER_rnryYiV0cdYFW3ExiP3xd2RLaG-qKkUtNc96MWQ@mail.gmail.com>
 <CADiSq7eNnDc=CL59Z-ACkaSWnJ7OWM1OECWxwQ6QW4M_ssWSJQ@mail.gmail.com>
 <F7116169-DF4B-45E5-983E-CD8C7EE86E27@stufft.io>
 <CADiSq7cL4VL-ikppSke223rsLXBsZfL8=dGQTz2X3uSfrLcx7A@mail.gmail.com>
Message-ID: <CAOTD34a8xKq-pkxMp89eL4oaAzgGCKPepZMQt1NkLMZB=DMa+g@mail.gmail.com>

On Thu, Oct 17, 2013 at 9:53 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Yes, but breaking these users' uploads is not the way to do that.
> That's an incredibly user hostile step to take, and the problem
> doesn't even come close to justifying breaking even a
> not-really-working process (since you can work around the current
> breakage by manually installing the dependencies - you can't work
> around an inability to upload without making changes to your code).

+1  I can imagine that from many users' perspectives "you're doing it
wrong" isn't even true.  The docs for distutils *still* read:

"Dependencies on other Python modules and packages can be specified by
supplying the requires keyword argument to setup()"

and

"A package can declare that it obsoletes other packages using the
obsoletes keyword argument"

and nowhere does it read "but these keywords are broken and obsolete
as of <some obscure PEP> and you're wrong to have used them".

> If you really wanted to push them towards fixing their releases, you
> could always have PyPI send them an email saying something like:
>
> "Hi, an automated scan of PyPI has shown that you're currently
> uploading metadata that uses the obsolete module dependency fields
> defined by distutils. These fields aren't actually checked by
> automated installation tools like pip, so if you're attempting to
> indicate that your project depends on other PyPI projects, this
> mechanism doesn't actually work to achieve that.
>
> At some point in the future, PyPI may disallow use of these fields
> entirely, so if you'd like to declare your dependencies in a way that
> automated tools can process, you may want to look at using the
> dependency declaration features in setuptools rather than using plain
> distutils".

I was just going to ask--why not at least try to e-mail the
maintainers of those packages?  If they're unreachable to begin with
then I'm a little less concerned.  But at least give those who might
care a direct heads up about where things are going.  Then when the
inevitable complaints do come in at least you've covered your ass :)

Best,

Erik

From marius at pov.lt  Thu Oct 17 16:38:57 2013
From: marius at pov.lt (Marius Gedminas)
Date: Thu, 17 Oct 2013 17:38:57 +0300
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <2EA1E6C1-95BA-4813-B8D5-F3A9DFFB0B9E@stufft.io>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CAPDm-Fi-ER_rnryYiV0cdYFW3ExiP3xd2RLaG-qKkUtNc96MWQ@mail.gmail.com>
 <CADiSq7eNnDc=CL59Z-ACkaSWnJ7OWM1OECWxwQ6QW4M_ssWSJQ@mail.gmail.com>
 <F7116169-DF4B-45E5-983E-CD8C7EE86E27@stufft.io>
 <CADiSq7cL4VL-ikppSke223rsLXBsZfL8=dGQTz2X3uSfrLcx7A@mail.gmail.com>
 <2EA1E6C1-95BA-4813-B8D5-F3A9DFFB0B9E@stufft.io>
Message-ID: <20131017143856.GA14465@fridge.pov.lt>

On Thu, Oct 17, 2013 at 09:59:58AM -0400, Donald Stufft wrote:
> On Oct 17, 2013, at 9:53 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > Yes, but breaking these users' uploads is not the way to do that.
> > That's an incredibly user hostile step to take, and the problem
> > doesn't even come close to justifying breaking even a
> > not-really-working process (since you can work around the current
> > breakage by manually installing the dependencies - you can't work
> > around an inability to upload without making changes to your code).
> > 
> > If you really wanted to push them towards fixing their releases, you
> > could always have PyPI send them an email saying something like:
>
...
> 
> This could be part of the deprecation process, but unless there's a plan
> in place to deprecate them I don't know how useful this email would be.
> It's a reactive warning to something that confuses people while they are
> creating a package. In other words by the time the email goes out they've
> already been confused.

Is it possible to make 'python setup.py sdist upload' emit a warning
message about using these deprecated fields, without rejecting the
upload?

Marius Gedminas
-- 
Q:  How many IBM CPU's does it take to execute a job?
A:  Four; three to hold it down, and one to rip its head off.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/4d7cb03b/attachment.sig>

From donald at stufft.io  Thu Oct 17 16:40:14 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 17 Oct 2013 10:40:14 -0400
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <20131017143856.GA14465@fridge.pov.lt>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CAPDm-Fi-ER_rnryYiV0cdYFW3ExiP3xd2RLaG-qKkUtNc96MWQ@mail.gmail.com>
 <CADiSq7eNnDc=CL59Z-ACkaSWnJ7OWM1OECWxwQ6QW4M_ssWSJQ@mail.gmail.com>
 <F7116169-DF4B-45E5-983E-CD8C7EE86E27@stufft.io>
 <CADiSq7cL4VL-ikppSke223rsLXBsZfL8=dGQTz2X3uSfrLcx7A@mail.gmail.com>
 <2EA1E6C1-95BA-4813-B8D5-F3A9DFFB0B9E@stufft.io>
 <20131017143856.GA14465@fridge.pov.lt>
Message-ID: <F98AE42D-2280-441D-B292-764A3A401D35@stufft.io>


On Oct 17, 2013, at 10:38 AM, Marius Gedminas <marius at pov.lt> wrote:

> On Thu, Oct 17, 2013 at 09:59:58AM -0400, Donald Stufft wrote:
>> On Oct 17, 2013, at 9:53 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> Yes, but breaking these users' uploads is not the way to do that.
>>> That's an incredibly user hostile step to take, and the problem
>>> doesn't even come close to justifying breaking even a
>>> not-really-working process (since you can work around the current
>>> breakage by manually installing the dependencies - you can't work
>>> around an inability to upload without making changes to your code).
>>> 
>>> If you really wanted to push them towards fixing their releases, you
>>> could always have PyPI send them an email saying something like:
>> 
> ...
>> 
>> This could be part of the deprecation process, but unless there's a plan
>> in place to deprecate them I don't know how useful this email would be.
>> It's a reactive warning to something that confuses people while they are
>> creating a package. In other words by the time the email goes out they've
>> already been confused.
> 
> Is it possible to make 'python setup.py sdist upload' emit a warning
> message about using these deprecated fields, without rejecting the
> upload?

I don't believe so (but don't quote me on that), besides the obvious
of patching distutils.

> 
> Marius Gedminas
> -- 
> Q:  How many IBM CPU's does it take to execute a job?
> A:  Four; three to hold it down, and one to rip its head off.
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/57eba78d/attachment.sig>

From fuzzyman at gmail.com  Thu Oct 17 17:20:49 2013
From: fuzzyman at gmail.com (Michael Foord)
Date: Thu, 17 Oct 2013 16:20:49 +0100
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <F33C6AA5-383B-4060-8643-CC6F561F3F5F@stufft.io>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>
 <F33C6AA5-383B-4060-8643-CC6F561F3F5F@stufft.io>
Message-ID: <CAKCKLWxCCx3QXGknicq4dgp2Lf-xrbWUGxStd0_ZXw5dD6j1Lg@mail.gmail.com>

On 17 October 2013 11:56, Donald Stufft <donald at stufft.io> wrote:

> Arguably it's already broken. I've had to explain to a number of people
> that it won't cause their dependencies to install. I think its way more
> user friendly to tell them up front then to confuse them when it doesn't
> work or when it appears to work but they get an error from a -
>
>
You're proposing replacing "arguably broken" (by some definition) with
"actually broken" (no longer works). That's not an acceptable trade-off.




> Packaging is confusing enough without leaving foot guns littered through
> it.
>
>
Packaging is confusing enough without breaking people's stuff! (Replacing a
foot gun with actually shooting people's feet off if we're going to stick
with the metaphor...)

Michael



> > On Oct 17, 2013, at 6:04 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> >
> > Don't break things when they don't pose an immediate security threat
> (and sometimes not even then) :)
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>



-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/64a15df2/attachment.html>

From jim at zope.com  Thu Oct 17 17:23:38 2013
From: jim at zope.com (Jim Fulton)
Date: Thu, 17 Oct 2013 11:23:38 -0400
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CAKCKLWxCCx3QXGknicq4dgp2Lf-xrbWUGxStd0_ZXw5dD6j1Lg@mail.gmail.com>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>
 <F33C6AA5-383B-4060-8643-CC6F561F3F5F@stufft.io>
 <CAKCKLWxCCx3QXGknicq4dgp2Lf-xrbWUGxStd0_ZXw5dD6j1Lg@mail.gmail.com>
Message-ID: <CAPDm-FiD8PhkzEZ3PvuHK8KLWkZJ0=6n-RkMUTmoUYyvUG5SiA@mail.gmail.com>

On Thu, Oct 17, 2013 at 11:20 AM, Michael Foord <fuzzyman at gmail.com> wrote:
>
>
>
> On 17 October 2013 11:56, Donald Stufft <donald at stufft.io> wrote:
>>
>> Arguably it's already broken. I've had to explain to a number of people
>> that it won't cause their dependencies to install. I think its way more user
>> friendly to tell them up front then to confuse them when it doesn't work or
>> when it appears to work but they get an error from a -
>>
>
> You're proposing replacing "arguably broken" (by some definition)

Is there any reason to think that it ever actually worked in any way?

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From donald at stufft.io  Thu Oct 17 17:26:04 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 17 Oct 2013 11:26:04 -0400
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CAPDm-FiD8PhkzEZ3PvuHK8KLWkZJ0=6n-RkMUTmoUYyvUG5SiA@mail.gmail.com>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>
 <F33C6AA5-383B-4060-8643-CC6F561F3F5F@stufft.io>
 <CAKCKLWxCCx3QXGknicq4dgp2Lf-xrbWUGxStd0_ZXw5dD6j1Lg@mail.gmail.com>
 <CAPDm-FiD8PhkzEZ3PvuHK8KLWkZJ0=6n-RkMUTmoUYyvUG5SiA@mail.gmail.com>
Message-ID: <F0809EBB-0029-433B-A97A-ADAF08A46A07@stufft.io>


On Oct 17, 2013, at 11:23 AM, Jim Fulton <jim at zope.com> wrote:

> On Thu, Oct 17, 2013 at 11:20 AM, Michael Foord <fuzzyman at gmail.com> wrote:
>> 
>> 
>> 
>> On 17 October 2013 11:56, Donald Stufft <donald at stufft.io> wrote:
>>> 
>>> Arguably it's already broken. I've had to explain to a number of people
>>> that it won't cause their dependencies to install. I think its way more user
>>> friendly to tell them up front then to confuse them when it doesn't work or
>>> when it appears to work but they get an error from a -
>>> 
>> 
>> You're proposing replacing "arguably broken" (by some definition)
> 
> Is there any reason to think that it ever actually worked in any way?

Depends what you mean by "worked", it does nothing except tell users
what they need available to ``import``. It doesn't tell them how to get that
or what provides it. As far as this piece of metadata is concerned requires
"xml" can be satisfied by ``touch xml.py``.

So it "works" in that it does what it was originally intended to do, it's just that
what it was originally intended to do is backwards and useless.

> 
> Jim
> 
> -- 
> Jim Fulton
> http://www.linkedin.com/in/jimfulton


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/c0342763/attachment.sig>

From jim at zope.com  Thu Oct 17 17:29:39 2013
From: jim at zope.com (Jim Fulton)
Date: Thu, 17 Oct 2013 11:29:39 -0400
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <F0809EBB-0029-433B-A97A-ADAF08A46A07@stufft.io>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>
 <F33C6AA5-383B-4060-8643-CC6F561F3F5F@stufft.io>
 <CAKCKLWxCCx3QXGknicq4dgp2Lf-xrbWUGxStd0_ZXw5dD6j1Lg@mail.gmail.com>
 <CAPDm-FiD8PhkzEZ3PvuHK8KLWkZJ0=6n-RkMUTmoUYyvUG5SiA@mail.gmail.com>
 <F0809EBB-0029-433B-A97A-ADAF08A46A07@stufft.io>
Message-ID: <CAPDm-Fj+65D_X99rgWVrvAbf7B6brMpmoKcjtNvzmx8gkXzM5A@mail.gmail.com>

On Thu, Oct 17, 2013 at 11:26 AM, Donald Stufft <donald at stufft.io> wrote:
>
> On Oct 17, 2013, at 11:23 AM, Jim Fulton <jim at zope.com> wrote:
>
>> On Thu, Oct 17, 2013 at 11:20 AM, Michael Foord <fuzzyman at gmail.com> wrote:
>>>
>>>
>>>
>>> On 17 October 2013 11:56, Donald Stufft <donald at stufft.io> wrote:
>>>>
>>>> Arguably it's already broken. I've had to explain to a number of people
>>>> that it won't cause their dependencies to install. I think its way more user
>>>> friendly to tell them up front then to confuse them when it doesn't work or
>>>> when it appears to work but they get an error from a -
>>>>
>>>
>>> You're proposing replacing "arguably broken" (by some definition)
>>
>> Is there any reason to think that it ever actually worked in any way?
>
> Depends what you mean by "worked", it does nothing except tell users
> what they need available to ``import``. It doesn't tell them how to get that
> or what provides it. As far as this piece of metadata is concerned requires
> "xml" can be satisfied by ``touch xml.py``.
>
> So it "works" in that it does what it was originally intended to do, it's just that
> what it was originally intended to do is backwards and useless.

I'm 92% sure it was intended to support automated dependency management,
but then setuptools went in a different (better) direction.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton

From donald at stufft.io  Thu Oct 17 17:30:20 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 17 Oct 2013 11:30:20 -0400
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CAKCKLWxCCx3QXGknicq4dgp2Lf-xrbWUGxStd0_ZXw5dD6j1Lg@mail.gmail.com>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>
 <F33C6AA5-383B-4060-8643-CC6F561F3F5F@stufft.io>
 <CAKCKLWxCCx3QXGknicq4dgp2Lf-xrbWUGxStd0_ZXw5dD6j1Lg@mail.gmail.com>
Message-ID: <8CEBC9E0-DFC5-4682-81E5-D21A808D78C4@stufft.io>


On Oct 17, 2013, at 11:20 AM, Michael Foord <fuzzyman at gmail.com> wrote:

> 
> 
> 
> On 17 October 2013 11:56, Donald Stufft <donald at stufft.io> wrote:
> Arguably it's already broken. I've had to explain to a number of people that it won't cause their dependencies to install. I think its way more user friendly to tell them up front then to confuse them when it doesn't work or when it appears to work but they get an error from a -
> 
> 
> You're proposing replacing "arguably broken" (by some definition) with "actually broken" (no longer works). That's not an acceptable trade-off.

I'm replacing Arguably broken for a small percentage of people with actually broken and easily fixed for a small percentage of people with no longer confusing for a large number of people. Of those people it would turn into "actually broken" a number of them are using it in a broken fashion and it's simply waiting for them
to try and use it with a name that doesn't match an importable module for them to have it actually break for them regardless.

> 
> 
>  
> Packaging is confusing enough without leaving foot guns littered through it.
> 
> 
> Packaging is confusing enough without breaking people's stuff! (Replacing a foot gun with actually shooting people's feet off if we're going to stick with the metaphor?)

As I said above, breaking it for a small number of people to simplify it for the larger group. By this rationale Python 3 should never have happened. Replacing Packaging with "Unicode".

> 
> Michael
> 
>  
> > On Oct 17, 2013, at 6:04 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> >
> > Don't break things when they don't pose an immediate security threat (and sometimes not even then) :)
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
> 
> 
> 
> -- 
> http://www.voidspace.org.uk/
> 
> May you do good and not evil
> May you find forgiveness for yourself and forgive others
> 
> May you share freely, never taking more than you give.
> -- the sqlite blessing http://www.sqlite.org/different.html


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/442e11f2/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/442e11f2/attachment.sig>

From donald at stufft.io  Thu Oct 17 17:30:53 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 17 Oct 2013 11:30:53 -0400
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CAPDm-Fj+65D_X99rgWVrvAbf7B6brMpmoKcjtNvzmx8gkXzM5A@mail.gmail.com>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>
 <F33C6AA5-383B-4060-8643-CC6F561F3F5F@stufft.io>
 <CAKCKLWxCCx3QXGknicq4dgp2Lf-xrbWUGxStd0_ZXw5dD6j1Lg@mail.gmail.com>
 <CAPDm-FiD8PhkzEZ3PvuHK8KLWkZJ0=6n-RkMUTmoUYyvUG5SiA@mail.gmail.com>
 <F0809EBB-0029-433B-A97A-ADAF08A46A07@stufft.io>
 <CAPDm-Fj+65D_X99rgWVrvAbf7B6brMpmoKcjtNvzmx8gkXzM5A@mail.gmail.com>
Message-ID: <49A73BF9-7706-4518-A516-95B6BEFFA8BA@stufft.io>


On Oct 17, 2013, at 11:29 AM, Jim Fulton <jim at zope.com> wrote:

> I'm 92% sure it was intended to support automated dependency management,
> but then setuptools went in a different (better) direction.

Hmm, that's entirely possible! I was around back then so i'm taking the intent from
the PEP.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/a9770ed0/attachment-0001.sig>

From ncoghlan at gmail.com  Thu Oct 17 17:36:44 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Oct 2013 01:36:44 +1000
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CAOTD34a8xKq-pkxMp89eL4oaAzgGCKPepZMQt1NkLMZB=DMa+g@mail.gmail.com>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CAPDm-Fi-ER_rnryYiV0cdYFW3ExiP3xd2RLaG-qKkUtNc96MWQ@mail.gmail.com>
 <CADiSq7eNnDc=CL59Z-ACkaSWnJ7OWM1OECWxwQ6QW4M_ssWSJQ@mail.gmail.com>
 <F7116169-DF4B-45E5-983E-CD8C7EE86E27@stufft.io>
 <CADiSq7cL4VL-ikppSke223rsLXBsZfL8=dGQTz2X3uSfrLcx7A@mail.gmail.com>
 <CAOTD34a8xKq-pkxMp89eL4oaAzgGCKPepZMQt1NkLMZB=DMa+g@mail.gmail.com>
Message-ID: <CADiSq7fFZhwUEfGmEvOgOZc8y=OvZGNSSnXk38nmsn+gaoh6MA@mail.gmail.com>

On 18 October 2013 00:24, Erik Bray <erik.m.bray at gmail.com> wrote:
> On Thu, Oct 17, 2013 at 9:53 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Yes, but breaking these users' uploads is not the way to do that.
>> That's an incredibly user hostile step to take, and the problem
>> doesn't even come close to justifying breaking even a
>> not-really-working process (since you can work around the current
>> breakage by manually installing the dependencies - you can't work
>> around an inability to upload without making changes to your code).
>
> +1  I can imagine that from many users' perspectives "you're doing it
> wrong" isn't even true.  The docs for distutils *still* read:
>
> "Dependencies on other Python modules and packages can be specified by
> supplying the requires keyword argument to setup()"
>
> and
>
> "A package can declare that it obsoletes other packages using the
> obsoletes keyword argument"
>
> and nowhere does it read "but these keywords are broken and obsolete
> as of <some obscure PEP> and you're wrong to have used them".

Indeed - the metadata PEPs really aren't aimed at end users. Until
something is marked as deprecated in the CPython docs, and actually
deprecated in the standard library, it isn't really deprecated :)

We can at least do a documented deprecation in the CPython docs when
we're implementing the PEP 453 documentation updates (I'll hopefully
get the Windows path changes in this weekend sometime, and after that
it should finally be ready for MvL's formal pronouncement)

Cheers,
Nick.

>
>> If you really wanted to push them towards fixing their releases, you
>> could always have PyPI send them an email saying something like:
>>
>> "Hi, an automated scan of PyPI has shown that you're currently
>> uploading metadata that uses the obsolete module dependency fields
>> defined by distutils. These fields aren't actually checked by
>> automated installation tools like pip, so if you're attempting to
>> indicate that your project depends on other PyPI projects, this
>> mechanism doesn't actually work to achieve that.
>>
>> At some point in the future, PyPI may disallow use of these fields
>> entirely, so if you'd like to declare your dependencies in a way that
>> automated tools can process, you may want to look at using the
>> dependency declaration features in setuptools rather than using plain
>> distutils".
>
> I was just going to ask--why not at least try to e-mail the
> maintainers of those packages?  If they're unreachable to begin with
> then I'm a little less concerned.  But at least give those who might
> care a direct heads up about where things are going.  Then when the
> inevitable complaints do come in at least you've covered your ass :)
>
> Best,
>
> Erik



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From donald at stufft.io  Thu Oct 17 17:45:51 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 17 Oct 2013 11:45:51 -0400
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CADiSq7fFZhwUEfGmEvOgOZc8y=OvZGNSSnXk38nmsn+gaoh6MA@mail.gmail.com>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CAPDm-Fi-ER_rnryYiV0cdYFW3ExiP3xd2RLaG-qKkUtNc96MWQ@mail.gmail.com>
 <CADiSq7eNnDc=CL59Z-ACkaSWnJ7OWM1OECWxwQ6QW4M_ssWSJQ@mail.gmail.com>
 <F7116169-DF4B-45E5-983E-CD8C7EE86E27@stufft.io>
 <CADiSq7cL4VL-ikppSke223rsLXBsZfL8=dGQTz2X3uSfrLcx7A@mail.gmail.com>
 <CAOTD34a8xKq-pkxMp89eL4oaAzgGCKPepZMQt1NkLMZB=DMa+g@mail.gmail.com>
 <CADiSq7fFZhwUEfGmEvOgOZc8y=OvZGNSSnXk38nmsn+gaoh6MA@mail.gmail.com>
Message-ID: <84C62C21-F51B-4F80-A212-F47D9D3BBCC1@stufft.io>


On Oct 17, 2013, at 11:36 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> Indeed - the metadata PEPs really aren't aimed at end users. Until
> something is marked as deprecated in the CPython docs, and actually
> deprecated in the standard library, it isn't really deprecated :)
> 
> We can at least do a documented deprecation in the CPython docs when
> we're implementing the PEP 453 documentation updates (I'll hopefully
> get the Windows path changes in this weekend sometime, and after that
> it should finally be ready for MvL's formal pronouncement)

Ironically the documentation shows how confusing it is because for someone
new coming into packaging that documentation reads like this is what you
put in in order to depend on other things. I would challenge anyone to write
docs that aren't confusing that actually explains what requires actually does.

So again I ask what I need to do to deprecate and remove requires, provides, 
obsoletes. Do I need to make a PEP? Do I need to make a bug tracker issue
and a patch? What's the path forward.

Can we at least agree that these can removed?
  obsoletes_dist - Never been used, never been implemented besides in Distutils2
  requires_external - Used 9 times by 1 project, never been implemented outside of Distutils2
  requires_python - Used 61 times, never been implemented outside of Distutils2

These either have a different method of supporting them in PEP426 (and
thus keeping around two ways of doing the same thing is confusing,
especially when it was never formally implemented) or were omitted
completely.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/2934e8fb/attachment.sig>

From fuzzyman at gmail.com  Thu Oct 17 17:49:40 2013
From: fuzzyman at gmail.com (Michael Foord)
Date: Thu, 17 Oct 2013 16:49:40 +0100
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CAPDm-FiD8PhkzEZ3PvuHK8KLWkZJ0=6n-RkMUTmoUYyvUG5SiA@mail.gmail.com>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>
 <F33C6AA5-383B-4060-8643-CC6F561F3F5F@stufft.io>
 <CAKCKLWxCCx3QXGknicq4dgp2Lf-xrbWUGxStd0_ZXw5dD6j1Lg@mail.gmail.com>
 <CAPDm-FiD8PhkzEZ3PvuHK8KLWkZJ0=6n-RkMUTmoUYyvUG5SiA@mail.gmail.com>
Message-ID: <CAKCKLWwpsuAjOXQRteNxGvNrn9Am5OG_=H0CqXrJDVvr5f0r7Q@mail.gmail.com>

On 17 October 2013 16:23, Jim Fulton <jim at zope.com> wrote:

> On Thu, Oct 17, 2013 at 11:20 AM, Michael Foord <fuzzyman at gmail.com>
> wrote:
> >
> >
> >
> > On 17 October 2013 11:56, Donald Stufft <donald at stufft.io> wrote:
> >>
> >> Arguably it's already broken. I've had to explain to a number of people
> >> that it won't cause their dependencies to install. I think its way more
> user
> >> friendly to tell them up front then to confuse them when it doesn't
> work or
> >> when it appears to work but they get an error from a -
> >>
> >
> > You're proposing replacing "arguably broken" (by some definition)
>
> Is there any reason to think that it ever actually worked in any way?
>
>

Package upload certainly worked, and that is what is going to be broken.

Michael


> Jim
>
> --
> Jim Fulton
> http://www.linkedin.com/in/jimfulton
>



-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/31902b18/attachment.html>

From donald at stufft.io  Thu Oct 17 17:53:46 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 17 Oct 2013 11:53:46 -0400
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CAKCKLWwpsuAjOXQRteNxGvNrn9Am5OG_=H0CqXrJDVvr5f0r7Q@mail.gmail.com>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>
 <F33C6AA5-383B-4060-8643-CC6F561F3F5F@stufft.io>
 <CAKCKLWxCCx3QXGknicq4dgp2Lf-xrbWUGxStd0_ZXw5dD6j1Lg@mail.gmail.com>
 <CAPDm-FiD8PhkzEZ3PvuHK8KLWkZJ0=6n-RkMUTmoUYyvUG5SiA@mail.gmail.com>
 <CAKCKLWwpsuAjOXQRteNxGvNrn9Am5OG_=H0CqXrJDVvr5f0r7Q@mail.gmail.com>
Message-ID: <BAC77324-A849-4D0E-AAA1-96A9995310B0@stufft.io>


On Oct 17, 2013, at 11:49 AM, Michael Foord <fuzzyman at gmail.com> wrote:

> Package upload certainly worked, and that is what is going to be broken.

So would you be ok with deprecating and removing to equal "this metadata silently
gets sent to /dev/null" in order to not break uploads for what would have affected
roughly 4% of the total new releases on PyPI in 2013.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/b6a05544/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/b6a05544/attachment-0001.sig>

From oscar.j.benjamin at gmail.com  Thu Oct 17 18:03:08 2013
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Thu, 17 Oct 2013 17:03:08 +0100
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <BAC77324-A849-4D0E-AAA1-96A9995310B0@stufft.io>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>
 <F33C6AA5-383B-4060-8643-CC6F561F3F5F@stufft.io>
 <CAKCKLWxCCx3QXGknicq4dgp2Lf-xrbWUGxStd0_ZXw5dD6j1Lg@mail.gmail.com>
 <CAPDm-FiD8PhkzEZ3PvuHK8KLWkZJ0=6n-RkMUTmoUYyvUG5SiA@mail.gmail.com>
 <CAKCKLWwpsuAjOXQRteNxGvNrn9Am5OG_=H0CqXrJDVvr5f0r7Q@mail.gmail.com>
 <BAC77324-A849-4D0E-AAA1-96A9995310B0@stufft.io>
Message-ID: <CAHVvXxS7Z+2ygi1pd1YMLxOmMHULhuGMPspWNvLFD6fdSjGwPA@mail.gmail.com>

On 17 October 2013 16:53, Donald Stufft <donald at stufft.io> wrote:
>
> On Oct 17, 2013, at 11:49 AM, Michael Foord <fuzzyman at gmail.com> wrote:
>
> Package upload certainly worked, and that is what is going to be broken.
>
>
> So would you be ok with deprecating and removing to equal "this metadata
> silently
> gets sent to /dev/null" in order to not break uploads for what would have
> affected
> roughly 4% of the total new releases on PyPI in 2013.

What about emitting a warning on upload/download for deprecated
metadata and a warning on the PyPI page for the distribution?

I don't know whether it's possible to implement that server side or if
it would only apply to newer versions of distutils/setuptools etc. but
it would give people who are still uploading sdists with this metadata
a chance to make the fix in their own time rather than suddenly being
unable to release updates.


Cheers,
Oscar

From ncoghlan at gmail.com  Thu Oct 17 18:03:49 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Oct 2013 02:03:49 +1000
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <84C62C21-F51B-4F80-A212-F47D9D3BBCC1@stufft.io>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CAPDm-Fi-ER_rnryYiV0cdYFW3ExiP3xd2RLaG-qKkUtNc96MWQ@mail.gmail.com>
 <CADiSq7eNnDc=CL59Z-ACkaSWnJ7OWM1OECWxwQ6QW4M_ssWSJQ@mail.gmail.com>
 <F7116169-DF4B-45E5-983E-CD8C7EE86E27@stufft.io>
 <CADiSq7cL4VL-ikppSke223rsLXBsZfL8=dGQTz2X3uSfrLcx7A@mail.gmail.com>
 <CAOTD34a8xKq-pkxMp89eL4oaAzgGCKPepZMQt1NkLMZB=DMa+g@mail.gmail.com>
 <CADiSq7fFZhwUEfGmEvOgOZc8y=OvZGNSSnXk38nmsn+gaoh6MA@mail.gmail.com>
 <84C62C21-F51B-4F80-A212-F47D9D3BBCC1@stufft.io>
Message-ID: <CADiSq7dhTe7v-uc-=JnH5x1wBYo9G+0_-7kN7vNdT8Qro_wqzA@mail.gmail.com>

On 18 October 2013 01:45, Donald Stufft <donald at stufft.io> wrote:
>
> On Oct 17, 2013, at 11:36 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>> Indeed - the metadata PEPs really aren't aimed at end users. Until
>> something is marked as deprecated in the CPython docs, and actually
>> deprecated in the standard library, it isn't really deprecated :)
>>
>> We can at least do a documented deprecation in the CPython docs when
>> we're implementing the PEP 453 documentation updates (I'll hopefully
>> get the Windows path changes in this weekend sometime, and after that
>> it should finally be ready for MvL's formal pronouncement)
>
> Ironically the documentation shows how confusing it is because for someone
> new coming into packaging that documentation reads like this is what you
> put in in order to depend on other things. I would challenge anyone to write
> docs that aren't confusing that actually explains what requires actually does.

The docs don't have to explain what it does, they have to explain that
it's obsolete and shouldn't be used. Deprecating it in the metadata
PEPs isn't enough.

> So again I ask what I need to do to deprecate and remove requires, provides,
> obsoletes. Do I need to make a PEP? Do I need to make a bug tracker issue
> and a patch? What's the path forward.

The docs changes can probably be done as part of the PEP 453 docs updates.

Emitting a deprecation warning in 3.4+ would need a patch on the
CPython issue tracker.

> Can we at least agree that these can removed?
>   obsoletes_dist - Never been used, never been implemented besides in Distutils2
>   requires_external - Used 9 times by 1 project, never been implemented outside of Distutils2
>   requires_python - Used 61 times, never been implemented outside of Distutils2
>
> These either have a different method of supporting them in PEP426 (and
> thus keeping around two ways of doing the same thing is confusing,
> especially when it was never formally implemented) or were omitted
> completely.

Sure, but what's the hurry? Is it just a matter of wanting to be able
to leave them out of the Warehouse data model?

As I stated earlier, I'm essentially OK with PyPI rejecting these with
a clear error message and a pointer towards setuptools for *new*
projects. I'm only averse to breaking them for projects that already
have these populated in at least one release, and upload a new release
that still includes them.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From donald at stufft.io  Thu Oct 17 18:13:42 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 17 Oct 2013 12:13:42 -0400
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CADiSq7dhTe7v-uc-=JnH5x1wBYo9G+0_-7kN7vNdT8Qro_wqzA@mail.gmail.com>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CAPDm-Fi-ER_rnryYiV0cdYFW3ExiP3xd2RLaG-qKkUtNc96MWQ@mail.gmail.com>
 <CADiSq7eNnDc=CL59Z-ACkaSWnJ7OWM1OECWxwQ6QW4M_ssWSJQ@mail.gmail.com>
 <F7116169-DF4B-45E5-983E-CD8C7EE86E27@stufft.io>
 <CADiSq7cL4VL-ikppSke223rsLXBsZfL8=dGQTz2X3uSfrLcx7A@mail.gmail.com>
 <CAOTD34a8xKq-pkxMp89eL4oaAzgGCKPepZMQt1NkLMZB=DMa+g@mail.gmail.com>
 <CADiSq7fFZhwUEfGmEvOgOZc8y=OvZGNSSnXk38nmsn+gaoh6MA@mail.gmail.com>
 <84C62C21-F51B-4F80-A212-F47D9D3BBCC1@stufft.io>
 <CADiSq7dhTe7v-uc-=JnH5x1wBYo9G+0_-7kN7vNdT8Qro_wqzA@mail.gmail.com>
Message-ID: <8D99075C-9673-4379-AF55-58C8B66726C5@stufft.io>


On Oct 17, 2013, at 12:03 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 18 October 2013 01:45, Donald Stufft <donald at stufft.io> wrote:
>> 
>> On Oct 17, 2013, at 11:36 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> 
>>> Indeed - the metadata PEPs really aren't aimed at end users. Until
>>> something is marked as deprecated in the CPython docs, and actually
>>> deprecated in the standard library, it isn't really deprecated :)
>>> 
>>> We can at least do a documented deprecation in the CPython docs when
>>> we're implementing the PEP 453 documentation updates (I'll hopefully
>>> get the Windows path changes in this weekend sometime, and after that
>>> it should finally be ready for MvL's formal pronouncement)
>> 
>> Ironically the documentation shows how confusing it is because for someone
>> new coming into packaging that documentation reads like this is what you
>> put in in order to depend on other things. I would challenge anyone to write
>> docs that aren't confusing that actually explains what requires actually does.
> 
> The docs don't have to explain what it does, they have to explain that
> it's obsolete and shouldn't be used. Deprecating it in the metadata
> PEPs isn't enough.

I meant supporting the viewpoint that these fields are confusing, not supporting
that they've been deprecated.

> 
>> So again I ask what I need to do to deprecate and remove requires, provides,
>> obsoletes. Do I need to make a PEP? Do I need to make a bug tracker issue
>> and a patch? What's the path forward.
> 
> The docs changes can probably be done as part of the PEP 453 docs updates.
> 
> Emitting a deprecation warning in 3.4+ would need a patch on the
> CPython issue tracker.

Distutils already accepts arbitrary keyword values and prints a warning so I
think all that would need to be done is to remove them and let the regular
machinery take care of it. I can work up a patch for that.

> 
>> Can we at least agree that these can removed?
>>  obsoletes_dist - Never been used, never been implemented besides in Distutils2
>>  requires_external - Used 9 times by 1 project, never been implemented outside of Distutils2
>>  requires_python - Used 61 times, never been implemented outside of Distutils2
>> 
>> These either have a different method of supporting them in PEP426 (and
>> thus keeping around two ways of doing the same thing is confusing,
>> especially when it was never formally implemented) or were omitted
>> completely.
> 
> Sure, but what's the hurry? Is it just a matter of wanting to be able
> to leave them out of the Warehouse data model?

The "hurry" on these is less a hurry and more wanting to clear them
out to prevent confusion. Warehouse is backed by the same schema
that PyPI itself uses. If nobody (or practically nobody) is using them
there's very little benefit to waiting to remove them.

Cleaning up the database to try and remove some of the bit rot, bad ideas,
deprecated and removed features is an ongoing process i've been engaged
in. This is the area I was working on this morning. So it can wait, but I don't
see a need to wait on these fields which will help clean up the DB. In this
case when I came across these I thought about it and realized that these
fields only really exist to confuse people and they should probably be removed
out right.

> 
> As I stated earlier, I'm essentially OK with PyPI rejecting these with
> a clear error message and a pointer towards setuptools for *new*
> projects. I'm only averse to breaking them for projects that already
> have these populated in at least one release, and upload a new release
> that still includes them.

Are you fine with PyPI just discarding these pieces of metadata?

> 
> Cheers,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/d14e0628/attachment.sig>

From fuzzyman at gmail.com  Thu Oct 17 18:26:28 2013
From: fuzzyman at gmail.com (Michael Foord)
Date: Thu, 17 Oct 2013 17:26:28 +0100
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <BAC77324-A849-4D0E-AAA1-96A9995310B0@stufft.io>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>
 <F33C6AA5-383B-4060-8643-CC6F561F3F5F@stufft.io>
 <CAKCKLWxCCx3QXGknicq4dgp2Lf-xrbWUGxStd0_ZXw5dD6j1Lg@mail.gmail.com>
 <CAPDm-FiD8PhkzEZ3PvuHK8KLWkZJ0=6n-RkMUTmoUYyvUG5SiA@mail.gmail.com>
 <CAKCKLWwpsuAjOXQRteNxGvNrn9Am5OG_=H0CqXrJDVvr5f0r7Q@mail.gmail.com>
 <BAC77324-A849-4D0E-AAA1-96A9995310B0@stufft.io>
Message-ID: <CAKCKLWwrU5jL42FNzOEG5RUc1EQdKRfHg46FZ7RD+HYWB_es=g@mail.gmail.com>

On 17 October 2013 16:53, Donald Stufft <donald at stufft.io> wrote:

>
> On Oct 17, 2013, at 11:49 AM, Michael Foord <fuzzyman at gmail.com> wrote:
>
> Package upload certainly worked, and that is what is going to be broken.
>
>
> So would you be ok with deprecating and removing to equal "this metadata
> silently
> gets sent to /dev/null" in order to not break uploads for what would have
> affected
> roughly 4% of the total new releases on PyPI in 2013.
>
>

Fine with me.

Michael



> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
> DCFA
>
>


-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/eaa359f1/attachment-0001.html>

From noah at coderanger.net  Thu Oct 17 20:33:16 2013
From: noah at coderanger.net (Noah Kantrowitz)
Date: Thu, 17 Oct 2013 11:33:16 -0700
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CAKCKLWwrU5jL42FNzOEG5RUc1EQdKRfHg46FZ7RD+HYWB_es=g@mail.gmail.com>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>
 <F33C6AA5-383B-4060-8643-CC6F561F3F5F@stufft.io>
 <CAKCKLWxCCx3QXGknicq4dgp2Lf-xrbWUGxStd0_ZXw5dD6j1Lg@mail.gmail.com>
 <CAPDm-FiD8PhkzEZ3PvuHK8KLWkZJ0=6n-RkMUTmoUYyvUG5SiA@mail.gmail.com>
 <CAKCKLWwpsuAjOXQRteNxGvNrn9Am5OG_=H0CqXrJDVvr5f0r7Q@mail.gmail.com>
 <BAC77324-A849-4D0E-AAA1-96A9995310B0@stufft.io>
 <CAKCKLWwrU5jL42FNzOEG5RUc1EQdKRfHg46FZ7RD+HYWB_es=g@mail.gmail.com>
Message-ID: <26C9A997-7F2F-43DD-844A-4B376ED14490@coderanger.net>


On Oct 17, 2013, at 9:26 AM, Michael Foord <fuzzyman at gmail.com> wrote:

> 
> 
> 
> On 17 October 2013 16:53, Donald Stufft <donald at stufft.io> wrote:
> 
> On Oct 17, 2013, at 11:49 AM, Michael Foord <fuzzyman at gmail.com> wrote:
> 
>> Package upload certainly worked, and that is what is going to be broken.
> 
> So would you be ok with deprecating and removing to equal "this metadata silently
> gets sent to /dev/null" in order to not break uploads for what would have affected
> roughly 4% of the total new releases on PyPI in 2013.
> 

My vote on this whole thing in the general context of how to handle deprecating metadata fields
* Email anyone using deprecated metadata at the time of deprecation (or now, in the case of this stuff)
* Deprecation would follow a somewhat normal arc:
  * Initially it is just marked as deprecated in the docs (pending deprecation phase).
  * "One major release" (which is fuzzy in this case, but 6-12 months) later it goes to dev null on input and is removed from all output.
  * "One major release" later it is a fatal error.

Having this whole schedule formalized will help everyone to know how we evolve the metadata spec, and because it is key-value pairs we have some wiggle room to sometimes ignore certain keys or treat them as opaque blobs (a la HTTP/MIME headers). In the case of this instance, I would say we should do the email and dev-null-ing immediately and then just pick up as normal and in 6-12 months (whatever we decide, not that it should actually be an ill-defined time period) it becomes a fatal error.

--Noah

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/00013aa5/attachment.sig>

From donald at stufft.io  Thu Oct 17 20:38:15 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 17 Oct 2013 14:38:15 -0400
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <26C9A997-7F2F-43DD-844A-4B376ED14490@coderanger.net>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>
 <F33C6AA5-383B-4060-8643-CC6F561F3F5F@stufft.io>
 <CAKCKLWxCCx3QXGknicq4dgp2Lf-xrbWUGxStd0_ZXw5dD6j1Lg@mail.gmail.com>
 <CAPDm-FiD8PhkzEZ3PvuHK8KLWkZJ0=6n-RkMUTmoUYyvUG5SiA@mail.gmail.com>
 <CAKCKLWwpsuAjOXQRteNxGvNrn9Am5OG_=H0CqXrJDVvr5f0r7Q@mail.gmail.com>
 <BAC77324-A849-4D0E-AAA1-96A9995310B0@stufft.io>
 <CAKCKLWwrU5jL42FNzOEG5RUc1EQdKRfHg46FZ7RD+HYWB_es=g@mail.gmail.com>
 <26C9A997-7F2F-43DD-844A-4B376ED14490@coderanger.net>
Message-ID: <3D98FF61-1815-4F01-8CA4-346182F59FD3@stufft.io>

On Oct 17, 2013, at 2:33 PM, Noah Kantrowitz <noah at coderanger.net> wrote:

> 
> On Oct 17, 2013, at 9:26 AM, Michael Foord <fuzzyman at gmail.com> wrote:
> 
>> 
>> 
>> 
>> On 17 October 2013 16:53, Donald Stufft <donald at stufft.io> wrote:
>> 
>> On Oct 17, 2013, at 11:49 AM, Michael Foord <fuzzyman at gmail.com> wrote:
>> 
>>> Package upload certainly worked, and that is what is going to be broken.
>> 
>> So would you be ok with deprecating and removing to equal "this metadata silently
>> gets sent to /dev/null" in order to not break uploads for what would have affected
>> roughly 4% of the total new releases on PyPI in 2013.
>> 
> 
> My vote on this whole thing in the general context of how to handle deprecating metadata fields
> * Email anyone using deprecated metadata at the time of deprecation (or now, in the case of this stuff)
> * Deprecation would follow a somewhat normal arc:
>  * Initially it is just marked as deprecated in the docs (pending deprecation phase).
>  * "One major release" (which is fuzzy in this case, but 6-12 months) later it goes to dev null on input and is removed from all output.
>  * "One major release" later it is a fatal error.
> 
> Having this whole schedule formalized will help everyone to know how we evolve the metadata spec, and because it is key-value pairs we have some wiggle room to sometimes ignore certain keys or treat them as opaque blobs (a la HTTP/MIME headers). In the case of this instance, I would say we should do the email and dev-null-ing immediately and then just pick up as normal and in 6-12 months (whatever we decide, not that it should actually be an ill-defined time period) it becomes a fatal error.
> 
> --Noah
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

This sounds reasonable to me.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/fd07e129/attachment.sig>

From ncoghlan at gmail.com  Fri Oct 18 00:50:21 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Oct 2013 08:50:21 +1000
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <3D98FF61-1815-4F01-8CA4-346182F59FD3@stufft.io>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>
 <F33C6AA5-383B-4060-8643-CC6F561F3F5F@stufft.io>
 <CAKCKLWxCCx3QXGknicq4dgp2Lf-xrbWUGxStd0_ZXw5dD6j1Lg@mail.gmail.com>
 <CAPDm-FiD8PhkzEZ3PvuHK8KLWkZJ0=6n-RkMUTmoUYyvUG5SiA@mail.gmail.com>
 <CAKCKLWwpsuAjOXQRteNxGvNrn9Am5OG_=H0CqXrJDVvr5f0r7Q@mail.gmail.com>
 <BAC77324-A849-4D0E-AAA1-96A9995310B0@stufft.io>
 <CAKCKLWwrU5jL42FNzOEG5RUc1EQdKRfHg46FZ7RD+HYWB_es=g@mail.gmail.com>
 <26C9A997-7F2F-43DD-844A-4B376ED14490@coderanger.net>
 <3D98FF61-1815-4F01-8CA4-346182F59FD3@stufft.io>
Message-ID: <CADiSq7fpO7KAXHoGqPM1RdkXwYaBam+og3GcsQOC7M8qcnA9eA@mail.gmail.com>

On 18 Oct 2013 04:48, "Donald Stufft" <donald at stufft.io> wrote:
>
> On Oct 17, 2013, at 2:33 PM, Noah Kantrowitz <noah at coderanger.net> wrote:
>
> >
> > On Oct 17, 2013, at 9:26 AM, Michael Foord <fuzzyman at gmail.com> wrote:
> >
> >>
> >>
> >>
> >> On 17 October 2013 16:53, Donald Stufft <donald at stufft.io> wrote:
> >>
> >> On Oct 17, 2013, at 11:49 AM, Michael Foord <fuzzyman at gmail.com> wrote:
> >>
> >>> Package upload certainly worked, and that is what is going to be
broken.
> >>
> >> So would you be ok with deprecating and removing to equal "this
metadata silently
> >> gets sent to /dev/null" in order to not break uploads for what would
have affected
> >> roughly 4% of the total new releases on PyPI in 2013.
> >>
> >
> > My vote on this whole thing in the general context of how to handle
deprecating metadata fields
> > * Email anyone using deprecated metadata at the time of deprecation (or
now, in the case of this stuff)
> > * Deprecation would follow a somewhat normal arc:
> >  * Initially it is just marked as deprecated in the docs (pending
deprecation phase).
> >  * "One major release" (which is fuzzy in this case, but 6-12 months)
later it goes to dev null on input and is removed from all output.
> >  * "One major release" later it is a fatal error.
> >
> > Having this whole schedule formalized will help everyone to know how we
evolve the metadata spec, and because it is key-value pairs we have some
wiggle room to sometimes ignore certain keys or treat them as opaque blobs
(a la HTTP/MIME headers). In the case of this instance, I would say we
should do the email and dev-null-ing immediately and then just pick up as
normal and in 6-12 months (whatever we decide, not that it should actually
be an ill-defined time period) it becomes a fatal error.
> >
> > --Noah
> >
> > _______________________________________________
> > Distutils-SIG maillist  -  Distutils-SIG at python.org
> > https://mail.python.org/mailman/listinfo/distutils-sig
>
> This sounds reasonable to me.

And to me. A general "Evolution of PyPI APIs" process PEP could be a very
helpful thing to avoid having to rehash this discussion for every change :)

Given that PyPI doesn't have releases as such, perhaps we could tie this to
the feature release cadence of pip? And officially recommend twine as the
upload tool over using distutils directly? (Is twine ready for that at this
point?)

The only other thing I would add is that when previous output is
/dev/null'ed we may want to have a placeholder for a while with a link to
an explanation for the removal.

Cheers,
Nick.
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
DCFA
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131018/96091339/attachment.html>

From donald at stufft.io  Fri Oct 18 00:55:07 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 17 Oct 2013 18:55:07 -0400
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CADiSq7fpO7KAXHoGqPM1RdkXwYaBam+og3GcsQOC7M8qcnA9eA@mail.gmail.com>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>
 <F33C6AA5-383B-4060-8643-CC6F561F3F5F@stufft.io>
 <CAKCKLWxCCx3QXGknicq4dgp2Lf-xrbWUGxStd0_ZXw5dD6j1Lg@mail.gmail.com>
 <CAPDm-FiD8PhkzEZ3PvuHK8KLWkZJ0=6n-RkMUTmoUYyvUG5SiA@mail.gmail.com>
 <CAKCKLWwpsuAjOXQRteNxGvNrn9Am5OG_=H0CqXrJDVvr5f0r7Q@mail.gmail.com>
 <BAC77324-A849-4D0E-AAA1-96A9995310B0@stufft.io>
 <CAKCKLWwrU5jL42FNzOEG5RUc1EQdKRfHg46FZ7RD+HYWB_es=g@mail.gmail.com>
 <26C9A997-7F2F-43DD-844A-4B376ED14490@coderanger.net>
 <3D98FF61-1815-4F01-8CA4-346182F59FD3@stufft.io>
 <CADiSq7fpO7KAXHoGqPM1RdkXwYaBam+og3GcsQOC7M8qcnA9eA@mail.gmail.com>
Message-ID: <7FBD8EA7-156E-42FF-8FD9-3D493CE2F13B@stufft.io>


On Oct 17, 2013, at 6:50 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> And to me. A general "Evolution of PyPI APIs" process PEP could be a very helpful thing to avoid having to rehash this discussion for every change :)
> 
PEPapolza
> Given that PyPI doesn't have releases as such, perhaps we could tie this to the feature release cadence of pip? And officially recommend twine as the upload tool over using distutils directly? (Is twine ready for that at this point?)
> 
Possibly, but I think it probably makes more sense to just do date based. Individual proposals can include special casings that depend on a release of a piece of tooling if it makes sense for that proposal.
> The only other thing I would add is that when previous output is /dev/null'ed we may want to have a placeholder for a while with a link to an explanation for the removal.
> 
A placeholder where? On the PyPI UX or something?

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/74b1752d/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/74b1752d/attachment-0001.sig>

From noah at coderanger.net  Fri Oct 18 01:03:01 2013
From: noah at coderanger.net (Noah Kantrowitz)
Date: Thu, 17 Oct 2013 16:03:01 -0700
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <CADiSq7fpO7KAXHoGqPM1RdkXwYaBam+og3GcsQOC7M8qcnA9eA@mail.gmail.com>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>
 <F33C6AA5-383B-4060-8643-CC6F561F3F5F@stufft.io>
 <CAKCKLWxCCx3QXGknicq4dgp2Lf-xrbWUGxStd0_ZXw5dD6j1Lg@mail.gmail.com>
 <CAPDm-FiD8PhkzEZ3PvuHK8KLWkZJ0=6n-RkMUTmoUYyvUG5SiA@mail.gmail.com>
 <CAKCKLWwpsuAjOXQRteNxGvNrn9Am5OG_=H0CqXrJDVvr5f0r7Q@mail.gmail.com>
 <BAC77324-A849-4D0E-AAA1-96A9995310B0@stufft.io>
 <CAKCKLWwrU5jL42FNzOEG5RUc1EQdKRfHg46FZ7RD+HYWB_es=g@mail.gmail.com>
 <26C9A997-7F2F-43DD-844A-4B376ED14490@coderanger.net>
 <3D98FF61-1815-4F01-8CA4-346182F59FD3@stufft.io>
 <CADiSq7fpO7KAXHoGqPM1RdkXwYaBam+og3GcsQOC7M8qcnA9eA@mail.gmail.com>
Message-ID: <4ADC09FC-1C2B-4234-AB4E-2046FFE04C48@coderanger.net>


On Oct 17, 2013, at 3:50 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> 
> On 18 Oct 2013 04:48, "Donald Stufft" <donald at stufft.io> wrote:
> >
> > On Oct 17, 2013, at 2:33 PM, Noah Kantrowitz <noah at coderanger.net> wrote:
> >
> > >
> > > On Oct 17, 2013, at 9:26 AM, Michael Foord <fuzzyman at gmail.com> wrote:
> > >
> > >>
> > >>
> > >>
> > >> On 17 October 2013 16:53, Donald Stufft <donald at stufft.io> wrote:
> > >>
> > >> On Oct 17, 2013, at 11:49 AM, Michael Foord <fuzzyman at gmail.com> wrote:
> > >>
> > >>> Package upload certainly worked, and that is what is going to be broken.
> > >>
> > >> So would you be ok with deprecating and removing to equal "this metadata silently
> > >> gets sent to /dev/null" in order to not break uploads for what would have affected
> > >> roughly 4% of the total new releases on PyPI in 2013.
> > >>
> > >
> > > My vote on this whole thing in the general context of how to handle deprecating metadata fields
> > > * Email anyone using deprecated metadata at the time of deprecation (or now, in the case of this stuff)
> > > * Deprecation would follow a somewhat normal arc:
> > >  * Initially it is just marked as deprecated in the docs (pending deprecation phase).
> > >  * "One major release" (which is fuzzy in this case, but 6-12 months) later it goes to dev null on input and is removed from all output.
> > >  * "One major release" later it is a fatal error.
> > >
> > > Having this whole schedule formalized will help everyone to know how we evolve the metadata spec, and because it is key-value pairs we have some wiggle room to sometimes ignore certain keys or treat them as opaque blobs (a la HTTP/MIME headers). In the case of this instance, I would say we should do the email and dev-null-ing immediately and then just pick up as normal and in 6-12 months (whatever we decide, not that it should actually be an ill-defined time period) it becomes a fatal error.
> > >
> > > --Noah
> > >
> > > _______________________________________________
> > > Distutils-SIG maillist  -  Distutils-SIG at python.org
> > > https://mail.python.org/mailman/listinfo/distutils-sig
> >
> > This sounds reasonable to me.
> 
> And to me. A general "Evolution of PyPI APIs" process PEP could be a very helpful thing to avoid having to rehash this discussion for every change :)

+1, especially because the process is asymmetric, pip needs to accept and silently ignore unknown metadata fields indefinitely.

--Noah

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131017/87b74b26/attachment.sig>

From ncoghlan at gmail.com  Fri Oct 18 01:44:16 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Oct 2013 09:44:16 +1000
Subject: [Distutils] Deprecate and Block requires/provides
In-Reply-To: <7FBD8EA7-156E-42FF-8FD9-3D493CE2F13B@stufft.io>
References: <1DDFA594-66D7-4233-A714-550B8FE4E3A9@stufft.io>
 <CADiSq7f7=QvOd9mkeNn0MnPbaCJd6Ycfa_nY312P+GdUt7Dsbg@mail.gmail.com>
 <F33C6AA5-383B-4060-8643-CC6F561F3F5F@stufft.io>
 <CAKCKLWxCCx3QXGknicq4dgp2Lf-xrbWUGxStd0_ZXw5dD6j1Lg@mail.gmail.com>
 <CAPDm-FiD8PhkzEZ3PvuHK8KLWkZJ0=6n-RkMUTmoUYyvUG5SiA@mail.gmail.com>
 <CAKCKLWwpsuAjOXQRteNxGvNrn9Am5OG_=H0CqXrJDVvr5f0r7Q@mail.gmail.com>
 <BAC77324-A849-4D0E-AAA1-96A9995310B0@stufft.io>
 <CAKCKLWwrU5jL42FNzOEG5RUc1EQdKRfHg46FZ7RD+HYWB_es=g@mail.gmail.com>
 <26C9A997-7F2F-43DD-844A-4B376ED14490@coderanger.net>
 <3D98FF61-1815-4F01-8CA4-346182F59FD3@stufft.io>
 <CADiSq7fpO7KAXHoGqPM1RdkXwYaBam+og3GcsQOC7M8qcnA9eA@mail.gmail.com>
 <7FBD8EA7-156E-42FF-8FD9-3D493CE2F13B@stufft.io>
Message-ID: <CADiSq7duWrAGEXQTJEaLJUFQcYxCe=+kVngg6cJgwmh904fLcg@mail.gmail.com>

On 18 Oct 2013 08:55, "Donald Stufft" <donald at stufft.io> wrote:
>
>
> On Oct 17, 2013, at 6:50 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>> And to me. A general "Evolution of PyPI APIs" process PEP could be a
very helpful thing to avoid having to rehash this discussion for every
change :)
>
> PEPapolza

One PEP, two PEP, three PEP, more PEP :)

It occurs to me the numbering for new process PEPs is different from
normal. Still, just a matter of looking at PEP 0 to pick the right
subrange.

>> Given that PyPI doesn't have releases as such, perhaps we could tie this
to the feature release cadence of pip? And officially recommend twine as
the upload tool over using distutils directly? (Is twine ready for that at
this point?)
>
> Possibly, but I think it probably makes more sense to just do date based.
Individual proposals can include special casings that depend on a release
of a piece of tooling if it makes sense for that proposal.

That works, too.

>
>> The only other thing I would add is that when previous output is
/dev/null'ed we may want to have a placeholder for a while with a link to
an explanation for the removal.
>
> A placeholder where? On the PyPI UX or something?

Yeah, I was thinking of the part at the bottom of the package info page
that currently displays this metadata.

Cheers,
Nick.

>
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
DCFA
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131018/ab17abdf/attachment.html>

From vinay_sajip at yahoo.co.uk  Fri Oct 18 19:44:53 2013
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 18 Oct 2013 17:44:53 +0000 (UTC)
Subject: [Distutils] distlib 0.1.3 released on PyPI
Message-ID: <loom.20131018T193505-8@post.gmane.org>

I've just released version 0.1.3 of distlib on PyPI [1]. For newcomers, 
distlib is a library of packaging functionality which is intended to be 
usable as the basis for third-party packaging tools.

The changes in this release are as follows:

database.py:

    Added support for PEP 426 JSON metadata (pydist.json).

    Generalised digests to support e.g. SHA256.

    Fixed a bug in parsing legacy metadata from .egg directories.

    Removed duplicated code relating to parsing "provides" fields.

index.py:

    Changes relating to support for PEP 426 JSON metadata (pydist.json).

locators.py:

    Changes relating to support for PEP 426 JSON metadata (pydist.json).

    Fixed a bug in scoring download URLs for preference when multiple URLs
    are available.

    The legacy scheme is used for the default locator.

    Made changes relating to parsing "provides" fields.

    Generalised digests to support e.g. SHA256.

    If no release version is found for a requirement, prereleases are
    now considered even if not explicitly requested.

markers.py:

    Added support for markers as specified in PEP 426.

metadata.py:

    Added support for PEP 426 JSON metadata (pydist.json). The old
    metadata class is renamed to LegacyMetadata, and the (new)
    Metadata class wraps the JSON format (and also the legacy format,
    through LegacyMetadata).

    Removed code which was only used if docutils was installed. This code
    implemented validation of .rst descriptions, which is not done in
    distlib.

scripts.py:

    Updated the logic for writing executable files to deal as best we can
    with files which are already in use and hence cannot be deleted on
    Windows.

    Changed the script generation when launchers are used to write a
    single executable which wraps a script (whether pre-built or
    generated) and includes a manifest to avoid UAC prompts on Windows.

    Changed the interface for script generation options: the "make" and
    "make_multiple" methods of ScriptMaker now take an optional
    options dictionary.

util.py:

    Added extract_by_key() to copy selected keys from one dict to another.

    Added parse_name_and_version() for use in parsing "provides" fields.

    Made split_filename more flexible.

version.py:

    Added support for PEP 440 version matching.

    Removed AdaptiveVersion, AdaptiveMatcher etc. as they don't add
    sufficient value to justify keeping them in.

wheel.py:

    Added wheel_version kwarg to Wheel.build API.

    Changed Wheel.install API (after consultation on distutils-sig).

    Added support for PEP 426 JSON metadata (pydist.json).

    Added lib_only flag to install() method.

docs:

    Numerous documentation updates, not detailed further here.

tests:

    Numerous test refinements, not detailed further here.

other:

    Corrected setup.py to ensure that executable wrappers are included.

Please try it out, and give some feedback using the issue tracker! [2]

Regards,

Vinay Sajip

[1] https://pypi.python.org/pypi/distlib/0.1.3
[2] https://bitbucket.org/pypa/distlib/issues/new


From vinay_sajip at yahoo.co.uk  Fri Oct 18 20:45:57 2013
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 18 Oct 2013 18:45:57 +0000 (UTC)
Subject: [Distutils] distil 0.1.2 released
Message-ID: <loom.20131018T203020-218@post.gmane.org>

I've released version 0.1.2 of distil, downloadable from [1]. It's based
on distlib 0.1.3, which brings support for PEP 426. The other changes are as 
follows:

* Fixed a bug in pre-release handling which occurred when there were
  *only* pre-releases available.

* Added support to convert "bdist_wininst" installers to wheels. This is
  done by using "distil package", specifying only the wheel format and
  giving the path to the installer in place of the source directory.

* Added --wheel-version option to package command to allow specifying
  a wheel version when building wheels.

* Fixed bugs in uninstallation code which occurred only for some
  installations.

* Fixed a bug in package data installation where data in packages
  not explicitly named in package-data was missed.

* Updated "distil link" to show existing links when invoked with
  no arguments.

* When building wheels, native executable launchers are not included, and
  scripts are not generated from exports.

* Added support for PEP 426 installation hooks.

* Added --target argument to "distil install" to allow partial
  installation to arbitrary directories.

* Added --prereleases argument to "distil install", "distil test", and
  "distil download" to support inclusion of pre-releases when locating
  distributions and performing dependency resolution.

* Numerous documentation updates.

Your feedback will be gratefully received!

Regards,

Vinay Sajip

[1] https://bitbucket.org/vinay.sajip/docs-distil/downloads/distil.py



From chris.barker at noaa.gov  Fri Oct 18 18:58:34 2013
From: chris.barker at noaa.gov (Chris Barker)
Date: Fri, 18 Oct 2013 09:58:34 -0700
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
Message-ID: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>

Someone on another list indicated that pip installing binary wheels
from PyPi will ONLY work for Windows.

Is that the case? I think it's desperately needed for OS-X as well.

Linux is so diverse that I can't imagine it being useful, but OS-X has
only so many versions, and the python.org OS-X binaries are very clear
in their requirements -- it would be very useful if folks could easily
get binary wheels for OS-X

-Chris




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

From barry at python.org  Fri Oct 18 21:27:45 2013
From: barry at python.org (Barry Warsaw)
Date: Fri, 18 Oct 2013 15:27:45 -0400
Subject: [Distutils] URL Structure of Packages URLs
In-Reply-To: <6238CA7D-6DDE-437B-B2CF-ABFCDDDC9874@stufft.io>
References: <70E49064-C95C-4B81-ACE5-13F76688803B@stufft.io>
 <20131008140553.GA20713@fridge.pov.lt>
 <60EC66B9-7661-4F3E-BECC-6A6520BE57DD@stufft.io>
 <20131009140243.76b45d85@anarchist>
 <CAAWk_Dxs=r3pRdn9ATsTqUd1tJGQiD+f2oXF4uuNsym-Z09NDg@mail.gmail.com>
 <20131012174226.7fa3b965@anarchist>
 <6238CA7D-6DDE-437B-B2CF-ABFCDDDC9874@stufft.io>
Message-ID: <20131018152745.2af10b1d@anarchist>

On Oct 12, 2013, at 06:15 PM, Donald Stufft wrote:

>Just to be clear, I don't fault folks for using the /packages/ urls. I was
>just trying to get some sort of idea if anyone actually used that url
>structure or not. I also don't plan on breaking anything for people who do.
>
>That being said, do you know if the Debian case requires something to be
>served at /packages/source/D/Django/ in order to locate a file at
>/packages/source/D/Django/Django-*.tar.gz?

It does.

uscan(1) is the tool that parses and acts on debian/watch files.  My Perl-fu
is pretty rusty so I'm not sure I'm reading it correctly, but I think it's
evident what the tool does (modulo tons of corner cases that mostly aren't
relevant for PyPI urls).

Here for example is tox's debian/watch file:

-----snip snip-----
version=3
http://pypi.python.org/packages/source/t/tox/tox-(.*).tar.gz
-----snip snip-----

And verbose output:

% uscan --verbose
-- Scanning for watchfiles in .
-- Found watchfile in ./debian
-- In debian/watch, processing watchfile line:
   http://pypi.python.org/packages/source/t/tox/tox-(.*).tar.gz
-- Found the following matching hrefs:
     tox-0.5.tar.gz (0.5)
     tox-0.5.tar.gz (0.5)
     tox-0.6.tar.gz (0.6)
     tox-0.6.tar.gz (0.6)
     tox-0.7.tar.gz (0.7)
     tox-0.7.tar.gz (0.7)
     tox-0.8.tar.gz (0.8)
     tox-0.8.tar.gz (0.8)
     tox-1.4.3.tar.gz (1.4.3)
     tox-1.4.3.tar.gz (1.4.3)
     tox-1.5.0.tar.gz (1.5.0)
     tox-1.5.0.tar.gz (1.5.0)
     tox-1.6.0.tar.gz (1.6.0)
     tox-1.6.0.tar.gz (1.6.0)
     tox-1.6.1.tar.gz (1.6.1)
     tox-1.6.1.tar.gz (1.6.1)
Newest version on remote site is 1.6.1, local version is 1.6.0
 => Newer version available from
    http://pypi.python.org/packages/source/t/tox/tox-1.6.1.tar.gz
-- Downloading updated package tox-1.6.1.tar.gz
-- Successfully downloaded updated package tox-1.6.1.tar.gz
    and symlinked tox_1.6.1.orig.tar.gz to it
-- Scan finished


And a directory listing of the URL leading up to the pattern (edited for
readability):

Index of /raw-packages/source/t/tox/

../
tox-0.5.tar.gz	 	12-Jul-2010 11:30               40853
tox-0.6.tar.gz	 	12-Jul-2010 18:21               41424
tox-0.7.tar.gz	 	14-Jul-2010 20:48               45934
tox-0.8.tar.gz	 	30-Jul-2010 23:41               41828
tox-0.9.zip	 	25-Nov-2010 20:05               57001
tox-1.0.zip	 	28-May-2011 14:59               61367
tox-1.1.zip	 	09-Jul-2011 10:09               99297
tox-1.2.zip	 	10-Nov-2011 18:40               66284
tox-1.3.zip	 	21-Dec-2011 08:40               67862
tox-1.4.1.zip	 	03-Jul-2012 09:34               76920
tox-1.4.2.zip	 	20-Jul-2012 11:05               77565
tox-1.4.3.tar.gz 	01-Mar-2013 09:44               71661
tox-1.4.zip	 	13-Jun-2012 15:01               76147
tox-1.5.0.tar.gz 	22-Jun-2013 13:05               73866
tox-1.6.0.tar.gz 	15-Aug-2013 13:39              130734
tox-1.6.1.tar.gz 	04-Sep-2013 14:17              131654

So yes there needs to be some url that uscan can get a list of to determine
the highest available version matching a wildcarded pattern.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131018/c6722d9d/attachment.sig>

From donald at stufft.io  Fri Oct 18 22:42:17 2013
From: donald at stufft.io (Donald Stufft)
Date: Fri, 18 Oct 2013 16:42:17 -0400
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
Message-ID: <C6DB17E5-1BB4-4E6F-9F38-6BEE31AF7D63@stufft.io>


On Oct 18, 2013, at 12:58 PM, Chris Barker <chris.barker at noaa.gov> wrote:

> Someone on another list indicated that pip installing binary wheels
> from PyPi will ONLY work for Windows.
> 
> Is that the case? I think it's desperately needed for OS-X as well.

For the time being yes, it'll change in the future.

> 
> Linux is so diverse that I can't imagine it being useful, but OS-X has
> only so many versions, and the python.org OS-X binaries are very clear
> in their requirements -- it would be very useful if folks could easily
> get binary wheels for OS-X
> 
> -Chris
> 
> 
> 
> 
> -- 
> 
> Christopher Barker, Ph.D.
> Oceanographer
> 
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
> 
> Chris.Barker at noaa.gov
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131018/f62b68c8/attachment-0001.sig>

From ncoghlan at gmail.com  Sat Oct 19 03:14:13 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 19 Oct 2013 11:14:13 +1000
Subject: [Distutils] distlib 0.1.3 released on PyPI
In-Reply-To: <loom.20131018T193505-8@post.gmane.org>
References: <loom.20131018T193505-8@post.gmane.org>
Message-ID: <CADiSq7cfoDM5V+6jtoiYp6G_iJmVtA5zRCpr+UCuFOKVyRV8mQ@mail.gmail.com>

On 19 Oct 2013 03:45, "Vinay Sajip" <vinay_sajip at yahoo.co.uk> wrote:
>
> I've just released version 0.1.3 of distlib on PyPI [1]. For newcomers,
> distlib is a library of packaging functionality which is intended to be
> usable as the basis for third-party packaging tools.

Nice!

Although users should remember that PEP 426 is still officially unstable at
this point in time (that mainly applies to the stuff relating to
extensions, exports, install hooks and the metabuild framework, though -
the core metadata model should be pretty solid).

Cheers,
Nick.

>
> The changes in this release are as follows:
>
> database.py:
>
>     Added support for PEP 426 JSON metadata (pydist.json).
>
>     Generalised digests to support e.g. SHA256.
>
>     Fixed a bug in parsing legacy metadata from .egg directories.
>
>     Removed duplicated code relating to parsing "provides" fields.
>
> index.py:
>
>     Changes relating to support for PEP 426 JSON metadata (pydist.json).
>
> locators.py:
>
>     Changes relating to support for PEP 426 JSON metadata (pydist.json).
>
>     Fixed a bug in scoring download URLs for preference when multiple URLs
>     are available.
>
>     The legacy scheme is used for the default locator.
>
>     Made changes relating to parsing "provides" fields.
>
>     Generalised digests to support e.g. SHA256.
>
>     If no release version is found for a requirement, prereleases are
>     now considered even if not explicitly requested.
>
> markers.py:
>
>     Added support for markers as specified in PEP 426.
>
> metadata.py:
>
>     Added support for PEP 426 JSON metadata (pydist.json). The old
>     metadata class is renamed to LegacyMetadata, and the (new)
>     Metadata class wraps the JSON format (and also the legacy format,
>     through LegacyMetadata).
>
>     Removed code which was only used if docutils was installed. This code
>     implemented validation of .rst descriptions, which is not done in
>     distlib.
>
> scripts.py:
>
>     Updated the logic for writing executable files to deal as best we can
>     with files which are already in use and hence cannot be deleted on
>     Windows.
>
>     Changed the script generation when launchers are used to write a
>     single executable which wraps a script (whether pre-built or
>     generated) and includes a manifest to avoid UAC prompts on Windows.
>
>     Changed the interface for script generation options: the "make" and
>     "make_multiple" methods of ScriptMaker now take an optional
>     options dictionary.
>
> util.py:
>
>     Added extract_by_key() to copy selected keys from one dict to another.
>
>     Added parse_name_and_version() for use in parsing "provides" fields.
>
>     Made split_filename more flexible.
>
> version.py:
>
>     Added support for PEP 440 version matching.
>
>     Removed AdaptiveVersion, AdaptiveMatcher etc. as they don't add
>     sufficient value to justify keeping them in.
>
> wheel.py:
>
>     Added wheel_version kwarg to Wheel.build API.
>
>     Changed Wheel.install API (after consultation on distutils-sig).
>
>     Added support for PEP 426 JSON metadata (pydist.json).
>
>     Added lib_only flag to install() method.
>
> docs:
>
>     Numerous documentation updates, not detailed further here.
>
> tests:
>
>     Numerous test refinements, not detailed further here.
>
> other:
>
>     Corrected setup.py to ensure that executable wrappers are included.
>
> Please try it out, and give some feedback using the issue tracker! [2]
>
> Regards,
>
> Vinay Sajip
>
> [1] https://pypi.python.org/pypi/distlib/0.1.3
> [2] https://bitbucket.org/pypa/distlib/issues/new
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131019/ea6860be/attachment.html>

From ncoghlan at gmail.com  Sat Oct 19 03:22:07 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 19 Oct 2013 11:22:07 +1000
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
Message-ID: <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>

On 19 Oct 2013 04:59, "Chris Barker" <chris.barker at noaa.gov> wrote:
>
> Someone on another list indicated that pip installing binary wheels
> from PyPi will ONLY work for Windows.
>
> Is that the case? I think it's desperately needed for OS-X as well.
>
> Linux is so diverse that I can't imagine it being useful, but OS-X has
> only so many versions, and the python.org OS-X binaries are very clear
> in their requirements -- it would be very useful if folks could easily
> get binary wheels for OS-X

We do plan to support it, but the pip devs uncovered a hole in the current
wheel spec that means it generates the same filename on *nix systems for
wheels that need to have different names for the download side of things to
work properly - hence the current Windows-only restriction.

Once ensurepip has landed in Python 3.4 and pip 1.5 is released, we should
be able to get back to updating the various metadata specs, with the aim of
getting cross-platform wheel support in pip 1.6 :)

Cheers,
Nick.

>
> -Chris
>
>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131019/029824ab/attachment.html>

From donald at stufft.io  Mon Oct 21 00:10:37 2013
From: donald at stufft.io (Donald Stufft)
Date: Sun, 20 Oct 2013 18:10:37 -0400
Subject: [Distutils] Warehouse - Simple API
In-Reply-To: <CALeMXf4ogETyqOYy=L0vZQzRom_Pv0hVtbki3ax0RCwkHNbs6g@mail.gmail.com>
References: <EE9E9386-7737-41A0-85C6-B96FD9EE518C@stufft.io>
 <CALeMXf4ogETyqOYy=L0vZQzRom_Pv0hVtbki3ax0RCwkHNbs6g@mail.gmail.com>
Message-ID: <EA4A9F67-BCAC-4281-B2A6-0BAE92D99200@stufft.io>

This is fixed now.

The problem was a misnamed variable: https://github.com/dstufft/warehouse/pull/80

I have a ticket to start writing functional tests which will test the actual output
of the HTML as well which will prevent this sort of thing in the future.

On Oct 12, 2013, at 8:23 PM, PJ Eby <pje at telecommunity.com> wrote:

> On Wed, Oct 2, 2013 at 10:57 PM, Donald Stufft <donald at stufft.io> wrote:
>> I've recently deployed the current Warehouse code. So far the
>> simple API has been implemented and it would be great if
>> people could point their installers to it and report back to me
>> if they run into any problems.
>> 
>> The current PyPI simple api is not documented at all so I've had
>> to reverse engineer it going from what I know, pip, etc. Ideally
>> if i've missed anything it will be found out with testing from y'all.
> 
> FYI, compare:
> 
> https://preview-pypi.python.org/simple/setuptools/
> 
> and:
> 
> https://pypi.python.org/simple/setuptools/
> 
> The former is missing the #egg URLs, which are required to install SVN versions.
> 
> In general, extracted URLs don't seem to be working on the preview site.


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131020/ee2c50a7/attachment.sig>

From p.f.moore at gmail.com  Mon Oct 21 12:39:51 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 21 Oct 2013 11:39:51 +0100
Subject: [Distutils] A portable version of pip
Message-ID: <CACac1F8SSkcGM-i=2=-UGdUOk1eQd_5pmOKYqVpL9Sdry67Vsg@mail.gmail.com>

With all of the current interest in bundling pip with the Python
distribution, I had a sudden thought. Vinay has developed a package
called pyzzer (https://pypi.python.org/pypi/pyzzer) that bundles up
Python modules into runnable single-file executables.

Using a command

    pyzzer -m pip:main -l t64 -o pipsa.exe -s "#!python" -r <path-to-pip>\pip

you can build a single-file executable version of pip that can install
wheels (it can't install sdists until setuptools is installed, because
installing sdists depends on having setuptools at install time).

The -s "#!python" is not needed on Unix as the default is
"#!/usr/bin/env python". Also the -l t64 is a Windows requirement to
get an exe - you can produce a cross-platform pyz archive without it
(see the pyzzer docs for details).

This requires the latest development version of pip (which contains a
fix for running pip from an uninstalled zipfile) and the following
patch to pip\__init__.py:

Change:

from pip import cmdoptions

to:

import pip.cmdoptions
cmdoptions = pip.cmdoptions

This fixes a recursive-import issue - I'll update pip "real soon now"
(TM) for this.

This is just an experiment at the moment - I've done minimal testing
but it seems to work as I'd expect. It may be worth thinking about,
though - either as an alternative to bundling (a bit late, I know) or
as a way of providing pip for earlier versions of Python.

Hope this is of interest.

Paul

From hv at tbz-pariv.de  Mon Oct 21 12:38:23 2013
From: hv at tbz-pariv.de (=?ISO-8859-15?Q?Thomas_G=FCttler?=)
Date: Mon, 21 Oct 2013 12:38:23 +0200
Subject: [Distutils] post install hook
Message-ID: <5265041F.5060503@tbz-pariv.de>

Hi,

I can live without a post-install hook.

But I think the documentation of setuptools
should contain information about this.

https://bitbucket.org/pypa/setuptools/issue/92/docs-post-install-hook


From oscar.j.benjamin at gmail.com  Mon Oct 21 14:43:30 2013
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 21 Oct 2013 13:43:30 +0100
Subject: [Distutils] post install hook
In-Reply-To: <5265041F.5060503@tbz-pariv.de>
References: <5265041F.5060503@tbz-pariv.de>
Message-ID: <CAHVvXxSqDAth3Gmarmq4Lfg1QRH5-0Qe_gH8OSkKMJtyD35zjg@mail.gmail.com>

On 21 October 2013 11:38, Thomas G?ttler <hv at tbz-pariv.de> wrote:
> Hi,
>
> I can live without a post-install hook.
>
> But I think the documentation of setuptools
> should contain information about this.
>
> https://bitbucket.org/pypa/setuptools/issue/92/docs-post-install-hook

That seems reasonable to me. Why don't you write a patch (as suggested
in the issue)?


Oscar

From vinay_sajip at yahoo.co.uk  Mon Oct 21 15:25:05 2013
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 21 Oct 2013 14:25:05 +0100 (BST)
Subject: [Distutils] A portable version of pip
In-Reply-To: <CACac1F8SSkcGM-i=2=-UGdUOk1eQd_5pmOKYqVpL9Sdry67Vsg@mail.gmail.com>
Message-ID: <1382361905.22877.YahooMailBasic@web171401.mail.ir2.yahoo.com>

I did a bit of work looking at running pip from a .zip using pyzzer, a little while ago. What were the fixes to pip which you mentioned? There were a number of places where pip needed changing:

1. How it handles cacert.pem when running from a .zip (default_cert_path in pip/locations.py)
2. Identifying the running pip rather than any installed pip
3. Expecting an installed setuptools to check its version to confirm that wheels can be supported
4. The wheel build code, which uses subprocess, assumes running from the file system

>From what I can see 2 and 3 have been addressed in recent changes, but I couldn't tell from a quick reading whether 1 has been addressed (there have certainly been code changes in this area) or whether 4 has been addressed.

Regards,

Vinay Sajip

--------------------------------------------
On Mon, 21/10/13, Paul Moore <p.f.moore at gmail.com> wrote:

 Subject: [Distutils] A portable version of pip
 To: "Distutils" <distutils-sig at python.org>
 Date: Monday, 21 October, 2013, 11:39
 
 With all of the current interest in
 bundling pip with the Python
 distribution, I had a sudden thought. Vinay has developed a
 package
 called pyzzer (https://pypi.python.org/pypi/pyzzer) that
 bundles up
 Python modules into runnable single-file executables.
 
 Using a command
 
 ? ? pyzzer -m pip:main -l t64 -o pipsa.exe -s
 "#!python" -r <path-to-pip>\pip
 
 you can build a single-file executable version of pip that
 can install
 wheels (it can't install sdists until setuptools is
 installed, because
 installing sdists depends on having setuptools at install
 time).
 
 The -s "#!python" is not needed on Unix as the default is
 "#!/usr/bin/env python". Also the -l t64 is a Windows
 requirement to
 get an exe - you can produce a cross-platform pyz archive
 without it
 (see the pyzzer docs for details).
 
 This requires the latest development version of pip (which
 contains a
 fix for running pip from an uninstalled zipfile) and the
 following
 patch to pip\__init__.py:
 
 Change:
 
 from pip import cmdoptions
 
 to:
 
 import pip.cmdoptions
 cmdoptions = pip.cmdoptions
 
 This fixes a recursive-import issue - I'll update pip "real
 soon now"
 (TM) for this.
 
 This is just an experiment at the moment - I've done minimal
 testing
 but it seems to work as I'd expect. It may be worth thinking
 about,
 though - either as an alternative to bundling (a bit late, I
 know) or
 as a way of providing pip for earlier versions of Python.
 
 Hope this is of interest.
 
 Paul
 _______________________________________________
 Distutils-SIG maillist? -? Distutils-SIG at python.org
 https://mail.python.org/mailman/listinfo/distutils-sig
 

From p.f.moore at gmail.com  Mon Oct 21 16:11:55 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 21 Oct 2013 15:11:55 +0100
Subject: [Distutils] A portable version of pip
In-Reply-To: <1382361905.22877.YahooMailBasic@web171401.mail.ir2.yahoo.com>
References: <CACac1F8SSkcGM-i=2=-UGdUOk1eQd_5pmOKYqVpL9Sdry67Vsg@mail.gmail.com>
 <1382361905.22877.YahooMailBasic@web171401.mail.ir2.yahoo.com>
Message-ID: <CACac1F8Nf2CapJoTP6rex=-dKeYsnCTSQoXx5vvnn8VnU1pxZA@mail.gmail.com>

On 21 October 2013 14:25, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> From what I can see 2 and 3 have been addressed in recent changes, but I couldn't tell from a quick reading whether 1 has been addressed (there have certainly been code changes in this area) or whether 4 has been addressed.

Yes, 3 in particular was the dev version change I mentioned.

With regard to 1 and 4, I was only really considering installation
from local wheels (the bootstrap case that ensurepip is aiming at) and
neither of 1 and 4 affect that. In particular anything that builds
from sdist needs a locally installed setuptools. It may be that
"everything else" works if a local setuptools is installed, but I
haven't tried.

The cert issue is more annoying (insofar as installing wheels from
PyPI *is* a reasonable thing to want to do) but the whole issue of
certificate management (particularly from zip files) is messy. This
may be a bug for requests to address, if they feel that running
requests from a zipfile is a valid use case. Personally, I'd just say
hang it and use pip's equivalent of "--no-check-certificate" (assuming
there is one, I can't see it right now, but there'd better be!).

There are certainly a lot of rough edges, and it may never be
"production quality", but I was surprised at how usable it was.

Paul

PS In pyzzer, is there a reason that -s on Windows couldn't default to
something (maybe just "#!python") that works for "the currently active
Python" out of the box so it matches Unix? Maybe with a separate
option to hard-code sys.executable for when you want to "freeze" the
binary to the interpreter you're using for the build (which would be
useful on Unix too)?

From vinay_sajip at yahoo.co.uk  Mon Oct 21 18:25:03 2013
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 21 Oct 2013 17:25:03 +0100 (BST)
Subject: [Distutils] A portable version of pip
In-Reply-To: <CACac1F8Nf2CapJoTP6rex=-dKeYsnCTSQoXx5vvnn8VnU1pxZA@mail.gmail.com>
Message-ID: <1382372703.71810.YahooMailBasic@web171404.mail.ir2.yahoo.com>


--------------------------------------------
On Mon, 21/10/13, Paul Moore <p.f.moore at gmail.com> wrote:

 PS In pyzzer, is there a reason that -s on Windows couldn't
 default to
 something (maybe just "#!python") that works for "the
 currently active
 Python" out of the box so it matches Unix? Maybe with a
 separate
 option to hard-code sys.executable for when you want to
 "freeze" the
 binary to the interpreter you're using for the build (which
 would be
 useful on Unix too)?
 
The default is "#!/usr/bin/env python", which should search the path for a Python executable, including on Windows as long as the latest launcher version is installed.



From p.f.moore at gmail.com  Mon Oct 21 18:53:26 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 21 Oct 2013 17:53:26 +0100
Subject: [Distutils] A portable version of pip
In-Reply-To: <1382372703.71810.YahooMailBasic@web171404.mail.ir2.yahoo.com>
References: <CACac1F8Nf2CapJoTP6rex=-dKeYsnCTSQoXx5vvnn8VnU1pxZA@mail.gmail.com>
 <1382372703.71810.YahooMailBasic@web171404.mail.ir2.yahoo.com>
Message-ID: <CACac1F_ExtEeFiVYt52d1aMw9GRH6s5hTmsSXu3bSABySmWcTw@mail.gmail.com>

On 21 October 2013 17:25, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> The default is "#!/usr/bin/env python", which should search the path for a Python executable, including on Windows as long as the latest launcher version is installed.

Not if you're using the exe launchers rather than a pyz file. The
supplied launchers don't (as far as I can tell) include the
/usr/bin/env logic from py.exe. I may be wrong, though, I'll do some
more tests tomorrow.
Paul

From chris.barker at noaa.gov  Mon Oct 21 19:02:29 2013
From: chris.barker at noaa.gov (Chris Barker)
Date: Mon, 21 Oct 2013 10:02:29 -0700
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
Message-ID: <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>

On Fri, Oct 18, 2013 at 6:22 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>  -- it would be very useful if folks could easily
>> get binary wheels for OS-X
>
> We do plan to support it, but the pip devs uncovered a hole in the current
> wheel spec that means it generates the same filename on *nix systems for
> wheels that need to have different names for the download side of things to
> work properly

THanks -- but really? don't OS-X wheels get:

macosx_10_6_intel

or some such tacked on? Where does that go wrong?

> Once ensurepip has landed in Python 3.4 and pip 1.5 is released, we should
> be able to get back to updating the various metadata specs, with the aim of
> getting cross-platform wheel support in pip 1.6 :)

Getting there... thanks,

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

From donald at stufft.io  Mon Oct 21 20:52:18 2013
From: donald at stufft.io (Donald Stufft)
Date: Mon, 21 Oct 2013 14:52:18 -0400
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
Message-ID: <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>


On Oct 21, 2013, at 1:02 PM, Chris Barker <chris.barker at noaa.gov> wrote:

> On Fri, Oct 18, 2013 at 6:22 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> -- it would be very useful if folks could easily
>>> get binary wheels for OS-X
>> 
>> We do plan to support it, but the pip devs uncovered a hole in the current
>> wheel spec that means it generates the same filename on *nix systems for
>> wheels that need to have different names for the download side of things to
>> work properly
> 
> THanks -- but really? don't OS-X wheels get:
> 
> macosx_10_6_intel
> 
> or some such tacked on? Where does that go wrong?

Homebrew, Mac Ports, Fink. That would work OK if nobody ever installed things
that the system didn't provide.

> 
>> Once ensurepip has landed in Python 3.4 and pip 1.5 is released, we should
>> be able to get back to updating the various metadata specs, with the aim of
>> getting cross-platform wheel support in pip 1.6 :)
> 
> Getting there... thanks,
> 
> -Chris
> 
> -- 
> 
> Christopher Barker, Ph.D.
> Oceanographer
> 
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
> 
> Chris.Barker at noaa.gov
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131021/22a962e8/attachment.sig>

From p.f.moore at gmail.com  Tue Oct 22 11:02:16 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 22 Oct 2013 10:02:16 +0100
Subject: [Distutils] distlib 0.1.3 released on PyPI
In-Reply-To: <loom.20131018T193505-8@post.gmane.org>
References: <loom.20131018T193505-8@post.gmane.org>
Message-ID: <CACac1F88CkfQZ6qU1bFEOADy8MwcsqD7jgzn+ydf2VqM0OPd4w@mail.gmail.com>

On 18 October 2013 18:44, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> version.py:
>
>     Added support for PEP 440 version matching.
>
>     Removed AdaptiveVersion, AdaptiveMatcher etc. as they don't add
>     sufficient value to justify keeping them in.

AFAICT, you removed Version.suggest_normalized_version(), which breaks
pip if we re-vendor to this version.

Paul

From vinay_sajip at yahoo.co.uk  Tue Oct 22 11:54:58 2013
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 22 Oct 2013 10:54:58 +0100 (BST)
Subject: [Distutils] distlib 0.1.3 released on PyPI
In-Reply-To: <CACac1F88CkfQZ6qU1bFEOADy8MwcsqD7jgzn+ydf2VqM0OPd4w@mail.gmail.com>
Message-ID: <1382435698.93433.YahooMailBasic@web171406.mail.ir2.yahoo.com>

No, it's just been renamed to _suggest_normalized_version. Public access is now via the version scheme.

Regards,

Vinay Sajip
--------------------------------------------
On Tue, 22/10/13, Paul Moore <p.f.moore at gmail.com> wrote:

 Subject: Re: [Distutils] distlib 0.1.3 released on PyPI
 To: "Vinay Sajip" <vinay_sajip at yahoo.co.uk>
 Cc: "Distutils" <Distutils-Sig at python.org>
 Date: Tuesday, 22 October, 2013, 10:02
 
 On 18 October 2013 18:44, Vinay Sajip
 <vinay_sajip at yahoo.co.uk>
 wrote:
 > version.py:
 >
 >? ???Added support for PEP 440
 version matching.
 >
 >? ???Removed AdaptiveVersion,
 AdaptiveMatcher etc. as they don't add
 >? ???sufficient value to justify
 keeping them in.
 
 AFAICT, you removed Version.suggest_normalized_version(),
 which breaks
 pip if we re-vendor to this version.
 
 Paul
 

From p.f.moore at gmail.com  Tue Oct 22 13:26:31 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 22 Oct 2013 12:26:31 +0100
Subject: [Distutils] distlib 0.1.3 released on PyPI
In-Reply-To: <1382435698.93433.YahooMailBasic@web171406.mail.ir2.yahoo.com>
References: <CACac1F88CkfQZ6qU1bFEOADy8MwcsqD7jgzn+ydf2VqM0OPd4w@mail.gmail.com>
 <1382435698.93433.YahooMailBasic@web171406.mail.ir2.yahoo.com>
Message-ID: <CACac1F8ETqTOyNpgrwL2+r+vgG0BGp4SOSME3D4iY5YE_iH+bQ@mail.gmail.com>

On 22 October 2013 10:54, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>>  AFAICT, you removed Version.suggest_normalized_version(),
>>  which breaks pip if we re-vendor to this version.
> No, it's just been renamed to _suggest_normalized_version. Public access is now via the version scheme.

OK, so it's now private. Is pip OK to use it (just add the underscore
and move on)? I guess so, as I don't really feel like rewriting those
bits of code use the new API. I keep getting sidetracked by stuff like
this.

Paul

From ncoghlan at gmail.com  Tue Oct 22 14:05:26 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 22 Oct 2013 22:05:26 +1000
Subject: [Distutils] Fwd: PEP 453 (ensurepip) updated
Message-ID: <CADiSq7fZ7o2LibOsyxxH6TMUq3=G69fE+iqzPoWFfecTPiDqJw@mail.gmail.com>

For those distutils-sig residents that don't follow python-dev, a
certain PEP was formally accepted today :)

---------- Forwarded message ----------
From: "Martin v. L?wis" <martin at v.loewis.de>
Date: 22 October 2013 19:33
Subject: Re: [Python-Dev] PEP 453 (ensurepip) updated
To: Nick Coghlan <ncoghlan at gmail.com>, "python-dev at python.org"
<python-dev at python.org>


Am 20.10.13 14:27, schrieb Nick Coghlan:
> I have posted the latest version of PEP 453 to python.org.

This version looks good to me, and I accept it for inclusion in Python 3.4.

I'd like to thank Nick for carefully editing this PEP, and I'd like to
cite it as an archetype for a well-written PEP. It's very precise, and
it elaborates on rejected proposals and the motivation for rejection.

I'd also like to thank Donald for pushing this, and for continued work
on the implementation of the PEP. I see that this all took longer than
expected (also due to my fault in providing timely reviews). I suggest
that some of the stuff that needs to be done still is delegated, so that
Donald doesn't have to do all of it.

Regards,
Martin



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Tue Oct 22 15:27:46 2013
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 22 Oct 2013 14:27:46 +0100 (BST)
Subject: [Distutils] distlib 0.1.3 released on PyPI
In-Reply-To: <CACac1F8ETqTOyNpgrwL2+r+vgG0BGp4SOSME3D4iY5YE_iH+bQ@mail.gmail.com>
Message-ID: <1382448466.30173.YahooMailBasic@web171404.mail.ir2.yahoo.com>

Yes, you can use it that way - sorry for any inconvenience.

Regards,

Vinay Sajip
--------------------------------------------
On Tue, 22/10/13, Paul Moore <p.f.moore at gmail.com> wrote:

 Subject: Re: [Distutils] distlib 0.1.3 released on PyPI
 To: "Vinay Sajip" <vinay_sajip at yahoo.co.uk>
 Cc: "Distutils" <Distutils-Sig at python.org>
 Date: Tuesday, 22 October, 2013, 12:26
 
 On 22 October 2013 10:54, Vinay Sajip
 <vinay_sajip at yahoo.co.uk>
 wrote:
 >>? AFAICT, you removed
 Version.suggest_normalized_version(),
 >>? which breaks pip if we re-vendor to this
 version.
 > No, it's just been renamed to
 _suggest_normalized_version. Public access is now via the
 version scheme.
 
 OK, so it's now private. Is pip OK to use it (just add the
 underscore
 and move on)? I guess so, as I don't really feel like
 rewriting those
 bits of code use the new API. I keep getting sidetracked by
 stuff like
 this.
 
 Paul
 

From p.f.moore at gmail.com  Tue Oct 22 15:34:39 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 22 Oct 2013 14:34:39 +0100
Subject: [Distutils] distlib 0.1.3 released on PyPI
In-Reply-To: <1382448466.30173.YahooMailBasic@web171404.mail.ir2.yahoo.com>
References: <CACac1F8ETqTOyNpgrwL2+r+vgG0BGp4SOSME3D4iY5YE_iH+bQ@mail.gmail.com>
 <1382448466.30173.YahooMailBasic@web171404.mail.ir2.yahoo.com>
Message-ID: <CACac1F-McGuc87ETvTziV-krFQe9X4QiEKFdu3cqda5GWESHQA@mail.gmail.com>

On 22 October 2013 14:27, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Yes, you can use it that way - sorry for any inconvenience.

No problem, it's what we get for using a library that's not reached a
stable version yet :-)
Paul

From chris.barker at noaa.gov  Tue Oct 22 21:34:10 2013
From: chris.barker at noaa.gov (Chris Barker)
Date: Tue, 22 Oct 2013 12:34:10 -0700
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
 <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
Message-ID: <CALGmxEJeOra740T=3B6xBo7f6zvxh763Ud3vxLhsRrscM=rMuA@mail.gmail.com>

On Mon, Oct 21, 2013 at 11:52 AM, Donald Stufft <donald at stufft.io> wrote:

>> Thanks -- but really? don't OS-X wheels get:
>>
>> macosx_10_6_intel
>>
>> or some such tacked on? Where does that go wrong?
>
> Homebrew, Mac Ports, Fink. That would work OK if nobody ever installed things
> that the system didn't provide.

OK -- yes, that will NEVER work. It's worse than the Linux situation.

But then, it's the same everywhere -- if someone builds a binary wheel
for Windows that depends on some non-standard dll, or is built against
a weirdly custom-built Python, it won't work either.

It's been more or less a consensus in the python-mac community that we
seek to provide binaries for the Python.org pythons, and that they
shouldn't depend on non-standard external libs -- just like on
Windows. Major hard-to-build packages have been doing this for years:

wxPython
numpy, scipy, matplotlib

But these installers often don't work with virtualenv, and can't be
discovered or installed  pip or easy_install.

So I think it would be VERY useful if we established this standard for
PyPi and binary wheels.

macports, fink, and homebrew have been doing their own thing for ages,
and can continue to do so -- they HAVE package management built in,
just like the linux distros. If they want to do wheels, they will need
to make sure that the neccesary info is in the platform-tag. On my
python.org build:

'macosx-10.6-i386'

so they should patch their python to return something like:

'macosx-macports-10.6-i386'

or just:

'macports-10.6-i386'

and probably a macports version, rather than "10.6".

However, the _point_ of macports, homebrew, etc, is that your stuff is
all custom compiled for your system the way you have configured it --
so binary wheels really don't make sense.

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

From ncoghlan at gmail.com  Tue Oct 22 22:19:45 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 23 Oct 2013 06:19:45 +1000
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <CALGmxEJeOra740T=3B6xBo7f6zvxh763Ud3vxLhsRrscM=rMuA@mail.gmail.com>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
 <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
 <CALGmxEJeOra740T=3B6xBo7f6zvxh763Ud3vxLhsRrscM=rMuA@mail.gmail.com>
Message-ID: <CADiSq7cN41HbtLLqw2mDnJOHt=xQ+NTikg8SR9r4MxMKvDosVw@mail.gmail.com>

On 23 Oct 2013 05:42, "Chris Barker" <chris.barker at noaa.gov> wrote:
>
> On Mon, Oct 21, 2013 at 11:52 AM, Donald Stufft <donald at stufft.io> wrote:
>
> >> Thanks -- but really? don't OS-X wheels get:
> >>
> >> macosx_10_6_intel
> >>
> >> or some such tacked on? Where does that go wrong?
> >
> > Homebrew, Mac Ports, Fink. That would work OK if nobody ever installed
things
> > that the system didn't provide.
>
> OK -- yes, that will NEVER work. It's worse than the Linux situation.
>
> But then, it's the same everywhere -- if someone builds a binary wheel
> for Windows that depends on some non-standard dll, or is built against
> a weirdly custom-built Python, it won't work either.
>
> It's been more or less a consensus in the python-mac community that we
> seek to provide binaries for the Python.org pythons, and that they
> shouldn't depend on non-standard external libs -- just like on
> Windows. Major hard-to-build packages have been doing this for years:
>
> wxPython
> numpy, scipy, matplotlib
>
> But these installers often don't work with virtualenv, and can't be
> discovered or installed  pip or easy_install.
>
> So I think it would be VERY useful if we established this standard for
> PyPi and binary wheels.
>
> macports, fink, and homebrew have been doing their own thing for ages,
> and can continue to do so -- they HAVE package management built in,
> just like the linux distros. If they want to do wheels, they will need
> to make sure that the neccesary info is in the platform-tag. On my
> python.org build:
>
> 'macosx-10.6-i386'
>
> so they should patch their python to return something like:
>
> 'macosx-macports-10.6-i386'
>
> or just:
>
> 'macports-10.6-i386'
>
> and probably a macports version, rather than "10.6".
>
> However, the _point_ of macports, homebrew, etc, is that your stuff is
> all custom compiled for your system the way you have configured it --
> so binary wheels really don't make sense.

PEP 453 has had most of my attention lately, but my tentative thought has
been to introduce a relatively freeform "variant" field to the wheel spec.

Windows and Mac OS X would then default to an empty variant, while other
*nix systems would require a nominated variant

That would then cover things like SSE builds on Windows, alternative
sources on Mac OS X, etc.

However, worrying about that is a fair way down the todo list until
ensurepip and the CPython doc updates for PEP 453 are done, along with
everything else that has a 3.4b1 deadline :)

Cheers,
Nick.

>
> -Chris
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131023/c0b51b93/attachment-0001.html>

From chris.barker at noaa.gov  Tue Oct 22 23:23:13 2013
From: chris.barker at noaa.gov (Chris Barker)
Date: Tue, 22 Oct 2013 14:23:13 -0700
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <CADiSq7cN41HbtLLqw2mDnJOHt=xQ+NTikg8SR9r4MxMKvDosVw@mail.gmail.com>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
 <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
 <CALGmxEJeOra740T=3B6xBo7f6zvxh763Ud3vxLhsRrscM=rMuA@mail.gmail.com>
 <CADiSq7cN41HbtLLqw2mDnJOHt=xQ+NTikg8SR9r4MxMKvDosVw@mail.gmail.com>
Message-ID: <CALGmxEJ16Jx=e9h+ukbT69WSFr=4t4P1xovimPSzhP4Oi29K4Q@mail.gmail.com>

On Tue, Oct 22, 2013 at 1:19 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> PEP 453 has had most of my attention lately, but my tentative thought has
> been to introduce a relatively freeform "variant" field to the wheel spec.
>
> Windows and Mac OS X would then default to an empty variant, while other
> *nix systems would require a nominated variant
>
> That would then cover things like SSE builds on Windows, alternative sources
> on Mac OS X, etc.

sounds good.


> However, worrying about that is a fair way down the todo list until
> ensurepip and the CPython doc updates for PEP 453 are done, along with
> everything else that has a 3.4b1 deadline :)

Fair enough, but enabling binary wheels for OS-X (the pyton.org build,
NOT macports and friends) would be great, and shouldn't take much
work, yes?

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

From richard at python.org  Wed Oct 23 01:34:04 2013
From: richard at python.org (Richard Jones)
Date: Wed, 23 Oct 2013 10:34:04 +1100
Subject: [Distutils] Fwd: PEP 453 (ensurepip) updated
In-Reply-To: <CADiSq7fZ7o2LibOsyxxH6TMUq3=G69fE+iqzPoWFfecTPiDqJw@mail.gmail.com>
References: <CADiSq7fZ7o2LibOsyxxH6TMUq3=G69fE+iqzPoWFfecTPiDqJw@mail.gmail.com>
Message-ID: <CAHrZfZBe5LWQ1fkmijG144bUBMknZ8Auerz_tyjyY+NQHpYYCA@mail.gmail.com>

Yay! Thanks everyone involved, especially Donald and Nick.


On 22 October 2013 23:05, Nick Coghlan <ncoghlan at gmail.com> wrote:

> For those distutils-sig residents that don't follow python-dev, a
> certain PEP was formally accepted today :)
>
> ---------- Forwarded message ----------
> From: "Martin v. L?wis" <martin at v.loewis.de>
> Date: 22 October 2013 19:33
> Subject: Re: [Python-Dev] PEP 453 (ensurepip) updated
> To: Nick Coghlan <ncoghlan at gmail.com>, "python-dev at python.org"
> <python-dev at python.org>
>
>
> Am 20.10.13 14:27, schrieb Nick Coghlan:
> > I have posted the latest version of PEP 453 to python.org.
>
> This version looks good to me, and I accept it for inclusion in Python 3.4.
>
> I'd like to thank Nick for carefully editing this PEP, and I'd like to
> cite it as an archetype for a well-written PEP. It's very precise, and
> it elaborates on rejected proposals and the motivation for rejection.
>
> I'd also like to thank Donald for pushing this, and for continued work
> on the implementation of the PEP. I see that this all took longer than
> expected (also due to my fault in providing timely reviews). I suggest
> that some of the stuff that needs to be done still is delegated, so that
> Donald doesn't have to do all of it.
>
> Regards,
> Martin
>
>
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131023/02f38366/attachment.html>

From tk47 at students.poly.edu  Wed Oct 23 04:10:05 2013
From: tk47 at students.poly.edu (Trishank Karthik Kuppusamy)
Date: Tue, 22 Oct 2013 22:10:05 -0400
Subject: [Distutils] Fwd: PEP 453 (ensurepip) updated
In-Reply-To: <CAHrZfZBe5LWQ1fkmijG144bUBMknZ8Auerz_tyjyY+NQHpYYCA@mail.gmail.com>
References: <CADiSq7fZ7o2LibOsyxxH6TMUq3=G69fE+iqzPoWFfecTPiDqJw@mail.gmail.com>
 <CAHrZfZBe5LWQ1fkmijG144bUBMknZ8Auerz_tyjyY+NQHpYYCA@mail.gmail.com>
Message-ID: <52672FFD.6000806@students.poly.edu>

On Tue 22 Oct 2013 07:34:04 PM EDT, Richard Jones wrote:
> Yay! Thanks everyone involved, especially Donald and Nick.
>

Congrats :)

From g2p.code at gmail.com  Thu Oct 24 17:14:44 2013
From: g2p.code at gmail.com (Gabriel de Perthuis)
Date: Thu, 24 Oct 2013 15:14:44 +0000 (UTC)
Subject: [Distutils] PyPI audit log
Message-ID: <l4bdh4$u6b$1@ger.gmane.org>

Hello,
The IP log that shows who did what on PyPI pages shows
some IPs that have no business being there; they seem
to come from CDNs, in my case fastly and wikia.
Can this be fixed, to make this audit log useful again?



From dominique.orban at gmail.com  Thu Oct 24 22:04:27 2013
From: dominique.orban at gmail.com (Dominique Orban)
Date: Thu, 24 Oct 2013 16:04:27 -0400
Subject: [Distutils] AttributeError: 'tuple' object has no attribute 'split'
Message-ID: <etPan.52697d87.1d0a04e.8d60@Aragorn>


I hope this is the right place to ask for help. I'm not finding much comfort in the PyPi documentation or in Google searches. I uploaded my package `pykrylov` with `python setup.py sdist upload`. Installing it locally with `python setup.py` install works fine but `pip install pykrylov` breaks with the messages below. I since removed it from PyPI but I get the same error message if I try installing from the git repository. I'm hoping someone can put me on track as I've no idea what's wrong. You can see my setup.py here:

https://github.com/dpo/pykrylov/blob/ea553cdb287f6e685406ceadcb297fd6704af52d/setup.py

I'm using Python 2.7.5 on OSX installed with Homebrew and pip 1.4.1. Attempts to upgrade setuptools or pip result in another error message (AttributeError: 'str' object has no attribute 'rollback')...

Thanks!

--?
Dominique


$ pip install -e git://github.com/dpo/pykrylov.git at ea553cd#egg=pykrylov
Obtaining pykrylov from git+git://github.com/dpo/pykrylov.git at ea553cd#egg=pykrylov
? Updating ./src/pykrylov clone (to ea553cd)
? Could not find a tag or branch 'ea553cd', assuming commit.
? Running setup.py egg_info for package pykrylov
? ? build_src
? ? Traceback (most recent call last):
? ? ? File "<string>", line 16, in <module>
? ? ? File "/Users/dpo/.virtualenvs/test/src/pykrylov/setup.py", line 77, in <module>
? ? ? ? setup_package()
? ? ? File "/Users/dpo/.virtualenvs/test/src/pykrylov/setup.py", line 69, in setup_package
? ? ? ? configuration=configuration )
? ? ? File "/usr/local/lib/python2.7/site-packages/numpy/distutils/core.py", line 186, in setup
? ? ? ? return old_setup(**new_attr)
? ? ? File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup
? ? ? ? dist.run_commands()
? ? ? File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
? ? ? ? self.run_command(cmd)
? ? ? File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
? ? ? ? cmd_obj.run()
? ? ? File "/usr/local/lib/python2.7/site-packages/numpy/distutils/command/egg_info.py", line 9, in run
? ? ? ? _egg_info.run(self)
? ? ? File "<string>", line 13, in replacement_run
? ? ? File "/Users/dpo/.virtualenvs/test/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 441, in write_toplevel_names
? ? ? ? for k in cmd.distribution.iter_distribution_names()
? ? AttributeError: 'tuple' object has no attribute 'split'
? ? Complete output from command python setup.py egg_info:
? ? running egg_info

running build_src

build_src

writing pykrylov.egg-info/PKG-INFO

Traceback (most recent call last):

? File "<string>", line 16, in <module>

? File "/Users/dpo/.virtualenvs/test/src/pykrylov/setup.py", line 77, in <module>

? ? setup_package()

? File "/Users/dpo/.virtualenvs/test/src/pykrylov/setup.py", line 69, in setup_package

? ? configuration=configuration )

? File "/usr/local/lib/python2.7/site-packages/numpy/distutils/core.py", line 186, in setup

? ? return old_setup(**new_attr)

? File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup

? ? dist.run_commands()

? File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands

? ? self.run_command(cmd)

? File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command

? ? cmd_obj.run()

? File "/usr/local/lib/python2.7/site-packages/numpy/distutils/command/egg_info.py", line 9, in run

? ? _egg_info.run(self)

? File "<string>", line 13, in replacement_run

? File "/Users/dpo/.virtualenvs/test/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 441, in write_toplevel_names

? ? for k in cmd.distribution.iter_distribution_names()

AttributeError: 'tuple' object has no attribute 'split'

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /Users/dpo/.virtualenvs/test/src/pykrylov
Storing complete log in /Users/dpo/.pip/pip.log



From richard at python.org  Fri Oct 25 01:50:28 2013
From: richard at python.org (Richard Jones)
Date: Fri, 25 Oct 2013 10:50:28 +1100
Subject: [Distutils] PyPI audit log
In-Reply-To: <l4bdh4$u6b$1@ger.gmane.org>
References: <l4bdh4$u6b$1@ger.gmane.org>
Message-ID: <CAHrZfZBt0UGrFwJXQzKXeNa_V26zNv0xaqV3HcVk3G2KR5-uMg@mail.gmail.com>

Thanks for noticing, we'll have a look.


     Richard


On 25 October 2013 02:14, Gabriel de Perthuis <g2p.code at gmail.com> wrote:

> Hello,
> The IP log that shows who did what on PyPI pages shows
> some IPs that have no business being there; they seem
> to come from CDNs, in my case fastly and wikia.
> Can this be fixed, to make this audit log useful again?
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131025/4bf3aeed/attachment.html>

From pje at telecommunity.com  Fri Oct 25 04:25:42 2013
From: pje at telecommunity.com (PJ Eby)
Date: Thu, 24 Oct 2013 22:25:42 -0400
Subject: [Distutils] Egg name computation
In-Reply-To: <525D2FE9.6050904@intec.ugent.be>
References: <525D2FE9.6050904@intec.ugent.be>
Message-ID: <CALeMXf53xTaCyVOvd6L_yw3VeVojasPvfhJMq46qkc1hVNPJ8g@mail.gmail.com>

On Tue, Oct 15, 2013 at 8:07 AM, Martin Fiers
<Martin.Fiers at intec.ugent.be> wrote:
> So the platform argument now is
> self.distribution.has_ext_modules() and self.plat_name
> Shouldn't it just be
> self.plat_name
> ?

No.  The platform name is only included if the distribution has
extension modules, because extension modules are what make the egg
platform-specific.  If there is only Python code and data, then the
egg is considered platform independent.


> Would there be a workaround?

What do you want to work around this *for*?  If the egg doesn't
contain extension modules, what is platform-specific about it?

From oscar.j.benjamin at gmail.com  Fri Oct 25 15:30:56 2013
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Fri, 25 Oct 2013 14:30:56 +0100
Subject: [Distutils] AttributeError: 'tuple' object has no attribute
	'split'
In-Reply-To: <etPan.52697d87.1d0a04e.8d60@Aragorn>
References: <etPan.52697d87.1d0a04e.8d60@Aragorn>
Message-ID: <CAHVvXxRBC_TtcZwdA2gd=-U5AJvGBWtm8FNAzoTRUdLixic1ng@mail.gmail.com>

On 24 October 2013 21:04, Dominique Orban <dominique.orban at gmail.com> wrote:
>
> I hope this is the right place to ask for help. I'm not finding much comfort in the PyPi documentation or in Google searches. I uploaded my package `pykrylov` with `python setup.py sdist upload`. Installing it locally with `python setup.py` install works fine but `pip install pykrylov` breaks with the messages below. I since removed it from PyPI but I get the same error message if I try installing from the git repository. I'm hoping someone can put me on track as I've no idea what's wrong. You can see my setup.py here:
>
> https://github.com/dpo/pykrylov/blob/ea553cdb287f6e685406ceadcb297fd6704af52d/setup.py
>
> I'm using Python 2.7.5 on OSX installed with Homebrew and pip 1.4.1. Attempts to upgrade setuptools or pip result in another error message (AttributeError: 'str' object has no attribute 'rollback')...

Can you install a more recent setuptools by downloading it and running
the setup.py yourself?
https://pypi.python.org/packages/source/s/setuptools/setuptools-1.1.6.tar.gz


Oscar

From dominique.orban at gmail.com  Fri Oct 25 16:51:56 2013
From: dominique.orban at gmail.com (Dominique Orban)
Date: Fri, 25 Oct 2013 10:51:56 -0400
Subject: [Distutils] AttributeError: 'tuple' object has no attribute
 'split'
In-Reply-To: <CAHVvXxRBC_TtcZwdA2gd=-U5AJvGBWtm8FNAzoTRUdLixic1ng@mail.gmail.com>
References: <etPan.52697d87.1d0a04e.8d60@Aragorn>
 <CAHVvXxRBC_TtcZwdA2gd=-U5AJvGBWtm8FNAzoTRUdLixic1ng@mail.gmail.com>
Message-ID: <etPan.526a85c8.7aed0d9f.8d60@Aragorn>



On 25 October, 2013 at 9:31:16 AM, Oscar Benjamin (oscar.j.benjamin at gmail.com) wrote:
>
>On 24 October 2013 21:04, Dominique Orban wrote:
>>
>> I hope this is the right place to ask for help. I'm not finding much comfort in the PyPi documentation or in Google searches. I uploaded my package `pykrylov` with `python setup.py sdist upload`. Installing it locally with `python setup.py` install works fine but `pip install pykrylov` breaks with the messages below. I since removed it from PyPI but I get the same error message if I try installing from the git repository. I'm hoping someone can put me on track as I've no idea what's wrong. You can see my setup.py here:
>>
>> https://github.com/dpo/pykrylov/blob/ea553cdb287f6e685406ceadcb297fd6704af52d/setup.py
>>
>> I'm using Python 2.7.5 on OSX installed with Homebrew and pip 1.4.1. Attempts to upgrade setuptools or pip result in another error message (AttributeError: 'str' object has no attribute 'rollback')...
>
>Can you install a more recent setuptools by downloading it and running
>the setup.py yourself?
>https://pypi.python.org/packages/source/s/setuptools/setuptools-1.1.6.tar.gz

Thanks for the suggestion. I'm still getting the same error with setuptools 1.1.6. I also tried "upgrading" Numpy (since I'm using Numpy distutils) by installing from their git repository, and I'm still getting the same error.

Is anything obviously wrong with the setup.py?

Thanks again.

--?
Dominique

From dustin at randomingenuity.com  Fri Oct 25 19:22:25 2013
From: dustin at randomingenuity.com (Dustin Oprea)
Date: Fri, 25 Oct 2013 13:22:25 -0400
Subject: [Distutils] Inflated download counts
Message-ID: <CAOr1xOG1mgAFrv5b2_qFuAXwgmCi+VnD2+duKpFmMgr2hG=dTw@mail.gmail.com>

It seems like the download counts on PyPI aren't accurate. Though the
really useful packages seem to have higher numbers than the packages that
only apply to a specific target audience, I'm fairly certain that the
numbers are more affected by robots and such than actual users.

Recently I started a service that requires membership. In the last month,
PyPI reports 3000 downloads of the client, yet Google Analytics only
reports a handful of visits to the website. I have even less membership
signups (as expected, so soon after launch). Why are the download counts so
inflated?

What has to be done to get this to be accurate?

I've included two screenshots of PyPI and GA.



Dustin Oprea
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131025/b463c550/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Selection_001.png
Type: image/png
Size: 10502 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131025/b463c550/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Selection_002.png
Type: image/png
Size: 5389 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131025/b463c550/attachment-0003.png>

From donald at stufft.io  Fri Oct 25 19:49:05 2013
From: donald at stufft.io (Donald Stufft)
Date: Fri, 25 Oct 2013 13:49:05 -0400
Subject: [Distutils] Inflated download counts
In-Reply-To: <CAOr1xOG1mgAFrv5b2_qFuAXwgmCi+VnD2+duKpFmMgr2hG=dTw@mail.gmail.com>
References: <CAOr1xOG1mgAFrv5b2_qFuAXwgmCi+VnD2+duKpFmMgr2hG=dTw@mail.gmail.com>
Message-ID: <204256A1-2135-441F-AF7A-875924D6E571@stufft.io>

Mostly new packages will get roughly 2-3k of downloads from what appears to be
mirroring infrastructure. I?m hesitant to mess with the traffic numbers at all because
I don?t want them to be inaccurate *and* artificial vs just inaccurate (assuming you
think it?s the number of people downloading your project).

On Oct 25, 2013, at 1:22 PM, Dustin Oprea <dustin at randomingenuity.com> wrote:

> It seems like the download counts on PyPI aren't accurate. Though the really useful packages seem to have higher numbers than the packages that only apply to a specific target audience, I'm fairly certain that the numbers are more affected by robots and such than actual users.
> 
> Recently I started a service that requires membership. In the last month, PyPI reports 3000 downloads of the client, yet Google Analytics only reports a handful of visits to the website. I have even less membership signups (as expected, so soon after launch). Why are the download counts so inflated?
> 
> What has to be done to get this to be accurate?
> 
> I've included two screenshots of PyPI and GA.
> 
> 
> 
> Dustin Oprea
> <Selection_001.png><Selection_002.png>_______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131025/21e21968/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131025/21e21968/attachment.sig>

From holger at merlinux.eu  Fri Oct 25 19:57:56 2013
From: holger at merlinux.eu (holger krekel)
Date: Fri, 25 Oct 2013 17:57:56 +0000
Subject: [Distutils] Inflated download counts
In-Reply-To: <204256A1-2135-441F-AF7A-875924D6E571@stufft.io>
References: <CAOr1xOG1mgAFrv5b2_qFuAXwgmCi+VnD2+duKpFmMgr2hG=dTw@mail.gmail.com>
 <204256A1-2135-441F-AF7A-875924D6E571@stufft.io>
Message-ID: <20131025175756.GG3973@merlinux.eu>

On Fri, Oct 25, 2013 at 13:49 -0400, Donald Stufft wrote:
> Mostly new packages will get roughly 2-3k of downloads from what appears to be
> mirroring infrastructure. I?m hesitant to mess with the traffic numbers at all because
> I don?t want them to be inaccurate *and* artificial vs just inaccurate (assuming you
> think it?s the number of people downloading your project).

Is it not possible that the analysis code or Fastly's delivery of logs 
has bugs?  The "inflation" problem only happens sometimes after all.
If it were mirroring infrastructure it should be more consistent.

holger

> On Oct 25, 2013, at 1:22 PM, Dustin Oprea <dustin at randomingenuity.com> wrote:
> 
> > It seems like the download counts on PyPI aren't accurate. Though the really useful packages seem to have higher numbers than the packages that only apply to a specific target audience, I'm fairly certain that the numbers are more affected by robots and such than actual users.
> > 
> > Recently I started a service that requires membership. In the last month, PyPI reports 3000 downloads of the client, yet Google Analytics only reports a handful of visits to the website. I have even less membership signups (as expected, so soon after launch). Why are the download counts so inflated?
> > 
> > What has to be done to get this to be accurate?
> > 
> > I've included two screenshots of PyPI and GA.
> > 
> > 
> > 
> > Dustin Oprea
> > <Selection_001.png><Selection_002.png>_______________________________________________
> > Distutils-SIG maillist  -  Distutils-SIG at python.org
> > https://mail.python.org/mailman/listinfo/distutils-sig
> 
> 
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> 



> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


From donald at stufft.io  Fri Oct 25 19:59:36 2013
From: donald at stufft.io (Donald Stufft)
Date: Fri, 25 Oct 2013 13:59:36 -0400
Subject: [Distutils] Inflated download counts
In-Reply-To: <20131025175756.GG3973@merlinux.eu>
References: <CAOr1xOG1mgAFrv5b2_qFuAXwgmCi+VnD2+duKpFmMgr2hG=dTw@mail.gmail.com>
 <204256A1-2135-441F-AF7A-875924D6E571@stufft.io>
 <20131025175756.GG3973@merlinux.eu>
Message-ID: <B4D87287-4E16-4B04-904A-C21B65AA9F10@stufft.io>


On Oct 25, 2013, at 1:57 PM, holger krekel <holger at merlinux.eu> wrote:

> Is it not possible that the analysis code or Fastly's delivery of logs 
> has bugs?  The "inflation" problem only happens sometimes after all.
> If it were mirroring infrastructure it should be more consistent.

Sure it?s possible. The Analysis code is OSS in the pypi repo, the log delivery
is just done via syslog. I have no idea how Fastly creates those logs though.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131025/159f30a4/attachment.sig>

From oscar.j.benjamin at gmail.com  Fri Oct 25 19:56:26 2013
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Fri, 25 Oct 2013 18:56:26 +0100
Subject: [Distutils] AttributeError: 'tuple' object has no attribute
	'split'
In-Reply-To: <etPan.526a85c8.7aed0d9f.8d60@Aragorn>
References: <etPan.52697d87.1d0a04e.8d60@Aragorn>
 <CAHVvXxRBC_TtcZwdA2gd=-U5AJvGBWtm8FNAzoTRUdLixic1ng@mail.gmail.com>
 <etPan.526a85c8.7aed0d9f.8d60@Aragorn>
Message-ID: <CAHVvXxTpMDMhU2Q96K6e++2JGqCLp1RWAFKBSr24P29-y+1qSA@mail.gmail.com>

On Oct 25, 2013 3:52 PM, "Dominique Orban" <dominique.orban at gmail.com>
wrote:
>
>
>
> On 25 October, 2013 at 9:31:16 AM, Oscar Benjamin (
oscar.j.benjamin at gmail.com) wrote:
> >
> >On 24 October 2013 21:04, Dominique Orban wrote:
> >>
> >> I hope this is the right place to ask for help. I'm not finding much
comfort in the PyPi documentation or in Google searches. I uploaded my
package `pykrylov` with `python setup.py sdist upload`. Installing it
locally with `python setup.py` install works fine but `pip install
pykrylov` breaks with the messages below. I since removed it from PyPI but
I get the same error message if I try installing from the git repository.
I'm hoping someone can put me on track as I've no idea what's wrong. You
can see my setup.py here:
> >>
> >>
https://github.com/dpo/pykrylov/blob/ea553cdb287f6e685406ceadcb297fd6704af52d/setup.py
> >>
> >> I'm using Python 2.7.5 on OSX installed with Homebrew and pip 1.4.1.
Attempts to upgrade setuptools or pip result in another error message
(AttributeError: 'str' object has no attribute 'rollback')...
> >
> >Can you install a more recent setuptools by downloading it and running
> >the setup.py yourself?
> >
https://pypi.python.org/packages/source/s/setuptools/setuptools-1.1.6.tar.gz
>
> Thanks for the suggestion. I'm still getting the same error with
setuptools 1.1.6. I also tried "upgrading" Numpy (since I'm using Numpy
distutils) by installing from their git repository, and I'm still getting
the same error.
>
> Is anything obviously wrong with the setup.py?

I don't know but I'm not totally clear what you mean. Previously you
described multiple problems: with pip, setuptools and pykrylov. Have you
successfully installed setuptools now?

If the "same error" is with pykrylov's setup.py have you tried debugging
it? E.g. 'python -m pdb setup.py install'

Oscar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131025/c495d293/attachment-0001.html>

From dominique.orban at gmail.com  Fri Oct 25 20:05:32 2013
From: dominique.orban at gmail.com (Dominique Orban)
Date: Fri, 25 Oct 2013 14:05:32 -0400
Subject: [Distutils] AttributeError: 'tuple' object has no attribute
 'split'
In-Reply-To: <CAHVvXxTpMDMhU2Q96K6e++2JGqCLp1RWAFKBSr24P29-y+1qSA@mail.gmail.com>
References: <etPan.52697d87.1d0a04e.8d60@Aragorn>
 <CAHVvXxRBC_TtcZwdA2gd=-U5AJvGBWtm8FNAzoTRUdLixic1ng@mail.gmail.com>
 <etPan.526a85c8.7aed0d9f.8d60@Aragorn>
 <CAHVvXxTpMDMhU2Q96K6e++2JGqCLp1RWAFKBSr24P29-y+1qSA@mail.gmail.com>
Message-ID: <etPan.526ab328.48945699.8d60@Aragorn>



On 25 October, 2013 at 1:56:26 PM, Oscar Benjamin (oscar.j.benjamin at gmail.com) wrote:
>
>On Oct 25, 2013 3:52 PM, "Dominique Orban"  
>wrote:
>>
>>
>>
>> On 25 October, 2013 at 9:31:16 AM, Oscar Benjamin (
>oscar.j.benjamin at gmail.com) wrote:
>> >
>> >On 24 October 2013 21:04, Dominique Orban wrote:
>> >>
>> >> I hope this is the right place to ask for help. I'm not finding much
>comfort in the PyPi documentation or in Google searches. I uploaded my
>package `pykrylov` with `python setup.py sdist upload`. Installing it
>locally with `python setup.py` install works fine but `pip install
>pykrylov` breaks with the messages below. I since removed it from PyPI but
>I get the same error message if I try installing from the git repository.
>I'm hoping someone can put me on track as I've no idea what's wrong. You
>can see my setup.py here:
>> >>
>> >>
>https://github.com/dpo/pykrylov/blob/ea553cdb287f6e685406ceadcb297fd6704af52d/setup.py
>> >>
>> >> I'm using Python 2.7.5 on OSX installed with Homebrew and pip 1.4.1.
>Attempts to upgrade setuptools or pip result in another error message
>(AttributeError: 'str' object has no attribute 'rollback')...
>> >
>> >Can you install a more recent setuptools by downloading it and running
>> >the setup.py yourself?
>> >
>https://pypi.python.org/packages/source/s/setuptools/setuptools-1.1.6.tar.gz
>>
>> Thanks for the suggestion. I'm still getting the same error with
>setuptools 1.1.6. I also tried "upgrading" Numpy (since I'm using Numpy
>distutils) by installing from their git repository, and I'm still getting
>the same error.
>>
>> Is anything obviously wrong with the setup.py?
>
>I don't know but I'm not totally clear what you mean. Previously you
>described multiple problems: with pip, setuptools and pykrylov. Have you
>successfully installed setuptools now?
>
>If the "same error" is with pykrylov's setup.py have you tried debugging
>it? E.g. 'python -m pdb setup.py install'

"python setup.py install" works fine. It's the installation with pip that returns the error message I mentioned. I was wondering if something in setup.py didn't agree with pip/setuptools.

Yes I installed setuptools 1.1.6. But "pip install -e git://github.com/dpo/pykrylov.git at ea553cd#egg=pykrylov" still returns "AttributeError: 'tuple' object has no attribute 'split'".

I hope I'm making sense.

--?
Dominique

From myselfasunder at gmail.com  Fri Oct 25 20:04:13 2013
From: myselfasunder at gmail.com (Dustin Oprea)
Date: Fri, 25 Oct 2013 14:04:13 -0400
Subject: [Distutils] Inflated download counts
In-Reply-To: <204256A1-2135-441F-AF7A-875924D6E571@stufft.io>
References: <CAOr1xOG1mgAFrv5b2_qFuAXwgmCi+VnD2+duKpFmMgr2hG=dTw@mail.gmail.com>
 <204256A1-2135-441F-AF7A-875924D6E571@stufft.io>
Message-ID: <CAG0x-ECMR3V0A3z6odJMgt77tDVHzGJg9PZFzLzdnKUy2csU5Q@mail.gmail.com>

Is there any way that we can use the user-agent to either identify users or
identify mirrors?

Can we pass a flag or signature from "pip"? It won't reflect downloads from
website, but this probably won't affect the numbers much. In this case, we
might just reword it to "pip Downloads".

This is a distressing issue. It doesn't seem like package owners have any
usable usage data.

Dustin Oprea
On Oct 25, 2013 1:49 PM, "Donald Stufft" <donald at stufft.io> wrote:

> Mostly new packages will get roughly 2-3k of downloads from what appears
> to be
> mirroring infrastructure. I?m hesitant to mess with the traffic numbers at
> all because
> I don?t want them to be inaccurate *and* artificial vs just inaccurate
> (assuming you
> think it?s the number of people downloading your project).
>
> On Oct 25, 2013, at 1:22 PM, Dustin Oprea <dustin at randomingenuity.com>
> wrote:
>
> It seems like the download counts on PyPI aren't accurate. Though the
> really useful packages seem to have higher numbers than the packages that
> only apply to a specific target audience, I'm fairly certain that the
> numbers are more affected by robots and such than actual users.
>
> Recently I started a service that requires membership. In the last month,
> PyPI reports 3000 downloads of the client, yet Google Analytics only
> reports a handful of visits to the website. I have even less membership
> signups (as expected, so soon after launch). Why are the download counts so
> inflated?
>
> What has to be done to get this to be accurate?
>
> I've included two screenshots of PyPI and GA.
>
>
>
> Dustin Oprea
>  <Selection_001.png><Selection_002.png>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
> DCFA
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131025/82cc4eeb/attachment.html>

From ncoghlan at gmail.com  Sat Oct 26 03:35:12 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 26 Oct 2013 11:35:12 +1000
Subject: [Distutils] Inflated download counts
In-Reply-To: <CAG0x-ECMR3V0A3z6odJMgt77tDVHzGJg9PZFzLzdnKUy2csU5Q@mail.gmail.com>
References: <CAOr1xOG1mgAFrv5b2_qFuAXwgmCi+VnD2+duKpFmMgr2hG=dTw@mail.gmail.com>
 <204256A1-2135-441F-AF7A-875924D6E571@stufft.io>
 <CAG0x-ECMR3V0A3z6odJMgt77tDVHzGJg9PZFzLzdnKUy2csU5Q@mail.gmail.com>
Message-ID: <CADiSq7f1MTQWwu_Am4C-OS+qXWhHn8deyUa-bWf+S4VbSy8fZg@mail.gmail.com>

On 26 Oct 2013 04:51, "Dustin Oprea" <myselfasunder at gmail.com> wrote:
>
> Is there any way that we can use the user-agent to either identify users
or identify mirrors?
>
> Can we pass a flag or signature from "pip"? It won't reflect downloads
from website, but this probably won't affect the numbers much. In this
case, we might just reword it to "pip Downloads".
>
> This is a distressing issue. It doesn't seem like package owners have any
usable usage data.

Most downloads happen through the Fastly CDN - the numbers are derived from
the Fastly logs rather than being direct. The code that does that log
analysis is in https://bitbucket.org/pypa/pypi/src (Donald would be able to
provide a more direct reference to the relevant source).

However, separating downloads between mirroring, automatic deployments and
integration and actual direct downloads isn't something PyPI has ever done,
or is really able to do in a systematic way. "pip install thatproject" (and
equivalent commands for other tools) looks the same to PyPI regardless of
whether it's a human or a script running the command.

That's why Donald's recent download analysis was able to split it up by
tools, but not by purpose.

Now, exposing more of that analytical data to package owners on an ongoing
basis is an interesting idea, but one that would be a *very* long way down
the priority list for the current development team.

However, if someone else were to figure out a way to expose the data users
needed to do their own analysis, it might be possible to support that,
although it may be better to look at offering that through Warehouse (aka
PyPI.next) rather than the existing PyPI software (
https://github.com/dstufft/warehouse). There's a demo instance (using live
data) running at preview-pypi.python.org, but that's mostly focused on
backwards compatibility testing for the tool APIs at this point rather than
being navigable through a web browser.

Cheers,
Nick.

>
> Dustin Oprea
>
> On Oct 25, 2013 1:49 PM, "Donald Stufft" <donald at stufft.io> wrote:
>>
>> Mostly new packages will get roughly 2-3k of downloads from what appears
to be
>> mirroring infrastructure. I?m hesitant to mess with the traffic numbers
at all because
>> I don?t want them to be inaccurate *and* artificial vs just inaccurate
(assuming you
>> think it?s the number of people downloading your project).
>>
>> On Oct 25, 2013, at 1:22 PM, Dustin Oprea <dustin at randomingenuity.com>
wrote:
>>
>>> It seems like the download counts on PyPI aren't accurate. Though the
really useful packages seem to have higher numbers than the packages that
only apply to a specific target audience, I'm fairly certain that the
numbers are more affected by robots and such than actual users.
>>>
>>> Recently I started a service that requires membership. In the last
month, PyPI reports 3000 downloads of the client, yet Google Analytics only
reports a handful of visits to the website. I have even less membership
signups (as expected, so soon after launch). Why are the download counts so
inflated?
>>>
>>> What has to be done to get this to be accurate?
>>>
>>> I've included two screenshots of PyPI and GA.
>>>
>>>
>>>
>>> Dustin Oprea
>>>
<Selection_001.png><Selection_002.png>_______________________________________________
>>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>>
>>
>> -----------------
>> Donald Stufft
>> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
DCFA
>>
>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131026/4a659f5e/attachment-0001.html>

From donald at stufft.io  Sat Oct 26 05:03:46 2013
From: donald at stufft.io (Donald Stufft)
Date: Fri, 25 Oct 2013 23:03:46 -0400
Subject: [Distutils] Inflated download counts
In-Reply-To: <CADiSq7f1MTQWwu_Am4C-OS+qXWhHn8deyUa-bWf+S4VbSy8fZg@mail.gmail.com>
References: <CAOr1xOG1mgAFrv5b2_qFuAXwgmCi+VnD2+duKpFmMgr2hG=dTw@mail.gmail.com>
 <204256A1-2135-441F-AF7A-875924D6E571@stufft.io>
 <CAG0x-ECMR3V0A3z6odJMgt77tDVHzGJg9PZFzLzdnKUy2csU5Q@mail.gmail.com>
 <CADiSq7f1MTQWwu_Am4C-OS+qXWhHn8deyUa-bWf+S4VbSy8fZg@mail.gmail.com>
Message-ID: <7CEA60A1-5D72-48EE-9152-C1ED6C091BC5@stufft.io>


On Oct 25, 2013, at 9:35 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> Most downloads happen through the Fastly CDN - the numbers are derived from the Fastly logs rather than being direct. The code that does that log analysis is in https://bitbucket.org/pypa/pypi/src (Donald would be able to provide a more direct reference to the relevant source).
> 

https://bitbucket.org/pypa/pypi/src/0c749f947b167b6643ed94ceac2b3e1ab478d5bd/tools/rsyslog-cdn.py?at=default

It buckets data and puts it into redis. Really ?dumb? but it works well enough until better infrastructure can be put into place.
> However, separating downloads between mirroring, automatic deployments and integration and actual direct downloads isn't something PyPI has ever done, or is really able to do in a systematic way. "pip install thatproject" (and equivalent commands for other tools) looks the same to PyPI regardless of whether it's a human or a script running the command.
> 
> That's why Donald's recent download analysis was able to split it up by tools, but not by purpose.
> 
Yea this part is hard/impossible :/
> Now, exposing more of that analytical data to package owners on an ongoing basis is an interesting idea, but one that would be a *very* long way down the priority list for the current development team.
> 

Nice analytics for package owners is on the road map, but it?s, as you mentioned, down the road map a ways.
> However, if someone else were to figure out a way to expose the data users needed to do their own analysis, it might be possible to support that, although it may be better to look at offering that through Warehouse (aka PyPI.next) rather than the existing PyPI software (https://github.com/dstufft/warehouse). There's a demo instance (using live data) running at preview-pypi.python.org, but that's mostly focused on backwards compatibility testing for the tool APIs at this point rather than being navigable through a web browser.
> 

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131025/219acc75/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131025/219acc75/attachment.sig>

From donald at stufft.io  Sun Oct 27 03:14:59 2013
From: donald at stufft.io (Donald Stufft)
Date: Sat, 26 Oct 2013 22:14:59 -0400
Subject: [Distutils] Removing dependency_links
Message-ID: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>

I would like to remove dependency_links from pip, and ideally
also setuptools.

In implementing the ensurepip module from PEP453 I realized that
even with the ``--no-index`` flag pip was still attempting to
reach the internet. After a little bit of investigation I realized
that the reason for this was setuptools use of dependency links.
From my investigation it appears that setuptools uses these in order
to enable secure automatic installation of the ssl dependencies on
Python < 2.6.

Overall this feature is a security concern, a malicous package could
"pin" any package they want by depending on it and adding a dependency
link a version 100000. This would be more or less transparent to
the end user.

I was looking to see what sort of impact this would have. There are
currently 167,796 source files hosted on PyPI and of those files
4,005 of them have any dependency links at all. Looking at it a
different way, there are 36,070 total projects on PyPI and 411 of them
use this feature. So this is ~2% of the files or ~1% of the projects.

So it appears that this isn't a particularly popular feature, I believe
that it is a *bad* idea that inverts the expected control and should
be removed from both pip and setuptools. In setuptools case it does use
it in the only reasonable way I can imagine, however I think setuptools
should just stop trying to automatically install those dependencies
for Pythons < 2.6 and similarly to pip just print an error and expect users
to get and install them on their own. As a reminder there are very
few downloads from PyPI that are from Pythons < 2.6 [1]

[1] https://caremad.io/blog/a-look-at-pypi-downloads/
[2] https://gist.github.com/dstufft/7173539

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131026/591bf231/attachment.sig>

From donald at stufft.io  Sun Oct 27 03:59:55 2013
From: donald at stufft.io (Donald Stufft)
Date: Sat, 26 Oct 2013 22:59:55 -0400
Subject: [Distutils] Removing dependency_links
In-Reply-To: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
Message-ID: <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>


On Oct 26, 2013, at 10:14 PM, Donald Stufft <donald at stufft.io> wrote:

> I would like to remove dependency_links from pip, and ideally
> also setuptools.
> 
> In implementing the ensurepip module from PEP453 I realized that
> even with the ``--no-index`` flag pip was still attempting to
> reach the internet. After a little bit of investigation I realized
> that the reason for this was setuptools use of dependency links.
> From my investigation it appears that setuptools uses these in order
> to enable secure automatic installation of the ssl dependencies on
> Python < 2.6.
> 
> Overall this feature is a security concern, a malicous package could
> "pin" any package they want by depending on it and adding a dependency
> link a version 100000. This would be more or less transparent to
> the end user.
> 
> I was looking to see what sort of impact this would have. There are
> currently 167,796 source files hosted on PyPI and of those files
> 4,005 of them have any dependency links at all. Looking at it a
> different way, there are 36,070 total projects on PyPI and 411 of them
> use this feature. So this is ~2% of the files or ~1% of the projects.
> 
> So it appears that this isn't a particularly popular feature, I believe
> that it is a *bad* idea that inverts the expected control and should
> be removed from both pip and setuptools. In setuptools case it does use
> it in the only reasonable way I can imagine, however I think setuptools
> should just stop trying to automatically install those dependencies
> for Pythons < 2.6 and similarly to pip just print an error and expect users
> to get and install them on their own. As a reminder there are very
> few downloads from PyPI that are from Pythons < 2.6 [1]
> 
> [1] https://caremad.io/blog/a-look-at-pypi-downloads/
> [2] https://gist.github.com/dstufft/7173539
> 
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

A list of projects that use dependency links: https://gist.github.com/dstufft/7177500

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131026/b79bffaf/attachment.sig>

From donald at stufft.io  Sun Oct 27 04:00:21 2013
From: donald at stufft.io (Donald Stufft)
Date: Sat, 26 Oct 2013 23:00:21 -0400
Subject: [Distutils] Removing dependency_links
In-Reply-To: <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
Message-ID: <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>

Bleh scratch that, it was adding everything :(

On Oct 26, 2013, at 10:59 PM, Donald Stufft <donald at stufft.io> wrote:

> 
> On Oct 26, 2013, at 10:14 PM, Donald Stufft <donald at stufft.io> wrote:
> 
>> I would like to remove dependency_links from pip, and ideally
>> also setuptools.
>> 
>> In implementing the ensurepip module from PEP453 I realized that
>> even with the ``--no-index`` flag pip was still attempting to
>> reach the internet. After a little bit of investigation I realized
>> that the reason for this was setuptools use of dependency links.
>> From my investigation it appears that setuptools uses these in order
>> to enable secure automatic installation of the ssl dependencies on
>> Python < 2.6.
>> 
>> Overall this feature is a security concern, a malicous package could
>> "pin" any package they want by depending on it and adding a dependency
>> link a version 100000. This would be more or less transparent to
>> the end user.
>> 
>> I was looking to see what sort of impact this would have. There are
>> currently 167,796 source files hosted on PyPI and of those files
>> 4,005 of them have any dependency links at all. Looking at it a
>> different way, there are 36,070 total projects on PyPI and 411 of them
>> use this feature. So this is ~2% of the files or ~1% of the projects.
>> 
>> So it appears that this isn't a particularly popular feature, I believe
>> that it is a *bad* idea that inverts the expected control and should
>> be removed from both pip and setuptools. In setuptools case it does use
>> it in the only reasonable way I can imagine, however I think setuptools
>> should just stop trying to automatically install those dependencies
>> for Pythons < 2.6 and similarly to pip just print an error and expect users
>> to get and install them on their own. As a reminder there are very
>> few downloads from PyPI that are from Pythons < 2.6 [1]
>> 
>> [1] https://caremad.io/blog/a-look-at-pypi-downloads/
>> [2] https://gist.github.com/dstufft/7173539
>> 
>> -----------------
>> Donald Stufft
>> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>> 
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
> 
> A list of projects that use dependency links: https://gist.github.com/dstufft/7177500
> 
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> 


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131026/e3f0dfa9/attachment.sig>

From donald at stufft.io  Sun Oct 27 04:59:15 2013
From: donald at stufft.io (Donald Stufft)
Date: Sat, 26 Oct 2013 23:59:15 -0400
Subject: [Distutils] Removing dependency_links
In-Reply-To: <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
Message-ID: <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>

Ok here?s the real list: https://gist.github.com/dstufft/7177500

On Oct 26, 2013, at 11:00 PM, Donald Stufft <donald at stufft.io> wrote:

> Bleh scratch that, it was adding everything :(
> 
> On Oct 26, 2013, at 10:59 PM, Donald Stufft <donald at stufft.io> wrote:
> 
>> 
>> On Oct 26, 2013, at 10:14 PM, Donald Stufft <donald at stufft.io> wrote:
>> 
>>> I would like to remove dependency_links from pip, and ideally
>>> also setuptools.
>>> 
>>> In implementing the ensurepip module from PEP453 I realized that
>>> even with the ``--no-index`` flag pip was still attempting to
>>> reach the internet. After a little bit of investigation I realized
>>> that the reason for this was setuptools use of dependency links.
>>> From my investigation it appears that setuptools uses these in order
>>> to enable secure automatic installation of the ssl dependencies on
>>> Python < 2.6.
>>> 
>>> Overall this feature is a security concern, a malicous package could
>>> "pin" any package they want by depending on it and adding a dependency
>>> link a version 100000. This would be more or less transparent to
>>> the end user.
>>> 
>>> I was looking to see what sort of impact this would have. There are
>>> currently 167,796 source files hosted on PyPI and of those files
>>> 4,005 of them have any dependency links at all. Looking at it a
>>> different way, there are 36,070 total projects on PyPI and 411 of them
>>> use this feature. So this is ~2% of the files or ~1% of the projects.
>>> 
>>> So it appears that this isn't a particularly popular feature, I believe
>>> that it is a *bad* idea that inverts the expected control and should
>>> be removed from both pip and setuptools. In setuptools case it does use
>>> it in the only reasonable way I can imagine, however I think setuptools
>>> should just stop trying to automatically install those dependencies
>>> for Pythons < 2.6 and similarly to pip just print an error and expect users
>>> to get and install them on their own. As a reminder there are very
>>> few downloads from PyPI that are from Pythons < 2.6 [1]
>>> 
>>> [1] https://caremad.io/blog/a-look-at-pypi-downloads/
>>> [2] https://gist.github.com/dstufft/7173539
>>> 
>>> -----------------
>>> Donald Stufft
>>> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>>> 
>>> _______________________________________________
>>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>>> https://mail.python.org/mailman/listinfo/distutils-sig
>> 
>> A list of projects that use dependency links: https://gist.github.com/dstufft/7177500
>> 
>> -----------------
>> Donald Stufft
>> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>> 
> 
> 
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131026/2c41186e/attachment-0001.sig>

From donald at stufft.io  Sun Oct 27 05:13:04 2013
From: donald at stufft.io (Donald Stufft)
Date: Sun, 27 Oct 2013 00:13:04 -0400
Subject: [Distutils] Removing dependency_links
In-Reply-To: <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
Message-ID: <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>


On Oct 26, 2013, at 11:59 PM, Donald Stufft <donald at stufft.io> wrote:

> Ok here?s the real list: https://gist.github.com/dstufft/7177500

Quick note that this list is a list of projects that have *ever* used
dependency links on PyPI. Some of these projects are no longer
using them.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131027/8b9b604d/attachment.sig>

From ncoghlan at gmail.com  Sun Oct 27 05:30:24 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 27 Oct 2013 14:30:24 +1000
Subject: [Distutils] Removing dependency_links
In-Reply-To: <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
Message-ID: <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>

On 27 October 2013 14:13, Donald Stufft <donald at stufft.io> wrote:
>
> On Oct 26, 2013, at 11:59 PM, Donald Stufft <donald at stufft.io> wrote:
>
>> Ok here?s the real list: https://gist.github.com/dstufft/7177500
>
> Quick note that this list is a list of projects that have *ever* used
> dependency links on PyPI. Some of these projects are no longer
> using them.

Am I correct in thinking that providing a flag to disable them
completely will be enough to get ensurepip to behave itself?

If so, then the bare minimum is to provide such a flag in the bundled
versions of pip and setuptools and have ensurepip use it.

I also think it is reasonable to continue offering a feature like
dependency_links on an opt-in basis for controlled environments (I see
it as analagous to the direct references feature in PEP 440).

That would make the migration look something like:

pip 1.5 (and associated minimum required version of setuptools):
  - add a disable switch for dependency link handling
  - add at least a per-project opt-in for dependency link handling
(and perhaps a global opt-in)
  - deprecate implicit handling of dependency links

pip 1.6:
  - dependency links are disabled by default, must opt-in to process them

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From donald at stufft.io  Sun Oct 27 05:35:03 2013
From: donald at stufft.io (Donald Stufft)
Date: Sun, 27 Oct 2013 00:35:03 -0400
Subject: [Distutils] Removing dependency_links
In-Reply-To: <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
Message-ID: <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>


On Oct 27, 2013, at 12:30 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 27 October 2013 14:13, Donald Stufft <donald at stufft.io> wrote:
>> 
>> On Oct 26, 2013, at 11:59 PM, Donald Stufft <donald at stufft.io> wrote:
>> 
>>> Ok here?s the real list: https://gist.github.com/dstufft/7177500
>> 
>> Quick note that this list is a list of projects that have *ever* used
>> dependency links on PyPI. Some of these projects are no longer
>> using them.
> 
> Am I correct in thinking that providing a flag to disable them
> completely will be enough to get ensurepip to behave itself?
> 
> If so, then the bare minimum is to provide such a flag in the bundled
> versions of pip and setuptools and have ensurepip use it.

Yes, it only needs to exist in pip as well, it does not need to exist
in setuptools for ensurepip?s purposes.

> 
> I also think it is reasonable to continue offering a feature like
> dependency_links on an opt-in basis for controlled environments (I see
> it as analagous to the direct references feature in PEP 440).
> 
> That would make the migration look something like:
> 
> pip 1.5 (and associated minimum required version of setuptools):
>  - add a disable switch for dependency link handling
>  - add at least a per-project opt-in for dependency link handling
> (and perhaps a global opt-in)
>  - deprecate implicit handling of dependency links
> 
> pip 1.6:
>  - dependency links are disabled by default, must opt-in to process them
> 
> Cheers,
> Nick.

What if pip 1.5 added a ?no-dependency-links flag, and then pip 1.6 ignored
them by default but if a package cannot be installed it would print something
like?

The package {foo} was unable to be found which was depended on by {bar},
{bar} has suggested some additional links for locating dependencies, you
can use any of them by using the ?find-links flag such as pip install ?find-links <url>.

The suggested urls are:
    https://?./
    https://?./

This allows users to opt in on a per url basis (and under the covers the implementation
would be the same, dependency links just get added to find-links) without adding
yet another flag.


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131027/c7473d3d/attachment.sig>

From fred at fdrake.net  Sun Oct 27 06:06:21 2013
From: fred at fdrake.net (Fred Drake)
Date: Sun, 27 Oct 2013 01:06:21 -0400
Subject: [Distutils] Removing dependency_links
In-Reply-To: <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
Message-ID: <CAFT4OTE0=OzaDDPZQ6pESsmCnRA2dtz08==CJ6F5fr5i8=07VQ@mail.gmail.com>

On Sun, Oct 27, 2013 at 12:30 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Am I correct in thinking that providing a flag to disable them
> completely will be enough to get ensurepip to behave itself?

There's been a setting for this in buildout for some time, and I don't
build without it.

Your deprecation path sounds reasonable to me.


  -Fred

-- 
Fred L. Drake, Jr.    <fred at fdrake.net>
"A storm broke loose in my mind."  --Albert Einstein

From holger at merlinux.eu  Sun Oct 27 06:07:19 2013
From: holger at merlinux.eu (holger krekel)
Date: Sun, 27 Oct 2013 05:07:19 +0000
Subject: [Distutils] Removing dependency_links
In-Reply-To: <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
Message-ID: <20131027050719.GJ3973@merlinux.eu>

On Sun, Oct 27, 2013 at 14:30 +1000, Nick Coghlan wrote:
> On 27 October 2013 14:13, Donald Stufft <donald at stufft.io> wrote:
> >
> > On Oct 26, 2013, at 11:59 PM, Donald Stufft <donald at stufft.io> wrote:
> >
> >> Ok here?s the real list: https://gist.github.com/dstufft/7177500
> >
> > Quick note that this list is a list of projects that have *ever* used
> > dependency links on PyPI. Some of these projects are no longer
> > using them.
> 
> Am I correct in thinking that providing a flag to disable them
> completely will be enough to get ensurepip to behave itself?
> 
> If so, then the bare minimum is to provide such a flag in the bundled
> versions of pip and setuptools and have ensurepip use it.
> 
> I also think it is reasonable to continue offering a feature like
> dependency_links on an opt-in basis for controlled environments (I see
> it as analagous to the direct references feature in PEP 440).
> 
> That would make the migration look something like:
> 
> pip 1.5 (and associated minimum required version of setuptools):
>   - add a disable switch for dependency link handling
>   - add at least a per-project opt-in for dependency link handling
> (and perhaps a global opt-in)
>   - deprecate implicit handling of dependency links
> 
> pip 1.6:
>   - dependency links are disabled by default, must opt-in to process them

So 400 projects out of 35000 ever used dependency links.
I checked three random ones:

- flask-mongorest: does not use it anymore
- Pylons: deplink goes to 502 page, and has the latest release on pypi.
- OpenCoreRedirect: one of out three deplinks work but goes to a page 
  that doesn't appear to be one.  Latest release is 0.5.1, available
  on pypi Project, four years old.

Judging from this little sample: if a questionable feature is used by
<1% of projects and even they likely to not work/don't rely on it
anymore, i don't think we should spend or make Donald spend much efforts
on it.  Rather do the supposed 1.6 change for 1.5 already.

Note that I was the guy publically pressing for backward-compat but 
that was for the introduction of "--pre" which broke many usages.  This
does not start to compare to this change here.  Also pip-1.5 would
cleanly bail out and tell what to do whereas the need for "--pre" was
more implicit as people could get the wrong version suddenly without
noticing/understanding.

best,
holger

From donald at stufft.io  Sun Oct 27 06:12:59 2013
From: donald at stufft.io (Donald Stufft)
Date: Sun, 27 Oct 2013 01:12:59 -0400
Subject: [Distutils] Removing dependency_links
In-Reply-To: <20131027050719.GJ3973@merlinux.eu>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
 <20131027050719.GJ3973@merlinux.eu>
Message-ID: <46DAE7BD-3872-49ED-B0CE-2CA4EEA85D66@stufft.io>


On Oct 27, 2013, at 1:07 AM, holger krekel <holger at merlinux.eu> wrote:

> On Sun, Oct 27, 2013 at 14:30 +1000, Nick Coghlan wrote:
>> On 27 October 2013 14:13, Donald Stufft <donald at stufft.io> wrote:
>>> 
>>> On Oct 26, 2013, at 11:59 PM, Donald Stufft <donald at stufft.io> wrote:
>>> 
>>>> Ok here?s the real list: https://gist.github.com/dstufft/7177500
>>> 
>>> Quick note that this list is a list of projects that have *ever* used
>>> dependency links on PyPI. Some of these projects are no longer
>>> using them.
>> 
>> Am I correct in thinking that providing a flag to disable them
>> completely will be enough to get ensurepip to behave itself?
>> 
>> If so, then the bare minimum is to provide such a flag in the bundled
>> versions of pip and setuptools and have ensurepip use it.
>> 
>> I also think it is reasonable to continue offering a feature like
>> dependency_links on an opt-in basis for controlled environments (I see
>> it as analagous to the direct references feature in PEP 440).
>> 
>> That would make the migration look something like:
>> 
>> pip 1.5 (and associated minimum required version of setuptools):
>>  - add a disable switch for dependency link handling
>>  - add at least a per-project opt-in for dependency link handling
>> (and perhaps a global opt-in)
>>  - deprecate implicit handling of dependency links
>> 
>> pip 1.6:
>>  - dependency links are disabled by default, must opt-in to process them
> 
> So 400 projects out of 35000 ever used dependency links.
> I checked three random ones:
> 
> - flask-mongorest: does not use it anymore
> - Pylons: deplink goes to 502 page, and has the latest release on pypi.
> - OpenCoreRedirect: one of out three deplinks work but goes to a page 
>  that doesn't appear to be one.  Latest release is 0.5.1, available
>  on pypi Project, four years old.

Heh, Webtest and Flask-Security were two I checked who no longer use them.

> 
> Judging from this little sample: if a questionable feature is used by
> <1% of projects and even they likely to not work/don't rely on it
> anymore, i don't think we should spend or make Donald spend much efforts
> on it.  Rather do the supposed 1.6 change for 1.5 already.

I?m definitely +1 on doing the change in 1.5 instead. I really don?t think it?s
going to affect hardly anyone.

> 
> Note that I was the guy publically pressing for backward-compat but 
> that was for the introduction of "--pre" which broke many usages.  This
> does not start to compare to this change here.  Also pip-1.5 would
> cleanly bail out and tell what to do whereas the need for "--pre" was
> more implicit as people could get the wrong version suddenly without
> noticing/understanding.
> 
> best,
> holger


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131027/f9426902/attachment-0001.sig>

From ncoghlan at gmail.com  Sun Oct 27 06:16:32 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 27 Oct 2013 15:16:32 +1000
Subject: [Distutils] Removing dependency_links
In-Reply-To: <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
 <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>
Message-ID: <CADiSq7cLNgwVmfgO94NQZkEMCMREkOr3oXfWTh6SsOThwQq6Uw@mail.gmail.com>

On 27 October 2013 14:35, Donald Stufft <donald at stufft.io> wrote:
>
> On Oct 27, 2013, at 12:30 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>> On 27 October 2013 14:13, Donald Stufft <donald at stufft.io> wrote:
>>>
>>> On Oct 26, 2013, at 11:59 PM, Donald Stufft <donald at stufft.io> wrote:
>>>
>>>> Ok here?s the real list: https://gist.github.com/dstufft/7177500
>>>
>>> Quick note that this list is a list of projects that have *ever* used
>>> dependency links on PyPI. Some of these projects are no longer
>>> using them.
>>
>> Am I correct in thinking that providing a flag to disable them
>> completely will be enough to get ensurepip to behave itself?
>>
>> If so, then the bare minimum is to provide such a flag in the bundled
>> versions of pip and setuptools and have ensurepip use it.
>
> Yes, it only needs to exist in pip as well, it does not need to exist
> in setuptools for ensurepip?s purposes.

Excellent, so that's the only mandatory-due-to-PEP-453 part for pip 1.5.

>> I also think it is reasonable to continue offering a feature like
>> dependency_links on an opt-in basis for controlled environments (I see
>> it as analagous to the direct references feature in PEP 440).
>>
>> That would make the migration look something like:
>>
>> pip 1.5 (and associated minimum required version of setuptools):
>>  - add a disable switch for dependency link handling
>>  - add at least a per-project opt-in for dependency link handling
>> (and perhaps a global opt-in)
>>  - deprecate implicit handling of dependency links
>>
>> pip 1.6:
>>  - dependency links are disabled by default, must opt-in to process them
>>
>> Cheers,
>> Nick.
>
> What if pip 1.5 added a ?no-dependency-links flag, and then pip 1.6 ignored
> them by default but if a package cannot be installed it would print something
> like?
>
> The package {foo} was unable to be found which was depended on by {bar},
> {bar} has suggested some additional links for locating dependencies, you
> can use any of them by using the ?find-links flag such as pip install ?find-links <url>.
>
> The suggested urls are:
>     https://?./
>     https://?./
>
> This allows users to opt in on a per url basis (and under the covers the implementation
> would be the same, dependency links just get added to find-links) without adding
> yet another flag.

With that error message, I think it's reasonable to do as Holger
suggests and opt out of processing them by default even in 1.5.

To me, the best part of the more aggressive timeline is it means
CPython would never ship a version of pip that allows that particular
attack vector by default.

In that approach, I'd still suggest offering a
"--process-dependency-links" flag and there wouldn't be a flag to turn
the processing off (since they'd be off by default). This suggestion
is born out of the "we don't know what happens inside corporate
firewalls" perspective, so I think it's beneficial to have a "make it
work the way it used to" fallback for at least one release.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From techtonik at gmail.com  Sun Oct 27 07:49:38 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Sun, 27 Oct 2013 09:49:38 +0300
Subject: [Distutils] PyPI pull request
Message-ID: <CAPkN8xLvAsArPaf3T-V+86-6EhwUwV4BFtfdJw=a9ce4DNyS8g@mail.gmail.com>

I've heard that there is PyPI 2.0, but I still find current PyPI code
to be very suitable for educational purposes (unlike some complicated
framework based solutions, where much of the stuff is hidden in
internals of external lib abstractions), so I continue to send fixes
to improve code base.

Please merge this one.
https://bitbucket.org/pypa/pypi/pull-request/6/remove-unused-templatetoolspy-file/
--
anatoly t.

From qwcode at gmail.com  Sun Oct 27 09:14:11 2013
From: qwcode at gmail.com (Marcus Smith)
Date: Sun, 27 Oct 2013 01:14:11 -0700
Subject: [Distutils] Removing dependency_links
In-Reply-To: <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
Message-ID: <CAPYWazoXOPtN3+ik49q764cr95hmUYqrHOB07FScvwz3i0jfRA@mail.gmail.com>

>
> I also think it is reasonable to continue offering a feature like
> dependency_links on an opt-in basis for controlled environments (I see
> it as analagous to the direct references feature in PEP 440).
>
>
with pip already, the idea** is that you can enforce a concrete location
for a
dependency that will override the default pypi location. e.g. I can do

  pip install http://myserver/peppercorn-0.3.tar.gz#egg=peppercorn deform
  (where deform requires peppercorn)

and my peppercorn will be installed
the point being, I don't think we need setup.py dependency links to be able
enforce concrete url fulfillment.

** I say "idea", because I know there are bugs related to archive urls and
egg fragments and overriding other dependencies, but the point still stands.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131027/5b03879c/attachment.html>

From qwcode at gmail.com  Sun Oct 27 09:38:30 2013
From: qwcode at gmail.com (Marcus Smith)
Date: Sun, 27 Oct 2013 01:38:30 -0700
Subject: [Distutils] Removing dependency_links
In-Reply-To: <CADiSq7cLNgwVmfgO94NQZkEMCMREkOr3oXfWTh6SsOThwQq6Uw@mail.gmail.com>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
 <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>
 <CADiSq7cLNgwVmfgO94NQZkEMCMREkOr3oXfWTh6SsOThwQq6Uw@mail.gmail.com>
Message-ID: <CAPYWazpcjRP-0xCE0=e0fJUCGNS5vTJpUt20Us95QEqDVMsN+Q@mail.gmail.com>

> "we don't know what happens inside corporate firewalls"
>

non-published use of dependency links could turn out to be the use-cases
that we'd get complaints about



> To me, the best part of the more aggressive timeline is it means
> CPython would never ship a version of pip that allows that particular
> attack vector by default.
>
>
over IRC and on pypa-dev, I brought up the deprecate first point of view in
the context that we would be *removing the feature*.
It's less drastic to flip defaults (and add a turn on)

it's probably right that nobody will complain, but my thinking was this:
- donald can add a hidden option for now for the sake of ensurepip (it
wouldn't clutter the cli, and can be removed later care-free)
- separate from that,  pip and setuptools deprecates together, then
completely removes dep-links support.  if its bad, it's bad. get rid of it.
let's reduce the options and clutter.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131027/75ac56be/attachment.html>

From ncoghlan at gmail.com  Sun Oct 27 13:19:02 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 27 Oct 2013 22:19:02 +1000
Subject: [Distutils] Removing dependency_links
In-Reply-To: <CAPYWazpcjRP-0xCE0=e0fJUCGNS5vTJpUt20Us95QEqDVMsN+Q@mail.gmail.com>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
 <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>
 <CADiSq7cLNgwVmfgO94NQZkEMCMREkOr3oXfWTh6SsOThwQq6Uw@mail.gmail.com>
 <CAPYWazpcjRP-0xCE0=e0fJUCGNS5vTJpUt20Us95QEqDVMsN+Q@mail.gmail.com>
Message-ID: <CADiSq7fW6uqWGDz-MGotgX0rL1c+uOyfAj+gMGFPwqkx3GjXUg@mail.gmail.com>

On 27 Oct 2013 18:38, "Marcus Smith" <qwcode at gmail.com> wrote:
>
>
>>
>> "we don't know what happens inside corporate firewalls"
>
>
> non-published use of dependency links could turn out to be the use-cases
that we'd get complaints about
>
>
>>
>> To me, the best part of the more aggressive timeline is it means
>> CPython would never ship a version of pip that allows that particular
>> attack vector by default.
>>
>
> over IRC and on pypa-dev, I brought up the deprecate first point of view
in the context that we would be *removing the feature*.
> It's less drastic to flip defaults (and add a turn on)
>
> it's probably right that nobody will complain, but my thinking was this:
> - donald can add a hidden option for now for the sake of ensurepip (it
wouldn't clutter the cli, and can be removed later care-free)

Yeah, we at least need to do that much to meet the "ensurepip doesn't talk
to the internet" guarantee.

> - separate from that,  pip and setuptools deprecates together, then
completely removes dep-links support.  if its bad, it's bad. get rid of it.
let's reduce the options and clutter.

I'm happy to go with whatever you folks (as in pip & setuptools devs)
decide on that front. I prefer "flip the default & deprecate, then remove
later if nobody campaigns to keep it", but I'm also OK with the more
conservative "deprecate, then remove later".

Cheers,
Nick.

>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131027/43c4b329/attachment.html>

From donald at stufft.io  Sun Oct 27 18:00:01 2013
From: donald at stufft.io (Donald Stufft)
Date: Sun, 27 Oct 2013 13:00:01 -0400
Subject: [Distutils] Removing dependency_links
In-Reply-To: <CADiSq7fW6uqWGDz-MGotgX0rL1c+uOyfAj+gMGFPwqkx3GjXUg@mail.gmail.com>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
 <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>
 <CADiSq7cLNgwVmfgO94NQZkEMCMREkOr3oXfWTh6SsOThwQq6Uw@mail.gmail.com>
 <CAPYWazpcjRP-0xCE0=e0fJUCGNS5vTJpUt20Us95QEqDVMsN+Q@mail.gmail.com>
 <CADiSq7fW6uqWGDz-MGotgX0rL1c+uOyfAj+gMGFPwqkx3GjXUg@mail.gmail.com>
Message-ID: <3FC26D14-AB77-49DB-9856-E55C75FE5F7B@stufft.io>

More numbers, of the 411 projects who have ever used dependency links, only 311 of them use them in their latest release.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131027/ca42ce52/attachment.sig>

From donald at stufft.io  Sun Oct 27 18:44:28 2013
From: donald at stufft.io (Donald Stufft)
Date: Sun, 27 Oct 2013 13:44:28 -0400
Subject: [Distutils] Removing dependency_links
In-Reply-To: <3FC26D14-AB77-49DB-9856-E55C75FE5F7B@stufft.io>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
 <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>
 <CADiSq7cLNgwVmfgO94NQZkEMCMREkOr3oXfWTh6SsOThwQq6Uw@mail.gmail.com>
 <CAPYWazpcjRP-0xCE0=e0fJUCGNS5vTJpUt20Us95QEqDVMsN+Q@mail.gmail.com>
 <CADiSq7fW6uqWGDz-MGotgX0rL1c+uOyfAj+gMGFPwqkx3GjXUg@mail.gmail.com>
 <3FC26D14-AB77-49DB-9856-E55C75FE5F7B@stufft.io>
Message-ID: <D6B8D266-925E-4706-89CF-B0F4FA541EDC@stufft.io>

Here?s the list of dependency links for the projects that still use them in their latest releases:

https://gist.github.com/dstufft/7185162

A good number of them are either bogus, are pointing directly to PyPI, or are file:// urls that are highly unlikely to exist on anyones computer but the author?s. All in all there are 307 total unique links in this set of packages, and 99 of them are not reachable from my computer (requests.get(?) raises an exception).

So honestly I think this could just go away completely. I don?t see any use for it anymore and apparently neither does most of PyPI.

On Oct 27, 2013, at 1:00 PM, Donald Stufft <donald at stufft.io> wrote:

> More numbers, of the 411 projects who have ever used dependency links, only 311 of them use them in their latest release.
> 
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131027/2eb6285c/attachment.sig>

From chris.jerdonek at gmail.com  Sun Oct 27 20:04:50 2013
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sun, 27 Oct 2013 12:04:50 -0700
Subject: [Distutils] Removing dependency_links
In-Reply-To: <D6B8D266-925E-4706-89CF-B0F4FA541EDC@stufft.io>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
 <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>
 <CADiSq7cLNgwVmfgO94NQZkEMCMREkOr3oXfWTh6SsOThwQq6Uw@mail.gmail.com>
 <CAPYWazpcjRP-0xCE0=e0fJUCGNS5vTJpUt20Us95QEqDVMsN+Q@mail.gmail.com>
 <CADiSq7fW6uqWGDz-MGotgX0rL1c+uOyfAj+gMGFPwqkx3GjXUg@mail.gmail.com>
 <3FC26D14-AB77-49DB-9856-E55C75FE5F7B@stufft.io>
 <D6B8D266-925E-4706-89CF-B0F4FA541EDC@stufft.io>
Message-ID: <CAOTb1wfba-Y2zvpwv1GQCs4tAwqmg=8f69G+xS9e1se+6XY_DA@mail.gmail.com>

On Sun, Oct 27, 2013 at 10:44 AM, Donald Stufft <donald at stufft.io> wrote:

> Here?s the list of dependency links for the projects that still use them
> in their latest releases:
>
> https://gist.github.com/dstufft/7185162
>
> A good number of them are either bogus, are pointing directly to PyPI, or
> are file:// urls that are highly unlikely to exist on anyones computer but
> the author?s. All in all there are 307 total unique links in this set of
> packages, and 99 of them are not reachable from my computer
> (requests.get(?) raises an exception).
>


I actually know a couple people on this list.  I can ask them and see if
the list can be reduced further. :)

--Chris





>
> So honestly I think this could just go away completely. I don?t see any
> use for it anymore and apparently neither does most of PyPI.
>
> On Oct 27, 2013, at 1:00 PM, Donald Stufft <donald at stufft.io> wrote:
>
> > More numbers, of the 411 projects who have ever used dependency links,
> only 311 of them use them in their latest release.
> >
> > -----------------
> > Donald Stufft
> > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
> DCFA
> >
> > _______________________________________________
> > Distutils-SIG maillist  -  Distutils-SIG at python.org
> > https://mail.python.org/mailman/listinfo/distutils-sig
>
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
> DCFA
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131027/606e1432/attachment-0001.html>

From ncoghlan at gmail.com  Sun Oct 27 21:29:02 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 28 Oct 2013 06:29:02 +1000
Subject: [Distutils] Removing dependency_links
In-Reply-To: <D6B8D266-925E-4706-89CF-B0F4FA541EDC@stufft.io>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
 <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>
 <CADiSq7cLNgwVmfgO94NQZkEMCMREkOr3oXfWTh6SsOThwQq6Uw@mail.gmail.com>
 <CAPYWazpcjRP-0xCE0=e0fJUCGNS5vTJpUt20Us95QEqDVMsN+Q@mail.gmail.com>
 <CADiSq7fW6uqWGDz-MGotgX0rL1c+uOyfAj+gMGFPwqkx3GjXUg@mail.gmail.com>
 <3FC26D14-AB77-49DB-9856-E55C75FE5F7B@stufft.io>
 <D6B8D266-925E-4706-89CF-B0F4FA541EDC@stufft.io>
Message-ID: <CADiSq7fA2GKANeTaV_PPZ+rLwBtk1Ej4Cf-cEBuSxaLOFPKQMw@mail.gmail.com>

On 28 Oct 2013 03:44, "Donald Stufft" <donald at stufft.io> wrote:
>
> Here?s the list of dependency links for the projects that still use them
in their latest releases:
>
> https://gist.github.com/dstufft/7185162
>
> A good number of them are either bogus, are pointing directly to PyPI, or
are file:// urls that are highly unlikely to exist on anyones computer but
the author?s. All in all there are 307 total unique links in this set of
packages, and 99 of them are not reachable from my computer
(requests.get(?) raises an exception).
>
> So honestly I think this could just go away completely. I don?t see any
use for it anymore and apparently neither does most of PyPI.

When making compatibility decisions, it's worth remembering that
pre-packaged software (let alone the open source subset of that) is only
the tip of a very large software iceberg that, as far as I am aware, still
consists mostly of custom purpose specific code written for particular
organisations.

In this case, I think the vulnerability argument is strong enough and good
use cases rare enough to justify turning dependency link support off by
default, but it should be easy to turn back on in at least pip 1.5 as a
risk mitigation strategy.

Cheers,
Nick.
>
> On Oct 27, 2013, at 1:00 PM, Donald Stufft <donald at stufft.io> wrote:
>
> > More numbers, of the 411 projects who have ever used dependency links,
only 311 of them use them in their latest release.
> >
> > -----------------
> > Donald Stufft
> > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
DCFA
> >
> > _______________________________________________
> > Distutils-SIG maillist  -  Distutils-SIG at python.org
> > https://mail.python.org/mailman/listinfo/distutils-sig
>
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
DCFA
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131028/91997576/attachment.html>

From holger at merlinux.eu  Sun Oct 27 21:55:10 2013
From: holger at merlinux.eu (holger krekel)
Date: Sun, 27 Oct 2013 20:55:10 +0000
Subject: [Distutils] factor out project metadata in PEP426?
Message-ID: <20131027205510.GM3973@merlinux.eu>

Hi Nick, all,

PEP426 and its prior related peps see project-specific metadata as part
of distribution metadata.  Main examples are "project urls" such as
"home page", "repository", "issue tracker" or contact points such as
mailing lists, maintainer emails etc.  However, at any point in time
there is only one set of active project urls and contact points.
Putting those into distribution files does not really make sense.

I suggest to rather define "project metadata" and make distribution 
metadata reference it. Of course we then also want a simple way to
retrieve project metadata from pypi.  Which could be used from tools to 
tell a user where to send a mail to etc. ("pypi-report-bug warehouse" could
e.g. open the webbrowser with the create-issue page).  Or from PyPI
implementations to present proper project meta info even if looking
at an older version (the project might have switched to a different
repo meanwhile etc.)

It would also make life easier for project maintainers to change project
metadata even between releases.

I feel that PEP426 is a big chance to clean up the project/distribution
metadata confusion and provide a better base for future tooling and
organising human communication and collaboration around Python software.

cheers,
holger



From ncoghlan at gmail.com  Sun Oct 27 22:36:24 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 28 Oct 2013 07:36:24 +1000
Subject: [Distutils] factor out project metadata in PEP426?
In-Reply-To: <20131027205510.GM3973@merlinux.eu>
References: <20131027205510.GM3973@merlinux.eu>
Message-ID: <CADiSq7f5bwaEAH4q1Pf_0hkOW5aerSC+cT_gpqrmo3AHFjS0Fw@mail.gmail.com>

On 28 Oct 2013 06:56, "holger krekel" <holger at merlinux.eu> wrote:
>
> Hi Nick, all,
>
> PEP426 and its prior related peps see project-specific metadata as part
> of distribution metadata.  Main examples are "project urls" such as
> "home page", "repository", "issue tracker" or contact points such as
> mailing lists, maintainer emails etc.  However, at any point in time
> there is only one set of active project urls and contact points.
> Putting those into distribution files does not really make sense.

I think it still makes sense to include a snapshot of those details in a
metadata extension, but I agree it should be easy to find out how to get
more up to date project metadata from an index server.

I  expect to get back to metadata 2.0 related design work after 3.4b1 is
out the door, using the repo at
https://bitbucket.org/pypa/pypi-metadata-formats

Likely changes/additions:

- reserving the python.* namespace for "official" (i.e. defined in a PEP)
extensions and export groups
- formalising a process to give projects control over the PyPI namespaces
that match their distribution names (including the current default "open
access" behaviour and the ability to designate "contribution" namespaces)
- rewriting the install hook system to be based on export groups and export
hooks
- moving anything that isn't part of the essential dependency resolution
metadata into python.* extensions
- defining the minimum CLI command and option requirements for setup.py
(although I'd love to delegate this one to someone that either already has
greater setuptools/distutils knowledge than mine, or else is willing to do
the research on the currently supported command options)
- resolution of the issues on the BitBucket tracker

By further minimising the core, we'll be able to better assess the proposed
extension mechanisms, as well as reduce the coupling between different
parts of the spec (by giving the extensions their own version numbers).

Another (more dubious) idea is providing a mechanism to indicate a "home"
index server that is the authoritative source for namespace metadata
(defaulting to PyPI). This is potentially related to the "local version
numbering" redistribution problem on the issue tracker.

> I feel that PEP426 is a big chance to clean up the project/distribution
> metadata confusion and provide a better base for future tooling and
> organising human communication and collaboration around Python software.

Indeed :)

Cheers,
Nick.

>
> cheers,
> holger
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131028/270ceace/attachment.html>

From chris.jerdonek at gmail.com  Sun Oct 27 22:49:35 2013
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sun, 27 Oct 2013 14:49:35 -0700
Subject: [Distutils] Removing dependency_links
In-Reply-To: <CAOTb1wfba-Y2zvpwv1GQCs4tAwqmg=8f69G+xS9e1se+6XY_DA@mail.gmail.com>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
 <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>
 <CADiSq7cLNgwVmfgO94NQZkEMCMREkOr3oXfWTh6SsOThwQq6Uw@mail.gmail.com>
 <CAPYWazpcjRP-0xCE0=e0fJUCGNS5vTJpUt20Us95QEqDVMsN+Q@mail.gmail.com>
 <CADiSq7fW6uqWGDz-MGotgX0rL1c+uOyfAj+gMGFPwqkx3GjXUg@mail.gmail.com>
 <3FC26D14-AB77-49DB-9856-E55C75FE5F7B@stufft.io>
 <D6B8D266-925E-4706-89CF-B0F4FA541EDC@stufft.io>
 <CAOTb1wfba-Y2zvpwv1GQCs4tAwqmg=8f69G+xS9e1se+6XY_DA@mail.gmail.com>
Message-ID: <CAOTb1wd7NpzmWgJz=jro5M+FNoaY072_b3akGyM=qSHvRj6DFQ@mail.gmail.com>

On Sun, Oct 27, 2013 at 12:04 PM, Chris Jerdonek
<chris.jerdonek at gmail.com> wrote:
>
> On Sun, Oct 27, 2013 at 10:44 AM, Donald Stufft <donald at stufft.io> wrote:
>>
>> Here?s the list of dependency links for the projects that still use them in their latest releases:
>>
>> https://gist.github.com/dstufft/7185162
>>
>> A good number of them are either bogus, are pointing directly to PyPI, or are file:// urls that are highly unlikely to exist on anyones computer but the author?s. All in all there are 307 total unique links in this set of packages, and 99 of them are not reachable from my computer (requests.get(?) raises an exception).
>
> I actually know a couple people on this list.  I can ask them and see if the list can be reduced further. :)

So I asked the person I know, and this is what he said, "Yes, we have
to use it!  It's the only way to allow a package to install other
packages that aren't on PyPI-- for instance, a custom fork of a
library."

Is there another approach or work-around he can be using?  What is the
"right" way for him to do it?

--Chris

From donald at stufft.io  Sun Oct 27 23:05:39 2013
From: donald at stufft.io (Donald Stufft)
Date: Sun, 27 Oct 2013 18:05:39 -0400
Subject: [Distutils] Removing dependency_links
In-Reply-To: <CAOTb1wd7NpzmWgJz=jro5M+FNoaY072_b3akGyM=qSHvRj6DFQ@mail.gmail.com>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
 <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>
 <CADiSq7cLNgwVmfgO94NQZkEMCMREkOr3oXfWTh6SsOThwQq6Uw@mail.gmail.com>
 <CAPYWazpcjRP-0xCE0=e0fJUCGNS5vTJpUt20Us95QEqDVMsN+Q@mail.gmail.com>
 <CADiSq7fW6uqWGDz-MGotgX0rL1c+uOyfAj+gMGFPwqkx3GjXUg@mail.gmail.com>
 <3FC26D14-AB77-49DB-9856-E55C75FE5F7B@stufft.io>
 <D6B8D266-925E-4706-89CF-B0F4FA541EDC@stufft.io>
 <CAOTb1wfba-Y2zvpwv1GQCs4tAwqmg=8f69G+xS9e1se+6XY_DA@mail.gmail.com>
 <CAOTb1wd7NpzmWgJz=jro5M+FNoaY072_b3akGyM=qSHvRj6DFQ@mail.gmail.com>
Message-ID: <77E55F4D-02A5-45DE-A559-EFC582FCF691@stufft.io>


On Oct 27, 2013, at 5:49 PM, Chris Jerdonek <chris.jerdonek at gmail.com> wrote:

> On Sun, Oct 27, 2013 at 12:04 PM, Chris Jerdonek
> <chris.jerdonek at gmail.com> wrote:
>> 
>> On Sun, Oct 27, 2013 at 10:44 AM, Donald Stufft <donald at stufft.io> wrote:
>>> 
>>> Here?s the list of dependency links for the projects that still use them in their latest releases:
>>> 
>>> https://gist.github.com/dstufft/7185162
>>> 
>>> A good number of them are either bogus, are pointing directly to PyPI, or are file:// urls that are highly unlikely to exist on anyones computer but the author?s. All in all there are 307 total unique links in this set of packages, and 99 of them are not reachable from my computer (requests.get(?) raises an exception).
>> 
>> I actually know a couple people on this list.  I can ask them and see if the list can be reduced further. :)
> 
> So I asked the person I know, and this is what he said, "Yes, we have
> to use it!  It's the only way to allow a package to install other
> packages that aren't on PyPI-- for instance, a custom fork of a
> library."
> 
> Is there another approach or work-around he can be using?  What is the
> "right" way for him to do it?
> 
> --Chris

Upload the package to PyPI under a different name? Vendor the package inside the source?

Maybe his fork is incompatible, the way he?s doing it it?ll install, pretending to be the unforked library, and then if something *else* depends on it, it?ll get a fundamentally incompatible version of that library (in the theoretical situation).

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131027/fe72d0e4/attachment-0001.sig>

From qwcode at gmail.com  Sun Oct 27 23:52:04 2013
From: qwcode at gmail.com (Marcus Smith)
Date: Sun, 27 Oct 2013 15:52:04 -0700
Subject: [Distutils] Removing dependency_links
In-Reply-To: <CAOTb1wd7NpzmWgJz=jro5M+FNoaY072_b3akGyM=qSHvRj6DFQ@mail.gmail.com>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
 <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>
 <CADiSq7cLNgwVmfgO94NQZkEMCMREkOr3oXfWTh6SsOThwQq6Uw@mail.gmail.com>
 <CAPYWazpcjRP-0xCE0=e0fJUCGNS5vTJpUt20Us95QEqDVMsN+Q@mail.gmail.com>
 <CADiSq7fW6uqWGDz-MGotgX0rL1c+uOyfAj+gMGFPwqkx3GjXUg@mail.gmail.com>
 <3FC26D14-AB77-49DB-9856-E55C75FE5F7B@stufft.io>
 <D6B8D266-925E-4706-89CF-B0F4FA541EDC@stufft.io>
 <CAOTb1wfba-Y2zvpwv1GQCs4tAwqmg=8f69G+xS9e1se+6XY_DA@mail.gmail.com>
 <CAOTb1wd7NpzmWgJz=jro5M+FNoaY072_b3akGyM=qSHvRj6DFQ@mail.gmail.com>
Message-ID: <CAPYWazrBa4GEYQyQ7N+LXFDwEa+RN=zHGo_K=vVbsM48U4R+LA@mail.gmail.com>

> So I asked the person I know, and this is what he said, "Yes, we have
> to use it!  It's the only way to allow a package to install other
> packages that aren't on PyPI-- for instance, a custom fork of a
> library."
>
> Is there another approach or work-around he can be using?  What is the
> "right" way for him to do it?
>

e.g. you find a bug in SomeProject-1.4 on pypi (which is a dependency in
your app install)
fork it, fix it, and re-version it to SomeProject-1.4-myfork1, then package
it, and place it in "/my/forks"
and then "pip install --find-links=/my/forks/  my_app"

as for fork versioning in the long run, that is intended to be covered in
PEP440, with a conversation happening here:
https://bitbucket.org/pypa/pypi-metadata-formats/issue/1/add-local-numbering-to-pep-440




>
> --Chris
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131027/51eab5d5/attachment.html>

From qwcode at gmail.com  Mon Oct 28 00:10:45 2013
From: qwcode at gmail.com (Marcus Smith)
Date: Sun, 27 Oct 2013 16:10:45 -0700
Subject: [Distutils] Removing dependency_links
In-Reply-To: <CAPYWazrBa4GEYQyQ7N+LXFDwEa+RN=zHGo_K=vVbsM48U4R+LA@mail.gmail.com>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
 <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>
 <CADiSq7cLNgwVmfgO94NQZkEMCMREkOr3oXfWTh6SsOThwQq6Uw@mail.gmail.com>
 <CAPYWazpcjRP-0xCE0=e0fJUCGNS5vTJpUt20Us95QEqDVMsN+Q@mail.gmail.com>
 <CADiSq7fW6uqWGDz-MGotgX0rL1c+uOyfAj+gMGFPwqkx3GjXUg@mail.gmail.com>
 <3FC26D14-AB77-49DB-9856-E55C75FE5F7B@stufft.io>
 <D6B8D266-925E-4706-89CF-B0F4FA541EDC@stufft.io>
 <CAOTb1wfba-Y2zvpwv1GQCs4tAwqmg=8f69G+xS9e1se+6XY_DA@mail.gmail.com>
 <CAOTb1wd7NpzmWgJz=jro5M+FNoaY072_b3akGyM=qSHvRj6DFQ@mail.gmail.com>
 <CAPYWazrBa4GEYQyQ7N+LXFDwEa+RN=zHGo_K=vVbsM48U4R+LA@mail.gmail.com>
Message-ID: <CAPYWazpEKEqrKb0_Z--u+Fp3c7of8vTZDTmm_ra9CPuvagU2gw@mail.gmail.com>

Chris:
to be clear, after talking to Donald, we interpreted your question
differently.

- If you're distributing library Y, and it depends on X, but it now needs a
custom/fixed fork of X, then what Donald said,  rename and re-publish (or
vendor it).
- If you just need to override a buggy pypi package locally in your app Y
install, then what I said (you can also use a vcs requirement form, and not
have to package you fork)


On Sun, Oct 27, 2013 at 3:52 PM, Marcus Smith <qwcode at gmail.com> wrote:

>
>
>
>> So I asked the person I know, and this is what he said, "Yes, we have
>> to use it!  It's the only way to allow a package to install other
>> packages that aren't on PyPI-- for instance, a custom fork of a
>> library."
>>
>> Is there another approach or work-around he can be using?  What is the
>> "right" way for him to do it?
>>
>
> e.g. you find a bug in SomeProject-1.4 on pypi (which is a dependency in
> your app install)
> fork it, fix it, and re-version it to SomeProject-1.4-myfork1, then
> package it, and place it in "/my/forks"
> and then "pip install --find-links=/my/forks/  my_app"
>
> as for fork versioning in the long run, that is intended to be covered in
> PEP440, with a conversation happening here:
>
> https://bitbucket.org/pypa/pypi-metadata-formats/issue/1/add-local-numbering-to-pep-440
>
>
>
>
>>
>> --Chris
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131027/d75c2755/attachment.html>

From ncoghlan at gmail.com  Mon Oct 28 00:45:21 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 28 Oct 2013 09:45:21 +1000
Subject: [Distutils] Removing dependency_links
In-Reply-To: <CAPYWazpEKEqrKb0_Z--u+Fp3c7of8vTZDTmm_ra9CPuvagU2gw@mail.gmail.com>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
 <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>
 <CADiSq7cLNgwVmfgO94NQZkEMCMREkOr3oXfWTh6SsOThwQq6Uw@mail.gmail.com>
 <CAPYWazpcjRP-0xCE0=e0fJUCGNS5vTJpUt20Us95QEqDVMsN+Q@mail.gmail.com>
 <CADiSq7fW6uqWGDz-MGotgX0rL1c+uOyfAj+gMGFPwqkx3GjXUg@mail.gmail.com>
 <3FC26D14-AB77-49DB-9856-E55C75FE5F7B@stufft.io>
 <D6B8D266-925E-4706-89CF-B0F4FA541EDC@stufft.io>
 <CAOTb1wfba-Y2zvpwv1GQCs4tAwqmg=8f69G+xS9e1se+6XY_DA@mail.gmail.com>
 <CAOTb1wd7NpzmWgJz=jro5M+FNoaY072_b3akGyM=qSHvRj6DFQ@mail.gmail.com>
 <CAPYWazrBa4GEYQyQ7N+LXFDwEa+RN=zHGo_K=vVbsM48U4R+LA@mail.gmail.com>
 <CAPYWazpEKEqrKb0_Z--u+Fp3c7of8vTZDTmm_ra9CPuvagU2gw@mail.gmail.com>
Message-ID: <CADiSq7dtXMz_wpqtr-rKovtqTr4ziUFMPjeddzY+F87BUcLbQQ@mail.gmail.com>

On 28 Oct 2013 09:11, "Marcus Smith" <qwcode at gmail.com> wrote:
>
> Chris:
> to be clear, after talking to Donald, we interpreted your question
differently.
>
> - If you're distributing library Y, and it depends on X, but it now needs
a custom/fixed fork of X, then what Donald said,  rename and re-publish (or
vendor it).
> - If you just need to override a buggy pypi package locally in your app Y
install, then what I said (you can also use a vcs requirement form, and not
have to package you fork)

And to clarify the rationale for these as the official solutions:
installing modified software under the *same* name as the original is not
something we wish to support through PyPI, as it makes even cursory
security audits far more difficult than they should be, and can cause
additional confusion when debugging problems.

If a package maintainer for a dependency doesn't release regular updates or
is otherwise not responsive to bug reports, then the appropriate solutions
are:
- offering a function for runtime monkey patching of the dependency
- bundling a patched copy directly
- forking the dependency
- using application level dependencies to force installation of the patched
version

Implicit monkey patching and silently replacing the upstream dependency
with a patched version is quite user hostile when publishing software for
use by others in a Python installation that may be used to run multiple
independent applications.

Thinking about this further, perhaps we should institute a PyPI side
lockout for usage of dependency links in *new* projects? (similar to what
happened for the external hosting support and is proposed for direct
references in PEP 426)

(Which would put this in YAPP territory - *sigh*)

Cheers,
Nick.

P.S. Yet Another Packaging PEP. Feel free to infer the implied presence of
additional adjectives ;)

>
>
> On Sun, Oct 27, 2013 at 3:52 PM, Marcus Smith <qwcode at gmail.com> wrote:
>>
>>
>>
>>>
>>> So I asked the person I know, and this is what he said, "Yes, we have
>>> to use it!  It's the only way to allow a package to install other
>>> packages that aren't on PyPI-- for instance, a custom fork of a
>>> library."
>>>
>>> Is there another approach or work-around he can be using?  What is the
>>> "right" way for him to do it?
>>
>>
>> e.g. you find a bug in SomeProject-1.4 on pypi (which is a dependency in
your app install)
>> fork it, fix it, and re-version it to SomeProject-1.4-myfork1, then
package it, and place it in "/my/forks"
>> and then "pip install --find-links=/my/forks/  my_app"
>>
>> as for fork versioning in the long run, that is intended to be covered
in PEP440, with a conversation happening here:
>>
https://bitbucket.org/pypa/pypi-metadata-formats/issue/1/add-local-numbering-to-pep-440
>>
>>
>>
>>>
>>>
>>> --Chris
>>> _______________________________________________
>>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>>
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131028/2fe51caf/attachment-0001.html>

From donald at stufft.io  Mon Oct 28 00:47:29 2013
From: donald at stufft.io (Donald Stufft)
Date: Sun, 27 Oct 2013 19:47:29 -0400
Subject: [Distutils] Removing dependency_links
In-Reply-To: <CADiSq7dtXMz_wpqtr-rKovtqTr4ziUFMPjeddzY+F87BUcLbQQ@mail.gmail.com>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
 <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>
 <CADiSq7cLNgwVmfgO94NQZkEMCMREkOr3oXfWTh6SsOThwQq6Uw@mail.gmail.com>
 <CAPYWazpcjRP-0xCE0=e0fJUCGNS5vTJpUt20Us95QEqDVMsN+Q@mail.gmail.com>
 <CADiSq7fW6uqWGDz-MGotgX0rL1c+uOyfAj+gMGFPwqkx3GjXUg@mail.gmail.com>
 <3FC26D14-AB77-49DB-9856-E55C75FE5F7B@stufft.io>
 <D6B8D266-925E-4706-89CF-B0F4FA541EDC@stufft.io>
 <CAOTb1wfba-Y2zvpwv1GQCs4tAwqmg=8f69G+xS9e1se+6XY_DA@mail.gmail.com>
 <CAOTb1wd7NpzmWgJz=jro5M+FNoaY072_b3akGyM=qSHvRj6DFQ@mail.gmail.com>
 <CAPYWazrBa4GEYQyQ7N+LXFDwEa+RN=zHGo_K=vVbsM48U4R+LA@mail.gmail.com>
 <CAPYWazpEKEqrKb0_Z--u+Fp3c7of8vTZDTmm_ra9CPuvagU2gw@mail.gmail.com>
 <CADiSq7dtXMz_wpqtr-rKovtqTr4ziUFMPjeddzY+
 F87BUcLbQQ@mail.gmail.com>
Message-ID: <7185B5A9-5F35-4066-BECF-451153AA45B6@stufft.io>

https://github.com/pypa/pip/pull/1264


On Oct 27, 2013, at 7:45 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> 
> On 28 Oct 2013 09:11, "Marcus Smith" <qwcode at gmail.com> wrote:
> >
> > Chris:
> > to be clear, after talking to Donald, we interpreted your question differently.
> >
> > - If you're distributing library Y, and it depends on X, but it now needs a custom/fixed fork of X, then what Donald said,  rename and re-publish (or vendor it).  
> > - If you just need to override a buggy pypi package locally in your app Y install, then what I said (you can also use a vcs requirement form, and not have to package you fork)
> 
> And to clarify the rationale for these as the official solutions: installing modified software under the *same* name as the original is not something we wish to support through PyPI, as it makes even cursory security audits far more difficult than they should be, and can cause additional confusion when debugging problems.
> 
> If a package maintainer for a dependency doesn't release regular updates or is otherwise not responsive to bug reports, then the appropriate solutions are:
> - offering a function for runtime monkey patching of the dependency
> - bundling a patched copy directly
> - forking the dependency
> - using application level dependencies to force installation of the patched version
> 
> Implicit monkey patching and silently replacing the upstream dependency with a patched version is quite user hostile when publishing software for use by others in a Python installation that may be used to run multiple independent applications.
> 
> Thinking about this further, perhaps we should institute a PyPI side lockout for usage of dependency links in *new* projects? (similar to what happened for the external hosting support and is proposed for direct references in PEP 426)
> 
> (Which would put this in YAPP territory - *sigh*)
> 
> Cheers,
> Nick.
> 
> P.S. Yet Another Packaging PEP. Feel free to infer the implied presence of additional adjectives ;)
> 
> >
> >
> > On Sun, Oct 27, 2013 at 3:52 PM, Marcus Smith <qwcode at gmail.com> wrote:
> >>
> >>
> >>
> >>>
> >>> So I asked the person I know, and this is what he said, "Yes, we have
> >>> to use it!  It's the only way to allow a package to install other
> >>> packages that aren't on PyPI-- for instance, a custom fork of a
> >>> library."
> >>>
> >>> Is there another approach or work-around he can be using?  What is the
> >>> "right" way for him to do it?
> >>
> >>
> >> e.g. you find a bug in SomeProject-1.4 on pypi (which is a dependency in your app install)
> >> fork it, fix it, and re-version it to SomeProject-1.4-myfork1, then package it, and place it in "/my/forks"
> >> and then "pip install --find-links=/my/forks/  my_app"
> >>
> >> as for fork versioning in the long run, that is intended to be covered in PEP440, with a conversation happening here:
> >> https://bitbucket.org/pypa/pypi-metadata-formats/issue/1/add-local-numbering-to-pep-440
> >>
> >>
> >>  
> >>>
> >>>
> >>> --Chris
> >>> _______________________________________________
> >>> Distutils-SIG maillist  -  Distutils-SIG at python.org
> >>> https://mail.python.org/mailman/listinfo/distutils-sig
> >>
> >>
> >
> >
> > _______________________________________________
> > Distutils-SIG maillist  -  Distutils-SIG at python.org
> > https://mail.python.org/mailman/listinfo/distutils-sig
> >
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131027/dba814a7/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131027/dba814a7/attachment.sig>

From r1chardj0n3s at gmail.com  Mon Oct 28 00:53:21 2013
From: r1chardj0n3s at gmail.com (Richard Jones)
Date: Mon, 28 Oct 2013 10:53:21 +1100
Subject: [Distutils] PyPI pull request
In-Reply-To: <CAPkN8xLvAsArPaf3T-V+86-6EhwUwV4BFtfdJw=a9ce4DNyS8g@mail.gmail.com>
References: <CAPkN8xLvAsArPaf3T-V+86-6EhwUwV4BFtfdJw=a9ce4DNyS8g@mail.gmail.com>
Message-ID: <CAHrZfZD7hkkkkXiY1XU_QtfuKrQVcJn8SOb9bkbTq3kjUmyhNQ@mail.gmail.com>

I have merged that PR but I really don't see any point in making any
changes to the current codebase beyond fixing significant issues. Cleaning
it up is not a priority. I've merged this PR to clean up the PyPI project
page on bitbucket a little, but I would ask that no further cosmetic PRs be
submitted, thanks.

Warehouse is the name of the next version of PyPI being developed by Donald
Stufft.


     Richard



On 27 October 2013 17:49, anatoly techtonik <techtonik at gmail.com> wrote:

> I've heard that there is PyPI 2.0, but I still find current PyPI code
> to be very suitable for educational purposes (unlike some complicated
> framework based solutions, where much of the stuff is hidden in
> internals of external lib abstractions), so I continue to send fixes
> to improve code base.
>
> Please merge this one.
>
> https://bitbucket.org/pypa/pypi/pull-request/6/remove-unused-templatetoolspy-file/
> --
> anatoly t.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131028/531e8d2c/attachment.html>

From techtonik at gmail.com  Mon Oct 28 00:58:03 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 28 Oct 2013 02:58:03 +0300
Subject: [Distutils] PyPI pull request
In-Reply-To: <CAHrZfZD7hkkkkXiY1XU_QtfuKrQVcJn8SOb9bkbTq3kjUmyhNQ@mail.gmail.com>
References: <CAPkN8xLvAsArPaf3T-V+86-6EhwUwV4BFtfdJw=a9ce4DNyS8g@mail.gmail.com>
 <CAHrZfZD7hkkkkXiY1XU_QtfuKrQVcJn8SOb9bkbTq3kjUmyhNQ@mail.gmail.com>
Message-ID: <CAPkN8xLhXnEmP875dxKJyPON-f=Hbz6JKQ=DmBcj75Za1fjNWQ@mail.gmail.com>

Thanks.

Warehouse sounds very enterprisey. Any Roadmap for that, estimate time
to become operational? I'd need some features right now and not next
PyCon. Also, am I right that bus factor for this stuff is one?
--
anatoly t.


On Mon, Oct 28, 2013 at 2:53 AM, Richard Jones <r1chardj0n3s at gmail.com> wrote:
> I have merged that PR but I really don't see any point in making any changes
> to the current codebase beyond fixing significant issues. Cleaning it up is
> not a priority. I've merged this PR to clean up the PyPI project page on
> bitbucket a little, but I would ask that no further cosmetic PRs be
> submitted, thanks.
>
> Warehouse is the name of the next version of PyPI being developed by Donald
> Stufft.
>
>
>      Richard
>
>
>
> On 27 October 2013 17:49, anatoly techtonik <techtonik at gmail.com> wrote:
>>
>> I've heard that there is PyPI 2.0, but I still find current PyPI code
>> to be very suitable for educational purposes (unlike some complicated
>> framework based solutions, where much of the stuff is hidden in
>> internals of external lib abstractions), so I continue to send fixes
>> to improve code base.
>>
>> Please merge this one.
>>
>> https://bitbucket.org/pypa/pypi/pull-request/6/remove-unused-templatetoolspy-file/
>> --
>> anatoly t.
>
>

From r1chardj0n3s at gmail.com  Mon Oct 28 01:00:30 2013
From: r1chardj0n3s at gmail.com (Richard Jones)
Date: Mon, 28 Oct 2013 11:00:30 +1100
Subject: [Distutils] PyPI pull request
In-Reply-To: <CAPkN8xLhXnEmP875dxKJyPON-f=Hbz6JKQ=DmBcj75Za1fjNWQ@mail.gmail.com>
References: <CAPkN8xLvAsArPaf3T-V+86-6EhwUwV4BFtfdJw=a9ce4DNyS8g@mail.gmail.com>
 <CAHrZfZD7hkkkkXiY1XU_QtfuKrQVcJn8SOb9bkbTq3kjUmyhNQ@mail.gmail.com>
 <CAPkN8xLhXnEmP875dxKJyPON-f=Hbz6JKQ=DmBcj75Za1fjNWQ@mail.gmail.com>
Message-ID: <CAHrZfZAzSPN2vS95_ty2oj89vBfRYtC52t5NvQ75uCXabTGm8g@mail.gmail.com>

I'm not sure what you mean by it sounding "enterprisey" except perhaps just
the name?


On 28 October 2013 10:58, anatoly techtonik <techtonik at gmail.com> wrote:

> Thanks.
>
> Warehouse sounds very enterprisey. Any Roadmap for that, estimate time
> to become operational? I'd need some features right now and not next
> PyCon. Also, am I right that bus factor for this stuff is one?
> --
> anatoly t.
>
>
> On Mon, Oct 28, 2013 at 2:53 AM, Richard Jones <r1chardj0n3s at gmail.com>
> wrote:
> > I have merged that PR but I really don't see any point in making any
> changes
> > to the current codebase beyond fixing significant issues. Cleaning it up
> is
> > not a priority. I've merged this PR to clean up the PyPI project page on
> > bitbucket a little, but I would ask that no further cosmetic PRs be
> > submitted, thanks.
> >
> > Warehouse is the name of the next version of PyPI being developed by
> Donald
> > Stufft.
> >
> >
> >      Richard
> >
> >
> >
> > On 27 October 2013 17:49, anatoly techtonik <techtonik at gmail.com> wrote:
> >>
> >> I've heard that there is PyPI 2.0, but I still find current PyPI code
> >> to be very suitable for educational purposes (unlike some complicated
> >> framework based solutions, where much of the stuff is hidden in
> >> internals of external lib abstractions), so I continue to send fixes
> >> to improve code base.
> >>
> >> Please merge this one.
> >>
> >>
> https://bitbucket.org/pypa/pypi/pull-request/6/remove-unused-templatetoolspy-file/
> >> --
> >> anatoly t.
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131028/a22a1774/attachment-0001.html>

From techtonik at gmail.com  Mon Oct 28 06:02:59 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 28 Oct 2013 08:02:59 +0300
Subject: [Distutils] PyPI pull request
In-Reply-To: <CAHrZfZAzSPN2vS95_ty2oj89vBfRYtC52t5NvQ75uCXabTGm8g@mail.gmail.com>
References: <CAPkN8xLvAsArPaf3T-V+86-6EhwUwV4BFtfdJw=a9ce4DNyS8g@mail.gmail.com>
 <CAHrZfZD7hkkkkXiY1XU_QtfuKrQVcJn8SOb9bkbTq3kjUmyhNQ@mail.gmail.com>
 <CAPkN8xLhXnEmP875dxKJyPON-f=Hbz6JKQ=DmBcj75Za1fjNWQ@mail.gmail.com>
 <CAHrZfZAzSPN2vS95_ty2oj89vBfRYtC52t5NvQ75uCXabTGm8g@mail.gmail.com>
Message-ID: <CAPkN8xLumGJTwo2pe=YpfbgqDBrYRQ7Xpt+UNC84JydOsPKB7g@mail.gmail.com>

I mean that the name CheeseShop has more human touch in it than Warehouse.
--
anatoly t.


On Mon, Oct 28, 2013 at 3:00 AM, Richard Jones <r1chardj0n3s at gmail.com> wrote:
> I'm not sure what you mean by it sounding "enterprisey" except perhaps just
> the name?
>
>
> On 28 October 2013 10:58, anatoly techtonik <techtonik at gmail.com> wrote:
>>
>> Thanks.
>>
>> Warehouse sounds very enterprisey. Any Roadmap for that, estimate time
>> to become operational? I'd need some features right now and not next
>> PyCon. Also, am I right that bus factor for this stuff is one?
>> --
>> anatoly t.
>>
>>
>> On Mon, Oct 28, 2013 at 2:53 AM, Richard Jones <r1chardj0n3s at gmail.com>
>> wrote:
>> > I have merged that PR but I really don't see any point in making any
>> > changes
>> > to the current codebase beyond fixing significant issues. Cleaning it up
>> > is
>> > not a priority. I've merged this PR to clean up the PyPI project page on
>> > bitbucket a little, but I would ask that no further cosmetic PRs be
>> > submitted, thanks.
>> >
>> > Warehouse is the name of the next version of PyPI being developed by
>> > Donald
>> > Stufft.
>> >
>> >
>> >      Richard
>> >
>> >
>> >
>> > On 27 October 2013 17:49, anatoly techtonik <techtonik at gmail.com> wrote:
>> >>
>> >> I've heard that there is PyPI 2.0, but I still find current PyPI code
>> >> to be very suitable for educational purposes (unlike some complicated
>> >> framework based solutions, where much of the stuff is hidden in
>> >> internals of external lib abstractions), so I continue to send fixes
>> >> to improve code base.
>> >>
>> >> Please merge this one.
>> >>
>> >>
>> >> https://bitbucket.org/pypa/pypi/pull-request/6/remove-unused-templatetoolspy-file/
>> >> --
>> >> anatoly t.
>> >
>> >
>
>

From noah at coderanger.net  Mon Oct 28 06:20:34 2013
From: noah at coderanger.net (Noah Kantrowitz)
Date: Sun, 27 Oct 2013 22:20:34 -0700
Subject: [Distutils] PyPI pull request
In-Reply-To: <CAPkN8xLumGJTwo2pe=YpfbgqDBrYRQ7Xpt+UNC84JydOsPKB7g@mail.gmail.com>
References: <CAPkN8xLvAsArPaf3T-V+86-6EhwUwV4BFtfdJw=a9ce4DNyS8g@mail.gmail.com>
 <CAHrZfZD7hkkkkXiY1XU_QtfuKrQVcJn8SOb9bkbTq3kjUmyhNQ@mail.gmail.com>
 <CAPkN8xLhXnEmP875dxKJyPON-f=Hbz6JKQ=DmBcj75Za1fjNWQ@mail.gmail.com>
 <CAHrZfZAzSPN2vS95_ty2oj89vBfRYtC52t5NvQ75uCXabTGm8g@mail.gmail.com>
 <CAPkN8xLumGJTwo2pe=YpfbgqDBrYRQ7Xpt+UNC84JydOsPKB7g@mail.gmail.com>
Message-ID: <5A8787FD-413B-45EA-87A5-817BA9BEE184@coderanger.net>

Warehouse is the internal project name, and will be just one software component of the service collectively known as PyPI. That said, Donald started it so by law of the jungle he can call it whatever he wants as long as I don't get phone calls from the FBI.

--Noah

On Oct 27, 2013, at 10:02 PM, anatoly techtonik <techtonik at gmail.com> wrote:

> I mean that the name CheeseShop has more human touch in it than Warehouse.
> --
> anatoly t.
> 
> 
> On Mon, Oct 28, 2013 at 3:00 AM, Richard Jones <r1chardj0n3s at gmail.com> wrote:
>> I'm not sure what you mean by it sounding "enterprisey" except perhaps just
>> the name?
>> 
>> 
>> On 28 October 2013 10:58, anatoly techtonik <techtonik at gmail.com> wrote:
>>> 
>>> Thanks.
>>> 
>>> Warehouse sounds very enterprisey. Any Roadmap for that, estimate time
>>> to become operational? I'd need some features right now and not next
>>> PyCon. Also, am I right that bus factor for this stuff is one?
>>> --
>>> anatoly t.
>>> 
>>> 
>>> On Mon, Oct 28, 2013 at 2:53 AM, Richard Jones <r1chardj0n3s at gmail.com>
>>> wrote:
>>>> I have merged that PR but I really don't see any point in making any
>>>> changes
>>>> to the current codebase beyond fixing significant issues. Cleaning it up
>>>> is
>>>> not a priority. I've merged this PR to clean up the PyPI project page on
>>>> bitbucket a little, but I would ask that no further cosmetic PRs be
>>>> submitted, thanks.
>>>> 
>>>> Warehouse is the name of the next version of PyPI being developed by
>>>> Donald
>>>> Stufft.
>>>> 
>>>> 
>>>>     Richard
>>>> 
>>>> 
>>>> 
>>>> On 27 October 2013 17:49, anatoly techtonik <techtonik at gmail.com> wrote:
>>>>> 
>>>>> I've heard that there is PyPI 2.0, but I still find current PyPI code
>>>>> to be very suitable for educational purposes (unlike some complicated
>>>>> framework based solutions, where much of the stuff is hidden in
>>>>> internals of external lib abstractions), so I continue to send fixes
>>>>> to improve code base.
>>>>> 
>>>>> Please merge this one.
>>>>> 
>>>>> 
>>>>> https://bitbucket.org/pypa/pypi/pull-request/6/remove-unused-templatetoolspy-file/
>>>>> --
>>>>> anatoly t.
>>>> 
>>>> 
>> 
>> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131027/9d92c870/attachment.sig>

From aclark at aclark.net  Mon Oct 28 11:23:31 2013
From: aclark at aclark.net (Alex Clark)
Date: Mon, 28 Oct 2013 10:23:31 +0000 (UTC)
Subject: [Distutils] PyPI pull request
References: <CAPkN8xLvAsArPaf3T-V+86-6EhwUwV4BFtfdJw=a9ce4DNyS8g@mail.gmail.com>
 <CAHrZfZD7hkkkkXiY1XU_QtfuKrQVcJn8SOb9bkbTq3kjUmyhNQ@mail.gmail.com>
 <CAPkN8xLhXnEmP875dxKJyPON-f=Hbz6JKQ=DmBcj75Za1fjNWQ@mail.gmail.com>
 <CAHrZfZAzSPN2vS95_ty2oj89vBfRYtC52t5NvQ75uCXabTGm8g@mail.gmail.com>
 <CAPkN8xLumGJTwo2pe=YpfbgqDBrYRQ7Xpt+UNC84JydOsPKB7g@mail.gmail.com>
 <5A8787FD-413B-45EA-87A5-817BA9BEE184@coderanger.net>
Message-ID: <loom.20131028T112156-586@post.gmane.org>

Noah Kantrowitz <noah <at> coderanger.net> writes:

> Warehouse is the internal project name

Heh, well if we are bike shedding: IIRC, CheeseShop fell out of favor for
being too casual and/or too much of an inside joke.


From donald at stufft.io  Mon Oct 28 12:29:11 2013
From: donald at stufft.io (Donald Stufft)
Date: Mon, 28 Oct 2013 07:29:11 -0400
Subject: [Distutils] PyPI pull request
In-Reply-To: <5A8787FD-413B-45EA-87A5-817BA9BEE184@coderanger.net>
References: <CAPkN8xLvAsArPaf3T-V+86-6EhwUwV4BFtfdJw=a9ce4DNyS8g@mail.gmail.com>
 <CAHrZfZD7hkkkkXiY1XU_QtfuKrQVcJn8SOb9bkbTq3kjUmyhNQ@mail.gmail.com>
 <CAPkN8xLhXnEmP875dxKJyPON-f=Hbz6JKQ=DmBcj75Za1fjNWQ@mail.gmail.com>
 <CAHrZfZAzSPN2vS95_ty2oj89vBfRYtC52t5NvQ75uCXabTGm8g@mail.gmail.com>
 <CAPkN8xLumGJTwo2pe=YpfbgqDBrYRQ7Xpt+UNC84JydOsPKB7g@mail.gmail.com>
 <5A8787FD-413B-45EA-87A5-817BA9BEE184@coderanger.net>
Message-ID: <B6973CB3-A828-41F0-AD9A-5E44F3C88480@stufft.io>

Like Noah said, it?s just an internal name to make it easier to talk about the
web application portion of PyPI so we don?t have to say pypi the web app
vs PyPI the service/website.

I thought the name was funny :( You put packages in a Warehouse :3

On Oct 28, 2013, at 1:20 AM, Noah Kantrowitz <noah at coderanger.net> wrote:

> Warehouse is the internal project name, and will be just one software component of the service collectively known as PyPI. That said, Donald started it so by law of the jungle he can call it whatever he wants as long as I don't get phone calls from the FBI.
> 
> --Noah
> 
> On Oct 27, 2013, at 10:02 PM, anatoly techtonik <techtonik at gmail.com> wrote:
> 
>> I mean that the name CheeseShop has more human touch in it than Warehouse.
>> --
>> anatoly t.
>> 
>> 
>> On Mon, Oct 28, 2013 at 3:00 AM, Richard Jones <r1chardj0n3s at gmail.com> wrote:
>>> I'm not sure what you mean by it sounding "enterprisey" except perhaps just
>>> the name?
>>> 
>>> 
>>> On 28 October 2013 10:58, anatoly techtonik <techtonik at gmail.com> wrote:
>>>> 
>>>> Thanks.
>>>> 
>>>> Warehouse sounds very enterprisey. Any Roadmap for that, estimate time
>>>> to become operational? I'd need some features right now and not next
>>>> PyCon. Also, am I right that bus factor for this stuff is one?
>>>> --
>>>> anatoly t.
>>>> 
>>>> 
>>>> On Mon, Oct 28, 2013 at 2:53 AM, Richard Jones <r1chardj0n3s at gmail.com>
>>>> wrote:
>>>>> I have merged that PR but I really don't see any point in making any
>>>>> changes
>>>>> to the current codebase beyond fixing significant issues. Cleaning it up
>>>>> is
>>>>> not a priority. I've merged this PR to clean up the PyPI project page on
>>>>> bitbucket a little, but I would ask that no further cosmetic PRs be
>>>>> submitted, thanks.
>>>>> 
>>>>> Warehouse is the name of the next version of PyPI being developed by
>>>>> Donald
>>>>> Stufft.
>>>>> 
>>>>> 
>>>>>    Richard
>>>>> 
>>>>> 
>>>>> 
>>>>> On 27 October 2013 17:49, anatoly techtonik <techtonik at gmail.com> wrote:
>>>>>> 
>>>>>> I've heard that there is PyPI 2.0, but I still find current PyPI code
>>>>>> to be very suitable for educational purposes (unlike some complicated
>>>>>> framework based solutions, where much of the stuff is hidden in
>>>>>> internals of external lib abstractions), so I continue to send fixes
>>>>>> to improve code base.
>>>>>> 
>>>>>> Please merge this one.
>>>>>> 
>>>>>> 
>>>>>> https://bitbucket.org/pypa/pypi/pull-request/6/remove-unused-templatetoolspy-file/
>>>>>> --
>>>>>> anatoly t.
>>>>> 
>>>>> 
>>> 
>>> 
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131028/aa4845d2/attachment.sig>

From ncoghlan at gmail.com  Mon Oct 28 14:50:21 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 28 Oct 2013 23:50:21 +1000
Subject: [Distutils] PyPI pull request
In-Reply-To: <B6973CB3-A828-41F0-AD9A-5E44F3C88480@stufft.io>
References: <CAPkN8xLvAsArPaf3T-V+86-6EhwUwV4BFtfdJw=a9ce4DNyS8g@mail.gmail.com>
 <CAHrZfZD7hkkkkXiY1XU_QtfuKrQVcJn8SOb9bkbTq3kjUmyhNQ@mail.gmail.com>
 <CAPkN8xLhXnEmP875dxKJyPON-f=Hbz6JKQ=DmBcj75Za1fjNWQ@mail.gmail.com>
 <CAHrZfZAzSPN2vS95_ty2oj89vBfRYtC52t5NvQ75uCXabTGm8g@mail.gmail.com>
 <CAPkN8xLumGJTwo2pe=YpfbgqDBrYRQ7Xpt+UNC84JydOsPKB7g@mail.gmail.com>
 <5A8787FD-413B-45EA-87A5-817BA9BEE184@coderanger.net>
 <B6973CB3-A828-41F0-AD9A-5E44F3C88480@stufft.io>
Message-ID: <CADiSq7dthWhJuWueoQe_4hEi5r0k0QWyAy_NBarwjpDMLOQW5g@mail.gmail.com>

On 28 Oct 2013 21:29, "Donald Stufft" <donald at stufft.io> wrote:
>
> Like Noah said, it?s just an internal name to make it easier to talk
about the
> web application portion of PyPI so we don?t have to say pypi the web app
> vs PyPI the service/website.

Right, PyPI.next is an installation of Warehouse, but it makes sense to
separate the identity of the software from the particular service.

> I thought the name was funny :( You put packages in a Warehouse :3

Yeah, the name's fine.

Cheers,
Nick.

>
> On Oct 28, 2013, at 1:20 AM, Noah Kantrowitz <noah at coderanger.net> wrote:
>
> > Warehouse is the internal project name, and will be just one software
component of the service collectively known as PyPI. That said, Donald
started it so by law of the jungle he can call it whatever he wants as long
as I don't get phone calls from the FBI.
> >
> > --Noah
> >
> > On Oct 27, 2013, at 10:02 PM, anatoly techtonik <techtonik at gmail.com>
wrote:
> >
> >> I mean that the name CheeseShop has more human touch in it than
Warehouse.
> >> --
> >> anatoly t.
> >>
> >>
> >> On Mon, Oct 28, 2013 at 3:00 AM, Richard Jones <r1chardj0n3s at gmail.com>
wrote:
> >>> I'm not sure what you mean by it sounding "enterprisey" except
perhaps just
> >>> the name?
> >>>
> >>>
> >>> On 28 October 2013 10:58, anatoly techtonik <techtonik at gmail.com>
wrote:
> >>>>
> >>>> Thanks.
> >>>>
> >>>> Warehouse sounds very enterprisey. Any Roadmap for that, estimate
time
> >>>> to become operational? I'd need some features right now and not next
> >>>> PyCon. Also, am I right that bus factor for this stuff is one?
> >>>> --
> >>>> anatoly t.
> >>>>
> >>>>
> >>>> On Mon, Oct 28, 2013 at 2:53 AM, Richard Jones <
r1chardj0n3s at gmail.com>
> >>>> wrote:
> >>>>> I have merged that PR but I really don't see any point in making any
> >>>>> changes
> >>>>> to the current codebase beyond fixing significant issues. Cleaning
it up
> >>>>> is
> >>>>> not a priority. I've merged this PR to clean up the PyPI project
page on
> >>>>> bitbucket a little, but I would ask that no further cosmetic PRs be
> >>>>> submitted, thanks.
> >>>>>
> >>>>> Warehouse is the name of the next version of PyPI being developed by
> >>>>> Donald
> >>>>> Stufft.
> >>>>>
> >>>>>
> >>>>>    Richard
> >>>>>
> >>>>>
> >>>>>
> >>>>> On 27 October 2013 17:49, anatoly techtonik <techtonik at gmail.com>
wrote:
> >>>>>>
> >>>>>> I've heard that there is PyPI 2.0, but I still find current PyPI
code
> >>>>>> to be very suitable for educational purposes (unlike some
complicated
> >>>>>> framework based solutions, where much of the stuff is hidden in
> >>>>>> internals of external lib abstractions), so I continue to send
fixes
> >>>>>> to improve code base.
> >>>>>>
> >>>>>> Please merge this one.
> >>>>>>
> >>>>>>
> >>>>>>
https://bitbucket.org/pypa/pypi/pull-request/6/remove-unused-templatetoolspy-file/
> >>>>>> --
> >>>>>> anatoly t.
> >>>>>
> >>>>>
> >>>
> >>>
> >> _______________________________________________
> >> Distutils-SIG maillist  -  Distutils-SIG at python.org
> >> https://mail.python.org/mailman/listinfo/distutils-sig
> >
> > _______________________________________________
> > Distutils-SIG maillist  -  Distutils-SIG at python.org
> > https://mail.python.org/mailman/listinfo/distutils-sig
>
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
DCFA
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131028/11c97200/attachment.html>

From techtonik at gmail.com  Mon Oct 28 15:10:45 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 28 Oct 2013 17:10:45 +0300
Subject: [Distutils] PyPI pull request
In-Reply-To: <B6973CB3-A828-41F0-AD9A-5E44F3C88480@stufft.io>
References: <CAPkN8xLvAsArPaf3T-V+86-6EhwUwV4BFtfdJw=a9ce4DNyS8g@mail.gmail.com>
 <CAHrZfZD7hkkkkXiY1XU_QtfuKrQVcJn8SOb9bkbTq3kjUmyhNQ@mail.gmail.com>
 <CAPkN8xLhXnEmP875dxKJyPON-f=Hbz6JKQ=DmBcj75Za1fjNWQ@mail.gmail.com>
 <CAHrZfZAzSPN2vS95_ty2oj89vBfRYtC52t5NvQ75uCXabTGm8g@mail.gmail.com>
 <CAPkN8xLumGJTwo2pe=YpfbgqDBrYRQ7Xpt+UNC84JydOsPKB7g@mail.gmail.com>
 <5A8787FD-413B-45EA-87A5-817BA9BEE184@coderanger.net>
 <B6973CB3-A828-41F0-AD9A-5E44F3C88480@stufft.io>
Message-ID: <CAPkN8xJGkP6r=M1fD5Doi2sjVQ7t_bXtpgRYakD5Rvx=Bn0adg@mail.gmail.com>

Warehouse sounds scary. Thousands of pythons in packages were
constantly delivered to Warehouse, to be taken away by cold internet
machines and work for strange mechanisms. =)

I am ok with tmp name.

So, is there any plans or Roadmaps for Warehouse? I mean it is no good
if nobody is going to approve pull request things until Warehouse is
ready.
--
anatoly t.


On Mon, Oct 28, 2013 at 2:29 PM, Donald Stufft <donald at stufft.io> wrote:
> Like Noah said, it?s just an internal name to make it easier to talk about the
> web application portion of PyPI so we don?t have to say pypi the web app
> vs PyPI the service/website.
>
> I thought the name was funny :( You put packages in a Warehouse :3
>
> On Oct 28, 2013, at 1:20 AM, Noah Kantrowitz <noah at coderanger.net> wrote:
>
>> Warehouse is the internal project name, and will be just one software component of the service collectively known as PyPI. That said, Donald started it so by law of the jungle he can call it whatever he wants as long as I don't get phone calls from the FBI.
>>
>> --Noah
>>
>> On Oct 27, 2013, at 10:02 PM, anatoly techtonik <techtonik at gmail.com> wrote:
>>
>>> I mean that the name CheeseShop has more human touch in it than Warehouse.
>>> --
>>> anatoly t.
>>>
>>>
>>> On Mon, Oct 28, 2013 at 3:00 AM, Richard Jones <r1chardj0n3s at gmail.com> wrote:
>>>> I'm not sure what you mean by it sounding "enterprisey" except perhaps just
>>>> the name?
>>>>
>>>>
>>>> On 28 October 2013 10:58, anatoly techtonik <techtonik at gmail.com> wrote:
>>>>>
>>>>> Thanks.
>>>>>
>>>>> Warehouse sounds very enterprisey. Any Roadmap for that, estimate time
>>>>> to become operational? I'd need some features right now and not next
>>>>> PyCon. Also, am I right that bus factor for this stuff is one?
>>>>> --
>>>>> anatoly t.
>>>>>
>>>>>
>>>>> On Mon, Oct 28, 2013 at 2:53 AM, Richard Jones <r1chardj0n3s at gmail.com>
>>>>> wrote:
>>>>>> I have merged that PR but I really don't see any point in making any
>>>>>> changes
>>>>>> to the current codebase beyond fixing significant issues. Cleaning it up
>>>>>> is
>>>>>> not a priority. I've merged this PR to clean up the PyPI project page on
>>>>>> bitbucket a little, but I would ask that no further cosmetic PRs be
>>>>>> submitted, thanks.
>>>>>>
>>>>>> Warehouse is the name of the next version of PyPI being developed by
>>>>>> Donald
>>>>>> Stufft.
>>>>>>
>>>>>>
>>>>>>    Richard
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 27 October 2013 17:49, anatoly techtonik <techtonik at gmail.com> wrote:
>>>>>>>
>>>>>>> I've heard that there is PyPI 2.0, but I still find current PyPI code
>>>>>>> to be very suitable for educational purposes (unlike some complicated
>>>>>>> framework based solutions, where much of the stuff is hidden in
>>>>>>> internals of external lib abstractions), so I continue to send fixes
>>>>>>> to improve code base.
>>>>>>>
>>>>>>> Please merge this one.
>>>>>>>
>>>>>>>
>>>>>>> https://bitbucket.org/pypa/pypi/pull-request/6/remove-unused-templatetoolspy-file/
>>>>>>> --
>>>>>>> anatoly t.
>>>>>>
>>>>>>
>>>>
>>>>
>>> _______________________________________________
>>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>

From fred at fdrake.net  Mon Oct 28 15:15:02 2013
From: fred at fdrake.net (Fred Drake)
Date: Mon, 28 Oct 2013 10:15:02 -0400
Subject: [Distutils] PyPI pull request
In-Reply-To: <B6973CB3-A828-41F0-AD9A-5E44F3C88480@stufft.io>
References: <CAPkN8xLvAsArPaf3T-V+86-6EhwUwV4BFtfdJw=a9ce4DNyS8g@mail.gmail.com>
 <CAHrZfZD7hkkkkXiY1XU_QtfuKrQVcJn8SOb9bkbTq3kjUmyhNQ@mail.gmail.com>
 <CAPkN8xLhXnEmP875dxKJyPON-f=Hbz6JKQ=DmBcj75Za1fjNWQ@mail.gmail.com>
 <CAHrZfZAzSPN2vS95_ty2oj89vBfRYtC52t5NvQ75uCXabTGm8g@mail.gmail.com>
 <CAPkN8xLumGJTwo2pe=YpfbgqDBrYRQ7Xpt+UNC84JydOsPKB7g@mail.gmail.com>
 <5A8787FD-413B-45EA-87A5-817BA9BEE184@coderanger.net>
 <B6973CB3-A828-41F0-AD9A-5E44F3C88480@stufft.io>
Message-ID: <CAFT4OTHMa=f-gyf0NV84XEm=y=8FDPk=xBmwedwmzEwnGqL8XQ@mail.gmail.com>

On Mon, Oct 28, 2013 at 7:29 AM, Donald Stufft <donald at stufft.io> wrote:
> I thought the name was funny :( You put packages in a Warehouse :3

And Warehouse 13 is where the troublesome packages are stored.  :-)


  -Fred

-- 
Fred L. Drake, Jr.    <fred at fdrake.net>
"A storm broke loose in my mind."  --Albert Einstein

From donald at stufft.io  Mon Oct 28 15:17:40 2013
From: donald at stufft.io (Donald Stufft)
Date: Mon, 28 Oct 2013 10:17:40 -0400
Subject: [Distutils] PyPI pull request
In-Reply-To: <CAPkN8xJGkP6r=M1fD5Doi2sjVQ7t_bXtpgRYakD5Rvx=Bn0adg@mail.gmail.com>
References: <CAPkN8xLvAsArPaf3T-V+86-6EhwUwV4BFtfdJw=a9ce4DNyS8g@mail.gmail.com>
 <CAHrZfZD7hkkkkXiY1XU_QtfuKrQVcJn8SOb9bkbTq3kjUmyhNQ@mail.gmail.com>
 <CAPkN8xLhXnEmP875dxKJyPON-f=Hbz6JKQ=DmBcj75Za1fjNWQ@mail.gmail.com>
 <CAHrZfZAzSPN2vS95_ty2oj89vBfRYtC52t5NvQ75uCXabTGm8g@mail.gmail.com>
 <CAPkN8xLumGJTwo2pe=YpfbgqDBrYRQ7Xpt+UNC84JydOsPKB7g@mail.gmail.com>
 <5A8787FD-413B-45EA-87A5-817BA9BEE184@coderanger.net>
 <B6973CB3-A828-41F0-AD9A-5E44F3C88480@stufft.io>
 <CAPkN8xJGkP6r=M1fD5Doi2sjVQ7t_bXtpgRYakD5Rvx=Bn0adg@mail.gmail.com>
Message-ID: <C44B74A3-4715-411B-ADDD-AE176ABF3F60@stufft.io>


On Oct 28, 2013, at 10:10 AM, anatoly techtonik <techtonik at gmail.com> wrote:

> So, is there any plans or Roadmaps for Warehouse? I mean it is no good
> if nobody is going to approve pull request things until Warehouse is
> ready.

Currently Warehouse is focused on:

 A) Getting API compatibility with the current implementation [1]
 B) Getting a UI in place that can replace the existing UI [2]

Overall as I think of more things we need for Warehouse to ?Become PyPI? I
add it to the ?Becoming PyPI? milestone [3]

[1] https://github.com/dstufft/warehouse/issues/58
[2] https://github.com/dstufft/warehouse/issues/63
[3] https://github.com/dstufft/warehouse/issues?milestone=1&state=open

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131028/ccd26e2c/attachment.sig>

From michaelvantellingen at gmail.com  Mon Oct 28 15:45:41 2013
From: michaelvantellingen at gmail.com (Michael van Tellingen)
Date: Mon, 28 Oct 2013 15:45:41 +0100
Subject: [Distutils] Removing dependency_links
In-Reply-To: <CAOTb1wd7NpzmWgJz=jro5M+FNoaY072_b3akGyM=qSHvRj6DFQ@mail.gmail.com>
References: <76C3FDF4-4EA3-4FCE-9D84-F61E62905FCC@stufft.io>
 <80B80AA9-AF7D-4CF8-B7FA-FD2C60AD48A0@stufft.io>
 <088874FF-04C2-4201-9C07-9846A20D2818@stufft.io>
 <0BEC2FEB-5555-4696-AC2C-368816C29069@stufft.io>
 <EDF0B075-0A3C-45CD-B1F7-5DC832724185@stufft.io>
 <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
 <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>
 <CADiSq7cLNgwVmfgO94NQZkEMCMREkOr3oXfWTh6SsOThwQq6Uw@mail.gmail.com>
 <CAPYWazpcjRP-0xCE0=e0fJUCGNS5vTJpUt20Us95QEqDVMsN+Q@mail.gmail.com>
 <CADiSq7fW6uqWGDz-MGotgX0rL1c+uOyfAj+gMGFPwqkx3GjXUg@mail.gmail.com>
 <3FC26D14-AB77-49DB-9856-E55C75FE5F7B@stufft.io>
 <D6B8D266-925E-4706-89CF-B0F4FA541EDC@stufft.io>
 <CAOTb1wfba-Y2zvpwv1GQCs4tAwqmg=8f69G+xS9e1se+6XY_DA@mail.gmail.com>
 <CAOTb1wd7NpzmWgJz=jro5M+FNoaY072_b3akGyM=qSHvRj6DFQ@mail.gmail.com>
Message-ID: <CAHN_fn2JYJD3buUGLa7AyK-uz1MXuysirDJNJTBQZfs7SYq14g@mail.gmail.com>

On Sun, Oct 27, 2013 at 10:49 PM, Chris Jerdonek
<chris.jerdonek at gmail.com>wrote:

> On Sun, Oct 27, 2013 at 12:04 PM, Chris Jerdonek
> <chris.jerdonek at gmail.com> wrote:
> >
> > On Sun, Oct 27, 2013 at 10:44 AM, Donald Stufft <donald at stufft.io>
> wrote:
> >>
> >> Here?s the list of dependency links for the projects that still use
> them in their latest releases:
> >>
> >> https://gist.github.com/dstufft/7185162
> >>
> >> A good number of them are either bogus, are pointing directly to PyPI,
> or are file:// urls that are highly unlikely to exist on anyones computer
> but the author?s. All in all there are 307 total unique links in this set
> of packages, and 99 of them are not reachable from my computer
> (requests.get(?) raises an exception).
> >
> > I actually know a couple people on this list.  I can ask them and see if
> the list can be reduced further. :)
>
> So I asked the person I know, and this is what he said, "Yes, we have
> to use it!  It's the only way to allow a package to install other
> packages that aren't on PyPI-- for instance, a custom fork of a
> library."
>
> Is there another approach or work-around he can be using?  What is the
> "right" way for him to do it?
>
> --Chris


Note that some projects also use the dependency_links to point to
development packages. See for example
https://github.com/tangentlabs/django-oscar/blob/master/setup.py#L63

Although this approach doesn't always work reliably in my experience
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131028/026a6e18/attachment.html>

From lincoln at clarete.li  Mon Oct 28 21:39:15 2013
From: lincoln at clarete.li (Lincoln Clarete)
Date: Mon, 28 Oct 2013 16:39:15 -0400
Subject: [Distutils] Curdling: Concurrent package installer for python
Message-ID: <CAAX-7Wz+Ea=ym5r9oN5hvTOan3G9Mbzfz2fR5b3_XinKhGRB=A@mail.gmail.com>

Hi all,

I've been working on a project called curdling. It's a package manager for
python. It's pretty similar to pip but with a brand new design and way less
lines of code. It uses exciting concurrency technology to provide a
reliable and fast installer for Python packages.

The current feature set includes:

 * Robust Concurrent model: it?s FAST!
 * Really good Error Handling and Report;
 * Conflict resolution for repeated requirements;
 * Distributed Cache System that includes a built-in cache server;
 * Simple command line interface;
 * Usage of bleeding edge technology available in the Python community;
 * Concurrent and Parallel, but Ctrl-C still works;

It's currently a work in progress but I'm pretty excited about its
stability and performance. There's a good amount of documentation available
at

  http://clarete.github.io/curdling

This project uses both wheel and distlib packages as part of the dependency
stack. I'm working on an essay about my experience with both libraries that
I'm planning to share on another opportunity. It will probably be available
under the "Design and Implementation" section in the main website. Feel
free to check it out right now if you don't want to wait:

  http://clarete.github.io/curdling/design-and-implementation.html

Any feedback is highly appreciated.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131028/dd046988/attachment.html>

From martin.fiers at intec.ugent.be  Mon Oct 28 09:29:57 2013
From: martin.fiers at intec.ugent.be (Martin Fiers)
Date: Mon, 28 Oct 2013 09:29:57 +0100
Subject: [Distutils] Egg name computation
In-Reply-To: <CALeMXf53xTaCyVOvd6L_yw3VeVojasPvfhJMq46qkc1hVNPJ8g@mail.gmail.com>
References: <525D2FE9.6050904@intec.ugent.be>
 <CALeMXf53xTaCyVOvd6L_yw3VeVojasPvfhJMq46qkc1hVNPJ8g@mail.gmail.com>
Message-ID: <526E2085.30100@intec.ugent.be>

On 10/25/2013 04:25 AM, PJ Eby wrote:
> On Tue, Oct 15, 2013 at 8:07 AM, Martin Fiers
> <Martin.Fiers at intec.ugent.be> wrote:
>> So the platform argument now is
>> self.distribution.has_ext_modules() and self.plat_name
>> Shouldn't it just be
>> self.plat_name
>> ?
> No.  The platform name is only included if the distribution has
> extension modules, because extension modules are what make the egg
> platform-specific.  If there is only Python code and data, then the
> egg is considered platform independent.
Ok, I understand the reasoning.
>
>
>> Would there be a workaround?
> What do you want to work around this *for*?  If the egg doesn't
> contain extension modules, what is platform-specific about it?
Well, the way we build our extensions forces us to do this. We cythonize 
large parts of our modules. This is done in separate build scripts. It 
is included as package_data, which is 'agnostic' of the platform. For 
this reason, we want to force the naming. In any case, no harm is done 
by adding the platform name, as distutils recognizes them. I guess we'll 
have to rename them manually after the setup() function, unless there's 
a way to 'force' setup() to 'think' it has compiled extensions in them?

Thanks,
Martin



From pje at telecommunity.com  Mon Oct 28 23:50:01 2013
From: pje at telecommunity.com (PJ Eby)
Date: Mon, 28 Oct 2013 18:50:01 -0400
Subject: [Distutils] Egg name computation
In-Reply-To: <526E2085.30100@intec.ugent.be>
References: <525D2FE9.6050904@intec.ugent.be>
 <CALeMXf53xTaCyVOvd6L_yw3VeVojasPvfhJMq46qkc1hVNPJ8g@mail.gmail.com>
 <526E2085.30100@intec.ugent.be>
Message-ID: <CALeMXf6jS_3jEmnjE2y8NHEGfDjTPy4wbZiCwkncMaj_e9EE5g@mail.gmail.com>

On Mon, Oct 28, 2013 at 4:29 AM, Martin Fiers
<martin.fiers at intec.ugent.be> wrote:
> I guess we'll have to
> rename them manually after the setup() function, unless there's a way to
> 'force' setup() to 'think' it has compiled extensions in them?

You could include a dummy extension that does nothing, I suppose.  Or
which controls the building of your actual extensions.  Setuptools has
long supported Pyrex and I think that Cython might also work, i.e.,
that you could just specify your cython modules as extensions in
setup.py to start with.

From marius at pov.lt  Tue Oct 29 10:45:27 2013
From: marius at pov.lt (Marius Gedminas)
Date: Tue, 29 Oct 2013 11:45:27 +0200
Subject: [Distutils] Removing dependency_links
In-Reply-To: <CAPYWazrBa4GEYQyQ7N+LXFDwEa+RN=zHGo_K=vVbsM48U4R+LA@mail.gmail.com>
References: <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
 <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>
 <CADiSq7cLNgwVmfgO94NQZkEMCMREkOr3oXfWTh6SsOThwQq6Uw@mail.gmail.com>
 <CAPYWazpcjRP-0xCE0=e0fJUCGNS5vTJpUt20Us95QEqDVMsN+Q@mail.gmail.com>
 <CADiSq7fW6uqWGDz-MGotgX0rL1c+uOyfAj+gMGFPwqkx3GjXUg@mail.gmail.com>
 <3FC26D14-AB77-49DB-9856-E55C75FE5F7B@stufft.io>
 <D6B8D266-925E-4706-89CF-B0F4FA541EDC@stufft.io>
 <CAOTb1wfba-Y2zvpwv1GQCs4tAwqmg=8f69G+xS9e1se+6XY_DA@mail.gmail.com>
 <CAOTb1wd7NpzmWgJz=jro5M+FNoaY072_b3akGyM=qSHvRj6DFQ@mail.gmail.com>
 <CAPYWazrBa4GEYQyQ7N+LXFDwEa+RN=zHGo_K=vVbsM48U4R+LA@mail.gmail.com>
Message-ID: <20131029094527.GA24147@fridge.pov.lt>

On Sun, Oct 27, 2013 at 03:52:04PM -0700, Marcus Smith wrote:
> > So I asked the person I know, and this is what he said, "Yes, we have
> > to use it!  It's the only way to allow a package to install other
> > packages that aren't on PyPI-- for instance, a custom fork of a
> > library."
> >
> > Is there another approach or work-around he can be using?  What is the
> > "right" way for him to do it?
> 
> e.g. you find a bug in SomeProject-1.4 on pypi (which is a dependency in
> your app install)
> fork it, fix it, and re-version it to SomeProject-1.4-myfork1, then package
> it, and place it in "/my/forks"
> and then "pip install --find-links=/my/forks/  my_app"

This is a useful feature and losing it would cause some measure of pain.
I tend to use this via find-links in buildout.cfg files.  Not just for
forks, but also for private packages not released to PyPI.

Specifying dependency_links in random packages' setup.py's is a
nuisance and I would rather it went away.  I always turn it off by
specifying allow-hosts = *.python.org in buildout.cfg

So, two different things here.  One is specified by the user who runs
pip (or some other installation tool).  The other is specified by the
package builder.  One is fine, the other is not.

Marius Gedminas
-- 
Never assume the reader has read the subject line.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131029/16ce89d3/attachment.sig>

From donald at stufft.io  Tue Oct 29 12:06:58 2013
From: donald at stufft.io (Donald Stufft)
Date: Tue, 29 Oct 2013 07:06:58 -0400
Subject: [Distutils] Removing dependency_links
In-Reply-To: <20131029094527.GA24147@fridge.pov.lt>
References: <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
 <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>
 <CADiSq7cLNgwVmfgO94NQZkEMCMREkOr3oXfWTh6SsOThwQq6Uw@mail.gmail.com>
 <CAPYWazpcjRP-0xCE0=e0fJUCGNS5vTJpUt20Us95QEqDVMsN+Q@mail.gmail.com>
 <CADiSq7fW6uqWGDz-MGotgX0rL1c+uOyfAj+gMGFPwqkx3GjXUg@mail.gmail.com>
 <3FC26D14-AB77-49DB-9856-E55C75FE5F7B@stufft.io>
 <D6B8D266-925E-4706-89CF-B0F4FA541EDC@stufft.io>
 <CAOTb1wfba-Y2zvpwv1GQCs4tAwqmg=8f69G+xS9e1se+6XY_DA@mail.gmail.com>
 <CAOTb1wd7NpzmWgJz=jro5M+FNoaY072_b3akGyM=qSHvRj6DFQ@mail.gmail.com>
 <CAPYWazrBa4GEYQyQ7N+LXFDwEa+RN=zHGo_K=vVbsM48U4R+LA@mail.gmail.com>
 <20131029094527.GA24147@fridge.pov.lt>
Message-ID: <9DEADE05-693D-4BE2-8B1D-2A38F8DCCDC1@stufft.io>


On Oct 29, 2013, at 5:45 AM, Marius Gedminas <marius at pov.lt> wrote:

> On Sun, Oct 27, 2013 at 03:52:04PM -0700, Marcus Smith wrote:
>>> So I asked the person I know, and this is what he said, "Yes, we have
>>> to use it!  It's the only way to allow a package to install other
>>> packages that aren't on PyPI-- for instance, a custom fork of a
>>> library."
>>> 
>>> Is there another approach or work-around he can be using?  What is the
>>> "right" way for him to do it?
>> 
>> e.g. you find a bug in SomeProject-1.4 on pypi (which is a dependency in
>> your app install)
>> fork it, fix it, and re-version it to SomeProject-1.4-myfork1, then package
>> it, and place it in "/my/forks"
>> and then "pip install --find-links=/my/forks/  my_app"
> 
> This is a useful feature and losing it would cause some measure of pain.
> I tend to use this via find-links in buildout.cfg files.  Not just for
> forks, but also for private packages not released to PyPI.
> 
> Specifying dependency_links in random packages' setup.py's is a
> nuisance and I would rather it went away.  I always turn it off by
> specifying allow-hosts = *.python.org in buildout.cfg
> 
> So, two different things here.  One is specified by the user who runs
> pip (or some other installation tool).  The other is specified by the
> package builder.  One is fine, the other is not.
> 
> Marius Gedminas
> -- 
> Never assume the reader has read the subject line.
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

?find-links aren?t going anywhere, it?s strictly about the package builder
case.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131029/83220d2d/attachment.sig>

From fred at fdrake.net  Tue Oct 29 12:02:53 2013
From: fred at fdrake.net (Fred Drake)
Date: Tue, 29 Oct 2013 07:02:53 -0400
Subject: [Distutils] Removing dependency_links
In-Reply-To: <20131029094527.GA24147@fridge.pov.lt>
References: <CADiSq7cNh98MXiA44H+1V38DkhcwzVdTVM6Y-AtT1Y7Z-GQhdQ@mail.gmail.com>
 <3AF6A53C-89A3-4B35-A51C-B252152A12FC@stufft.io>
 <CADiSq7cLNgwVmfgO94NQZkEMCMREkOr3oXfWTh6SsOThwQq6Uw@mail.gmail.com>
 <CAPYWazpcjRP-0xCE0=e0fJUCGNS5vTJpUt20Us95QEqDVMsN+Q@mail.gmail.com>
 <CADiSq7fW6uqWGDz-MGotgX0rL1c+uOyfAj+gMGFPwqkx3GjXUg@mail.gmail.com>
 <3FC26D14-AB77-49DB-9856-E55C75FE5F7B@stufft.io>
 <D6B8D266-925E-4706-89CF-B0F4FA541EDC@stufft.io>
 <CAOTb1wfba-Y2zvpwv1GQCs4tAwqmg=8f69G+xS9e1se+6XY_DA@mail.gmail.com>
 <CAOTb1wd7NpzmWgJz=jro5M+FNoaY072_b3akGyM=qSHvRj6DFQ@mail.gmail.com>
 <CAPYWazrBa4GEYQyQ7N+LXFDwEa+RN=zHGo_K=vVbsM48U4R+LA@mail.gmail.com>
 <20131029094527.GA24147@fridge.pov.lt>
Message-ID: <CAFT4OTELeS0A32nQ4KL7CzwcLs8zY1RCWPpVOs=CSzzanF8bew@mail.gmail.com>

On Tue, Oct 29, 2013 at 5:45 AM, Marius Gedminas <marius at pov.lt> wrote:
> Specifying dependency_links in random packages' setup.py's is a
> nuisance and I would rather it went away.  I always turn it off by
> specifying allow-hosts = *.python.org in buildout.cfg

Some packages indexed on PyPI have downloads elsewhere for various reasons.

You can specify

    use-dependency-links = false

to get buildout to ignore dependency_links without limiting the
collection of hosts it
is willing to download from (useful if you use any private repositories).


  -Fred

-- 
Fred L. Drake, Jr.    <fred at fdrake.net>
"A storm broke loose in my mind."  --Albert Einstein

From techtonik at gmail.com  Wed Oct 30 07:17:00 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 30 Oct 2013 09:17:00 +0300
Subject: [Distutils] PyPI pull request
In-Reply-To: <C44B74A3-4715-411B-ADDD-AE176ABF3F60@stufft.io>
References: <CAPkN8xLvAsArPaf3T-V+86-6EhwUwV4BFtfdJw=a9ce4DNyS8g@mail.gmail.com>
 <CAHrZfZD7hkkkkXiY1XU_QtfuKrQVcJn8SOb9bkbTq3kjUmyhNQ@mail.gmail.com>
 <CAPkN8xLhXnEmP875dxKJyPON-f=Hbz6JKQ=DmBcj75Za1fjNWQ@mail.gmail.com>
 <CAHrZfZAzSPN2vS95_ty2oj89vBfRYtC52t5NvQ75uCXabTGm8g@mail.gmail.com>
 <CAPkN8xLumGJTwo2pe=YpfbgqDBrYRQ7Xpt+UNC84JydOsPKB7g@mail.gmail.com>
 <5A8787FD-413B-45EA-87A5-817BA9BEE184@coderanger.net>
 <B6973CB3-A828-41F0-AD9A-5E44F3C88480@stufft.io>
 <CAPkN8xJGkP6r=M1fD5Doi2sjVQ7t_bXtpgRYakD5Rvx=Bn0adg@mail.gmail.com>
 <C44B74A3-4715-411B-ADDD-AE176ABF3F60@stufft.io>
Message-ID: <CAPkN8xJzu0bchnyGatTAGPiVegRPwaj1Doy6SnE4htp8SVGmeQ@mail.gmail.com>

On Mon, Oct 28, 2013 at 5:17 PM, Donald Stufft <donald at stufft.io> wrote:
>
> On Oct 28, 2013, at 10:10 AM, anatoly techtonik <techtonik at gmail.com> wrote:
>
>> So, is there any plans or Roadmaps for Warehouse? I mean it is no good
>> if nobody is going to approve pull request things until Warehouse is
>> ready.
>
> Currently Warehouse is focused on:
>
>  A) Getting API compatibility with the current implementation [1]
>  B) Getting a UI in place that can replace the existing UI [2]
>
> Overall as I think of more things we need for Warehouse to ?Become PyPI? I
> add it to the ?Becoming PyPI? milestone [3]
>
> [1] https://github.com/dstufft/warehouse/issues/58
> [2] https://github.com/dstufft/warehouse/issues/63
> [3] https://github.com/dstufft/warehouse/issues?milestone=1&state=open

So, how long will it take? Do you have a (two-)weekly schedule for
features/fixes you want in? It would be nice to get some
approximation.

Also it looks like "all-or-nothing" approach. One of the WSGI features
is to chain web apps together. Do you feel it is possible to run PyPI
+ nextgen side by side? Daytime, XMLRPC, JSON, RSS, upload API from
#58 - these seem like they can be decoupled from persistence layer and
reused for both backends on some point. Rewriting interfaces from
scratch doesn't guarantee that behavior will match.
--
anatoly t.

From holger at merlinux.eu  Wed Oct 30 11:21:58 2013
From: holger at merlinux.eu (holger krekel)
Date: Wed, 30 Oct 2013 10:21:58 +0000
Subject: [Distutils] devpi 1.2 releases: many improvements + py33 support
Message-ID: <20131030102158.GQ3973@merlinux.eu>

The devpi-{server,client}-1.2 releases bring a lot of refinements
and improvements for serving and interacting with your own pypi indexes:

- devpi-server serves release files from URLs containing a MD5 hash
  allowing safe serving of those files through nginx

- devpi-server's USER/INDEX urls can now be used directly with 
  pip/easy_install without the previously required (and still valid)
  ``+simple/`` suffix.

- ``devpi use --set-cfg`` reads and writes pip/easy_install 
  configuration files, making those installers pick up the in-use
  index seemlessly.  You can even do ``devpi use --always-set-cfg`` to 
  always set those config files when issuing a "devpi use" afterwards.

- ``devpi upload`` got many improvements:

  - versioned files (git and hg) will be exported to a clean directory prior to the build step
  - distutils/setup.py is now only used for building a package
  - documentation upload is tied to a version now
  - you can directly upload distribution files, including wheel files

- both devpi-server and devpi-client are python3.3 compatible now and 
  depend on a new devpi-common package which consolidates various 
  pypi-interaction aspects to avoid code duplication.  Also,
  global http proxy settings are honoured.

If you have an existing devpi-server-1.1 installation serving
your own packages you can install devpi-server>=1.2 and 
migrate the data with::

    devpi-server --upgrade-state [--serverdir your_server_dir]
    
This upgrades your server state in-place.  Please make sure you
backup your serverdir ahead of doing the upgrade (default location 
is ~/.devpi/server).

WARNING: ``devpi-server --gendeploy`` is deprecated and will be removed 
probably in favor of just generating example config files for 
nginx/supervisor/cron. Also ``devpi install`` is deprecated now 
in favor of using pip/easy_install directly (see also the --set-cfg
and --always-set-cfg options).

For more information please refer to the extensive documentation at:

    http://doc.devpi.net/

or check the CHANGELOG below.

have fun,

holger krekel


1.2
----------------------------

devpi-server:

- serve links to files on simple pages and index root as relative
  paths so that it works more nicely with proxy-pass server setups.
  fixes issue56.

- make devpi-server and devpi-common python3.3 compatible, addresses
  issue57

- use system http/s proxy settings from devpi-server.  fixes issue58.

- refactor locations to allow nginx serving static files more directly.
  Also updated nginx template accordingly.

- rework "--upgrade-state" to detect the state version of the server dir
  and create an appropriate virtualenv with a devpi-server install in order
  to export data, and then import that version.

- allow to use /user/index as indexserver url for pip/easy_install by
  redirecting non-json queries to /user/index/PROJ[/] to 
  /user/index/+simple/PROJ/

- fix submission of multi-value fields like "classifiers" or "platform"
  (previously they would be wrongly collapsed to become the last value of a list)

- fix normalization import/export issue: pypi names take precendence
  for defining the "real" name of a project.

- always store uploaded documentation with a version.  While 
  "devpi upload" will make sure to pass in the version, "setup.py upload_docs"
  will not pass in a version.  In the latter case, devpi-server assumes
  the documentation belongs to the highest yet registered release.
  This change requires exporting with devpi-1.1 and importing with devpi-1.2
  in order to properly store versioned docs internally.

- use types/url/metadata/validation functionality of new depdency devpi_common 

- internal cleanup using pytest-flakes

- make devpi-server use a proper UserAgent string

devpi-client:

- "devpi list" and "devpi remove" now accept a pip/setuptools style
  requirement like "pkg>=1.0" instead of the former for limited "pkg-1.0".

- make devpi-client fully work with python3.3 and fix test bugs 

- use system http/s proxy settings.  fixes issue58.

- add "devpi test -c tox.ini package" to use a particular (external)
  tox.ini for running tox with the unpackaged package.
  also add "--fallback-ini tox.ini" option which will only 
  be used if the download package has no tox.ini.

- new "devpi use --set-cfg" option to set pip/easy_install configuration
  files when changing indexes.  Also new "devpi use --always-set-cfg=yes"
  option if you want to imply "--set-cfg" on future "devpi use" invocations
  and "devpi use --always-st-cfg=no" to disable this implication.

- support git and hg for exporting all versioned files of a directory 
  before performing the build step when uploading

- improve how upload works: setup.py is only used for building docs
  and release files but not for the remote upload part.  This gets rid of a 
  number of hacks that were done trying to get the Python shipped "distutils" 
  to pick the proper devpi index and allows proper SSL verification on Python2.6
  onwards.

- upload: show response when uploading documentation failed

- upload: allow to specify archive files as positional arguments (both files and
  directories can be specified but the latter additionally require
  a --upload-dirs option)

- fix issue54: upload now works on wheel files as well.
  As pkginfo does not support wheels directly, we use the ``twine``
  project which extends pkginfo for now.

- only show highest version in "devpi list PROJECT" output, unless
  "--all" is specified.

- on upload release files: skip rather than guess packages which contain no metadata

- strike BeautifulSoup dependency and re-use vendored pip-link parser

- use types/url/metadata/validation functionality of new depdency devpi_common 

- internal cleanup wrt pytest-flakes discoveries

- remove "archive" dependency in favour of a small implementation in
  devpi_common

- make devpi-client use a proper UserAgent string

From techtonik at gmail.com  Wed Oct 30 21:07:27 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 30 Oct 2013 23:07:27 +0300
Subject: [Distutils] PyPI pull request #7
Message-ID: <CAPkN8xKWmSJUqH0QU36YbS4oHnv93B1nM8VhQ5+JXwDFR7HqMA@mail.gmail.com>

https://bitbucket.org/pypa/pypi/pull-request/7/fix-development-mode/diff

This allows to run PyPI on local machine without configuring web server,
and fixes CSS warnings from Chrome.
--
anatoly t.

From noah at coderanger.net  Wed Oct 30 21:11:32 2013
From: noah at coderanger.net (Noah Kantrowitz)
Date: Wed, 30 Oct 2013 13:11:32 -0700
Subject: [Distutils] PyPI pull request #7
In-Reply-To: <CAPkN8xKWmSJUqH0QU36YbS4oHnv93B1nM8VhQ5+JXwDFR7HqMA@mail.gmail.com>
References: <CAPkN8xKWmSJUqH0QU36YbS4oHnv93B1nM8VhQ5+JXwDFR7HqMA@mail.gmail.com>
Message-ID: <A95C5014-4DD9-4ED8-B77A-B28BB716EDAE@coderanger.net>

Please stop submitting pull requests. Development on the existing codebase is halted except for critical fixes or security issues. You are making extra work for people on this list and it will not be tolerated. Please consider this your final warning.

--Noah

On Oct 30, 2013, at 1:07 PM, anatoly techtonik <techtonik at gmail.com> wrote:

> https://bitbucket.org/pypa/pypi/pull-request/7/fix-development-mode/diff
> 
> This allows to run PyPI on local machine without configuring web server,
> and fixes CSS warnings from Chrome.
> --
> anatoly t.
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131030/99873cfb/attachment.sig>

From techtonik at gmail.com  Thu Oct 31 12:32:33 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Thu, 31 Oct 2013 14:32:33 +0300
Subject: [Distutils] PyPI pull request #7
In-Reply-To: <A95C5014-4DD9-4ED8-B77A-B28BB716EDAE@coderanger.net>
References: <CAPkN8xKWmSJUqH0QU36YbS4oHnv93B1nM8VhQ5+JXwDFR7HqMA@mail.gmail.com>
 <A95C5014-4DD9-4ED8-B77A-B28BB716EDAE@coderanger.net>
Message-ID: <CAPkN8xKUWoK6ckqwaZTNe1_+9koLafhp9mYvJu5khBHyBkKR0g@mail.gmail.com>

On Wed, Oct 30, 2013 at 11:11 PM, Noah Kantrowitz <noah at coderanger.net> wrote:
> Please stop submitting pull requests. Development on the existing codebase is halted except for critical fixes or security issues. You are making extra work for people on this list and it will not be tolerated. Please consider this your final warning.

I can't live as long as you are to see the new incantation of Python
website (by PyCon 2013) or PyPI. I am willing to help, and this stuff
you're saying is rather discouraging and like "no, go waste your time
somewhere else, we are not giving any code reviews for free". I
understand that my reputation precedes me, but can we keep this
strictly technical?

What I am trying to do is to send small, incremental fixes. They don't
affect security. I can commit it directly to avoid distracting
overloaded PyPI (bus factor 2) team, and you can blame me for breaking
things - ok, and ban if I break something - that's also ok. If learn
previous PyPI and new PyPI, I can tell people more about it, and you
can expect more pull requests - not from me, for new PyPI, once it is
ready.

And if I am going to submit any new features, like reST validation on
edit and Markdown support - the code will be more decoupled than
existing one to be almost directly reused for the new site.


Why I am skeptical that new site will replace old one soon? Just
because I don't believe in rewrites by one man army. When you develop
public resource, you need to rely on external feedback. You also need
some designer guy in a team. You also need a backlog for
collaboration. My ETA for new PyPI is no earlier than PyCon 2014 if
Donald and Richard will be working on it full time. So, instead of
all-or-nothing scenario I can try to find some help with incremental
approach.
--
anatoly t.

From donald at stufft.io  Thu Oct 31 12:38:50 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 31 Oct 2013 07:38:50 -0400
Subject: [Distutils] PyPI pull request #7
In-Reply-To: <CAPkN8xKUWoK6ckqwaZTNe1_+9koLafhp9mYvJu5khBHyBkKR0g@mail.gmail.com>
References: <CAPkN8xKWmSJUqH0QU36YbS4oHnv93B1nM8VhQ5+JXwDFR7HqMA@mail.gmail.com>
 <A95C5014-4DD9-4ED8-B77A-B28BB716EDAE@coderanger.net>
 <CAPkN8xKUWoK6ckqwaZTNe1_+9koLafhp9mYvJu5khBHyBkKR0g@mail.gmail.com>
Message-ID: <2ED3542A-9E15-4145-A382-F841158658BF@stufft.io>


On Oct 31, 2013, at 7:32 AM, anatoly techtonik <techtonik at gmail.com> wrote:

> Why I am skeptical that new site will replace old one soon? Just
> because I don't believe in rewrites by one man army. When you develop
> public resource, you need to rely on external feedback.

Warehouse is live at https://preview-pypi.python.org/ (No landing page yet)

> You also need
> some designer guy in a team.

Have one who?ll have time in a few weeks to go over everything and make it really great :)

> You also need a backlog for
> collaboration. My ETA for new PyPI is no earlier than PyCon 2014 if
> Donald and Richard will be working on it full time.

Possibly! I?m unsure of how long it will take, it?s primarily Richard and I but we?ve a
few domain experts in particular pieces who have offered to help out as well when
we?re ready for their pieces.

> So, instead of
> all-or-nothing scenario I can try to find some help with incremental
> approach.

Mostly the problem with improving the current base is every change is particularly
dangerous. The code base is large, it?s untested, and it?s not very nice. It?s extremely
easy to break things by accident with seemingly unrelated change. Richard and I
have both done this multiple times. So every PR we accept has a danger of breaking
things. This incurs a high cognitive overload for accepting a PR because it means we
have to spin up a copy of the site and manually go through and test any feature we
think *might* be affected which typically catches most things but not always. It?s a time
and labor intensive process which none of us enjoy and which we?ve decided not to
prioritize unless the pay offs are large.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131031/b830cdd1/attachment.sig>

From ronaldoussoren at mac.com  Thu Oct 31 14:38:26 2013
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 31 Oct 2013 14:38:26 +0100
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
 <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
Message-ID: <E0DC4524-DF0A-4D64-9D43-D1B1B9694185@mac.com>


On 21 Oct, 2013, at 20:52, Donald Stufft <donald at stufft.io> wrote:

> 
> On Oct 21, 2013, at 1:02 PM, Chris Barker <chris.barker at noaa.gov> wrote:
> 
>> On Fri, Oct 18, 2013 at 6:22 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>>> -- it would be very useful if folks could easily
>>>> get binary wheels for OS-X
>>> 
>>> We do plan to support it, but the pip devs uncovered a hole in the current
>>> wheel spec that means it generates the same filename on *nix systems for
>>> wheels that need to have different names for the download side of things to
>>> work properly
>> 
>> THanks -- but really? don't OS-X wheels get:
>> 
>> macosx_10_6_intel
>> 
>> or some such tacked on? Where does that go wrong?
> 
> Homebrew, Mac Ports, Fink. That would work OK if nobody ever installed things
> that the system didn't provide.

I don't understand this, what would break? Binary wheels that reference shared libraries that aren't included in the wheel (or a default system install) won't work, but that's also true on Windows.    

What makes OSX more fun[1] than Linux is including shared libraries in the binary archive, unless you are careful with linking you'll end up with binaries that can only be installed in 1 filesystem location (that is, don't work in virtualenvs).

Ronald

[1] for a fairly twisted definition of fun ;-)


From ncoghlan at gmail.com  Thu Oct 31 15:26:50 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 1 Nov 2013 00:26:50 +1000
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <E0DC4524-DF0A-4D64-9D43-D1B1B9694185@mac.com>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
 <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
 <E0DC4524-DF0A-4D64-9D43-D1B1B9694185@mac.com>
Message-ID: <CADiSq7cZLUchLYDXyXOze_E=HtCJrByOKMGj_QagDCVuye=JzA@mail.gmail.com>

On 31 October 2013 23:38, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>
> On 21 Oct, 2013, at 20:52, Donald Stufft <donald at stufft.io> wrote:
>
>>
>> On Oct 21, 2013, at 1:02 PM, Chris Barker <chris.barker at noaa.gov> wrote:
>>
>>> On Fri, Oct 18, 2013 at 6:22 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>>>> -- it would be very useful if folks could easily
>>>>> get binary wheels for OS-X
>>>>
>>>> We do plan to support it, but the pip devs uncovered a hole in the current
>>>> wheel spec that means it generates the same filename on *nix systems for
>>>> wheels that need to have different names for the download side of things to
>>>> work properly
>>>
>>> THanks -- but really? don't OS-X wheels get:
>>>
>>> macosx_10_6_intel
>>>
>>> or some such tacked on? Where does that go wrong?
>>
>> Homebrew, Mac Ports, Fink. That would work OK if nobody ever installed things
>> that the system didn't provide.
>
> I don't understand this, what would break? Binary wheels that reference shared libraries that aren't included in the wheel (or a default system install) won't work, but that's also true on Windows.

The main difference I see is that on Windows, the ABI for the
*CPython* shared library is significantly less likely to vary across
machines.

By contrast, it's relatively easy to change the ABI on *nix systems,
as it depends on what you pass as configure options.

Will a C extension built with Homebrew Python work with the Python Mac
OS X installer from python.org? Probably, but given how painful ABI
mismatches with shared libraries can be to debug, "probably" doesn't
cut it until someone has the chance to thoroughly review the potential
for problems.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ronaldoussoren at mac.com  Thu Oct 31 14:33:21 2013
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 31 Oct 2013 14:33:21 +0100
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
Message-ID: <2E27D4CF-D61E-40D8-90FC-4541BEF76D80@mac.com>


On 19 Oct, 2013, at 3:22, Nick Coghlan <ncoghlan at gmail.com> wrote:

> 
> On 19 Oct 2013 04:59, "Chris Barker" <chris.barker at noaa.gov> wrote:
> >
> > Someone on another list indicated that pip installing binary wheels
> > from PyPi will ONLY work for Windows.
> >
> > Is that the case? I think it's desperately needed for OS-X as well.
> >
> > Linux is so diverse that I can't imagine it being useful, but OS-X has
> > only so many versions, and the python.org OS-X binaries are very clear
> > in their requirements -- it would be very useful if folks could easily
> > get binary wheels for OS-X
> 
> We do plan to support it, but the pip devs uncovered a hole in the current wheel spec that means it generates the same filename on *nix systems for wheels that need to have different names for the download side of things to work properly - hence the current Windows-only restriction.

Is that hole documented somewhere? 

Ronald

From ronaldoussoren at mac.com  Thu Oct 31 16:06:04 2013
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 31 Oct 2013 16:06:04 +0100
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <CADiSq7cZLUchLYDXyXOze_E=HtCJrByOKMGj_QagDCVuye=JzA@mail.gmail.com>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
 <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
 <E0DC4524-DF0A-4D64-9D43-D1B1B9694185@mac.com>
 <CADiSq7cZLUchLYDXyXOze_E=HtCJrByOKMGj_QagDCVuye=JzA@mail.gmail.com>
Message-ID: <7BA23D4B-E652-4874-8C59-EDB2104B6F29@mac.com>


On 31 Oct, 2013, at 15:26, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 31 October 2013 23:38, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>> 
>> On 21 Oct, 2013, at 20:52, Donald Stufft <donald at stufft.io> wrote:
>> 
>>> 
>>> On Oct 21, 2013, at 1:02 PM, Chris Barker <chris.barker at noaa.gov> wrote:
>>> 
>>>> On Fri, Oct 18, 2013 at 6:22 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>>>>> -- it would be very useful if folks could easily
>>>>>> get binary wheels for OS-X
>>>>> 
>>>>> We do plan to support it, but the pip devs uncovered a hole in the current
>>>>> wheel spec that means it generates the same filename on *nix systems for
>>>>> wheels that need to have different names for the download side of things to
>>>>> work properly
>>>> 
>>>> THanks -- but really? don't OS-X wheels get:
>>>> 
>>>> macosx_10_6_intel
>>>> 
>>>> or some such tacked on? Where does that go wrong?
>>> 
>>> Homebrew, Mac Ports, Fink. That would work OK if nobody ever installed things
>>> that the system didn't provide.
>> 
>> I don't understand this, what would break? Binary wheels that reference shared libraries that aren't included in the wheel (or a default system install) won't work, but that's also true on Windows.
> 
> The main difference I see is that on Windows, the ABI for the
> *CPython* shared library is significantly less likely to vary across
> machines.
> 
> By contrast, it's relatively easy to change the ABI on *nix systems,
> as it depends on what you pass as configure options.
> 
> Will a C extension built with Homebrew Python work with the Python Mac
> OS X installer from python.org?

As long as they use the same "ABI Tag" from PEP 425 extensions build with
one should work with the other. 

> Probably, but given how painful ABI
> mismatches with shared libraries can be to debug, "probably" doesn't
> cut it until someone has the chance to thoroughly review the potential
> for problems.

I'd have to make time for a thorough review but be sure, but AFAIK this is 
only a little more complicated on OSX than on (say) Linux and the additonal 
complication is already captured by the platform tag mentioned by Chris.  

That is, the CPython ABI is affected by:

* The usual configure flags (--with-pydebug, --enable-unicode, ...)
   
  These issues also exists on Windows, and the "ABI Tag" from PEP 425 means
  that wheels for different sets of configure flags can be recognized from
  their filename.

* The OSX deployment target (the minimal OSX release you want to support)

  This is already recorded by the distutils platform tag 

* The processor architectures supported

  This is also recorded in the distutils platform tag.

* Possibly the compiler

  This is pretty unlikely, as I (and others) already load extensions build 
  with clang in CPython binaries built with GCC (and v.v.), but the low-level 
  compiler support library might affect the ABI when using a compiler other
  than GCC or clang. And I'd expect that two-level namespaces on OSX make that
  question moot except for code that's explicitly built to disable that feature.

  And this does definitely affect CPython on Windows unless you use the limited ABI.
  The only reason this doesn't cause problems with binary installers on PyPI
  right now is that most users use the binary installers from www.python.org and hence
  are all using the same compiler version.

The deployment target can affect the unix level API a little, IIRC around 10.4 
or 10.5 the APIs on the POSIX level were adjusted a little to more fully comply 
with the UNIX specification and because of this the affected named resolve to 
different symbols based on the deployment target (more or less simular to symbol 
versioning in glibc). AFAIK that shouldn't affect the CPython ABI, but that would 
need to be carefully checked to be sure and I don't have time to do so right now.

To provide more detail on the disutils platform tag, on my machine:

>>> import distutils.util
>>> distutils.util.get_platform()
'macosx-10.8-intel'

This means the deployment target is OSX 10.8, with suppport for the 'intel' set 
of archictures (i386 and x86_64). This records the two variables that are most likely
to cause problems.

I'd expected more problems with binary eggs that are dynamicly linked to libraries
that are provided in the wheel because OSX's linker add an absolute path to the library
to be loaded unless you're careful.

Ronald


From dominique.orban at gmail.com  Thu Oct 31 17:08:09 2013
From: dominique.orban at gmail.com (Dominique Orban)
Date: Thu, 31 Oct 2013 12:08:09 -0400
Subject: [Distutils] AttributeError: 'tuple' object has no attribute
 'split'
In-Reply-To: <etPan.526ab328.48945699.8d60@Aragorn>
References: <etPan.52697d87.1d0a04e.8d60@Aragorn>
 <CAHVvXxRBC_TtcZwdA2gd=-U5AJvGBWtm8FNAzoTRUdLixic1ng@mail.gmail.com>
 <etPan.526a85c8.7aed0d9f.8d60@Aragorn>
 <CAHVvXxTpMDMhU2Q96K6e++2JGqCLp1RWAFKBSr24P29-y+1qSA@mail.gmail.com>
 <etPan.526ab328.48945699.8d60@Aragorn>
Message-ID: <etPan.527280a5.4775af33.8d60@Aragorn>

On 25 October, 2013 at 2:06:34 PM, Dominique Orban (dominique.orban at gmail.com) wrote:
>
>
>
>On 25 October, 2013 at 1:56:26 PM, Oscar Benjamin (oscar.j.benjamin at gmail.com) wrote:
>>
>>On Oct 25, 2013 3:52 PM, "Dominique Orban"
>>wrote:
>>>
>>>
>>>
>>> On 25 October, 2013 at 9:31:16 AM, Oscar Benjamin (
>>oscar.j.benjamin at gmail.com) wrote:
>>> >
>>> >On 24 October 2013 21:04, Dominique Orban wrote:
>>> >>
>>> >> I hope this is the right place to ask for help. I'm not finding much
>>comfort in the PyPi documentation or in Google searches. I uploaded my
>>package `pykrylov` with `python setup.py sdist upload`. Installing it
>>locally with `python setup.py` install works fine but `pip install
>>pykrylov` breaks with the messages below. I since removed it from PyPI but
>>I get the same error message if I try installing from the git repository.
>>I'm hoping someone can put me on track as I've no idea what's wrong. You
>>can see my setup.py here:
>>> >>
>>> >>
>>https://github.com/dpo/pykrylov/blob/ea553cdb287f6e685406ceadcb297fd6704af52d/setup.py
>>> >>
>>> >> I'm using Python 2.7.5 on OSX installed with Homebrew and pip 1.4.1.
>>Attempts to upgrade setuptools or pip result in another error message
>>(AttributeError: 'str' object has no attribute 'rollback')...
>>> >
>>> >Can you install a more recent setuptools by downloading it and running
>>> >the setup.py yourself?
>>> >
>>https://pypi.python.org/packages/source/s/setuptools/setuptools-1.1.6.tar.gz
>>>
>>> Thanks for the suggestion. I'm still getting the same error with
>>setuptools 1.1.6. I also tried "upgrading" Numpy (since I'm using Numpy
>>distutils) by installing from their git repository, and I'm still getting
>>the same error.
>>>
>>> Is anything obviously wrong with the setup.py?
>>
>>I don't know but I'm not totally clear what you mean. Previously you
>>described multiple problems: with pip, setuptools and pykrylov. Have you
>>successfully installed setuptools now?
>>
>>If the "same error" is with pykrylov's setup.py have you tried debugging
>>it? E.g. 'python -m pdb setup.py install'
>
>"python setup.py install" works fine. It's the installation with pip that returns the error message I mentioned. I was wondering if something in setup.py didn't agree with pip/setuptools.
>
>Yes I installed setuptools 1.1.6. But "pip install -e git://github.com/dpo/pykrylov.git at ea553cd#egg=pykrylov" still returns "AttributeError: 'tuple' object has no attribute 'split'".
>
>I hope I'm making sense.

Anybody can provide any help with this?

Many thanks!


--?
Dominique

From dholth at gmail.com  Thu Oct 31 17:49:28 2013
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 31 Oct 2013 12:49:28 -0400
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <CADiSq7cZLUchLYDXyXOze_E=HtCJrByOKMGj_QagDCVuye=JzA@mail.gmail.com>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
 <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
 <E0DC4524-DF0A-4D64-9D43-D1B1B9694185@mac.com>
 <CADiSq7cZLUchLYDXyXOze_E=HtCJrByOKMGj_QagDCVuye=JzA@mail.gmail.com>
Message-ID: <CAG8k2+6oD77FgPr6J9FUQb3tujY5UCJm5soyaSdE5Rtx=gHE5w@mail.gmail.com>

On Thu, Oct 31, 2013 at 10:26 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 31 October 2013 23:38, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>>
>> On 21 Oct, 2013, at 20:52, Donald Stufft <donald at stufft.io> wrote:
>>
>>>
>>> On Oct 21, 2013, at 1:02 PM, Chris Barker <chris.barker at noaa.gov> wrote:
>>>
>>>> On Fri, Oct 18, 2013 at 6:22 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>>>>> -- it would be very useful if folks could easily
>>>>>> get binary wheels for OS-X
>>>>>
>>>>> We do plan to support it, but the pip devs uncovered a hole in the current
>>>>> wheel spec that means it generates the same filename on *nix systems for
>>>>> wheels that need to have different names for the download side of things to
>>>>> work properly
>>>>
>>>> THanks -- but really? don't OS-X wheels get:
>>>>
>>>> macosx_10_6_intel
>>>>
>>>> or some such tacked on? Where does that go wrong?
>>>
>>> Homebrew, Mac Ports, Fink. That would work OK if nobody ever installed things
>>> that the system didn't provide.
>>
>> I don't understand this, what would break? Binary wheels that reference shared libraries that aren't included in the wheel (or a default system install) won't work, but that's also true on Windows.
>
> The main difference I see is that on Windows, the ABI for the
> *CPython* shared library is significantly less likely to vary across
> machines.
>
> By contrast, it's relatively easy to change the ABI on *nix systems,
> as it depends on what you pass as configure options.

I'm sure you could build your own broken Windows Python, but who
bothers? IMO it pretty much boils down to the fact that on Windows you
are probably using the python.org version of Python and not linking
with random shared libraries from C:\Program Files\, but on Linux you
are probably using one of a few dozen distro x distro-release Pythons
AND your extension probably dynamically links to some other things
your specific distro provides AND maybe you are depending on some
versioned symbols in glibc oh the horror.

On OS X I would hope you are uploading only wheels built with
python.org-Python but maybe you aren't, everyone has their
preferences.

> Will a C extension built with Homebrew Python work with the Python Mac
> OS X installer from python.org? Probably, but given how painful ABI
> mismatches with shared libraries can be to debug, "probably" doesn't
> cut it until someone has the chance to thoroughly review the potential
> for problems.

From chris.jerdonek at gmail.com  Thu Oct 31 18:48:50 2013
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Thu, 31 Oct 2013 10:48:50 -0700
Subject: [Distutils] AttributeError: 'tuple' object has no attribute
	'split'
In-Reply-To: <etPan.527280a5.4775af33.8d60@Aragorn>
References: <etPan.52697d87.1d0a04e.8d60@Aragorn>
 <CAHVvXxRBC_TtcZwdA2gd=-U5AJvGBWtm8FNAzoTRUdLixic1ng@mail.gmail.com>
 <etPan.526a85c8.7aed0d9f.8d60@Aragorn>
 <CAHVvXxTpMDMhU2Q96K6e++2JGqCLp1RWAFKBSr24P29-y+1qSA@mail.gmail.com>
 <etPan.526ab328.48945699.8d60@Aragorn>
 <etPan.527280a5.4775af33.8d60@Aragorn>
Message-ID: <CAOTb1wf0ECDQ68y9VgfWkW2tG72bD1_y7mSSssdbNvWDJ7PKOw@mail.gmail.com>

Why don't you study what's happening in the offending lines of code
and see where things are going wrong?  For example, in this file:

/Users/dpo/.virtualenvs/test/lib/python2.7/site-packages/setuptools/command/egg_info.py

see what's showing up as a tuple rather than a string.  You can add
debug statements if necessary.

--Chris


On Thu, Oct 31, 2013 at 9:08 AM, Dominique Orban
<dominique.orban at gmail.com> wrote:
> On 25 October, 2013 at 2:06:34 PM, Dominique Orban (dominique.orban at gmail.com) wrote:
>>
>>
>>
>>On 25 October, 2013 at 1:56:26 PM, Oscar Benjamin (oscar.j.benjamin at gmail.com) wrote:
>>>
>>>On Oct 25, 2013 3:52 PM, "Dominique Orban"
>>>wrote:
>>>>
>>>>
>>>>
>>>> On 25 October, 2013 at 9:31:16 AM, Oscar Benjamin (
>>>oscar.j.benjamin at gmail.com) wrote:
>>>> >
>>>> >On 24 October 2013 21:04, Dominique Orban wrote:
>>>> >>
>>>> >> I hope this is the right place to ask for help. I'm not finding much
>>>comfort in the PyPi documentation or in Google searches. I uploaded my
>>>package `pykrylov` with `python setup.py sdist upload`. Installing it
>>>locally with `python setup.py` install works fine but `pip install
>>>pykrylov` breaks with the messages below. I since removed it from PyPI but
>>>I get the same error message if I try installing from the git repository.
>>>I'm hoping someone can put me on track as I've no idea what's wrong. You
>>>can see my setup.py here:
>>>> >>
>>>> >>
>>>https://github.com/dpo/pykrylov/blob/ea553cdb287f6e685406ceadcb297fd6704af52d/setup.py
>>>> >>
>>>> >> I'm using Python 2.7.5 on OSX installed with Homebrew and pip 1.4.1.
>>>Attempts to upgrade setuptools or pip result in another error message
>>>(AttributeError: 'str' object has no attribute 'rollback')...
>>>> >
>>>> >Can you install a more recent setuptools by downloading it and running
>>>> >the setup.py yourself?
>>>> >
>>>https://pypi.python.org/packages/source/s/setuptools/setuptools-1.1.6.tar.gz
>>>>
>>>> Thanks for the suggestion. I'm still getting the same error with
>>>setuptools 1.1.6. I also tried "upgrading" Numpy (since I'm using Numpy
>>>distutils) by installing from their git repository, and I'm still getting
>>>the same error.
>>>>
>>>> Is anything obviously wrong with the setup.py?
>>>
>>>I don't know but I'm not totally clear what you mean. Previously you
>>>described multiple problems: with pip, setuptools and pykrylov. Have you
>>>successfully installed setuptools now?
>>>
>>>If the "same error" is with pykrylov's setup.py have you tried debugging
>>>it? E.g. 'python -m pdb setup.py install'
>>
>>"python setup.py install" works fine. It's the installation with pip that returns the error message I mentioned. I was wondering if something in setup.py didn't agree with pip/setuptools.
>>
>>Yes I installed setuptools 1.1.6. But "pip install -e git://github.com/dpo/pykrylov.git at ea553cd#egg=pykrylov" still returns "AttributeError: 'tuple' object has no attribute 'split'".
>>
>>I hope I'm making sense.
>
> Anybody can provide any help with this?
>
> Many thanks!
>
>
> --
> Dominique
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

From ronaldoussoren at mac.com  Thu Oct 31 18:15:36 2013
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 31 Oct 2013 18:15:36 +0100
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <CAG8k2+6oD77FgPr6J9FUQb3tujY5UCJm5soyaSdE5Rtx=gHE5w@mail.gmail.com>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
 <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
 <E0DC4524-DF0A-4D64-9D43-D1B1B9694185@mac.com>
 <CADiSq7cZLUchLYDXyXOze_E=HtCJrByOKMGj_QagDCVuye=JzA@mail.gmail.com>
 <CAG8k2+6oD77FgPr6J9FUQb3tujY5UCJm5soyaSdE5Rtx=gHE5w@mail.gmail.com>
Message-ID: <9EF006C3-5FB4-463D-87F2-D8E1BFBC8CDB@mac.com>


On 31 Oct, 2013, at 17:49, Daniel Holth <dholth at gmail.com> wrote:

> On Thu, Oct 31, 2013 at 10:26 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On 31 October 2013 23:38, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>>> 
>>> On 21 Oct, 2013, at 20:52, Donald Stufft <donald at stufft.io> wrote:
>>> 
>>>> 
>>>> On Oct 21, 2013, at 1:02 PM, Chris Barker <chris.barker at noaa.gov> wrote:
>>>> 
>>>>> On Fri, Oct 18, 2013 at 6:22 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>>>>>> -- it would be very useful if folks could easily
>>>>>>> get binary wheels for OS-X
>>>>>> 
>>>>>> We do plan to support it, but the pip devs uncovered a hole in the current
>>>>>> wheel spec that means it generates the same filename on *nix systems for
>>>>>> wheels that need to have different names for the download side of things to
>>>>>> work properly
>>>>> 
>>>>> THanks -- but really? don't OS-X wheels get:
>>>>> 
>>>>> macosx_10_6_intel
>>>>> 
>>>>> or some such tacked on? Where does that go wrong?
>>>> 
>>>> Homebrew, Mac Ports, Fink. That would work OK if nobody ever installed things
>>>> that the system didn't provide.
>>> 
>>> I don't understand this, what would break? Binary wheels that reference shared libraries that aren't included in the wheel (or a default system install) won't work, but that's also true on Windows.
>> 
>> The main difference I see is that on Windows, the ABI for the
>> *CPython* shared library is significantly less likely to vary across
>> machines.
>> 
>> By contrast, it's relatively easy to change the ABI on *nix systems,
>> as it depends on what you pass as configure options.
> 
> On OS X I would hope you are uploading only wheels built with
> python.org-Python but maybe you aren't, everyone has their
> preferences.

And that shouldn't be a problem, the various builds of Python on OSX should be compatible if the have the same platform tag in the wheel filename. Well, barring idiocy like installing a completely different libc, that might cause problems because (at least for 2.7) some libc types leak into the CPython ABI.

But anyway, I'd love to use binary wheels in the future to distribute prebuilt binaries for some of my projects and that's why I reacted on a message that mentioned that binary installs aren't supported yet. If I don't understand what the issue is I can't try to help finding I solution :-)

Is it "just" a matter of researching if the various build options on OSX really lead to binaries with the same ABI, or is more work needed?   

Ronald

> 
>> Will a C extension built with Homebrew Python work with the Python Mac
>> OS X installer from python.org? Probably, but given how painful ABI
>> mismatches with shared libraries can be to debug, "probably" doesn't
>> cut it until someone has the chance to thoroughly review the potential
>> for problems.


From donald at stufft.io  Thu Oct 31 19:24:25 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 31 Oct 2013 14:24:25 -0400
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <9EF006C3-5FB4-463D-87F2-D8E1BFBC8CDB@mac.com>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
 <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
 <E0DC4524-DF0A-4D64-9D43-D1B1B9694185@mac.com>
 <CADiSq7cZLUchLYDXyXOze_E=HtCJrByOKMGj_QagDCVuye=JzA@mail.gmail.com>
 <CAG8k2+6oD77FgPr6J9FUQb3tujY5UCJm5soyaSdE5Rtx=gHE5w@mail.gmail.com>
 <9EF006C3-5FB4-463D-87F2-D8E1BFBC8CDB@mac.com>
Message-ID: <049ABB88-78A4-43A9-B3D3-54110CE3D53B@stufft.io>


On Oct 31, 2013, at 1:15 PM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:

> Is it "just" a matter of researching if the various build options on OSX really lead to binaries with the same ABI, or is more work needed?   

Basically it?s this:

I was told a wheel built on Ubuntu probably won?t work on Linux, so I shut off Linux Wheels, at the same time I asked about Windows and OSX wheels, the answers I got from people were they would generally just work on Windows, and nobody gave me a straight answer on OSX.

If you build a Wheel with Homebrew Python will it work on the official OSX installers? What if I have a library installed from Homebrew? Essentially trying to figure out how likely it is that with the existing tags a wheel is going to work if we select based on them.

Once we know what, if any, problems exist for the various platforms then we can come up with fixes for them. This just hasn?t been high priority for me because of PEP453 work. 

To be honest the same problems likely exists on Windows, it?s just less likely and the benefits of prebuilt binaries greater.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131031/06ca633c/attachment.sig>

From chris.barker at noaa.gov  Thu Oct 31 16:41:42 2013
From: chris.barker at noaa.gov (Chris Barker)
Date: Thu, 31 Oct 2013 08:41:42 -0700
Subject: [Distutils] Egg name computation
In-Reply-To: <CALeMXf6jS_3jEmnjE2y8NHEGfDjTPy4wbZiCwkncMaj_e9EE5g@mail.gmail.com>
References: <525D2FE9.6050904@intec.ugent.be>
 <CALeMXf53xTaCyVOvd6L_yw3VeVojasPvfhJMq46qkc1hVNPJ8g@mail.gmail.com>
 <526E2085.30100@intec.ugent.be>
 <CALeMXf6jS_3jEmnjE2y8NHEGfDjTPy4wbZiCwkncMaj_e9EE5g@mail.gmail.com>
Message-ID: <CALGmxELcWNPCw+Qf0vDKqx5GLehmL_CBJk4Q7UJdAYDMUx-PqQ@mail.gmail.com>

On Mon, Oct 28, 2013 at 3:50 PM, PJ Eby <pje at telecommunity.com> wrote:

> You could include a dummy extension that does nothing, I suppose.  Or
> which controls the building of your actual extensions.  Setuptools has
> long supported Pyrex and I think that Cython might also work, i.e.,
> that you could just specify your cython modules as extensions in
> setup.py to start with.
>

Indeed -- recent versions of setuptools do support Cython. You also may
want to use Cython's "cythonize" on your Extension objects to make it a bit
"smarter".

-Chris




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131031/99fc3ee5/attachment.html>

From vinay_sajip at yahoo.co.uk  Thu Oct 31 20:55:14 2013
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 31 Oct 2013 19:55:14 +0000 (UTC)
Subject: [Distutils] distlib 0.1.3 released on PyPI
Message-ID: <loom.20131031T205108-720@post.gmane.org>

I've just released version 0.1.4 of distlib on PyPI [1]. For newcomers, 
distlib is a library of packaging functionality which is intended to be 
usable as the basis for third-party packaging tools.

The changes in this release are as follows:

scripts.py:

    Updated the logic for finding the distlib package using a relative,
    rather than absolute method. This fixes a problem for pip, where
    distlib is kept in the pip.vendor.distlib package.

_backport/sysconfig.py:

    The analogous change to that made for scripts.py, described above.

Please try it out, and if you find any problems or have any suggestions for
improvements, please give some feedback using the issue tracker! [2]

Regards,

Vinay Sajip

[1] https://pypi.python.org/pypi/distlib/0.1.4
[2] https://bitbucket.org/pypa/distlib/issues/new



From vinay_sajip at yahoo.co.uk  Thu Oct 31 21:11:45 2013
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 31 Oct 2013 20:11:45 +0000 (UTC)
Subject: [Distutils] distlib 0.1.4 released on PyPI
References: <loom.20131031T205108-720@post.gmane.org>
Message-ID: <loom.20131031T211046-184@post.gmane.org>

Vinay Sajip <vinay_sajip <at> yahoo.co.uk> writes:

> 
> I've just released version 0.1.4 of distlib on PyPI [1]. For newcomers, 
> distlib is a library of packaging functionality which is intended to be 
> usable as the basis for third-party packaging tools.

Aargh, sorry about the typo in the subject - that should have been 0.1.4.

Regards,

Vinay Sajip


From tseaver at palladion.com  Thu Oct 31 21:49:48 2013
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 31 Oct 2013 16:49:48 -0400
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <049ABB88-78A4-43A9-B3D3-54110CE3D53B@stufft.io>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
 <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
 <E0DC4524-DF0A-4D64-9D43-D1B1B9694185@mac.com>
 <CADiSq7cZLUchLYDXyXOze_E=HtCJrByOKMGj_QagDCVuye=JzA@mail.gmail.com>
 <CAG8k2+6oD77FgPr6J9FUQb3tujY5UCJm5soyaSdE5Rtx=gHE5w@mail.gmail.com>
 <9EF006C3-5FB4-463D-87F2-D8E1BFBC8CDB@mac.com>
 <049ABB88-78A4-43A9-B3D3-54110CE3D53B@stufft.io>
Message-ID: <l4ufp3$gbs$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/31/2013 02:24 PM, Donald Stufft wrote:
> To be honest the same problems likely exists on Windows, it?s just
> less likely and the benefits of prebuilt binaries greater.

For all platforms *except* Windows, wheels are essentially caches --
there is no real reason to distribute them via PyPI at all, because OSx
and Linux develpoers will have tools to build them from sdists.

Pip *does* need to allow installing them on those platforms, but probably
only via explicit paths / URLS (rather than finding them on PyPI).


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlJywmwACgkQ+gerLs4ltQ5P/ACfXcMJj4dmnlNJEccZ8gxi8FLR
GrQAoLxxgeVnKRvuoscR6GuabwGxfsnF
=gZW1
-----END PGP SIGNATURE-----


From donald at stufft.io  Thu Oct 31 21:55:45 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 31 Oct 2013 16:55:45 -0400
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <l4ufp3$gbs$1@ger.gmane.org>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
 <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
 <E0DC4524-DF0A-4D64-9D43-D1B1B9694185@mac.com>
 <CADiSq7cZLUchLYDXyXOze_E=HtCJrByOKMGj_QagDCVuye=JzA@mail.gmail.com>
 <CAG8k2+6oD77FgPr6J9FUQb3tujY5UCJm5soyaSdE5Rtx=gHE5w@mail.gmail.com>
 <9EF006C3-5FB4-463D-87F2-D8E1BFBC8CDB@mac.com>
 <049ABB88-78A4-43A9-B3D3-54110CE3D53B@stufft.io> <l4ufp3$gbs$1@ger.gmane.org>
Message-ID: <1123DCC9-589D-405A-B567-BD39654866E2@stufft.io>


On Oct 31, 2013, at 4:49 PM, Tres Seaver <tseaver at palladion.com> wrote:

> Pip *does* need to allow installing them on those platforms, but probably
> only via explicit paths / URLS (rather than finding them on PyPI).

You can install them just fine on any platform, the only restrictions are PyPI won?t
let you upload them, and pip won?t try to install them from pypi.python.org.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131031/847f518f/attachment-0001.sig>

From ncoghlan at gmail.com  Thu Oct 31 22:34:19 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 1 Nov 2013 07:34:19 +1000
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <l4ufp3$gbs$1@ger.gmane.org>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
 <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
 <E0DC4524-DF0A-4D64-9D43-D1B1B9694185@mac.com>
 <CADiSq7cZLUchLYDXyXOze_E=HtCJrByOKMGj_QagDCVuye=JzA@mail.gmail.com>
 <CAG8k2+6oD77FgPr6J9FUQb3tujY5UCJm5soyaSdE5Rtx=gHE5w@mail.gmail.com>
 <9EF006C3-5FB4-463D-87F2-D8E1BFBC8CDB@mac.com>
 <049ABB88-78A4-43A9-B3D3-54110CE3D53B@stufft.io>
 <l4ufp3$gbs$1@ger.gmane.org>
Message-ID: <CADiSq7enQUVchS9e9DEgNbBWPG4CjwEABQdprU6e+c-2f0yunQ@mail.gmail.com>

On 1 Nov 2013 06:50, "Tres Seaver" <tseaver at palladion.com> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 10/31/2013 02:24 PM, Donald Stufft wrote:
> > To be honest the same problems likely exists on Windows, it?s just
> > less likely and the benefits of prebuilt binaries greater.
>
> For all platforms *except* Windows, wheels are essentially caches --
> there is no real reason to distribute them via PyPI at all, because OSx
> and Linux develpoers will have tools to build them from sdists.

They're caches that can speed things up a *lot*, though, and for new users
"can build C extensions" is a higher bar than "can run Python and pip".

Given PEP 453, it's probably worth allowing wheels on Mac OS X in pip 1.5,
then we can tackle the thornier general *nix problem in the pip 1.6 time
frame (which should also improve external dependency handling on Windows
and Mac OS X as well).

Cheers,
Nick.

>
> Pip *does* need to allow installing them on those platforms, but probably
> only via explicit paths / URLS (rather than finding them on PyPI).
>
>
> Tres.
> - --
> ===================================================================
> Tres Seaver          +1 540-429-0999          tseaver at palladion.com
> Palladion Software   "Excellence by Design"    http://palladion.com
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iEYEARECAAYFAlJywmwACgkQ+gerLs4ltQ5P/ACfXcMJj4dmnlNJEccZ8gxi8FLR
> GrQAoLxxgeVnKRvuoscR6GuabwGxfsnF
> =gZW1
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131101/26f7fdea/attachment.html>

From eric at trueblade.com  Thu Oct 31 22:52:44 2013
From: eric at trueblade.com (Eric V. Smith)
Date: Thu, 31 Oct 2013 17:52:44 -0400
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <CADiSq7enQUVchS9e9DEgNbBWPG4CjwEABQdprU6e+c-2f0yunQ@mail.gmail.com>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
 <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
 <E0DC4524-DF0A-4D64-9D43-D1B1B9694185@mac.com>
 <CADiSq7cZLUchLYDXyXOze_E=HtCJrByOKMGj_QagDCVuye=JzA@mail.gmail.com>
 <CAG8k2+6oD77FgPr6J9FUQb3tujY5UCJm5soyaSdE5Rtx=gHE5w@mail.gmail.com>
 <9EF006C3-5FB4-463D-87F2-D8E1BFBC8CDB@mac.com>
 <049ABB88-78A4-43A9-B3D3-54110CE3D53B@stufft.io> <l4ufp3$gbs$1@ger.gmane.org>
 <CADiSq7enQUVchS9e9DEgNbBWPG4CjwEABQdprU6e+c-2f0yunQ@mail.gmail.com>
Message-ID: <5272D12C.4040907@trueblade.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/31/2013 5:34 PM, Nick Coghlan wrote:
> 
> On 1 Nov 2013 06:50, "Tres Seaver" <tseaver at palladion.com 
> <mailto:tseaver at palladion.com>> wrote:
>> 
> On 10/31/2013 02:24 PM, Donald Stufft wrote:
>> To be honest the same problems likely exists on Windows, it?s
>> just less likely and the benefits of prebuilt binaries greater.
> 
> For all platforms *except* Windows, wheels are essentially caches
> -- there is no real reason to distribute them via PyPI at all,
> because OSx and Linux develpoers will have tools to build them from
> sdists.
> 
>> They're caches that can speed things up a *lot*, though, and for
>> new users "can build C extensions" is a higher bar than "can run
>> Python and pip".

I don't think it's true that they're caches on all Linux boxes. I have
many production machines that do not have compilers on them, and do
not have the development headers, etc. installed. They're no better
off than a Windows machine!

Sure, I could install all of this, but:
- - I'd rather not increase the attack surface by installing lots
  of software
- - Some of these are small VM instances, and I don't have room to
  install a full build environment

Where I can, right now I typically install the tools I need to
install, do the installation from source, and then remove the tools.
It's a pain. I'd much rather have binary Linux wheels. I'm more than
happy to build them myself on a dedicated build machine.

Eric.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (Cygwin)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJSctEsAAoJENxauZFcKtNxSYYIAJW3BArMWX+Z669AYe/lpoj9
WMggmdfr6u6BXrOVdujHJpXG43U9UhcqJTYLOitNHyO9XKVq/ODDfbV7SN9LTlHI
atkEf87xxjuaBI5nBiS0q/ozKMmU4AR82aHYQ2wcxlLry/CSt9zgBBM/vt4I0so2
UXeUa5CgkXGGsG5nYvk9zOAQmMknUSESZrE/WNsgjImk9oBBySs9Tvz+hUF407Y2
sRFuWVZCoDjFwWg8/e98YI6sxuqx7fFhdzn5uoXq9wQ27xg9FjKQlbAWPYcBVK1M
ZAQj0SgSMY4paKQ9QsOdQchieJjRZ21C7ue8WGaca/IeyY280RTA0ztuRVMFz58=
=nmb3
-----END PGP SIGNATURE-----

From donald at stufft.io  Thu Oct 31 23:05:05 2013
From: donald at stufft.io (Donald Stufft)
Date: Thu, 31 Oct 2013 18:05:05 -0400
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <5272D12C.4040907@trueblade.com>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
 <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
 <E0DC4524-DF0A-4D64-9D43-D1B1B9694185@mac.com>
 <CADiSq7cZLUchLYDXyXOze_E=HtCJrByOKMGj_QagDCVuye=JzA@mail.gmail.com>
 <CAG8k2+6oD77FgPr6J9FUQb3tujY5UCJm5soyaSdE5Rtx=gHE5w@mail.gmail.com>
 <9EF006C3-5FB4-463D-87F2-D8E1BFBC8CDB@mac.com>
 <049ABB88-78A4-43A9-B3D3-54110CE3D53B@stufft.io> <l4ufp3$gbs$1@ger.gmane.org>
 <CADiSq7enQUVchS9e9DEgNbBWPG4CjwEABQdprU6e+c-2f0yunQ@mail.gmail.com>
 <5272D12C.4040907@trueblade.com>
Message-ID: <3F93CCA5-D5D6-473F-86DA-5E0879AD0C7E@stufft.io>


On Oct 31, 2013, at 5:52 PM, Eric V. Smith <eric at trueblade.com> wrote:

>  I'm more than
> happy to build them myself on a dedicated build machine.

This works! You just can?t upload/install from PyPI.

The restriction is *only* centered around PyPI.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131031/c7e8bc63/attachment.sig>

From eric at trueblade.com  Thu Oct 31 23:11:19 2013
From: eric at trueblade.com (Eric V. Smith)
Date: Thu, 31 Oct 2013 18:11:19 -0400
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <3F93CCA5-D5D6-473F-86DA-5E0879AD0C7E@stufft.io>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
 <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
 <E0DC4524-DF0A-4D64-9D43-D1B1B9694185@mac.com>
 <CADiSq7cZLUchLYDXyXOze_E=HtCJrByOKMGj_QagDCVuye=JzA@mail.gmail.com>
 <CAG8k2+6oD77FgPr6J9FUQb3tujY5UCJm5soyaSdE5Rtx=gHE5w@mail.gmail.com>
 <9EF006C3-5FB4-463D-87F2-D8E1BFBC8CDB@mac.com>
 <049ABB88-78A4-43A9-B3D3-54110CE3D53B@stufft.io> <l4ufp3$gbs$1@ger.gmane.org>
 <CADiSq7enQUVchS9e9DEgNbBWPG4CjwEABQdprU6e+c-2f0yunQ@mail.gmail.com>
 <5272D12C.4040907@trueblade.com>
 <3F93CCA5-D5D6-473F-86DA-5E0879AD0C7E@stufft.io>
Message-ID: <5272D587.9080001@trueblade.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/31/2013 6:05 PM, Donald Stufft wrote:
> 
> On Oct 31, 2013, at 5:52 PM, Eric V. Smith <eric at trueblade.com>
> wrote:
> 
>> I'm more than happy to build them myself on a dedicated build
>> machine.
> 
> This works! You just can?t upload/install from PyPI.
> 
> The restriction is *only* centered around PyPI.

Ah, I misunderstood. Cool, and thanks for all of the work!

Eric.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (Cygwin)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJSctWHAAoJENxauZFcKtNxzXEH/0XhiAoMeS8WODLrKSREkDFi
trvaIFU/AaC0fIidkjqSmSk/axaD8mp8yPCjo/PVeq+ZM7Mi1pmZmGmU7k6KkpeS
xdEYCpMBg72yYFzo/fyvIQGgVm3Y/mMaIVc3fBbzfEk08vMbNUVFwgg0Jtz8OOfh
F2oqTLxqLYhaLQ36hpWuQpJYHDNEEa0fDMiomzKhvzoFEu4e5XhgMpi2DrkYQWbM
q7mmLOhJ76SVzODx+fx+jvAu8xQ++MGNxcQ90O4m1pP+RPHnw5DAXVE6jJ1Ko8jB
S5zKKvInFeLB5eQwxgQ1S3JMLLQOD02y2pI+w3T7fbhUBtiKVctL02dXxOUw6/Q=
=1dem
-----END PGP SIGNATURE-----

From tseaver at palladion.com  Thu Oct 31 23:17:32 2013
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 31 Oct 2013 18:17:32 -0400
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <5272D12C.4040907@trueblade.com>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
 <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
 <E0DC4524-DF0A-4D64-9D43-D1B1B9694185@mac.com>
 <CADiSq7cZLUchLYDXyXOze_E=HtCJrByOKMGj_QagDCVuye=JzA@mail.gmail.com>
 <CAG8k2+6oD77FgPr6J9FUQb3tujY5UCJm5soyaSdE5Rtx=gHE5w@mail.gmail.com>
 <9EF006C3-5FB4-463D-87F2-D8E1BFBC8CDB@mac.com>
 <049ABB88-78A4-43A9-B3D3-54110CE3D53B@stufft.io> <l4ufp3$gbs$1@ger.gmane.org>
 <CADiSq7enQUVchS9e9DEgNbBWPG4CjwEABQdprU6e+c-2f0yunQ@mail.gmail.com>
 <5272D12C.4040907@trueblade.com>
Message-ID: <l4uktj$8q1$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/31/2013 05:52 PM, Eric V. Smith wrote:
> I'm more than happy to build them myself on a dedicated build
> machine.

Right -- that makes them back into caches. ;)  You can safely deploy them
because you know the architecture / libc / etc. against which you built
them.  They are highly unlikely to be useful to anyone *else*, however,
except by accident.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlJy1vwACgkQ+gerLs4ltQ7HqQCgs3toEwtRSn5q+USRXdOWDB+C
dusAn0fSfRyUmRSvqGQ3IUh1Kl6Jh36X
=OCEp
-----END PGP SIGNATURE-----


From tseaver at palladion.com  Thu Oct 31 23:18:49 2013
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 31 Oct 2013 18:18:49 -0400
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <1123DCC9-589D-405A-B567-BD39654866E2@stufft.io>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
 <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
 <E0DC4524-DF0A-4D64-9D43-D1B1B9694185@mac.com>
 <CADiSq7cZLUchLYDXyXOze_E=HtCJrByOKMGj_QagDCVuye=JzA@mail.gmail.com>
 <CAG8k2+6oD77FgPr6J9FUQb3tujY5UCJm5soyaSdE5Rtx=gHE5w@mail.gmail.com>
 <9EF006C3-5FB4-463D-87F2-D8E1BFBC8CDB@mac.com>
 <049ABB88-78A4-43A9-B3D3-54110CE3D53B@stufft.io> <l4ufp3$gbs$1@ger.gmane.org>
 <1123DCC9-589D-405A-B567-BD39654866E2@stufft.io>
Message-ID: <l4ukvv$8q1$2@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/31/2013 04:55 PM, Donald Stufft wrote:
> 
> On Oct 31, 2013, at 4:49 PM, Tres Seaver <tseaver at palladion.com>
> wrote:
> 
>> Pip *does* need to allow installing them on those platforms, but
>> probably only via explicit paths / URLS (rather than finding them on
>> PyPI).
> 
> You can install them just fine on any platform, the only restrictions
> are PyPI won?t let you upload them, and pip won?t try to install them
> from pypi.python.org.

Excellent -- sorry I misunderstood.  AFAICT, you could leave that
restriction in place for ever for Linux.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlJy10kACgkQ+gerLs4ltQ5C0QCePWr4hX6zTff9hyCO9+w5s0Ac
R8sAn11NPAr3d+SCpricLcdLOhtk/nmp
=V/rv
-----END PGP SIGNATURE-----


From oscar.j.benjamin at gmail.com  Thu Oct 31 23:33:01 2013
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Thu, 31 Oct 2013 22:33:01 +0000
Subject: [Distutils] AttributeError: 'tuple' object has no attribute
	'split'
In-Reply-To: <etPan.527280a5.4775af33.8d60@Aragorn>
References: <etPan.52697d87.1d0a04e.8d60@Aragorn>
 <CAHVvXxRBC_TtcZwdA2gd=-U5AJvGBWtm8FNAzoTRUdLixic1ng@mail.gmail.com>
 <etPan.526a85c8.7aed0d9f.8d60@Aragorn>
 <CAHVvXxTpMDMhU2Q96K6e++2JGqCLp1RWAFKBSr24P29-y+1qSA@mail.gmail.com>
 <etPan.526ab328.48945699.8d60@Aragorn>
 <etPan.527280a5.4775af33.8d60@Aragorn>
Message-ID: <CAHVvXxSkVEV7e+dkXRQMc1X2Rqj+tnBMndA6Bu0ph3gidkFOCQ@mail.gmail.com>

On Oct 31, 2013 4:09 PM, "Dominique Orban" <dominique.orban at gmail.com>
wrote:
>
> On 25 October, 2013 at 2:06:34 PM, Dominique Orban (
dominique.orban at gmail.com) wrote:
> >
> >
> >
> >On 25 October, 2013 at 1:56:26 PM, Oscar Benjamin (
oscar.j.benjamin at gmail.com) wrote:
> >>
> >>On Oct 25, 2013 3:52 PM, "Dominique Orban"
> >>wrote:
> >>>
> >>>
> >>>
> >>> On 25 October, 2013 at 9:31:16 AM, Oscar Benjamin (
> >>oscar.j.benjamin at gmail.com) wrote:
> >>> >
> >>> >On 24 October 2013 21:04, Dominique Orban wrote:
> >>> >>
> >>> >> I hope this is the right place to ask for help. I'm not finding
much
> >>comfort in the PyPi documentation or in Google searches. I uploaded my
> >>package `pykrylov` with `python setup.py sdist upload`. Installing it
> >>locally with `python setup.py` install works fine but `pip install
> >>pykrylov` breaks with the messages below. I since removed it from PyPI
but
> >>I get the same error message if I try installing from the git
repository.
> >>I'm hoping someone can put me on track as I've no idea what's wrong. You
> >>can see my setup.py here:
> >>> >>
> >>> >>
> >>
https://github.com/dpo/pykrylov/blob/ea553cdb287f6e685406ceadcb297fd6704af52d/setup.py
> >>> >>
> >>> >> I'm using Python 2.7.5 on OSX installed with Homebrew and pip
1.4.1.
> >>Attempts to upgrade setuptools or pip result in another error message
> >>(AttributeError: 'str' object has no attribute 'rollback')...
> >>> >
> >>> >Can you install a more recent setuptools by downloading it and
running
> >>> >the setup.py yourself?
> >>> >
> >>
https://pypi.python.org/packages/source/s/setuptools/setuptools-1.1.6.tar.gz
> >>>
> >>> Thanks for the suggestion. I'm still getting the same error with
> >>setuptools 1.1.6. I also tried "upgrading" Numpy (since I'm using Numpy
> >>distutils) by installing from their git repository, and I'm still
getting
> >>the same error.
> >>>
> >>> Is anything obviously wrong with the setup.py?
> >>
> >>I don't know but I'm not totally clear what you mean. Previously you
> >>described multiple problems: with pip, setuptools and pykrylov. Have you
> >>successfully installed setuptools now?
> >>
> >>If the "same error" is with pykrylov's setup.py have you tried debugging
> >>it? E.g. 'python -m pdb setup.py install'
> >
> >"python setup.py install" works fine. It's the installation with pip
that returns the error message I mentioned. I was wondering if something in
setup.py didn't agree with pip/setuptools.
> >
> >Yes I installed setuptools 1.1.6. But "pip install -e git://
github.com/dpo/pykrylov.git at ea553cd#egg=pykrylov" still returns
"AttributeError: 'tuple' object has no attribute 'split'".
> >
> >I hope I'm making sense.
>
> Anybody can provide any help with this?

Run pip under pdb. Find the location of the pip script and run:
python -m pdb /path/to/pip install ...

Then find out which object has the wrong type and see if you can trace it
back to something in the setup.py.

Oscar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131031/d5f47c63/attachment.html>

From oscar.j.benjamin at gmail.com  Thu Oct 31 23:52:14 2013
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Thu, 31 Oct 2013 22:52:14 +0000
Subject: [Distutils] Plans for binary wheels, and PyPi and OS-X
In-Reply-To: <l4ufp3$gbs$1@ger.gmane.org>
References: <CALGmxEKPEs0gAun3_SWVXPXjRo17NyTRbkrh0pN2tSkX5CfNJQ@mail.gmail.com>
 <CADiSq7cfFyE4gaa5DvphpJYq0iv2rzJqgQ2uygQGF3RS_ep34w@mail.gmail.com>
 <CALGmxEJ7Wy-AAMFvzS5MjFwCGZKD3HHJBPXzfasekmf42SSH8w@mail.gmail.com>
 <EB992840-5763-496B-AEFB-5644701F85A9@stufft.io>
 <E0DC4524-DF0A-4D64-9D43-D1B1B9694185@mac.com>
 <CADiSq7cZLUchLYDXyXOze_E=HtCJrByOKMGj_QagDCVuye=JzA@mail.gmail.com>
 <CAG8k2+6oD77FgPr6J9FUQb3tujY5UCJm5soyaSdE5Rtx=gHE5w@mail.gmail.com>
 <9EF006C3-5FB4-463D-87F2-D8E1BFBC8CDB@mac.com>
 <049ABB88-78A4-43A9-B3D3-54110CE3D53B@stufft.io>
 <l4ufp3$gbs$1@ger.gmane.org>
Message-ID: <CAHVvXxQNN5YN4Pcj+vytL=Eok5Oev9ciYfqvyRbLDaebuWVj=A@mail.gmail.com>

On Oct 31, 2013 8:50 PM, "Tres Seaver" <tseaver at palladion.com> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 10/31/2013 02:24 PM, Donald Stufft wrote:
> > To be honest the same problems likely exists on Windows, it?s just
> > less likely and the benefits of prebuilt binaries greater.
>
> For all platforms *except* Windows, wheels are essentially caches --
> there is no real reason to distribute them via PyPI at all, because OSx
> and Linux develpoers will have tools to build them from sdists.

What if an OSX user wants to install numpy/scipy? How easy is it to do this
from source (I really don't know)?

Building the necessary BLAS/LAPACK libraries isn't easy on any platform.
It's just easier on a linux distro when the package manager can do it for
you.

Oscar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131031/04698c8d/attachment.html>