From ncoghlan at gmail.com Wed Nov 1 00:33:44 2017 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 1 Nov 2017 14:33:44 +1000 Subject: [Distutils] [Python-ideas] Add processor generation to wheel metadata In-Reply-To: References: <4d49183c-d1f5-136b-f185-87fa19575bc3@mail.mipt.ru> <1fd22c98-32f3-3b0d-2dd1-8b90bf65083a@mail.mipt.ru> Message-ID: On 1 November 2017 at 02:23, Wes Turner wrote: > Maybe the anaconda team has some insight on a standard way to capture (& > configure) compiler versions and flags in package metadata? The short answer is "You don't, and instead live with the related uncertainty". Similar to Linux distro packages, these expectations are set by the build environment, and it's up to whoever's publishing a conda channel to make sure all the packages it contains are consistent both with each other, and with 3rd party wheel files (if they want to support layered applications which mix third party wheel files with platform provided binaries). https://reproducible-builds.org/docs/perimeter/ discusses some of the aspects of a system that may or may not affect a build process, and hence the ABI compatibility of a result. There unfortunately isn't a generic way of knowing which CPU flags are actually going to be important in determining a project's ABI or its platform requirements (e.g. CPython's independence of SSE2 doesn't arise from specific choice - it arises from the fact that there isn't any code in CPython that relies on CPU provided vector operations). In the PEP 426/459 draft, my proposal was to have an optional extension to the metadata called "python.constraints" that allowed projects to declare particular compatibility constraints on their installation environments: https://www.python.org/dev/peps/pep-0459/#the-python-constraints-extension That way, an installer could download a wheel file, check if its usage constraints were met, and if not, fall back to building from source. That still seems like a reasonable way to go to me, although we may want to look at defining it as a separate JSON file stored in dist-info (similar to entry_points.txt), rather than tying it to a new version of the main metadata spec. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From vano at mail.mipt.ru Wed Nov 1 04:14:40 2017 From: vano at mail.mipt.ru (Ivan Pozdeev) Date: Wed, 1 Nov 2017 11:14:40 +0300 Subject: [Distutils] [Python-ideas] Add processor generation to wheel metadata In-Reply-To: References: <4d49183c-d1f5-136b-f185-87fa19575bc3@mail.mipt.ru> <1fd22c98-32f3-3b0d-2dd1-8b90bf65083a@mail.mipt.ru> Message-ID: <025bc1a6-1b03-2462-3d11-c8cb44359578@mail.mipt.ru> On 31.10.2017 19:23, Wes Turner wrote: > Maybe the anaconda team has some insight on a standard way to capture > (& configure) compiler versions and flags in package metadata? > > From > https://www.anaconda.com/blog/developer-blog/announcing-the-release-of-anaconda-distribution-5-0/ > : > > >?The Anaconda 5.0 release used very modern compilers to rebuild > almost everything (~99.5%) provided in the installers for x86 Linux > and MacOS. This enables Anaconda users to get the benefits of the > latest compilers? still allowing support for older operating > systems?back to MacOS 10.9 and CentOS 6. Our own builds of GCC 7.2 > (Linux) and Clang 4.0.1 (MacOS) are used, and every reasonable > security flag has been enabled. CFLAGS and CXXFLAGS are no longer > managed by each package; instead compiler activation sets them globally. > > > > The packages built with the new compilers are in a different channel > from packages built the old way, and as we build out this new channel, > we will eventually be able to change the default experience to only > using these packages. Interested in using this approach to build your > own conda packages? Stay tuned for a more developer-focused blog post! It's all the same -- all packages are supposed to be built with the same compiler settings. So there's no problem of saving the flags into package metadata and checking compatibility with the local system. There's, in fact, the same problem of potential unchecked incompatibility if a package is compiled in a different environment: see https://stackoverflow.com/questions/46912969/osx-c14-compiler-not-detected-multiple-versions-of-gcc-when-compiling-graph (in that case, it's a different compiler version). -- Regards, Ivan From stuaxo2 at yahoo.com Sun Nov 5 04:49:42 2017 From: stuaxo2 at yahoo.com (Stuart Axon) Date: Sun, 5 Nov 2017 09:49:42 +0000 (UTC) Subject: [Distutils] Where to get help changing ownership of Pypi package? References: <282224689.2537964.1509875382280.ref@mail.yahoo.com> Message-ID: <282224689.2537964.1509875382280@mail.yahoo.com> Hi,?? I realise this is the wrong place, but does anyone know who could help resolve ownership of the Pypi package PyCairo https://sourceforge.net/p/pypi/support-requests/728/ There are also a bunch of similar tickets, I guess nobody checks these any more ? :( S++ -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Sun Nov 5 10:45:51 2017 From: wes.turner at gmail.com (Wes Turner) Date: Sun, 5 Nov 2017 10:45:51 -0500 Subject: [Distutils] Where to get help changing ownership of Pypi package? In-Reply-To: <282224689.2537964.1509875382280@mail.yahoo.com> References: <282224689.2537964.1509875382280.ref@mail.yahoo.com> <282224689.2537964.1509875382280@mail.yahoo.com> Message-ID: There's a "package claim" label in the pypa/pypi-legacy Github issue tracker: https://github.com/pypa/pypi-legacy/issues What is the documented procedure for handling transfer of package ownership? On Sunday, November 5, 2017, Stuart Axon via Distutils-SIG < distutils-sig at python.org> wrote: > Hi, > I realise this is the wrong place, but does anyone know who could help > resolve ownership of the Pypi package PyCairo > > https://sourceforge.net/p/pypi/support-requests/728/ > > > There are also a bunch of similar tickets, I guess nobody checks these any > more ? :( > > S++ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pradyunsg at gmail.com Sun Nov 5 11:38:02 2017 From: pradyunsg at gmail.com (Pradyun Gedam) Date: Sun, 05 Nov 2017 16:38:02 +0000 Subject: [Distutils] Where to get help changing ownership of Pypi package? In-Reply-To: References: <282224689.2537964.1509875382280.ref@mail.yahoo.com> <282224689.2537964.1509875382280@mail.yahoo.com> Message-ID: On Sun, 5 Nov 2017, 21:16 Wes Turner, wrote: > There's a "package claim" label in the pypa/pypi-legacy Github issue > tracker: > > https://github.com/pypa/pypi-legacy/issues > > What is the documented procedure for handling transfer of package > ownership? > > There's PEP 541; it's still a draft though. I think there was a thread for this in early this year and also more recently around September. Until that PEP is accepted (likely even after), the URL that Wes posted is probably the right place... Best, Pradyun > > On Sunday, November 5, 2017, Stuart Axon via Distutils-SIG < > distutils-sig at python.org> wrote: > >> Hi, >> I realise this is the wrong place, but does anyone know who could help >> resolve ownership of the Pypi package PyCairo >> >> https://sourceforge.net/p/pypi/support-requests/728/ >> >> >> There are also a bunch of similar tickets, I guess nobody checks these >> any more ? :( >> >> S++ >> > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vano at mail.mipt.ru Tue Nov 7 16:22:29 2017 From: vano at mail.mipt.ru (Ivan Pozdeev) Date: Wed, 8 Nov 2017 00:22:29 +0300 Subject: [Distutils] [Python-ideas] Add processor generation to wheel metadata In-Reply-To: References: <4d49183c-d1f5-136b-f185-87fa19575bc3@mail.mipt.ru> <1fd22c98-32f3-3b0d-2dd1-8b90bf65083a@mail.mipt.ru> Message-ID: <62be3b1f-d74c-e2a1-b385-ea965ab30060@mail.mipt.ru> On 01.11.2017 7:33, Nick Coghlan wrote: > https://reproducible-builds.org/docs/perimeter/ discusses some of the > aspects of a system that may or may not affect a build process, and > hence the ABI compatibility of a result. There unfortunately isn't a > generic way of knowing which CPU flags are actually going to be > important in determining a project's ABI or its platform requirements There's no such thing as "a set of compiler flags that unequivocally defines ABI." Any proprocessor macro that changes a type or a subroutine's signature will result in an incompatibility, too. So, "ABI" can only be defined on a case by case basis. > (e.g. CPython's independence of SSE2 doesn't arise from specific > choice - it arises from the fact that there isn't any code in CPython > that relies on CPU provided vector operations). CPython is compiled with the corresponding /arch and doesn't use SSE2 intrinsics (I did check that cl.exe doesn't trigger an error on the latter even with lower /arch). That's enough to guarantee the instruction set that it uses. So, I don't see a problem here. > That way, an installer could download a wheel file, check if its usage > constraints were met, and if not, fall back to building from source. > That still seems like a reasonable way to go to me, although we may > want to look at defining it as a separate JSON file stored in > dist-info (similar to entry_points.txt), rather than tying it to a new > version of the main metadata spec. Having to download a wheel just to check compatibility is a deal breaker. There are dozens of them for a given package (pip examines old versions, too), and each one can weigh quite a bit. With more flavors to add, the number will multiply. Metadata required to check if the wheel is suitable for install must be available separately from its contents. Currently, pip examines the triplets in package names. What I'm suggesting is to put/append something more specific than "win32" into it when applicable. > In the PEP 426/459 draft, my proposal was to have an optional > extension to the metadata called "python.constraints" that allowed > projects to declare particular compatibility constraints on their > installation environments: > https://www.python.org/dev/peps/pep-0459/#the-python-constraints-extension I checked it out. It's not applicable here for the aforementioned reason, and one more thing. A package's file name should be unique - to easily distinguish it from others. The _one obviouls way_(tm) is to make it contain all the metadata fields that distinguish this package from others. I mean, you _can_ make metadata available otherwise (with some API, in a small companion file etc.) -- but that'd be suboptimal to simply a meaningful file name that will go with this file wherever it goes. (For an illustration why this is an anti-pattern, you can check out a WSUS server contents -- that's an epitome of meaningless filenames, you can't know anything about a file without a database query. Not only is that grossly inconvenient; if the database goes under, the entire multigigabyte storage becomes useless.) -- Regards, Ivan; -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas at kluyver.me.uk Sun Nov 12 13:59:23 2017 From: thomas at kluyver.me.uk (Thomas Kluyver) Date: Sun, 12 Nov 2017 18:59:23 +0000 Subject: [Distutils] PEP 517 reference consumer Message-ID: <1510513163.2647691.1169959736.4E3617D9@webmail.messagingengine.com> I have found a bit more time to work on my PEP 517 consumer API: https://github.com/takluyver/pep517 It provides two main interfaces: - 'pep517.wrappers' is a lower-level interface to call the backend hooks in a subprocess. - 'pep517.envbuild' is a higher-level interface which creates a temporary environment, uses pip to install build dependencies, and builds a wheel or an sdist. I hope that some frontend tools will be able to use these interfaces directly, but I also intend it as a reference consumer which backend authors can use to test their implementation. As such, I'd like to propose moving the project into the control of the Python Packagaing Authority - i.e. transferring the Github repo and giving other people upload rights on PyPI. In other news, Flit 0.12 is out, providing a PEP 517 backend. Though come to think of it, I may have forgotten to use the tar pax format. Thomas From p.f.moore at gmail.com Sun Nov 12 14:21:57 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 12 Nov 2017 19:21:57 +0000 Subject: [Distutils] PEP 517 reference consumer In-Reply-To: <1510513163.2647691.1169959736.4E3617D9@webmail.messagingengine.com> References: <1510513163.2647691.1169959736.4E3617D9@webmail.messagingengine.com> Message-ID: On 12 November 2017 at 18:59, Thomas Kluyver wrote: > I have found a bit more time to work on my PEP 517 consumer API: > > https://github.com/takluyver/pep517 > > It provides two main interfaces: > - 'pep517.wrappers' is a lower-level interface to call the backend hooks > in a subprocess. > - 'pep517.envbuild' is a higher-level interface which creates a > temporary environment, uses pip to install build dependencies, and > builds a wheel or an sdist. Nice :-) > I hope that some frontend tools will be able to use these interfaces > directly, Is it (in your opinion) something that pip might be able to use? The biggest potential hurdle, I would imagine, is that if pep517.envbuild uses pip to install dependencies, and pip uses pep517.envbuild, there's a risk of a recursive loop that could result in starting an arbitrarily large number of processes on the user's machine (this is one of the issues that has stalled the current work on PEP 517 for pip). On the other hand, other frontends depending on pip (indirectly, via pep517.envbuild) might seem a bit odd. > but I also intend it as a reference consumer which backend > authors can use to test their implementation. As such, I'd like to > propose moving the project into the control of the Python Packagaing > Authority - i.e. transferring the Github repo and giving other people > upload rights on PyPI. +1 to that suggestion. Paul From thomas at kluyver.me.uk Sun Nov 12 15:08:02 2017 From: thomas at kluyver.me.uk (Thomas Kluyver) Date: Sun, 12 Nov 2017 20:08:02 +0000 Subject: [Distutils] PEP 517 reference consumer In-Reply-To: References: <1510513163.2647691.1169959736.4E3617D9@webmail.messagingengine.com> Message-ID: <1510517282.2665303.1170000776.43266557@webmail.messagingengine.com> On Sun, Nov 12, 2017, at 07:21 PM, Paul Moore wrote: > Is it (in your opinion) something that pip might be able to use? The > biggest potential hurdle, I would imagine, is that if pep517.envbuild > uses pip to install dependencies, and pip uses pep517.envbuild, > there's a risk of a recursive loop that could result in starting an > arbitrarily large number of processes on the user's machine (this is > one of the issues that has stalled the current work on PEP 517 for > pip). On the other hand, other frontends depending on pip (indirectly, > via pep517.envbuild) might seem a bit odd. I would guess that pip could use the lower-level interface directly, but it might want to control how dependencies are found and installed. The higher level part is based on code already in pip (pip._internal.wheel.BuildEnvironment), so hopefully integrating it shouldn't be too much work. I've also just added another module to test the hooks for a project and point out any problems. It's used like this: python3 -m pep517.check path/to/source Thomas From ncoghlan at gmail.com Sun Nov 12 17:51:28 2017 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Nov 2017 08:51:28 +1000 Subject: [Distutils] PEP 517 reference consumer In-Reply-To: <1510513163.2647691.1169959736.4E3617D9@webmail.messagingengine.com> References: <1510513163.2647691.1169959736.4E3617D9@webmail.messagingengine.com> Message-ID: On 13 November 2017 at 04:59, Thomas Kluyver wrote: > I have found a bit more time to work on my PEP 517 consumer API: > > https://github.com/takluyver/pep517 > > It provides two main interfaces: > - 'pep517.wrappers' is a lower-level interface to call the backend hooks > in a subprocess. > - 'pep517.envbuild' is a higher-level interface which creates a > temporary environment, uses pip to install build dependencies, and > builds a wheel or an sdist. > > I hope that some frontend tools will be able to use these interfaces > directly, but I also intend it as a reference consumer which backend > authors can use to test their implementation. As such, I'd like to > propose moving the project into the control of the Python Packagaing > Authority - i.e. transferring the Github repo and giving other people > upload rights on PyPI. +1 from me, too. > In other news, Flit 0.12 is out, providing a PEP 517 backend Cool! Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From phil at riverbankcomputing.com Mon Nov 13 05:07:51 2017 From: phil at riverbankcomputing.com (Phil Thompson) Date: Mon, 13 Nov 2017 10:07:51 +0000 Subject: [Distutils] PyPI Limits of Wheel Size Message-ID: I am hitting a limit on the size of a wheel when uploading to PyPI - the wheel is about 100M. When this has happened in the past I have contacted Donald who has been kind enough to increase the limit (I believe it is a per-project limit). Currently where should I make this request? Thanks, Phil From matthew.brett at gmail.com Mon Nov 13 05:21:16 2017 From: matthew.brett at gmail.com (Matthew Brett) Date: Mon, 13 Nov 2017 10:21:16 +0000 Subject: [Distutils] PyPI Limits of Wheel Size In-Reply-To: References: Message-ID: Hi, On Mon, Nov 13, 2017 at 10:07 AM, Phil Thompson wrote: > I am hitting a limit on the size of a wheel when uploading to PyPI - the wheel is about 100M. When this has happened in the past I have contacted Donald who has been kind enough to increase the limit (I believe it is a per-project limit). > > Currently where should I make this request? I'm hoping / guessing that this is a PyQt wheel? That would be a huge help ... Cheers, Matthew From thomas at kluyver.me.uk Thu Nov 16 06:22:25 2017 From: thomas at kluyver.me.uk (Thomas Kluyver) Date: Thu, 16 Nov 2017 11:22:25 +0000 Subject: [Distutils] PEP 517 reference consumer In-Reply-To: References: <1510513163.2647691.1169959736.4E3617D9@webmail.messagingengine.com> Message-ID: <1510831345.3847030.1174495240.3D9A22F9@webmail.messagingengine.com> On Sun, Nov 12, 2017, at 10:51 PM, Nick Coghlan wrote: > +1 from me, too. Whose further agreement should we get for this? And - if there is agreement - how do we do the transfer? If someone briefly gives me permission to create repos in the PyPA org, I can use Github's transfer feature. Otherwise, we can do it manually by creating a new repo and removing the old one. Thanks, Thomas From ncoghlan at gmail.com Thu Nov 16 07:46:40 2017 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 16 Nov 2017 22:46:40 +1000 Subject: [Distutils] PEP 517 reference consumer In-Reply-To: <1510831345.3847030.1174495240.3D9A22F9@webmail.messagingengine.com> References: <1510513163.2647691.1169959736.4E3617D9@webmail.messagingengine.com> <1510831345.3847030.1174495240.3D9A22F9@webmail.messagingengine.com> Message-ID: On 16 November 2017 at 21:22, Thomas Kluyver wrote: > On Sun, Nov 12, 2017, at 10:51 PM, Nick Coghlan wrote: > > +1 from me, too. > > Whose further agreement should we get for this? And - if there is > agreement - how do we do the transfer? If someone briefly gives me > permission to create repos in the PyPA org, I can use Github's transfer > feature. Otherwise, we can do it manually by creating a new repo and > removing the old one. > Paul and Donald can accept transfer requests. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas at kluyver.me.uk Thu Nov 16 08:16:28 2017 From: thomas at kluyver.me.uk (Thomas Kluyver) Date: Thu, 16 Nov 2017 13:16:28 +0000 Subject: [Distutils] PEP 517 reference consumer In-Reply-To: References: <1510513163.2647691.1169959736.4E3617D9@webmail.messagingengine.com> <1510831345.3847030.1174495240.3D9A22F9@webmail.messagingengine.com> Message-ID: <1510838188.3876464.1174600984.235258FC@webmail.messagingengine.com> On Thu, Nov 16, 2017, at 12:46 PM, Nick Coghlan wrote: > On 16 November 2017 at 21:22, Thomas Kluyver > wrote:>> On Sun, Nov 12, 2017, at 10:51 PM, Nick Coghlan wrote: >> > +1 from me, too. >> >> Whose further agreement should we get for this? And - if there is >> agreement - how do we do the transfer? If someone briefly gives me >> permission to create repos in the PyPA org, I can use Github's >> transfer>> feature. Otherwise, we can do it manually by creating a new repo and>> removing the old one. > > Paul and Donald can accept transfer requests. It appears that you can't make transfer requests to an organisation. Paul, Donald, if you're happy to accept it I could make a transfer request to one of your individual accounts, and you'd then transfer it again to pypa. Or maybe I've missed something about how to do it. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Thu Nov 16 08:31:20 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 16 Nov 2017 13:31:20 +0000 Subject: [Distutils] PEP 517 reference consumer In-Reply-To: <1510838188.3876464.1174600984.235258FC@webmail.messagingengine.com> References: <1510513163.2647691.1169959736.4E3617D9@webmail.messagingengine.com> <1510831345.3847030.1174495240.3D9A22F9@webmail.messagingengine.com> <1510838188.3876464.1174600984.235258FC@webmail.messagingengine.com> Message-ID: I'm happy to (not sure what needs to be done, but I'm sure I can muddle it through ;-)) Paul On 16 November 2017 at 13:16, Thomas Kluyver wrote: > On Thu, Nov 16, 2017, at 12:46 PM, Nick Coghlan wrote: > > On 16 November 2017 at 21:22, Thomas Kluyver wrote: > > On Sun, Nov 12, 2017, at 10:51 PM, Nick Coghlan wrote: >> +1 from me, too. > > Whose further agreement should we get for this? And - if there is > agreement - how do we do the transfer? If someone briefly gives me > permission to create repos in the PyPA org, I can use Github's transfer > feature. Otherwise, we can do it manually by creating a new repo and > removing the old one. > > > Paul and Donald can accept transfer requests. > > > It appears that you can't make transfer requests to an organisation. Paul, > Donald, if you're happy to accept it I could make a transfer request to one > of your individual accounts, and you'd then transfer it again to pypa. Or > maybe I've missed something about how to do it. > > Thomas > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > From thomas at kluyver.me.uk Thu Nov 16 10:04:16 2017 From: thomas at kluyver.me.uk (Thomas Kluyver) Date: Thu, 16 Nov 2017 15:04:16 +0000 Subject: [Distutils] PEP 517 reference consumer In-Reply-To: References: <1510513163.2647691.1169959736.4E3617D9@webmail.messagingengine.com> <1510831345.3847030.1174495240.3D9A22F9@webmail.messagingengine.com> <1510838188.3876464.1174600984.235258FC@webmail.messagingengine.com> Message-ID: <1510844656.3908360.1174727952.640CB84D@webmail.messagingengine.com> We have now done this, and you can see the new repo here: https://github.com/pypa/pep517 On Thu, Nov 16, 2017, at 01:31 PM, Paul Moore wrote: > I'm happy to (not sure what needs to be done, but I'm sure I can > muddle it through ;-)) > Paul > > On 16 November 2017 at 13:16, Thomas Kluyver > wrote: > > On Thu, Nov 16, 2017, at 12:46 PM, Nick Coghlan wrote: > > > > On 16 November 2017 at 21:22, Thomas Kluyver wrote: > > > > On Sun, Nov 12, 2017, at 10:51 PM, Nick Coghlan wrote: > >> +1 from me, too. > > > > Whose further agreement should we get for this? And - if there is > > agreement - how do we do the transfer? If someone briefly gives me > > permission to create repos in the PyPA org, I can use Github's transfer > > feature. Otherwise, we can do it manually by creating a new repo and > > removing the old one. > > > > > > Paul and Donald can accept transfer requests. > > > > > > It appears that you can't make transfer requests to an organisation. Paul, > > Donald, if you're happy to accept it I could make a transfer request to one > > of your individual accounts, and you'd then transfer it again to pypa. Or > > maybe I've missed something about how to do it. > > > > Thomas > > > > _______________________________________________ > > Distutils-SIG maillist - Distutils-SIG at python.org > > https://mail.python.org/mailman/listinfo/distutils-sig > > From p.f.moore at gmail.com Thu Nov 16 10:36:40 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 16 Nov 2017 15:36:40 +0000 Subject: [Distutils] PEP 517 reference consumer In-Reply-To: <1510844656.3908360.1174727952.640CB84D@webmail.messagingengine.com> References: <1510513163.2647691.1169959736.4E3617D9@webmail.messagingengine.com> <1510831345.3847030.1174495240.3D9A22F9@webmail.messagingengine.com> <1510838188.3876464.1174600984.235258FC@webmail.messagingengine.com> <1510844656.3908360.1174727952.640CB84D@webmail.messagingengine.com> Message-ID: On 16 November 2017 at 15:04, Thomas Kluyver wrote: > We have now done this, and you can see the new repo here: > > https://github.com/pypa/pep517 I've set up a PEP 517 Committers group under the PyPA organisation. At the moment only Thomas and I are members - if anyone else wants to be added, just let me know. Paul From ronaldoussoren at mac.com Mon Nov 20 07:57:22 2017 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Mon, 20 Nov 2017 13:57:22 +0100 Subject: [Distutils] pythonhosted.org doc upload no longer works In-Reply-To: <3FC71839-40AD-497B-8ADD-A343CDB2336A@mac.com> References: <3FC71839-40AD-497B-8ADD-A343CDB2336A@mac.com> Message-ID: <592017EA-7006-4C12-AA7B-C3EBD05A2E8A@mac.com> > On 20 Oct 2017, at 14:42, Ronald Oussoren wrote: > > >> On 7 Oct 2017, at 09:12, Giampaolo Rodola' > wrote: >> >> Any news about this please? This is becoming a problem as my outdated doc keeps being around and I have no way to remove it. > > I have the same problem with my projects: the documentation on pythonhosted.org can no longer be updated, and it is also not possible to change that documentation into a link to a new location. Sigh? I still have outdated documentation on pythonhosted.org and no way to replace this with a link to the new location. As the documentation on pythonhosted is the top hit on Google I?d rather not remove the documentation there without any replacement. Ronald -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Mon Nov 20 13:41:30 2017 From: wes.turner at gmail.com (Wes Turner) Date: Mon, 20 Nov 2017 13:41:30 -0500 Subject: [Distutils] pythonhosted.org doc upload no longer works In-Reply-To: <592017EA-7006-4C12-AA7B-C3EBD05A2E8A@mac.com> References: <3FC71839-40AD-497B-8ADD-A343CDB2336A@mac.com> <592017EA-7006-4C12-AA7B-C3EBD05A2E8A@mac.com> Message-ID: Is there a recommended way to redirect docs that weren't updated before new uploads were prevented? (The mailing list thread about this change is further up in the thread) On Monday, November 20, 2017, Ronald Oussoren wrote: > > On 20 Oct 2017, at 14:42, Ronald Oussoren > wrote: > > > On 7 Oct 2017, at 09:12, Giampaolo Rodola' > wrote: > > Any news about this please? This is becoming a problem as my outdated doc > keeps being around and I have no way to remove it. > > > I have the same problem with my projects: the documentation on > pythonhosted.org can no longer be updated, and it is also not possible to > change that documentation into a link to a new location. > > > Sigh? I still have outdated documentation on pythonhosted.org and no way > to replace this with a link to the new location. As the documentation on > pythonhosted is the top hit on Google I?d rather not remove the > documentation there without any replacement. > > Ronald > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sat Nov 25 20:49:34 2017 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 26 Nov 2017 11:49:34 +1000 Subject: [Distutils] pythonhosted.org doc upload no longer works In-Reply-To: References: <3FC71839-40AD-497B-8ADD-A343CDB2336A@mac.com> <592017EA-7006-4C12-AA7B-C3EBD05A2E8A@mac.com> Message-ID: On 21 November 2017 at 04:41, Wes Turner wrote: > Is there a recommended way to redirect docs that weren't updated before new > uploads were prevented? Unfortunately not currently. https://github.com/pypa/warehouse/issues/582 is the relevant Warehouse issue if folks want to subscribe to that for updates. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From sandro at e-den.it Tue Nov 28 06:33:21 2017 From: sandro at e-den.it (Alessandro Dentella) Date: Tue, 28 Nov 2017 12:33:21 +0100 Subject: [Distutils] buildout & extra-path Message-ID: <20171128113321.GA25989@bluff.e-den.it> Hi, I realized that extra-path is not always honored in buildout. I have a Django application that uses django-recipe with extra-path. I also want a pure ipython program that share the same sys.path. but if I write: [ipython] recipe = zc.recipe.egg eggs = ${buildout:eggs} extra-paths = ${buildout:directory}/apps I get a script that doesn't have the extra-path. I explicitly have to add: interpreter = ipython-dj is this the desired behaviour? sandro -- Sandro Dentella *:-) http://trepalchi.it Il portale degli artisti http://www.reteisi.org Soluzioni libere per le scuole http://sqlkit.argolinux.org SQLkit home page - PyGTK/python/sqlalchemy From jim at jimfulton.info Tue Nov 28 08:40:48 2017 From: jim at jimfulton.info (Jim Fulton) Date: Tue, 28 Nov 2017 06:40:48 -0700 Subject: [Distutils] buildout & extra-path In-Reply-To: <20171128113321.GA25989@bluff.e-den.it> References: <20171128113321.GA25989@bluff.e-den.it> Message-ID: On Tue, Nov 28, 2017 at 4:33 AM, Alessandro Dentella wrote: > Hi, > > I realized that extra-path is not always honored in buildout. I have a > Django application that uses django-recipe with extra-path. > > I also want a pure ipython program that share the same sys.path. but > if I write: > > [ipython] > recipe = zc.recipe.egg > eggs = ${buildout:eggs} > extra-paths = ${buildout:directory}/apps > > I get a script that doesn't have the extra-path. > I explicitly have to add: > > interpreter = ipython-dj > > is this the desired behaviour? > The extra-paths egg recipe option should work, regardless of whether the interpreter option is used. Please create an issue: https://github.com/buildout/buildout/issues/new and include a full configuration file that demonstrates the problem so we can try to reproduce. Jim -- Jim Fulton http://jimfulton.info -------------- next part -------------- An HTML attachment was scrubbed... URL: From donald at stufft.io Tue Nov 28 09:41:13 2017 From: donald at stufft.io (Donald Stufft) Date: Tue, 28 Nov 2017 09:41:13 -0500 Subject: [Distutils] Funding Warehouse Development/Deployment Message-ID: <6E6CAC40-8C73-4DEE-B1C3-48B9477F03E4@stufft.io> As I?m sure most or everyone on this list is aware, the work to replace PyPI with Warehouse has been underway for quite awhile. I?m happy to say that we?ve now got funding from a MOSS grant that should finally help us get this across the finish line and deployed for ?real?. You can read more about this at https://pyfound.blogspot.com/2017/11/the-psf-awarded-moss-grant-pypi.html but the tl;dr is basically we?ve been awarded 170k with the explicit goal of that money being to launch Warehouse and shutdown legacy PyPI. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Nov 28 22:30:31 2017 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 29 Nov 2017 13:30:31 +1000 Subject: [Distutils] Funding Warehouse Development/Deployment In-Reply-To: <6E6CAC40-8C73-4DEE-B1C3-48B9477F03E4@stufft.io> References: <6E6CAC40-8C73-4DEE-B1C3-48B9477F03E4@stufft.io> Message-ID: On 29 November 2017 at 00:41, Donald Stufft wrote: > As I?m sure most or everyone on this list is aware, the work to replace PyPI > with Warehouse has been underway for quite awhile. I?m happy to say that > we?ve now got funding from a MOSS grant that should finally help us get this > across the finish line and deployed for ?real?. You can read more about this > at https://pyfound.blogspot.com/2017/11/the-psf-awarded-moss-grant-pypi.html > but the tl;dr is basically we?ve been awarded 170k with the explicit goal of > that money being to launch Warehouse and shutdown legacy PyPI. And to provide a bit more info on where that funding is going: - project management work to help figure out and document what's actually missing (there's some initial work on this in the Warehouse repo milestones, but it isn't clear if that's complete) - design & development work to actually close the capability gaps and define transition plans where needed This should include things like figuring out what we're going to do with the pythonhosted.org situation (updates to pythonhosted.org docs were broken when the old upload API was shut down, but we don't yet offer a way for folks to redirect their old docs pages to Read The Docs or another static HTML hosting service like GitHub Pages) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From sh at changeset.nyc Wed Nov 29 23:10:42 2017 From: sh at changeset.nyc (Sumana Harihareswara) Date: Wed, 29 Nov 2017 23:10:42 -0500 Subject: [Distutils] Outstanding questions for PEP 541: Package Index Name Retention Message-ID: While getting up to speed on Warehouse I saw how many package transfer requests are waiting for PEP 541[0] to be accepted, so I thought it might be helpful to round up what I see as the outstanding questions. 1) Usage criteria for abandoned projects. > The tricky part there is that "being used" is a tough concept to > define. Over what time period? What amount of downloading counts as > "used"?-- Chris Rose[1] Perhaps a month? suggests Matthias Bussonnier.[2] The ensuing discussion includes thoughts on locking old versions of the project[3]; as I see it, that's a potential feature request for Warehouse, but not something to build into this PEP. (Similarly, Nick Timkovich's idea of "salting" hot-button names on PyPI so it isn't possible to register projects like "android"[4] is a feature idea I like but I think this PEP does not need to wait for it.) 2) A few copyedits from Chris Barker.[5] 3) "How would I, for example, start the process of flagging a project as abandoned?" -- Nick Timkovich[6] It seems to me that the PEP's wording in "Removal of an abandoned project" says we'll allow transfer of abandoned projects but will not remove them only for being abandoned. However, the PEP currently doesn't include a "where to file a ticket" line in that section (as it does in "Invalid projects"). Also, if there's some other reason we should be willing to remove abandoned projects even without a transfer, e.g., the project has a critical security flaw, we should say that somewhere in this PEP or in a different policy document. There is one item ?ukasz mentioned this in the "I decided to not otherwise touch on:" list regarding the revision on January 14th[7] that I think deserves one more chance for discussion. :) A few people brought up making the reachability criteria and instructions crisper. > Regarding reachability: contact attempts should also include the > relevant project's issue tracker if attempts at private contact have > failed. > > This step is important as it allows a project's *user* community to > respond, even if the person that actually pushes the buttons to > upload new releases to PyPI is out of contact for some reason. -- Nick Coghlan[8] Nick Timkovich also suggested some specific instructions[9] that would help set expectations to, among other things, reassure maintainers about the length of offline vacations they can take without worrying about having their packages usurped. :) I'm totally fine with folks saying to me: no, these are all addressed, or not important enough to slow adoption of this PEP. In which case, yay, I hope Donald can accept it and we can start processing the backlog. [0] https://www.python.org/dev/peps/pep-0541/ [1] https://mail.python.org/pipermail/distutils-sig/2017-January/030017.html [2] https://mail.python.org/pipermail/distutils-sig/2017-January/030020.html [3] https://mail.python.org/pipermail/distutils-sig/2017-January/030034.html [4] https://mail.python.org/pipermail/distutils-sig/2017-January/030006.html [5] https://mail.python.org/pipermail/distutils-sig/2017-September/031517.html [6] https://mail.python.org/pipermail/distutils-sig/2017-January/030005.html [7] https://mail.python.org/pipermail/distutils-sig/2017-January/030009.html [8] https://mail.python.org/pipermail/distutils-sig/2017-January/030008.html [9] https://mail.python.org/pipermail/distutils-sig/2017-January/030005.html -- Sumana Harihareswara Changeset Consulting https://changeset.nyc From tesrin at gmail.com Wed Nov 29 22:50:39 2017 From: tesrin at gmail.com (Jimmy Jia) Date: Wed, 29 Nov 2017 22:50:39 -0500 Subject: [Distutils] Policy for deleting packages, releases, and artifacts Message-ID: Hi everyone, This is moved over from https://github.com/pypa/packaging-problems/issues/112. Currently, PyPI has no limitations around deleting packages, releases, or artifacts. This can be problematic for users, as user builds can break in an unsolicited manner if a dependency is removed from PyPI. In the Node ecosystem, a similar lack of limitations there caused significant problems about a year and a half ago, when a widely-used package was deleted following a dispute: http://blog.npmjs.org/post/141577284765/kik-left-pad-and-npm, http://blog.npmjs.org/post/141905368000/changes-to-npms-unpublish-policy. At the time, the scope of the impact was characterized as "breaking the world". To resolve this, npm adopted a policy where package deletions (there's no distinction between a release and an artifact there) could only be done for the first 24 hours after a release was published. Deletions after the 24 hour mark require contacting npm support, and are contingent on the absence of dependents for the deleted release. Of course, npm is a venture-backed for-profit enterprise that has a paid support team ? that's not the case here, so the "support" half of the solution above doesn't make sense here. However, as a starting point, it might still be a good idea to restrict package deletion after that 24-hour window. At least, in the examples given in the links above, the deletion of old packages is strictly a nice-to-have for the package maintainers, balanced against potentially breaking impacts for users. Thanks in advance for any feedback. Thanks, Jimmy Jia From thomas at kluyver.me.uk Thu Nov 30 13:24:54 2017 From: thomas at kluyver.me.uk (Thomas Kluyver) Date: Thu, 30 Nov 2017 18:24:54 +0000 Subject: [Distutils] Caching entry points - performance Message-ID: <1512066294.2990306.1189709976.5C98FF20@webmail.messagingengine.com> To pick up the caching discussion again, I've started to experiment with a couple of different caching techniques. The headline results: a cold-start scan of entry points goes from about 4.5 seconds with no caching, to 0.45 seconds with caching. There's only a small difference (for me) between having a cache file per sys.path directory and a single per-user cache file. I've posted lots more details here: https://github.com/takluyver/entrypoints/issues/16 I keep my sys.path quite short, and the difference between the two approaches presumably increases the more sys.path entries you have. If anyone has time to test the different techniques, I'd be interested to know what your results look like. I've put (Linux) instructions on the same issue. If you'd like to try with a different format or caching mechanism, feel free to add some scripts and see how they compare. I haven't yet tried to integrate either technique into the entrypoints module. Thomas From dholth at gmail.com Thu Nov 30 16:13:46 2017 From: dholth at gmail.com (Daniel Holth) Date: Thu, 30 Nov 2017 21:13:46 +0000 Subject: [Distutils] Caching entry points - performance In-Reply-To: <1512066294.2990306.1189709976.5C98FF20@webmail.messagingengine.com> References: <1512066294.2990306.1189709976.5C98FF20@webmail.messagingengine.com> Message-ID: Neat On Thu, Nov 30, 2017 at 1:25 PM Thomas Kluyver wrote: > To pick up the caching discussion again, I've started to experiment with > a couple of different caching techniques. > > The headline results: a cold-start scan of entry points goes from about > 4.5 seconds with no caching, to 0.45 seconds with caching. There's only > a small difference (for me) between having a cache file per sys.path > directory and a single per-user cache file. I've posted lots more > details here: https://github.com/takluyver/entrypoints/issues/16 > > I keep my sys.path quite short, and the difference between the two > approaches presumably increases the more sys.path entries you have. If > anyone has time to test the different techniques, I'd be interested to > know what your results look like. I've put (Linux) instructions on the > same issue. > > If you'd like to try with a different format or caching mechanism, feel > free to add some scripts and see how they compare. > > I haven't yet tried to integrate either technique into the entrypoints > module. > > Thomas > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: