From ncoghlan at gmail.com Fri Apr 1 10:03:56 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 2 Apr 2016 00:03:56 +1000 Subject: [Distutils] Thank you for the ability to do `pip install git+https://...` In-Reply-To: <56F9297C.9050402@nextday.fi> References: <56F9297C.9050402@nextday.fi> Message-ID: On 28 March 2016 at 22:54, Alex Gr?nholm wrote: > You could always point pip to the automatically generated zip > (https://github.com/nchammas/flintrock/archive/master.zip). That way you > won't have to wait for git to pointlessly download your entire version > history when you just want to install something. I'd assume pip is doing shallow clones when installing from version control, and if that currently isn't the case due to lack of time for implementation and testing, it should be a relatively straightforward PR for someone to submit as a performance optimisation. It's also the case that the automatically generated zip archives are a GitHub specific feature rather than a general property of version control systems, so at least some folks are going to avoid depending on that service specific feature and instead favour the host-independent approach of using the relevant version control protocol. Regards, Nick. P.S. More generally: is a thread where someone is saying "Thanks for this feature!" *really* the best place to be telling them "you shouldn't be using that feature"? It's rare enough for maintainers of any open source project to hear a heartfelt "thank you" that it's at least worth considering taking other replies off list :) -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From alex.gronholm at nextday.fi Fri Apr 1 10:16:15 2016 From: alex.gronholm at nextday.fi (=?UTF-8?Q?Alex_Gr=c3=b6nholm?=) Date: Fri, 1 Apr 2016 17:16:15 +0300 Subject: [Distutils] Thank you for the ability to do `pip install git+https://...` In-Reply-To: References: <56F9297C.9050402@nextday.fi> Message-ID: <56FE82AF.9000602@nextday.fi> I'm sorry if I offended anyone. I was just trying to point out that in the case of Github (or any other service that provides automated tarball generation) it's better to install from those rather than using the VCS integration plugins. Oh, and for the record, I just tested -- it does a deep clone at this time, which would be responsible for the slowness compared to installing from a tarball. 01.04.2016, 17:03, Nick Coghlan kirjoitti: > On 28 March 2016 at 22:54, Alex Gr?nholm wrote: >> You could always point pip to the automatically generated zip >> (https://github.com/nchammas/flintrock/archive/master.zip). That way you >> won't have to wait for git to pointlessly download your entire version >> history when you just want to install something. > I'd assume pip is doing shallow clones when installing from version > control, and if that currently isn't the case due to lack of time for > implementation and testing, it should be a relatively straightforward > PR for someone to submit as a performance optimisation. > > It's also the case that the automatically generated zip archives are a > GitHub specific feature rather than a general property of version > control systems, so at least some folks are going to avoid depending > on that service specific feature and instead favour the > host-independent approach of using the relevant version control > protocol. > > Regards, > Nick. > > P.S. More generally: is a thread where someone is saying "Thanks for > this feature!" *really* the best place to be telling them "you > shouldn't be using that feature"? It's rare enough for maintainers of > any open source project to hear a heartfelt "thank you" that it's at > least worth considering taking other replies off list :) > From erik.m.bray at gmail.com Fri Apr 1 11:11:28 2016 From: erik.m.bray at gmail.com (Erik Bray) Date: Fri, 1 Apr 2016 17:11:28 +0200 Subject: [Distutils] PEP 516 nitpicks Message-ID: Hi all, I was just getting around to reading the latest published draft of PEP 516 and had a few minor nitpicks. Sorry if this unhelpful, as I know it's still being drafted and may change anyways. "The programmatic interface allows decoupling of pip from its current hard dependency on setuptools [2] able for two key reasons:" The "able" at the end of this sentence seems out of place. *** "The file pypa.json acts as a neutral configuration file for pip and other tools..." It's not clear to me what "neutral" means here? From context I'm pretty sure it means not specific to any build tool, but I think it's distracting. I also don't think it's entirely neutral since its *absence* implies a specific build tool. *** "The JSON has the following schema. Extra keys are ignored, which permits the use of pypa.json as a configuration file for other related tools. If doing that the chosen keys must be namespaced under tools" This last part seems inconsistent. It implies that additional keys may be used for other tools, but that they must be under a "tools" key. Okay, but what does that imply for top-level keys not specified by the schema? Will they be ignored? Or should they be explicitly disallowed in order to enable expansion of the format? *** "As usual with processes, a non-zero exit status indicates an error." The "As usual with processes" is somewhat *NIX-biased. I think simply stating the expectation is better: "Non-zero exit statuses are assumed to indicate an error." Of course, any program for which that does not apply for some reason would have to have a wrapper around itself. *** "Note that doing so will cause use operations like pip install -e foo to fail." "use" -> "use of"? Also, how will they fail? Will pip still install dependencies or roll those back too? *** "develop [--prefix PREFIX]" ... "flit develop --root /tmp/ --prefix /usr/local" So does develop have a "--root" option or not? (Maybe this is a moot point if the interface is going to switch back to a Python API? It's not clear to me what the last word was on that.) That's all for now. Thanks! From fungi at yuggoth.org Fri Apr 1 12:29:04 2016 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 1 Apr 2016 16:29:04 +0000 Subject: [Distutils] Thank you for the ability to do `pip install git+https://...` In-Reply-To: <56FE82AF.9000602@nextday.fi> References: <56F9297C.9050402@nextday.fi> <56FE82AF.9000602@nextday.fi> Message-ID: <20160401162904.GP15295@yuggoth.org> On 2016-04-01 17:16:15 +0300 (+0300), Alex Gr?nholm wrote: > I'm sorry if I offended anyone. I was just trying to point out > that in the case of Github (or any other service that provides > automated tarball generation) it's better to install from those > rather than using the VCS integration plugins. Oh, and for the > record, I just tested -- it does a deep clone at this time, which > would be responsible for the slowness compared to installing from > a tarball. Whether this can work depends entirely on the project being installed. Some don't check package metadata into their repos in a form consumable by pip, and require an additional sdist build step which may need information from the VCS itself or manually provided during that step. A prime example of this is projects using PBR, which will want access to Git tag and commit details to determine the package version. With a local clone of the project's Git repository (not just a tarball/zip snapshot of its content) you can build a pip-supported sdist, but otherwise you may need to manually determine and set version information in the calling environment or by editing to be able to then generate a usable sdist from the raw source tree. -- Jeremy Stanley From p.f.moore at gmail.com Mon Apr 4 07:26:50 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 4 Apr 2016 12:26:50 +0100 Subject: [Distutils] abstract build system approaches redux In-Reply-To: References: Message-ID: On 15 March 2016 at 18:34, Robert Collins wrote: > Just to set expectations: this whole process seems stalled to me; I'm > going to context switch and focus on things that can move forward. > Someone please ping me when its relevant to put effort in again :). I'm not sure what is needed to get this process moving again, but when it does, it would probably be worth reviewing https://github.com/pypa/pip/issues/562 The issue there is that --editable and --target don't work well together in pip, if I understand the details it's because setuptools needs extra information to know how to do "setup.py develop" in the face of --target. The question for this thread is, does the build system interface need to know what *other* options get passed to pip as part of deciding what the correct "develop" command should be (or indeed any of the other commands)? Note that I don't have any vested interest in the specific issue in question, and "this is a corner case we shouldn't try to support going forward" is a perfectly acceptable answer as far as I'm concerned. But I thought it was worth raising in case it points to a limitation of the build system command discovery process. Paul From guettliml at thomas-guettler.de Tue Apr 5 14:46:21 2016 From: guettliml at thomas-guettler.de (=?UTF-8?Q?Thomas_G=c3=bcttler?=) Date: Tue, 5 Apr 2016 20:46:21 +0200 Subject: [Distutils] How to deprecate a python package Message-ID: <570407FD.7050305@thomas-guettler.de> I wasted some time because I used a deprecated python package. I asked the maintainer to remove it, and he looked at the usage statistics: I still gets downloaded. What is the official way to deprecate a python package? Related discussion: https://github.com/riklaunim/django-ckeditor/issues/60#issuecomment-205021579 Regards, Thomas G?ttler -- http://www.thomas-guettler.de/ From geoffspear at gmail.com Tue Apr 5 15:07:06 2016 From: geoffspear at gmail.com (Geoffrey Spear) Date: Tue, 5 Apr 2016 15:07:06 -0400 Subject: [Distutils] How to deprecate a python package In-Reply-To: <570407FD.7050305@thomas-guettler.de> References: <570407FD.7050305@thomas-guettler.de> Message-ID: I don't have an answer to your actual question, but I'd not advocate people removing packages; we don't want a npm situation here. :( On Tue, Apr 5, 2016 at 2:46 PM, Thomas G?ttler wrote: > I wasted some time because I used a deprecated python package. > > I asked the maintainer to remove it, and he looked at the usage > statistics: I still gets > downloaded. > > What is the official way to deprecate a python package? > > Related discussion: > > > https://github.com/riklaunim/django-ckeditor/issues/60#issuecomment-205021579 > > Regards, > Thomas G?ttler > > -- > http://www.thomas-guettler.de/ > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From contact at ionelmc.ro Tue Apr 5 15:59:37 2016 From: contact at ionelmc.ro (=?UTF-8?Q?Ionel_Cristian_M=C4=83rie=C8=99?=) Date: Tue, 5 Apr 2016 22:59:37 +0300 Subject: [Distutils] How to deprecate a python package In-Reply-To: <570407FD.7050305@thomas-guettler.de> References: <570407FD.7050305@thomas-guettler.de> Message-ID: What's wrong with a new release that just depends on replacement (assuming there's identical API)? This might be of help: https://pypi.python.org/pypi/pypi-alias If there are API changes or other breakages then maybe a release with a wrapper that emits warnings would be better and generate less surprise/anger. Thanks, -- Ionel Cristian M?rie?, http://blog.ionelmc.ro On Tue, Apr 5, 2016 at 9:46 PM, Thomas G?ttler wrote: > I wasted some time because I used a deprecated python package. > > I asked the maintainer to remove it, and he looked at the usage > statistics: I still gets > downloaded. > > What is the official way to deprecate a python package? > > Related discussion: > > > https://github.com/riklaunim/django-ckeditor/issues/60#issuecomment-205021579 > > Regards, > Thomas G?ttler > > -- > http://www.thomas-guettler.de/ > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.gronholm at nextday.fi Tue Apr 5 16:17:55 2016 From: alex.gronholm at nextday.fi (=?UTF-8?Q?Alex_Gr=c3=b6nholm?=) Date: Tue, 5 Apr 2016 23:17:55 +0300 Subject: [Distutils] How to deprecate a python package In-Reply-To: References: <570407FD.7050305@thomas-guettler.de> Message-ID: <57041D73.3010608@nextday.fi> I think an ideal solution would be to add a feature to Warehouse that would "redirect" any downloads of a library to another. Though I'm not saying it would be simple. 05.04.2016, 22:59, Ionel Cristian M?rie? kirjoitti: > What's wrong with a new release that just depends on replacement > (assuming there's identical API)? This might be of help: > https://pypi.python.org/pypi/pypi-alias > > If there are API changes or other breakages then maybe a release with > a wrapper that emits warnings would be better and generate less > surprise/anger. > > > Thanks, > -- IonelCristian M?rie?, http://blog.ionelmc.ro > > On Tue, Apr 5, 2016 at 9:46 PM, Thomas G?ttler > > > wrote: > > I wasted some time because I used a deprecated python package. > > I asked the maintainer to remove it, and he looked at the usage > statistics: I still gets > downloaded. > > What is the official way to deprecate a python package? > > Related discussion: > > https://github.com/riklaunim/django-ckeditor/issues/60#issuecomment-205021579 > > Regards, > Thomas G?ttler > > -- > http://www.thomas-guettler.de/ > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > > https://mail.python.org/mailman/listinfo/distutils-sig > > > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From tritium-list at sdamon.com Tue Apr 5 17:41:55 2016 From: tritium-list at sdamon.com (Alexander Walters) Date: Tue, 5 Apr 2016 17:41:55 -0400 Subject: [Distutils] How to deprecate a python package In-Reply-To: <570407FD.7050305@thomas-guettler.de> References: <570407FD.7050305@thomas-guettler.de> Message-ID: <57043123.6050808@sdamon.com> The ideal solution is for the maintainer to release one last version of the package with copious use of the warnings module. We really don't want to redirect people blindly - They may be depending on undocumented-but-still-api portions of the original's code that a replacement package might not implement - or more likely - the replacement would only have a similar, but not identical, api. We really don't want to just remove the package - then dependencies break for people who can't upgrade their own codebase for whatever reason, or who just need to deploy fresh again. We might want to implement a package metadata property - Pip can give a big flashing warning on install that the package is deprecated by the maintainer. This should be the ONLY use of this property; let's not start making rules based on deprecation metadata, that's as bad as just removing the package. This leaves, for me, one real option maintainers can do right now, and that's just warn the dickens out of the developer. On 4/5/2016 14:46, Thomas G?ttler wrote: > I wasted some time because I used a deprecated python package. > > I asked the maintainer to remove it, and he looked at the usage statistics: I still gets > downloaded. > > What is the official way to deprecate a python package? > > Related discussion: > > https://github.com/riklaunim/django-ckeditor/issues/60#issuecomment-205021579 > > Regards, > Thomas G?ttler > From tseaver at palladion.com Tue Apr 5 18:05:10 2016 From: tseaver at palladion.com (Tres Seaver) Date: Tue, 5 Apr 2016 18:05:10 -0400 Subject: [Distutils] How to deprecate a python package In-Reply-To: <57041D73.3010608@nextday.fi> References: <570407FD.7050305@thomas-guettler.de> <57041D73.3010608@nextday.fi> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/05/2016 04:17 PM, Alex Gr?nholm wrote: > I think an ideal solution would be to add a feature to Warehouse that > would "redirect" any downloads of a library to another. Though I'm not > saying it would be simple. Such a feature would be doing a huge disservice: repeatability *matters* for package consumers. Unless an already-uploaded package is known to contain malware, or the author is under force majeur compulsion (governmental / legal injunction), removing a distribution is much worse than giving the humans involved flexibility to deal with an issue. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJXBDaKAAoJEPKpaDSJE9HYGYsP/AtJGhNFXPXjtUlTVoDHw6oz ohzb5js31Dps86V/CQELl4cxFhfPMpPCPxcfA9z/E9B4Wk3HaFTPxOUunZKrbUJA 2MguOnsYOjeWCBSlBEOdjCSTHiYse07NRMV4NN+b0mfdnj6VvTk17uY9UW96GTiN xigRgysRgN71JnE41ZNL+4qKjvCL/6dYFrga21rdvOGnZrTNUBwP8mbbACrdz9lh jeOSVkbWMqKazAXIZB3y7KaByIHIfes5fguBnsjqpgdL9c9r8WsE5nhBCdlkUm8N NAiNEpTy+5G5o0NhGz/4AXFtamkVLTGSZhWcUprHOtJUgjzer+b0WWijFcBtcQGY Ugbijhotlbx+zI/QPqArqDemU++UhGr+oiI839KfyzV3viZ4MEr8jC/BchM88Jmn 8lR3Fyv25Tc2bDTC96hv8A5zcwM08i5FYHlPhW2a96xue5Vl9wZ6rmpRUTtqhErJ vwPu/Yps/l1nXzRXPc8NcHTH8daDVIgaNNp8EeDHV+vYJgy066zzzSQ4dTJddWbZ mcf6aFQDP50vrloZ81GaeByUJ1xlcVfyODdvpKj350YlqPqyv7y7uMJv026csRax l/3DyhChbqzU/be9f6xaGL+KzJU3Xt2L0XY/annNkBWrsbRKISpiiGc+21rNo23P EB9Sax3Uoa47h5GWQWH5 =CblP -----END PGP SIGNATURE----- From alex.gronholm at nextday.fi Tue Apr 5 18:26:38 2016 From: alex.gronholm at nextday.fi (=?UTF-8?Q?Alex_Gr=c3=b6nholm?=) Date: Wed, 6 Apr 2016 01:26:38 +0300 Subject: [Distutils] How to deprecate a python package In-Reply-To: References: <570407FD.7050305@thomas-guettler.de> <57041D73.3010608@nextday.fi> Message-ID: <57043B9E.5020004@nextday.fi> You make a valid point. This made me recall something -- there is a classifier "Development Status :: 7 - Inactive". As a quick fix, pip could be modified to emit a warning when a distribution containing this classifier is installed. But the problem seems more social than technical. The author of the package the OP referred to has not chosen to add any status classifier or any kind of warning in the description that would notify the user of its deprecated status. 06.04.2016, 01:05, Tres Seaver kirjoitti: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 04/05/2016 04:17 PM, Alex Gr?nholm wrote: > >> I think an ideal solution would be to add a feature to Warehouse that >> would "redirect" any downloads of a library to another. Though I'm not >> saying it would be simple. > Such a feature would be doing a huge disservice: repeatability *matters* > for package consumers. Unless an already-uploaded package is known to > contain malware, or the author is under force majeur compulsion > (governmental / legal injunction), removing a distribution is much worse > than giving the humans involved flexibility to deal with an issue. > > > Tres. > - -- > =================================================================== > Tres Seaver +1 540-429-0999 tseaver at palladion.com > Palladion Software "Excellence by Design" http://palladion.com > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQIcBAEBAgAGBQJXBDaKAAoJEPKpaDSJE9HYGYsP/AtJGhNFXPXjtUlTVoDHw6oz > ohzb5js31Dps86V/CQELl4cxFhfPMpPCPxcfA9z/E9B4Wk3HaFTPxOUunZKrbUJA > 2MguOnsYOjeWCBSlBEOdjCSTHiYse07NRMV4NN+b0mfdnj6VvTk17uY9UW96GTiN > xigRgysRgN71JnE41ZNL+4qKjvCL/6dYFrga21rdvOGnZrTNUBwP8mbbACrdz9lh > jeOSVkbWMqKazAXIZB3y7KaByIHIfes5fguBnsjqpgdL9c9r8WsE5nhBCdlkUm8N > NAiNEpTy+5G5o0NhGz/4AXFtamkVLTGSZhWcUprHOtJUgjzer+b0WWijFcBtcQGY > Ugbijhotlbx+zI/QPqArqDemU++UhGr+oiI839KfyzV3viZ4MEr8jC/BchM88Jmn > 8lR3Fyv25Tc2bDTC96hv8A5zcwM08i5FYHlPhW2a96xue5Vl9wZ6rmpRUTtqhErJ > vwPu/Yps/l1nXzRXPc8NcHTH8daDVIgaNNp8EeDHV+vYJgy066zzzSQ4dTJddWbZ > mcf6aFQDP50vrloZ81GaeByUJ1xlcVfyODdvpKj350YlqPqyv7y7uMJv026csRax > l/3DyhChbqzU/be9f6xaGL+KzJU3Xt2L0XY/annNkBWrsbRKISpiiGc+21rNo23P > EB9Sax3Uoa47h5GWQWH5 > =CblP > -----END PGP SIGNATURE----- > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig From glyph at twistedmatrix.com Tue Apr 5 18:34:23 2016 From: glyph at twistedmatrix.com (Glyph) Date: Tue, 5 Apr 2016 15:34:23 -0700 Subject: [Distutils] How to deprecate a python package In-Reply-To: <57043B9E.5020004@nextday.fi> References: <570407FD.7050305@thomas-guettler.de> <57041D73.3010608@nextday.fi> <57043B9E.5020004@nextday.fi> Message-ID: Perhaps, before anyone tries to make pip doing something mechanical about deprecations, we should just have the website itself do this sort of redirect? Removing the download would be drastic; giving people an interstitial that says "This package is no longer maintained, please use $X instead" would be very informative. -glyph > On Apr 5, 2016, at 3:26 PM, Alex Gr?nholm wrote: > > You make a valid point. This made me recall something -- there is a classifier "Development Status :: 7 - Inactive". As a quick fix, pip could be modified to emit a warning when a distribution containing this classifier is installed. But the problem seems more social than technical. The author of the package the OP referred to has not chosen to add any status classifier or any kind of warning in the description that would notify the user of its deprecated status. > > 06.04.2016, 01:05, Tres Seaver kirjoitti: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On 04/05/2016 04:17 PM, Alex Gr?nholm wrote: >> >>> I think an ideal solution would be to add a feature to Warehouse that >>> would "redirect" any downloads of a library to another. Though I'm not >>> saying it would be simple. >> Such a feature would be doing a huge disservice: repeatability *matters* >> for package consumers. Unless an already-uploaded package is known to >> contain malware, or the author is under force majeur compulsion >> (governmental / legal injunction), removing a distribution is much worse >> than giving the humans involved flexibility to deal with an issue. >> >> >> Tres. >> - -- =================================================================== >> Tres Seaver +1 540-429-0999 tseaver at palladion.com >> Palladion Software "Excellence by Design" http://palladion.com >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1 >> >> iQIcBAEBAgAGBQJXBDaKAAoJEPKpaDSJE9HYGYsP/AtJGhNFXPXjtUlTVoDHw6oz >> ohzb5js31Dps86V/CQELl4cxFhfPMpPCPxcfA9z/E9B4Wk3HaFTPxOUunZKrbUJA >> 2MguOnsYOjeWCBSlBEOdjCSTHiYse07NRMV4NN+b0mfdnj6VvTk17uY9UW96GTiN >> xigRgysRgN71JnE41ZNL+4qKjvCL/6dYFrga21rdvOGnZrTNUBwP8mbbACrdz9lh >> jeOSVkbWMqKazAXIZB3y7KaByIHIfes5fguBnsjqpgdL9c9r8WsE5nhBCdlkUm8N >> NAiNEpTy+5G5o0NhGz/4AXFtamkVLTGSZhWcUprHOtJUgjzer+b0WWijFcBtcQGY >> Ugbijhotlbx+zI/QPqArqDemU++UhGr+oiI839KfyzV3viZ4MEr8jC/BchM88Jmn >> 8lR3Fyv25Tc2bDTC96hv8A5zcwM08i5FYHlPhW2a96xue5Vl9wZ6rmpRUTtqhErJ >> vwPu/Yps/l1nXzRXPc8NcHTH8daDVIgaNNp8EeDHV+vYJgy066zzzSQ4dTJddWbZ >> mcf6aFQDP50vrloZ81GaeByUJ1xlcVfyODdvpKj350YlqPqyv7y7uMJv026csRax >> l/3DyhChbqzU/be9f6xaGL+KzJU3Xt2L0XY/annNkBWrsbRKISpiiGc+21rNo23P >> EB9Sax3Uoa47h5GWQWH5 >> =CblP >> -----END PGP SIGNATURE----- >> >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> https://mail.python.org/mailman/listinfo/distutils-sig > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig From tritium-list at sdamon.com Tue Apr 5 18:37:58 2016 From: tritium-list at sdamon.com (Alexander Walters) Date: Tue, 5 Apr 2016 18:37:58 -0400 Subject: [Distutils] How to deprecate a python package In-Reply-To: References: <570407FD.7050305@thomas-guettler.de> <57041D73.3010608@nextday.fi> <57043B9E.5020004@nextday.fi> Message-ID: <57043E46.9080007@sdamon.com> On 4/5/2016 18:34, Glyph wrote: > Perhaps, before anyone tries to make pip doing something mechanical about deprecations, we should just have the website itself do this sort of redirect? Removing the download would be drastic; giving people an interstitial that says "This package is no longer maintained, please use $X instead" would be very informative. > > -glyph I don't remember the last time I used the pypi website. I use pypi every day. I don't know if I am weird, but redirecting web views would do nothing for me. Redirecting packages is pure evil. I really think the best course of action is for the maintainer to release a final version of the package that warns on install and import. From alex.gronholm at nextday.fi Tue Apr 5 18:40:47 2016 From: alex.gronholm at nextday.fi (=?UTF-8?Q?Alex_Gr=c3=b6nholm?=) Date: Wed, 6 Apr 2016 01:40:47 +0300 Subject: [Distutils] How to deprecate a python package In-Reply-To: <57043E46.9080007@sdamon.com> References: <570407FD.7050305@thomas-guettler.de> <57041D73.3010608@nextday.fi> <57043B9E.5020004@nextday.fi> <57043E46.9080007@sdamon.com> Message-ID: <57043EEF.1060901@nextday.fi> Wouldn't my suggestion or Glyph's be easier for the maintainers? That way they wouldn't even have to make a new release, just modify a setting on the package settings page on PyPI. Also, are you going you see the warning if it's emitted on installation? 06.04.2016, 01:37, Alexander Walters kirjoitti: > > > On 4/5/2016 18:34, Glyph wrote: >> Perhaps, before anyone tries to make pip doing something mechanical >> about deprecations, we should just have the website itself do this >> sort of redirect? Removing the download would be drastic; giving >> people an interstitial that says "This package is no longer >> maintained, please use $X instead" would be very informative. >> >> -glyph > > I don't remember the last time I used the pypi website. I use pypi > every day. I don't know if I am weird, but redirecting web views > would do nothing for me. Redirecting packages is pure evil. > > I really think the best course of action is for the maintainer to > release a final version of the package that warns on install and import. > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig From tritium-list at sdamon.com Tue Apr 5 18:47:08 2016 From: tritium-list at sdamon.com (Alexander Walters) Date: Tue, 5 Apr 2016 18:47:08 -0400 Subject: [Distutils] How to deprecate a python package In-Reply-To: <57043EEF.1060901@nextday.fi> References: <570407FD.7050305@thomas-guettler.de> <57041D73.3010608@nextday.fi> <57043B9E.5020004@nextday.fi> <57043E46.9080007@sdamon.com> <57043EEF.1060901@nextday.fi> Message-ID: <5704406C.4070704@sdamon.com> I am not 100% sure if pip will let stderr messages though, but i THINK it does. Warnings on import will work regardless. Honestly, I don't care if its marginally easier (and it would only be marginally easier) to mark a package deprecated by flipping a bit on the site - it's the last thing they will ever do with the package. On 4/5/2016 18:40, Alex Gr?nholm wrote: > Wouldn't my suggestion or Glyph's be easier for the maintainers? That > way they wouldn't even have to make a new release, just modify a > setting on the package settings page on PyPI. > Also, are you going you see the warning if it's emitted on installation? > > 06.04.2016, 01:37, Alexander Walters kirjoitti: >> >> >> On 4/5/2016 18:34, Glyph wrote: >>> Perhaps, before anyone tries to make pip doing something mechanical >>> about deprecations, we should just have the website itself do this >>> sort of redirect? Removing the download would be drastic; giving >>> people an interstitial that says "This package is no longer >>> maintained, please use $X instead" would be very informative. >>> >>> -glyph >> >> I don't remember the last time I used the pypi website. I use pypi >> every day. I don't know if I am weird, but redirecting web views >> would do nothing for me. Redirecting packages is pure evil. >> >> I really think the best course of action is for the maintainer to >> release a final version of the package that warns on install and import. >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> https://mail.python.org/mailman/listinfo/distutils-sig > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig From alex.gronholm at nextday.fi Tue Apr 5 18:50:18 2016 From: alex.gronholm at nextday.fi (=?UTF-8?Q?Alex_Gr=c3=b6nholm?=) Date: Wed, 6 Apr 2016 01:50:18 +0300 Subject: [Distutils] How to deprecate a python package In-Reply-To: <5704406C.4070704@sdamon.com> References: <570407FD.7050305@thomas-guettler.de> <57041D73.3010608@nextday.fi> <57043B9E.5020004@nextday.fi> <57043E46.9080007@sdamon.com> <57043EEF.1060901@nextday.fi> <5704406C.4070704@sdamon.com> Message-ID: <5704412A.8050907@nextday.fi> Implementing this on Warehouse and pip would have the added benefit of warning users who have a specific version pinned. As for pip letting stderr messages through, that'd be irrelevant if pip had direct support for this. 06.04.2016, 01:47, Alexander Walters kirjoitti: > I am not 100% sure if pip will let stderr messages though, but i THINK > it does. Warnings on import will work regardless. > > Honestly, I don't care if its marginally easier (and it would only be > marginally easier) to mark a package deprecated by flipping a bit on > the site - it's the last thing they will ever do with the package. > > On 4/5/2016 18:40, Alex Gr?nholm wrote: >> Wouldn't my suggestion or Glyph's be easier for the maintainers? That >> way they wouldn't even have to make a new release, just modify a >> setting on the package settings page on PyPI. >> Also, are you going you see the warning if it's emitted on installation? >> >> 06.04.2016, 01:37, Alexander Walters kirjoitti: >>> >>> >>> On 4/5/2016 18:34, Glyph wrote: >>>> Perhaps, before anyone tries to make pip doing something mechanical >>>> about deprecations, we should just have the website itself do this >>>> sort of redirect? Removing the download would be drastic; giving >>>> people an interstitial that says "This package is no longer >>>> maintained, please use $X instead" would be very informative. >>>> >>>> -glyph >>> >>> I don't remember the last time I used the pypi website. I use pypi >>> every day. I don't know if I am weird, but redirecting web views >>> would do nothing for me. Redirecting packages is pure evil. >>> >>> I really think the best course of action is for the maintainer to >>> release a final version of the package that warns on install and >>> import. >>> _______________________________________________ >>> Distutils-SIG maillist - Distutils-SIG at python.org >>> https://mail.python.org/mailman/listinfo/distutils-sig >> >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> https://mail.python.org/mailman/listinfo/distutils-sig > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig From tritium-list at sdamon.com Tue Apr 5 18:52:38 2016 From: tritium-list at sdamon.com (Alexander Walters) Date: Tue, 5 Apr 2016 18:52:38 -0400 Subject: [Distutils] How to deprecate a python package In-Reply-To: <5704412A.8050907@nextday.fi> References: <570407FD.7050305@thomas-guettler.de> <57041D73.3010608@nextday.fi> <57043B9E.5020004@nextday.fi> <57043E46.9080007@sdamon.com> <57043EEF.1060901@nextday.fi> <5704406C.4070704@sdamon.com> <5704412A.8050907@nextday.fi> Message-ID: <570441B6.6060708@sdamon.com> Please keep in mind, my suggestion can be done *today* with zero changes to tooling. On 4/5/2016 18:50, Alex Gr?nholm wrote: > Implementing this on Warehouse and pip would have the added benefit of > warning users who have a specific version pinned. As for pip letting > stderr messages through, that'd be irrelevant if pip had direct > support for this. From greg.ewing at canterbury.ac.nz Tue Apr 5 19:32:13 2016 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 06 Apr 2016 11:32:13 +1200 Subject: [Distutils] How to deprecate a python package In-Reply-To: References: <570407FD.7050305@thomas-guettler.de> Message-ID: <57044AFD.7060609@canterbury.ac.nz> Geoffrey Spear wrote: > I don't have an answer to your actual question, but I'd not advocate > people removing packages; we don't want a npm situation here. :( Perhaps there should be a way of marking a package as deprecated on PyPI, so that it shows a big red warning flag? -- Greg From alex.gronholm at nextday.fi Tue Apr 5 19:36:36 2016 From: alex.gronholm at nextday.fi (=?UTF-8?Q?Alex_Gr=c3=b6nholm?=) Date: Wed, 6 Apr 2016 02:36:36 +0300 Subject: [Distutils] How to deprecate a python package In-Reply-To: <57044AFD.7060609@canterbury.ac.nz> References: <570407FD.7050305@thomas-guettler.de> <57044AFD.7060609@canterbury.ac.nz> Message-ID: <57044C04.2060708@nextday.fi> You can already modify the description and add the "Development Status :: 7 - Inactive" classifier. It would be really nice to filter these out of search results though. 06.04.2016, 02:32, Greg Ewing kirjoitti: > Geoffrey Spear wrote: >> I don't have an answer to your actual question, but I'd not advocate >> people removing packages; we don't want a npm situation here. :( > > Perhaps there should be a way of marking a package as > deprecated on PyPI, so that it shows a big red warning > flag? > From nicholas.chammas at gmail.com Wed Apr 6 10:14:41 2016 From: nicholas.chammas at gmail.com (Nicholas Chammas) Date: Wed, 06 Apr 2016 14:14:41 +0000 Subject: [Distutils] How to deprecate a python package In-Reply-To: <57044C04.2060708@nextday.fi> References: <570407FD.7050305@thomas-guettler.de> <57044AFD.7060609@canterbury.ac.nz> <57044C04.2060708@nextday.fi> Message-ID: FYI, there is an existing issue on Warehouse's tracker for this: https://github.com/pypa/warehouse/issues/345 -------------- next part -------------- An HTML attachment was scrubbed... URL: From tymoteusz.jankowski at gmail.com Sat Apr 9 05:02:24 2016 From: tymoteusz.jankowski at gmail.com (Tymoteusz Jankowski) Date: Sat, 09 Apr 2016 09:02:24 +0000 Subject: [Distutils] [Python-ideas] Version control system link in package meta data In-Reply-To: References: <56E3F313.7060908@sdamon.com> Message-ID: Here's a draft of the feature. Any feedback is appreciated. On Sun, Mar 13, 2016 at 10:52 AM Tymoteusz Jankowski < tymoteusz.jankowski at gmail.com> wrote: > Thanks for information. I think that's enough to me. I'm gonna check all > URLs in package data and if anyone match to VCS i clone it. If not fallback > to regular package installation. > > On Sat, Mar 12, 2016 at 3:46 PM Nick Coghlan wrote: > >> On 12 March 2016 at 20:44, Alexander Walters >> wrote: >> > I agree, it would be nice if everyone used git (or any of a small set of >> > VCS), and all the packages on pypi listed their repositories in the >> > metadata. If that were the case, this tool might already exist. In the >> > current state of things, though, i don't think it makes much sense to >> > produce a general purpose tool for this. >> >> We don't place any particular requirements on the development >> practices of projects publishing their releases through PyPI, so >> there's no requirement for a public VCS URL to even exist for a >> project, let alone for it to be mentioned in the project metadata. >> >> That said, since project URLs do make it possible for projects to >> share that metadata if they want to, this is a situation where a >> "checkout-pypi-project" that gained popularity might provide more >> incentive for maintainers to provide that metadata and keep it up to >> date. As a fallback for projects without that metadata, searching >> popular hosting sites like GitHub, BitBucket, GitLab and even >> SourceForge, would provide some initial links to investigate. >> >> Cheers, >> Nick. >> >> -- >> Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Sat Apr 9 10:04:42 2016 From: wes.turner at gmail.com (Wes Turner) Date: Sat, 9 Apr 2016 09:04:42 -0500 Subject: [Distutils] [Python-ideas] Version control system link in package meta data In-Reply-To: References: <56E3F313.7060908@sdamon.com> Message-ID: * JSONLD / RDFa would be easier to parse, someday: https://github.com/pypa/interoperability-peps/issues/31 #PEP426JSONLD Warehouse had placeholders for e.g. extra metadata like source URLs. * https://github.com/pypa/warehouse * https://warehouse.python.org/project/pip/ cases where we can try to 'infer' a source URL: * 1. when specified as an extra package metadata attribute * how to specify (version, VCS_Rev) pairs? * semver 3.0.0+buildVCSREV ? * https://warehouse.python.org/project/semantic_version/ * 2. when homepage.domain in [github.com, bitbucket.org] * sphinxcontrib-srclinks has sane defaults and expansions: * | Warehouse: https://warehouse.python.org/project/sphinxcontrib-srclinks/ * | PyPI: https://pypi.python.org/pypi/sphinxcontrib-srclinks/ * | Src: https://github.com/westurner/sphinxcontrib-srclinks/blob/master/sphinxcontrib/srclinks/__init__.py * | Src: git https://bitbucket.org/westurner/sphinxcontrib-srclinks * git+ssh://git at github.com/pypa/pip at tag#egg=pip sure would be nice to specify a Src URL as a ## comment for each Requirement string block e.g. in requirements.txt; once: * https://github.com/westurner/pyleset/blob/release/0.1.0/setup.py * class Requirement * class RequirementsMap * def generate_packages_txt * def generate_requirements_txt ... I would be more than happy to relicense/contribute any of this with my name on it here On Apr 9, 2016 4:04 AM, "Tymoteusz Jankowski" wrote: > Here's a draft of the feature. > Any feedback is appreciated. > > On Sun, Mar 13, 2016 at 10:52 AM Tymoteusz Jankowski < > tymoteusz.jankowski at gmail.com> wrote: > >> Thanks for information. I think that's enough to me. I'm gonna check all >> URLs in package data and if anyone match to VCS i clone it. If not fallback >> to regular package installation. >> >> On Sat, Mar 12, 2016 at 3:46 PM Nick Coghlan wrote: >> >>> On 12 March 2016 at 20:44, Alexander Walters >>> wrote: >>> > I agree, it would be nice if everyone used git (or any of a small set >>> of >>> > VCS), and all the packages on pypi listed their repositories in the >>> > metadata. If that were the case, this tool might already exist. In >>> the >>> > current state of things, though, i don't think it makes much sense to >>> > produce a general purpose tool for this. >>> >>> We don't place any particular requirements on the development >>> practices of projects publishing their releases through PyPI, so >>> there's no requirement for a public VCS URL to even exist for a >>> project, let alone for it to be mentioned in the project metadata. >>> >>> That said, since project URLs do make it possible for projects to >>> share that metadata if they want to, this is a situation where a >>> "checkout-pypi-project" that gained popularity might provide more >>> incentive for maintainers to provide that metadata and keep it up to >>> date. As a fallback for projects without that metadata, searching >>> popular hosting sites like GitHub, BitBucket, GitLab and even >>> SourceForge, would provide some initial links to investigate. >>> >>> Cheers, >>> Nick. >>> >>> -- >>> Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia >>> >> > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tymoteusz.jankowski at gmail.com Sun Apr 10 06:15:59 2016 From: tymoteusz.jankowski at gmail.com (Tymoteusz Jankowski) Date: Sun, 10 Apr 2016 10:15:59 +0000 Subject: [Distutils] [Python-ideas] Version control system link in package meta data In-Reply-To: References: <56E3F313.7060908@sdamon.com> Message-ID: Thanks for hints and solutions Wes. However I'm suspending it, becuase of this . On Sat, Apr 9, 2016 at 4:04 PM Wes Turner wrote: > * JSONLD / RDFa would be easier to parse, someday: > https://github.com/pypa/interoperability-peps/issues/31 #PEP426JSONLD > > Warehouse had placeholders for e.g. extra metadata like source URLs. > > * https://github.com/pypa/warehouse > * https://warehouse.python.org/project/pip/ > > cases where we can try to 'infer' a source URL: > > * 1. when specified as an extra package metadata attribute > * how to specify (version, VCS_Rev) pairs? > * semver 3.0.0+buildVCSREV ? > * https://warehouse.python.org/project/semantic_version/ > > * 2. when homepage.domain in [github.com, bitbucket.org] > > * sphinxcontrib-srclinks has sane defaults and expansions: > * | Warehouse: > https://warehouse.python.org/project/sphinxcontrib-srclinks/ > * | PyPI: https://pypi.python.org/pypi/sphinxcontrib-srclinks/ > * | Src: > https://github.com/westurner/sphinxcontrib-srclinks/blob/master/sphinxcontrib/srclinks/__init__.py > * | Src: git https://bitbucket.org/westurner/sphinxcontrib-srclinks > > * git+ssh://git at github.com/pypa/pip at tag#egg=pip > > sure would be nice to specify a Src URL as a ## comment for each > Requirement string block e.g. in requirements.txt; once: > > * https://github.com/westurner/pyleset/blob/release/0.1.0/setup.py > * class Requirement > * class RequirementsMap > * def generate_packages_txt > * def generate_requirements_txt > > ... I would be more than happy to relicense/contribute any of this with my > name on it here > On Apr 9, 2016 4:04 AM, "Tymoteusz Jankowski" < > tymoteusz.jankowski at gmail.com> wrote: > >> Here's a draft of the feature. >> Any feedback is appreciated. >> >> On Sun, Mar 13, 2016 at 10:52 AM Tymoteusz Jankowski < >> tymoteusz.jankowski at gmail.com> wrote: >> >>> Thanks for information. I think that's enough to me. I'm gonna check all >>> URLs in package data and if anyone match to VCS i clone it. If not fallback >>> to regular package installation. >>> >>> On Sat, Mar 12, 2016 at 3:46 PM Nick Coghlan wrote: >>> >>>> On 12 March 2016 at 20:44, Alexander Walters >>>> wrote: >>>> > I agree, it would be nice if everyone used git (or any of a small set >>>> of >>>> > VCS), and all the packages on pypi listed their repositories in the >>>> > metadata. If that were the case, this tool might already exist. In >>>> the >>>> > current state of things, though, i don't think it makes much sense to >>>> > produce a general purpose tool for this. >>>> >>>> We don't place any particular requirements on the development >>>> practices of projects publishing their releases through PyPI, so >>>> there's no requirement for a public VCS URL to even exist for a >>>> project, let alone for it to be mentioned in the project metadata. >>>> >>>> That said, since project URLs do make it possible for projects to >>>> share that metadata if they want to, this is a situation where a >>>> "checkout-pypi-project" that gained popularity might provide more >>>> incentive for maintainers to provide that metadata and keep it up to >>>> date. As a fallback for projects without that metadata, searching >>>> popular hosting sites like GitHub, BitBucket, GitLab and even >>>> SourceForge, would provide some initial links to investigate. >>>> >>>> Cheers, >>>> Nick. >>>> >>>> -- >>>> Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia >>>> >>> >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> https://mail.python.org/mailman/listinfo/distutils-sig >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From njs at pobox.com Mon Apr 11 05:07:35 2016 From: njs at pobox.com (Nathaniel Smith) Date: Mon, 11 Apr 2016 02:07:35 -0700 Subject: [Distutils] fyi - wheel-builders mailing list Message-ID: Hi all, Just wanted to give distutils-sig a (somewhat belated, mea culpa) heads up on this new list for non-standards-related discussion around getting wheels built: https://mail.python.org/mailman/listinfo/wheel-builders In particular, I just posted a proposal for shipping native libraries as wheels that may be of interest to some folks here: http://thread.gmane.org/gmane.comp.python.wheel-builders/91 -n -- Nathaniel J. Smith -- https://vorpus.org From guettliml at thomas-guettler.de Mon Apr 11 07:04:55 2016 From: guettliml at thomas-guettler.de (=?UTF-8?Q?Thomas_G=c3=bcttler?=) Date: Mon, 11 Apr 2016 13:04:55 +0200 Subject: [Distutils] For maximum performance, Python packages are best installed as zip files. Message-ID: <570B84D7.1080703@thomas-guettler.de> From https://pythonhosted.org/setuptools/setuptools.html#setting-the-zip-safe-flag > For maximum performance, Python packages are best installed as zip files. What kind of performance improvement is this? Is this improvement really measurable? What improvement numbers do you get? Regards, Thomas G?ttler -- Thomas Guettler http://www.thomas-guettler.de/ From donald at stufft.io Mon Apr 11 07:08:19 2016 From: donald at stufft.io (Donald Stufft) Date: Mon, 11 Apr 2016 07:08:19 -0400 Subject: [Distutils] For maximum performance, Python packages are best installed as zip files. In-Reply-To: <570B84D7.1080703@thomas-guettler.de> References: <570B84D7.1080703@thomas-guettler.de> Message-ID: <639935EB-16E1-4872-87D6-C882B0A4E1AA@stufft.io> > On Apr 11, 2016, at 7:04 AM, Thomas G?ttler wrote: > > From https://pythonhosted.org/setuptools/setuptools.html#setting-the-zip-safe-flag > > > For maximum performance, Python packages are best installed as zip files. > > What kind of performance improvement is this? > > Is this improvement really measurable? > > What improvement numbers do you get? > I?m not sure if that is still the case with modern SSDs, but I think the idea is that by putting everything inside of zip files you reduce the number of stat calls that Python needs to do (they flip side of this is that pkg_resources is incredibly slow because it needs to issue a ton of stat calls on import). ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 842 bytes Desc: Message signed with OpenPGP using GPGMail URL: From p.f.moore at gmail.com Mon Apr 11 07:16:47 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 11 Apr 2016 12:16:47 +0100 Subject: [Distutils] For maximum performance, Python packages are best installed as zip files. In-Reply-To: <639935EB-16E1-4872-87D6-C882B0A4E1AA@stufft.io> References: <570B84D7.1080703@thomas-guettler.de> <639935EB-16E1-4872-87D6-C882B0A4E1AA@stufft.io> Message-ID: On 11 April 2016 at 12:08, Donald Stufft wrote: >> On Apr 11, 2016, at 7:04 AM, Thomas G?ttler wrote: >> >> From https://pythonhosted.org/setuptools/setuptools.html#setting-the-zip-safe-flag >> >> > For maximum performance, Python packages are best installed as zip files. >> >> What kind of performance improvement is this? >> >> Is this improvement really measurable? >> >> What improvement numbers do you get? >> > > I?m not sure if that is still the case with modern SSDs, but I think the idea is that by putting everything inside of zip files you reduce the number of stat calls that Python needs to do (they flip side of this is that pkg_resources is incredibly slow because it needs to issue a ton of stat calls on import). I believe that recent versions of Python also did a lot to improve the number of stat calls done on import. So very definitely YMMV. If the performance difference matters, then you should test carefully. Paul From solipsis at pitrou.net Mon Apr 11 07:23:13 2016 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 11 Apr 2016 13:23:13 +0200 Subject: [Distutils] For maximum performance, Python packages are best installed as zip files. References: <570B84D7.1080703@thomas-guettler.de> <639935EB-16E1-4872-87D6-C882B0A4E1AA@stufft.io> Message-ID: <20160411132313.4966de34@fsol> On Mon, 11 Apr 2016 07:08:19 -0400 Donald Stufft wrote: > > I?m not sure if that is still the case with modern SSDs, but I think the idea is that by putting everything inside of zip files you reduce the number of stat calls that Python needs to do (they flip side of this is that pkg_resources is incredibly slow because it needs to issue a ton of stat calls on import). I don't think SSDs have anything to do in it since the kernel should cache directory contents, rather it's the number of system calls issued. But as Paul says, in Python 3 importlib got a lot of optimization work on this front, so this advice probably doesn't apply anymore. Regards Antoine. From donald at stufft.io Mon Apr 11 07:33:05 2016 From: donald at stufft.io (Donald Stufft) Date: Mon, 11 Apr 2016 07:33:05 -0400 Subject: [Distutils] For maximum performance, Python packages are best installed as zip files. In-Reply-To: <20160411132313.4966de34@fsol> References: <570B84D7.1080703@thomas-guettler.de> <639935EB-16E1-4872-87D6-C882B0A4E1AA@stufft.io> <20160411132313.4966de34@fsol> Message-ID: <2A700196-860D-481E-BA97-2C9D0B6F019D@stufft.io> > On Apr 11, 2016, at 7:23 AM, Antoine Pitrou wrote: > > On Mon, 11 Apr 2016 07:08:19 -0400 > Donald Stufft wrote: >> >> I?m not sure if that is still the case with modern SSDs, but I think the idea is that by putting everything inside of zip files you reduce the number of stat calls that Python needs to do (they flip side of this is that pkg_resources is incredibly slow because it needs to issue a ton of stat calls on import). > > I don't think SSDs have anything to do in it since the kernel should > cache directory contents, rather it's the number of system calls issued. > But as Paul says, in Python 3 importlib got a lot of optimization work > on this front, so this advice probably doesn't apply anymore. > Surely SSDs are faster at returning the data (metadata or actual data) than spinning rust and thus would speed up importing on their own. Sure you might have that data cached in memory if you?ve recently accessed it, but you just as easily might not have that data cached and the OS might need to hit the disk to find out. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 842 bytes Desc: Message signed with OpenPGP using GPGMail URL: From solipsis at pitrou.net Mon Apr 11 07:36:20 2016 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 11 Apr 2016 13:36:20 +0200 Subject: [Distutils] For maximum performance, Python packages are best installed as zip files. In-Reply-To: <2A700196-860D-481E-BA97-2C9D0B6F019D@stufft.io> References: <570B84D7.1080703@thomas-guettler.de> <639935EB-16E1-4872-87D6-C882B0A4E1AA@stufft.io> <20160411132313.4966de34@fsol> <2A700196-860D-481E-BA97-2C9D0B6F019D@stufft.io> Message-ID: <20160411133620.3de5f424@fsol> On Mon, 11 Apr 2016 07:33:05 -0400 Donald Stufft wrote: > > > On Apr 11, 2016, at 7:23 AM, Antoine Pitrou wrote: > > > > On Mon, 11 Apr 2016 07:08:19 -0400 > > Donald Stufft wrote: > >> > >> I?m not sure if that is still the case with modern SSDs, but I think the idea is that by putting everything inside of zip files you reduce the number of stat calls that Python needs to do (they flip side of this is that pkg_resources is incredibly slow because it needs to issue a ton of stat calls on import). > > > > I don't think SSDs have anything to do in it since the kernel should > > cache directory contents, rather it's the number of system calls issued. > > But as Paul says, in Python 3 importlib got a lot of optimization work > > on this front, so this advice probably doesn't apply anymore. > > > > Surely SSDs are faster at returning the data (metadata or actual data) than spinning rust and thus would speed up importing on their own. Sure you might have that data cached in memory if you?ve recently accessed it, but you just as easily might not have that data cached and the OS might need to hit the disk to find out. That doesn't really follow. There is a certain number of disk accesses that is necessary to get the required metadata in a correct way. That number of accesses hasn't changed between Python 2 and Python 3. What has changed is the number of spurious stat() calls asking for information that can be cached. In Python 2, the OS would have done the caching (sparing disk accesses but not system calls). On Python 3, the caching is done inside importlib, which spares system calls in addition to disk accesses. (yes, this is a bit simplified because of NFS and friends ;-)) Regards Antoine. From donald at stufft.io Mon Apr 11 07:40:05 2016 From: donald at stufft.io (Donald Stufft) Date: Mon, 11 Apr 2016 07:40:05 -0400 Subject: [Distutils] For maximum performance, Python packages are best installed as zip files. In-Reply-To: <20160411133620.3de5f424@fsol> References: <570B84D7.1080703@thomas-guettler.de> <639935EB-16E1-4872-87D6-C882B0A4E1AA@stufft.io> <20160411132313.4966de34@fsol> <2A700196-860D-481E-BA97-2C9D0B6F019D@stufft.io> <20160411133620.3de5f424@fsol> Message-ID: <1377C5E9-F968-4C39-8EA4-13B0B86BC7F5@stufft.io> > On Apr 11, 2016, at 7:36 AM, Antoine Pitrou wrote: > > There is a certain number of disk accesses > that is necessary to get the required metadata in a correct way. Right, but a SSD will perform those disk accesses quicker. Obviously doing less of them is better than doing more of them faster, but doing less of them faster seems to me to be better than doing less of them slower? ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 842 bytes Desc: Message signed with OpenPGP using GPGMail URL: From solipsis at pitrou.net Mon Apr 11 07:41:47 2016 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 11 Apr 2016 13:41:47 +0200 Subject: [Distutils] For maximum performance, Python packages are best installed as zip files. In-Reply-To: <1377C5E9-F968-4C39-8EA4-13B0B86BC7F5@stufft.io> References: <570B84D7.1080703@thomas-guettler.de> <639935EB-16E1-4872-87D6-C882B0A4E1AA@stufft.io> <20160411132313.4966de34@fsol> <2A700196-860D-481E-BA97-2C9D0B6F019D@stufft.io> <20160411133620.3de5f424@fsol> <1377C5E9-F968-4C39-8EA4-13B0B86BC7F5@stufft.io> Message-ID: <20160411134147.6d23b97c@fsol> On Mon, 11 Apr 2016 07:40:05 -0400 Donald Stufft wrote: > > > On Apr 11, 2016, at 7:36 AM, Antoine Pitrou wrote: > > > > There is a certain number of disk accesses > > that is necessary to get the required metadata in a correct way. > > Right, but a SSD will perform those disk accesses quicker. Obviously doing less of them is better than doing more of them faster, but doing less of them faster seems to me to be better than doing less of them slower? Yes, that's true. Regards Antoine. From marius at gedmin.as Mon Apr 11 07:53:11 2016 From: marius at gedmin.as (Marius Gedminas) Date: Mon, 11 Apr 2016 14:53:11 +0300 Subject: [Distutils] For maximum performance, Python packages are best installed as zip files. In-Reply-To: <570B84D7.1080703@thomas-guettler.de> References: <570B84D7.1080703@thomas-guettler.de> Message-ID: <20160411115311.GA8531@platonas> On Mon, Apr 11, 2016 at 01:04:55PM +0200, Thomas G?ttler wrote: > From https://pythonhosted.org/setuptools/setuptools.html#setting-the-zip-safe-flag > > > For maximum performance, Python packages are best installed as zip files. > > What kind of performance improvement is this? > > Is this improvement really measurable? > > What improvement numbers do you get? A long time ago I saw a thread (maybe on this very list) with concrete numbers. I only remember the conclusion: - everything in one zip file (including the Python stdlib) was fastest - everything unzipped was 2nd fastest - multiple zipped eggs were slowest This was probably done in the times of rotational hard disks. It would be interesting to see someone repeat that experiment. Marius Gedminas -- I've never understood the pathological fear some companies seem to have about the idea of their advertisements being seen by too many people. -- Brett Tamahori -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 173 bytes Desc: Digital signature URL: From phil at riverbankcomputing.com Mon Apr 11 04:26:49 2016 From: phil at riverbankcomputing.com (Phil Thompson) Date: Mon, 11 Apr 2016 09:26:49 +0100 Subject: [Distutils] PyPi Size Limits Message-ID: I hope this is the right place to ask this, apologies if not... I want to make my project's wheels available via PyPi. However they exceed the size limits imposed by PyPi. Each release has three wheels (four once Linux wheels are supported) each being about 70M. What are my options? Is there a way to get PyPi to accept larger wheels? Is there a way for PyPi to reference wheels hosted externally? Should I just set up my own public pypi? Thanks, Phil From donald at stufft.io Mon Apr 11 08:27:34 2016 From: donald at stufft.io (Donald Stufft) Date: Mon, 11 Apr 2016 08:27:34 -0400 Subject: [Distutils] PyPi Size Limits In-Reply-To: References: Message-ID: <0FDEBCCF-9484-4BFE-BD71-DD76899EAF59@stufft.io> > On Apr 11, 2016, at 4:26 AM, Phil Thompson wrote: > > I hope this is the right place to ask this, apologies if not... > > I want to make my project's wheels available via PyPi. However they exceed the size limits imposed by PyPi. Each release has three wheels (four once Linux wheels are supported) each being about 70M. > > What are my options? > > Is there a way to get PyPi to accept larger wheels? Sort of. Legacy PyPI doesn?t do it but Warehouse (PyPI 2.0) does. Warehouse is currently in development but is available for people to use right now (with a couple of caveats). Caveats (off the top of my head): * Warehouse isn?t ?production? so it may or may not be working at any given time and no priority is given to trying to keep it running (no monitoring, etc). * Warehouse doesn?t issue PURGE statements to PyPI, so you may need to either wait for caches to timeout or carefully order how you upload (e.g. if sdist is small enough but wheels aren?t, upload Wheels to Warehouse first, then sdist to PyPI). * If it?s broken, it may be broken in a way that corrupts your upload, requiring you to reissue the release. On the other hand, I?d love more people to be using Warehouse and seeing what bugs they hit so we can make Warehouse better before it launches at pypi.python.org. > > Is there a way for PyPi to reference wheels hosted externally? No. > > Should I just set up my own public pypi? That?s an option (and currently the only option if you can?t make your wheels smaller and you?re unhappy with the caveats of using Warehouse atm). > > Thanks, > Phil > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 842 bytes Desc: Message signed with OpenPGP using GPGMail URL: From contact at ionelmc.ro Mon Apr 11 08:46:16 2016 From: contact at ionelmc.ro (=?UTF-8?Q?Ionel_Cristian_M=C4=83rie=C8=99?=) Date: Mon, 11 Apr 2016 15:46:16 +0300 Subject: [Distutils] For maximum performance, Python packages are best installed as zip files. In-Reply-To: <20160411115311.GA8531@platonas> References: <570B84D7.1080703@thomas-guettler.de> <20160411115311.GA8531@platonas> Message-ID: On Mon, Apr 11, 2016 at 2:53 PM, Marius Gedminas wrote: > - multiple zipped eggs were slowest ?Afaik importing became slower and slower as you had more more eggs around. Haven't investigated what actually was the cause as I don't use eggs anymore but the idea that zips are good for performance is wrong - you need to look at a lot of zips (and unpack) to see what's importable. The zip importing caches worse than stating lots of directories, since it needs to cache file content while unzipped only needs metadata - correct me if I'm wrong. -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Mon Apr 11 09:00:41 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 11 Apr 2016 14:00:41 +0100 Subject: [Distutils] For maximum performance, Python packages are best installed as zip files. In-Reply-To: References: <570B84D7.1080703@thomas-guettler.de> <20160411115311.GA8531@platonas> Message-ID: On 11 April 2016 at 13:46, Ionel Cristian M?rie? wrote: > On Mon, Apr 11, 2016 at 2:53 PM, Marius Gedminas wrote: >> >> - multiple zipped eggs were slowest > > > Afaik importing became slower and slower as you had more more eggs around. > Haven't investigated what actually was the cause as I don't use eggs anymore > but the idea that zips are good for performance is wrong - you need to look > at a lot of zips (and unpack) to see what's importable. The zip importing > caches worse than stating lots of directories, since it needs to cache file > content while unzipped only needs metadata - correct me if I'm wrong. I believe the issue with eggs is around the fact that they encourage large numbers of items on sys.path (each egg gets its own sys.path entry). That's slow, as there are lots of locations to be scanned for each new import. TBH, though, I consider this a misuse of sys.path - I see sys.path as analogous to $PATH, and you wouldn't typically install each application in its own directory and add that directory to $PATH.], so why do the equivalent with sys.path? Paul From contact at ionelmc.ro Mon Apr 11 09:30:15 2016 From: contact at ionelmc.ro (=?UTF-8?Q?Ionel_Cristian_M=C4=83rie=C8=99?=) Date: Mon, 11 Apr 2016 16:30:15 +0300 Subject: [Distutils] For maximum performance, Python packages are best installed as zip files. In-Reply-To: References: <570B84D7.1080703@thomas-guettler.de> <20160411115311.GA8531@platonas> Message-ID: On Mon, Apr 11, 2016 at 4:00 PM, Paul Moore wrote: > you wouldn't typically install > each application in its own directory and add that directory to > $PATH.], so why do the equivalent with sys.path? > ?Funny that Nixos and Redhat's SCLs? do exactly that with $PATH other vars:-) Thanks, -- Ionel Cristian M?rie?, http://blog.ionelmc.ro -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Mon Apr 11 09:41:19 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 11 Apr 2016 23:41:19 +1000 Subject: [Distutils] For maximum performance, Python packages are best installed as zip files. In-Reply-To: References: <570B84D7.1080703@thomas-guettler.de> <20160411115311.GA8531@platonas> Message-ID: On 11 April 2016 at 23:30, Ionel Cristian M?rie? wrote: > > On Mon, Apr 11, 2016 at 4:00 PM, Paul Moore wrote: > >> you wouldn't typically install >> each application in its own directory and add that directory to >> $PATH.], so why do the equivalent with sys.path? >> > > ?Funny that Nixos and Redhat's SCLs? do exactly that with $PATH other > vars:-) > For SCLs, we don't really expect you to have dozens active at once, though - usually only 1 or 2 for any given process (language runtime + web server). In terms of this thread, though, the advice in the setuptools docs is indeed out of date - the downside of making sys.path longer overwhelms the benefit of zipimporter loading content listings into memory, especially in Python 3, which also caches normal directory listings for sys.path entries since the switch to importlib in 3.3 (the caching avoided a slowdown when importing from local disks, and provided a dramatic speed *up* when importing from network file shares). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From guettliml at thomas-guettler.de Mon Apr 11 09:46:22 2016 From: guettliml at thomas-guettler.de (=?UTF-8?Q?Thomas_G=c3=bcttler?=) Date: Mon, 11 Apr 2016 15:46:22 +0200 Subject: [Distutils] For maximum performance, ... docs out of date In-Reply-To: References: <570B84D7.1080703@thomas-guettler.de> <20160411115311.GA8531@platonas> Message-ID: <570BAAAE.8000309@thomas-guettler.de> Am 11.04.2016 um 15:41 schrieb Nick Coghlan: > On 11 April 2016 at 23:30, Ionel Cristian M?rie? > wrote: > > > On Mon, Apr 11, 2016 at 4:00 PM, Paul Moore > wrote: > > you wouldn't typically install > each application in its own directory and add that directory to > $PATH.], so why do the equivalent with sys.path? > > > ?Funny that Nixos and Redhat's SCLs? do exactly that with $PATH other vars:-) > > > For SCLs, we don't really expect you to have dozens active at once, though - usually only 1 or 2 for any given process > (language runtime + web server). > > In terms of this thread, though, the advice in the setuptools docs is indeed out of date The docs are out of date: What is the next step? Just remove this sentence from the docs? Here is the current original: https://github.com/pypa/setuptools/blob/master/docs/setuptools.txt#L1273 Regards, Thomas G?ttler -- Thomas Guettler http://www.thomas-guettler.de/ From p.f.moore at gmail.com Mon Apr 11 10:07:50 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 11 Apr 2016 15:07:50 +0100 Subject: [Distutils] For maximum performance, ... docs out of date In-Reply-To: <570BAAAE.8000309@thomas-guettler.de> References: <570B84D7.1080703@thomas-guettler.de> <20160411115311.GA8531@platonas> <570BAAAE.8000309@thomas-guettler.de> Message-ID: On 11 April 2016 at 14:46, Thomas G?ttler wrote: > Am 11.04.2016 um 15:41 schrieb Nick Coghlan: >> >> On 11 April 2016 at 23:30, Ionel Cristian M?rie? > > wrote: >> >> >> On Mon, Apr 11, 2016 at 4:00 PM, Paul Moore > > wrote: >> >> you wouldn't typically install >> each application in its own directory and add that directory to >> $PATH.], so why do the equivalent with sys.path? >> >> >> Funny that Nixos and Redhat's SCLs do exactly that with $PATH other >> vars:-) >> >> >> For SCLs, we don't really expect you to have dozens active at once, though >> - usually only 1 or 2 for any given process >> (language runtime + web server). >> >> In terms of this thread, though, the advice in the setuptools docs is >> indeed out of date > > > The docs are out of date: What is the next step? > > Just remove this sentence from the docs? > > Here is the current original: > > https://github.com/pypa/setuptools/blob/master/docs/setuptools.txt#L1273 I guess a PR against the setuptools docs to suggest updated wording would be a good start. Paul From ncoghlan at gmail.com Mon Apr 11 10:11:33 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 12 Apr 2016 00:11:33 +1000 Subject: [Distutils] For maximum performance, ... docs out of date In-Reply-To: References: <570B84D7.1080703@thomas-guettler.de> <20160411115311.GA8531@platonas> <570BAAAE.8000309@thomas-guettler.de> Message-ID: On 12 April 2016 at 00:07, Paul Moore wrote: > On 11 April 2016 at 14:46, Thomas G?ttler wrote: >> The docs are out of date: What is the next step? >> >> Just remove this sentence from the docs? >> >> Here is the current original: >> >> https://github.com/pypa/setuptools/blob/master/docs/setuptools.txt#L1273 > > I guess a PR against the setuptools docs to suggest updated wording > would be a good start. Sent: https://github.com/pypa/setuptools/pull/542 Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From phil at riverbankcomputing.com Mon Apr 11 13:48:17 2016 From: phil at riverbankcomputing.com (Phil Thompson) Date: Mon, 11 Apr 2016 18:48:17 +0100 Subject: [Distutils] PyPi Size Limits In-Reply-To: <0FDEBCCF-9484-4BFE-BD71-DD76899EAF59@stufft.io> References: <0FDEBCCF-9484-4BFE-BD71-DD76899EAF59@stufft.io> Message-ID: <496E8CE4-3429-4690-8178-F2DE2A39F54F@riverbankcomputing.com> On 11 Apr 2016, at 1:27 pm, Donald Stufft wrote: > > >> On Apr 11, 2016, at 4:26 AM, Phil Thompson wrote: >> >> I hope this is the right place to ask this, apologies if not... >> >> I want to make my project's wheels available via PyPi. However they exceed the size limits imposed by PyPi. Each release has three wheels (four once Linux wheels are supported) each being about 70M. >> >> What are my options? >> >> Is there a way to get PyPi to accept larger wheels? > > Sort of. Legacy PyPI doesn?t do it but Warehouse (PyPI 2.0) does. Warehouse is currently in development but is available for people to use right now (with a couple of caveats). > > Caveats (off the top of my head): > > * Warehouse isn?t ?production? so it may or may not be working at any given time and no priority is given to trying to keep it running (no monitoring, etc). > * Warehouse doesn?t issue PURGE statements to PyPI, so you may need to either wait for caches to timeout or carefully order how you upload (e.g. if sdist is small enough but wheels aren?t, upload Wheels to Warehouse first, then sdist to PyPI). > * If it?s broken, it may be broken in a way that corrupts your upload, requiring you to reissue the release. > > On the other hand, I?d love more people to be using Warehouse and seeing what bugs they hit so we can make Warehouse better before it launches at pypi.python.org. I'm happy to give Warehouse a go. How do I go about it? Phil From chris.barker at noaa.gov Mon Apr 11 18:05:07 2016 From: chris.barker at noaa.gov (Chris Barker) Date: Mon, 11 Apr 2016 15:05:07 -0700 Subject: [Distutils] For maximum performance, Python packages are best installed as zip files. In-Reply-To: References: <570B84D7.1080703@thomas-guettler.de> <20160411115311.GA8531@platonas> Message-ID: one other note: on systems with aggressive virus scanning, python startup can ber REALLY slow. _maybe_ zipped packages would help with that. (or maybe not -- I haven't tested) -CHB On Mon, Apr 11, 2016 at 6:41 AM, Nick Coghlan wrote: > On 11 April 2016 at 23:30, Ionel Cristian M?rie? > wrote: > >> >> On Mon, Apr 11, 2016 at 4:00 PM, Paul Moore wrote: >> >>> you wouldn't typically install >>> each application in its own directory and add that directory to >>> $PATH.], so why do the equivalent with sys.path? >>> >> >> ?Funny that Nixos and Redhat's SCLs? do exactly that with $PATH other >> vars:-) >> > > For SCLs, we don't really expect you to have dozens active at once, though > - usually only 1 or 2 for any given process (language runtime + web server). > > In terms of this thread, though, the advice in the setuptools docs is > indeed out of date - the downside of making sys.path longer overwhelms the > benefit of zipimporter loading content listings into memory, especially in > Python 3, which also caches normal directory listings for sys.path entries > since the switch to importlib in 3.3 (the caching avoided a slowdown when > importing from local disks, and provided a dramatic speed *up* when > importing from network file shares). > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From luis.de.sousa at protonmail.ch Tue Apr 12 08:24:17 2016 From: luis.de.sousa at protonmail.ch (=?UTF-8?Q?Lu=C3=AD=C2=ADs_de_Sousa?=) Date: Tue, 12 Apr 2016 08:24:17 -0400 Subject: [Distutils] How to add GDAL as a dependency to a Python package Message-ID: <8jEiYTMeVM6P4F53PRNX50g_AN_q0GLnx3-ThLBNlVifyAdTbzEo8IhUUeoumiO_i5UG-1MDdFlglBxagFG6aw==@protonmail.ch> Dear all, I am trying to package a Python script for PyPI that uses GDAL. I started by including a direct reference in my setup.py: install_requires=['GDAL==1.11.2'], This way the package failed to install in my test virtual environment: extensions/gdal_wrap.cpp:2855:22: fatal error: cpl_port.h: No such file or directory #include "cpl_port.h" ^ compilation terminated. error: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 I then tried it with a reference to pygdal, since it is marked as a virtualenv friendly version: install_requires=['pygdal'], This way the installation finishes without errors (but with the usual load of compilation warnings). However, when then I invoke the script I get this error back: Traceback (most recent call last): File "/home/desouslu/.virtualenvs/test_p3/bin/hasc2gml", line 5, in from pkg_resources import load_entry_point File "/home/desouslu/.virtualenvs/test_p3/lib/python3.4/site-packages/pkg_resources.py", line 2716, in working_set.require(__requires__) File "/home/desouslu/.virtualenvs/test_p3/lib/python3.4/site-packages/pkg_resources.py", line 685, in require needed = self.resolve(parse_requirements(requirements)) File "/home/desouslu/.virtualenvs/test_p3/lib/python3.4/site-packages/pkg_resources.py", line 588, in resolve raise DistributionNotFound(req) pkg_resources.DistributionNotFound: pygdal What is the correct way of setting GDAL as a dependency? Thank you, Lu?s Sent from [ProtonMail](https://protonmail.ch), encrypted email based in Switzerland. -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Tue Apr 12 12:45:35 2016 From: chris.barker at noaa.gov (Chris Barker) Date: Tue, 12 Apr 2016 09:45:35 -0700 Subject: [Distutils] How to add GDAL as a dependency to a Python package In-Reply-To: <8jEiYTMeVM6P4F53PRNX50g_AN_q0GLnx3-ThLBNlVifyAdTbzEo8IhUUeoumiO_i5UG-1MDdFlglBxagFG6aw==@protonmail.ch> References: <8jEiYTMeVM6P4F53PRNX50g_AN_q0GLnx3-ThLBNlVifyAdTbzEo8IhUUeoumiO_i5UG-1MDdFlglBxagFG6aw==@protonmail.ch> Message-ID: setting gdal as a dependency is only going to work if one of these is true: * There is a binary wheel on PyPi for gdal (which is very hard to do, though with teh manylinux effort, maybe there will be some day) * the system on which you are trying to install is all set up to compile gdal -- which is a major effort, and indeed, I'm not sure is possible (I haven't looked) - The GDAL source ships with the python bindings, so the usual thing is for gdal itself and the pyton bindings to all get compiled and installed at once. I"m not srue that anyone has made a setup.py that will do that. This kind of thing is exactly why conda exists -- python packages with complex non-python dependencies are not well handled by pip / pypi -- if I were you, I"d give it a try :-) -CHB On Tue, Apr 12, 2016 at 5:24 AM, Lu??s de Sousa wrote: > Dear all, > I am trying to package a Python script for PyPI that uses GDAL. I started > by including a direct reference in my setup.py: > > install_requires=['GDAL==1.11.2'], > > This way the package failed to install in my test virtual environment: > > extensions/gdal_wrap.cpp:2855:22: fatal error: cpl_port.h: No such file or directory > #include "cpl_port.h" > ^ > compilation terminated. > error: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 > > I then tried it with a reference to pygdal, since it is marked as a > virtualenv friendly version: > > install_requires=['pygdal'], > > This way the installation finishes without errors (but with the usual load > of compilation warnings). However, when then I invoke the script I get this > error back: > > Traceback (most recent call last): > File "/home/desouslu/.virtualenvs/test_p3/bin/hasc2gml", line 5, in > from pkg_resources import load_entry_point > File "/home/desouslu/.virtualenvs/test_p3/lib/python3.4/site-packages/pkg_resources.py", line 2716, in > working_set.require(__requires__) > File "/home/desouslu/.virtualenvs/test_p3/lib/python3.4/site-packages/pkg_resources.py", line 685, in require > needed = self.resolve(parse_requirements(requirements)) > File "/home/desouslu/.virtualenvs/test_p3/lib/python3.4/site-packages/pkg_resources.py", line 588, in resolve > raise DistributionNotFound(req) > pkg_resources.DistributionNotFound: pygdal > > What is the correct way of setting GDAL as a dependency? > Thank you, > > Lu?s > > *Sent from ProtonMail , encrypted email based in > Switzerland.* > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From luis.de.sousa at protonmail.ch Wed Apr 13 03:02:24 2016 From: luis.de.sousa at protonmail.ch (=?UTF-8?Q?Lu=C3=AD=C2=ADs_de_Sousa?=) Date: Wed, 13 Apr 2016 03:02:24 -0400 Subject: [Distutils] How to add GDAL as a dependency to a Python package In-Reply-To: References: <8jEiYTMeVM6P4F53PRNX50g_AN_q0GLnx3-ThLBNlVifyAdTbzEo8IhUUeoumiO_i5UG-1MDdFlglBxagFG6aw==@protonmail.ch> Message-ID: <6xTG73bNItec__n3VDkqn_yUVrhL1B1HPb_MxdwbYLYUgPNvJq1GfEqzHVu7BQTTtY7oLoKKOOqToy-4Ib6fuQ==@protonmail.ch> Thank you for the reply Chris. I just tried to install pygdal directly from PiPY on a stock Ubuntu distribution and it fails [0], even if I supposedly have all the necessary headers installed. I will try easier ways before going with Conda, but thanks for the suggestion in any case. Lu?s [0] http://pastebin.com/gzrimLvB Sent from [ProtonMail](https://protonmail.ch), encrypted email based in Switzerland. -------- Original Message -------- Subject: Re: [Distutils] How to add GDAL as a dependency to a Python package Local Time: April 12, 2016 6:45 PM UTC Time: April 12, 2016 4:45 PM From: chris.barker at noaa.gov To: luis.de.sousa at protonmail.ch CC: distutils-sig at python.org setting gdal as a dependency is only going to work if one of these is true: * There is a binary wheel on PyPi for gdal (which is very hard to do, though with teh manylinux effort, maybe there will be some day) * the system on which you are trying to install is all set up to compile gdal -- which is a major effort, and indeed, I'm not sure is possible (I haven't looked) - The GDAL source ships with the python bindings, so the usual thing is for gdal itself and the pyton bindings to all get compiled and installed at once. I"m not srue that anyone has made a setup.py that will do that. This kind of thing is exactly why conda exists -- python packages with complex non-python dependencies are not well handled by pip / pypi -- if I were you, I"d give it a try :-) -CHB -------------- next part -------------- An HTML attachment was scrubbed... URL: From donald at stufft.io Wed Apr 13 07:23:08 2016 From: donald at stufft.io (Donald Stufft) Date: Wed, 13 Apr 2016 07:23:08 -0400 Subject: [Distutils] PyPi Size Limits In-Reply-To: <496E8CE4-3429-4690-8178-F2DE2A39F54F@riverbankcomputing.com> References: <0FDEBCCF-9484-4BFE-BD71-DD76899EAF59@stufft.io> <496E8CE4-3429-4690-8178-F2DE2A39F54F@riverbankcomputing.com> Message-ID: > On Apr 11, 2016, at 1:48 PM, Phil Thompson wrote: > > I'm happy to give Warehouse a go. How do I go about it? In your ~/.pypirc you?ll want something like https://bpaste.net/show/7749b2b903b1 You can then upload to PyPI using ``twine upload [-s] dist/*`` and upload to Warehouse using ``twine upload -r warehouse [-s] dist/*``. You?ll want to make sure you upload something to PyPI itself after you upload something to Warehouse to trigger a purge on the CDN. What?s the name of your project and how large of a limit do you need? ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 842 bytes Desc: Message signed with OpenPGP using GPGMail URL: From chris.barker at noaa.gov Wed Apr 13 14:26:54 2016 From: chris.barker at noaa.gov (Chris Barker) Date: Wed, 13 Apr 2016 11:26:54 -0700 Subject: [Distutils] How to add GDAL as a dependency to a Python package In-Reply-To: <6xTG73bNItec__n3VDkqn_yUVrhL1B1HPb_MxdwbYLYUgPNvJq1GfEqzHVu7BQTTtY7oLoKKOOqToy-4Ib6fuQ==@protonmail.ch> References: <8jEiYTMeVM6P4F53PRNX50g_AN_q0GLnx3-ThLBNlVifyAdTbzEo8IhUUeoumiO_i5UG-1MDdFlglBxagFG6aw==@protonmail.ch> <6xTG73bNItec__n3VDkqn_yUVrhL1B1HPb_MxdwbYLYUgPNvJq1GfEqzHVu7BQTTtY7oLoKKOOqToy-4Ib6fuQ==@protonmail.ch> Message-ID: On Wed, Apr 13, 2016 at 12:02 AM, Lu??s de Sousa < luis.de.sousa at protonmail.ch> wrote: > Thank you for the reply Chris. > > I just tried to install pygdal directly from PiPY on a stock Ubuntu > distribution and it fails [0], even if I supposedly have all the necessary > headers installed. > exactly -- not a pip issue :-( are there not Ubuntu packages for GDAL? -CHB I will try easier ways before going with Conda, but thanks for the > suggestion in any case. > > Lu?s > > [0] http://pastebin.com/gzrimLvB > > *Sent from ProtonMail , encrypted email based in > Switzerland.* > > > -------- Original Message -------- > Subject: Re: [Distutils] How to add GDAL as a dependency to a Python > package > Local Time: April 12, 2016 6:45 PM > UTC Time: April 12, 2016 4:45 PM > From: chris.barker at noaa.gov > To: luis.de.sousa at protonmail.ch > CC: distutils-sig at python.org > > setting gdal as a dependency is only going to work if one of these is true: > > * There is a binary wheel on PyPi for gdal (which is very hard to do, > though with teh manylinux effort, maybe there will be some day) > > * the system on which you are trying to install is all set up to compile > gdal -- which is a major effort, and indeed, I'm not sure is possible (I > haven't looked) > - The GDAL source ships with the python bindings, so the usual thing is > for gdal itself and the pyton bindings to all get compiled and installed at > once. I"m not srue that anyone has made a setup.py that will do that. > > This kind of thing is exactly why conda exists -- python packages with > complex non-python dependencies are not well handled by pip / pypi -- if I > were you, I"d give it a try :-) > > -CHB > > > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Wed Apr 13 14:31:52 2016 From: chris.barker at noaa.gov (Chris Barker) Date: Wed, 13 Apr 2016 11:31:52 -0700 Subject: [Distutils] How to add GDAL as a dependency to a Python package In-Reply-To: <6xTG73bNItec__n3VDkqn_yUVrhL1B1HPb_MxdwbYLYUgPNvJq1GfEqzHVu7BQTTtY7oLoKKOOqToy-4Ib6fuQ==@protonmail.ch> References: <8jEiYTMeVM6P4F53PRNX50g_AN_q0GLnx3-ThLBNlVifyAdTbzEo8IhUUeoumiO_i5UG-1MDdFlglBxagFG6aw==@protonmail.ch> <6xTG73bNItec__n3VDkqn_yUVrhL1B1HPb_MxdwbYLYUgPNvJq1GfEqzHVu7BQTTtY7oLoKKOOqToy-4Ib6fuQ==@protonmail.ch> Message-ID: On Wed, Apr 13, 2016 at 12:02 AM, Lu??s de Sousa < luis.de.sousa at protonmail.ch> wrote: > Thank you for the reply Chris. > > I just tried to install pygdal directly from PiPY on a stock Ubuntu > distribution and it fails [0], even if I supposedly have all the necessary > headers installed. > I took a quick look, and it looks like the buld/install failed on numpy -- numpy is a dependency of gdal. so you may want to get numpy installed first. (that surprises me -- I thought numpy built out of the box pretty easily -- if not with an optimized BLAS) and it looks like it did build, but somehow failed to install.... anyway, this is all why Anaconda and Canopy exist... -CHB > I will try easier ways before going with Conda, but thanks for the > suggestion in any case. > > Lu?s > > [0] http://pastebin.com/gzrimLvB > > *Sent from ProtonMail , encrypted email based in > Switzerland.* > > > -------- Original Message -------- > Subject: Re: [Distutils] How to add GDAL as a dependency to a Python > package > Local Time: April 12, 2016 6:45 PM > UTC Time: April 12, 2016 4:45 PM > From: chris.barker at noaa.gov > To: luis.de.sousa at protonmail.ch > CC: distutils-sig at python.org > > setting gdal as a dependency is only going to work if one of these is true: > > * There is a binary wheel on PyPi for gdal (which is very hard to do, > though with teh manylinux effort, maybe there will be some day) > > * the system on which you are trying to install is all set up to compile > gdal -- which is a major effort, and indeed, I'm not sure is possible (I > haven't looked) > - The GDAL source ships with the python bindings, so the usual thing is > for gdal itself and the pyton bindings to all get compiled and installed at > once. I"m not srue that anyone has made a setup.py that will do that. > > This kind of thing is exactly why conda exists -- python packages with > complex non-python dependencies are not well handled by pip / pypi -- if I > were you, I"d give it a try :-) > > -CHB > > > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From njs at pobox.com Wed Apr 13 16:33:46 2016 From: njs at pobox.com (Nathaniel Smith) Date: Wed, 13 Apr 2016 13:33:46 -0700 Subject: [Distutils] How to add GDAL as a dependency to a Python package In-Reply-To: References: <8jEiYTMeVM6P4F53PRNX50g_AN_q0GLnx3-ThLBNlVifyAdTbzEo8IhUUeoumiO_i5UG-1MDdFlglBxagFG6aw==@protonmail.ch> <6xTG73bNItec__n3VDkqn_yUVrhL1B1HPb_MxdwbYLYUgPNvJq1GfEqzHVu7BQTTtY7oLoKKOOqToy-4Ib6fuQ==@protonmail.ch> Message-ID: On Wed, Apr 13, 2016 at 11:31 AM, Chris Barker wrote: > > On Wed, Apr 13, 2016 at 12:02 AM, Lu??s de Sousa wrote: >> >> Thank you for the reply Chris. >> >> I just tried to install pygdal directly from PiPY on a stock Ubuntu distribution and it fails [0], even if I supposedly have all the necessary headers installed. > > > I took a quick look, and it looks like the buld/install failed on numpy -- numpy is a dependency of gdal. so you may want to get numpy installed first. Yeah, due to various infelicities in the distutils stack, automated dependency installation when building stuff is pretty wonky, and a lot of things need numpy to be installed before you try building them. So doing 'pip install numpy' first is a good idea before trying to install any of the rest of the numerical stack. (Part of the motivation for PEP 516/517 is to fix this.) > (that surprises me -- I thought numpy built out of the box pretty easily -- if not with an optimized BLAS) and it looks like it did build, but somehow failed to install.... Thanks to Matthew Brett's tireless efforts (hat tip to Zhang Xianyi for fixing a bunch of bugs in OpenBLAS just for us), there are now[1] numpy linux wheels with optimized BLAS up on PyPI, so if you have pip >= 8.1.0, then 'pip install numpy' should Just Work on linux without needing a compiler at all (and likewise on windows and osx). -n [1] now = "since ~1 hour ago" ;-) -- Nathaniel J. Smith -- https://vorpus.org From p.f.moore at gmail.com Wed Apr 13 17:47:54 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 13 Apr 2016 22:47:54 +0100 Subject: [Distutils] How to add GDAL as a dependency to a Python package In-Reply-To: References: <8jEiYTMeVM6P4F53PRNX50g_AN_q0GLnx3-ThLBNlVifyAdTbzEo8IhUUeoumiO_i5UG-1MDdFlglBxagFG6aw==@protonmail.ch> <6xTG73bNItec__n3VDkqn_yUVrhL1B1HPb_MxdwbYLYUgPNvJq1GfEqzHVu7BQTTtY7oLoKKOOqToy-4Ib6fuQ==@protonmail.ch> Message-ID: On 13 April 2016 at 21:33, Nathaniel Smith wrote: > Thanks to Matthew Brett's tireless efforts (hat tip to Zhang Xianyi > for fixing a bunch of bugs in OpenBLAS just for us), there are now[1] > numpy linux wheels with optimized BLAS up on PyPI, so if you have pip >>= 8.1.0, then 'pip install numpy' should Just Work on linux without > needing a compiler at all (and likewise on windows and osx). W00t. That's really good news :-) Thanks to all who worked on making that possible. Paul From ncoghlan at gmail.com Wed Apr 13 23:30:57 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 14 Apr 2016 13:30:57 +1000 Subject: [Distutils] How to add GDAL as a dependency to a Python package In-Reply-To: References: <8jEiYTMeVM6P4F53PRNX50g_AN_q0GLnx3-ThLBNlVifyAdTbzEo8IhUUeoumiO_i5UG-1MDdFlglBxagFG6aw==@protonmail.ch> <6xTG73bNItec__n3VDkqn_yUVrhL1B1HPb_MxdwbYLYUgPNvJq1GfEqzHVu7BQTTtY7oLoKKOOqToy-4Ib6fuQ==@protonmail.ch> Message-ID: On 14 April 2016 at 06:33, Nathaniel Smith wrote: > Thanks to Matthew Brett's tireless efforts (hat tip to Zhang Xianyi > for fixing a bunch of bugs in OpenBLAS just for us), there are now[1] > numpy linux wheels with optimized BLAS up on PyPI, so if you have pip >>= 8.1.0, then 'pip install numpy' should Just Work on linux without > needing a compiler at all (and likewise on windows and osx). Woohoo! Congrats to all involved in making that possible :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From luis.de.sousa at protonmail.ch Thu Apr 14 03:19:43 2016 From: luis.de.sousa at protonmail.ch (=?UTF-8?Q?Lu=C3=AD=C2=ADs_de_Sousa?=) Date: Thu, 14 Apr 2016 03:19:43 -0400 Subject: [Distutils] How to add GDAL as a dependency to a Python package In-Reply-To: References: <8jEiYTMeVM6P4F53PRNX50g_AN_q0GLnx3-ThLBNlVifyAdTbzEo8IhUUeoumiO_i5UG-1MDdFlglBxagFG6aw==@protonmail.ch> <6xTG73bNItec__n3VDkqn_yUVrhL1B1HPb_MxdwbYLYUgPNvJq1GfEqzHVu7BQTTtY7oLoKKOOqToy-4Ib6fuQ==@protonmail.ch> Message-ID: Thank you all for the replies. I have tried various tests on Ubuntu, and even installing numpy beforehand, apt has to come into play at some point (as has sudo). I guess it is simpler to leave dependencies up to the user. And perhaps provide a package hooked up to python-dev and python-gdal in Debian based distros. Regards, Lu?s -------------- next part -------------- An HTML attachment was scrubbed... URL: From guettliml at thomas-guettler.de Thu Apr 14 03:20:02 2016 From: guettliml at thomas-guettler.de (=?UTF-8?Q?Thomas_G=c3=bcttler?=) Date: Thu, 14 Apr 2016 09:20:02 +0200 Subject: [Distutils] pip install -e foo (without Repo URL) Message-ID: <570F44A2.3080902@thomas-guettler.de> I think it would be very cool if you could install a package editable without the repo-url. The default repo-url can be defined in the meta-data of the package. Background: I came across this becaus saltstack prefers the branch "develop", but most other repos use the branch "master". Yes, this is no big problem. I figured out the right repo url soon. Next use case: you use software third-party-foo-lib in your project. Up to now you use it as package. You find a bug and want to fix it. Wouldn't it be great if you could just type "pip install -e third-party-foo-lib" and now you are read to create pull requests? What do you think? Regards, Thomas G?ttler -- Thomas Guettler http://www.thomas-guettler.de/ From graffatcolmingov at gmail.com Thu Apr 14 08:31:58 2016 From: graffatcolmingov at gmail.com (Ian Cordasco) Date: Thu, 14 Apr 2016 07:31:58 -0500 Subject: [Distutils] pip install -e foo (without Repo URL) In-Reply-To: <570F44A2.3080902@thomas-guettler.de> References: <570F44A2.3080902@thomas-guettler.de> Message-ID: On Apr 14, 2016 2:20 AM, "Thomas G?ttler" wrote: > > I think it would be very cool if you could install a package editable > without the repo-url. > > The default repo-url can be defined in the meta-data of the package. > > Background: I came across this becaus saltstack prefers the branch "develop", but > most other repos use the branch "master". > > Yes, this is no big problem. I figured out the right repo url soon. > > Next use case: you use software third-party-foo-lib in your project. > Up to now you use it as package. You find a bug and want to fix it. > Wouldn't it be great if you could just type "pip install -e third-party-foo-lib" > and now you are read to create pull requests? I don't understand how this makes me ready to submit pull requests. Can you explain a little more? > What do you think? > > Regards, > Thomas G?ttler > > > > -- > Thomas Guettler http://www.thomas-guettler.de/ > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From luis.de.sousa at protonmail.ch Thu Apr 14 11:28:09 2016 From: luis.de.sousa at protonmail.ch (=?UTF-8?Q?Lu=C3=AD=C2=ADs_de_Sousa?=) Date: Thu, 14 Apr 2016 11:28:09 -0400 Subject: [Distutils] 'Invalid module name' creating package with setuptools Message-ID: Hi everyone, I have a project with these contents: proj ??? proj ? ??? scriptA.py ? ??? scriptB.py ? ??? __init__.py ??? LICENCE ??? README.md ??? setup.py The setup.py file looks like: #!/usr/bin/python3 # coding=utf8 from setuptools import setup setup( name = "proj", version = "0.2", packages = ['proj'], install_requires=[], entry_points={ 'console_scripts': [ 'scriptA=proj:scriptA', 'scriptB=proj:scriptB' ], }, # metadata for upload to PyPI author = "Lu?s", author_email = "luis at mail.com", description = "Some package", license = "EUPL v1.1", keywords = "pip package", url = "https://some.place.com", # project home page, if any classifiers = [ "Programming Language :: Python", "Programming Language :: Python :: 3", "Development Status :: 4 - Beta", "Environment :: Console", "Intended Audience :: Science/Research", "License :: OSI Approved :: European Union Public Licence 1.1 (EUPL 1.1)", "Operating System :: OS Independent", "Topic :: Scientific/Engineering :: GIS" ], ) When I try to build I get the following error: $ python setup.py bdist_wheel --universal error in proj setup command: ('Invalid module name', 'proj') What am I doing wrong? Thank you, Lu?s Sent from [ProtonMail](https://protonmail.ch), encrypted email based in Switzerland. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.gronholm at nextday.fi Thu Apr 14 11:32:28 2016 From: alex.gronholm at nextday.fi (=?UTF-8?Q?Alex_Gr=c3=b6nholm?=) Date: Thu, 14 Apr 2016 18:32:28 +0300 Subject: [Distutils] 'Invalid module name' creating package with setuptools In-Reply-To: References: Message-ID: <570FB80C.5030504@nextday.fi> That setup.py worked fine here, so I'm guessing this is not the actual setup.py you're running. What is the exact value of the "name" argument in your real setup.py? BTW, source encoding is utf-8 by default on Python 3 so the "# coding=utf8" is unnecessary. 14.04.2016, 18:28, Lu??s de Sousa via Distutils-SIG kirjoitti: > Hi everyone, > > I have a project with these contents: > > proj > ??? proj > ? ??? scriptA.py > ? ??? scriptB.py > ? ??? __init__.py > ??? LICENCE > ??? README.md > ??? setup.py > > The setup.py file looks like: > > #!/usr/bin/python3 > # coding=utf8 > > from setuptools import setup > > setup( > name = "proj", > version = "0.2", > packages = ['proj'], > install_requires=[], > entry_points={ > 'console_scripts': [ > 'scriptA=proj:scriptA', > 'scriptB=proj:scriptB' > ], > }, > > # metadata for upload to PyPI > author = "Lu?s", > author_email = "luis at mail.com ", > description = "Some package", > license = "EUPL v1.1", > keywords = "pip package", > url = "https://some.place.com", # project home page, if any > classifiers = [ > "Programming Language :: Python", > "Programming Language :: Python :: 3", > "Development Status :: 4 - Beta", > "Environment :: Console", > "Intended Audience :: Science/Research", > "License :: OSI Approved :: European Union Public Licence 1.1 > (EUPL 1.1)", > "Operating System :: OS Independent", > "Topic :: Scientific/Engineering :: GIS" > ], > ) > > When I try to build I get the following error: > > $ python setup.py bdist_wheel --universal > error in proj setup command: ('Invalid module name', 'proj') > > What am I doing wrong? Thank you, > > Lu?s > > /Sent from ProtonMail , encrypted email based > in Switzerland./ > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From gokoproject at gmail.com Thu Apr 14 14:50:04 2016 From: gokoproject at gmail.com (John Wong) Date: Thu, 14 Apr 2016 14:50:04 -0400 Subject: [Distutils] pip install -e foo (without Repo URL) In-Reply-To: References: <570F44A2.3080902@thomas-guettler.de> Message-ID: > On Apr 14, 2016 2:20 AM, "Thomas G?ttler" > wrote: > > Next use case: you use software third-party-foo-lib in your project. > > Up to now you use it as package. You find a bug and want to fix it. > > Wouldn't it be great if you could just type "pip install -e > third-party-foo-lib" > > and now you are read to create pull requests? > How is this different from 1) fork, 2) update your requirements.txt file pointing to the new fork? I guess -e directly from pip install would be an ad-hoc install of a python library... -------------- next part -------------- An HTML attachment was scrubbed... URL: From marius at gedmin.as Fri Apr 15 02:24:42 2016 From: marius at gedmin.as (Marius Gedminas) Date: Fri, 15 Apr 2016 09:24:42 +0300 Subject: [Distutils] 'Invalid module name' creating package with setuptools In-Reply-To: References: Message-ID: <20160415062442.GC24187@platonas> On Thu, Apr 14, 2016 at 11:28:09AM -0400, Lu??s de Sousa via Distutils-SIG wrote: > I have a project with these contents: > > proj > ??? proj > ? ??? scriptA.py > ? ??? scriptB.py > ? ??? __init__.py > ??? LICENCE > ??? README.md > ??? setup.py > > The setup.py file looks like: ... > entry_points={ > 'console_scripts': [ > 'scriptA=proj:scriptA', > 'scriptB=proj:scriptB' I'm not sure this is right -- an entry point should point to a python module (not a package) and a function in that module, so something like 'scriptA=proj.scriptA:main', 'scriptB=proj.scriptB:main', I've never tried to define entry points pointing to functions defined in the __init__.py of a package, so I don't know if you're allowed to write scriptX=proj:fn or if you have to explicitly say scriptX=proj.__init__:fn > When I try to build I get the following error: > > $ python setup.py bdist_wheel --universal > error in proj setup command: ('Invalid module name', 'proj') (That is not a good error message indeed.) Marius Gedminas -- This loads a GDT entry into the "Task Register": that entry points to a structure called the Task State Segment. Some comments scattered though the kernel code indicate that this used for task switching in ages past, along with blood sacrifice and astrology. -- lguest source code -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 173 bytes Desc: Digital signature URL: From luis.de.sousa at protonmail.ch Fri Apr 15 02:52:17 2016 From: luis.de.sousa at protonmail.ch (=?UTF-8?Q?Lu=C3=AD=C2=ADs_de_Sousa?=) Date: Fri, 15 Apr 2016 02:52:17 -0400 Subject: [Distutils] 'Invalid module name' creating package with setuptools In-Reply-To: <20160415062442.GC24187@platonas> References: <20160415062442.GC24187@platonas> Message-ID: Hi Marius, thank you for the reply. If I remove the line # coding=utf8, I get the following error: $ python setup.py bdist_wheel --universal File "setup.py", line 18 SyntaxError: Non-ASCII character '\xc3' in file setup.py on line 18, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details I created main functions in each of these scripts and altered the entry points bit to: entry_points={ 'console_scripts': [ 'scriptA=proj:scriptA:main', 'scriptB=proj:scriptB:main' ], }, This way I get this error: $ python setup.py bdist_wheel --universal error in proj setup command: ("EntryPoint must be in 'name=module:attrs [extras]' format", 'scriptA=proj:scriptA:main') The __init__.py file is empty, by the way. Any other hints? Thank you, Lu?s Sent from [ProtonMail](https://protonmail.ch), encrypted email based in Switzerland. -------- Original Message -------- Subject: Re: [Distutils] 'Invalid module name' creating package with setuptools Local Time: April 15, 2016 8:24 AM UTC Time: April 15, 2016 6:24 AM From: marius at gedmin.as To: distutils-sig at python.org On Thu, Apr 14, 2016 at 11:28:09AM -0400, Lu??s de Sousa via Distutils-SIG wrote: > I have a project with these contents: > > proj > ??? proj > ? ??? scriptA.py > ? ??? scriptB.py > ? ??? __init__.py > ??? LICENCE > ??? README.md > ??? setup.py > > The setup.py file looks like: ... > entry_points={ > 'console_scripts': [ > 'scriptA=proj:scriptA', > 'scriptB=proj:scriptB' I'm not sure this is right -- an entry point should point to a python module (not a package) and a function in that module, so something like 'scriptA=proj.scriptA:main', 'scriptB=proj.scriptB:main', I've never tried to define entry points pointing to functions defined in the __init__.py of a package, so I don't know if you're allowed to write scriptX=proj:fn or if you have to explicitly say scriptX=proj.__init__:fn > When I try to build I get the following error: > > $ python setup.py bdist_wheel --universal > error in proj setup command: ('Invalid module name', 'proj') (That is not a good error message indeed.) Marius Gedminas -- This loads a GDT entry into the "Task Register": that entry points to a structure called the Task State Segment. Some comments scattered though the kernel code indicate that this used for task switching in ages past, along with blood sacrifice and astrology. -- lguest source code _______________________________________________ Distutils-SIG maillist - Distutils-SIG at python.org https://mail.python.org/mailman/listinfo/distutils-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From luis.de.sousa at protonmail.ch Fri Apr 15 02:53:27 2016 From: luis.de.sousa at protonmail.ch (=?UTF-8?Q?Lu=C3=AD=C2=ADs_de_Sousa?=) Date: Fri, 15 Apr 2016 02:53:27 -0400 Subject: [Distutils] 'Invalid module name' creating package with setuptools In-Reply-To: <570FB80C.5030504@nextday.fi> References: <570FB80C.5030504@nextday.fi> Message-ID: Hi Alex, Could you please detail the steps you took to get this setup.py file to work? There is no other in this project. Thank you, Lu?s Sent from [ProtonMail](https://protonmail.ch), encrypted email based in Switzerland. -------- Original Message -------- Subject: Re: [Distutils] 'Invalid module name' creating package with setuptools Local Time: April 14, 2016 5:32 PM UTC Time: April 14, 2016 3:32 PM From: alex.gronholm at nextday.fi To: distutils-sig at python.org That setup.py worked fine here, so I'm guessing this is not the actual setup.py you're running. What is the exact value of the "name" argument in your real setup.py? BTW, source encoding is utf-8 by default on Python 3 so the "# coding=utf8" is unnecessary. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guettliml at thomas-guettler.de Fri Apr 15 06:05:53 2016 From: guettliml at thomas-guettler.de (=?UTF-8?Q?Thomas_G=c3=bcttler?=) Date: Fri, 15 Apr 2016 12:05:53 +0200 Subject: [Distutils] pip install -e foo (without Repo URL) In-Reply-To: References: <570F44A2.3080902@thomas-guettler.de> Message-ID: <5710BD01.1070808@thomas-guettler.de> Am 14.04.2016 um 14:31 schrieb Ian Cordasco: > > On Apr 14, 2016 2:20 AM, "Thomas G?ttler" > wrote: > > > > I think it would be very cool if you could install a package editable > > without the repo-url. > > > > The default repo-url can be defined in the meta-data of the package. > > > > Background: I came across this becaus saltstack prefers the branch "develop", but > > most other repos use the branch "master". > > > > Yes, this is no big problem. I figured out the right repo url soon. > > > > Next use case: you use software third-party-foo-lib in your project. > > Up to now you use it as package. You find a bug and want to fix it. > > Wouldn't it be great if you could just type "pip install -e third-party-foo-lib" > > and now you are read to create pull requests? > > I don't understand how this makes me ready to submit pull requests. Can you explain a little more? This: pip install -e third-party-foo-lib should be the same as: pip install -e git+https://..../@mydevbranch#egg=third-party-foo-lib And the information needed to build the URL (in above example git+https://..../@mydevbranch) should be provided by setup.py. Result: you have an editable checkout. You are right, pull requests most often need a fork first. This is far out of scope. But for custom (non public) projects hosted in a corporate lan, you get a ready-to-push checkout. Is this enough explanation? Please ask if you have further question. Regards, Thomas G?ttler -- Thomas Guettler http://www.thomas-guettler.de/ From guettliml at thomas-guettler.de Fri Apr 15 06:07:12 2016 From: guettliml at thomas-guettler.de (=?UTF-8?Q?Thomas_G=c3=bcttler?=) Date: Fri, 15 Apr 2016 12:07:12 +0200 Subject: [Distutils] pip install -e foo (without Repo URL) In-Reply-To: References: <570F44A2.3080902@thomas-guettler.de> Message-ID: <5710BD50.9050908@thomas-guettler.de> Am 14.04.2016 um 20:50 schrieb John Wong: > > On Apr 14, 2016 2:20 AM, "Thomas G?ttler" > wrote: > > Next use case: you use software third-party-foo-lib in your project. > > Up to now you use it as package. You find a bug and want to fix it. > > Wouldn't it be great if you could just type "pip install -e third-party-foo-lib" > > and now you are read to create pull requests? > > How is this different from 1) fork, 2) update your requirements.txt file pointing to the new fork? I guess -e directly > from pip install would be an ad-hoc install of a python library... Sorry, the term "pull request" was too much. See my other reply from a second ago. Please ask again, if you have further question. Regards, Thomas G?ttler -- Thomas Guettler http://www.thomas-guettler.de/ From alex.gronholm at nextday.fi Fri Apr 15 08:12:17 2016 From: alex.gronholm at nextday.fi (=?UTF-8?Q?Alex_Gr=c3=b6nholm?=) Date: Fri, 15 Apr 2016 15:12:17 +0300 Subject: [Distutils] pip install -e foo (without Repo URL) In-Reply-To: <5710BD01.1070808@thomas-guettler.de> References: <570F44A2.3080902@thomas-guettler.de> <5710BD01.1070808@thomas-guettler.de> Message-ID: <5710DAA1.3020800@nextday.fi> To my knowledge, metadata 2.0 does not have any reliable means to specify a repository URL. So how do you propose to retrieve this information? 15.04.2016, 13:05, Thomas G?ttler kirjoitti: > > > Am 14.04.2016 um 14:31 schrieb Ian Cordasco: >> >> On Apr 14, 2016 2:20 AM, "Thomas G?ttler" >> > >> wrote: >> > >> > I think it would be very cool if you could install a package editable >> > without the repo-url. >> > >> > The default repo-url can be defined in the meta-data of the package. >> > >> > Background: I came across this becaus saltstack prefers the branch >> "develop", but >> > most other repos use the branch "master". >> > >> > Yes, this is no big problem. I figured out the right repo url soon. >> > >> > Next use case: you use software third-party-foo-lib in your project. >> > Up to now you use it as package. You find a bug and want to fix it. >> > Wouldn't it be great if you could just type "pip install -e >> third-party-foo-lib" >> > and now you are read to create pull requests? >> >> I don't understand how this makes me ready to submit pull requests. >> Can you explain a little more? > > > This: > > pip install -e third-party-foo-lib > > should be the same as: > > pip install -e git+https://..../@mydevbranch#egg=third-party-foo-lib > > And the information needed to build the URL (in above example > git+https://..../@mydevbranch) > should be provided by setup.py. > > Result: you have an editable checkout. > > You are right, pull requests most often need a fork first. This is far > out of scope. > > But for custom (non public) projects hosted in a corporate lan, you > get a ready-to-push > checkout. > > Is this enough explanation? Please ask if you have further question. > > Regards, > Thomas G?ttler > > > From njs at pobox.com Fri Apr 15 11:04:27 2016 From: njs at pobox.com (Nathaniel Smith) Date: Fri, 15 Apr 2016 08:04:27 -0700 Subject: [Distutils] 'Invalid module name' creating package with setuptools In-Reply-To: References: <20160415062442.GC24187@platonas> Message-ID: On Apr 14, 2016 11:52 PM, "Lu??s de Sousa" wrote: > > Hi Marius, thank you for the reply. > > If I remove the line # coding=utf8, I get the following error: > > $ python setup.py bdist_wheel --universal > File "setup.py", line 18 > SyntaxError: Non-ASCII character '\xc3' in file setup.py on line 18, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details > > I created main functions in each of these scripts and altered the entry points bit to: > > entry_points={ > 'console_scripts': [ > 'scriptA=proj:scriptA:main', > 'scriptB=proj:scriptB:main' > ], > }, > > This way I get this error: > > $ python setup.py bdist_wheel --universal > error in proj setup command: ("EntryPoint must be in 'name=module:attrs [extras]' format", 'scriptA=proj:scriptA:main') Here your problem is that entry points should have only one : in them. You want proj.scriptA:main (This means: do 'import proj.scriptA', then call 'proj.scriptA.main'. You can have as many dots as you want on either side of the :, and the colon marks how much of your dotted string gets passed to 'import'.) -n -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.gronholm at nextday.fi Fri Apr 15 11:06:51 2016 From: alex.gronholm at nextday.fi (=?UTF-8?Q?Alex_Gr=c3=b6nholm?=) Date: Fri, 15 Apr 2016 18:06:51 +0300 Subject: [Distutils] 'Invalid module name' creating package with setuptools In-Reply-To: References: <20160415062442.GC24187@platonas> Message-ID: <5711038B.4090003@nextday.fi> If you get that error, it means you're not actually using python 3 to run setup.py. 15.04.2016, 09:52, Lu??s de Sousa via Distutils-SIG kirjoitti: > Hi Marius, thank you for the reply. > > If I remove the line # coding=utf8, I get the following error: > > $ python setup.py bdist_wheel --universal > File "setup.py", line 18 > SyntaxError: Non-ASCII character '\xc3' in file setup.py on line 18, > but no encoding declared; see http://www.python.org/peps/pep-0263.html > for details > > I created main functions in each of these scripts and altered the > entry points bit to: > > entry_points={ > 'console_scripts': [ > 'scriptA=proj:scriptA:main', > 'scriptB=proj:scriptB:main' > ], > }, > > This way I get this error: > > $ python setup.py bdist_wheel --universal > error in proj setup command: ("EntryPoint must be in > 'name=module:attrs [extras]' format", 'scriptA=proj:scriptA:main') > > The __init__.py file is empty, by the way. Any other hints? > > Thank you, > > Lu?s > > > /Sent from ProtonMail , encrypted email based > in Switzerland./ > > >> -------- Original Message -------- >> Subject: Re: [Distutils] 'Invalid module name' creating package with >> setuptools >> Local Time: April 15, 2016 8:24 AM >> UTC Time: April 15, 2016 6:24 AM >> From: marius at gedmin.as >> To: distutils-sig at python.org >> >> On Thu, Apr 14, 2016 at 11:28:09AM -0400, Lu??s de Sousa via >> Distutils-SIG wrote: >> > I have a project with these contents: >> > >> > proj >> > ??? proj >> > ? ??? scriptA.py >> > ? ??? scriptB.py >> > ? ??? __init__.py >> > ??? LICENCE >> > ??? README.md >> > ??? setup.py >> > >> > The setup.py file looks like: >> ... >> > entry_points={ >> > 'console_scripts': [ >> > 'scriptA=proj:scriptA', >> > 'scriptB=proj:scriptB' >> >> I'm not sure this is right -- an entry point should point to a python >> module (not a package) and a function in that module, so something like >> >> 'scriptA=proj.scriptA:main', >> 'scriptB=proj.scriptB:main', >> >> I've never tried to define entry points pointing to functions defined in >> the __init__.py of a package, so I don't know if you're allowed to write >> >> scriptX=proj:fn >> >> or if you have to explicitly say >> >> scriptX=proj.__init__:fn >> >> > When I try to build I get the following error: >> > >> > $ python setup.py bdist_wheel --universal >> > error in proj setup command: ('Invalid module name', 'proj') >> >> (That is not a good error message indeed.) >> >> Marius Gedminas >> -- >> This loads a GDT entry into the "Task Register": that entry points to a >> structure called the Task State Segment. Some comments scattered >> though the >> kernel code indicate that this used for task switching in ages past, >> along >> with blood sacrifice and astrology. >> -- lguest source code >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> https://mail.python.org/mailman/listinfo/distutils-sig > > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Fri Apr 15 19:29:40 2016 From: guido at python.org (Guido van Rossum) Date: Fri, 15 Apr 2016 16:29:40 -0700 Subject: [Distutils] The mypy package Message-ID: Brett suggested I ask the kind folks here. As you may or may not know, there's an old unmaintained "mypy" package on PyPI that attracts a fair amount of downloads from people trying to download mypy the type checker. We then get bug reports and have to explain in our tracker that they have to use "pip install mypy-lang" instead. Query: https://github.com/python/mypy/issues?utf8=%E2%9C%93&q=is%3Aissue++dbutils+ That mypy package was last updated in 2011, and it's a quite forgettable combination of copied open-source packages and a little bit of glue code presumably written by the package author. Both Donald and myself have approached the owner (zsp007 at gmail.com) but not received any response. Is there a "higher authority" to whom we can appeal this, or are we just stuck with this situation? As Brett wrote part of the problem, though, is the mypy project has 2244 downloads in the last month which shows it's being used and we don't want to end up in an npm/left_pad situation. (But how many of those downloads are misguided attempts to install mypy-lang?) One possibility, if people aren't happy with me or Jukka taking over owhership of the old mypy package, would be for someone (not me or Jukka) to take ownership of that package just so they can update the PyPI home page for that package with a prominent note telling people looking for Jukka's type checker to use mypy-lang instead. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.gronholm at nextday.fi Fri Apr 15 19:32:42 2016 From: alex.gronholm at nextday.fi (=?UTF-8?Q?Alex_Gr=c3=b6nholm?=) Date: Sat, 16 Apr 2016 02:32:42 +0300 Subject: [Distutils] The mypy package In-Reply-To: References: Message-ID: <57117A1A.6040102@nextday.fi> I've fallen into this trap as well, so +1 for the takeover. It might be a good idea to come up with a standardized process for taking over old, unmaintained packages. 16.04.2016, 02:29, Guido van Rossum kirjoitti: > Brett suggested I ask the kind folks here. > > As you may or may not know, there's an old unmaintained "mypy" package > on PyPI that attracts a fair amount of downloads from people trying to > download mypy the type checker. We then get bug reports and have to > explain in our tracker that they have to use "pip install mypy-lang" > instead. > > Query: > https://github.com/python/mypy/issues?utf8=%E2%9C%93&q=is%3Aissue++dbutils+ > > That mypy package was last updated in 2011, and it's a quite > forgettable combination of copied open-source packages and a little > bit of glue code presumably written by the package author. Both Donald > and myself have approached the owner (zsp007 at gmail.com > ) but not received any response. Is there a > "higher authority" to whom we can appeal this, or are we just stuck > with this situation? > > As Brett wrote part of the problem, though, is the mypy project has > 2244 downloads in the last month which shows it's being used and we > don't want to end up in an npm/left_pad situation. (But how many of > those downloads are misguided attempts to install mypy-lang?) > > One possibility, if people aren't happy with me or Jukka taking over > owhership of the old mypy package, would be for someone (not me or > Jukka) to take ownership of that package just so they can update the > PyPI home page for that package with a prominent note telling people > looking for Jukka's type checker to use mypy-lang instead. > > -- > --Guido van Rossum (python.org/~guido ) > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard at python.org Fri Apr 15 20:10:33 2016 From: richard at python.org (Richard Jones) Date: Sat, 16 Apr 2016 10:10:33 +1000 Subject: [Distutils] The mypy package In-Reply-To: References: Message-ID: Hi Guido, Because this sort of thing has come up a lot in the past, and because I've copped trouble for mishandling it in the past, I took the trouble of writing up a formal description of how I handle these sorts of issues: https://docs.google.com/document/d/1elum7ENjQb0dLB4ATfYNtnXYVLUzsKacc0VWnHHJb2A/edit?usp=sharing I believe Donald follows the same, or a very similar procedure. In short, all that can be done has been done, from my perspective. Someone has published their module, and regardless of any opinion of it, or desire to also use that name, I have to respect that they published first. In the absence of explicit consent from them to do anything, my hands are tied. I've taken unilateral action in the past to my personal detriment. Of course, once I'm no longer a PyPI admin (I look forward to the day so very much) someone else will have to make these decisions. Richard On 16 April 2016 at 09:29, Guido van Rossum wrote: > Brett suggested I ask the kind folks here. > > As you may or may not know, there's an old unmaintained "mypy" package on > PyPI that attracts a fair amount of downloads from people trying to > download mypy the type checker. We then get bug reports and have to explain > in our tracker that they have to use "pip install mypy-lang" instead. > > Query: > https://github.com/python/mypy/issues?utf8=%E2%9C%93&q=is%3Aissue++dbutils+ > > That mypy package was last updated in 2011, and it's a quite forgettable > combination of copied open-source packages and a little bit of glue code > presumably written by the package author. Both Donald and myself have > approached the owner (zsp007 at gmail.com) but not received any response. Is > there a "higher authority" to whom we can appeal this, or are we just stuck > with this situation? > > As Brett wrote part of the problem, though, is the mypy project has 2244 > downloads in the last month which shows it's being used and we don't want > to end up in an npm/left_pad situation. (But how many of those downloads > are misguided attempts to install mypy-lang?) > > One possibility, if people aren't happy with me or Jukka taking over > owhership of the old mypy package, would be for someone (not me or Jukka) > to take ownership of that package just so they can update the PyPI home > page for that package with a prominent note telling people looking for > Jukka's type checker to use mypy-lang instead. > > -- > --Guido van Rossum (python.org/~guido) > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Fri Apr 15 20:26:59 2016 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 16 Apr 2016 12:26:59 +1200 Subject: [Distutils] The mypy package In-Reply-To: References: Message-ID: <571186D3.1010409@canterbury.ac.nz> Why are you so determined to use the name "mypy" in the first place? Seems to me it's a terrible name. It sounds more like a working title for someone's personal implementation of Python, and certainly gives no clue that it has anything to do with type checking. So instead of trying to steal the name "mypy", how about coming up with a new, more appropriate name? -- Greg From guido at python.org Fri Apr 15 21:10:01 2016 From: guido at python.org (Guido van Rossum) Date: Fri, 15 Apr 2016 18:10:01 -0700 Subject: [Distutils] The mypy package In-Reply-To: References: Message-ID: Oh well. I wonder if offering money would change the situation. On Fri, Apr 15, 2016 at 5:10 PM, Richard Jones wrote: > Hi Guido, > > Because this sort of thing has come up a lot in the past, and because I've > copped trouble for mishandling it in the past, I took the trouble of > writing up a formal description of how I handle these sorts of issues: > > > https://docs.google.com/document/d/1elum7ENjQb0dLB4ATfYNtnXYVLUzsKacc0VWnHHJb2A/edit?usp=sharing > > I believe Donald follows the same, or a very similar procedure. > > In short, all that can be done has been done, from my perspective. Someone > has published their module, and regardless of any opinion of it, or desire > to also use that name, I have to respect that they published first. In the > absence of explicit consent from them to do anything, my hands are tied. > I've taken unilateral action in the past to my personal detriment. > > Of course, once I'm no longer a PyPI admin (I look forward to the day so > very much) someone else will have to make these decisions. > > > Richard > > > On 16 April 2016 at 09:29, Guido van Rossum wrote: > >> Brett suggested I ask the kind folks here. >> >> As you may or may not know, there's an old unmaintained "mypy" package on >> PyPI that attracts a fair amount of downloads from people trying to >> download mypy the type checker. We then get bug reports and have to explain >> in our tracker that they have to use "pip install mypy-lang" instead. >> >> Query: >> https://github.com/python/mypy/issues?utf8=%E2%9C%93&q=is%3Aissue++dbutils+ >> >> That mypy package was last updated in 2011, and it's a quite forgettable >> combination of copied open-source packages and a little bit of glue code >> presumably written by the package author. Both Donald and myself have >> approached the owner (zsp007 at gmail.com) but not received any response. >> Is there a "higher authority" to whom we can appeal this, or are we just >> stuck with this situation? >> >> As Brett wrote part of the problem, though, is the mypy project has 2244 >> downloads in the last month which shows it's being used and we don't want >> to end up in an npm/left_pad situation. (But how many of those downloads >> are misguided attempts to install mypy-lang?) >> >> One possibility, if people aren't happy with me or Jukka taking over >> owhership of the old mypy package, would be for someone (not me or Jukka) >> to take ownership of that package just so they can update the PyPI home >> page for that package with a prominent note telling people looking for >> Jukka's type checker to use mypy-lang instead. >> >> -- >> --Guido van Rossum (python.org/~guido) >> >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> https://mail.python.org/mailman/listinfo/distutils-sig >> >> > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Fri Apr 15 23:59:00 2016 From: guido at python.org (Guido van Rossum) Date: Fri, 15 Apr 2016 20:59:00 -0700 Subject: [Distutils] The mypy package In-Reply-To: References: Message-ID: Another thing. Is the search index on pypi.python.org no longer being updated? Searching for mypy-lang still takes you to mypy-lang 0.2.0, even though 0.3.1 has been released many weeks ago. I just updated the typing package and searching for that still shows the old versions. On Fri, Apr 15, 2016 at 6:10 PM, Guido van Rossum wrote: > Oh well. I wonder if offering money would change the situation. > > On Fri, Apr 15, 2016 at 5:10 PM, Richard Jones wrote: > >> Hi Guido, >> >> Because this sort of thing has come up a lot in the past, and because >> I've copped trouble for mishandling it in the past, I took the trouble of >> writing up a formal description of how I handle these sorts of issues: >> >> >> https://docs.google.com/document/d/1elum7ENjQb0dLB4ATfYNtnXYVLUzsKacc0VWnHHJb2A/edit?usp=sharing >> >> I believe Donald follows the same, or a very similar procedure. >> >> In short, all that can be done has been done, from my perspective. >> Someone has published their module, and regardless of any opinion of it, or >> desire to also use that name, I have to respect that they published first. >> In the absence of explicit consent from them to do anything, my hands are >> tied. I've taken unilateral action in the past to my personal detriment. >> >> Of course, once I'm no longer a PyPI admin (I look forward to the day so >> very much) someone else will have to make these decisions. >> >> >> Richard >> >> >> On 16 April 2016 at 09:29, Guido van Rossum wrote: >> >>> Brett suggested I ask the kind folks here. >>> >>> As you may or may not know, there's an old unmaintained "mypy" package >>> on PyPI that attracts a fair amount of downloads from people trying to >>> download mypy the type checker. We then get bug reports and have to explain >>> in our tracker that they have to use "pip install mypy-lang" instead. >>> >>> Query: >>> https://github.com/python/mypy/issues?utf8=%E2%9C%93&q=is%3Aissue++dbutils+ >>> >>> That mypy package was last updated in 2011, and it's a quite forgettable >>> combination of copied open-source packages and a little bit of glue code >>> presumably written by the package author. Both Donald and myself have >>> approached the owner (zsp007 at gmail.com) but not received any response. >>> Is there a "higher authority" to whom we can appeal this, or are we just >>> stuck with this situation? >>> >>> As Brett wrote part of the problem, though, is the mypy project has 2244 >>> downloads in the last month which shows it's being used and we don't want >>> to end up in an npm/left_pad situation. (But how many of those downloads >>> are misguided attempts to install mypy-lang?) >>> >>> One possibility, if people aren't happy with me or Jukka taking over >>> owhership of the old mypy package, would be for someone (not me or Jukka) >>> to take ownership of that package just so they can update the PyPI home >>> page for that package with a prominent note telling people looking for >>> Jukka's type checker to use mypy-lang instead. >>> >>> -- >>> --Guido van Rossum (python.org/~guido) >>> >>> _______________________________________________ >>> Distutils-SIG maillist - Distutils-SIG at python.org >>> https://mail.python.org/mailman/listinfo/distutils-sig >>> >>> >> > > > -- > --Guido van Rossum (python.org/~guido) > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From njs at pobox.com Sat Apr 16 01:21:17 2016 From: njs at pobox.com (Nathaniel Smith) Date: Fri, 15 Apr 2016 22:21:17 -0700 Subject: [Distutils] The mypy package In-Reply-To: References: Message-ID: On Fri, Apr 15, 2016 at 8:59 PM, Guido van Rossum wrote: > Another thing. Is the search index on pypi.python.org no longer being > updated? Searching for mypy-lang still takes you to mypy-lang 0.2.0, even > though 0.3.1 has been released many weeks ago. I just updated the typing > package and searching for that still shows the old versions. Yes, search and download statistics have both been broken on pypi for a while. Unfortunately it seems that everyone who's in a position to figure out why or fix it is already scrambling to fight other equally high priority fires. There was some more discussion in this thread: https://mail.python.org/pipermail/distutils-sig/2016-March/028464.html -n -- Nathaniel J. Smith -- https://vorpus.org From p.f.moore at gmail.com Sat Apr 16 04:36:45 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 16 Apr 2016 09:36:45 +0100 Subject: [Distutils] The mypy package In-Reply-To: References: Message-ID: On 16 April 2016 at 01:10, Richard Jones wrote: > Because this sort of thing has come up a lot in the past, and because I've > copped trouble for mishandling it in the past, I took the trouble of writing > up a formal description of how I handle these sorts of issues: > > https://docs.google.com/document/d/1elum7ENjQb0dLB4ATfYNtnXYVLUzsKacc0VWnHHJb2A/edit?usp=sharing > > I believe Donald follows the same, or a very similar procedure. > > In short, all that can be done has been done, from my perspective. Someone > has published their module, and regardless of any opinion of it, or desire > to also use that name, I have to respect that they published first. In the > absence of explicit consent from them to do anything, my hands are tied. > I've taken unilateral action in the past to my personal detriment. > > Of course, once I'm no longer a PyPI admin (I look forward to the day so > very much) someone else will have to make these decisions. I can understand your reasons for not wanting to take unilateral action. I wonder, however, whether it would be reasonable to add an explicit policy to PyPI (probably at the point of the switch to Warehouse) requiring project owners to provide an active email address (where "active" means, say, responding to an annual automated ping email to confirm the project is still alive). There would obviously have to be some sort of "legacy" exemption from this, and maybe a transition process. It still wouldn't help if the author doesn't respond to requests like Guido's, but at least it avoids the possibility that the actual email address is dead. Other than this, I don't think there's much that can be done here. The "first come, first serve" nature of claiming names on PyPI is pretty much part of the culture. Changing that would be a pretty hard sell (as well as being a ton of work that no-one is likely to want to do...) Paul From contact at ionelmc.ro Sat Apr 16 08:23:11 2016 From: contact at ionelmc.ro (=?UTF-8?Q?Ionel_Cristian_M=C4=83rie=C8=99?=) Date: Sat, 16 Apr 2016 15:23:11 +0300 Subject: [Distutils] pip install -e foo (without Repo URL) In-Reply-To: <5710BD01.1070808@thomas-guettler.de> References: <570F44A2.3080902@thomas-guettler.de> <5710BD01.1070808@thomas-guettler.de> Message-ID: On Fri, Apr 15, 2016 at 1:05 PM, Thomas G?ttler < guettliml at thomas-guettler.de> wrote: > This: > > pip install -e > ?? > third-party-foo-lib > > should be the same as: > > pip install -e git+https://..../@mydevbranch#egg=third-party-foo-lib > In the first form, how would you tell pip which branch you want?? (assuming you don't want whatever is default) If you have a local directory `?third-party-foo-lib` would that take precedence over a scm checkout? ?There would need to be a way to add this extra metadata (scm url and default branch)? in distributions ... uh, that means changing setuptools/distutils? That means adoption issues, people won't always have latest and greatest. Thanks, -- Ionel Cristian M?rie?, http://blog.ionelmc.ro -------------- next part -------------- An HTML attachment was scrubbed... URL: From tritium-list at sdamon.com Sat Apr 16 12:42:57 2016 From: tritium-list at sdamon.com (Alexander Walters) Date: Sat, 16 Apr 2016 12:42:57 -0400 Subject: [Distutils] The mypy package In-Reply-To: References: Message-ID: <57126B91.9090106@sdamon.com> To what end? As much as old packages cluttering the namespace of pypi is annoying, the only thing that will accomplish is orphaning projects. We cant unilaterally hand over names on pypi to unrelated.. or even related.. projects because they have a name someone else wants. Another solution like adding namespaces to pypi sounds better to me... but then I think about the nightmare of implementing that in a backwards compatible way. On 4/16/2016 04:36, Paul Moore wrote: > I wonder, however, whether it would be reasonable to add an explicit > policy to PyPI (probably at the point of the switch to Warehouse) > requiring project owners to provide an active email address (where > "active" means, say, responding to an annual automated ping email to > confirm the project is still alive). From donald at stufft.io Sat Apr 16 12:53:10 2016 From: donald at stufft.io (Donald Stufft) Date: Sat, 16 Apr 2016 12:53:10 -0400 Subject: [Distutils] The mypy package In-Reply-To: <57126B91.9090106@sdamon.com> References: <57126B91.9090106@sdamon.com> Message-ID: <82192EA5-B7F1-4C42-9007-743D81FAEB92@stufft.io> > On Apr 16, 2016, at 12:42 PM, Alexander Walters wrote: > > Another solution like adding namespaces to pypi sounds better to me... but then I think about the nightmare of implementing that in a backwards compatible way. We already have namespacing sort of, people can make something like dstufft.mypy and that works fine. The main thing that doesn?t exist is there?s no way to claim an entire namespace to prevent someone else from taking something in your namespace. However, I don?t think most people really want that, people like having shorter names that aren?t tied to a specific namespace generally. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 842 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tritium-list at sdamon.com Sat Apr 16 13:00:32 2016 From: tritium-list at sdamon.com (Alexander Walters) Date: Sat, 16 Apr 2016 13:00:32 -0400 Subject: [Distutils] Namespaces (was: The mypy package) In-Reply-To: <82192EA5-B7F1-4C42-9007-743D81FAEB92@stufft.io> References: <57126B91.9090106@sdamon.com> <82192EA5-B7F1-4C42-9007-743D81FAEB92@stufft.io> Message-ID: <57126FB0.9080000@sdamon.com> On 4/16/2016 12:53, Donald Stufft wrote: >> On Apr 16, 2016, at 12:42 PM, Alexander Walters wrote: >> >> Another solution like adding namespaces to pypi sounds better to me... but then I think about the nightmare of implementing that in a backwards compatible way. > We already have namespacing sort of, people can make something like dstufft.mypy and that works fine. The main thing that doesn?t exist is there?s no way to claim an entire namespace to prevent someone else from taking something in your namespace. > > However, I don?t think most people really want that, people like having shorter names that aren?t tied to a specific namespace generally. > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA > My thought when I made that suggestion (which I already dismissed as a nightmare for you brave souls who provide the index free of charge... thank you) was less in the realm of 'tritium.mypackage', where a user would claim their package namespace for all their code, but more along the lines of 'django/my.super.cool.app' or 'flask/my_extension' or 'qa/mypy' and 'qa/flake8' - curated (or open...) namespaces of related packages. From p.f.moore at gmail.com Sat Apr 16 14:48:41 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 16 Apr 2016 19:48:41 +0100 Subject: [Distutils] The mypy package In-Reply-To: <57126B91.9090106@sdamon.com> References: <57126B91.9090106@sdamon.com> Message-ID: On 16 April 2016 at 17:42, Alexander Walters wrote: > To what end? To the end of ensuring that people can get in touch with project owners. > As much as old packages cluttering the namespace of pypi is > annoying, the only thing that will accomplish is orphaning projects. Not at all. The projects that would be affected by this are those that are already orphaned, in the sense that there's no means to contact the owner. > We > cant unilaterally hand over names on pypi to unrelated.. or even related.. > projects because they have a name someone else wants. I'm not proposing anything like that - I guess when I said "requiring project owners to provide an email address..." it read like I was suggesting removing stuff if the owner went away. I hadn't really thought much about what we *do* with projects where the contact email becomes unmonitored (I *so* think it's acceptable to require a working contact email when a project is registered, or maybe when a user registers From p.f.moore at gmail.com Sat Apr 16 14:52:11 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 16 Apr 2016 19:52:11 +0100 Subject: [Distutils] The mypy package In-Reply-To: References: <57126B91.9090106@sdamon.com> Message-ID: Sorry, please ignore that email. I hit "Send" too soon. > We > cant unilaterally hand over names on pypi to unrelated.. or even related.. > projects because they have a name someone else wants. What I *meant* to say here was that when a request for a name transfer gets no reply, it's helpful to know if the email address is no longer responding at all. Nothing more than that. That might help anyone making a decision to decide what to do. But no, I agree absolutely we can't just hand over names. Paul On 16 April 2016 at 19:48, Paul Moore wrote: > On 16 April 2016 at 17:42, Alexander Walters wrote: >> To what end? > > To the end of ensuring that people can get in touch with project owners. > >> As much as old packages cluttering the namespace of pypi is >> annoying, the only thing that will accomplish is orphaning projects. > > Not at all. The projects that would be affected by this are those that > are already orphaned, in the sense that there's no means to contact > the owner. > >> We >> cant unilaterally hand over names on pypi to unrelated.. or even related.. >> projects because they have a name someone else wants. > > I'm not proposing anything like that - I guess when I said "requiring > project owners to provide an email address..." it read like I was > suggesting removing stuff if the owner went away. I hadn't really > thought much about what we *do* with projects where the contact email > becomes unmonitored (I *so* think it's acceptable to require a working > contact email when a project is registered, or maybe when a user > registers From tritium-list at sdamon.com Sat Apr 16 20:26:50 2016 From: tritium-list at sdamon.com (Alexander Walters) Date: Sat, 16 Apr 2016 20:26:50 -0400 Subject: [Distutils] The mypy package In-Reply-To: References: <57126B91.9090106@sdamon.com> Message-ID: <5712D84A.1070804@sdamon.com> On 4/16/2016 14:52, Paul Moore wrote: >> We >> >cant unilaterally hand over names on pypi to unrelated.. or even related.. >> >projects because they have a name someone else wants. > What I*meant* to say here was that when a request for a name transfer > gets no reply, it's helpful to know if the email address is no longer > responding at all. Nothing more than that. That might help anyone > making a decision to decide what to do. > > But no, I agree absolutely we can't just hand over names. > > Paul If what you intend is to just flag the index entries where the owner has a bouncing email as "Heads up, the owner's email doesn't work anymore", then I don't have any problems with that. Though I do wonder how effective that would be in this case. For all we know, in the case of mypy, the maintainer is simply ignoring someone else who is trying to take the name they registered. (I get emails all the time for people trying to get me to sign over my domain names; even though I am not doing much with them, they are my names and my identity, so that is one possible reason.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From contact at ionelmc.ro Sun Apr 17 02:38:01 2016 From: contact at ionelmc.ro (=?UTF-8?Q?Ionel_Cristian_M=C4=83rie=C8=99?=) Date: Sun, 17 Apr 2016 09:38:01 +0300 Subject: [Distutils] The mypy package In-Reply-To: References: Message-ID: On Sat, Apr 16, 2016 at 2:29 AM, Guido van Rossum wrote: > Both Donald and myself have approached the owner (zsp007 at gmail.com) but > not received any response. > ?Have you considered getting someone to write an email in Chinese?? I suspect he did not understand what was asked in Enligsh. Thanks, -- Ionel Cristian M?rie?, http://blog.ionelmc.ro -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Sun Apr 17 18:46:05 2016 From: guido at python.org (Guido van Rossum) Date: Sun, 17 Apr 2016 15:46:05 -0700 Subject: [Distutils] The mypy package In-Reply-To: References: Message-ID: Hm, interesting idea. I probably know someone at work who can help me translate. In a similar vein, the package distributor is listed as "zuroc" -- would this be someone else or just an alias for the owner? On Sat, Apr 16, 2016 at 11:38 PM, Ionel Cristian M?rie? wrote: > > On Sat, Apr 16, 2016 at 2:29 AM, Guido van Rossum > wrote: > >> Both Donald and myself have approached the owner (zsp007 at gmail.com) but >> not received any response. >> > > ?Have you considered getting someone to write an email in Chinese?? I > suspect he did not understand what was asked in Enligsh. > > > > Thanks, > -- Ionel Cristian M?rie?, http://blog.ionelmc.ro > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard at python.org Sun Apr 17 20:48:56 2016 From: richard at python.org (Richard Jones) Date: Mon, 18 Apr 2016 10:48:56 +1000 Subject: [Distutils] The mypy package In-Reply-To: References: Message-ID: On 18 April 2016 at 08:46, Guido van Rossum wrote: > In a similar vein, the package distributor is listed as "zuroc" -- would > this be someone else or just an alias for the owner? > That's the same person mentioned in your original mail. Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From guettliml at thomas-guettler.de Mon Apr 18 09:17:52 2016 From: guettliml at thomas-guettler.de (=?UTF-8?Q?Thomas_G=c3=bcttler?=) Date: Mon, 18 Apr 2016 15:17:52 +0200 Subject: [Distutils] pip install -e foo (without Repo URL) In-Reply-To: <5710DAA1.3020800@nextday.fi> References: <570F44A2.3080902@thomas-guettler.de> <5710BD01.1070808@thomas-guettler.de> <5710DAA1.3020800@nextday.fi> Message-ID: <5714DE80.7030006@thomas-guettler.de> Am 15.04.2016 um 14:12 schrieb Alex Gr?nholm: > To my knowledge, metadata 2.0 does not have any reliable means to specify a repository URL. > So how do you propose to retrieve this information? How to retrieve the information? I don't know. Where is the best place to store meta information for python packages? Regards, Thomas G?ttler -- Thomas Guettler http://www.thomas-guettler.de/ From chris.barker at noaa.gov Mon Apr 18 11:18:37 2016 From: chris.barker at noaa.gov (Chris Barker - NOAA Federal) Date: Mon, 18 Apr 2016 08:18:37 -0700 Subject: [Distutils] The mypy package In-Reply-To: <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> Message-ID: <-3275783620647160143@unknownmsgid> > Though I do wonder how effective that would be in this case. For all we know, in the case of mypy, the maintainer is simply ignoring someone else who is trying to take the name they registered. (I get emails all the time for people trying to get me to sign over my domain names; Domain names are a different system -- you need to maintain your registration. PyPi names, on the other hand, are all too easy to setup, and then completely ignore, maybe even forget you used it. We really should have SOME way to determine if a PyPi name has been abandoned. Or even be proactive--PyPi names must be maintained in SOME way, perhaps: Push a change or update at least once a year (or some other interval). Or Respond to some sort of "do you still want this" email. At least once a year. If neither of these occurs, then we could have a deprecation period. Details aside, as PyPi continues to grow, we really need a way to clear out the abandoned stuff -- the barrier to entry for creating a new name on PyPi is just too low. This is all too late for MyPy, but it has certainly come up before, and will again, more and more. -CHB From jim at jimfulton.info Mon Apr 18 11:21:04 2016 From: jim at jimfulton.info (Jim Fulton) Date: Mon, 18 Apr 2016 11:21:04 -0400 Subject: [Distutils] The mypy package In-Reply-To: <-3275783620647160143@unknownmsgid> References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> <-3275783620647160143@unknownmsgid> Message-ID: I suggest measuring activity by downloads, not releases. Sometimes maintained packages are boring enough not to need releases, while many projects depend on them. Jim On Mon, Apr 18, 2016 at 11:18 AM, Chris Barker - NOAA Federal wrote: >> Though I do wonder how effective that would be in this case. For all we know, in the case of mypy, the maintainer is simply ignoring someone else who is trying to take the name they registered. (I get emails all the time for people trying to get me to sign over my domain names; > > Domain names are a different system -- you need to maintain your registration. > > PyPi names, on the other hand, are all too easy to setup, and then > completely ignore, maybe even forget you used it. > > We really should have SOME way to determine if a PyPi name has been > abandoned. Or even be proactive--PyPi names must be maintained in SOME > way, perhaps: > > Push a change or update at least once a year (or some other interval). > > Or > > Respond to some sort of "do you still want this" email. At least once a year. > > If neither of these occurs, then we could have a deprecation period. > > Details aside, as PyPi continues to grow, we really need a way to > clear out the abandoned stuff -- the barrier to entry for creating a > new name on PyPi is just too low. > > This is all too late for MyPy, but it has certainly come up before, > and will again, more and more. > > -CHB > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig -- Jim Fulton http://jimfulton.info From waynejwerner at gmail.com Mon Apr 18 11:53:03 2016 From: waynejwerner at gmail.com (Wayne Werner) Date: Mon, 18 Apr 2016 10:53:03 -0500 Subject: [Distutils] The mypy package In-Reply-To: References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> <-3275783620647160143@unknownmsgid> Message-ID: On Mon, Apr 18, 2016 at 10:21 AM, Jim Fulton wrote: > I suggest measuring activity by downloads, not releases. Sometimes > maintained packages are boring enough not to need releases, while many > projects depend on them. > I don't know about on pypi, but I know in general there are plenty of packages that could be considered "complete". Although when it comes to this particular case it *has* a lot of downloads, but there's no real reasonable way to tell if it's an intentional download or a, "oh, that's definitely not the package I meant to download" kind of case. One instance that I can think about - Kenneth Reitz (afaict) used to have the inbox package. I'm assuming that Nylas contacted him, and now there's inbox and inbox.py. Neither package is particularly popular, with only a couple hundred downloads each in the last month. But occasionally I've downloaded the Nylas package when I didn't intend to. I think you could *guess* that I meant to download the other package because probably for each of the times I downloaded inbox, in 5-10 minutes I downloaded inbox.py. But that's not a guarantee, of course. Personally I'm more in favor of at least requiring some kind of, "Yeah, this is still my project," (at least for projects that aren't being actively updated, e.g. maybe no uploads for a year?). One example of how things *could* be done is to follow the ICANN - you register your domains, and after N months your domain expires and can be scooped up by another owner. That's nice because people who don't care about their property automatically release that back into the pool. Of course on the other hand you have absolutely horrible people, like the ones who registered my late brother's domain name that literally has no relation to anyone besides him and wants several hundred dollars to buy it back. And of course there's yet another option - re-branding your own project into a name that's not taken by someone else. I think it's a hard question, and I don't know if there can be a right answer for all circumstances. But it may not be obvious because I'm not Dutch ;) -Wayne -------------- next part -------------- An HTML attachment was scrubbed... URL: From tritium-list at sdamon.com Mon Apr 18 12:30:44 2016 From: tritium-list at sdamon.com (Alexander Walters) Date: Mon, 18 Apr 2016 12:30:44 -0400 Subject: [Distutils] The mypy package In-Reply-To: <-3275783620647160143@unknownmsgid> References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> <-3275783620647160143@unknownmsgid> Message-ID: <57150BB4.8080101@sdamon.com> On 4/18/2016 11:18, Chris Barker - NOAA Federal wrote: > Domain names are a different system -- you need to maintain your registration. Except, that wasn't my point. My point was I ignore people asking to buy my domain from me because the registered name is part of my identity. > PyPi names, on the other hand, are all too easy to setup, and then > completely ignore, maybe even forget you used it. > > We really should have SOME way to determine if a PyPi name has been > abandoned. Or even be proactive--PyPi names must be maintained in SOME > way, perhaps: Why? > Push a change or update at least once a year (or some other interval). What if your code doesn't need an update? > > Or > > Respond to some sort of "do you still want this" email. At least once a year. And how many times have you missed an automated email? > If neither of these occurs, then we could have a deprecation period. > > Details aside, as PyPi continues to grow, we really need a way to > clear out the abandoned stuff -- the barrier to entry for creating a > new name on PyPi is just too low. We absolutely do not. Names are first come, first serve, in perpetuity. Changing this changes the security model of pypi. If all an attacker has to do is wait out an old, but still highly downloaded package... why wouldn't they do it? > > This is all too late for MyPy, but it has certainly come up before, > and will again, more and more. > > -CHB From chris.barker at noaa.gov Mon Apr 18 12:31:55 2016 From: chris.barker at noaa.gov (Chris Barker) Date: Mon, 18 Apr 2016 09:31:55 -0700 Subject: [Distutils] The mypy package In-Reply-To: References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> <-3275783620647160143@unknownmsgid> Message-ID: On Mon, Apr 18, 2016 at 8:21 AM, Jim Fulton wrote: > I suggest measuring activity by downloads, not releases. Sometimes > maintained packages are boring enough not to need releases, while many > projects depend on them. > so this is the tricky bit -- in the mypy case, it's pretty clear that many of the downloads are mistakes. maybe many of them are real use cases also -- how to know? there is no way to know! Yes, it's possible that: - someone puts something up n pypi. - that someone abandons the project. - no one picks up maintenance for the project - other people still find it useful --- many years into the future. This is the one edge-case that is the real trick. Personally, I would argue that a completely and totally abandoned project would be OK to remove from pypi (Not any time really fast). If you can't even get anyone to step and say: "sure -- I'll respond to an email once a year to keep this alive", then you really have a pretty darn dead project. The trick is how to make this process work. Maybe an apparently abandoned project gets a "might-be-abandoned" tag. Then its page geta a prominent -- "We need someone to take this over" message. Maybe even a way to pass a message off to folks that install it via pypi. If no one steps up for, say, a year or so, then it gets removed. We could even make that provisional, for another period of time, so folks trying to download it would get a message saying: This package appears to be abandoned -- if you would like adopt it, please do such and such" They key point ist hat the barrier to entry for grabbing a name on pypi is very, very low. maybe the barrier to KEEP that name should be equally low -- rather than non-existant. I think the domain name system is a fine parallel -- if you want to keep your domain name, you need to keep your registration up -- all that takes is a few bucks a year (and in this case, we wouldn't even ask for a few bucks) but you need to do SOMETHING -- you can't just abandon it and keep everyone else from using it forever. And note that it's possible for someone to put up a useful web site on a free hosting service, attach it to their domain name, and then abandon it -- sure, there may be someone out there that finds that site useful years from now -- but those are the breaks. -CHB > Jim > > On Mon, Apr 18, 2016 at 11:18 AM, Chris Barker - NOAA Federal > wrote: > >> Though I do wonder how effective that would be in this case. For all > we know, in the case of mypy, the maintainer is simply ignoring someone > else who is trying to take the name they registered. (I get emails all the > time for people trying to get me to sign over my domain names; > > > > Domain names are a different system -- you need to maintain your > registration. > > > > PyPi names, on the other hand, are all too easy to setup, and then > > completely ignore, maybe even forget you used it. > > > > We really should have SOME way to determine if a PyPi name has been > > abandoned. Or even be proactive--PyPi names must be maintained in SOME > > way, perhaps: > > > > Push a change or update at least once a year (or some other interval). > > > > Or > > > > Respond to some sort of "do you still want this" email. At least once a > year. > > > > If neither of these occurs, then we could have a deprecation period. > > > > Details aside, as PyPi continues to grow, we really need a way to > > clear out the abandoned stuff -- the barrier to entry for creating a > > new name on PyPi is just too low. > > > > This is all too late for MyPy, but it has certainly come up before, > > and will again, more and more. > > > > -CHB > > _______________________________________________ > > Distutils-SIG maillist - Distutils-SIG at python.org > > https://mail.python.org/mailman/listinfo/distutils-sig > > > > -- > Jim Fulton > http://jimfulton.info > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Mon Apr 18 12:34:09 2016 From: chris.barker at noaa.gov (Chris Barker) Date: Mon, 18 Apr 2016 09:34:09 -0700 Subject: [Distutils] The mypy package In-Reply-To: <20160418154820.GA10377@k3> References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> <-3275783620647160143@unknownmsgid> <20160418154820.GA10377@k3> Message-ID: On Mon, Apr 18, 2016 at 8:48 AM, David Wilson wrote: > Namespaces seem like a great idea, then these problems disappear > entirely, huh? as far as I can tell, namespaces greatly expand the pool of available names, but other than that, we've got the same problem. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From tritium-list at sdamon.com Mon Apr 18 12:37:12 2016 From: tritium-list at sdamon.com (Alexander Walters) Date: Mon, 18 Apr 2016 12:37:12 -0400 Subject: [Distutils] The mypy package In-Reply-To: References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> <-3275783620647160143@unknownmsgid> <20160418154820.GA10377@k3> Message-ID: <57150D38.3010102@sdamon.com> Greatly expanding the pool of names solves the problem. On 4/18/2016 12:34, Chris Barker wrote: > On Mon, Apr 18, 2016 at 8:48 AM, David Wilson > > wrote: > > Namespaces seem like a great idea, then these problems disappear > entirely, > > > huh? as far as I can tell, namespaces greatly expand the pool of > available names, but other than that, we've got the same problem. > > -CHB > > > > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chris.Barker at noaa.gov > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From dw+distutils-sig at hmmz.org Mon Apr 18 11:48:20 2016 From: dw+distutils-sig at hmmz.org (David Wilson) Date: Mon, 18 Apr 2016 15:48:20 +0000 Subject: [Distutils] The mypy package In-Reply-To: <-3275783620647160143@unknownmsgid> References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> <-3275783620647160143@unknownmsgid> Message-ID: <20160418154820.GA10377@k3> On Mon, Apr 18, 2016 at 08:18:37AM -0700, Chris Barker - NOAA Federal wrote: > We really should have SOME way to determine if a PyPi name has been > abandoned. Or even be proactive--PyPi names must be maintained in SOME > way, perhaps: +1 > Respond to some sort of "do you still want this" email. At least once a year. +0.. this is along the right track but still seems too invasive for what is mostly an edge case. I'm interested in this conversation as I have two package names registered on PyPy for unreleased projects, one with months of work spanning years put into it (but not yet fit for release) and another with actual years put into it. I'd be disappointed to lack the ability to prevent either name being annexed for someone's weekend project, although life would continue just fine if this were to occur. :) > Details aside, as PyPi continues to grow, we really need a way to > clear out the abandoned stuff -- the barrier to entry for creating a > new name on PyPi is just too low. Namespaces seem like a great idea, then these problems disappear entirely, e.g. have the server consult a one-time-generated list of aliases should a package name be requested that is not prefixed with an alias, and insist any new registrations include one. David From dw+distutils-sig at hmmz.org Mon Apr 18 12:41:33 2016 From: dw+distutils-sig at hmmz.org (David Wilson) Date: Mon, 18 Apr 2016 16:41:33 +0000 Subject: [Distutils] The mypy package In-Reply-To: References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> <-3275783620647160143@unknownmsgid> <20160418154820.GA10377@k3> Message-ID: <20160418164133.GA27753@k3> On Mon, Apr 18, 2016 at 09:34:09AM -0700, Chris Barker wrote: > Namespaces seem like a great idea, then these problems disappear > entirely, > huh? as far as I can tell, namespaces greatly expand the pool of available > names, but other than that, we've got the same problem. They seem to have worked well enough from the 1980s through to the 3.5bn or so Internet users we have today. David From tritium-list at sdamon.com Mon Apr 18 13:37:28 2016 From: tritium-list at sdamon.com (Alexander Walters) Date: Mon, 18 Apr 2016 13:37:28 -0400 Subject: [Distutils] Name arbitration on PyPI (was: The mypy package) In-Reply-To: References: Message-ID: <57151B58.1090305@sdamon.com> The idea of expiring out names has been brought up recently to resolve an issue of two packages, one popular and large; another someone's weekend project. The general idea being that a project maintainer should be forced to renew their contact information, or face the possibility of the PyPI name they registered being de-registered and made available for another package to use. Preamble done, let me enumerate why this is just a disaster: 1. PyYAML is a package that would be de-registered in such a scheme. It is a highly used, extremely popular, package that unserializes text into arbitrary python objects. It is a trusted package... and one that hasn't been active in ages. This is prime malware bait. 2. the package tooling already assumes that names will always point to one, and only one package. ever. until the heat death of the universe or the death of the language whichever is first. If I am the one person in the world who actually depends on the 'mypy' (not mypy-lang) package, you have broken that trust. 3. Who in the PSF really wants that bureaucratic nightmare of arbitrating cases when this inevitably messes up, be this system manual or automatic? To the specifics of the mypy-lang package that brought this up... It's like naming your company "Yahoo", and getting upset that yahoo.com is getting a bump in traffic because of your popularity. It is unfortunate that the mypy-lang developers failed to check pypi for name availability before they named their package, but it is by no means a reason to invite malicious code into the index, break the trust of the tooling, or create a bureaucracy to manage when the first two happen. From chris.barker at noaa.gov Mon Apr 18 16:47:09 2016 From: chris.barker at noaa.gov (Chris Barker) Date: Mon, 18 Apr 2016 13:47:09 -0700 Subject: [Distutils] The mypy package In-Reply-To: <57150BB4.8080101@sdamon.com> References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> <-3275783620647160143@unknownmsgid> <57150BB4.8080101@sdamon.com> Message-ID: On Mon, Apr 18, 2016 at 9:30 AM, Alexander Walters wrote: > We absolutely do not. Names are first come, first serve, in perpetuity. I'm suggesting that the "in perpetuity" bit is NOT a good way to go -- packages are abandoned, and the longer this goes on, the more issues will arise. > Changing this changes the security model of pypi. If all an attacker has > to do is wait out an old, but still highly downloaded package... why > wouldn't they do it? I'd suggest that a highly downloaded package isn't abandoned. granted, it may be hard to tell, but I image any package that is frequently, or even occasionally, downloaded would have *someone* willing to act as maintainer -- which, at a minimum, is simply replying to an email once a year or so saying "yes, this is still an active package" All that being said -- yes, we wouldn't want to provide an avenue for someone to post malware to the exact same download-ability as a previously valid package. But there has GOT to be a solution to that -- maybe a vetting porcess for re-using names? This really isn't going to come up all that often. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Mon Apr 18 16:54:31 2016 From: chris.barker at noaa.gov (Chris Barker) Date: Mon, 18 Apr 2016 13:54:31 -0700 Subject: [Distutils] The mypy package In-Reply-To: <20160418164133.GA27753@k3> References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> <-3275783620647160143@unknownmsgid> <20160418154820.GA10377@k3> <20160418164133.GA27753@k3> Message-ID: On Mon, Apr 18, 2016 at 9:41 AM, David Wilson wrote: > > huh? as far as I can tell, namespaces greatly expand the pool of > available > > names, but other than that, we've got the same problem. > > They seem to have worked well enough from the 1980s through to the 3.5bn > or so Internet users we have today. > but people don't get domain names in perpetuity without maintaining them -- not the same thing. And the problem at hand is that the mypy folks don't to use a different name, and it would better serve the community if an abandoned name could be used by a more useful, productive package. BTW -- as of the security issue -- isn't it the name with domain names? Someone puts up a website with something at least a little useful on it. They abandon the domain A malicious individual puts up up new site with that name that is a clone of the old one except that it distributed Malware. I agree that it would be a bit easier to do with PyPi, as it does specifically distribute software, but the risk is there in any case. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Mon Apr 18 17:00:20 2016 From: chris.barker at noaa.gov (Chris Barker) Date: Mon, 18 Apr 2016 14:00:20 -0700 Subject: [Distutils] The mypy package In-Reply-To: <57150D38.3010102@sdamon.com> References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> <-3275783620647160143@unknownmsgid> <20160418154820.GA10377@k3> <57150D38.3010102@sdamon.com> Message-ID: On Mon, Apr 18, 2016 at 9:37 AM, Alexander Walters wrote: > Greatly expanding the pool of names solves the problem. > some of it, maybe, but not the problem at hand -- mypy has already put itself up as "mypy-lang", an namespace would be pretty much the same thing. if you do pip search mpypi, you get a handful of results, two of which are: mypy (0.256) - A wsgi framework ... mypy-lang (0.2.0) - Optional static typing for Python if we're OK with that, we're already done. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Mon Apr 18 17:16:40 2016 From: chris.barker at noaa.gov (Chris Barker) Date: Mon, 18 Apr 2016 14:16:40 -0700 Subject: [Distutils] Name arbitration on PyPI (was: The mypy package) In-Reply-To: <57151B58.1090305@sdamon.com> References: <57151B58.1090305@sdamon.com> Message-ID: You've made a strong case, and this is probably not where PyPi should go -- but it would hardly be a disaster: > The idea of expiring out names has been brought up recently to resolve an > issue of two packages, one popular and large; another someone's weekend > project. The issue here is not that it's a weekend project, but that it may be an abandoned project. I don't think anyone suggest that we should have a popularity or quality test to see who gets to trump whom with name allocation -- I sure didn't. Which is quite relevant to below: > 1. PyYAML is a package that would be de-registered in such a scheme. It > is a highly used, extremely popular, package that unserializes text into > arbitrary python objects. It is a trusted package... and one that hasn't > been active in ages. and you don't think ANYONE would be willing to take on the miniscule amount of work to maintain the name? Plus there would be any number of other schemes for determining whether a project name is abandoned. > 2. the package tooling already assumes that names will always point to > one, and only one package. ever. until the heat death of the universe or > the death of the language whichever is first. IIUC, the current scheme allows for a name to be "taken over" by a new package if the original author so desires -- i.e. if the current owner of the mypy name was happy to relinquish it, then "pip install mypy" would get users something totally different 6 months from now. So no -- we don't currently guarantee anything about future use of names. Other that that the original author can do whatever they want with it. 3. Who in the PSF really wants that bureaucratic nightmare of arbitrating > cases when this inevitably messes up, be this system manual or automatic? > I think bureaucratic nightmare is pretty hyperbolic, but yes, there would be some overhead, for sure. And given that no one has the time and motivation to even maintain PyPi at this point -- this will probably kill the idea altogether. To the specifics of the mypy-lang package that brought this up... It's like > naming your company "Yahoo", and getting upset that yahoo.com is getting > a bump in traffic because of your popularity. Again - this is not about minor weekend projects not be "important". It's about potential abandonware -- with domain registration, "Yahoo" can offer to buy the domain from the current holder, or, if the current owner has abandoned it, it will go into the public pool again when they stop paying to maintain the registration. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From graffatcolmingov at gmail.com Mon Apr 18 17:31:49 2016 From: graffatcolmingov at gmail.com (Ian Cordasco) Date: Mon, 18 Apr 2016 16:31:49 -0500 Subject: [Distutils] Name arbitration on PyPI (was: The mypy package) In-Reply-To: References: <57151B58.1090305@sdamon.com> Message-ID: On Mon, Apr 18, 2016 at 4:16 PM, Chris Barker wrote: > You've made a strong case, and this is probably not where PyPi should go -- > but it would hardly be a disaster: > >> >> The idea of expiring out names has been brought up recently to resolve an >> issue of two packages, one popular and large; another someone's weekend >> project. > > > The issue here is not that it's a weekend project, but that it may be an > abandoned project. I don't think anyone suggest that we should have a > popularity or quality test to see who gets to trump whom with name > allocation -- I sure didn't. > > Which is quite relevant to below: > >> >> 1. PyYAML is a package that would be de-registered in such a scheme. It >> is a highly used, extremely popular, package that unserializes text into >> arbitrary python objects. It is a trusted package... and one that hasn't >> been active in ages. > > > and you don't think ANYONE would be willing to take on the miniscule amount > of work to maintain the name? Plus there would be any number of other > schemes for determining whether a project name is abandoned. I have in fact offered but the author refuses to accept help from anyone. They're also the author of the C library (libyaml) and they do not maintain that either. It's actually quite frustrating as someone who wants to fix some of the numerous bugs in the library + improve it and add support for YAML 1.2 which is years old at this point. >> >> 2. the package tooling already assumes that names will always point to >> one, and only one package. ever. until the heat death of the universe or >> the death of the language whichever is first. > > > IIUC, the current scheme allows for a name to be "taken over" by a new > package if the original author so desires -- i.e. if the current owner of > the mypy name was happy to relinquish it, then "pip install mypy" would get > users something totally different 6 months from now. So no -- we don't > currently guarantee anything about future use of names. Other that that the > original author can do whatever they want with it. > >> 3. Who in the PSF really wants that bureaucratic nightmare of arbitrating >> cases when this inevitably messes up, be this system manual or automatic? > > > I think bureaucratic nightmare is pretty hyperbolic, but yes, there would be > some overhead, for sure. And given that no one has the time and motivation > to even maintain PyPi at this point -- this will probably kill the idea > altogether. > > >> To the specifics of the mypy-lang package that brought this up... It's >> like naming your company "Yahoo", and getting upset that yahoo.com is >> getting a bump in traffic because of your popularity. > > > Again - this is not about minor weekend projects not be "important". It's > about potential abandonware -- with domain registration, "Yahoo" can offer > to buy the domain from the current holder, or, if the current owner has > abandoned it, it will go into the public pool again when they stop paying to > maintain the registration. > > -CHB > > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chris.Barker at noaa.gov > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > From chris.barker at noaa.gov Mon Apr 18 17:56:53 2016 From: chris.barker at noaa.gov (Chris Barker) Date: Mon, 18 Apr 2016 14:56:53 -0700 Subject: [Distutils] Name arbitration on PyPI (was: The mypy package) In-Reply-To: References: <57151B58.1090305@sdamon.com> Message-ID: On Mon, Apr 18, 2016 at 2:31 PM, Ian Cordasco wrote: > >> 1. PyYAML is a package that would be de-registered in such a scheme. > It > > > and you don't think ANYONE would be willing to take on the miniscule > amount > > of work to maintain the name? Plus there would be any number of other > > schemes for determining whether a project name is abandoned. > > I have in fact offered but the author refuses to accept help from > anyone. They're also the author of the C library (libyaml) and they do > not maintain that either. It's actually quite frustrating as someone > who wants to fix some of the numerous bugs in the library + improve it > and add support for YAML 1.2 which is years old at this point. Interesting third case I hadn't considered -- the original author is still "active", but not actually maintaining the software or accepting help. I don't think there is anything PyPi policy can do about that -- too bad. Time for a fork? -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.gronholm at nextday.fi Mon Apr 18 18:07:01 2016 From: alex.gronholm at nextday.fi (=?UTF-8?Q?Alex_Gr=c3=b6nholm?=) Date: Tue, 19 Apr 2016 01:07:01 +0300 Subject: [Distutils] Name arbitration on PyPI In-Reply-To: References: <57151B58.1090305@sdamon.com> Message-ID: <57155A85.5080205@nextday.fi> Another high profile example of such a project: PIL. 19.04.2016, 00:56, Chris Barker kirjoitti: > On Mon, Apr 18, 2016 at 2:31 PM, Ian Cordasco > > wrote: > > >> 1. PyYAML is a package that would be de-registered in such a scheme. It > > > and you don't think ANYONE would be willing to take on the miniscule amount > > of work to maintain the name? Plus there would be any number of > other > > schemes for determining whether a project name is abandoned. > > I have in fact offered but the author refuses to accept help from > anyone. They're also the author of the C library (libyaml) and they do > not maintain that either. It's actually quite frustrating as someone > who wants to fix some of the numerous bugs in the library + improve it > and add support for YAML 1.2 which is years old at this point. > > > Interesting third case I hadn't considered -- the original author is > still "active", but not actually maintaining the software or accepting > help. I don't think there is anything PyPi policy can do about that -- > too bad. > > Time for a fork? > > -CHB > > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chris.Barker at noaa.gov > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From glyph at twistedmatrix.com Mon Apr 18 18:11:00 2016 From: glyph at twistedmatrix.com (Glyph) Date: Mon, 18 Apr 2016 15:11:00 -0700 Subject: [Distutils] The mypy package In-Reply-To: References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> <-3275783620647160143@unknownmsgid> <20160418154820.GA10377@k3> <57150D38.3010102@sdamon.com> Message-ID: <10E4582F-F1A2-4E93-BF0B-A6F500EAE571@twistedmatrix.com> > On Apr 18, 2016, at 2:00 PM, Chris Barker wrote: > > On Mon, Apr 18, 2016 at 9:37 AM, Alexander Walters > wrote: > Greatly expanding the pool of names solves the problem. > > some of it, maybe, but not the problem at hand -- mypy has already put itself up as "mypy-lang", an namespace would be pretty much the same thing. > > if you do pip search mpypi, you get a handful of results, two of which are: > > mypy (0.256) - A wsgi framework > ... > mypy-lang (0.2.0) - Optional static typing for Python > > if we're OK with that, we're already done. I think there's still a general question here about orphaning packages, but maybe in this very specific case a simple name change is in order? 'mypy' never really made much sense to me as "python with types"; for many months after I started hearing the name, I thought it was some kind of personalizable / portable distribution of Python, or maybe a macro system (allowing you to personalize python to your tastes). Might I suggest 'typy' for TYped PYthon? Nothing shows up in a pip search for that just yet. Just a thought, -glyph -------------- next part -------------- An HTML attachment was scrubbed... URL: From donald at stufft.io Mon Apr 18 18:11:06 2016 From: donald at stufft.io (Donald Stufft) Date: Mon, 18 Apr 2016 18:11:06 -0400 Subject: [Distutils] Name arbitration on PyPI (was: The mypy package) In-Reply-To: References: <57151B58.1090305@sdamon.com> Message-ID: <39B26098-F2C4-43B9-9A5C-FF8B06180C37@stufft.io> > On Apr 18, 2016, at 5:16 PM, Chris Barker wrote: > > 1. PyYAML is a package that would be de-registered in such a scheme. It is a highly used, extremely popular, package that unserializes text into arbitrary python objects. It is a trusted package... and one that hasn't been active in ages. > > and you don't think ANYONE would be willing to take on the miniscule amount of work to maintain the name? Plus there would be any number of other schemes for determining whether a project name is abandoned. This sort of gets to the core of one of the major questions that has to be answered before you talk about expiring names and such. Who ?owns? a name on PyPI? Right now, as implemented, the people who ?own? (in terms of ACL) the name on PyPI are the owners of said name. In this vein PyPI has traditionally been first come first serve on names and generally will not release a name to someone else without the current owner?s permission [1]. This leads to projects, even popular ones, sometimes falling into abandonment, sometimes even when you have a new maintainer willing to step up (as is the case with Ian and pyYAML) or in some cases, has already forked the project under another name (as is the case with PIL and Pillow). However, this also means that in general, if you decide you trust the current owner of something on PyPI, you know that it?s not suddenly going to be owned by someone else *unless* the person who you previously trusted decides to give it away to them (and thus, implicitly transferring the trust you granted them to another person). So, before we try and think up some mechanism for expiring names, or forcibly taking a name from the current owner and giving it to another, we [2] would first need to decide that names on PyPI are not owned by the individuals who hold the registration, but are instead owned by the community and the current individuals are only borrowing them. This would mean that instead of having packages wither and get abandoned we could instead transfer them to new maintainers who can keep the project going. However, that of course raises new questions like: * Should we only transfer projects to maintainers who want to keep the current project going? Such as with PIL or pyYAML? - This will be generally less surprising to end users, but it doesn?t solve the problem of rarely used packages ?blocking? other?s use of the name. * How do we determine who to give a name to? Surely we shouldn?t hand off a package that people are using to the first random passerby who happens to ask for it and we can probably trust Guido not to be malicious (or else we have bigger problems ;) ), but how do we determine if someone is to be trusted when they inevitably fall somewhere in between? * The current policy is pretty black and white with only a little bit of grey area (what is a ?real? package that?s been uploaded to a project, what is a malicious package, etc), but moving over to community owned names would open things up to a far wider set of grey areas, and who is going to arbitrate when a project is abandoned or when it?s ?popular? or not? - If folks are unaware, npmjs.org just recently had a bit of a kerfuffle over a package on their repository going missing, ?left-pad?, due to the grey-ness of their policy. - The current, fairly rigid policy came around because the PyPI admins decided to give what appeared to be an unmaintained library to someone new who had forked the library, after the original author hadn?t responded to an email. This ended up causing our own kerfuffle when it turned out this person *wasn?t* absent and *wasn?t* OK with this. [3]. * If we start giving projects out to other people, what tools should we give people to deal with that? - The latest version of pip has a mode that allows you to specify a number of hashes and it won?t install anything that doesn?t match a hash. That could protect end users from installing something inadvertently, but it?s very opt in and requires hard pins ``==`` so it?s not likely to get a large uptick. - If we mandated semver (or something like it) we could make it so that transferring a name *forced* a major version bump and the new author would be unable to release anything using a smaller major version number, people could then pin to -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 842 bytes Desc: Message signed with OpenPGP using GPGMail URL: From glyph at twistedmatrix.com Mon Apr 18 18:14:03 2016 From: glyph at twistedmatrix.com (Glyph) Date: Mon, 18 Apr 2016 15:14:03 -0700 Subject: [Distutils] Name arbitration on PyPI (was: The mypy package) In-Reply-To: References: <57151B58.1090305@sdamon.com> Message-ID: <741971F7-F67F-414D-8147-615408C91FB5@twistedmatrix.com> > On Apr 18, 2016, at 2:31 PM, Ian Cordasco wrote: > > I have in fact offered but the author refuses to accept help from > anyone. They're also the author of the C library (libyaml) and they do > not maintain that either. It's actually quite frustrating as someone > who wants to fix some of the numerous bugs in the library + improve it > and add support for YAML 1.2 which is years old at this point. Since the spectre of malware has been raised in this thread, I feel I should point out that the reverse is also true. Although libyaml / pyyaml are "trusted" today, what happens after the inevitable 0-day RCE drops which the author refuses to patch it? Does PyPI have a responsibility to re-assign the name in that case? Specifically, YAML does have a heritage of vulnerabilities, even if this specific instance doesn't. -glyph -------------- next part -------------- An HTML attachment was scrubbed... URL: From glyph at twistedmatrix.com Mon Apr 18 18:16:18 2016 From: glyph at twistedmatrix.com (Glyph) Date: Mon, 18 Apr 2016 15:16:18 -0700 Subject: [Distutils] Name arbitration on PyPI (was: The mypy package) In-Reply-To: <39B26098-F2C4-43B9-9A5C-FF8B06180C37@stufft.io> References: <57151B58.1090305@sdamon.com> <39B26098-F2C4-43B9-9A5C-FF8B06180C37@stufft.io> Message-ID: <194A8A26-8D0D-4853-8AE6-ACA207D17843@twistedmatrix.com> > On Apr 18, 2016, at 3:11 PM, Donald Stufft wrote: > > If we mandated semver (or something like it) we could make it so that transferring a name *forced* a major version bump and the new author would be unable to release anything using a smaller major version number, people could then pin to From alex.gronholm at nextday.fi Mon Apr 18 18:17:05 2016 From: alex.gronholm at nextday.fi (=?UTF-8?Q?Alex_Gr=c3=b6nholm?=) Date: Tue, 19 Apr 2016 01:17:05 +0300 Subject: [Distutils] The mypy package In-Reply-To: <10E4582F-F1A2-4E93-BF0B-A6F500EAE571@twistedmatrix.com> References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> <-3275783620647160143@unknownmsgid> <20160418154820.GA10377@k3> <57150D38.3010102@sdamon.com> <10E4582F-F1A2-4E93-BF0B-A6F500EAE571@twistedmatrix.com> Message-ID: <57155CE1.5060008@nextday.fi> This name is unfortunately a bit awkward in the author's native language -- it is the colloquial word for "babe" or "broad" :) 19.04.2016, 01:11, Glyph kirjoitti: > >> On Apr 18, 2016, at 2:00 PM, Chris Barker > > wrote: >> >> On Mon, Apr 18, 2016 at 9:37 AM, Alexander Walters >> > wrote: >> >> Greatly expanding the pool of names solves the problem. >> >> >> some of it, maybe, but not the problem at hand -- mypy has already >> put itself up as "mypy-lang", an namespace would be pretty much the >> same thing. >> >> if you do pip search mpypi, you get a handful of results, two of >> which are: >> >> mypy (0.256) - A wsgi framework >> ... >> mypy-lang (0.2.0) - Optional static typing for Python >> >> if we're OK with that, we're already done. > > I think there's still a general question here about orphaning > packages, but maybe in this very specific case a simple name change is > in order? 'mypy' never really made much sense to me as "python with > types"; for many months after I started hearing the name, I thought it > was some kind of personalizable / portable distribution of Python, or > maybe a macro system (allowing you to personalize python to your tastes). > > Might I suggest 'typy' for TYped PYthon? Nothing shows up in a pip > search for that just yet. > > Just a thought, > > -glyph > > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From contact at ionelmc.ro Mon Apr 18 18:21:07 2016 From: contact at ionelmc.ro (=?UTF-8?Q?Ionel_Cristian_M=C4=83rie=C8=99?=) Date: Tue, 19 Apr 2016 01:21:07 +0300 Subject: [Distutils] The mypy package In-Reply-To: References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> <-3275783620647160143@unknownmsgid> <57150BB4.8080101@sdamon.com> Message-ID: On Mon, Apr 18, 2016 at 11:47 PM, Chris Barker wrote: > > I'm suggesting that the "in perpetuity" bit is NOT a good way to go -- > packages are abandoned, and the longer this goes on, the more issues will > arise. > > ?Problem is cat's out of the bag here. There are three issues: - Can't just change the rules underneath everyone. If we'd be making a package repository today, it would be fine - everything would know the rules. There's a huge surprise factor here and a recipe for drama if this is changed. If we'd change this perpetuity rule it wouldn't be possible to let everyone know about it - read receipts for email don't really exist. - Would taking mypy package from random Chinese dude no one seems to care about be fair? First release of mypy is in 2009 while mypy-lang's first commit is in? 2012. Jukka would had done well if he'd used a different name for the project, or resolved the ownership issue back then. - Where do you draw the line for "abandoned"? Whom would you allow to confiscate ownership? It's impossible to come up with a non-arbitrary set of rules. Plus I'm pretty sure the Chinese dude didn't even read or understood the mail - we're talking about taking his package while he didn't even reply. Seriously? Give it time it will sort itself out. Thanks, -- Ionel Cristian M?rie?, http://blog.ionelmc.ro -------------- next part -------------- An HTML attachment was scrubbed... URL: From glyph at twistedmatrix.com Mon Apr 18 18:21:31 2016 From: glyph at twistedmatrix.com (Glyph) Date: Mon, 18 Apr 2016 15:21:31 -0700 Subject: [Distutils] The mypy package In-Reply-To: <57155CE1.5060008@nextday.fi> References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> <-3275783620647160143@unknownmsgid> <20160418154820.GA10377@k3> <57150D38.3010102@sdamon.com> <10E4582F-F1A2-4E93-BF0B-A6F500EAE571@twistedmatrix.com> <57155CE1.5060008@nextday.fi> Message-ID: > On Apr 18, 2016, at 3:17 PM, Alex Gr?nholm wrote: > > This name is unfortunately a bit awkward in the author's native language -- it is the colloquial word for "babe" or "broad" :) OK, I didn't see that one coming :-). "stapy", then? "static type annotation python"? -glyph -------------- next part -------------- An HTML attachment was scrubbed... URL: From donald at stufft.io Mon Apr 18 18:21:32 2016 From: donald at stufft.io (Donald Stufft) Date: Mon, 18 Apr 2016 18:21:32 -0400 Subject: [Distutils] Name arbitration on PyPI (was: The mypy package) In-Reply-To: <741971F7-F67F-414D-8147-615408C91FB5@twistedmatrix.com> References: <57151B58.1090305@sdamon.com> <741971F7-F67F-414D-8147-615408C91FB5@twistedmatrix.com> Message-ID: > On Apr 18, 2016, at 6:14 PM, Glyph wrote: > > >> On Apr 18, 2016, at 2:31 PM, Ian Cordasco > wrote: >> >> I have in fact offered but the author refuses to accept help from >> anyone. They're also the author of the C library (libyaml) and they do >> not maintain that either. It's actually quite frustrating as someone >> who wants to fix some of the numerous bugs in the library + improve it >> and add support for YAML 1.2 which is years old at this point. > > Since the spectre of malware has been raised in this thread, I feel I should point out that the reverse is also true. Although libyaml / pyyaml are "trusted" today, what happens after the inevitable 0-day RCE drops which the author refuses to patch it? Does PyPI have a responsibility to re-assign the name in that case? Specifically, YAML does have a heritage of vulnerabilities, even if this specific instance doesn't. > We don?t currently have much in the way of mechanisms to deal with that. Although I could think of a few that we could do which *wouldn?t* require handing over the name and which could generalize out to other maintenance/abandonment problems as well, like (in order of severity): * Add a warning on the PyPI page indicating that the project is abandoned/unmaintained/etc suggesting they find something else (possibly with specific suggestions, like PIL -> Pillow). * Add some mechanism to pip/PyPI that would allow PyPI to provide a message to people installing a particular project (or perhaps a specific version). This could also be exposed to authors who want to mark specific versions of their project as insecure. * Delete the files from PyPI or otherwise prevent them from being discovered by pip (likely paired with the a warning of some kind on the PyPI page). ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 842 bytes Desc: Message signed with OpenPGP using GPGMail URL: From glyph at twistedmatrix.com Mon Apr 18 18:24:32 2016 From: glyph at twistedmatrix.com (Glyph) Date: Mon, 18 Apr 2016 15:24:32 -0700 Subject: [Distutils] Name arbitration on PyPI (was: The mypy package) In-Reply-To: References: <57151B58.1090305@sdamon.com> <741971F7-F67F-414D-8147-615408C91FB5@twistedmatrix.com> Message-ID: > On Apr 18, 2016, at 3:21 PM, Donald Stufft wrote: > >> >> On Apr 18, 2016, at 6:14 PM, Glyph > wrote: >> >> >>> On Apr 18, 2016, at 2:31 PM, Ian Cordasco > wrote: >>> >>> I have in fact offered but the author refuses to accept help from >>> anyone. They're also the author of the C library (libyaml) and they do >>> not maintain that either. It's actually quite frustrating as someone >>> who wants to fix some of the numerous bugs in the library + improve it >>> and add support for YAML 1.2 which is years old at this point. >> >> Since the spectre of malware has been raised in this thread, I feel I should point out that the reverse is also true. Although libyaml / pyyaml are "trusted" today, what happens after the inevitable 0-day RCE drops which the author refuses to patch it? Does PyPI have a responsibility to re-assign the name in that case? Specifically, YAML does have a heritage of vulnerabilities, even if this specific instance doesn't. >> > > We don?t currently have much in the way of mechanisms to deal with that. Although I could think of a few that we could do which *wouldn?t* require handing over the name and which could generalize out to other maintenance/abandonment problems as well, like (in order of severity): > > * Add a warning on the PyPI page indicating that the project is abandoned/unmaintained/etc suggesting they find something else (possibly with specific suggestions, like PIL -> Pillow). This is the sort of thing I had in mind with https://github.com/pypa/warehouse/issues/933 - it seems like any kind of annotation like this should be a matter of last resort and authors should be given every opportunity to respond first. > > * Add some mechanism to pip/PyPI that would allow PyPI to provide a message to people installing a particular project (or perhaps a specific version). This could also be exposed to authors who want to mark specific versions of their project as insecure. > > * Delete the files from PyPI or otherwise prevent them from being discovered by pip (likely paired with the a warning of some kind on the PyPI page). > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Mon Apr 18 18:49:14 2016 From: chris.barker at noaa.gov (Chris Barker - NOAA Federal) Date: Mon, 18 Apr 2016 15:49:14 -0700 Subject: [Distutils] Name arbitration on PyPI In-Reply-To: <1393180650.7600.1461019677522.JavaMail.mobile-sync@iecvi8> References: <57151B58.1090305@sdamon.com> <1393180650.7600.1461019677522.JavaMail.mobile-sync@iecvi8> Message-ID: <5651377664198208096@unknownmsgid> Another high profile example of such a project: PIL. Was PIL ever on PyPi? Anyway, yup, the solution there was to fork give it s new name -- Pillow was born. CHB 19.04.2016, 00:56, Chris Barker kirjoitti: On Mon, Apr 18, 2016 at 2:31 PM, Ian Cordasco wrote: > >> 1.??? PyYAML is a package that would be de-registered in such a > scheme.??? It > ??? > > and you don't think ANYONE would be willing to take on the miniscule > amount > > of work to maintain the name? Plus there would be any number of other > > schemes for determining whether a project name is abandoned. > > I have in fact offered but the author refuses to accept help from > anyone. They're also the author of the C library (libyaml) and they do > not maintain that either. It's actually quite frustrating as someone > who wants to fix some of the numerous bugs in the library + improve it > and add support for YAML 1.2 which is years old at this point. Interesting third case I hadn't considered -- the original author is still "active", but not actually maintaining the software or accepting help. I don't think there is anything PyPi policy can do about that -- too bad. Time for a fork???? -CHB ??? -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R ??? ??? ??? ??? ??? ???(206) 526-6959?????? voice 7600 Sand Point Way NE ??????(206) 526-6329?????? fax Seattle, WA ???98115 ??? ??? ??????(206) 526-6317?????? main reception Chris.Barker at noaa.gov _______________________________________________ Distutils-SIG maillist - Distutils-SIG at python.orghttps://mail.python.org/mailman/listinfo/distutils-sig _______________________________________________ Distutils-SIG maillist - Distutils-SIG at python.org https://mail.python.org/mailman/listinfo/distutils-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.gronholm at nextday.fi Mon Apr 18 18:56:43 2016 From: alex.gronholm at nextday.fi (=?UTF-8?Q?Alex_Gr=c3=b6nholm?=) Date: Tue, 19 Apr 2016 01:56:43 +0300 Subject: [Distutils] Name arbitration on PyPI In-Reply-To: <5651377664198208096@unknownmsgid> References: <57151B58.1090305@sdamon.com> <1393180650.7600.1461019677522.JavaMail.mobile-sync@iecvi8> <5651377664198208096@unknownmsgid> Message-ID: <5715662B.9010703@nextday.fi> Here: https://pypi.python.org/pypi/PIL It was downloadable until external downloads were disallowed. 19.04.2016, 01:49, Chris Barker - NOAA Federal kirjoitti: > >> Another high profile example of such a project: PIL. > > Was PIL ever on PyPi? Anyway, yup, the solution there was to fork give > it s new name -- Pillow was born. > > CHB > > >> 19.04.2016, 00:56, Chris Barker kirjoitti: >>> On Mon, Apr 18, 2016 at 2:31 PM, Ian Cordasco >>> > wrote: >>> >>> >> 1.??? PyYAML is a package that would be de-registered in such a scheme.??? It >>> >>> ??? >>> >>> > and you don't think ANYONE would be willing to take on the miniscule amount >>> > of work to maintain the name? Plus there would be any number >>> of other >>> > schemes for determining whether a project name is abandoned. >>> >>> I have in fact offered but the author refuses to accept help from >>> anyone. They're also the author of the C library (libyaml) and >>> they do >>> not maintain that either. It's actually quite frustrating as someone >>> who wants to fix some of the numerous bugs in the library + >>> improve it >>> and add support for YAML 1.2 which is years old at this point. >>> >>> >>> Interesting third case I hadn't considered -- the original author is >>> still "active", but not actually maintaining the software or >>> accepting help. I don't think there is anything PyPi policy can do >>> about that -- too bad. >>> >>> Time for a fork???? >>> >>> -CHB >>> >>> >>> ??? >>> -- >>> >>> Christopher Barker, Ph.D. >>> Oceanographer >>> >>> Emergency Response Division >>> NOAA/NOS/OR&R ??? ??? ??? ??? ??? ???(206) 526-6959?????? voice >>> 7600 Sand Point Way NE ??????(206) 526-6329?????? fax >>> Seattle, WA ???98115 ??? ??? ??????(206) 526-6317?????? main reception >>> >>> Chris.Barker at noaa.gov >>> >>> >>> _______________________________________________ >>> Distutils-SIG maillist -Distutils-SIG at python.org >>> https://mail.python.org/mailman/listinfo/distutils-sig >> >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> >> https://mail.python.org/mailman/listinfo/distutils-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From tritium-list at sdamon.com Mon Apr 18 20:34:15 2016 From: tritium-list at sdamon.com (Alexander Walters) Date: Mon, 18 Apr 2016 20:34:15 -0400 Subject: [Distutils] Name arbitration on PyPI In-Reply-To: <194A8A26-8D0D-4853-8AE6-ACA207D17843@twistedmatrix.com> References: <57151B58.1090305@sdamon.com> <39B26098-F2C4-43B9-9A5C-FF8B06180C37@stufft.io> <194A8A26-8D0D-4853-8AE6-ACA207D17843@twistedmatrix.com> Message-ID: <57157D07.1040500@sdamon.com> Here here. As to the concept of a name being community owned vs. registrant owned - I am very opposed to changing the rules mid-stream. If community ownership is to be a thing (and I do not want it to be a thing at all, but if it is), it should be *from this point forward* the names are community owned. On 4/18/2016 18:16, Glyph wrote: > >> On Apr 18, 2016, at 3:11 PM, Donald Stufft > > wrote: >> >> If we mandated semver (or something like it) we could make it so that >> transferring a name *forced* a major version bump and the new author >> would be unable to release anything using a smaller major version >> number, people could then pin to > somebody new won?t take over a project without their knowing about >> it. However we haven?t historically mandated this and there are a lot >> of projects using date based versions that would b very unhappy (in >> addition to the fact upper pins often are major contributors to being >> unable to resolve a version tree of dependencies). > > Just want to raise my hand to be counted here among the people that > would be /extremely/ unhappy if PyPI started mandating semver. > > -glyph > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From tritium-list at sdamon.com Mon Apr 18 20:52:14 2016 From: tritium-list at sdamon.com (Alexander Walters) Date: Mon, 18 Apr 2016 20:52:14 -0400 Subject: [Distutils] The mypy package In-Reply-To: References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> <-3275783620647160143@unknownmsgid> <20160418154820.GA10377@k3> <57150D38.3010102@sdamon.com> <10E4582F-F1A2-4E93-BF0B-A6F500EAE571@twistedmatrix.com> <57155CE1.5060008@nextday.fi> Message-ID: <5715813E.7030105@sdamon.com> I described at a high level what mypy-lang does to my wife, and the brief history of this issue. Her blerted out solution was to "just change mypy-lang to annopy" (for annotated python). I am required by marital obligation to bring that forward. On 4/18/2016 18:21, Glyph wrote: > >> On Apr 18, 2016, at 3:17 PM, Alex Gr?nholm > > wrote: >> >> This name is unfortunately a bit awkward in the author's native >> language -- it is the colloquial word for "babe" or "broad" :) > > OK, I didn't see that one coming :-). > > "stapy", then? "static type annotation python"? > > -glyph > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Apr 19 10:59:11 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 20 Apr 2016 00:59:11 +1000 Subject: [Distutils] The mypy package In-Reply-To: References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> <-3275783620647160143@unknownmsgid> Message-ID: On 19 April 2016 at 01:21, Jim Fulton wrote: > I suggest measuring activity by downloads, not releases. Sometimes > maintained packages are boring enough not to need releases, while many > projects depend on them. > Standard library backport packages are fairly prone to this - contextlib2 went without a release for ~4 years, for example, since it was only in 3.5 that we made sufficient changes to the standard library version that it seemed worthwhile for me to update the backport (the previous release of contextlib2 was from just prior to Python 3.3). However, as others have noted, we don't really have the resources to administer a PyPI name dispute resolution system - when there are legal issues, the PyPI admins can escalate matters to the PSF Board (but those are fortunately rare), and for other cases, establishing contact with the current owners is the most practical course of action currently available. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From opensource at ronnypfannschmidt.de Tue Apr 19 12:45:45 2016 From: opensource at ronnypfannschmidt.de (Ronny Pfannschmidt) Date: Tue, 19 Apr 2016 18:45:45 +0200 Subject: [Distutils] Name arbitration on PyPI - how about administrative abandonment/replacement meta-data In-Reply-To: References: <57151B58.1090305@sdamon.com> Message-ID: <571660B9.3040309@ronnypfannschmidt.de> Instead of overtaking, how about clearly marking packages as abandoned/maintained clearly pointing out the mark was imposed by community action and listing potential/primary replacements this would have the possibility of also supporting normal abandonment when people voluntary stop maintenance and cannot find a successor its important that community imposed abandonment is not simply removable by doing a minor "noop"-release, after all if abandonment had to be applied by proof of 3rd party, there is need of proof of continued maintenance pip could warn on installation/update -- Ronny Am 18.04.2016 um 23:31 schrieb Ian Cordasco: > On Mon, Apr 18, 2016 at 4:16 PM, Chris Barker wrote: >> You've made a strong case, and this is probably not where PyPi should go -- >> but it would hardly be a disaster: >> >>> The idea of expiring out names has been brought up recently to resolve an >>> issue of two packages, one popular and large; another someone's weekend >>> project. >> >> The issue here is not that it's a weekend project, but that it may be an >> abandoned project. I don't think anyone suggest that we should have a >> popularity or quality test to see who gets to trump whom with name >> allocation -- I sure didn't. >> >> Which is quite relevant to below: >> >>> 1. PyYAML is a package that would be de-registered in such a scheme. It >>> is a highly used, extremely popular, package that unserializes text into >>> arbitrary python objects. It is a trusted package... and one that hasn't >>> been active in ages. >> >> and you don't think ANYONE would be willing to take on the miniscule amount >> of work to maintain the name? Plus there would be any number of other >> schemes for determining whether a project name is abandoned. > I have in fact offered but the author refuses to accept help from > anyone. They're also the author of the C library (libyaml) and they do > not maintain that either. It's actually quite frustrating as someone > who wants to fix some of the numerous bugs in the library + improve it > and add support for YAML 1.2 which is years old at this point. > >>> 2. the package tooling already assumes that names will always point to >>> one, and only one package. ever. until the heat death of the universe or >>> the death of the language whichever is first. >> >> IIUC, the current scheme allows for a name to be "taken over" by a new >> package if the original author so desires -- i.e. if the current owner of >> the mypy name was happy to relinquish it, then "pip install mypy" would get >> users something totally different 6 months from now. So no -- we don't >> currently guarantee anything about future use of names. Other that that the >> original author can do whatever they want with it. >> >>> 3. Who in the PSF really wants that bureaucratic nightmare of arbitrating >>> cases when this inevitably messes up, be this system manual or automatic? >> >> I think bureaucratic nightmare is pretty hyperbolic, but yes, there would be >> some overhead, for sure. And given that no one has the time and motivation >> to even maintain PyPi at this point -- this will probably kill the idea >> altogether. >> >> >>> To the specifics of the mypy-lang package that brought this up... It's >>> like naming your company "Yahoo", and getting upset that yahoo.com is >>> getting a bump in traffic because of your popularity. >> >> Again - this is not about minor weekend projects not be "important". It's >> about potential abandonware -- with domain registration, "Yahoo" can offer >> to buy the domain from the current holder, or, if the current owner has >> abandoned it, it will go into the public pool again when they stop paying to >> maintain the registration. >> >> -CHB >> >> >> -- >> >> Christopher Barker, Ph.D. >> Oceanographer >> >> Emergency Response Division >> NOAA/NOS/OR&R (206) 526-6959 voice >> 7600 Sand Point Way NE (206) 526-6329 fax >> Seattle, WA 98115 (206) 526-6317 main reception >> >> Chris.Barker at noaa.gov >> >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> https://mail.python.org/mailman/listinfo/distutils-sig >> > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig From chris.barker at noaa.gov Tue Apr 19 18:31:26 2016 From: chris.barker at noaa.gov (Chris Barker) Date: Tue, 19 Apr 2016 15:31:26 -0700 Subject: [Distutils] The mypy package In-Reply-To: References: <57126B91.9090106@sdamon.com> <722245622.43823.1460950233370.JavaMail.mobile-sync@ioiw24> <-3275783620647160143@unknownmsgid> Message-ID: On Tue, Apr 19, 2016 at 7:59 AM, Nick Coghlan wrote: > However, as others have noted, we don't really have the resources to > administer a PyPI name dispute resolution system - when there are legal > issues, the PyPI admins can escalate matters to the PSF Board (but those > are fortunately rare), and for other cases, establishing contact with the > current owners is the most practical course of action currently available. > which doesn't help for "abandoned" packages -- maybe that's too rare to worry about though. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Tue Apr 19 18:38:58 2016 From: chris.barker at noaa.gov (Chris Barker) Date: Tue, 19 Apr 2016 15:38:58 -0700 Subject: [Distutils] Name arbitration on PyPI - how about administrative abandonment/replacement meta-data In-Reply-To: <571660B9.3040309@ronnypfannschmidt.de> References: <57151B58.1090305@sdamon.com> <571660B9.3040309@ronnypfannschmidt.de> Message-ID: On Tue, Apr 19, 2016 at 9:45 AM, Ronny Pfannschmidt < opensource at ronnypfannschmidt.de> wrote: > Instead of overtaking, > how about clearly marking packages as abandoned/maintained clearly > pointing out the mark was imposed by community action > I think that would be a good idea -- and maybe start with just that -- then we'd learn how big an issue it really was, etc. > and listing potential/primary replacements > that required real work on someone's part -- so not sure when that would actually happen. > its important that community imposed abandonment is not simply removable > by doing a minor "noop"-release, > why not? I brought tis all up to address truly abandoned projects -- maybe we want to go some day to the idea of the names being community owned, but that's not the way ti is now -- and if someone makes the effort to do a noop release, then they have no abandoned the name -- maybe aren't maintaining it worth a damn, but there you go. Personally, I think there is no point in anything between the current free for all, and a "curated" package repo -- a curated repo would support the idea that anything on it had met some minimum standard: no malware, some maintenance, some minimum usefulness, etc. It's kind of a shame that there are so many "toy" packages and experiments on PyPi, but in fact, it's worked pretty darn well.. pip could warn on installation/update > I think that would be good too -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at crwilcox.com Tue Apr 19 17:52:39 2016 From: python at crwilcox.com (Christopher Wilcox) Date: Tue, 19 Apr 2016 21:52:39 +0000 Subject: [Distutils] Parked Names in PyPI under user rodmena Message-ID: DistUtils-Sig: I was searching warehouse for all Microsoft owned packages today and came across a certain user that seems to have parked on a few different package names that I don?t believe he has any intention of using (@rodmena). https://warehouse.python.org/user/rodmena/ Can we get these released to the proper owners? He seems to have done this rather broadly. If possible can the user @Microsoft be marked as the owner of the Microsoft package? Thanks! Chris Wilcox -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve.dower at python.org Tue Apr 19 23:39:44 2016 From: steve.dower at python.org (Steve Dower) Date: Tue, 19 Apr 2016 20:39:44 -0700 Subject: [Distutils] Parked Names in PyPI under user rodmena In-Reply-To: References: Message-ID: Also the "windows" package. Might just want to release all of those package names, as he's clearly a troll, but in light of the other discussions I think a case of well established and enforceable trademarks should be straightforward. (Don't honestly know what we'd _do_ with packages with those names, but better for us to be squatting than someone else.) Cheers, Steve Top-posted from my Windows Phone -----Original Message----- From: "Christopher Wilcox" Sent: ?4/?19/?2016 18:50 To: "distutils-sig at python.org" Subject: [Distutils] Parked Names in PyPI under user rodmena DistUtils-Sig: I was searching warehouse for all Microsoft owned packages today and came across a certain user that seems to have parked on a few different package names that I don?t believe he has any intention of using (@rodmena). https://warehouse.python.org/user/rodmena/ Can we get these released to the proper owners? He seems to have done this rather broadly. If possible can the user @Microsoft be marked as the owner of the Microsoft package? Thanks! Chris Wilcox -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard at python.org Wed Apr 20 00:20:10 2016 From: richard at python.org (Richard Jones) Date: Wed, 20 Apr 2016 14:20:10 +1000 Subject: [Distutils] Parked Names in PyPI under user rodmena In-Reply-To: References: Message-ID: The usual process is to request such things through the support tracker so there's a 'paper trail', but I've been unable to attend to the queue of requests there recently so I'm going to make a special effort here. Please do consider making a request in the support tracker though, thanks. I'll get the ball rolling by contacting the current owner. Richard On 20 April 2016 at 07:52, Christopher Wilcox wrote: > DistUtils-Sig: > > I was searching warehouse for all Microsoft owned packages today and came > across a certain user that seems to have parked on a few different package > names that I don?t believe he has any intention of using (@rodmena). > https://warehouse.python.org/user/rodmena/ > > Can we get these released to the proper owners? He seems to have done this > rather broadly. > > If possible can the user @Microsoft be marked as the owner of the > Microsoft package? > > Thanks! > Chris Wilcox > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard at python.org Wed Apr 20 00:24:38 2016 From: richard at python.org (Richard Jones) Date: Wed, 20 Apr 2016 14:24:38 +1000 Subject: [Distutils] Parked Names in PyPI under user rodmena In-Reply-To: References: Message-ID: Just to be clear, are you the user "Microsoft"? You're not posting from a @ microsoft.com email domain, is all. Or are you just a "concerned citizen"? Because in the case of the latter there's really nothing for me to do here without a request from someone actually wanting to do something with the name. Richard On 20 April 2016 at 07:52, Christopher Wilcox wrote: > DistUtils-Sig: > > I was searching warehouse for all Microsoft owned packages today and came > across a certain user that seems to have parked on a few different package > names that I don?t believe he has any intention of using (@rodmena). > https://warehouse.python.org/user/rodmena/ > > Can we get these released to the proper owners? He seems to have done this > rather broadly. > > If possible can the user @Microsoft be marked as the owner of the > Microsoft package? > > Thanks! > Chris Wilcox > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From donald at stufft.io Wed Apr 20 00:51:27 2016 From: donald at stufft.io (Donald Stufft) Date: Wed, 20 Apr 2016 00:51:27 -0400 Subject: [Distutils] Parked Names in PyPI under user rodmena In-Reply-To: References: Message-ID: <0D30025F-1B69-4C9C-8753-D164D8E5EF39@stufft.io> I?m 100% sure Steve is a Microsoft employee and I?m like 95% sure Christopher is too :) > On Apr 20, 2016, at 12:24 AM, Richard Jones wrote: > > Just to be clear, are you the user "Microsoft"? You're not posting from a @microsoft.com email domain, is all. Or are you just a "concerned citizen"? Because in the case of the latter there's really nothing for me to do here without a request from someone actually wanting to do something with the name. > > > Richard > > On 20 April 2016 at 07:52, Christopher Wilcox > wrote: > DistUtils-Sig: > > I was searching warehouse for all Microsoft owned packages today and came across a certain user that seems to have parked on a few different package names that I don?t believe he has any intention of using (@rodmena). https://warehouse.python.org/user/rodmena/ > > Can we get these released to the proper owners? He seems to have done this rather broadly. > > If possible can the user @Microsoft be marked as the owner of the Microsoft package? > > Thanks! > Chris Wilcox > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 842 bytes Desc: Message signed with OpenPGP using GPGMail URL: From steve.dower at python.org Wed Apr 20 08:14:16 2016 From: steve.dower at python.org (Steve Dower) Date: Wed, 20 Apr 2016 05:14:16 -0700 Subject: [Distutils] Parked Names in PyPI under user rodmena In-Reply-To: <0D30025F-1B69-4C9C-8753-D164D8E5EF39@stufft.io> References: <0D30025F-1B69-4C9C-8753-D164D8E5EF39@stufft.io> Message-ID: Thanks for the vouch, we are indeed both current Microsoft employees. I stopped using my work email for Python stuff when our server started corrupting URLs to add a phishing/malware filter. Feel free to email the python at microsoft.com address attached to the Microsoft user and I'll reply to you. Cheers, Steve Top-posted from my Windows Phone -----Original Message----- From: "Donald Stufft" Sent: ?4/?19/?2016 21:52 To: "Richard Jones" Cc: "disutils-sig" ; "Christopher Wilcox" Subject: Re: [Distutils] Parked Names in PyPI under user rodmena I?m 100% sure Steve is a Microsoft employee and I?m like 95% sure Christopher is too :) On Apr 20, 2016, at 12:24 AM, Richard Jones wrote: Just to be clear, are you the user "Microsoft"? You're not posting from a @microsoft.com email domain, is all. Or are you just a "concerned citizen"? Because in the case of the latter there's really nothing for me to do here without a request from someone actually wanting to do something with the name. Richard On 20 April 2016 at 07:52, Christopher Wilcox wrote: DistUtils-Sig: I was searching warehouse for all Microsoft owned packages today and came across a certain user that seems to have parked on a few different package names that I don?t believe he has any intention of using (@rodmena). https://warehouse.python.org/user/rodmena/ Can we get these released to the proper owners? He seems to have done this rather broadly. If possible can the user @Microsoft be marked as the owner of the Microsoft package? Thanks! Chris Wilcox _______________________________________________ Distutils-SIG maillist - Distutils-SIG at python.org https://mail.python.org/mailman/listinfo/distutils-sig _______________________________________________ Distutils-SIG maillist - Distutils-SIG at python.org https://mail.python.org/mailman/listinfo/distutils-sig ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard at python.org Wed Apr 20 16:23:31 2016 From: richard at python.org (Richard Jones) Date: Thu, 21 Apr 2016 06:23:31 +1000 Subject: [Distutils] Parked Names in PyPI under user rodmena In-Reply-To: References: <0D30025F-1B69-4C9C-8753-D164D8E5EF39@stufft.io> Message-ID: Thanks Steve, it's all good! I have met you, but just the once, and my memory is terrible :-) On 20 April 2016 at 22:14, Steve Dower wrote: > Thanks for the vouch, we are indeed both current Microsoft employees. I > stopped using my work email for Python stuff when our server started > corrupting URLs to add a phishing/malware filter. > > Feel free to email the python at microsoft.com address attached to the > Microsoft user and I'll reply to you. > > Cheers, > Steve > > Top-posted from my Windows Phone > ------------------------------ > From: Donald Stufft > Sent: ?4/?19/?2016 21:52 > To: Richard Jones > Cc: disutils-sig ; Christopher Wilcox > > Subject: Re: [Distutils] Parked Names in PyPI under user rodmena > > I?m 100% sure Steve is a Microsoft employee and I?m like 95% sure > Christopher is too :) > > On Apr 20, 2016, at 12:24 AM, Richard Jones wrote: > > Just to be clear, are you the user "Microsoft"? You're not posting from a @ > microsoft.com email domain, is all. Or are you just a "concerned > citizen"? Because in the case of the latter there's really nothing for me > to do here without a request from someone actually wanting to do something > with the name. > > > Richard > > On 20 April 2016 at 07:52, Christopher Wilcox wrote: > >> DistUtils-Sig: >> >> I was searching warehouse for all Microsoft owned packages today and came >> across a certain user that seems to have parked on a few different package >> names that I don?t believe he has any intention of using (@rodmena). >> https://warehouse.python.org/user/rodmena/ >> >> Can we get these released to the proper owners? He seems to have done >> this rather broadly. >> >> If possible can the user @Microsoft be marked as the owner of the >> Microsoft package? >> >> Thanks! >> Chris Wilcox >> >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> https://mail.python.org/mailman/listinfo/distutils-sig >> >> > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > > > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 > DCFA > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From opensource at ronnypfannschmidt.de Wed Apr 20 16:36:22 2016 From: opensource at ronnypfannschmidt.de (Ronny Pfannschmidt) Date: Wed, 20 Apr 2016 22:36:22 +0200 Subject: [Distutils] Name arbitration on PyPI - how about administrative abandonment/replacement meta-data In-Reply-To: References: <57151B58.1090305@sdamon.com> <571660B9.3040309@ronnypfannschmidt.de> Message-ID: <5717E846.9000707@ronnypfannschmidt.de> Am 20.04.2016 um 00:38 schrieb Chris Barker: > On Tue, Apr 19, 2016 at 9:45 AM, Ronny Pfannschmidt > > wrote: > > Instead of overtaking, > how about clearly marking packages as abandoned/maintained clearly > pointing out the mark was imposed by community action > > > I think that would be a good idea -- and maybe start with just that -- > then we'd learn how big an issue it really was, etc. > > > and listing potential/primary replacements > > > that required real work on someone's part -- so not sure when that > would actually happen. > > > its important that community imposed abandonment is not simply > removable > by doing a minor "noop"-release, > > > why not? I brought tis all up to address truly abandoned projects -- > maybe we want to go some day to the idea of the names being community > owned, but that's not the way ti is now -- and if someone makes the > effort to do a noop release, then they have no abandoned the name -- > maybe aren't maintaining it worth a damn, but there you go. a community action is supposed to be imposed after extended non-reaction, an next to no effort way to get out of it seems counter such an invsive move. its not given lightly, and it shouldn't be easy to weasel out of it. Actually a noop release is a good indicator that the mark is well-deserved and should be keept. Making an effort to remove a mark without making the reason for its existence go as well is a lie in plain sight. this reminds me of the whole setuptools/distribute situation. -- Ronny > Personally, I think there is no point in anything between the current > free for all, and a "curated" package repo -- a curated repo would > support the idea that anything on it had met some minimum standard: no > malware, some maintenance, some minimum usefulness, etc. > > It's kind of a shame that there are so many "toy" packages and > experiments on PyPi, but in fact, it's worked pretty darn well.. > > pip could warn on installation/update > > > I think that would be good too > > -Chris > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From tseaver at palladion.com Wed Apr 20 18:45:28 2016 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 20 Apr 2016 18:45:28 -0400 Subject: [Distutils] Name arbitration on PyPI - how about administrative abandonment/replacement meta-data In-Reply-To: <5717E846.9000707@ronnypfannschmidt.de> References: <57151B58.1090305@sdamon.com> <571660B9.3040309@ronnypfannschmidt.de> <5717E846.9000707@ronnypfannschmidt.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/20/2016 04:36 PM, Ronny Pfannschmidt wrote: > its not given lightly, and it shouldn't be easy to weasel out of it. > Actually a noop release is a good indicator that the mark is > well-deserved and should be keept. Making an effort to remove a mark > without making the reason for its existence go as well is a lie in > plain sight. A release, any release, is a sign that the maintainer is around and cares about the package enough to act. That should be sufficient to block any takeover attempt. > this reminds me of the whole setuptools/distribute situation. I don't think that means what you think it does: the situation was resolved amicably, via negotiation with the owner of the original name. The fact that it took longer than some wanted was *not* an indication that it was the wrong outcome. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJXGAaCAAoJEPKpaDSJE9HY3FIQAL87jOyqytF1MuC5SHqiw0y1 p7BYWrLf8MVl+DFRJ0x0s9NPVgvkeF+rbx17WD7k8Eik5iuXzTpjIun0oI9UBkwC iTpogEwDrFg3T/s3bM0ObAphcLFHAY4YFMXsUThODtCHg4qi/04Z9RiMhnabwitP GfzuI33aDZbgkyi3ozEse5rP1xTn3SmhtHwu7k5gQvAmg8ntpXFzP7l1IXNWtPr+ SGXzsgohZwsDdOBwOibbvQ3VQSmeLBPrfsP0xVzNOIh7GEHT6AXMM8qK52Er4vMd SQmR2r0WAShqDm+LadfK/KjrvwSgQILpp1z4jvY6k3PWhtLABEeotDZj5mgRY8Hf 0+NTBuifYo7SHs+ElI4nryDbp48q2DZH2PLdh3WgyMlLzTocXTjQcwOTEZGcbb7t s5u+/cQLzfvg4Z23i7wPGrUrBZvl4YM3YKZfsC3Pl5v9jfwiJm/ay7G7O2NHstdB inI23H427ELXgPLI8Ffi9u7MQ6zTCd0H/XNxaiWAhG4JmMZyFuNqJQAuZ9+cR9N+ WWPEpfpHhWW9pYdxKWPHJLWKoGTu+6qPqIMnCPn3sY8ACnqH7z5puNu+/uAJ3hBt PNVh+wKE4R1wNXHKHaHqVqMH/Ut057hEh3EIOwX5e8W/zCojZy8O6PjmaFW7EGdd BMtyIkp2jgQJMY3ClfP6 =w60N -----END PGP SIGNATURE----- From chris.barker at noaa.gov Thu Apr 21 15:02:15 2016 From: chris.barker at noaa.gov (Chris Barker) Date: Thu, 21 Apr 2016 12:02:15 -0700 Subject: [Distutils] Parked Names in PyPI under user rodmena In-Reply-To: References: Message-ID: Interesting. Good evidence that the "first come first served, and then you get to keep it forever" is not ideal. As someone pointed out in the other thread, we probably don't want to change policy on existing packages, but maybe it would be good to get *some* policy in place for when warehouse goes live. -CHB On Tue, Apr 19, 2016 at 2:52 PM, Christopher Wilcox wrote: > DistUtils-Sig: > > I was searching warehouse for all Microsoft owned packages today and came > across a certain user that seems to have parked on a few different package > names that I don?t believe he has any intention of using (@rodmena). > https://warehouse.python.org/user/rodmena/ > > Can we get these released to the proper owners? He seems to have done this > rather broadly. > > If possible can the user @Microsoft be marked as the owner of the > Microsoft package? > > Thanks! > Chris Wilcox > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From tritium-list at sdamon.com Thu Apr 21 17:24:08 2016 From: tritium-list at sdamon.com (Alexander Walters) Date: Thu, 21 Apr 2016 17:24:08 -0400 Subject: [Distutils] Parked Names in PyPI under user rodmena In-Reply-To: References: Message-ID: <571944F8.2020603@sdamon.com> On 4/21/2016 15:02, Chris Barker wrote: > Good evidence that the "first come first served, and then you get to > keep it forever" is not ideal. Criminal violations of trademark are evidence that its not ideal, and therefor we should make pypi untrustworthy for all other cases? This case is /criminal/ violation of trademarks. This is different than 'I have a package that hasn't been updated for a year and you want my name on pypi'. From chris.barker at noaa.gov Thu Apr 21 18:14:00 2016 From: chris.barker at noaa.gov (Chris Barker) Date: Thu, 21 Apr 2016 15:14:00 -0700 Subject: [Distutils] Parked Names in PyPI under user rodmena In-Reply-To: <571944F8.2020603@sdamon.com> References: <571944F8.2020603@sdamon.com> Message-ID: On Thu, Apr 21, 2016 at 2:24 PM, Alexander Walters wrote: > On 4/21/2016 15:02, Chris Barker wrote: > >> Good evidence that the "first come first served, and then you get to keep >> it forever" is not ideal. >> > > Criminal violations of trademark are evidence that its not ideal, and > therefor we should make pypi untrustworthy for all other cases? This case > is /criminal/ violation of trademarks. IANL, but I don't think there is anything criminal about using a registered trademark for a Pypi name -- it all depends on how you represent your use of the name. But even if it is, we really don't want to have to go through a legal proceeding for this sort of thing, do we? This is different than 'I have a package that hasn't been updated for a > year and you want my name on pypi'. Yup. But again, I at least never proposed anything about "I have a package that hasn't been updated for a year and you want my name on pypi." I was suggesting we do something about: "I put up a package on pypi in a whim, and no longer am paying any attention to it years later" The mypy situation has gotten attention because it's a high profile package with high profile people interested in it. but I just took a look at mypy on PiPy: https://pypi.python.org/pypi/mypy/ " a wsgi framework" it has published ONE version, in 2011. no activity of any sort since then, no documentation, no meta-data, nada. And 82 downloads in the last day. Do you REALLY think that 82 people decided to use a half-baked, undocumented, ancient wsgi framework today? This in fact, looks like a perfect example of an abandoned name -- regardless of whether anyone wants to re-use that name or not. And I was just thinking: if we are worried about security -- this is a pretty good example of a dangerous situation: If that author were to suddenly decide to publish some malware under that name -- it would get a lot of traffic!. Highly unlikely, I grant you (after all, if I'm right, that person is no longer paying any attention) But it wouldn't be hard to publish all sorts of stuff under all sorts of names, and if you hit a name that was close to a popular project, you'd get a lot of hits --maybe "jango"? it doesn't seem to be taken. Anyway, all I'm saying is that current free for all leaves a lot to be desired -- but anything else will take administrative energy, and since I'm not offering to do that work, I'll shut up now. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From contact at ionelmc.ro Thu Apr 21 19:19:28 2016 From: contact at ionelmc.ro (=?UTF-8?Q?Ionel_Cristian_M=C4=83rie=C8=99?=) Date: Fri, 22 Apr 2016 02:19:28 +0300 Subject: [Distutils] Parked Names in PyPI under user rodmena In-Reply-To: References: <571944F8.2020603@sdamon.com> Message-ID: On Fri, Apr 22, 2016 at 1:14 AM, Chris Barker wrote: > it has published ONE version, in 2011. Have you actually checked? There are 40 releases since 2009: ? https://pypi.python.org/pypi/mypy/json? Thanks, -- Ionel Cristian M?rie?, http://blog.ionelmc.ro -------------- next part -------------- An HTML attachment was scrubbed... URL: From contact at ionelmc.ro Thu Apr 21 19:40:45 2016 From: contact at ionelmc.ro (=?UTF-8?Q?Ionel_Cristian_M=C4=83rie=C8=99?=) Date: Fri, 22 Apr 2016 02:40:45 +0300 Subject: [Distutils] Parked Names in PyPI under user rodmena In-Reply-To: References: <571944F8.2020603@sdamon.com> Message-ID: On Fri, Apr 22, 2016 at 1:14 AM, Chris Barker wrote: > If that author were to suddenly decide to publish some malware under that > name -- it would get a lot of traffic! ?That's the problem with badly chosen names.? I mean, what do you expect when you give a name taken 3 years ago to your project? Mypy is a poor name anyway, it's hard/ambiguous to spell and write [1], and doesn't tell anything about functionality. This is hard to understand, especially if you don't know any other language than English, but for non-native English speakers these things really matter. Thanks, -- Ionel Cristian M?rie?, http://blog.ionelmc.ro -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Apr 22 01:39:45 2016 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 22 Apr 2016 15:39:45 +1000 Subject: [Distutils] Parked Names in PyPI under user rodmena In-Reply-To: References: <571944F8.2020603@sdamon.com> Message-ID: On 22 April 2016 at 09:40, Ionel Cristian M?rie? wrote: > > On Fri, Apr 22, 2016 at 1:14 AM, Chris Barker > wrote: > >> If that author were to suddenly decide to publish some malware under that >> name -- it would get a lot of traffic! > > > ?That's the problem with badly chosen names.? I mean, what do you expect > when you give a name taken 3 years ago to your project? > > Mypy is a poor name anyway, it's hard/ambiguous to spell and write [1], > and doesn't tell anything about functionality. > > This is hard to understand, especially if you don't know any other > language than English, but for non-native English speakers these things > really matter. > FWIW, mypy isn't great as a name for English speakers either - I always have to remind myself that it has nothing to do with Mython [1]. Naming projects in general is hard though, especially for relatively arcane tasks like typechecking annotated Python code. Perhaps it would be worth having a "Choosing a name" section in https://packaging.python.org/en/latest/distributing/ similar to the "Choosing a version" one, where we provided some pragmatic suggestions on things to check for once you have a name you're considering, like: 1. Is the name already claimed on PyPI? 2. What comes up in a web search for that name? 3. What comes up if you qualify the search with "python" as a second keyword? Those 3 cursory checks will find most potential name conflicts before someone commits themselves to a particular one. Cheers, Nick. [1] http://mython.org/ -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Fri Apr 22 01:44:09 2016 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 22 Apr 2016 17:44:09 +1200 Subject: [Distutils] Parked Names in PyPI under user rodmena In-Reply-To: References: <571944F8.2020603@sdamon.com> Message-ID: <5719BA29.4030508@canterbury.ac.nz> Nick Coghlan wrote: > Naming projects in general is hard though, especially for relatively > arcane tasks like typechecking annotated Python code. Maybe obtaining a public name on PyPI should be a little bit harder than just using the first name that comes into one's head? Such as getting approval from a moderator, or something. -- Greg From gnurser at gmail.com Fri Apr 22 10:58:44 2016 From: gnurser at gmail.com (George Nurser) Date: Fri, 22 Apr 2016 15:58:44 +0100 Subject: [Distutils] pkg_resources/func_sigs requires ordereddict package rather than collections.OrderedDict in python 2.7.11 Message-ID: Hi, I've been having a strange problem with an executable that I create from a previously downloaded package using pip install -e --user nemosphere. This is under python 2.7.11 from macports. As it should be, an executable is created, ~/Library/Python/2.7/bin/3ddriver which I then link to ~/bin/3ddriver and looks like this: #!/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python # EASY-INSTALL-ENTRY-SCRIPT: 'nemosphere','console_scripts','3ddriver' __requires__ = 'nemosphere' import sys from pkg_resources import load_entry_point if __name__ == '__main__': sys.exit( load_entry_point('nemosphere', 'console_scripts', '3ddriver')() ) However I try to run 3ddriver, I get $ 3ddriver ..... Traceback (most recent call last): File "/Users/agn/bin/3ddriver", line 5, in from pkg_resources import load_entry_point File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3138, in @_call_aside File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3124, in _call_aside f(*args, **kwargs) File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3151, in _initialize_master_working_set working_set = WorkingSet._build_master() File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 661, in _build_master ws.require(__requires__) File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 962, in require needed = self.resolve(parse_requirements(requirements)) File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 849, in resolve raise DistributionNotFound(req, requirers) pkg_resources.DistributionNotFound: The 'ordereddict' distribution was not found and is required by funcsigs *The problem is solved if I install ordereddict*, but I am using python 2.7, so it shouldn't be necessary since python 2.7 already has collections.OrderedDict (and indeed it was not necessary until this week when I upgraded my macports set up, and funcsigs was upgraded to v1.0.0). IIs the problem that the current version of pkg_resources (which I believe is part of setuptools) automatically searches for ordereddict or is it a funcsigs problem? I am running v 19.2 of setuptools and v 1.0.0 of funcsigs: ~ $python Python 2.7.11 (default, Mar 1 2016, 18:08:21) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import setuptools >>> print setuptools.__version__ 19.2 >>> import funcsigs >>> print funcsigs.__version__ 1.0.0 >>> Regards, George Nurser. -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Fri Apr 22 11:33:18 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 22 Apr 2016 16:33:18 +0100 Subject: [Distutils] pkg_resources/func_sigs requires ordereddict package rather than collections.OrderedDict in python 2.7.11 In-Reply-To: References: Message-ID: On 22 April 2016 at 15:58, George Nurser wrote: > IIs the problem that the current version of pkg_resources (which I believe > is part of setuptools) automatically searches for ordereddict or is it a > funcsigs problem? I am running v 19.2 of setuptools and v 1.0.0 of > funcsigs: Funcsigs declares that it requires ordereddict in order to run. So the message is correct. Whether funcsigs could be modified to use collections.OrderedDict if present as an alternative to ordereddict, is a question for the funcsigs project. It looks like it's already been raised: https://github.com/testing-cabal/funcsigs/issues/18 Paul From tritium-list at sdamon.com Sat Apr 23 01:10:32 2016 From: tritium-list at sdamon.com (Alexander Walters) Date: Sat, 23 Apr 2016 01:10:32 -0400 Subject: [Distutils] Parked Names in PyPI under user rodmena In-Reply-To: <5719BA29.4030508@canterbury.ac.nz> References: <571944F8.2020603@sdamon.com> <5719BA29.4030508@canterbury.ac.nz> Message-ID: <571B03C8.4090802@sdamon.com> Are you volunteering your time and email inbox for that task? On 4/22/2016 01:44, Greg Ewing wrote: > Nick Coghlan wrote: >> Naming projects in general is hard though, especially for relatively >> arcane tasks like typechecking annotated Python code. > > Maybe obtaining a public name on PyPI should be a little > bit harder than just using the first name that comes into > one's head? Such as getting approval from a moderator, > or something. > From phil at riverbankcomputing.com Sun Apr 24 10:49:34 2016 From: phil at riverbankcomputing.com (Phil Thompson) Date: Sun, 24 Apr 2016 15:49:34 +0100 Subject: [Distutils] Current Warehouse Problems Message-ID: I am currently unable to upload to warehouse and get... ConnectionError: ('Connection aborted.', BrokenPipeError(32, 'Broken pipe')) ...from twine. Is this a known problem? Thanks, Phil From donald at stufft.io Sun Apr 24 11:00:04 2016 From: donald at stufft.io (Donald Stufft) Date: Sun, 24 Apr 2016 11:00:04 -0400 Subject: [Distutils] Current Warehouse Problems In-Reply-To: References: Message-ID: > On Apr 24, 2016, at 10:49 AM, Phil Thompson wrote: > > I am currently unable to upload to warehouse and get... > > ConnectionError: ('Connection aborted.', BrokenPipeError(32, 'Broken pipe')) > > ...from twine. > > Is this a known problem? > > Thanks, > Phil > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig Yea I?m working on fixing it. We?ve had some problems with gevent/eventlet either taking progressively more time on every single request (gevent) or blocking forever on uploads to S3 (eventlet). This PR reverts an earlier change to attempt to work around it https://github.com/pypa/warehouse/pull/1110 and once that lands I?ll have another PR switching away from eventlet. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 842 bytes Desc: Message signed with OpenPGP using GPGMail URL: From contact at ionelmc.ro Mon Apr 25 10:37:20 2016 From: contact at ionelmc.ro (=?UTF-8?Q?Ionel_Cristian_M=C4=83rie=C8=99?=) Date: Mon, 25 Apr 2016 17:37:20 +0300 Subject: [Distutils] Current Warehouse Problems In-Reply-To: References: Message-ID: Probably none of my damn business but why does Warehouse use gevent/eventlet? Does it really need them? Are there hosting constraints that severely limit available RAM (so threads/processes aren't an option)? Thanks, -- Ionel Cristian M?rie?, http://blog.ionelmc.ro On Sun, Apr 24, 2016 at 6:00 PM, Donald Stufft wrote: > > > On Apr 24, 2016, at 10:49 AM, Phil Thompson > wrote: > > > > I am currently unable to upload to warehouse and get... > > > > ConnectionError: ('Connection aborted.', BrokenPipeError(32, 'Broken > pipe')) > > > > ...from twine. > > > > Is this a known problem? > > > > Thanks, > > Phil > > _______________________________________________ > > Distutils-SIG maillist - Distutils-SIG at python.org > > https://mail.python.org/mailman/listinfo/distutils-sig > > > Yea I?m working on fixing it. We?ve had some problems with gevent/eventlet > either taking progressively more time on every single request (gevent) or > blocking forever on uploads to S3 (eventlet). This PR reverts an earlier > change to attempt to work around it > https://github.com/pypa/warehouse/pull/1110 and once that lands I?ll have > another PR switching away from eventlet. > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 > DCFA > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From donald at stufft.io Mon Apr 25 10:46:09 2016 From: donald at stufft.io (Donald Stufft) Date: Mon, 25 Apr 2016 10:46:09 -0400 Subject: [Distutils] Current Warehouse Problems In-Reply-To: References: Message-ID: > On Apr 25, 2016, at 10:37 AM, Ionel Cristian M?rie? wrote: > > Probably none of my damn business but why does Warehouse use gevent/eventlet? Does it really need them? Are there hosting constraints that severely limit available RAM (so threads/processes aren't an option)? > > Primarily it did that so that it could dynamically reject file uploads without having to buffer them entirely in memory or on disk. Without something asynchronous to the Python process then we end up having a hard limit set in whatever the WSGI server is that buffers files to disk or rejects file uploads over a certain size. Other than that, it doesn?t need it. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 842 bytes Desc: Message signed with OpenPGP using GPGMail URL: From contact at ionelmc.ro Mon Apr 25 11:06:06 2016 From: contact at ionelmc.ro (=?UTF-8?Q?Ionel_Cristian_M=C4=83rie=C8=99?=) Date: Mon, 25 Apr 2016 18:06:06 +0300 Subject: [Distutils] Current Warehouse Problems In-Reply-To: References: Message-ID: On Mon, Apr 25, 2016 at 5:46 PM, Donald Stufft wrote: > > Primarily it did that so that it could dynamically reject file uploads > without having to buffer them entirely in memory or on disk. Without > something asynchronous to the Python process then we end up having a hard > limit set in whatever the WSGI server is that buffers files to disk or > rejects file uploads over a certain size. > ?Sounds like a webserver feature (being able to stream the request? body) rather than a constraint that flows from the types of syscalls it use. Doesn't gunicorn (prefork) allow streaming the request body? Thanks, -- Ionel Cristian M?rie?, http://blog.ionelmc.ro -------------- next part -------------- An HTML attachment was scrubbed... URL: From donald at stufft.io Mon Apr 25 11:07:33 2016 From: donald at stufft.io (Donald Stufft) Date: Mon, 25 Apr 2016 11:07:33 -0400 Subject: [Distutils] Current Warehouse Problems In-Reply-To: References: Message-ID: > On Apr 25, 2016, at 11:06 AM, Ionel Cristian M?rie? wrote: > > > On Mon, Apr 25, 2016 at 5:46 PM, Donald Stufft > wrote: > > Primarily it did that so that it could dynamically reject file uploads without having to buffer them entirely in memory or on disk. Without something asynchronous to the Python process then we end up having a hard limit set in whatever the WSGI server is that buffers files to disk or rejects file uploads over a certain size. > > ?Sounds like a webserver feature (being able to stream the request? body) rather than a constraint that flows from the types of syscalls it use. Doesn't gunicorn (prefork) allow streaming the request body? > It does by default yes, but the problem is if you stream the request body instead of buffer it, then you need either a lot of threads or you need asynchronous IO to prevent a slowloris attack. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 842 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jim at jimfulton.info Mon Apr 25 14:45:55 2016 From: jim at jimfulton.info (Jim Fulton) Date: Mon, 25 Apr 2016 14:45:55 -0400 Subject: [Distutils] Heads up: https://bootstrap.pypa.io/ez_setup.py is broken Message-ID: Due to recent changes in PyPI URLs. This breaks buildout's bootstrap.py as well as other tools. Jim -- Jim Fulton http://jimfulton.info From jim at jimfulton.info Mon Apr 25 15:40:11 2016 From: jim at jimfulton.info (Jim Fulton) Date: Mon, 25 Apr 2016 15:40:11 -0400 Subject: [Distutils] Heads up: https://bootstrap.pypa.io/ez_setup.py is broken In-Reply-To: References: Message-ID: On Mon, Apr 25, 2016 at 2:45 PM, Jim Fulton wrote: > Due to recent changes in PyPI URLs. This breaks buildout's > bootstrap.py as well as other tools. I created an PR against the setuptools repo that should fix this. Jim -- Jim Fulton http://jimfulton.info From akashdepth at gmail.com Mon Apr 25 15:15:45 2016 From: akashdepth at gmail.com (akash chaudhary) Date: Tue, 26 Apr 2016 00:45:45 +0530 Subject: [Distutils] Python Project distribution Message-ID: Hi I want to distribute my python project that select best proxy and changes it to chrome browser or running current application that uses internet behind proxy.it has also features of username and password in it. I want to know how to distribute my python setup using distutils. Please response me soon as possible. Thanking You. Akash Chaudhary +918979078763 MNNIT,Allahabad India -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Tue Apr 26 09:46:28 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 26 Apr 2016 06:46:28 -0700 Subject: [Distutils] wheel including files it shouldn't Message-ID: <571F7134.80709@stoneleaf.us> I just received a bug report for enum34 for python3 code in a test file. Further research revealed that that file should not be included in the distribution, and is not included in the .tar nor .zip bundles, only the wheel bundle. Is this a known/fixed bug? If not, where do I report it? -- ~Ethan~ From p.f.moore at gmail.com Tue Apr 26 09:57:49 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 26 Apr 2016 14:57:49 +0100 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <571F7134.80709@stoneleaf.us> References: <571F7134.80709@stoneleaf.us> Message-ID: On 26 April 2016 at 14:46, Ethan Furman wrote: > I just received a bug report for enum34 for python3 code in a test file. > Further research revealed that that file should not be included in the > distribution, and is not included in the .tar nor .zip bundles, only the > wheel bundle. > > Is this a known/fixed bug? If not, where do I report it? What file is included that shouldn't be? test.py? Your setup.py says packages=['enum'], which will pick up the whole contents of the "enum" directory. So I'm not sure what you're saying when you say the file "should not be included". Paul From contact at ionelmc.ro Tue Apr 26 10:06:45 2016 From: contact at ionelmc.ro (=?UTF-8?Q?Ionel_Cristian_M=C4=83rie=C8=99?=) Date: Tue, 26 Apr 2016 17:06:45 +0300 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <571F7134.80709@stoneleaf.us> References: <571F7134.80709@stoneleaf.us> Message-ID: On Tue, Apr 26, 2016 at 4:46 PM, Ethan Furman wrote: > I just received a bug report for enum34 for python3 code in a test file. > Further research revealed that that file should not be included in the > distribution, and is not included in the .tar nor .zip bundles, only the > wheel bundle. > > Is this a known/fixed bug? If not, where do I report it? > It looks like you have published a wheel built from a dirty checkout. The 1.1.3 zip sdist produces a clean wheel, as expected. ?You should use https://pypi.python.org/pypi/check-manifest in your release process. Thanks, -- Ionel Cristian M?rie?, http://blog.ionelmc.ro -------------- next part -------------- An HTML attachment was scrubbed... URL: From donald at stufft.io Tue Apr 26 10:10:37 2016 From: donald at stufft.io (Donald Stufft) Date: Tue, 26 Apr 2016 10:10:37 -0400 (EDT) Subject: [Distutils] wheel including files it shouldn't In-Reply-To: References: <571F7134.80709@stoneleaf.us> Message-ID: <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> Alternatively, he could have just produced a wheel from any checkout at all if the MANIFEST.in excluded a file that would otherwise have been installed. This sort of thing is why I'm an advocate that we should only build sdists from checkouts, and wheels from sdists (at the low level anyways, even if the UI allows people to appear to create a wheel straight from a checkout). On Apr 26 2016, at 10:07 am, Ionel Cristian M?rie? <contact at ionelmc.ro> wrote: > > > It looks like you have published a wheel built from a dirty checkout. The 1.1.3 zip sdist produces a clean wheel, as expected. ?You should use in your release process. -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Tue Apr 26 10:11:30 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 26 Apr 2016 15:11:30 +0100 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: References: <571F7134.80709@stoneleaf.us> Message-ID: On 26 April 2016 at 15:06, Ionel Cristian M?rie? wrote: > On Tue, Apr 26, 2016 at 4:46 PM, Ethan Furman wrote: >> >> I just received a bug report for enum34 for python3 code in a test file. >> Further research revealed that that file should not be included in the >> distribution, and is not included in the .tar nor .zip bundles, only the >> wheel bundle. >> >> Is this a known/fixed bug? If not, where do I report it? > > > It looks like you have published a wheel built from a dirty checkout. The > 1.1.3 zip sdist produces a clean wheel, as expected. You should use > https://pypi.python.org/pypi/check-manifest in your release process. Ah, yes. enum/py35_test_enum.py is not part of the source tree or sdist, but is in the "enum" package directory, so will get picked up by packages=['enum'] Paul From contact at ionelmc.ro Tue Apr 26 10:15:15 2016 From: contact at ionelmc.ro (=?UTF-8?Q?Ionel_Cristian_M=C4=83rie=C8=99?=) Date: Tue, 26 Apr 2016 17:15:15 +0300 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: References: <571F7134.80709@stoneleaf.us> Message-ID: On Tue, Apr 26, 2016 at 5:11 PM, Paul Moore wrote: > > It looks like you have published a wheel built from a dirty checkout. The > > 1.1.3 zip sdist produces a clean wheel, as expected. You should use > > https://pypi.python.org/pypi/check-manifest in your release process. > > Ah, yes. enum/py35_test_enum.py is not part of the source tree or > sdist, but is in the "enum" package directory, so will get picked up > by packages=['enum'] ?So yes, technically check-manifest don't have anything to do with bdist_wheel (it does checks for sdist). But most probably it would had? complained about unversioned file going into the sdist. Maybe there should be a similar tool for bdist_wheel, "check-wheel" anyone? What other checks could it have? Thanks, -- Ionel Cristian M?rie?, http://blog.ionelmc.ro -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Tue Apr 26 11:56:51 2016 From: brett at python.org (Brett Cannon) Date: Tue, 26 Apr 2016 15:56:51 +0000 Subject: [Distutils] Python Project distribution In-Reply-To: References: Message-ID: https://packaging.python.org/en/latest/ should have all the information you need to learn how to package up and distribute your project. On Tue, 26 Apr 2016 at 05:42 akash chaudhary wrote: > Hi > > I want to distribute my python project that select best proxy and changes > it to chrome browser or running current application that uses internet > behind proxy.it has also features of username and password in it. > I want to know how to distribute my python setup using distutils. > Please response me soon as possible. > > Thanking You. > > Akash Chaudhary > +918979078763 > MNNIT,Allahabad > India > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Wed Apr 27 13:39:12 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 27 Apr 2016 10:39:12 -0700 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> Message-ID: <5720F940.4020307@stoneleaf.us> On 04/26/2016 07:10 AM, Donald Stufft wrote: > Alternatively, he could have just produced a wheel from any checkout at > all if the MANIFEST.in excluded a file that would otherwise have been > installed. Yes. My MANIFEST.in starts with an 'exclude enum/*' and then includes all files it wants. > This sort of thing is why I'm an advocate that we should only > build sdists from checkouts, and wheels from sdists (at the low level > anyways, even if the UI allows people to appear to create a wheel > straight from a checkout). My current process is: python3.5 setup.py sdist --format=gztar,zip bdist_wheel upload What should I be doing instead? -- ~Ethan~ From donald at stufft.io Wed Apr 27 13:52:37 2016 From: donald at stufft.io (Donald Stufft) Date: Wed, 27 Apr 2016 13:52:37 -0400 (EDT) Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <5720F940.4020307@stoneleaf.us> References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> Message-ID: <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> This isn't really a problem with what you're doing. Rather it's an issue with the toolchain and and open question whether or not wheels should conceptually be able to be produced from a checkout, or if they should only be produced from a sdist. Problems like this are why I advocate the Checkout -> sdist -> wheel being the only path, but others feel differently. ? Donald Stufft On Apr 27 2016, at 1:38 pm, Ethan Furman <ethan at stoneleaf.us> wrote: > On 04/26/2016 07:10 AM, Donald Stufft wrote: > > > Alternatively, he could have just produced a wheel from any checkout at > all if the MANIFEST.in excluded a file that would otherwise have been > installed. > > Yes. My MANIFEST.in starts with an 'exclude enum/*' and then includes all files it wants. > > > This sort of thing is why I'm an advocate that we should only > build sdists from checkouts, and wheels from sdists (at the low level > anyways, even if the UI allows people to appear to create a wheel > straight from a checkout). > > My current process is: > > python3.5 setup.py sdist --format=gztar,zip bdist_wheel upload > > What should I be doing instead? > > \-- ~Ethan~ > > _______________________________________________ Distutils-SIG maillist - Distutils-SIG at python.org https://mail.python.org/mailman/listinfo/distutils-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From dholth at gmail.com Wed Apr 27 14:05:33 2016 From: dholth at gmail.com (Daniel Holth) Date: Wed, 27 Apr 2016 18:05:33 +0000 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> Message-ID: I feel differently. Bdist_wheel just doesn't understand MANIFEST.in . Perhaps support could be adapted from bdist_egg or other bdist implementations. Wheel doesn't do everything the setuptools or distutils implementations put into their dist commands. Even something as simple as a forced clean of the dist directory on init may fix the problem? On Wed, Apr 27, 2016, 13:52 Donald Stufft wrote: > This isn't really a problem with what you're doing. Rather it's an issue > with the toolchain and and open question whether or not wheels should > conceptually be able to be produced from a checkout, or if they should only > be produced from a sdist. Problems like this are why I advocate the > Checkout -> sdist -> wheel being the only path, but others feel differently. > > > ? Donald Stufft > > On Apr 27 2016, at 1:38 pm, Ethan Furman wrote: > >> On 04/26/2016 07:10 AM, Donald Stufft wrote: >> >> > Alternatively, he could have just produced a wheel from any checkout at >> > all if the MANIFEST.in excluded a file that would otherwise have been >> > installed. >> >> Yes. My MANIFEST.in starts with an 'exclude enum/*' and then includes >> all files it wants. >> >> > This sort of thing is why I'm an advocate that we should only >> > build sdists from checkouts, and wheels from sdists (at the low level >> > anyways, even if the UI allows people to appear to create a wheel >> > straight from a checkout). >> >> My current process is: >> >> python3.5 setup.py sdist --format=gztar,zip bdist_wheel upload >> >> What should I be doing instead? >> >> -- >> ~Ethan~ >> >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> https://mail.python.org/mailman/listinfo/distutils-sig >> > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Wed Apr 27 14:10:40 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 27 Apr 2016 11:10:40 -0700 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> Message-ID: <572100A0.1070408@stoneleaf.us> On 04/27/2016 10:52 AM, Donald Stufft wrote: > This isn't really a problem with what you're doing. Rather it's an issue > with the toolchain and and open question whether or not wheels should > conceptually be able to be produced from a checkout, or if they should > only be produced from a sdist. Problems like this are why I advocate the > Checkout -> sdist -> wheel being the only path, but others feel differently. As a simple user, my feelings are that the command I used should have generated three equivalent distributions, but did not. That feels like a bug. :( Let me rephrase my question: what command do I use to build the wheel from the sdist I just made? For bonus points: why can't setup do that for me? -- ~Ethan~ From donald at stufft.io Wed Apr 27 14:09:34 2016 From: donald at stufft.io (Donald Stufft) Date: Wed, 27 Apr 2016 14:09:34 -0400 (EDT) Subject: [Distutils] wheel including files it shouldn't In-Reply-To: References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> Message-ID: <6jcv0c90una7hvlr1lcnlp0rf-0@mailer.nylas.com> You can try to patch around the problem (and do it in every single alternative build system once we enable support for that) or you can just fix the problem at the "protocol" level once and for all. The fundamental problem isn't with MANIFEST.in, it's that some indeterminite steps need to run to produce a sdist and unless you make sure you run every single one of those steps you can have no confidence that the results of producing a wheel from a VCS checkout will be the same as installing from a sdist or building a wheel from a sdist. ? Donald Stufft On Apr 27 2016, at 2:05 pm, Daniel Holth <dholth at gmail.com> wrote: > I feel differently. > > Bdist_wheel just doesn't understand MANIFEST.in . Perhaps support could be adapted from bdist_egg or other bdist implementations. Wheel doesn't do everything the setuptools or distutils implementations put into their dist commands. Even something as simple as a forced clean of the dist directory on init may fix the problem? > > > > On Wed, Apr 27, 2016, 13:52 Donald Stufft <[donald at stufft.io](mailto:donald at stufft.io)> wrote: > >> This isn't really a problem with what you're doing. Rather it's an issue with the toolchain and and open question whether or not wheels should conceptually be able to be produced from a checkout, or if they should only be produced from a sdist. Problems like this are why I advocate the Checkout -> sdist -> wheel being the only path, but others feel differently. >> >> ? Donald Stufft >> >> >> >> On Apr 27 2016, at 1:38 pm, Ethan Furman <[ethan at stoneleaf.us](mailto:ethan at stoneleaf.us)> wrote: >> >>> On 04/26/2016 07:10 AM, Donald Stufft wrote: >>> >>> > Alternatively, he could have just produced a wheel from any checkout at > all if the MANIFEST.in excluded a file that would otherwise have been > installed. >>> >>> Yes. My MANIFEST.in starts with an 'exclude enum/*' and then includes all files it wants. >>> >>> > This sort of thing is why I'm an advocate that we should only > build sdists from checkouts, and wheels from sdists (at the low level > anyways, even if the UI allows people to appear to create a wheel > straight from a checkout). >>> >>> My current process is: >>> >>> python3.5 setup.py sdist --format=gztar,zip bdist_wheel upload >>> >>> What should I be doing instead? >>> >>> \-- ~Ethan~ >>> >>> _______________________________________________ Distutils-SIG maillist - [Distutils-SIG at python.org](mailto:Distutils- SIG at python.org) >> >> _______________________________________________ Distutils-SIG maillist - [Distutils-SIG at python.org](mailto:Distutils- SIG at python.org) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Wed Apr 27 14:12:28 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 27 Apr 2016 11:12:28 -0700 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> Message-ID: <5721010C.6060203@stoneleaf.us> On 04/27/2016 11:05 AM, Daniel Holth wrote: > Bdist_wheel just doesn't understand MANIFEST.in . Perhaps support could > be adapted from bdist_egg or other bdist implementations. Wheel doesn't > do everything the setuptools or distutils implementations put into their > dist commands. Even something as simple as a forced clean of the dist > directory on init may fix the problem? Um, does that mean it would destroy my files-in-progress? 'Cause that would really [insert colloquial phrase meaning extreme irritation here]. -- ~Ethan~ From dholth at gmail.com Wed Apr 27 14:12:00 2016 From: dholth at gmail.com (Daniel Holth) Date: Wed, 27 Apr 2016 18:12:00 +0000 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <5721010C.6060203@stoneleaf.us> References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> <5721010C.6060203@stoneleaf.us> Message-ID: No I mean the build directory. On Wed, Apr 27, 2016, 14:11 Ethan Furman wrote: > On 04/27/2016 11:05 AM, Daniel Holth wrote: > > > Bdist_wheel just doesn't understand MANIFEST.in . Perhaps support could > > be adapted from bdist_egg or other bdist implementations. Wheel doesn't > > do everything the setuptools or distutils implementations put into their > > dist commands. Even something as simple as a forced clean of the dist > > directory on init may fix the problem? > > Um, does that mean it would destroy my files-in-progress? 'Cause that > would really [insert colloquial phrase meaning extreme irritation here]. > > -- > ~Ethan~ > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From donald at stufft.io Wed Apr 27 14:12:33 2016 From: donald at stufft.io (Donald Stufft) Date: Wed, 27 Apr 2016 14:12:33 -0400 (EDT) Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <572100A0.1070408@stoneleaf.us> References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> <572100A0.1070408@stoneleaf.us> Message-ID: <6t5t01cluqn0h0hdi84ujagm4-0@mailer.nylas.com> On Apr 27 2016, at 2:09 pm, Ethan Furman <ethan at stoneleaf.us> wrote: > On 04/27/2016 10:52 AM, Donald Stufft wrote: > > > This isn't really a problem with what you're doing. Rather it's an issue > with the toolchain and and open question whether or not wheels should > conceptually be able to be produced from a checkout, or if they should > only be produced from a sdist. Problems like this are why I advocate the > Checkout -> sdist -> wheel being the only path, but others feel differently. > > As a simple user, my feelings are that the command I used should have generated three equivalent distributions, but did not. That feels like a bug. :( I agree with you. Others don't believe it to be a big deal. > Let me rephrase my question: what command do I use to build the wheel from the sdist I just made? For bonus points: why can't setup do that for me? > > You can first build the sdist, then unpack it into a directory and then run ``setup.py bdist_wheel`` inside of that directory. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.gronholm at nextday.fi Wed Apr 27 14:13:39 2016 From: alex.gronholm at nextday.fi (=?UTF-8?Q?Alex_Gr=c3=b6nholm?=) Date: Wed, 27 Apr 2016 21:13:39 +0300 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <572100A0.1070408@stoneleaf.us> References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> <572100A0.1070408@stoneleaf.us> Message-ID: <57210153.5030504@nextday.fi> Are you seriously saying that you want your bdists to include tests, documentation etc.? Most developers would not agree with you, including yours truly. 27.04.2016, 21:10, Ethan Furman kirjoitti: > On 04/27/2016 10:52 AM, Donald Stufft wrote: > >> This isn't really a problem with what you're doing. Rather it's an issue >> with the toolchain and and open question whether or not wheels should >> conceptually be able to be produced from a checkout, or if they should >> only be produced from a sdist. Problems like this are why I advocate the >> Checkout -> sdist -> wheel being the only path, but others feel >> differently. > > As a simple user, my feelings are that the command I used should have > generated three equivalent distributions, but did not. That feels > like a bug. :( > > Let me rephrase my question: what command do I use to build the wheel > from the sdist I just made? For bonus points: why can't setup do > that for me? > > -- > ~Ethan~ > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig From dholth at gmail.com Wed Apr 27 14:18:07 2016 From: dholth at gmail.com (Daniel Holth) Date: Wed, 27 Apr 2016 18:18:07 +0000 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <57210153.5030504@nextday.fi> References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> <572100A0.1070408@stoneleaf.us> <57210153.5030504@nextday.fi> Message-ID: Whatever the ideal situation (not using distutils at all would be a suggestion) bdist_wheel certainly has some simple bugs that make it less fun to use sometimes. On Wed, Apr 27, 2016 at 2:14 PM Alex Gr?nholm wrote: > Are you seriously saying that you want your bdists to include tests, > documentation etc.? > Most developers would not agree with you, including yours truly. > > 27.04.2016, 21:10, Ethan Furman kirjoitti: > > On 04/27/2016 10:52 AM, Donald Stufft wrote: > > > >> This isn't really a problem with what you're doing. Rather it's an issue > >> with the toolchain and and open question whether or not wheels should > >> conceptually be able to be produced from a checkout, or if they should > >> only be produced from a sdist. Problems like this are why I advocate the > >> Checkout -> sdist -> wheel being the only path, but others feel > >> differently. > > > > As a simple user, my feelings are that the command I used should have > > generated three equivalent distributions, but did not. That feels > > like a bug. :( > > > > Let me rephrase my question: what command do I use to build the wheel > > from the sdist I just made? For bonus points: why can't setup do > > that for me? > > > > -- > > ~Ethan~ > > > > _______________________________________________ > > Distutils-SIG maillist - Distutils-SIG at python.org > > https://mail.python.org/mailman/listinfo/distutils-sig > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Wed Apr 27 14:40:03 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 27 Apr 2016 11:40:03 -0700 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <57210153.5030504@nextday.fi> References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> <572100A0.1070408@stoneleaf.us> <57210153.5030504@nextday.fi> Message-ID: <57210783.908@stoneleaf.us> On 04/27/2016 11:13 AM, Alex Gr?nholm wrote: > Are you seriously saying that you want your bdists to include tests, > documentation etc.? However you and I agree or disagree on what should be in a bdist, the command I ran should have produced a bdist based on the sdists I just created in the same command. > Most developers would not agree with you, including yours truly. Well, we disagree. To me, the salient difference between an sdist and a bdist is whether binary artifacts are, um, already built. I certianly enjoy having docs (so I know how to use the binaries I just installed) and tests (so I can assure myself the binaries work as advertised). If a project is big enough I can see making separate packages for docs and/or tests, but mine are small. And whichever way we decide to do the packaging, the tools should work for us, not us for the tools. -- ~Ethan~ From alex.gronholm at nextday.fi Wed Apr 27 15:00:07 2016 From: alex.gronholm at nextday.fi (=?UTF-8?Q?Alex_Gr=c3=b6nholm?=) Date: Wed, 27 Apr 2016 22:00:07 +0300 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <57210783.908@stoneleaf.us> References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> <572100A0.1070408@stoneleaf.us> <57210153.5030504@nextday.fi> <57210783.908@stoneleaf.us> Message-ID: <57210C37.2080905@nextday.fi> The sdist should include all the source files, including tests and documentation. In binary distributions, however, they are just dead weight. Do you want the full documentation and test suites to be installed for every single dependency when you deploy your application? I sure don't. 27.04.2016, 21:40, Ethan Furman kirjoitti: > On 04/27/2016 11:13 AM, Alex Gr?nholm wrote: > >> Are you seriously saying that you want your bdists to include tests, >> documentation etc.? > > However you and I agree or disagree on what should be in a bdist, the > command I ran should have produced a bdist based on the sdists I just > created in the same command. > >> Most developers would not agree with you, including yours truly. > > Well, we disagree. To me, the salient difference between an sdist and > a bdist is whether binary artifacts are, um, already built. I > certianly enjoy having docs (so I know how to use the binaries I just > installed) and tests (so I can assure myself the binaries work as > advertised). > > If a project is big enough I can see making separate packages for docs > and/or tests, but mine are small. > > And whichever way we decide to do the packaging, the tools should work > for us, not us for the tools. > > -- > ~Ethan~ > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig From ethan at stoneleaf.us Wed Apr 27 15:11:06 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 27 Apr 2016 12:11:06 -0700 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <57210C37.2080905@nextday.fi> References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> <572100A0.1070408@stoneleaf.us> <57210153.5030504@nextday.fi> <57210783.908@stoneleaf.us> <57210C37.2080905@nextday.fi> Message-ID: <57210ECA.5030108@stoneleaf.us> On 04/27/2016 12:00 PM, Alex Gr?nholm wrote: > The sdist should include all the source files, including tests and > documentation. In binary distributions, however, they are just dead > weight. Do you want the full documentation and test suites to be > installed for every single dependency when you deploy your application? > I sure don't. That's makes a certain amount of sense. It also raises some questions: - are wheels a binary-only distribution? - if yes, we still have to use distutils to make source distributions? - if no, what are the commands to make source vs binary distributions? - is this just a matter of a properly configured setup.py? - if yes, what is that configuration? -- ~Ethan~ From dholth at gmail.com Wed Apr 27 15:18:12 2016 From: dholth at gmail.com (Daniel Holth) Date: Wed, 27 Apr 2016 19:18:12 +0000 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <57210ECA.5030108@stoneleaf.us> References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> <572100A0.1070408@stoneleaf.us> <57210153.5030504@nextday.fi> <57210783.908@stoneleaf.us> <57210C37.2080905@nextday.fi> <57210ECA.5030108@stoneleaf.us> Message-ID: To answer the original question, report the bug here https://bitbucket.org/pypa/wheel On Wed, Apr 27, 2016 at 3:10 PM Ethan Furman wrote: > On 04/27/2016 12:00 PM, Alex Gr?nholm wrote: > > > The sdist should include all the source files, including tests and > > documentation. In binary distributions, however, they are just dead > > weight. Do you want the full documentation and test suites to be > > installed for every single dependency when you deploy your application? > > I sure don't. > > That's makes a certain amount of sense. > > It also raises some questions: > > - are wheels a binary-only distribution? > - if yes, we still have to use distutils to make source distributions? > - if no, what are the commands to make source vs binary distributions? > > - is this just a matter of a properly configured setup.py? > - if yes, what is that configuration? > > -- > ~Ethan~ > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Wed Apr 27 15:32:31 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 27 Apr 2016 12:32:31 -0700 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> <572100A0.1070408@stoneleaf.us> <57210153.5030504@nextday.fi> <57210783.908@stoneleaf.us> <57210C37.2080905@nextday.fi> <57210ECA.5030108@stoneleaf.us> Message-ID: <572113CF.6070106@stoneleaf.us> On 04/27/2016 12:18 PM, Daniel Holth wrote: > To answer the original question, report the bug here > https://bitbucket.org/pypa/wheel What do you know, it's already there! ;) https://bitbucket.org/pypa/wheel/issues/99/cannot-exclude-directory -- ~Ethan~ From mmericke at gmail.com Wed Apr 27 15:31:08 2016 From: mmericke at gmail.com (Michael Merickel) Date: Wed, 27 Apr 2016 14:31:08 -0500 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> <572100A0.1070408@stoneleaf.us> <57210153.5030504@nextday.fi> <57210783.908@stoneleaf.us> <57210C37.2080905@nextday.fi> <57210ECA.5030108@stoneleaf.us> Message-ID: On Wed, Apr 27, 2016 at 2:18 PM, Daniel Holth wrote: > To answer the original question, report the bug here > https://bitbucket.org/pypa/wheel > It looksl ike there's already a similar bug opened at https://bitbucket.org/pypa/wheel/issues/147/bdist_wheel-should-start-by-cleaning-up I ran into this when releasing the latest versions of Pyramid as well. It's definitely not clear how building a wheel interacts with the build folder or manifest. Things I've never had to think about when building sdists. -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Wed Apr 27 17:45:00 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 27 Apr 2016 22:45:00 +0100 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <5720F940.4020307@stoneleaf.us> References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> Message-ID: On 27 April 2016 at 18:39, Ethan Furman wrote: > My current process is: > > python3.5 setup.py sdist --format=gztar,zip bdist_wheel upload > > What should I be doing instead? My suggestion would be # Build the sdist python3.5 setup.py sdist --format=gztar,zip # Build the wheel *from that sdist* python3.5 -m pip wheel dist/*.zip # the path to the sdist built just above # Upload the files once you've checked them twine upload *.whl dist/* # because setup.py upload can't upload prebuilt files But I agree that's somewhat clumsy compared to the command you used. It has the benefit of working correctly though :-) I agree that it's a bug that the bdist_wheel command doesn't respect your MANIFEST.in. It's also a (design-level) bug that bdist_wheel *needs* to implement MANIFEST.in support for itself. IMO, it's a further bug that setup.py upload doesn't allow you to build the files first, test them, and *then* upload them. Personally, I agree with Donald that the "normal" process of building a distribution should be: 1. Build the sdist. 2. Build wheels *from* that sdist. 3. Check the built sdist and wheels. 4. Upload the files once you've confirmed they are OK. The tools should make that as transparent as possible (I'm not averse to a command that builds sdist and wheels together, but such a command could easily use the sdist to build the wheels "behind the scenes") and there may be special cases (incremental builds of wheels when a full recompile is a significant cost) but those are quality of implementation details. Paul From ethan at stoneleaf.us Wed Apr 27 19:57:36 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 27 Apr 2016 16:57:36 -0700 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> Message-ID: <572151F0.1050802@stoneleaf.us> On 04/27/2016 02:45 PM, Paul Moore wrote: > On 27 April 2016 at 18:39, Ethan Furman wrote: > # Upload the files once you've checked them > twine upload *.whl dist/* # because setup.py upload can't upload prebuilt files What a pain. :( > Personally, I agree with Donald that the "normal" process of building > a distribution should be: > > 1. Build the sdist. > 2. Build wheels *from* that sdist. > 3. Check the built sdist and wheels. > 4. Upload the files once you've confirmed they are OK. What types of things should I be doing for (3) ? -- ~Ethan~ From p.f.moore at gmail.com Thu Apr 28 04:03:48 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 28 Apr 2016 09:03:48 +0100 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <572151F0.1050802@stoneleaf.us> References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <572151F0.1050802@stoneleaf.us> Message-ID: On 28 April 2016 at 00:57, Ethan Furman wrote: > What types of things should I be doing for (3) ? You could check the contents of the wheels to ensure they don't contain unexpected files :-) But in all honesty, probably not much. I only noted it as it's the only place you can verify the actual files you're uploading. Paul From leorochael at gmail.com Thu Apr 28 04:07:33 2016 From: leorochael at gmail.com (Leonardo Rochael Almeida) Date: Thu, 28 Apr 2016 05:07:33 -0300 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <572151F0.1050802@stoneleaf.us> Message-ID: Actually, another thing you can do with the wheels at this point is to take them for a spin :-) Have pip in another virtualenv install it by using --find-links, or even setting your own local index. If you have tests embedded in your wheel, now would be a good time to run them. Best regards, Leo On 28 April 2016 at 05:03, Paul Moore wrote: > On 28 April 2016 at 00:57, Ethan Furman wrote: > > What types of things should I be doing for (3) ? > > You could check the contents of the wheels to ensure they don't > contain unexpected files :-) > > But in all honesty, probably not much. I only noted it as it's the > only place you can verify the actual files you're uploading. > Paul > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven at wedontsleep.org Thu Apr 28 11:43:40 2016 From: steven at wedontsleep.org (Steve Kowalik) Date: Thu, 28 Apr 2016 10:43:40 -0500 Subject: [Distutils] Use of direct references in requirements for setuptools Message-ID: <57222FAC.2090103@wedontsleep.org> Hi, Currently, pip supports use of direct references[1][2] in requirement files, which means it includes code that will deal with downloading files over HTTP, or git, etc. setuptools does not have this support, and in order for setuptools to completely implement PEP 508 support, we should come up with a plan what to do when setuptools encounters a requirement that uses a direct reference. The options that I can see are: 1) When egg_info is run, exit with an error saying "Please don't do that" 2) Warn about the requirement, and that we will just depend on it, but drop the direct reference. 3) Add support to setuptools. Personally, I don't like this plan, but perhaps it's easy. 1: For example: testtools@ https://github.com/testing-cabal/testtools 2: It doesn't yet support the format specified by PEP 508, but the functionality exists and works. -- Steve Oh, in case you got covered in that Repulsion Gel, here's some advice the lab boys gave me: [paper rustling] DO NOT get covered in the Repulsion Gel. - Cave Johnson, CEO of Aperture Science From robertc at robertcollins.net Thu Apr 28 12:34:41 2016 From: robertc at robertcollins.net (Robert Collins) Date: Thu, 28 Apr 2016 11:34:41 -0500 Subject: [Distutils] Use of direct references in requirements for setuptools In-Reply-To: <57222FAC.2090103@wedontsleep.org> References: <57222FAC.2090103@wedontsleep.org> Message-ID: On 28 April 2016 at 10:43, Steve Kowalik wrote: > Hi, > > Currently, pip supports use of direct references[1][2] in requirement > files, which means it includes code that will deal with downloading > files over HTTP, or git, etc. > > setuptools does not have this support, and in order for setuptools to > completely implement PEP 508 support, we should come up with a plan > what to do when setuptools encounters a requirement that uses a direct > reference. > > The options that I can see are: > 1) When egg_info is run, exit with an error saying "Please don't do > that" > 2) Warn about the requirement, and that we will just depend on it, but > drop the direct reference. > 3) Add support to setuptools. Personally, I don't like this plan, but > perhaps it's easy. What about putting the reference into the egg_info, warning that setuptools itself cannot deal with direct references, and for easy-install itself drop back to using the name? I'm thinking we probably do want egg-info to have direct references - so that the use case of a private network of packages all hosted on e.g. github, can work with pip. Donald, WDYT? -Rob From steve.dower at python.org Thu Apr 28 12:46:02 2016 From: steve.dower at python.org (Steve Dower) Date: Thu, 28 Apr 2016 09:46:02 -0700 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> Message-ID: <3eb78e47-1963-e2f2-1737-cd3e10e26744@python.org> On 27Apr2016 1445, Paul Moore wrote: > Personally, I agree with Donald that the "normal" process of building > a distribution should be: > > 1. Build the sdist. > 2. Build wheels *from* that sdist. > 3. Check the built sdist and wheels. > 4. Upload the files once you've confirmed they are OK. > > The tools should make that as transparent as possible (I'm not averse > to a command that builds sdist and wheels together, but such a command > could easily use the sdist to build the wheels "behind the scenes") > and there may be special cases (incremental builds of wheels when a > full recompile is a significant cost) but those are quality of > implementation details. One extra task that I often need is the ability to separate bdist_ext from bdist_wheel, so that I can apply modifications to built modules (e.g. code signing) before packaging up the wheel. There's probably a way to insert this step using a setuptools extension or a setup.py hack, but it's actually most convenient to have two completely separate commands (or options) for building and packaging. Cheers, Steve From dholth at gmail.com Thu Apr 28 13:25:57 2016 From: dholth at gmail.com (Daniel Holth) Date: Thu, 28 Apr 2016 17:25:57 +0000 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <3eb78e47-1963-e2f2-1737-cd3e10e26744@python.org> References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <3eb78e47-1963-e2f2-1737-cd3e10e26744@python.org> Message-ID: Wheel has a proposal for a "wheel pack" command that might help you. On Thu, Apr 28, 2016, 12:46 Steve Dower wrote: > On 27Apr2016 1445, Paul Moore wrote: > > Personally, I agree with Donald that the "normal" process of building > > a distribution should be: > > > > 1. Build the sdist. > > 2. Build wheels *from* that sdist. > > 3. Check the built sdist and wheels. > > 4. Upload the files once you've confirmed they are OK. > > > > The tools should make that as transparent as possible (I'm not averse > > to a command that builds sdist and wheels together, but such a command > > could easily use the sdist to build the wheels "behind the scenes") > > and there may be special cases (incremental builds of wheels when a > > full recompile is a significant cost) but those are quality of > > implementation details. > > One extra task that I often need is the ability to separate bdist_ext > from bdist_wheel, so that I can apply modifications to built modules > (e.g. code signing) before packaging up the wheel. > > There's probably a way to insert this step using a setuptools extension > or a setup.py hack, but it's actually most convenient to have two > completely separate commands (or options) for building and packaging. > > Cheers, > Steve > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Thu Apr 28 14:21:32 2016 From: barry at python.org (Barry Warsaw) Date: Thu, 28 Apr 2016 14:21:32 -0400 Subject: [Distutils] wheel including files it shouldn't References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> <572100A0.1070408@stoneleaf.us> <57210153.5030504@nextday.fi> <57210783.908@stoneleaf.us> <57210C37.2080905@nextday.fi> Message-ID: <20160428142132.2776e847@anarchist.wooz.org> On Apr 27, 2016, at 10:00 PM, Alex Gr?nholm wrote: >The sdist should include all the source files, including tests and >documentation. In binary distributions, however, they are just dead >weight. Do you want the full documentation and test suites to be installed >for every single dependency when you deploy your application? I sure don't. Shouldn't that be left to the package author to decide? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From alex.gronholm at nextday.fi Thu Apr 28 14:31:30 2016 From: alex.gronholm at nextday.fi (=?UTF-8?Q?Alex_Gr=c3=b6nholm?=) Date: Thu, 28 Apr 2016 21:31:30 +0300 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <20160428142132.2776e847@anarchist.wooz.org> References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> <572100A0.1070408@stoneleaf.us> <57210153.5030504@nextday.fi> <57210783.908@stoneleaf.us> <57210C37.2080905@nextday.fi> <20160428142132.2776e847@anarchist.wooz.org> Message-ID: <57225702.7080700@nextday.fi> To what end? 28.04.2016, 21:21, Barry Warsaw kirjoitti: > On Apr 27, 2016, at 10:00 PM, Alex Gr?nholm wrote: > >> The sdist should include all the source files, including tests and >> documentation. In binary distributions, however, they are just dead >> weight. Do you want the full documentation and test suites to be installed >> for every single dependency when you deploy your application? I sure don't. > Shouldn't that be left to the package author to decide? > > -Barry > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From dholth at gmail.com Thu Apr 28 14:50:57 2016 From: dholth at gmail.com (Daniel Holth) Date: Thu, 28 Apr 2016 18:50:57 +0000 Subject: [Distutils] wheel including files it shouldn't In-Reply-To: <57225702.7080700@nextday.fi> References: <571F7134.80709@stoneleaf.us> <8n801vq4ljpalzetnxnpjhx6i-0@mailer.nylas.com> <5720F940.4020307@stoneleaf.us> <1n09a68i2n7eap7zl9yf12k9a-0@mailer.nylas.com> <572100A0.1070408@stoneleaf.us> <57210153.5030504@nextday.fi> <57210783.908@stoneleaf.us> <57210C37.2080905@nextday.fi> <20160428142132.2776e847@anarchist.wooz.org> <57225702.7080700@nextday.fi> Message-ID: If you'd like to have more control over what's installed, we could re-visit the idea of adding more categories of files within distributions based on the model of "copy each category of files to a [configurable] target directory". Perhaps there could even be a "necessary" flag on each category to indicate whether the code accessed those files at runtime or not. On Thu, Apr 28, 2016 at 2:31 PM Alex Gr?nholm wrote: > To what end? > > > 28.04.2016, 21:21, Barry Warsaw kirjoitti: > > On Apr 27, 2016, at 10:00 PM, Alex Gr?nholm wrote: > > > The sdist should include all the source files, including tests and > documentation. In binary distributions, however, they are just dead > weight. Do you want the full documentation and test suites to be installed > for every single dependency when you deploy your application? I sure don't. > > > Shouldn't that be left to the package author to decide? > > -Barry > > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.orghttps://mail.python.org/mailman/listinfo/distutils-sig > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > https://mail.python.org/mailman/listinfo/distutils-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stuaxo2 at yahoo.com Sat Apr 30 20:15:25 2016 From: stuaxo2 at yahoo.com (Stuart Axon) Date: Sun, 1 May 2016 00:15:25 +0000 (UTC) Subject: [Distutils] Pypi - procedure if user email / password changed? References: <999136496.4265249.1462061725392.JavaMail.yahoo.ref@mail.yahoo.com> Message-ID: <999136496.4265249.1462061725392.JavaMail.yahoo@mail.yahoo.com> This is probably the wrong mailinglist - does anybody here know who a maintainer needs to contact to change their details if they don't have access to the mailing address they registered a package on pypi with ? This is for the package 'fabulous', which can't be updated for this reason.https://github.com/jart/fabulous/issues/7 So far I've tried mailing webmaster at upython.org + got an automated message. S++ -------------- next part -------------- An HTML attachment was scrubbed... URL: