From techtonik at gmail.com Wed Jun 13 13:55:44 2012 From: techtonik at gmail.com (anatoly techtonik) Date: Wed, 13 Jun 2012 14:55:44 +0300 Subject: [Catalog-sig] Dependencies Message-ID: I need to build Fabric RPM package for Fedora. I use py2pack to generate .spec file, which in turn uses PyPI information for it. The problem that PyPI completely misses information about dependencies. Is that intentional, broken or just not implemented? Any previous discussions/outcomes? http://pypi.python.org/pypi/Fabric/json - here should be a reference to ssh package dependency which is present in setup.py as: install_requires=['ssh>=1.7.14'], P.S. My patch queue is still awaiting review comments at this changeset https://bitbucket.org/techtonik/pypi-techtonik/changeset/5396f8c60d49#comments -- anatoly t. From donald.stufft at gmail.com Wed Jun 13 14:20:05 2012 From: donald.stufft at gmail.com (Donald Stufft) Date: Wed, 13 Jun 2012 08:20:05 -0400 Subject: [Catalog-sig] Dependencies In-Reply-To: References: Message-ID: setuptools is a non standard addition to Python packaging which is impossible to safely extract requirements in a 100% generic way. Distutils2 / metadata 1.2 includes a safe, good way to specify requirements similar to setup tools but it's not ready for primetime yet. On Wednesday, June 13, 2012 at 7:55 AM, anatoly techtonik wrote: > I need to build Fabric RPM package for Fedora. I use py2pack to > generate .spec file, which in turn uses PyPI information for it. The > problem that PyPI completely misses information about dependencies. Is > that intentional, broken or just not implemented? Any previous > discussions/outcomes? > > http://pypi.python.org/pypi/Fabric/json - here should be a reference > to ssh package dependency which is present in setup.py as: > install_requires=['ssh>=1.7.14'], > > > P.S. My patch queue is still awaiting review comments at this changeset > https://bitbucket.org/techtonik/pypi-techtonik/changeset/5396f8c60d49#comments > -- > anatoly t. > _______________________________________________ > Catalog-SIG mailing list > Catalog-SIG at python.org (mailto:Catalog-SIG at python.org) > http://mail.python.org/mailman/listinfo/catalog-sig > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From techtonik at gmail.com Wed Jun 13 14:48:29 2012 From: techtonik at gmail.com (anatoly techtonik) Date: Wed, 13 Jun 2012 15:48:29 +0300 Subject: [Catalog-sig] Dependencies In-Reply-To: References: Message-ID: Thanks for the explanation, but it doesn't directly answer my question. Why PyPI doesn't include dependency information? On Wed, Jun 13, 2012 at 3:20 PM, Donald Stufft wrote: > setuptools is a non standard addition to Python packaging which > is impossible to safely extract requirements in a 100% generic > way. > > Distutils2 / metadata 1.2 includes a safe, good way to specify > requirements similar to setup tools but it's not ready for primetime yet. > > On Wednesday, June 13, 2012 at 7:55 AM, anatoly techtonik wrote: > > I need to build Fabric RPM package for Fedora. I use py2pack to > generate .spec file, which in turn uses PyPI information for it. The > problem that PyPI completely misses information about dependencies. Is > that intentional, broken or just not implemented? Any previous > discussions/outcomes? > > http://pypi.python.org/pypi/Fabric/json - here should be a reference > to ssh package dependency which is present in setup.py as: > install_requires=['ssh>=1.7.14'], > > > P.S. My patch queue is still awaiting review comments at this changeset > https://bitbucket.org/techtonik/pypi-techtonik/changeset/5396f8c60d49#comments > -- > anatoly t. > _______________________________________________ > Catalog-SIG mailing list > Catalog-SIG at python.org > http://mail.python.org/mailman/listinfo/catalog-sig > > From techtonik at gmail.com Wed Jun 13 15:20:49 2012 From: techtonik at gmail.com (anatoly techtonik) Date: Wed, 13 Jun 2012 16:20:49 +0300 Subject: [Catalog-sig] Dependencies In-Reply-To: References: Message-ID: I mean - PyPI is an external project. It is not a part of client/server packaging suite, so why should it depend on whenever some functionality is landed into setuptools, distutils or other stuff? I'll be pretty comfortable with setting dependencies manually for a while. On Wed, Jun 13, 2012 at 3:48 PM, anatoly techtonik wrote: > Thanks for the explanation, but it doesn't directly answer my question. > Why PyPI doesn't include dependency information? > > On Wed, Jun 13, 2012 at 3:20 PM, Donald Stufft wrote: >> setuptools is a non standard addition to Python packaging which >> is impossible to safely extract requirements in a 100% generic >> way. >> >> Distutils2 / metadata 1.2 includes a safe, good way to specify >> requirements similar to setup tools but it's not ready for primetime yet. >> >> On Wednesday, June 13, 2012 at 7:55 AM, anatoly techtonik wrote: >> >> I need to build Fabric RPM package for Fedora. I use py2pack to >> generate .spec file, which in turn uses PyPI information for it. The >> problem that PyPI completely misses information about dependencies. Is >> that intentional, broken or just not implemented? Any previous >> discussions/outcomes? >> >> http://pypi.python.org/pypi/Fabric/json - here should be a reference >> to ssh package dependency which is present in setup.py as: >> install_requires=['ssh>=1.7.14'], >> >> >> P.S. My patch queue is still awaiting review comments at this changeset >> https://bitbucket.org/techtonik/pypi-techtonik/changeset/5396f8c60d49#comments >> -- >> anatoly t. >> _______________________________________________ >> Catalog-SIG mailing list >> Catalog-SIG at python.org >> http://mail.python.org/mailman/listinfo/catalog-sig >> >> From martin at v.loewis.de Wed Jun 13 18:47:02 2012 From: martin at v.loewis.de (martin at v.loewis.de) Date: Wed, 13 Jun 2012 18:47:02 +0200 Subject: [Catalog-sig] Dependencies In-Reply-To: References: Message-ID: <20120613184702.Horde.6JpMJbuWis5P2MQGBsoy-ZA@webmail.df.eu> Zitat von anatoly techtonik : > I need to build Fabric RPM package for Fedora. I use py2pack to > generate .spec file, which in turn uses PyPI information for it. The > problem that PyPI completely misses information about dependencies. Is > that intentional, broken or just not implemented? It's intentional. > Any previous discussions/outcomes? Yes. Search the archives of the previous years. The outcome is that you shall use PEP 345. Regards, Martin From donald.stufft at gmail.com Fri Jun 15 20:36:18 2012 From: donald.stufft at gmail.com (Donald Stufft) Date: Fri, 15 Jun 2012 14:36:18 -0400 Subject: [Catalog-sig] Dependencies In-Reply-To: <4FDB65C6.1040108@python.org> References: <4FDB65C6.1040108@python.org> Message-ID: On Friday, June 15, 2012 at 12:41 PM, Chris Withers wrote: > On 13/06/2012 13:20, Donald Stufft wrote: > > setuptools is a non standard addition to Python packaging which > > is impossible to safely extract requirements in a 100% generic > > way. > > > > > I would argue setuptools is the de facto python packaging dependency > specification, which PyPI should support as a result. > > I've started doing work on this front for Crate, but it really isn't possible to cover all the bases trying to get setuptools dependencies. I've sort of decided to attempt to cover as much of it as I reasonably and safely can and just mark them as setup tools dependencies (vs distutils2/packaging which can be sorted out 100%). > > > Distutils2 / metadata 1.2 includes a safe, good way to specify > > requirements similar to setup tools but it's not ready for primetime yet. > > > > > ...except that only a fraction of packagers even know what these are, > let alone use them... > > Chris > > -- > Simplistix - Content Management, Batch Processing & Python Consulting > - http://www.simplistix.co.uk > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris at python.org Fri Jun 15 18:41:42 2012 From: chris at python.org (Chris Withers) Date: Fri, 15 Jun 2012 17:41:42 +0100 Subject: [Catalog-sig] Dependencies In-Reply-To: References: Message-ID: <4FDB65C6.1040108@python.org> On 13/06/2012 13:20, Donald Stufft wrote: > setuptools is a non standard addition to Python packaging which > is impossible to safely extract requirements in a 100% generic > way. I would argue setuptools is the de facto python packaging dependency specification, which PyPI should support as a result. > Distutils2 / metadata 1.2 includes a safe, good way to specify > requirements similar to setup tools but it's not ready for primetime yet. ...except that only a fraction of packagers even know what these are, let alone use them... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From r1chardj0n3s at gmail.com Sat Jun 16 05:01:58 2012 From: r1chardj0n3s at gmail.com (Richard Jones) Date: Sat, 16 Jun 2012 13:01:58 +1000 Subject: [Catalog-sig] Dependencies In-Reply-To: <4FDB65C6.1040108@python.org> References: <4FDB65C6.1040108@python.org> Message-ID: "impossible to safely extract requirements in a 100% generic way." It has nothing to do with it being the de facto standard and everything to do with executing untrusted code on pydotorg systems with no guarantee that we'll even get the setup.py to work in our environment anyway. Sent from my portable device, please excuse the brevity. On Jun 16, 2012 2:41 AM, "Chris Withers" wrote: > On 13/06/2012 13:20, Donald Stufft wrote: > >> setuptools is a non standard addition to Python packaging which >> is impossible to safely extract requirements in a 100% generic >> way. >> > > I would argue setuptools is the de facto python packaging dependency > specification, which PyPI should support as a result. > > Distutils2 / metadata 1.2 includes a safe, good way to specify >> requirements similar to setup tools but it's not ready for primetime yet. >> > > ...except that only a fraction of packagers even know what these are, let > alone use them... > > Chris > > -- > Simplistix - Content Management, Batch Processing & Python Consulting > - http://www.simplistix.co.uk > ______________________________**_________________ > Catalog-SIG mailing list > Catalog-SIG at python.org > http://mail.python.org/**mailman/listinfo/catalog-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Sat Jun 16 08:41:23 2012 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 16 Jun 2012 08:41:23 +0200 Subject: [Catalog-sig] Dependencies In-Reply-To: <4FDB65C6.1040108@python.org> References: <4FDB65C6.1040108@python.org> Message-ID: <4FDC2A93.1010902@v.loewis.de> On 15.06.2012 18:41, Chris Withers wrote: > On 13/06/2012 13:20, Donald Stufft wrote: >> setuptools is a non standard addition to Python packaging which >> is impossible to safely extract requirements in a 100% generic >> way. > > I would argue setuptools is the de facto python packaging dependency > specification, which PyPI should support as a result. I tried to implement it once, and it was shot down by catalog-sig. Regards, Martin From martin at v.loewis.de Sat Jun 16 08:42:51 2012 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 16 Jun 2012 08:42:51 +0200 Subject: [Catalog-sig] Dependencies In-Reply-To: References: <4FDB65C6.1040108@python.org> Message-ID: <4FDC2AEB.6080904@v.loewis.de> On 16.06.2012 05:01, Richard Jones wrote: > "impossible to safely extract requirements in a 100% generic way." > > It has nothing to do with it being the de facto standard and everything > to do with executing untrusted code on pydotorg systems with no > guarantee that we'll even get the setup.py to work in our environment > anyway. Of course, it would be possible to extract the information in a safe way from packages for which pre-built distributions have been uploaded. Regards, Martin From techtonik at gmail.com Sat Jun 16 11:01:39 2012 From: techtonik at gmail.com (anatoly techtonik) Date: Sat, 16 Jun 2012 12:01:39 +0300 Subject: [Catalog-sig] Dependencies In-Reply-To: References: <4FDB65C6.1040108@python.org> Message-ID: On Sat, Jun 16, 2012 at 6:01 AM, Richard Jones wrote: > "impossible to safely extract requirements in a 100% generic way." > > It has nothing to do with it being the de facto standard and everything to > do with executing untrusted code on pydotorg systems with no guarantee that > we'll even get the setup.py to work in our environment anyway. I'd say that automatic dependency extraction is a problem of package management tools. PyPI is just a catalog, which critical role is to provide ability to store, query and get dependency information. In the end all dependency information is always provided by package maintainer - setup.py or metadata or whatever is only a medium. So if there is a way to submit the info through the API manually - the tools will follow, From jim at zope.com Sat Jun 16 15:03:51 2012 From: jim at zope.com (Jim Fulton) Date: Sat, 16 Jun 2012 09:03:51 -0400 Subject: [Catalog-sig] Dependencies In-Reply-To: References: Message-ID: On Wed, Jun 13, 2012 at 8:20 AM, Donald Stufft wrote: > setuptools is a non standard addition to Python packaging which > is impossible to safely extract requirements in a 100% generic > way. That's not really fair. Setuptools tried hard to build on the standard, distutils and was blessed by Guido at one point. But that's beside the point.... > Distutils2 / metadata 1.2 includes a safe, good way to specify > requirements similar to setup tools but it's not ready for primetime yet. Perhaps support for the new meta-data configuration files could be added to distribute. (Perhaps it has already?) If distribute supported (or with help, could support) the new meta-data standard, that would allow us to move forward on proven software. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm From tseaver at palladion.com Sun Jun 17 18:24:12 2012 From: tseaver at palladion.com (Tres Seaver) Date: Sun, 17 Jun 2012 12:24:12 -0400 Subject: [Catalog-sig] Dependencies In-Reply-To: References: <4FDB65C6.1040108@python.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 06/15/2012 11:01 PM, Richard Jones wrote: > "impossible to safely extract requirements in a 100% generic way." > > It has nothing to do with it being the de facto standard and > everything to do with executing untrusted code on pydotorg systems > with no guarantee that we'll even get the setup.py to work in our > environment anyway. > > Sent from my portable device, please excuse the brevity. On Jun 16, > 2012 2:41 AM, "Chris Withers" wrote: > >> On 13/06/2012 13:20, Donald Stufft wrote: >> >>> setuptools is a non standard addition to Python packaging which is >>> impossible to safely extract requirements in a 100% generic way. You can avoid executing 'setup.py' by looking for 'requires.txt' in the egg-info directory within the sdist. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/eBKYACgkQ+gerLs4ltQ6vjQCg0h8HL6wX/1KjPNpXBQqQJwK1 SGAAn2EW0HK6eUYRVsE1UtFP+RjsmiaE =+THe -----END PGP SIGNATURE----- From jim at zope.com Sun Jun 17 19:01:44 2012 From: jim at zope.com (Jim Fulton) Date: Sun, 17 Jun 2012 13:01:44 -0400 Subject: [Catalog-sig] Dependencies In-Reply-To: References: <4FDB65C6.1040108@python.org> Message-ID: On Sun, Jun 17, 2012 at 12:24 PM, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 06/15/2012 11:01 PM, Richard Jones wrote: >> "impossible to safely extract requirements in a 100% generic way." >> >> It has nothing to do with it being the de facto standard and >> everything to do with executing untrusted code on pydotorg systems >> with no guarantee that we'll even get the setup.py to work in our >> environment anyway. >> >> Sent from my portable device, please excuse the brevity. On Jun 16, >> 2012 2:41 AM, "Chris Withers" wrote: >> >>> On 13/06/2012 13:20, Donald Stufft wrote: >>> >>>> setuptools is a non standard addition to Python packaging which is >>>> impossible to safely extract requirements in a 100% generic way. > > You can avoid executing 'setup.py' by looking for 'requires.txt' in the > egg-info directory within the sdist. Except that sdists don't have egg-info directories, presumably because egg-info can depend on the environment the project is installed in. For example, it's not unheard of for dependencies to depend on the Python version (e.g. json). Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm From donald.stufft at gmail.com Sun Jun 17 19:02:48 2012 From: donald.stufft at gmail.com (Donald Stufft) Date: Sun, 17 Jun 2012 13:02:48 -0400 Subject: [Catalog-sig] Dependencies In-Reply-To: References: <4FDB65C6.1040108@python.org> Message-ID: <924C4AB2A21546748E3E8E4C022C12E9@gmail.com> On Sunday, June 17, 2012 at 12:24 PM, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 06/15/2012 11:01 PM, Richard Jones wrote: > > "impossible to safely extract requirements in a 100% generic way." > > > > It has nothing to do with it being the de facto standard and > > everything to do with executing untrusted code on pydotorg systems > > with no guarantee that we'll even get the setup.py to work in our > > environment anyway. > > > > Sent from my portable device, please excuse the brevity. On Jun 16, > > 2012 2:41 AM, "Chris Withers" wrote: > > > > > On 13/06/2012 13:20, Donald Stufft wrote: > > > > > > > setuptools is a non standard addition to Python packaging which is > > > > impossible to safely extract requirements in a 100% generic way. > > > > > > > > > > > > > > > > > You can avoid executing 'setup.py' by looking for 'requires.txt' in the > egg-info directory within the sdist. > > This will only include dependencies required for the system that wrote the package. It won't include dependencies required on other systems (say via an if statement for windows platform). Hence the impossible to do it completely. > > > > Tres. > - -- > =================================================================== > Tres Seaver +1 540-429-0999 tseaver at palladion.com (mailto:tseaver at palladion.com) > Palladion Software "Excellence by Design" http://palladion.com > > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.11 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEARECAAYFAk/eBKYACgkQ+gerLs4ltQ6vjQCg0h8HL6wX/1KjPNpXBQqQJwK1 > SGAAn2EW0HK6eUYRVsE1UtFP+RjsmiaE > =+THe > -----END PGP SIGNATURE----- > > _______________________________________________ > Catalog-SIG mailing list > Catalog-SIG at python.org (mailto:Catalog-SIG at python.org) > http://mail.python.org/mailman/listinfo/catalog-sig > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tseaver at palladion.com Sun Jun 17 19:17:26 2012 From: tseaver at palladion.com (Tres Seaver) Date: Sun, 17 Jun 2012 13:17:26 -0400 Subject: [Catalog-sig] Dependencies In-Reply-To: <924C4AB2A21546748E3E8E4C022C12E9@gmail.com> References: <4FDB65C6.1040108@python.org> <924C4AB2A21546748E3E8E4C022C12E9@gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 06/17/2012 01:02 PM, Donald Stufft wrote: > On Sunday, June 17, 2012 at 12:24 PM, Tres Seaver wrote: > You can avoid executing 'setup.py' by looking for 'requires.txt' in > the egg-info directory within the sdist. > >> This will only include dependencies required for the system that >> wrote the package. It won't include dependencies required on other >> systems (say via an if statement for windows platform). > >> Hence the impossible to do it completely. Surely it isn't perfect, but it would be correct for most packages on PyPI today, and "close enough" to be useful even for the rest. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/eESYACgkQ+gerLs4ltQ6ooQCeLWpShxnb0Luw6qlfOmh2bxaH L1UAn36lE5618yTTONvVFOtAMZSv0ytx =/QSx -----END PGP SIGNATURE----- From tseaver at palladion.com Sun Jun 17 19:18:52 2012 From: tseaver at palladion.com (Tres Seaver) Date: Sun, 17 Jun 2012 13:18:52 -0400 Subject: [Catalog-sig] Dependencies In-Reply-To: References: <4FDB65C6.1040108@python.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 06/17/2012 01:01 PM, Jim Fulton wrote: > On Sun, Jun 17, 2012 at 12:24 PM, Tres Seaver > wrote: >> You can avoid executing 'setup.py' by looking for 'requires.txt' in >> the egg-info directory within the sdist. > > Except that sdists don't have egg-info directories, presumably > because egg-info can depend on the environment the project is > installed in. For example, it's not unheard of for dependencies to > depend on the Python version (e.g. json). sdists do have egg-info:: $ svn co http://svn.zope.org/repos/main/zope.interface/trunk xxx ... $ cd xxx $ /path/to/python setup.py sdist ... $ tar tzf dist/zope.interface-4.0.2dev.tar.gz | grep requires zope.interface-4.0.2dev/src/zope.interface.egg-info/requires.txt Now, that doesn't mean that the contents of requires.txt might not be different when installed on a different platform or Python version, but for the vast majority of projects (/me waves hands), this works just fine. Tres. - - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com - -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/eEH0ACgkQ+gerLs4ltQ42vgCeNAA0yNZB9YrZTGajI+HlXQJK 5LcAoLS+7KjDMxfWf6S0ncLu0SKxQSZh =r7TG - -----END PGP SIGNATURE----- - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/eEXwACgkQ+gerLs4ltQ5w+gCdHRlCLTL/vEASnz9JLxkJAY+K e58An3soJ41FEQykuNdYdpeXPM+hqT9W =pVB1 -----END PGP SIGNATURE----- From lunaryorn at googlemail.com Mon Jun 18 19:47:26 2012 From: lunaryorn at googlemail.com (Sebastian Wiesner) Date: Mon, 18 Jun 2012 19:47:26 +0200 Subject: [Catalog-sig] ReST not rendered for sphinxcontrib-issuetracker Message-ID: Hello, the long description of sphinxcontrib-issuetracker [1] is not properly rendered into HTML, though it is syntactically correct ReST, "python setup.py --long-description | rst2html > /dev/null" doesn't show warnings or errors. Any ideas? Sebastian Wiesner From martin at v.loewis.de Mon Jun 18 23:38:06 2012 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 18 Jun 2012 23:38:06 +0200 Subject: [Catalog-sig] ReST not rendered for sphinxcontrib-issuetracker In-Reply-To: References: Message-ID: <4FDF9FBE.60608@v.loewis.de> On 18.06.2012 19:47, Sebastian Wiesner wrote: > Hello, > > the long description of sphinxcontrib-issuetracker [1] is not properly > rendered into HTML, though it is syntactically correct ReST, "python > setup.py --long-description | rst2html > /dev/null" doesn't show > warnings or errors. > > Any ideas? The PyPI rendering has restrictions as to what you can do in the ReST, see https://bitbucket.org/loewis/pypi/src/0bcc875f5130/description_utils.py Most likely, it fails some of the restrictions. Regards, Martin From djay at pretaweb.com Tue Jun 19 00:32:34 2012 From: djay at pretaweb.com (Dylan Jay) Date: Tue, 19 Jun 2012 08:32:34 +1000 Subject: [Catalog-sig] Dependencies In-Reply-To: References: Message-ID: <7789515265364996792@unknownmsgid> On 16/06/2012, at 11:04 PM, Jim Fulton wrote: > On Wed, Jun 13, 2012 at 8:20 AM, Donald Stufft wrote: >> setuptools is a non standard addition to Python packaging which >> is impossible to safely extract requirements in a 100% generic >> way. > > That's not really fair. Setuptools tried hard to build on the > standard, distutils > and was blessed by Guido at one point. But that's beside the point.... > >> Distutils2 / metadata 1.2 includes a safe, good way to specify >> requirements similar to setup tools but it's not ready for primetime yet. > > Perhaps support for the new meta-data configuration files could be > added to distribute. (Perhaps it has already?) > > If distribute supported (or with help, could support) the new meta-data > standard, that would allow us to move forward on proven software. +1 Particularly if it was on by default. Then for the 99.9% of packages that have the same set of dependencies regardless of environment, on their next release we'd have dependencies pypi would have no excuse not to read. Then it would finally be possible for someone to make tools that take dependency information into account *before* downloading software, like every other decent package management system in the world. > > Jim > > -- > Jim Fulton > http://www.linkedin.com/in/jimfulton > Jerky is better than bacon! http://zo.pe/Kqm > _______________________________________________ > Catalog-SIG mailing list > Catalog-SIG at python.org > http://mail.python.org/mailman/listinfo/catalog-sig From lunaryorn at googlemail.com Mon Jun 18 23:47:34 2012 From: lunaryorn at googlemail.com (Sebastian Wiesner) Date: Mon, 18 Jun 2012 23:47:34 +0200 Subject: [Catalog-sig] ReST not rendered for sphinxcontrib-issuetracker In-Reply-To: <4FDF9FBE.60608@v.loewis.de> References: <4FDF9FBE.60608@v.loewis.de> Message-ID: 2012/6/18 "Martin v. L?wis" : > On 18.06.2012 19:47, Sebastian Wiesner wrote: >> Hello, >> >> the long description of sphinxcontrib-issuetracker [1] is not properly >> rendered into HTML, though it is syntactically correct ReST, "python >> setup.py --long-description | rst2html > /dev/null" doesn't show >> warnings or errors. >> >> Any ideas? > > The PyPI rendering has restrictions as to what you can do in the ReST, > see > > https://bitbucket.org/loewis/pypi/src/0bcc875f5130/description_utils.py > > Most likely, it fails some of the restrictions. I searched for the PyPI sources to debug the problem, but failed to find them. Thanks for pointing me to them, now I can debug this problem. From lunaryorn at googlemail.com Tue Jun 19 11:59:53 2012 From: lunaryorn at googlemail.com (Sebastian Wiesner) Date: Tue, 19 Jun 2012 11:59:53 +0200 Subject: [Catalog-sig] ReST not rendered for sphinxcontrib-issuetracker In-Reply-To: References: <4FDF9FBE.60608@v.loewis.de> Message-ID: 2012/6/18 Sebastian Wiesner : > 2012/6/18 "Martin v. L?wis" : >> On 18.06.2012 19:47, Sebastian Wiesner wrote: >>> Hello, >>> >>> the long description of sphinxcontrib-issuetracker [1] is not properly >>> rendered into HTML, though it is syntactically correct ReST, "python >>> setup.py --long-description | rst2html > /dev/null" doesn't show >>> warnings or errors. >>> >>> Any ideas? >> >> The PyPI rendering has restrictions as to what you can do in the ReST, >> see >> >> https://bitbucket.org/loewis/pypi/src/0bcc875f5130/description_utils.py >> >> Most likely, it fails some of the restrictions. > > I searched for the PyPI sources to debug the problem, but failed to > find them. Thanks for pointing me to them, now I can debug this > problem. I was able to fix the rendering based on the source code you linked. The cause was a missing colon in a link, causing the scheme check to fail. Thanks for your help. From asmeurer at gmail.com Sat Jun 23 02:21:10 2012 From: asmeurer at gmail.com (Aaron Meurer) Date: Fri, 22 Jun 2012 18:21:10 -0600 Subject: [Catalog-sig] Flag to tell pip to only install uploaded files Message-ID: <5256255447244850384@unknownmsgid> Hi. I'm following up on a discussion on the pip mailing list (https://groups.google.com/forum/#!topic/python-virtualenv/PZNj9pC6aKA/discussion), where I was directed here. Would it be possible to add some kind of a flag to PyPI that would let package maintainers tell pip to install only the uploaded file (or possibly also the file given by a direct link), and no others? Currently, pip aggressively tries to find the latest version of a package by crawling all links on the PyPI page, even those from older versions. This is a headache to me as a package maintainer because it means that pip is quite often installing the wrong thing. Recently, pip was trying to install our html docs because we had a file uploaded at Google Code named "sympy-0.7.1-html-docs", which it deemed to be a newer version than "sympy-0.7.1". There's also the issue that every time we put out a release candidate for a new version, pip starts installing that, when I would prefer it to only install stable final releases. It's also, as I noted on the other discussion list, a bit of a security risk. According to the pip guys (namely, Carl Meyer), this is not so easy to change from their end because of backwards compatibility issues. I suggested that such a flag be added to PyPI, and they told me that if it were, they would accept a patch supporting it in pip. This would make it much less of a headache for me as a package maintainer, because I could know that pip will always install exactly what I want. It could be off by default to enable backwards compatibility. Aaron Meurer From pje at telecommunity.com Sat Jun 23 02:58:28 2012 From: pje at telecommunity.com (PJ Eby) Date: Fri, 22 Jun 2012 20:58:28 -0400 Subject: [Catalog-sig] Flag to tell pip to only install uploaded files In-Reply-To: <5256255447244850384@unknownmsgid> References: <5256255447244850384@unknownmsgid> Message-ID: On Fri, Jun 22, 2012 at 8:21 PM, Aaron Meurer wrote: > Hi. > > I'm following up on a discussion on the pip mailing list > ( > https://groups.google.com/forum/#!topic/python-virtualenv/PZNj9pC6aKA/discussion > ), > where I was directed here. > > Would it be possible to add some kind of a flag to PyPI that would let > package maintainers tell pip to install only the uploaded file (or > possibly also the file given by a direct link), and no others? > > Currently, pip aggressively tries to find the latest version of a > package by crawling all links on the PyPI page, even those from older > versions. This is a headache to me as a package maintainer because it > means that pip is quite often installing the wrong thing. Recently, > pip was trying to install our html docs because we had a file uploaded > at Google Code named "sympy-0.7.1-html-docs", The simple way to correct this problem is to rename the file 'sympy-html-docs-0.7.1' - this will fix things for all installers that follow easy_install's discovery protocol, including pip and zc.buildout. > which it deemed to be a > newer version than "sympy-0.7.1". There's also the issue that every > time we put out a release candidate for a new version, pip starts > installing that, when I would prefer it to only install stable final > releases. It's also, as I noted on the other discussion list, a bit > of a security risk. > zc.buildout includes a flag to prefer stable releases, and I believe some other installation tools do as well. You might suggest they add such a flag to pip and move towards using it by default. -------------- next part -------------- An HTML attachment was scrubbed... URL: From asmeurer at gmail.com Sat Jun 23 03:45:20 2012 From: asmeurer at gmail.com (Aaron Meurer) Date: Fri, 22 Jun 2012 19:45:20 -0600 Subject: [Catalog-sig] Flag to tell pip to only install uploaded files In-Reply-To: References: <5256255447244850384@unknownmsgid> Message-ID: <-9162853598252893873@unknownmsgid> On Jun 22, 2012, at 6:58 PM, PJ Eby wrote: On Fri, Jun 22, 2012 at 8:21 PM, Aaron Meurer wrote: > Hi. > > I'm following up on a discussion on the pip mailing list > ( > https://groups.google.com/forum/#!topic/python-virtualenv/PZNj9pC6aKA/discussion > ), > where I was directed here. > > Would it be possible to add some kind of a flag to PyPI that would let > package maintainers tell pip to install only the uploaded file (or > possibly also the file given by a direct link), and no others? > > Currently, pip aggressively tries to find the latest version of a > package by crawling all links on the PyPI page, even those from older > versions. This is a headache to me as a package maintainer because it > means that pip is quite often installing the wrong thing. Recently, > pip was trying to install our html docs because we had a file uploaded > at Google Code named "sympy-0.7.1-html-docs", The simple way to correct this problem is to rename the file 'sympy-html-docs-0.7.1' - this will fix things for all installers that follow easy_install's discovery protocol, including pip and zc.buildout. Yes, I did this. But it doesn't solve the issue of installing our release candidates, or trying to install who knows what because of the discovery "protocol" (which I would call the discovery magic). > which it deemed to be a > newer version than "sympy-0.7.1". There's also the issue that every > time we put out a release candidate for a new version, pip starts > installing that, when I would prefer it to only install stable final > releases. It's also, as I noted on the other discussion list, a bit > of a security risk. > zc.buildout includes a flag to prefer stable releases, and I believe some other installation tools do as well. You might suggest they add such a flag to pip and move towards using it by default. The pip guys don't want to make this change, I guess because of the problems it would cause with who knows how many packages wouldn't be following this. See the discussion I linked to. I do agree that this would be the better way to do it (but I can think of about a thousand "better ways to do it" as far as Python packaging is concerned, but none of them will happen, or at least not within the timeline that I'm hoping for). Aaron Meurer -------------- next part -------------- An HTML attachment was scrubbed... URL: From donald.stufft at gmail.com Sat Jun 23 03:50:09 2012 From: donald.stufft at gmail.com (Donald Stufft) Date: Fri, 22 Jun 2012 21:50:09 -0400 Subject: [Catalog-sig] Flag to tell pip to only install uploaded files In-Reply-To: <5256255447244850384@unknownmsgid> References: <5256255447244850384@unknownmsgid> Message-ID: <065322F6F6C4435BAF5E7490C750C57E@gmail.com> pip install -i https://restricted.crate.io/ (Restricted is an endpoint of Crate that _only_ lists uploaded files) Not exactly what you were looking for, but it's *a* solution. On Friday, June 22, 2012 at 8:21 PM, Aaron Meurer wrote: > Hi. > > I'm following up on a discussion on the pip mailing list > (https://groups.google.com/forum/#!topic/python-virtualenv/PZNj9pC6aKA/discussion), > where I was directed here. > > Would it be possible to add some kind of a flag to PyPI that would let > package maintainers tell pip to install only the uploaded file (or > possibly also the file given by a direct link), and no others? > > Currently, pip aggressively tries to find the latest version of a > package by crawling all links on the PyPI page, even those from older > versions. This is a headache to me as a package maintainer because it > means that pip is quite often installing the wrong thing. Recently, > pip was trying to install our html docs because we had a file uploaded > at Google Code named "sympy-0.7.1-html-docs", which it deemed to be a > newer version than "sympy-0.7.1". There's also the issue that every > time we put out a release candidate for a new version, pip starts > installing that, when I would prefer it to only install stable final > releases. It's also, as I noted on the other discussion list, a bit > of a security risk. > > According to the pip guys (namely, Carl Meyer), this is not so easy to > change from their end because of backwards compatibility issues. I > suggested that such a flag be added to PyPI, and they told me that if > it were, they would accept a patch supporting it in pip. This would > make it much less of a headache for me as a package maintainer, > because I could know that pip will always install exactly what I want. > It could be off by default to enable backwards compatibility. > > Aaron Meurer > _______________________________________________ > Catalog-SIG mailing list > Catalog-SIG at python.org (mailto:Catalog-SIG at python.org) > http://mail.python.org/mailman/listinfo/catalog-sig > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carl at oddbird.net Sat Jun 23 05:03:14 2012 From: carl at oddbird.net (Carl Meyer) Date: Fri, 22 Jun 2012 21:03:14 -0600 Subject: [Catalog-sig] Flag to tell pip to only install uploaded files In-Reply-To: <-9162853598252893873@unknownmsgid> References: <5256255447244850384@unknownmsgid> <-9162853598252893873@unknownmsgid> Message-ID: <4FE531F2.40601@oddbird.net> On 06/22/2012 07:45 PM, Aaron Meurer wrote: > On Jun 22, 2012, at 6:58 PM, PJ Eby > wrote: >> zc.buildout includes a flag to prefer stable releases, and I believe >> some other installation tools do as well. You might suggest they add >> such a flag to pip and move towards using it by default. >> > > The pip guys don't want to make this change, I guess because of the > problems it would cause with who knows how many packages wouldn't be > following this. See the discussion I linked to. I do agree that this > would be the better way to do it (but I can think of about a thousand > "better ways to do it" as far as Python packaging is concerned, but none > of them will happen, or at least not within the timeline that I'm hoping > for). To be clear, I'd have no objection to adding such a flag to pip, presuming someone's willing to contribute a pull request for it (probably borrowing the what-is-a-final-release algorithm from zc.buildout so we are consistent). I don't think the flag would really achieve what you are looking for unless it became the default; that's the part that is (at best) a ways out in the future. I guess if the flag were present you could at least tell your users to use it... Carl