From pje at telecommunity.com Thu Feb 1 01:07:27 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 31 Jan 2007 19:07:27 -0500 Subject: [Distutils] Optional C extensions in packages Message-ID: <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> I'm thinking setuptools should allow you to specify that one or more C extensions in a project are optional. At minimum, there would then be a command-line option you could give to easy_install to say, "don't build any optional extensions", so if you don't have a compiler or need a cross-platform egg, you can skip those extensions. There would also need to be a way to specify this when using the regular build commands. This feature would address projects that include optional C versions of certain code, that can fall back to pure Python implementations. It would not address projects that have C extensions requiring external libraries that might or might not exist. (Such as Twisted, for instance, but Twisted falls into the category of projects requiring "extreme measures" to be supported anyway.) Comments, anyone? From jim at zope.com Thu Feb 1 13:11:16 2007 From: jim at zope.com (Jim Fulton) Date: Thu, 01 Feb 2007 07:11:16 -0500 Subject: [Distutils] Optional C extensions in packages In-Reply-To: <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> References: <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> Message-ID: <45C1D8E4.7030007@zope.com> Phillip J. Eby wrote: > I'm thinking setuptools should allow you to specify that one or more C > extensions in a project are optional. > > At minimum, there would then be a command-line option you could give to > easy_install to say, "don't build any optional extensions", so if you don't > have a compiler or need a cross-platform egg, you can skip those extensions. > > There would also need to be a way to specify this when using the regular > build commands. > > This feature would address projects that include optional C versions of > certain code, that can fall back to pure Python implementations. > > It would not address projects that have C extensions requiring external > libraries that might or might not exist. (Such as Twisted, for instance, > but Twisted falls into the category of projects requiring "extreme > measures" to be supported anyway.) > > Comments, anyone? I like the goal, since most of our extensions are in this category, however, I have a feeling that this use case might be better served by packaging the extensions as separate distributions. Once concern I have is that I think I'd want an egg with the extensions and the egg without the extensions to have different names, and perhaps different requirements. Of course, the names would be different because an egg with a C extension would be platform dependent and an egg without would be platform independent. What if both a platform-independent egg and a platform-dependent egg were available? Which would take precedence? I think they have the same precedence now. I think I'd be +1 if platform-dependent eggs were prefered over platform-independent ones. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From chris at trichech.us Thu Feb 1 15:15:54 2007 From: chris at trichech.us (Chris Fonnesbeck) Date: Thu, 1 Feb 2007 09:15:54 -0500 Subject: [Distutils] setupegg.py omits important file Message-ID: <723eb6930702010615r26279ea9g951882dd5f5db596@mail.gmail.com> I have built a matplotlib installer for OS X, using setupegg.py: python setupegg.py config -L../staticlibs build bdist_mpkg However, when I import the module after installing, I get the follwing: ImportError: No module named matplotlib.pylab The problem appears to be that the module's __init__.py is not included in the package; when I do a standard install using setup.py, the file is copied over as it should be. Another oddity is that this only happens on my Intel Mac; it seems to work fine on PPC. Any advice on this is appreciated. -- Chris Fonnesbeck + Atlanta, GA + http://trichech.us -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070201/fcdf252b/attachment.htm From jim at zope.com Thu Feb 1 18:15:16 2007 From: jim at zope.com (Jim Fulton) Date: Thu, 1 Feb 2007 12:15:16 -0500 Subject: [Distutils] Building an egg from Subversion using zc.buildout In-Reply-To: References: <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <45BE6CAF.3010206@gmx.net> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <1163956881.5763.21.camel@Jane> <5.1.1.6.0.20061119134913.01f3d088@sparrow.telecommunity.com> <45BD22C2.7030403@colorstudy.com> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <5.1.1.6.0.20070128202027.04461008@sparrow.telecommunity.com> <9D0FD265-B323-42C3-80A1-85638A067C9A@zope.com> <45BE6CAF.3010206@gmx.net> <65788D22-5FA7-4AC1-A6FA-08FDA4D173FE@zope.com> <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <5.1.1.6.0.20070129180219.045646e0@sparrow.telecommunity.com> Message-ID: <15D4D085-221F-400B-B231-DB1986764B56@zope.com> Note that I'm not ignoring this. I just need to find some time to dig into it. Jim On Jan 29, 2007, at 6:14 PM, Martin Aspeli wrote: > Phillip J. Eby wrote: >> At 10:37 PM 1/29/2007 +0000, Martin Aspeli wrote: >>> Phillip J. Eby wrote: >>> >>>> Development snapshots of setuptools can be checked out with >>>> "setuptools==dev", and that will always download a new version >>>> because the >>>> installed version is always a version like 0.6c2dev-r51290 or >>>> something. This doesn't match ==dev, thereby forcing >>>> easy_install to >>>> download a fresh copy. >>> Could you explain how the find-link would work with an ==dev version >>> specification? >> >> #egg=archetypes.kss-dev >> >> That is, use 'dev' as the version in the link. The combination of >> an ==dev >> version specification and a #egg=projectname-dev tag means that >> the code >> will be downloaded and rebuilt *every* time, however. > > Alas, this gives me this error: > > Traceback (most recent call last): > File "./bin/buildout", line 12, in ? > zc.buildout.buildout.main() > File > "/private/tmp/tmp29h1vY/zc.buildout-1.0.0b19-py2.4.egg/zc/buildout/ > buildout.py", > line 1056, in main > File > "/private/tmp/tmp29h1vY/zc.buildout-1.0.0b19-py2.4.egg/zc/buildout/ > buildout.py", > line 309, in install > File > "/Users/optilude/Development/Plone/Code/Products/ZopeSkel/trunk/ > test/bot/eggs/z2c.recipe.zope2instance-0.1dev_r12104-py2.4.egg/z2c/ > recipe/zope2instance/__init__.py", > line 36, in install > requirements, ws = self.egg.working_set() > File > "/Users/optilude/Development/Plone/Code/Products/ZopeSkel/trunk/ > test/bot/eggs/zc.recipe.egg-1.0.0b4-py2.4.egg/zc/recipe/egg/egg.py", > line 79, in working_set > path=[options['develop-eggs-directory']] > File > "/Users/optilude/Development/Plone/Code/Products/ZopeSkel/trunk/ > test/bot/eggs/setuptools-0.6c5-py2.4.egg/easy_install.py", > line 355, in install > > File > "/Users/optilude/Development/Plone/Code/Products/ZopeSkel/trunk/ > test/bot/eggs/setuptools-0.6c5-py2.4.egg/easy_install.py", > line 294, in _get_dist > > ValueError: ("Couldn't find", Requirement.parse > ('archetypes.kss==dev')) > > Is there something else I need to do to the svn repository > (http://svn.plone.org/svn/archetypes/archetypes.kss/trunk)? Or is > this a > zc.buildout limitation? > > Martin > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From pje at telecommunity.com Thu Feb 1 19:03:53 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 01 Feb 2007 13:03:53 -0500 Subject: [Distutils] Optional C extensions in packages In-Reply-To: <45C1D8E4.7030007@zope.com> References: <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070201125400.02794c90@sparrow.telecommunity.com> At 07:11 AM 2/1/2007 -0500, Jim Fulton wrote: >I like the goal, since most of our extensions are in this category, >however, I have a feeling that this use case might be better served >by packaging the extensions as separate distributions. True, but without automation, it's more work for the package author to "do the right thing". If your extension is in a package, you have to make it a namespace package. You have to create another setup.py and directory structure, and manage another set of releases. This is a lot more work than just marking the extension as "optional". >Once concern I have is that I think I'd want an egg with the extensions >and the egg without the extensions to have different names, and perhaps >different requirements. Of course, the names would be different >because an egg with a C extension would be platform dependent and an >egg without would be platform independent. What if both a >platform-independent egg and a platform-dependent egg were available? >Which would take precedence? I think they have the same precedence now. > >I think I'd be +1 if platform-dependent eggs were prefered over >platform-independent ones. ...of the same version? That's easy enough to do by changing the Distribution.hashcmp property to put the 'platform' part earlier in the comparison. If they're different versions, however, it's a whole 'nother kettle of fish. The solution I was thinking of, however, has the additional benefit of working right now. If you're generating eggs for a package with optional extensions, you would simply *not* generate a cross-platform egg, but instead just ship an sdist tarball, plus platform-specific eggs for the platforms you care about. That way, anybody on an unsupported platform will end up building a local egg from the source, and either producing the extensions or skipping them locally. (And if you don't have access to a compiler, it seems unlikely that you will be then *distributing* the eggs you produced.) Notice that this setup is a natural side-effect of the way most people would produce and upload packages to the Cheeseshop anyway - you would have to make an intentional effort to suppress extension-building and upload a cross-platform egg; otherwise you will just end up with a source distribution and a platform-specific egg, which then works as intended. From theller at ctypes.org Thu Feb 1 19:37:39 2007 From: theller at ctypes.org (Thomas Heller) Date: Thu, 01 Feb 2007 19:37:39 +0100 Subject: [Distutils] setuptools problem? Message-ID: Philip, I was trying to install yolk with c:\python25\python -m easy_install yolk (on Windows, as you can see). A yolk.py script is installed in c:\Python25\Scripts, but between each source line there is an empty line. In other words, line feeds are doubled each time. This leads to syntax errors because continuation lines don't work any longer. It seems easy_install gets the newlines wrong. Thanks, Thomas PS: This is not the first time I had strange problems with Scripts in distributions installed by easy_install, but it is the first time I'm reporting it. From pje at telecommunity.com Thu Feb 1 20:05:25 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 01 Feb 2007 14:05:25 -0500 Subject: [Distutils] setuptools problem? In-Reply-To: Message-ID: <5.1.1.6.0.20070201135832.02851ff8@sparrow.telecommunity.com> At 07:37 PM 2/1/2007 +0100, Thomas Heller wrote: >Philip, > >I was trying to install yolk with > >c:\python25\python -m easy_install yolk > >(on Windows, as you can see). > >A yolk.py script is installed in c:\Python25\Scripts, but between each >source line >there is an empty line. In other words, line feeds are doubled each time. >This leads to syntax errors because continuation lines don't work any longer. > >It seems easy_install gets the newlines wrong. Yep, the problem was exclusive to "old-style" scripts, which yolk is using for some reason. (It could just move yolk's main() function to yolklib and list in the the setup script's entry points.) It's fixed it now in SVN. From theller at ctypes.org Thu Feb 1 20:22:49 2007 From: theller at ctypes.org (Thomas Heller) Date: Thu, 01 Feb 2007 20:22:49 +0100 Subject: [Distutils] setuptools problem? In-Reply-To: <5.1.1.6.0.20070201135832.02851ff8@sparrow.telecommunity.com> References: <5.1.1.6.0.20070201135832.02851ff8@sparrow.telecommunity.com> Message-ID: Phillip J. Eby schrieb: > At 07:37 PM 2/1/2007 +0100, Thomas Heller wrote: >>Philip, >> >>I was trying to install yolk with >> >>c:\python25\python -m easy_install yolk >> >>(on Windows, as you can see). >> >>A yolk.py script is installed in c:\Python25\Scripts, but between each >>source line >>there is an empty line. In other words, line feeds are doubled each time. >>This leads to syntax errors because continuation lines don't work any longer. >> >>It seems easy_install gets the newlines wrong. > > Yep, the problem was exclusive to "old-style" scripts, which yolk is using > for some reason. (It could just move yolk's main() function to yolklib and > list in the the setup script's entry points.) > > It's fixed it now in SVN. > and it works for me, now. Boy, are you quick! For my education: what are "old-style" scripts? Thanks, Thomas From pje at telecommunity.com Thu Feb 1 21:04:29 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 01 Feb 2007 15:04:29 -0500 Subject: [Distutils] setuptools problem? In-Reply-To: References: <5.1.1.6.0.20070201135832.02851ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070201135832.02851ff8@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070201144438.0276efb0@sparrow.telecommunity.com> At 08:22 PM 2/1/2007 +0100, Thomas Heller wrote: >Phillip J. Eby schrieb: > > At 07:37 PM 2/1/2007 +0100, Thomas Heller wrote: > >>Philip, > >> > >>I was trying to install yolk with > >> > >>c:\python25\python -m easy_install yolk > >> > >>(on Windows, as you can see). > >> > >>A yolk.py script is installed in c:\Python25\Scripts, but between each > >>source line > >>there is an empty line. In other words, line feeds are doubled each time. > >>This leads to syntax errors because continuation lines don't work any > longer. > >> > >>It seems easy_install gets the newlines wrong. > > > > Yep, the problem was exclusive to "old-style" scripts, which yolk is using > > for some reason. (It could just move yolk's main() function to yolklib > and > > list in the the setup script's entry points.) > > > > It's fixed it now in SVN. > > > >and it works for me, now. Boy, are you quick! And just as quick to create new bugs... see below. :( >For my education: what are "old-style" scripts? A script specified in the 'scripts' option to setup(), as opposed to via entry_points. Now, what I just realized when I typed that answer above, is that normally Python scripts specified in setup() would be installed using a wrapper that handles require()-ing the package's dependencies. So, the real problem turned out to be that the 'yolk.py' script was not being detected as being a *Python* script, if the build_scripts command was run on Windows. This would put a #! line in place with 'python.exe' instead of 'python', which distutils' "first_line_re" pattern would not match at install time. So, when I fixed the broken line endings, I missed the fact that the broken code should never have been running at all. In effect, only non-Python scripts should've had their contents messed up! (Well, they shouldn't be messed up, but the point is that there were *two* bugs here, and I missed one of them.) So, now I've fixed the script language detection to be more lenient regarding #! lines, and in fact it will only bother with the #! line if the script filename is not .py/.pyw and the content isn't syntactically valid Python. (By the way, you might want to mention to the yolk author that his/her package depends on ElementTree, but it's not listed in the project's install_requires. So, when I tried running the 'yolk' script it failed because I didn't have ElementTree installed.) From theller at ctypes.org Thu Feb 1 21:47:39 2007 From: theller at ctypes.org (Thomas Heller) Date: Thu, 01 Feb 2007 21:47:39 +0100 Subject: [Distutils] setuptools problem? In-Reply-To: <5.1.1.6.0.20070201144438.0276efb0@sparrow.telecommunity.com> References: <5.1.1.6.0.20070201135832.02851ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070201135832.02851ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070201144438.0276efb0@sparrow.telecommunity.com> Message-ID: Phillip J. Eby schrieb: >>For my education: what are "old-style" scripts? > > A script specified in the 'scripts' option to setup(), as opposed to via > entry_points. > > Now, what I just realized when I typed that answer above, is that normally > Python scripts specified in setup() would be installed using a wrapper that > handles require()-ing the package's dependencies. > > So, the real problem turned out to be that the 'yolk.py' script was not > being detected as being a *Python* script, if the build_scripts command was > run on Windows. This would put a #! line in place with 'python.exe' > instead of 'python', which distutils' "first_line_re" pattern would not > match at install time. > > So, when I fixed the broken line endings, I missed the fact that the broken > code should never have been running at all. In effect, only non-Python > scripts should've had their contents messed up! (Well, they shouldn't be > messed up, but the point is that there were *two* bugs here, and I missed > one of them.) > > So, now I've fixed the script language detection to be more lenient > regarding #! lines, and in fact it will only bother with the #! line if the > script filename is not .py/.pyw and the content isn't syntactically valid > Python. This explains at least one of the other problems that I didn't remember first. IIRC, easy_install also added a #!whatever line to batch files ;-), but it I have then removed these packages (or installed them in the traditional way). > (By the way, you might want to mention to the yolk author that his/her > package depends on ElementTree, but it's not listed in the project's > install_requires. So, when I tried running the 'yolk' script it failed > because I didn't have ElementTree installed.) Yes, I found that also, and even noticed another problem (I post this here so that I can point the guy to this post): c:\>py25 \python25\scripts\yolk.py -l argparse (0.5.0) Traceback (most recent call last): File "\python25\scripts\yolk.py", line 462, in sys.exit(main()) File "\python25\scripts\yolk.py", line 426, in main options.fields) File "\python25\scripts\yolk.py", line 49, in show_distributions print_metadata(show, metadata, active, show_metadata, fields) File "\python25\scripts\yolk.py", line 87, in print_metadata print " %s" % metadata['Summary'] KeyError: 'Summary' > c:\python25\scripts\yolk.py(87)print_metadata() -> print " %s" % metadata['Summary'] (Pdb) Thanks, Thomas From ianb at colorstudy.com Thu Feb 1 22:32:18 2007 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 01 Feb 2007 15:32:18 -0600 Subject: [Distutils] Optional C extensions in packages In-Reply-To: <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> References: <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> Message-ID: <45C25C62.10006@colorstudy.com> Phillip J. Eby wrote: > I'm thinking setuptools should allow you to specify that one or more C > extensions in a project are optional. > > At minimum, there would then be a command-line option you could give to > easy_install to say, "don't build any optional extensions", so if you don't > have a compiler or need a cross-platform egg, you can skip those extensions. > > There would also need to be a way to specify this when using the regular > build commands. > > This feature would address projects that include optional C versions of > certain code, that can fall back to pure Python implementations. I don't have anything I maintain that falls under this, but I've had a hard time supporting dependencies which do have this problem (Cheetah in particular). It would be quite handy to have this. And actually my experience with Cheetah would make me reluctant to introduce a zope.interfaces requirement for all the same reasons -- even if I'm willing to get degraded performance, that's not an option I have (which in many cases is quite acceptable, as it would be for the way I use Cheetah). -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Thu Feb 1 22:34:07 2007 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 01 Feb 2007 15:34:07 -0600 Subject: [Distutils] Building an egg from Subversion using zc.buildout In-Reply-To: References: <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <45BE6CAF.3010206@gmx.net> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <1163956881.5763.21.camel@Jane> <5.1.1.6.0.20061119134913.01f3d088@sparrow.telecommunity.com> <45BD22C2.7030403@colorstudy.com> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <5.1.1.6.0.20070128202027.04461008@sparrow.telecommunity.com> <9D0FD265-B323-42C3-80A1-85638A067C9A@zope.com> <45BE6CAF.3010206@gmx.net> <65788D22-5FA7-4AC1-A6FA-08FDA4D173FE@zope.com> <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <5.1.1.6.0.20070129180219.045646e0@sparrow.telecommunity.com> Message-ID: <45C25CCF.3080104@colorstudy.com> Martin Aspeli wrote: > Is there something else I need to do to the svn repository > (http://svn.plone.org/svn/archetypes/archetypes.kss/trunk)? Or is this a > zc.buildout limitation? I think this is an issue with indicating ways that you find a package (==dev), and the requirement you have (which might be, I dunno, >1.0). Setuptools/easy_install can find the package given ==dev, but once it's installed the thing that was just installed doesn't satisfy the requirement ==dev. The typical way to deal with this is something like archetypes.kss==dev,>1.0. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From jim at zope.com Thu Feb 1 23:32:35 2007 From: jim at zope.com (Jim Fulton) Date: Thu, 1 Feb 2007 17:32:35 -0500 Subject: [Distutils] Optional C extensions in packages In-Reply-To: <5.1.1.6.0.20070201125400.02794c90@sparrow.telecommunity.com> References: <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> <5.1.1.6.0.20070201125400.02794c90@sparrow.telecommunity.com> Message-ID: <5A6B57FB-8505-4FA2-B4C3-EEEBD88E273F@zope.com> On Feb 1, 2007, at 1:03 PM, Phillip J. Eby wrote: > At 07:11 AM 2/1/2007 -0500, Jim Fulton wrote: >> I like the goal, since most of our extensions are in this category, >> however, I have a feeling that this use case might be better served >> by packaging the extensions as separate distributions. > > True, but without automation, it's more work for the package author > to "do the right thing". Yes. It also makes it easier for the consumer to be explicit about what they want. Although maybe there are better ways. > If your extension is in a package, you have to make it a > namespace package. You have to create another setup.py and > directory structure, and manage another set of releases. This is a > lot more work than just marking the extension as "optional". Yes. See below... > > >> Once concern I have is that I think I'd want an egg with the >> extensions >> and the egg without the extensions to have different names, and >> perhaps >> different requirements. Of course, the names would be different >> because an egg with a C extension would be platform dependent and an >> egg without would be platform independent. What if both a >> platform-independent egg and a platform-dependent egg were available? >> Which would take precedence? I think they have the same precedence >> now. >> >> I think I'd be +1 if platform-dependent eggs were prefered over >> platform-independent ones. > > ...of the same version? Yes > That's easy enough to do by changing the Distribution.hashcmp > property to put the 'platform' part earlier in the comparison. If > they're different versions, however, it's a whole 'nother kettle of > fish. Yup. Of course, that points up that the ambiguity remains, on some level. So I retract my precedence idea, > The solution I was thinking of, however, has the additional benefit > of working right now. If you're generating eggs for a package with > optional extensions, you would simply *not* generate a cross- > platform egg, but instead just ship an sdist tarball, plus platform- > specific eggs for the platforms you care about. That way, anybody > on an unsupported platform will end up building a local egg from > the source, and either producing the extensions or skipping them > locally. (And if you don't have access to a compiler, it seems > unlikely that you will be then *distributing* the eggs you produced.) > > Notice that this setup is a natural side-effect of the way most > people would produce and upload packages to the Cheeseshop anyway - > you would have to make an intentional effort to suppress extension- > building and upload a cross-platform egg; otherwise you will just > end up with a source distribution and a platform-specific egg, > which then works as intended. I'm still worried about the ambiguous case when there are both platform-dependent and platform-independent eggs installed. I think you were proposing an easy_install option. This helps when someone installs a distribution directly, but doesn't help when a distribution is installed as a dependency. It also doesn't help with controlling selection of eggs after installation. And I think it doesn't make it easy to change one's mind. For example, one might install an egg with extensions and then install one without extensions to debug a problem using the Python debugger. Would the option let them do that? Is it possible to control this as part of the requirement specification? Perhaps this could be some kind of standard extra? I'd strongly prefer to be able to control this via the requirements mechanism. I'd like to be able to say that I want or don't want extensions as part of a requirement string. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From pje at telecommunity.com Fri Feb 2 00:23:20 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 01 Feb 2007 18:23:20 -0500 Subject: [Distutils] Optional C extensions in packages In-Reply-To: <5A6B57FB-8505-4FA2-B4C3-EEEBD88E273F@zope.com> References: <5.1.1.6.0.20070201125400.02794c90@sparrow.telecommunity.com> <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> <5.1.1.6.0.20070201125400.02794c90@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070201175051.0436d158@sparrow.telecommunity.com> At 05:32 PM 2/1/2007 -0500, Jim Fulton wrote: >I'm still worried about the ambiguous case when there are both >platform-dependent and platform-independent eggs installed. How would this happen? I think you're trying to solve a broader problem than the one I'm trying to solve, which is that I'd like to make it possible for people who don't have working compilers (i.e. mostly Windows, with some Mac users and some people in virtual hosting environments) to install packages that contain C extensions. In that scenario, you're going to *always* want to use this option to suppress optional extensions, because there isn't any way for you to build them. But, you would presumably still want to know about packages that *require* their extensions to be built. >I think you were proposing an easy_install option. This helps when >someone installs a distribution directly, but doesn't help when a >distribution is installed as a dependency. This would be an option to suppress compiling *all* optional C extensions, period. > It also doesn't help with >controlling selection of eggs after installation. And I think it >doesn't make it easy to change one's mind. For example, one might >install an egg with extensions and then install one without >extensions to debug a problem using the Python debugger. Would the >option let them do that? The idea was that it would be a build-time option. >Is it possible to control this as part of the requirement specification? >Perhaps this could be some kind of standard extra? > >I'd strongly prefer to be able to control this via the requirements >mechanism. I'd like to be able to say that I want or don't want >extensions as part of a requirement string. Yeah, I see the benefit of that, certainly. The problem is that we're trying to solve different problems. I just want to make it *possible* to suppress building extensions during easy_install. I'll give some more thought to what you're asking for. I have an inkling of an idea, but the problems have to do with things like having to actually check the egg's contents to see if it meets requirements, and there are problems regarding the need to clean up the build/ directory if you change what features you build something with. You see, setuptools has an undocumented 'feature' mechanism (which is still used by some PEAK projects) to control the inclusion of various packages, extensions, etc. The main reason this is undocumented is because it turns out that it's fragile to specify what features to use or not use on the command line alone, due to some distutils' commands just taking whatever's in the build/ directory as gospel. Anyway, that feature mechanism could probably be tied in to the requirements system, as long as there was a way to wipe the build/ directory whenever the features changed between runs of setup.py, and there was a way to list the features in the .egg-info, and pkg_resources was changed to query a distribution's "features" info when validating a requirement that includes "extras". I'm a little concerned that this will incur additional disk access under various circumstances, unless there is some way to statically distinguish between extras that denote "features" and ones that indicate additional requirements. Of course, matching a requirement against a distribution when the requirement doesn't list any extras, will not incur overhead. I guess we could do something like this for 0.7. One thing that concerns me, however, is that it potentially *increases* the amount of conflicts and confusion possible regarding a single egg, unless there's a way to include the features in the filename. You can't tell just by looking at it, if it meets your needs. In contrast, the benefit of my current proposal is that it's intended strictly for those circumstances where the eggs are *supposed to be* interchangeable except for platform-specificity and performance, and you should be able to at least tell from the filename which kind you have. In the case where we allow other choices of features, you would need some kind of tool to tell you what features the egg was built with. Maybe another possibility is to have *subprojects* instead, where a subproject is something built using the same setup.py, but has a distinct project name, like "PyProtocols-CExtensions" or "Twisted-Foo". By default, perhaps such a multi-project setup script would run each subproject with its own build directory, and dump multiple eggs or source distributions into the dist/ directory. This might take some munging of EasyInstall to support picking up the distributions produced when running the bdist_egg, but it might be doable. The principal downsides to this approach are the doubling up of eggs involved, and the need to keep a precise match of versions between the packages. In particular, if someone installs a new version of a package without its C extensions, and the C extensions still exist for an older version, it will end up importing the wrong extensions -- and it will be hard to tell what happened and why. The package will just seem broken. Sigh. I guess at this point I don't really see a way to do optional extensions that doesn't turn into a crazy madhouse of support later. It seems to me that at least the problems with my approach would at most boil down to, "how come this thing is so slow"? :) From constant.beta at gmail.com Fri Feb 2 02:09:57 2007 From: constant.beta at gmail.com (=?ISO-8859-2?Q?Micha=B3_Kwiatkowski?=) Date: Fri, 2 Feb 2007 02:09:57 +0100 Subject: [Distutils] parallel package Message-ID: <5e8b0f6b0702011709h116eaea5ndc53b0ce99ade95a@mail.gmail.com> Hi, I'm not able to install parallel version 0.2.1, although it is visible and accessible from PyPI. What can be the cause? Going here works: http://python.org/pypi/parallel/0.2.1 . Doing this doesn't: $ easy_install parallel==0.2.1 Searching for parallel==0.2.1 Reading http://cheeseshop.python.org/pypi/parallel/ Reading http://cheeseshop.python.org/pypi/parallel/0.2.3 No local packages or download links found for parallel==0.2.1 error: Could not find suitable distribution for Requirement.parse('parallel==0.2.1') I'm using setuptools 0.6c5 on Python 2.4 (which I've learned by browsing easy_install source code; IMHO easy_install should support --version option). Cheers, mk From pje at telecommunity.com Fri Feb 2 07:18:34 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 02 Feb 2007 01:18:34 -0500 Subject: [Distutils] parallel package In-Reply-To: <5e8b0f6b0702011709h116eaea5ndc53b0ce99ade95a@mail.gmail.co m> Message-ID: <5.1.1.6.0.20070202011717.029cd660@sparrow.telecommunity.com> At 02:09 AM 2/2/2007 +0100, =?ISO-8859-2?Q?Micha=B3_Kwiatkowski?= wrote: >Going here works: http://python.org/pypi/parallel/0.2.1 . > >Doing this doesn't: > >$ easy_install parallel==0.2.1 >Searching for parallel==0.2.1 >Reading http://cheeseshop.python.org/pypi/parallel/ >Reading http://cheeseshop.python.org/pypi/parallel/0.2.3 >No local packages or download links found for parallel==0.2.1 >error: Could not find suitable distribution for >Requirement.parse('parallel==0.2.1') easy_install -f http://python.org/pypi/parallel/0.2.1 parallel==0.2.1 From jim at zope.com Fri Feb 2 14:11:20 2007 From: jim at zope.com (Jim Fulton) Date: Fri, 02 Feb 2007 08:11:20 -0500 Subject: [Distutils] Optional C extensions in packages In-Reply-To: <5.1.1.6.0.20070201175051.0436d158@sparrow.telecommunity.com> References: <5.1.1.6.0.20070201125400.02794c90@sparrow.telecommunity.com> <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> <5.1.1.6.0.20070201125400.02794c90@sparrow.telecommunity.com> <5.1.1.6.0.20070201175051.0436d158@sparrow.telecommunity.com> Message-ID: <45C33878.2040106@zope.com> Phillip J. Eby wrote: > At 05:32 PM 2/1/2007 -0500, Jim Fulton wrote: >> I'm still worried about the ambiguous case when there are both >> platform-dependent and platform-independent eggs installed. > > How would this happen? At least in a couple of ways. 1. As I mentioned in my previous note, when a package has optional extensions, one will often want to disable the extensions for debugging purposes. It is easier debugging Python code than C code, especially in combination with other Python code. In the past, this was typically done by removing .so (or .pyd) files. This can still be done with eggs, but I thik it will be attractive to do this by selecting diffeent eggs. 2. Consider the following scenario: Someone has a mac without a development environment installed. They install some eggs and get versions without extensions. Later, they install the development tools that came on the CD with their mac. How do they reinstall the eggs with extensions? If they install in multi-version mode, won't they have a mix of eggs with and without extensions? > I think you're trying to solve a broader problem than the one I'm trying > to solve, which is that I'd like to make it possible for people who > don't have working compilers (i.e. mostly Windows, with some Mac users > and some people in virtual hosting environments) to install packages > that contain C extensions. I'm trying to avoid a problem I think you may create. As soon as there can be two eggs that satisfy the same requirements but with different semantics, I think there is a problem. I understand that in the use case you are thinking of, this would normally not happen, but it still can happen and I suggest will happen. > In that scenario, you're going to *always* want to use this option to > suppress optional extensions, because there isn't any way for you to > build them. But, you would presumably still want to know about packages > that *require* their extensions to be built. >> I think you were proposing an easy_install option. This helps when >> someone installs a distribution directly, but doesn't help when a >> distribution is installed as a dependency. > > This would be an option to suppress compiling *all* optional C > extensions, period. So it would apply to dependencies as well. Yeah, that makes sense. >> It also doesn't help with >> controlling selection of eggs after installation. And I think it >> doesn't make it easy to change one's mind. For example, one might >> install an egg with extensions and then install one without >> extensions to debug a problem using the Python debugger. Would the >> option let them do that? > > The idea was that it would be a build-time option. Will it be possible to reinstall eggs with the same versions but with different choices wrt optional extensions? I guess even if it isn't supported by easy_install, I could make it work with buildout. >> Is it possible to control this as part of the requirement specification? >> Perhaps this could be some kind of standard extra? >> >> I'd strongly prefer to be able to control this via the requirements >> mechanism. I'd like to be able to say that I want or don't want >> extensions as part of a requirement string. > > Yeah, I see the benefit of that, certainly. The problem is that we're > trying to solve different problems. I just want to make it *possible* > to suppress building extensions during easy_install. I want that too, certainly. > I'll give some more thought to what you're asking for. Super. > I have an > inkling of an idea, but the problems have to do with things like having > to actually check the egg's contents to see if it meets requirements, > and there are problems regarding the need to clean up the build/ > directory if you change what features you build something with. > > You see, setuptools has an undocumented 'feature' mechanism (which is > still used by some PEAK projects) to control the inclusion of various > packages, extensions, etc. The main reason this is undocumented is > because it turns out that it's fragile to specify what features to use > or not use on the command line alone, due to some distutils' commands > just taking whatever's in the build/ directory as gospel. > > Anyway, that feature mechanism could probably be tied in to the > requirements system, as long as there was a way to wipe the build/ > directory whenever the features changed between runs of setup.py, and > there was a way to list the features in the .egg-info, and pkg_resources > was changed to query a distribution's "features" info when validating a > requirement that includes "extras". > > I'm a little concerned that this will incur additional disk access under > various circumstances, unless there is some way to statically > distinguish between extras that denote "features" and ones that indicate > additional requirements. Of course, matching a requirement against a > distribution when the requirement doesn't list any extras, will not > incur overhead. > > I guess we could do something like this for 0.7. One thing that > concerns me, however, is that it potentially *increases* the amount of > conflicts and confusion possible regarding a single egg, unless there's > a way to include the features in the filename. You can't tell just by > looking at it, if it meets your needs. Yup. (I think this is related to the 2-byte/4-byte unicode issue.) Are there so many of these potential features that we couldn't reflect them in the file name? In the specific case of the presence or absence of extensions, that is already part of the file name. Eggs with extensions will have the platform reflected in the file name. Eggs without won't, so it should be easy to tell them apart. > In contrast, the benefit of my current proposal is that it's intended > strictly for those circumstances where the eggs are *supposed to be* > interchangeable except for platform-specificity and performance, and you > should be able to at least tell from the filename which kind you have. As I mention above, you'll be able to easily distinguish platform-specific and platform-independent eggs apart based on their file names. > In the case where we allow other choices of features, you would need > some kind of tool to tell you what features the egg was built with. In the general case, yes, unless you reflected the features in the file name. In the specific case of extensions, I don't think this is a problem. > Maybe another possibility is to have *subprojects* instead, where a > subproject is something built using the same setup.py, but has a > distinct project name, like "PyProtocols-CExtensions" or "Twisted-Foo". > By default, perhaps such a multi-project setup script would run each > subproject with its own build directory, and dump multiple eggs or > source distributions into the dist/ directory. Yup. And this could be automated triggered easily using minimal meta-data in the setup file. > This might take some > munging of EasyInstall to support picking up the distributions produced > when running the bdist_egg, but it might be doable. > > The principal downsides to this approach are the doubling up of eggs > involved, and the need to keep a precise match of versions between the > packages. In particular, if someone installs a new version of a package > without its C extensions, and the C extensions still exist for an older > version, it will end up importing the wrong extensions -- and it will be > hard to tell what happened and why. The package will just seem broken. I see your point. This arises from the way that easy_install incrementally installs distributions. This potentially wouldn't be a problem for buildout, but I wouldn't want to break easy_install (or workingenv). > Sigh. I guess at this point I don't really see a way to do optional > extensions that doesn't turn into a crazy madhouse of support later. It > seems to me that at least the problems with my approach would at most > boil down to, "how come this thing is so slow"? :) OK, so based on this discussion, I'm in favor of your original proposal as a start. I think there should be a way to cause building/installation of a platform-dependent egg even if there is a platform-independent egg with the same installed already, and the other way around, to deal with the use cases I described earlier. Even in multiple-version mode, this is not a problem, because the eggs will have different file names. I'd really *like* to be able to reflect the selection of these somehow in requirement specifications, but, if need be, this can be dealt with at the tool (e.g. buildout) level. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From jim at zope.com Fri Feb 2 14:18:39 2007 From: jim at zope.com (Jim Fulton) Date: Fri, 02 Feb 2007 08:18:39 -0500 Subject: [Distutils] parallel package In-Reply-To: <5.1.1.6.0.20070202011717.029cd660@sparrow.telecommunity.com> References: <5.1.1.6.0.20070202011717.029cd660@sparrow.telecommunity.com> Message-ID: <45C33A2F.4020501@zope.com> Phillip J. Eby wrote: > At 02:09 AM 2/2/2007 +0100, =?ISO-8859-2?Q?Micha=B3_Kwiatkowski?= wrote: > >> Going here works: http://python.org/pypi/parallel/0.2.1 . >> >> Doing this doesn't: >> >> $ easy_install parallel==0.2.1 >> Searching for parallel==0.2.1 >> Reading http://cheeseshop.python.org/pypi/parallel/ >> Reading http://cheeseshop.python.org/pypi/parallel/0.2.3 >> No local packages or download links found for parallel==0.2.1 >> error: Could not find suitable distribution for >> Requirement.parse('parallel==0.2.1') > > easy_install -f http://python.org/pypi/parallel/0.2.1 parallel==0.2.1 Can this be automated? It is annoying that PyPI hides old versions from setuptools. Perhaps the index API should be expanded reflect that older versions can often be found this way? That is, perhaps setuptools should look at INDEX_URL/PROJECT/VERSION when a version can't be found another way. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From p.f.moore at gmail.com Fri Feb 2 14:49:27 2007 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 2 Feb 2007 13:49:27 +0000 Subject: [Distutils] Compiler support for VC++ 2005 In-Reply-To: <45B0AADD.13806.25AF95@howard.eegsoftware.com> References: <45B0AADD.13806.25AF95@howard.eegsoftware.com> Message-ID: <79990c6b0702020549q7fa212b2s6fc687b4cc3fc9e1@mail.gmail.com> On 19/01/07, Howard Lightstone wrote: > I am unable to rebuild things using disutils using Visual Studio > 2005. I get reports of "No compiler found". > > Since VC8 is the only thing available in the future (from > Micro$soft), is there any progress toward updating distuils to > allow this compiler (and its own locations....) to be supported? If you want to build extensions with VS 2005, you need to build Python with that compiler. At present, the Windows binaries of Python are built with VC 7.1 (VS 2003). Sadly, MS no longer make the free VC 7.1 compiler ("Toolkit compiler") available, otherwise you could use that (you may still be able to find it somewhere, or persuade someone to send you a copy). So, unless you're building your own Python executable, you can't use VS 2005 regardless of distutils support. On the other hand, mingw gcc is fully supported, if that's an option for you. Paul. From p.f.moore at gmail.com Fri Feb 2 15:23:14 2007 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 2 Feb 2007 14:23:14 +0000 Subject: [Distutils] Compiler support for VC++ 2005 In-Reply-To: <45C3437B.1020905@voidspace.org.uk> References: <45B0AADD.13806.25AF95@howard.eegsoftware.com> <79990c6b0702020549q7fa212b2s6fc687b4cc3fc9e1@mail.gmail.com> <45C3437B.1020905@voidspace.org.uk> Message-ID: <79990c6b0702020623tb184f4dya5484b40e7285d4e@mail.gmail.com> On 02/02/07, Michael Foord wrote: > Paul Moore wrote: > But, IIUC there can still be issues with using gcc for extensions - > because you are still using a different CRT layer and this can cause > problems. Not true. When using mingw/gcc, distutils ensures that msvcr71.dll is used - the same CRT as Python. (I know, I was involved in implementing the change). > *If* distutils supported VC8 it would be as effective as using gcc. (?) My understanding is that VC8 only supports using msvcr80.dll, which is incompatible. Paul. From fuzzyman at voidspace.org.uk Fri Feb 2 15:22:33 2007 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 02 Feb 2007 14:22:33 +0000 Subject: [Distutils] Compiler support for VC++ 2005 In-Reply-To: <79990c6b0702020623tb184f4dya5484b40e7285d4e@mail.gmail.com> References: <45B0AADD.13806.25AF95@howard.eegsoftware.com> <79990c6b0702020549q7fa212b2s6fc687b4cc3fc9e1@mail.gmail.com> <45C3437B.1020905@voidspace.org.uk> <79990c6b0702020623tb184f4dya5484b40e7285d4e@mail.gmail.com> Message-ID: <45C34929.4010701@voidspace.org.uk> Paul Moore wrote: > On 02/02/07, Michael Foord wrote: >> Paul Moore wrote: >> But, IIUC there can still be issues with using gcc for extensions - >> because you are still using a different CRT layer and this can cause >> problems. > > Not true. When using mingw/gcc, distutils ensures that msvcr71.dll is > used - the same CRT as Python. (I know, I was involved in implementing > the change). > >> *If* distutils supported VC8 it would be as effective as using gcc. (?) > > My understanding is that VC8 only supports using msvcr80.dll, which is > incompatible. Oh right - so gcc supports you building against msvcr71.dll which VC8 specifically prevents. Fair enough, I wasn't aware of this. I *have* heard reports of problems for certain projects caused by extensions built with gcc on windows, but they seemed like corner cases. (One was a database project which passed file handles to the CRT - or something like that - IIRC.) Fuzzyman > > > Paul. > From p.f.moore at gmail.com Fri Feb 2 15:39:31 2007 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 2 Feb 2007 14:39:31 +0000 Subject: [Distutils] Compiler support for VC++ 2005 In-Reply-To: <45C34929.4010701@voidspace.org.uk> References: <45B0AADD.13806.25AF95@howard.eegsoftware.com> <79990c6b0702020549q7fa212b2s6fc687b4cc3fc9e1@mail.gmail.com> <45C3437B.1020905@voidspace.org.uk> <79990c6b0702020623tb184f4dya5484b40e7285d4e@mail.gmail.com> <45C34929.4010701@voidspace.org.uk> Message-ID: <79990c6b0702020639s526cf7efw2e065706318eea95@mail.gmail.com> On 02/02/07, Michael Foord wrote: > Fair enough, I wasn't aware of this. I *have* heard reports of problems > for certain projects caused by extensions built with gcc on windows, but > they seemed like corner cases. (One was a database project which passed > file handles to the CRT - or something like that - IIRC.) You could get issues with an extension which uses an external library. If that external library's API returns things like file handles or pointers to malloc'ed memory, *and* the external library is built to use a CRT other than msvcr71, then you could have problems (which you could fix by rebuilding the external library to use msvcr71). IIRC, the gd library used to have issues like this. But it's very rare, in my experience, and points to an issue with the external library, rather than with the extension or distutils. A much more common problem is extension code which relies on MSVC features or libraries (for example pywin32) or which relies on external libraries which only come in a compiled form compatible with MSVC and which are too much of a pain to build yourself (wxPython comes to mind here). These days, I generally go with prebuilt binaries for anything 3rd party, but I'd build my own extensions using mingw. Paul. From fuzzyman at voidspace.org.uk Fri Feb 2 14:58:19 2007 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 02 Feb 2007 13:58:19 +0000 Subject: [Distutils] Compiler support for VC++ 2005 In-Reply-To: <79990c6b0702020549q7fa212b2s6fc687b4cc3fc9e1@mail.gmail.com> References: <45B0AADD.13806.25AF95@howard.eegsoftware.com> <79990c6b0702020549q7fa212b2s6fc687b4cc3fc9e1@mail.gmail.com> Message-ID: <45C3437B.1020905@voidspace.org.uk> Paul Moore wrote: > On 19/01/07, Howard Lightstone wrote: > >> I am unable to rebuild things using disutils using Visual Studio >> 2005. I get reports of "No compiler found". >> >> Since VC8 is the only thing available in the future (from >> Micro$soft), is there any progress toward updating distuils to >> allow this compiler (and its own locations....) to be supported? >> > > If you want to build extensions with VS 2005, you need to build Python > with that compiler. At present, the Windows binaries of Python are > built with VC 7.1 (VS 2003). Sadly, MS no longer make the free VC 7.1 > compiler ("Toolkit compiler") available, otherwise you could use that > (you may still be able to find it somewhere, or persuade someone to > send you a copy). > > So, unless you're building your own Python executable, you can't use > VS 2005 regardless of distutils support. > > On the other hand, mingw gcc is fully supported, if that's an option for you. > But, IIUC there can still be issues with using gcc for extensions - because you are still using a different CRT layer and this can cause problems. *If* distutils supported VC8 it would be as effective as using gcc. (?) All the best, Fuzzyman http://www.voidspace.org.uk/python/articles.shtml > Paul. > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > > From pje at telecommunity.com Fri Feb 2 18:49:29 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 02 Feb 2007 12:49:29 -0500 Subject: [Distutils] package_data not used by sdist command In-Reply-To: <45B4CB27.3000005@zope.com> Message-ID: <5.1.1.6.0.20070202124556.02992cb8@sparrow.telecommunity.com> At 09:33 AM 1/22/2007 -0500, Jim Fulton wrote: >It appears that the sdist command doesn't consult the data passed >in the package_data option to setup. I want to name data >that isn't under version control for inclusion. Because >sdist doesn't pay attention to package_data, the data I'm >naming isn't included in the source distribution. It is included >if I build an egg directly. This appears to me to be a bug. Yeah, there's a long and uninteresting history there. :) When I implemented package_data, I was already using a MANIFEST.in to include data in the sdist. Then, Fred ported package_data back into the distutils from setuptools, and then *after* that I added sdist-thru-revision-control support, so the distutils never got it, and setuptools never appeared to need it (at least to me). Anyway, I don't know if a fix for this can go into 0.6 or not, depending on how extensive a change is required. A patch would go a long way towards getting it into 0.6c6. There are way too many 'c' releases and I'd really like 0.6 final out before *this* year's Pycon. :) From pje at telecommunity.com Fri Feb 2 18:58:01 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 02 Feb 2007 12:58:01 -0500 Subject: [Distutils] parallel package In-Reply-To: <45C33A2F.4020501@zope.com> References: <5.1.1.6.0.20070202011717.029cd660@sparrow.telecommunity.com> <5.1.1.6.0.20070202011717.029cd660@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070202125226.04665cb0@sparrow.telecommunity.com> At 08:18 AM 2/2/2007 -0500, Jim Fulton wrote: >Phillip J. Eby wrote: >>At 02:09 AM 2/2/2007 +0100, =?ISO-8859-2?Q?Micha=B3_Kwiatkowski?= wrote: >> >>>Going here works: http://python.org/pypi/parallel/0.2.1 . >>> >>>Doing this doesn't: >>> >>>$ easy_install parallel==0.2.1 >>>Searching for parallel==0.2.1 >>>Reading http://cheeseshop.python.org/pypi/parallel/ >>>Reading http://cheeseshop.python.org/pypi/parallel/0.2.3 >>>No local packages or download links found for parallel==0.2.1 >>>error: Could not find suitable distribution for >>>Requirement.parse('parallel==0.2.1') >>easy_install -f http://python.org/pypi/parallel/0.2.1 parallel==0.2.1 > >Can this be automated? It is annoying that PyPI hides >old versions from setuptools. Perhaps the index API should >be expanded reflect that older versions can often be found >this way? That is, perhaps setuptools should look at >INDEX_URL/PROJECT/VERSION when a version can't be >found another way. Yeah, that's a possibility. It'd have to look for any <=, >=, or == specifiers in the requirement, and then scan those URLs. I'll add it to my list. Of course, if you don't know a specific version, you won't be able to get it this way, but presumably the use case here is to get a known-to-work version, especially one specified as a dependency of another project. From pje at telecommunity.com Fri Feb 2 19:13:35 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 02 Feb 2007 13:13:35 -0500 Subject: [Distutils] Optional C extensions in packages In-Reply-To: <45C33878.2040106@zope.com> References: <5.1.1.6.0.20070201175051.0436d158@sparrow.telecommunity.com> <5.1.1.6.0.20070201125400.02794c90@sparrow.telecommunity.com> <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> <5.1.1.6.0.20070201125400.02794c90@sparrow.telecommunity.com> <5.1.1.6.0.20070201175051.0436d158@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070202125932.02781ad8@sparrow.telecommunity.com> At 08:11 AM 2/2/2007 -0500, Jim Fulton wrote: >Phillip J. Eby wrote: >>At 05:32 PM 2/1/2007 -0500, Jim Fulton wrote: >>>I'm still worried about the ambiguous case when there are both >>>platform-dependent and platform-independent eggs installed. >>How would this happen? > >At least in a couple of ways. > >1. As I mentioned in my previous note, when a package has optional > extensions, one will often want to disable the extensions for > debugging purposes. It is easier debugging Python code than > C code, especially in combination with other Python code. In the > past, this was typically done by removing .so (or .pyd) files. > This can still be done with eggs, but I thik it will be attractive > to do this by selecting diffeent eggs. > >2. Consider the following scenario: Someone has a mac without a development > environment installed. They install some eggs and get versions without > extensions. Later, they install the development tools that came on the CD > with their mac. How do they reinstall the eggs with extensions? If they > install in multi-version mode, won't they have a mix of eggs with and > without extensions? Well, they can always "rm -rf *.egg" and reinstall. :) Otherwise, they'll get them by attrition when packages are upgraded to newer versions. In fact, using -U might be sufficient, although I think EasyInstall actually has some quirks with respect to determining whether -U will end up in a reinstall or not. >In the specific case of the presence or absence of extensions, that >is already part of the file name. Eggs with extensions will >have the platform reflected in the file name. Eggs without won't, >so it should be easy to tell them apart. Yep. >I see your point. This arises from the way that easy_install incrementally >installs distributions. This potentially wouldn't be a problem for buildout, >but I wouldn't want to break easy_install (or workingenv). Yes, so these features would have to wait until 0.7, and a possible redesign of EasyInstall to be based on buildout (or something like it, anyway), instead of the other way around. ;) >>Sigh. I guess at this point I don't really see a way to do optional >>extensions that doesn't turn into a crazy madhouse of support later. It >>seems to me that at least the problems with my approach would at most >>boil down to, "how come this thing is so slow"? :) > >OK, so based on this discussion, I'm in favor of your original proposal >as a start. I think there should be a way to cause building/installation >of a platform-dependent egg even if there is a platform-independent egg >with the same installed already, and the other way around, to deal with >the use cases I described earlier. Even in multiple-version mode, >this is not a problem, because the eggs will have different file names. >I'd really *like* to be able to reflect the selection of these somehow >in requirement specifications, but, if need be, this can be dealt with >at the tool (e.g. buildout) level. EasyInstall probably just needs to grow an option to force reinstallation of a package, as that's a generally useful feature. I.e., sort of a "don't allow the requirement to be satisfied with an egg that's already on sys.path" option. From jim at zope.com Fri Feb 2 19:26:12 2007 From: jim at zope.com (Jim Fulton) Date: Fri, 02 Feb 2007 13:26:12 -0500 Subject: [Distutils] Optional C extensions in packages In-Reply-To: <5.1.1.6.0.20070202125932.02781ad8@sparrow.telecommunity.com> References: <5.1.1.6.0.20070201175051.0436d158@sparrow.telecommunity.com> <5.1.1.6.0.20070201125400.02794c90@sparrow.telecommunity.com> <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> <5.1.1.6.0.20070201125400.02794c90@sparrow.telecommunity.com> <5.1.1.6.0.20070201175051.0436d158@sparrow.telecommunity.com> <5.1.1.6.0.20070202125932.02781ad8@sparrow.telecommunity.com> Message-ID: <45C38244.2020001@zope.com> Phillip J. Eby wrote: > At 08:11 AM 2/2/2007 -0500, Jim Fulton wrote: >> Phillip J. Eby wrote: >>> At 05:32 PM 2/1/2007 -0500, Jim Fulton wrote: >>>> I'm still worried about the ambiguous case when there are both >>>> platform-dependent and platform-independent eggs installed. >>> How would this happen? >> >> At least in a couple of ways. >> >> 1. As I mentioned in my previous note, when a package has optional >> extensions, one will often want to disable the extensions for >> debugging purposes. It is easier debugging Python code than >> C code, especially in combination with other Python code. In the >> past, this was typically done by removing .so (or .pyd) files. >> This can still be done with eggs, but I thik it will be attractive >> to do this by selecting diffeent eggs. >> >> 2. Consider the following scenario: Someone has a mac without a >> development >> environment installed. They install some eggs and get versions >> without >> extensions. Later, they install the development tools that came on >> the CD >> with their mac. How do they reinstall the eggs with extensions? >> If they >> install in multi-version mode, won't they have a mix of eggs with and >> without extensions? > > Well, they can always "rm -rf *.egg" and reinstall. :) Otherwise, > they'll get them by attrition when packages are upgraded to newer > versions. In fact, using -U might be sufficient, although I think > EasyInstall actually has some quirks with respect to determining whether > -U will end up in a reinstall or not. I'd be surprised if upgrade would reinstall if there weren't a later version available. ... >> I see your point. This arises from the way that easy_install >> incrementally >> installs distributions. This potentially wouldn't be a problem for >> buildout, >> but I wouldn't want to break easy_install (or workingenv). > > Yes, so these features would have to wait until 0.7, and a possible > redesign of EasyInstall to be based on buildout (or something like it, > anyway), instead of the other way around. ;) I didn't mean to imply that buildout was better than easy_install, merely noting that they were different. >>> Sigh. I guess at this point I don't really see a way to do optional >>> extensions that doesn't turn into a crazy madhouse of support later. >>> It seems to me that at least the problems with my approach would at >>> most boil down to, "how come this thing is so slow"? :) >> >> OK, so based on this discussion, I'm in favor of your original proposal >> as a start. I think there should be a way to cause building/installation >> of a platform-dependent egg even if there is a platform-independent egg >> with the same installed already, and the other way around, to deal with >> the use cases I described earlier. Even in multiple-version mode, >> this is not a problem, because the eggs will have different file names. >> I'd really *like* to be able to reflect the selection of these somehow >> in requirement specifications, but, if need be, this can be dealt with >> at the tool (e.g. buildout) level. > > EasyInstall probably just needs to grow an option to force > reinstallation of a package, as that's a generally useful feature. > I.e., sort of a "don't allow the requirement to be satisfied with an egg > that's already on sys.path" option. That seems like a rather big stick and a round-about way to do it. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From pje at telecommunity.com Fri Feb 2 19:40:40 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 02 Feb 2007 13:40:40 -0500 Subject: [Distutils] Optional C extensions in packages In-Reply-To: <45C38244.2020001@zope.com> References: <5.1.1.6.0.20070202125932.02781ad8@sparrow.telecommunity.com> <5.1.1.6.0.20070201175051.0436d158@sparrow.telecommunity.com> <5.1.1.6.0.20070201125400.02794c90@sparrow.telecommunity.com> <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> <5.1.1.6.0.20070131184008.043ccf38@sparrow.telecommunity.com> <5.1.1.6.0.20070201125400.02794c90@sparrow.telecommunity.com> <5.1.1.6.0.20070201175051.0436d158@sparrow.telecommunity.com> <5.1.1.6.0.20070202125932.02781ad8@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070202133545.0515b900@sparrow.telecommunity.com> At 01:26 PM 2/2/2007 -0500, Jim Fulton wrote: >Phillip J. Eby wrote: >>Yes, so these features would have to wait until 0.7, and a possible >>redesign of EasyInstall to be based on buildout (or something like it, >>anyway), instead of the other way around. ;) > >I didn't mean to imply that buildout was better than easy_install, >merely noting that they were different. Well, I did mean to imply it's better, because it is. Take the compliment and go quietly. ;) Seriously, I do intend for the "nest" tool to be "more like buildout", in the sense that it will target the management of individual nests (analagous to individual buildouts), that it will likely be more transactional, and better able to support plugins (analagous to recipes). So, I think that the basic ideas of buildout are good and should be emulated in "nest"; whether any actual code sharing or other similarity-in-detail will exist, I don't yet know. >>EasyInstall probably just needs to grow an option to force reinstallation >>of a package, as that's a generally useful feature. >>I.e., sort of a "don't allow the requirement to be satisfied with an egg >>that's already on sys.path" option. > >That seems like a rather big stick and a round-about way to do it. True, but that statement applies to EasyInstall as a whole already, doesn't it? :) More seriously, it's the only thing I can reasonably see doing in the 0.6 timeframe unless somebody else can contribute good patches. I really want to put 0.6 to bed so that serious work on 0.7 can start -- something that's now almost a year overdue, compared to my druthers. From jorge.vargas at gmail.com Sat Feb 3 04:43:31 2007 From: jorge.vargas at gmail.com (Jorge Vargas) Date: Fri, 2 Feb 2007 23:43:31 -0400 Subject: [Distutils] Compiler support for VC++ 2005 In-Reply-To: <45C34929.4010701@voidspace.org.uk> References: <45B0AADD.13806.25AF95@howard.eegsoftware.com> <79990c6b0702020549q7fa212b2s6fc687b4cc3fc9e1@mail.gmail.com> <45C3437B.1020905@voidspace.org.uk> <79990c6b0702020623tb184f4dya5484b40e7285d4e@mail.gmail.com> <45C34929.4010701@voidspace.org.uk> Message-ID: <32822fe60702021943y4233aal24cca20bc7b5537b@mail.gmail.com> On 2/2/07, Michael Foord wrote: > > Paul Moore wrote: > > On 02/02/07, Michael Foord wrote: > >> Paul Moore wrote: > >> But, IIUC there can still be issues with using gcc for extensions - > >> because you are still using a different CRT layer and this can cause > >> problems. > > > > Not true. When using mingw/gcc, distutils ensures that msvcr71.dll is > > used - the same CRT as Python. (I know, I was involved in implementing > > the change). > > > >> *If* distutils supported VC8 it would be as effective as using gcc. (?) > > > > My understanding is that VC8 only supports using msvcr80.dll, which is > > incompatible. > Oh right - so gcc supports you building against msvcr71.dll which VC8 > specifically prevents. > > Fair enough, I wasn't aware of this. I *have* heard reports of problems > for certain projects caused by extensions built with gcc on windows, but > they seemed like corner cases. (One was a database project which passed > file handles to the CRT - or something like that - IIRC.) yes indeed I had a huge fight with mysqldb driver because of this. Since the windows version of mysql is compile with VS2003 and python too getting those drivers to build with gcc will require recompiling everything and making it incompatible with the stock mysql. This is something that shock me a lot, why python (the official) on windows is build with a tool that is not accessible to all developers. Building extensions on windows is just a pain... > Fuzzyman > > > > > > > Paul. > > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070202/3327ec6a/attachment.html From arve.knudsen at gmail.com Sat Feb 3 12:48:02 2007 From: arve.knudsen at gmail.com (Arve Knudsen) Date: Sat, 3 Feb 2007 12:48:02 +0100 Subject: [Distutils] package_data not used by sdist command In-Reply-To: <5.1.1.6.0.20070202124556.02992cb8@sparrow.telecommunity.com> References: <45B4CB27.3000005@zope.com> <5.1.1.6.0.20070202124556.02992cb8@sparrow.telecommunity.com> Message-ID: Funny, I seem to recall having raised the same issue, but not receiving much in way of an answer. Glad to see it's being taken care of at least. Arve On 2/2/07, Phillip J. Eby wrote: > > At 09:33 AM 1/22/2007 -0500, Jim Fulton wrote: > >It appears that the sdist command doesn't consult the data passed > >in the package_data option to setup. I want to name data > >that isn't under version control for inclusion. Because > >sdist doesn't pay attention to package_data, the data I'm > >naming isn't included in the source distribution. It is included > >if I build an egg directly. This appears to me to be a bug. > > Yeah, there's a long and uninteresting history there. :) When I > implemented package_data, I was already using a MANIFEST.in to include > data > in the sdist. Then, Fred ported package_data back into the distutils from > setuptools, and then *after* that I added sdist-thru-revision-control > support, so the distutils never got it, and setuptools never appeared to > need it (at least to me). > > Anyway, I don't know if a fix for this can go into 0.6 or not, depending > on > how extensive a change is required. A patch would go a long way towards > getting it into 0.6c6. There are way too many 'c' releases and I'd really > like 0.6 final out before *this* year's Pycon. :) > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070203/6f22236f/attachment.htm From pje at telecommunity.com Sat Feb 3 17:03:27 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 03 Feb 2007 11:03:27 -0500 Subject: [Distutils] package_data not used by sdist command In-Reply-To: References: <5.1.1.6.0.20070202124556.02992cb8@sparrow.telecommunity.com> <45B4CB27.3000005@zope.com> <5.1.1.6.0.20070202124556.02992cb8@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070203110203.028f7ab0@sparrow.telecommunity.com> At 12:48 PM 2/3/2007 +0100, Arve Knudsen wrote: >Funny, I seem to recall having raised the same issue, but not receiving >much in way of an answer. Glad to see it's being taken care of at least. Um, it's not being "taken care of". As I said, "a patch would go a long way towards getting it in." That means it's probably *not* being taken care of any time soon, unless somebody submits one. From aknuds-1 at broadpark.no Sat Feb 3 22:28:33 2007 From: aknuds-1 at broadpark.no (Arve Knudsen) Date: Sat, 03 Feb 2007 22:28:33 +0100 Subject: [Distutils] package_data not used by sdist command In-Reply-To: <5.1.1.6.0.20070203110203.028f7ab0@sparrow.telecommunity.com> References: <5.1.1.6.0.20070202124556.02992cb8@sparrow.telecommunity.com> <45B4CB27.3000005@zope.com> <5.1.1.6.0.20070203110203.028f7ab0@sparrow.telecommunity.com> Message-ID: On Sat, 03 Feb 2007 17:03:27 +0100, Phillip J. Eby wrote: > At 12:48 PM 2/3/2007 +0100, Arve Knudsen wrote: >> Funny, I seem to recall having raised the same issue, but not receiving >> much in way of an answer. Glad to see it's being taken care of at least. > > Um, it's not being "taken care of". As I said, "a patch would go a long > way towards getting it in." That means it's probably *not* being taken > care of any time soon, unless somebody submits one. I interpreted you as having recognized the problem, but that it would not make it into 0.6. As long as it can be scheduled for a future version I can try my hand at contributing a patch (unless someone else wants to). Arve From jorge.vargas at gmail.com Sat Feb 3 22:09:01 2007 From: jorge.vargas at gmail.com (Jorge Vargas) Date: Sat, 3 Feb 2007 17:09:01 -0400 Subject: [Distutils] setuptools-0.6c5-py2.5.egg in cheeseshop seems to be broken Message-ID: <32822fe60702031309y5ec522fejc5a7683dd8ced224@mail.gmail.com> elpargo at notebook:~$ sh setuptools-0.6c5-py2.5.egg setuptools-0.6c5-py2.5.egg: 8: [[: not found setuptools-0.6c5-py2.5.egg is not the correct name for this egg file. Please rename it back to setuptools-0.6c5-py2.5.egg and try again. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070203/777a3a6d/attachment.html From matt at matt-good.net Sat Feb 3 23:18:25 2007 From: matt at matt-good.net (Matt Good) Date: Sat, 03 Feb 2007 17:18:25 -0500 Subject: [Distutils] setuptools-0.6c5-py2.5.egg in cheeseshop seems to be broken In-Reply-To: <32822fe60702031309y5ec522fejc5a7683dd8ced224@mail.gmail.com> References: <32822fe60702031309y5ec522fejc5a7683dd8ced224@mail.gmail.com> Message-ID: <1170541106.7671.18.camel@nny> On Sat, 2007-02-03 at 17:09 -0400, Jorge Vargas wrote: > elpargo at notebook:~$ sh setuptools-0.6c5-py2.5.egg > setuptools-0.6c5-py2.5.egg: 8: [[: not found > setuptools-0.6c5-py2.5.egg is not the correct name for this egg file. > Please rename it back to setuptools-0.6c5-py2.5.egg and try again. There was a previous mention of this on the mailing list. The Egg uses a bash-ism there, so "sh" doesn't quite work. I don't remember if PJE was going to change that in SVN, but I believe you should be able to run the Egg with bash instead. -- Matt Good From pje at telecommunity.com Sat Feb 3 23:52:02 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 03 Feb 2007 17:52:02 -0500 Subject: [Distutils] setuptools-0.6c5-py2.5.egg in cheeseshop seems to be broken In-Reply-To: <1170541106.7671.18.camel@nny> References: <32822fe60702031309y5ec522fejc5a7683dd8ced224@mail.gmail.com> <32822fe60702031309y5ec522fejc5a7683dd8ced224@mail.gmail.com> Message-ID: <5.1.1.6.0.20070203175010.02d9df78@sparrow.telecommunity.com> At 05:18 PM 2/3/2007 -0500, Matt Good wrote: >On Sat, 2007-02-03 at 17:09 -0400, Jorge Vargas wrote: > > elpargo at notebook:~$ sh setuptools-0.6c5-py2.5.egg > > setuptools-0.6c5-py2.5.egg: 8: [[: not found > > setuptools-0.6c5-py2.5.egg is not the correct name for this egg file. > > Please rename it back to setuptools-0.6c5-py2.5.egg and try again. > >There was a previous mention of this on the mailing list. The Egg uses >a bash-ism there, so "sh" doesn't quite work. I don't remember if PJE >was going to change that in SVN, but I believe you should be able to run >the Egg with bash instead. That's correct; 0.6c5 actually requires bash, but 0.6c6 uses '[' instead of '[[' and so in theory should work with a non-bash "sh". If you're good with a binary editor, you could probably hack the header of the current .egg to work. :) Or at least, you could manually run the shell commands that are in it. From p.f.moore at gmail.com Mon Feb 5 12:33:49 2007 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 5 Feb 2007 11:33:49 +0000 Subject: [Distutils] Compiler support for VC++ 2005 In-Reply-To: <32822fe60702021943y4233aal24cca20bc7b5537b@mail.gmail.com> References: <45B0AADD.13806.25AF95@howard.eegsoftware.com> <79990c6b0702020549q7fa212b2s6fc687b4cc3fc9e1@mail.gmail.com> <45C3437B.1020905@voidspace.org.uk> <79990c6b0702020623tb184f4dya5484b40e7285d4e@mail.gmail.com> <45C34929.4010701@voidspace.org.uk> <32822fe60702021943y4233aal24cca20bc7b5537b@mail.gmail.com> Message-ID: <79990c6b0702050333p7c12a227m73e6c314ced74b5f@mail.gmail.com> On 03/02/07, Jorge Vargas wrote: > yes indeed I had a huge fight with mysqldb driver because of this. Since the > windows version of mysql is compile with VS2003 and python too getting those > drivers to build with gcc will require recompiling everything and making it > incompatible with the stock mysql. You don't need to recompile Python with gcc. Whether you need to recompile MySQL, I can't say. > This is something that shock me a lot, why python (the official) on windows > is build with a tool that is not accessible to all developers. Building > extensions on windows is just a pain... I agree that building extensions on Windows can be a real pain. But it's generally the fault of the 3rd party code, or the extension code, not of Python itself. As an example, mod_python used to be a real pain to build, because Apache (the standard binary) was built with MSVC 6, and Python (the standard build) with MSVC 7.1. But I believe the mod_python developers have made some changes, and as a result it *is* now possible to build mod_python with gcc. At the end of the day, it can be a real pain to build your own extensions on Windows. But in my view, the Python developers have made a good job of easing that pain as much as possible. Whether choosing to use MSVC 7.1 was a good *political* move, rather than a good *technical* one, is a different question, and one I don't intend to get into :-) Paul. From lxander.m at gmail.com Tue Feb 6 21:58:34 2007 From: lxander.m at gmail.com (Alexander Michael) Date: Tue, 6 Feb 2007 15:58:34 -0500 Subject: [Distutils] SetupTools: Forcefully Ignoring Dependencies Message-ID: <525f23e80702061258x75adfew24cd11bc60bc5ea4@mail.gmail.com> How can I force easy_install to not worry about (i.e. ignore) dependencies? I tried --no-deps, but easy_install (from setuptools 0.6c5) still attempts to find dependencies and then errors out. Thanks, Alex From jorge.vargas at gmail.com Wed Feb 7 20:20:29 2007 From: jorge.vargas at gmail.com (Jorge Vargas) Date: Wed, 7 Feb 2007 15:20:29 -0400 Subject: [Distutils] Compiler support for VC++ 2005 In-Reply-To: <79990c6b0702050333p7c12a227m73e6c314ced74b5f@mail.gmail.com> References: <45B0AADD.13806.25AF95@howard.eegsoftware.com> <79990c6b0702020549q7fa212b2s6fc687b4cc3fc9e1@mail.gmail.com> <45C3437B.1020905@voidspace.org.uk> <79990c6b0702020623tb184f4dya5484b40e7285d4e@mail.gmail.com> <45C34929.4010701@voidspace.org.uk> <32822fe60702021943y4233aal24cca20bc7b5537b@mail.gmail.com> <79990c6b0702050333p7c12a227m73e6c314ced74b5f@mail.gmail.com> Message-ID: <32822fe60702071120s4d5ccb75ha726b13aae84a1c6@mail.gmail.com> On 2/5/07, Paul Moore wrote: > On 03/02/07, Jorge Vargas wrote: > > yes indeed I had a huge fight with mysqldb driver because of this. Since the > > windows version of mysql is compile with VS2003 and python too getting those > > drivers to build with gcc will require recompiling everything and making it > > incompatible with the stock mysql. > > You don't need to recompile Python with gcc. Whether you need to > recompile MySQL, I can't say. > no but you need to patch distutils to force it to use the windows "free compiler" it can't be doen with gcc because of mysql. > > This is something that shock me a lot, why python (the official) on windows > > is build with a tool that is not accessible to all developers. Building > > extensions on windows is just a pain... > > I agree that building extensions on Windows can be a real pain. But > it's generally the fault of the 3rd party code, or the extension code, > not of Python itself. > yes that's true the problem comes when you need windows.h or other file that the mingw people can't distribute. > As an example, mod_python used to be a real pain to build, because > Apache (the standard binary) was built with MSVC 6, and Python (the > standard build) with MSVC 7.1. But I believe the mod_python developers > have made some changes, and as a result it *is* now possible to build > mod_python with gcc. > > At the end of the day, it can be a real pain to build your own > extensions on Windows. But in my view, the Python developers have made > a good job of easing that pain as much as possible. Whether choosing > to use MSVC 7.1 was a good *political* move, rather than a good > *technical* one, is a different question, and one I don't intend to > get into :-) > ok lets not :) > Paul. > From jorge.vargas at gmail.com Wed Feb 7 20:24:04 2007 From: jorge.vargas at gmail.com (Jorge Vargas) Date: Wed, 7 Feb 2007 15:24:04 -0400 Subject: [Distutils] setuptools-0.6c5-py2.5.egg in cheeseshop seems to be broken In-Reply-To: <5.1.1.6.0.20070203175010.02d9df78@sparrow.telecommunity.com> References: <32822fe60702031309y5ec522fejc5a7683dd8ced224@mail.gmail.com> <1170541106.7671.18.camel@nny> <5.1.1.6.0.20070203175010.02d9df78@sparrow.telecommunity.com> Message-ID: <32822fe60702071124s6235bfd6g5855e0b976fead85@mail.gmail.com> On 2/3/07, Phillip J. Eby wrote: > At 05:18 PM 2/3/2007 -0500, Matt Good wrote: > >On Sat, 2007-02-03 at 17:09 -0400, Jorge Vargas wrote: > > > elpargo at notebook:~$ sh setuptools-0.6c5-py2.5.egg > > > setuptools-0.6c5-py2.5.egg: 8: [[: not found > > > setuptools-0.6c5-py2.5.egg is not the correct name for this egg file. > > > Please rename it back to setuptools-0.6c5-py2.5.egg and try again. > > > >There was a previous mention of this on the mailing list. The Egg uses > >a bash-ism there, so "sh" doesn't quite work. I don't remember if PJE > >was going to change that in SVN, but I believe you should be able to run > >the Egg with bash instead. > > That's correct; 0.6c5 actually requires bash, but 0.6c6 uses '[' instead of > '[[' and so in theory should work with a non-bash "sh". > Oh I see my bad I don't read this mailing list that often I probably scroll over it without noticing. > If you're good with a binary editor, you could probably hack the header of > the current .egg to work. :) Or at least, you could manually run the > shell commands that are in it. or use the old install method :) > > thanks From jorge.vargas at gmail.com Wed Feb 7 20:31:48 2007 From: jorge.vargas at gmail.com (Jorge Vargas) Date: Wed, 7 Feb 2007 15:31:48 -0400 Subject: [Distutils] SetupTools: Forcefully Ignoring Dependencies In-Reply-To: <525f23e80702061258x75adfew24cd11bc60bc5ea4@mail.gmail.com> References: <525f23e80702061258x75adfew24cd11bc60bc5ea4@mail.gmail.com> Message-ID: <32822fe60702071131j4666a8a1u1194b006caa15964@mail.gmail.com> On 2/6/07, Alexander Michael wrote: > How can I force easy_install to not worry about (i.e. ignore) > dependencies? I tried --no-deps, but easy_install (from setuptools > 0.6c5) still attempts to find dependencies and then errors out. > --single-version-externally-managed ? > Thanks, > Alex > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From pje at telecommunity.com Wed Feb 7 21:41:06 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 07 Feb 2007 15:41:06 -0500 Subject: [Distutils] SetupTools: Forcefully Ignoring Dependencies In-Reply-To: <525f23e80702061258x75adfew24cd11bc60bc5ea4@mail.gmail.com> Message-ID: <5.1.1.6.0.20070207153758.03965390@sparrow.telecommunity.com> At 03:58 PM 2/6/2007 -0500, Alexander Michael wrote: >How can I force easy_install to not worry about (i.e. ignore) >dependencies? I tried --no-deps, but easy_install (from setuptools >0.6c5) still attempts to find dependencies and then errors out. Are you using --always-copy? That overrides --no-deps. You may wish to check your configuration files. Also, if you are building any packages from source, please note that they may specify build-time requirements using 'setup_requires', and that *cannot* be overridden by anything, ever, for any reason. :) You would have to install a pre-built egg instead of building from source to avoid those requirements. That's about all I can tell you without knowing what packages you're talking about and what messages you're seeing. From pje at telecommunity.com Wed Feb 7 21:41:42 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 07 Feb 2007 15:41:42 -0500 Subject: [Distutils] SetupTools: Forcefully Ignoring Dependencies In-Reply-To: <32822fe60702071131j4666a8a1u1194b006caa15964@mail.gmail.co m> References: <525f23e80702061258x75adfew24cd11bc60bc5ea4@mail.gmail.com> <525f23e80702061258x75adfew24cd11bc60bc5ea4@mail.gmail.com> Message-ID: <5.1.1.6.0.20070207154112.041f52e8@sparrow.telecommunity.com> At 03:31 PM 2/7/2007 -0400, Jorge Vargas wrote: >On 2/6/07, Alexander Michael wrote: > > How can I force easy_install to not worry about (i.e. ignore) > > dependencies? I tried --no-deps, but easy_install (from setuptools > > 0.6c5) still attempts to find dependencies and then errors out. > > >--single-version-externally-managed ? That's an 'install' option, not an easy_install option. From thomas at thomas-lotze.de Thu Feb 8 17:05:37 2007 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Thu, 08 Feb 2007 17:05:37 +0100 Subject: [Distutils] Setuptools: omit namespace package directories? Message-ID: Hi, when using namespace packages, the corresponding package directories and __init__.py files must physically exist in the source tree, even though they can't, by definition of a namespace package, contain anything other than subordinate directories and a fixed stanza of Python code, resp. This gets annoying, especially when using nested namespaces. For example, when writing a buildout recipe, the cleanest thing is to use several namespaces, like zc.recipe.egg does. However, you end up jumping through essentially empty directories all the time while working on the source. There's evidence that this problem is a real one: buildout recipes registered with PyPI tend to omit the "recipe" part at the expense of namespace hygiene and PEP 8 compliance, for example gocept.download or buildout_script. Since there's basically not information in those boilerplate directories and __init__.py files that couldn't be inferred from a keyword parameter to setup(), would it be a sensible feature request that setuptools do without the physical namespace directories in the future? -- Thomas From jim at zope.com Thu Feb 8 19:24:42 2007 From: jim at zope.com (Jim Fulton) Date: Thu, 8 Feb 2007 13:24:42 -0500 Subject: [Distutils] Setuptools: omit namespace package directories? In-Reply-To: References: Message-ID: <7B34A97C-C758-4289-BCAE-69B18F22AB45@zope.com> On Feb 8, 2007, at 11:05 AM, Thomas Lotze wrote: > Hi, > > when using namespace packages, the corresponding package > directories and > __init__.py files must physically exist in the source tree, even > though > they can't, by definition of a namespace package, contain anything > other > than subordinate directories and a fixed stanza of Python code, resp. > > This gets annoying, especially when using nested namespaces. For > example, > when writing a buildout recipe, the cleanest thing is to use several > namespaces, like zc.recipe.egg does. I don't agree that this is cleanest. I made a mistake by introducing the recipe namespace. New recipes I write won't use this. "Flat is better than nested." Also, for smaller projects, I'm avoiding intermediate src directories to avoid the annoying nesting. (I'm a little uneasy about this, but can't see a serious downside other than setup being importable in develop mode.) > However, you end up jumping through > essentially empty directories all the time while working on the > source. Yup > There's evidence that this problem is a real one: buildout recipes > registered with PyPI tend to omit the "recipe" part at the expense of > namespace hygiene and PEP 8 compliance, for example gocept.download or > buildout_script. IMO, the only needed hygiene is the top-level namespace. (Maybe a huge organization would need sub namespaces, but we don't.) > Since there's basically not information in those boilerplate > directories > and __init__.py files that couldn't be inferred from a keyword > parameter > to setup(), would it be a sensible feature request that setuptools do > without the physical namespace directories in the future? I would love to see this. I haven't been able to think of a good way to do it reliably, especially considering the important use case of develop eggs, which want to run directly from a checkout. Again, for smaller project, the pain can be mitigated by skipping the src directory and avoiding nested namespaces. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From p.f.moore at gmail.com Thu Feb 8 20:34:22 2007 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 8 Feb 2007 19:34:22 +0000 Subject: [Distutils] Compiler support for VC++ 2005 In-Reply-To: <32822fe60702071120s4d5ccb75ha726b13aae84a1c6@mail.gmail.com> References: <45B0AADD.13806.25AF95@howard.eegsoftware.com> <79990c6b0702020549q7fa212b2s6fc687b4cc3fc9e1@mail.gmail.com> <45C3437B.1020905@voidspace.org.uk> <79990c6b0702020623tb184f4dya5484b40e7285d4e@mail.gmail.com> <45C34929.4010701@voidspace.org.uk> <32822fe60702021943y4233aal24cca20bc7b5537b@mail.gmail.com> <79990c6b0702050333p7c12a227m73e6c314ced74b5f@mail.gmail.com> <32822fe60702071120s4d5ccb75ha726b13aae84a1c6@mail.gmail.com> Message-ID: <79990c6b0702081134s484503g353220e9423298c1@mail.gmail.com> On 07/02/07, Jorge Vargas wrote: > yes that's true the problem comes when you need windows.h or other > file that the mingw people can't distribute. Um, windows.h comes with mingw - it's in the win32api package. (Which doesn't alter the fact that MySQL may not support being compiled by mingw - but that's not because mingw doesn't supply the Windows headers and library files). Paul. From pje at telecommunity.com Thu Feb 8 23:04:56 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 08 Feb 2007 17:04:56 -0500 Subject: [Distutils] Setuptools: omit namespace package directories? In-Reply-To: Message-ID: <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> At 05:05 PM 2/8/2007 +0100, Thomas Lotze wrote: >Hi, > >when using namespace packages, the corresponding package directories and >__init__.py files must physically exist in the source tree, even though >they can't, by definition of a namespace package, contain anything other >than subordinate directories and a fixed stanza of Python code, resp. I'm not sure that's entirely true, although they *do* have to exist when using "setup.py develop", or they won't be importable. Have you tried using the distutils package_dir mapping to remap your source tree? >This gets annoying, especially when using nested namespaces. For example, >when writing a buildout recipe, the cleanest thing is to use several >namespaces, like zc.recipe.egg does. As Jim points out, nested namespace packages are usually a bad idea and shouldn't be created for new systems, as entry points are a better way of supporting third-party extensions to a package. Thus, having a single top-level namespace to denote a publisher or product line is usually sufficient. >However, you end up jumping through >essentially empty directories all the time while working on the source. >There's evidence that this problem is a real one: buildout recipes >registered with PyPI tend to omit the "recipe" part at the expense of >namespace hygiene and PEP 8 compliance, for example gocept.download or >buildout_script. I don't understand. >Since there's basically not information in those boilerplate directories >and __init__.py files that couldn't be inferred from a keyword parameter >to setup(), would it be a sensible feature request that setuptools do >without the physical namespace directories in the future? As I said, I'm not sure it needs them *now*, except to support setup.py develop. I'm not really fond of the package_dirs feature, preferring to use an importable layout when I do development. But the feature does exist and please feel free to let me know whether it solves your problem, or report *specifically* what it does or does not do correctly and what output is produced. Thanks. From thomas at thomas-lotze.de Fri Feb 9 09:08:25 2007 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Fri, 09 Feb 2007 09:08:25 +0100 Subject: [Distutils] Setuptools: omit namespace package directories? References: <7B34A97C-C758-4289-BCAE-69B18F22AB45@zope.com> Message-ID: Jim Fulton wrote: >> when writing a buildout recipe, the cleanest thing is to use several >> namespaces, like zc.recipe.egg does. > > I don't agree that this is cleanest. I made a mistake by introducing the > recipe namespace. New recipes I write won't use this. "Flat is better > than nested." To comment on buildout specifically, which I had only brought up as an example, though: While I agree with "flat is better than nested" most of the time, I don't when it comes to buildout recipes. It's the nature of buildout that some recipes deal with software that has a name of its own, or with general concepts. You wouldn't want to call a recipe for installing libfoo something like zc.foo or zc.libfoo as everybody would think it's your libfoo binding, and a name such as gocept.download is a bit too non-descriptive for my taste. So there should be a hint at buildout or recipes in the (dotted) name. Personally, I prefer a recipe namespace for recipes that deal with third-party software or generalities, and a recipe subpackage for one's own projects. Anything else feels rather non-elegant to me, but this might just be a matter of taste. What's your preferred naming scheme for new recipes? > Also, for smaller projects, I'm avoiding intermediate src > directories to avoid the annoying nesting. (I'm a little uneasy about > this, but can't see a serious downside other than setup being importable > in develop mode.) I do this all the time myself. > IMO, the only needed hygiene is the top-level namespace. (Maybe a huge > organization would need sub namespaces, but we don't.) I don't think it's a matter of running out of names when you have to name too many unrelated things, but one of finding concise and descriptive names for related or generic stuff. Descriptive and unique names in a flat namespace tend to grow long and in some cases, unreadable unless you violate PEP 8 and stick in an underscore or two. For example, I'm writing a bunch of buildout recipes for Apache stuff, and I'm not going to choose names such as tl.apache or tl.buildoutmodpython. Without introducing a namespace for my buildout recipes, buildout_apache and buildout_modpython is the least annoying naming scheme I can come up with. (BTW: Did I just miss something, or do recipes for these really not exist yet?) >> Since there's basically not information in those boilerplate directories >> and __init__.py files that couldn't be inferred from a keyword parameter >> to setup(), would it be a sensible feature request that setuptools do >> without the physical namespace directories in the future? > > I would love to see this. I haven't been able to think of a good way to > do it reliably, especially considering the important use case of develop > eggs, which want to run directly from a checkout. I'd have to look closer at what eggs do to Python's importing mechanism, but naively I'd hope that if it's possible to find a subpackage in one of several eggs that all hold parts of its top-level package, it should be possible to find it in some other place. Installing a develop egg would then have to mean more than placing a link file somewhere, possibly a stub egg for the namespace package. Maybe this is day-dreaming, though - I'll have to study setuptools first. -- Thomas From thomas at thomas-lotze.de Fri Feb 9 10:51:47 2007 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Fri, 09 Feb 2007 10:51:47 +0100 Subject: [Distutils] Setuptools: omit namespace package directories? References: <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 05:05 PM 2/8/2007 +0100, Thomas Lotze wrote: > I'm not sure that's entirely true, although they *do* have to exist when > using "setup.py develop", or they won't be importable. Have you tried > using the distutils package_dir mapping to remap your source tree? To achieve what? One does want to support setup.py develop, after all. > As Jim points out, nested namespace packages are usually a bad idea and > shouldn't be created for new systems, as entry points are a better way of > supporting third-party extensions to a package. Thus, having a single > top-level namespace to denote a publisher or product line is usually > sufficient. But if you want to distribute a package as separate eggs, you have to split it into subpackages, which you have to name somehow. If I'm not mistaken about the concept of entry points, they aren't relevant for this. >>However, you end up jumping through >>essentially empty directories all the time while working on the source. >>There's evidence that this problem is a real one: buildout recipes >>registered with PyPI tend to omit the "recipe" part at the expense of >>namespace hygiene and PEP 8 compliance, for example gocept.download or >>buildout_script. > > I don't understand. What don't you understand? > As I said, I'm not sure it needs them *now*, except to support setup.py > develop. I'm not really fond of the package_dirs feature, preferring to > use an importable layout when I do development. But the feature does > exist and please feel free to let me know whether it solves your problem, > or report *specifically* what it does or does not do correctly and what > output is produced. Thanks. For me, package_dirs does correctly what it's supposed to do, which does not include setup.py develop. Not having that is a show stopper for development. -- Thomas From jim at zope.com Fri Feb 9 15:15:42 2007 From: jim at zope.com (Jim Fulton) Date: Fri, 9 Feb 2007 09:15:42 -0500 Subject: [Distutils] Setuptools: omit namespace package directories? In-Reply-To: <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> References: <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> Message-ID: On Feb 8, 2007, at 5:04 PM, Phillip J. Eby wrote: > At 05:05 PM 2/8/2007 +0100, Thomas Lotze wrote: >> Hi, >> >> when using namespace packages, the corresponding package >> directories and >> __init__.py files must physically exist in the source tree, even >> though >> they can't, by definition of a namespace package, contain anything >> other >> than subordinate directories and a fixed stanza of Python code, resp. > > I'm not sure that's entirely true, although they *do* have to exist > when > using "setup.py develop", Yes. I think that's the main issue, as this is a very important use case. We rely on using develop eggs. > As Jim points out, nested namespace packages are usually a bad idea > and > shouldn't be created for new systems, as entry points are a better > way of > supporting third-party extensions to a package. I don't think that the nested namespace was motivated specifically by this. It was simply a result of a natural human tendency to organize. :) >> Since there's basically not information in those boilerplate >> directories >> and __init__.py files that couldn't be inferred from a keyword >> parameter >> to setup(), would it be a sensible feature request that setuptools do >> without the physical namespace directories in the future? > > As I said, I'm not sure it needs them *now*, except to support > setup.py > develop. Yes, as we need this. > I'm not really fond of the package_dirs feature, preferring to > use an importable layout when I do development. The question is whether a less annoying layout can be made importable. > But the feature does exist > and please feel free to let me know whether it solves your problem, It won't because of the need to support the develop use case. I haven't really had a chance to think about this, so I don't know what a solution, if there is one, would look like. For me, this isn't critical, as it is merely annoying to have to create namespace package directories. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From jim at zope.com Fri Feb 9 15:15:44 2007 From: jim at zope.com (Jim Fulton) Date: Fri, 9 Feb 2007 09:15:44 -0500 Subject: [Distutils] Setuptools: omit namespace package directories? In-Reply-To: References: <7B34A97C-C758-4289-BCAE-69B18F22AB45@zope.com> Message-ID: <3D3DE32E-F16D-4E2D-8A68-E493EEE3D705@zope.com> On Feb 9, 2007, at 3:08 AM, Thomas Lotze wrote: > Jim Fulton wrote: > >>> when writing a buildout recipe, the cleanest thing is to use several >>> namespaces, like zc.recipe.egg does. >> >> I don't agree that this is cleanest. I made a mistake by >> introducing the >> recipe namespace. New recipes I write won't use this. "Flat is >> better >> than nested." > > To comment on buildout specifically, which I had only brought up as an > example, though: > > While I agree with "flat is better than nested" most of the time, I > don't when it comes to buildout recipes. It's the nature of buildout > that some recipes deal with software that has a name of its own, or > with general concepts. You wouldn't want to call a recipe for > installing > libfoo something like zc.foo or zc.libfoo Right, I'd call it zc.libfoorecipe. > as everybody would think it's > your libfoo binding, and a name such as gocept.download is a bit too > non-descriptive for my taste. So there should be a hint at > buildout or > recipes in the (dotted) name. Personally, I prefer a recipe > namespace > for recipes that deal with third-party software or generalities, > and a > recipe subpackage for one's own projects. Anything else feels rather > non-elegant to me, but this might just be a matter of taste. > What's your > preferred naming scheme for new recipes? namespace.foorecipe. Abstractly, namespace.recipe.foo is appealing except for the directory management issue. But the directory management issue is a significant one. ... >> IMO, the only needed hygiene is the top-level namespace. (Maybe a >> huge >> organization would need sub namespaces, but we don't.) > > I don't think it's a matter of running out of names when you have > to name > too many unrelated things, Maybe not for you, but it is for me. In particular, I want to be able to name something without worrying about whether someone else has taken the name. The zc namespace protects me from that. I only need one level of namespace for that. > but one of finding concise and descriptive > names for related or generic stuff. Descriptive and unique names in > a flat > namespace tend to grow long and in some cases, unreadable unless you > violate PEP 8 and stick in an underscore or two. > > For example, I'm writing a bunch of buildout recipes for Apache > stuff, and > I'm not going to choose names such as tl.apache or > tl.buildoutmodpython. > Without introducing a namespace for my buildout recipes, > buildout_apache > and buildout_modpython is the least annoying naming scheme I can > come up > with. Why not just tl.apacherecipes, which could contain all of your apache recipes. > (BTW: Did I just miss something, or do recipes for these really not > exist yet?) I don't know, are you done yet? ;) Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From tseaver at palladion.com Fri Feb 9 15:29:19 2007 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 09 Feb 2007 09:29:19 -0500 Subject: [Distutils] setuptools extras_require In-Reply-To: <1166082725.27648.50.camel@nny> References: <5.1.1.6.0.20061210155520.029828f0@sparrow.telecommunity.com> <5035593a0612111344w7f5c3490t9c0424aa863fabcc@mail.gmail.com> <5035593a0612120732j7eaec5anb48234c2b056d9b0@mail.gmail.com> <1165941302.6958.23.camel@nny> <5035593a0612132215u3c08dda0h7f2f6fd9011c1697@mail.gmail.com> <1166082725.27648.50.camel@nny> Message-ID: <45CC853F.5020908@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Matt Good wrote: > On Thu, 2006-12-14 at 07:15 +0100, Elvelind Grandin wrote: >> Perhaps I wasn't totally clear. what I meant was something like this. >> >> extras_require = { >> "default" : ["SQLObject==bugfix,>=0.7.1dev-r1860,<=0.7.99"], >> "future" : ["Genshi>=0.3", "SQLAlchemy>=0.3"] >> }, > > That may be ok for the example, but what happens if you add another > extra "foo" for some optional feature. Installing "TurboGears[foo]" > won't select either the "default" or "future" extras, so you don't > really get all the requirements. > >>From a semantic perspective I think listing these as "extras" is odd. > One of either SQLObject or SQLAlchemy is required for installation, so > it's not just an "extra" feature. I think extras have been used for > this as a workaround to setuptools not supporting ORed requirements, but > I'd like to see that fixed. I tend to think in terms of Debian packages > where stuff like SA or SO would be listed as "depends" meaning that > they're strictly required, so in setuptools this would correspond to > "install_requires". On the other hand I believe Paste is still an > optional package used in TurboGears for stuff like the interactive > traceback display, so it'd be a Debian "recommends", or setuptools > "extras". > > Using "|" as a separator for ORing TurboGear's requirements might look > something like this (ignoring version numbers for simplicity): > > install_requires = [ > "SQLObject | SQLAlchemy", > "Kid | Genshi", > ], > extras_require = { > "fancy_tb": ["Paste"], > } +1 for the pipe symbol: to my eye, it means "pick the first of these", which is what you want. > Setuptools would check if either SQLObject or SQLAlchemy was already > installed, if not it'd try from left to right until it successfully > installed one of the requirements. > > I suppose extras could still be used as aliases for something like the > "future" requirements to make it easier for users to install this set of > packages: > > extras_require = { > "standard": ["Kid", "SQLObject"], > "future": ["Genshi", "SQLAlchemy"], > "fancy_tb": ["Paste"], > } > > Alternatively tuples could be substituted to define ORed requirements > like: > > install_requires = [ > ("SQLObject", "SQLAlchemy"), > ("Kid", "Genshi"), > ] > > Comments? I have a need for the "alternate dependencies" myself, e.g.: install_requires = ['lxml | elementtree'] where my package prefers lxml, but can fall back to using elementtree. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFzIU/+gerLs4ltQ4RAseRAJ4xhRcRmJbPegE1+ExInh9/VlZ4cgCbBhHO WlAJp1wRhxvnWQgoJ/T61oU= =yDKv -----END PGP SIGNATURE----- From arve.knudsen at gmail.com Fri Feb 9 15:31:30 2007 From: arve.knudsen at gmail.com (Arve Knudsen) Date: Fri, 9 Feb 2007 15:31:30 +0100 Subject: [Distutils] package_data not used by sdist command In-Reply-To: References: <5.1.1.6.0.20070202124556.02992cb8@sparrow.telecommunity.com> <45B4CB27.3000005@zope.com> <5.1.1.6.0.20070203110203.028f7ab0@sparrow.telecommunity.com> Message-ID: Philip, I've now modified the latest setuptools (0.6c5) so that the egg_info command respects package_data and exclude_package_data when creating its sources manifest (and thus influencing sdist's choice of files). Does this sound like the right approach to you (I figured sdist should base its choice of files on egg_info)? I've encountered one problem so far, which is that if data files are both listed in *.egg-info/SOURCES.txt and exclude_package_data they are not excluded. This may happen if files are added to the exclude list after a .egg-info is already generated. I'm not familiar enough with setuptools' design to know how to best solve this. Please let me know what you think. Best, Arve On 2/3/07, Arve Knudsen wrote: > > On Sat, 03 Feb 2007 17:03:27 +0100, Phillip J. Eby > wrote: > > > At 12:48 PM 2/3/2007 +0100, Arve Knudsen wrote: > >> Funny, I seem to recall having raised the same issue, but not receiving > >> much in way of an answer. Glad to see it's being taken care of at > least. > > > > Um, it's not being "taken care of". As I said, "a patch would go a long > > way towards getting it in." That means it's probably *not* being taken > > care of any time soon, unless somebody submits one. > > I interpreted you as having recognized the problem, but that it would not > make it into 0.6. As long as it can be scheduled for a future version I > can try my hand at contributing a patch (unless someone else wants to). > > Arve > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070209/8268edd3/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: setuptools-egginfo.patch Type: text/x-patch Size: 4080 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20070209/8268edd3/attachment.bin From tseaver at palladion.com Fri Feb 9 15:29:19 2007 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 09 Feb 2007 09:29:19 -0500 Subject: [Distutils] setuptools extras_require In-Reply-To: <1166082725.27648.50.camel@nny> References: <5.1.1.6.0.20061210155520.029828f0@sparrow.telecommunity.com> <5035593a0612111344w7f5c3490t9c0424aa863fabcc@mail.gmail.com> <5035593a0612120732j7eaec5anb48234c2b056d9b0@mail.gmail.com> <1165941302.6958.23.camel@nny> <5035593a0612132215u3c08dda0h7f2f6fd9011c1697@mail.gmail.com> <1166082725.27648.50.camel@nny> Message-ID: <45CC853F.5020908@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Matt Good wrote: > On Thu, 2006-12-14 at 07:15 +0100, Elvelind Grandin wrote: >> Perhaps I wasn't totally clear. what I meant was something like this. >> >> extras_require = { >> "default" : ["SQLObject==bugfix,>=0.7.1dev-r1860,<=0.7.99"], >> "future" : ["Genshi>=0.3", "SQLAlchemy>=0.3"] >> }, > > That may be ok for the example, but what happens if you add another > extra "foo" for some optional feature. Installing "TurboGears[foo]" > won't select either the "default" or "future" extras, so you don't > really get all the requirements. > >>From a semantic perspective I think listing these as "extras" is odd. > One of either SQLObject or SQLAlchemy is required for installation, so > it's not just an "extra" feature. I think extras have been used for > this as a workaround to setuptools not supporting ORed requirements, but > I'd like to see that fixed. I tend to think in terms of Debian packages > where stuff like SA or SO would be listed as "depends" meaning that > they're strictly required, so in setuptools this would correspond to > "install_requires". On the other hand I believe Paste is still an > optional package used in TurboGears for stuff like the interactive > traceback display, so it'd be a Debian "recommends", or setuptools > "extras". > > Using "|" as a separator for ORing TurboGear's requirements might look > something like this (ignoring version numbers for simplicity): > > install_requires = [ > "SQLObject | SQLAlchemy", > "Kid | Genshi", > ], > extras_require = { > "fancy_tb": ["Paste"], > } +1 for the pipe symbol: to my eye, it means "pick the first of these", which is what you want. > Setuptools would check if either SQLObject or SQLAlchemy was already > installed, if not it'd try from left to right until it successfully > installed one of the requirements. > > I suppose extras could still be used as aliases for something like the > "future" requirements to make it easier for users to install this set of > packages: > > extras_require = { > "standard": ["Kid", "SQLObject"], > "future": ["Genshi", "SQLAlchemy"], > "fancy_tb": ["Paste"], > } > > Alternatively tuples could be substituted to define ORed requirements > like: > > install_requires = [ > ("SQLObject", "SQLAlchemy"), > ("Kid", "Genshi"), > ] > > Comments? I have a need for the "alternate dependencies" myself, e.g.: install_requires = ['lxml | elementtree'] where my package prefers lxml, but can fall back to using elementtree. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFzIU/+gerLs4ltQ4RAseRAJ4xhRcRmJbPegE1+ExInh9/VlZ4cgCbBhHO WlAJp1wRhxvnWQgoJ/T61oU= =yDKv -----END PGP SIGNATURE----- From pje at telecommunity.com Fri Feb 9 18:08:24 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 09 Feb 2007 12:08:24 -0500 Subject: [Distutils] Setuptools: omit namespace package directories? In-Reply-To: References: <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> At 09:15 AM 2/9/2007 -0500, Jim Fulton wrote: >The question is whether a less annoying layout can be made importable. Well, what's "annoying" is relative. I personally find the flat layout "annoying" because it's not WYSIWYG. :) But the answer to your question is "yes". Anything can be made importable, with sufficient effort. When a project that includes a namespace package is installed using --single-version-externally-managed, a special .pth file for that project is created that sets up empty modules in sys.modules with appropriate __path__ values. A similar technique could be applied in setup.py develop, if a project uses package_dirs. I'm just a bit reluctant to try to shoehorn something like that into 0.6. But, if you'd like to experiment with creating a patch (or a subclass of "develop") that would support creating and uninstalling this .pth file, see the 'install_namespaces()' method of the install_egg_info command in setuptools. The code you'd need for this would actually be *simpler* in some ways, because that code is trying to work relative to whatever directory it's installed in, but for what "develop" needs you could just bake the absolute paths right in. The uninstall mode of "develop" could just remove the .pth file. One possible complication, however, is that if someone didn't uninstall but left the .pth file around, it would produce strange results (if e.g., they installed an egg for the project without uninstalling the develop mode). So, the code in the .pth should probably check that the original project is still on sys.path before creating dummy modules. From strawman at astraw.com Fri Feb 9 18:34:49 2007 From: strawman at astraw.com (Andrew Straw) Date: Fri, 09 Feb 2007 09:34:49 -0800 Subject: [Distutils] setuptools: including shared libraries the Right Way Message-ID: <45CCB0B9.5040305@astraw.com> I'm trying to include shared libraries (built by scons invoked from my setup.py script) into built eggs. I can do this just fine with the package_data command, but then the egg isn't known to be platform dependent. This is somewhat puzzling, because setuptools notices the library and lists it in .egg-info/native_libs.txt. Perhaps this is expected behavior because, in theory, I could include pre-built native libraries of more than one platform, or the native libraries might only be required on one platform. This leads me to the bdist_egg --plat-name option, which doesn't seem to work in this case. "python setup.py bdist_egg --plat-name=abc" doesn't produce an .egg with "abc" in the filename. This is apparently because setuptools thinks the egg doesn't contain C extensions. Perhaps the solution to this issue is to tell setuptools "yes, I did build C extensions". How would I achieve this? Any suggestions? -Andrew From pje at telecommunity.com Fri Feb 9 19:02:48 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 09 Feb 2007 13:02:48 -0500 Subject: [Distutils] setuptools: including shared libraries the Right Way In-Reply-To: <45CCB0B9.5040305@astraw.com> Message-ID: <5.1.1.6.0.20070209130035.028e7678@sparrow.telecommunity.com> At 09:34 AM 2/9/2007 -0800, Andrew Straw wrote: >I'm trying to include shared libraries (built by scons invoked from my >setup.py script) into built eggs. I can do this just fine with the >package_data command, but then the egg isn't known to be platform >dependent. This is somewhat puzzling, because setuptools notices the >library and lists it in .egg-info/native_libs.txt. Perhaps this is >expected behavior because, in theory, I could include pre-built native >libraries of more than one platform, or the native libraries might only >be required on one platform. > >This leads me to the bdist_egg --plat-name option, which doesn't seem to >work in this case. "python setup.py bdist_egg --plat-name=abc" doesn't >produce an .egg with "abc" in the filename. This is apparently because >setuptools thinks the egg doesn't contain C extensions. More precisely, it's distutils, which setuptools asks to find out if there are any. Unfortunately, at the time setuptools needs to know this, it hasn't checked for native libraries yet, because that's an egg-building thing. >Perhaps the solution to this issue is to tell setuptools "yes, I did >build C extensions". How would I achieve this? A quick workaround: from setuptools.dist import Distribution class MyDist(Distribution): def has_ext_modules(self): return True setup( ... distclass = MyDist, ) From jim at zope.com Fri Feb 9 19:21:37 2007 From: jim at zope.com (Jim Fulton) Date: Fri, 9 Feb 2007 13:21:37 -0500 Subject: [Distutils] Setuptools: omit namespace package directories? In-Reply-To: <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> References: <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> Message-ID: On Feb 9, 2007, at 12:08 PM, Phillip J. Eby wrote: > At 09:15 AM 2/9/2007 -0500, Jim Fulton wrote: >> The question is whether a less annoying layout can be made >> importable. > > Well, what's "annoying" is relative. I personally find the flat > layout "annoying" because it's not WYSIWYG. :) > > But the answer to your question is "yes". Anything can be made > importable, with sufficient effort. Yup. > When a project that includes a namespace package is installed using > --single-version-externally-managed, a special .pth file for that > project is created that sets up empty modules in sys.modules with > appropriate __path__ values. I really don't fathom --single-version-externally-managed. :) Where is this .pth file created? > A similar technique could be applied in setup.py develop, if a > project uses package_dirs. I'm just a bit reluctant to try to > shoehorn something like that into 0.6. Of course. > But, if you'd like to experiment with creating a patch (or a > subclass of "develop") that would support creating and uninstalling > this .pth file, see the 'install_namespaces()' method of the > install_egg_info command in setuptools. The code you'd need for > this would actually be *simpler* in some ways, because that code is > trying to work relative to whatever directory it's installed in, > but for what "develop" needs you could just bake the absolute paths > right in. What would read this .pth file? > The uninstall mode of "develop" could just remove the .pth file. > One possible complication, however, is that if someone didn't > uninstall but left the .pth file around, it would produce strange > results (if e.g., they installed an egg for the project without > uninstalling the develop mode). So, the code in the .pth should > probably check that the original project is still on sys.path > before creating dummy modules. I kinda doubt I understand this enough to pursue it. In any case, I won't have time until after PyCon. I may ask you more about this there (assuming that you'll be there.) Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From thomas at thomas-lotze.de Fri Feb 9 20:10:08 2007 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Fri, 09 Feb 2007 20:10:08 +0100 Subject: [Distutils] Setuptools: omit namespace package directories? References: <7B34A97C-C758-4289-BCAE-69B18F22AB45@zope.com> <3D3DE32E-F16D-4E2D-8A68-E493EEE3D705@zope.com> Message-ID: Jim Fulton wrote: > Abstractly, namespace.recipe.foo is appealing except for the directory > management issue. But the directory management issue is a significant one. And it is a technical one, so I think it should be fixed technically, in setuptools, instead of by bending our usage pattern around it. > Maybe not for you, but it is for me. In particular, I want to be able to > name something without worrying about whether someone else has taken the > name. The zc namespace protects me from that. I only need one level of > namespace for that. Ah, concerning the top level, it is of course an issue of namespace clutter for me as well. And one of modesty and responsibility: for example, I would neither want nor dare publish a PDF library as a top-level 'pdf' package until it has earned a reputation as _the_ standard Python PDF library. What I actually meant was that I'm personally not too afraid of polluting a namespace of my own but that organizing a few but related packages seems to be the bigger problem there. > Why not just tl.apacherecipes, which could contain all of your apache > recipes. I used to shy away from that name because of two issues that are probably very much up to personal taste and the degree one cares about this kind of detail: running the words right together looks ugly to me, and appending the "recipe" part to its end conflicts with the way hierarchical package names get more specific from left to right. But now that I see it actually spelled out, I think I might even like it better than my other suggestion in case I really decide to collect the recipes under a two-part name. >> (BTW: Did I just miss something, or do recipes for these really not >> exist yet?) > > I don't know, are you done yet? ;) I wouldn't call it done. Setting up a server root (an instance in Zope speak) with a preinstalled Apache does work but could still do with some more configurability. I hope to add a recipe for building Apache today, and one for adding mod_python on the weekend. If I can stick to this plan, I'll do a first release next week. -- Thomas From pje at telecommunity.com Fri Feb 9 21:33:23 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 09 Feb 2007 15:33:23 -0500 Subject: [Distutils] Setuptools: omit namespace package directories? In-Reply-To: References: <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070209152759.05074008@sparrow.telecommunity.com> At 01:21 PM 2/9/2007 -0500, Jim Fulton wrote: >I really don't fathom --single-version-externally-managed. :) It's the same as good old distutils "install" -- with a couple of additions. The additions are that an .egg-info directory is installed alongside the package(s), and if there are namespace packages involved, a .pth file is also added. >Where is this .pth file created? In site-packages (or whatever the effective --install-lib target directory is). >>But, if you'd like to experiment with creating a patch (or a >>subclass of "develop") that would support creating and uninstalling >>this .pth file, see the 'install_namespaces()' method of the >>install_egg_info command in setuptools. The code you'd need for >>this would actually be *simpler* in some ways, because that code is >>trying to work relative to whatever directory it's installed in, >>but for what "develop" needs you could just bake the absolute paths >>right in. > >What would read this .pth file? Python, at startup, causing the empty namespace packages to be created in sys.modules with usable __path__ settings. >I kinda doubt I understand this enough to pursue it. In any case, I >won't have time until after PyCon. I may ask you more about this >there (assuming that you'll be there.) No, I'm not going this year. From jim at zope.com Fri Feb 9 21:42:15 2007 From: jim at zope.com (Jim Fulton) Date: Fri, 9 Feb 2007 15:42:15 -0500 Subject: [Distutils] Setuptools: omit namespace package directories? In-Reply-To: <5.1.1.6.0.20070209152759.05074008@sparrow.telecommunity.com> References: <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> <5.1.1.6.0.20070209152759.05074008@sparrow.telecommunity.com> Message-ID: <48701E33-297D-496D-8398-C7AABBD6E180@zope.com> On Feb 9, 2007, at 3:33 PM, Phillip J. Eby wrote: > At 01:21 PM 2/9/2007 -0500, Jim Fulton wrote: >> I really don't fathom --single-version-externally-managed. :) > > It's the same as good old distutils "install" -- with a couple of > additions. Yeah, but I don't fathom distutils either. ;) > The additions are that an .egg-info directory is installed > alongside the package(s), and if there are namespace packages > involved, a .pth file is also added. > > >> Where is this .pth file created? > > In site-packages (or whatever the effective --install-lib target > directory is). This makes this approach uninteresting for buildout, which doesn't write to site-packages or have any site-packages equivalent. It sound like it also violates the egg promise that you just have to put the egg in sys.path for it to be useable. buildout relies on this promise. ... >> I kinda doubt I understand this enough to pursue it. In any case, I >> won't have time until after PyCon. I may ask you more about this >> there (assuming that you'll be there.) > > No, I'm not going this year. Darn. We'll miss you. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From pje at telecommunity.com Fri Feb 9 21:57:15 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 09 Feb 2007 15:57:15 -0500 Subject: [Distutils] setuptools extras_require In-Reply-To: <45CC853F.5020908@palladion.com> References: <1166082725.27648.50.camel@nny> <5.1.1.6.0.20061210155520.029828f0@sparrow.telecommunity.com> <5035593a0612111344w7f5c3490t9c0424aa863fabcc@mail.gmail.com> <5035593a0612120732j7eaec5anb48234c2b056d9b0@mail.gmail.com> <1165941302.6958.23.camel@nny> <5035593a0612132215u3c08dda0h7f2f6fd9011c1697@mail.gmail.com> <1166082725.27648.50.camel@nny> Message-ID: <5.1.1.6.0.20070209154357.0500fe68@sparrow.telecommunity.com> At 09:29 AM 2/9/2007 -0500, Tres Seaver wrote: > > Using "|" as a separator for ORing TurboGear's requirements might look > > something like this (ignoring version numbers for simplicity): > > > > install_requires = [ > > "SQLObject | SQLAlchemy", > > "Kid | Genshi", > > ], > > extras_require = { > > "fancy_tb": ["Paste"], > > } > >+1 for the pipe symbol: to my eye, it means "pick the first of these", >which is what you want. It's also implementable; the tuple concept isn't. Eggs' requirements get stored in text files, so there has to be a pure-text representation possible. > > Setuptools would check if either SQLObject or SQLAlchemy was already > > installed, if not it'd try from left to right until it successfully > > installed one of the requirements. Yeah, it would be pretty straightforward to add the parsing and a new type of "Requirement" object to support alternatives, *but* nearly all of setuptools expects individual Requirement instances to refer to a single project, not multiple alternative projects. Integrating an alternative type of "MultiRequirement" object could introduce random breakage. Essentially, everything that uses Requirement.parse would need to be changed to handle the case where alternative requirements were included, and for some parts of setuptools it would be a significant change. So, this is something that'll have to wait for 0.7. Also, there are some open design questions with this idea. For example, will we need command-line options to allow someone to force installing the "best" choices, even if a less-desirable option is still available? (I suppose this could be handled with an extra, e.g. "foo[recommended]".) And what kinds of installation problems should be considered "recoverable"? Do we want to install a lesser alternative just because a server is down at the moment? Should we in fact ever try to *install* an alternative, or merely allow already-installed alternatives to satisfy a dependency? Yet another problem is backtracking. Right now, the dependency resolving algorithm never backtracks. But in the world of allowing alternatives, it seems as though it would need to be able to back out if the first thing is installed, but one of *its* dependencies is missing. Although I suppose that could just produce an error... but right now there's no easy_uninstall to let you remove the broken item easily. (These types of questions are why I never implemented this in the first place.) From pje at telecommunity.com Fri Feb 9 22:16:45 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 09 Feb 2007 16:16:45 -0500 Subject: [Distutils] Setuptools: omit namespace package directories? In-Reply-To: <48701E33-297D-496D-8398-C7AABBD6E180@zope.com> References: <5.1.1.6.0.20070209152759.05074008@sparrow.telecommunity.com> <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> <5.1.1.6.0.20070209152759.05074008@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070209161150.03d72140@sparrow.telecommunity.com> At 03:42 PM 2/9/2007 -0500, Jim Fulton wrote: >This makes this approach uninteresting for buildout, which doesn't >write to site-packages or have any site-packages equivalent. > >It sound like it also violates the egg promise that you just have to >put the egg in sys.path for it to be useable. buildout relies on >this promise. Well, the backward-compatibility mode is for making system packages like .rpm files, and the namespace package support for that is a hack to deal with the fact that such packaging tools don't like to have multiple .rpm's or whatever containing the same file (i.e. the namespace package's __init__.py). This would be an even more extreme hack, if we tried to support 'develop' mode for procrustean package_dirs setups. You do, however, make a good point regarding the egg promise. As far as I can see, then, there is no way to support crazy package_dirs in combination with namespace packages, without automatically creating a bunch of directories and __init__.py files, along with some other crazy hacks. So the idea is probably a dead duck. From thomas at thomas-lotze.de Fri Feb 9 23:01:20 2007 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Fri, 09 Feb 2007 23:01:20 +0100 Subject: [Distutils] Setuptools: omit namespace package directories? References: <5.1.1.6.0.20070209152759.05074008@sparrow.telecommunity.com> <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> <5.1.1.6.0.20070209152759.05074008@sparrow.telecommunity.com> <48701E33-297D-496D-8398-C7AABBD6E180@zope.com> <5.1.1.6.0.20070209161150.03d72140@sparrow.telecommunity.com> Message-ID: Am Fri, 09 Feb 2007 16:16:45 -0500 schrieb Phillip J. Eby: > As far as I can see, then, there is no > way to support crazy package_dirs in combination with namespace packages, > without automatically creating a bunch of directories and __init__.py > files, along with some other crazy hacks. So the idea is probably a dead > duck. Not necessarily. If I get all of this right, then namespace packages work by creating a bunch of module objects in sys.modules which then take care of finding the right place to import any given subpackage from. So what is needed is to get some code executed that does the Right Thing when first trying to import a package or module that resides inside a namespace. If the egg promise is that putting an egg on sys.path suffices for making it available to Python, this precludes relying on .pth files in any way for doing something other than modifying sys.path itself, right? After all, the egg could be added to sys.path after .pth files have been evaluated - which I understand is the approach buildout takes. The next chance to get code executed is when the top-level namespace's __init__.py is imported. Thus there absolutely must exist a directory for the top-level namespace inside the source tree. But - what prevents the __init__.py file in this directory from setting up any intermediate namespaces in sys.modules and causing the real package to be looked for in some convenient place that avoids directory jumping in the source tree? -- Viele Gr??e, Thomas From jim at zope.com Fri Feb 9 23:26:17 2007 From: jim at zope.com (Jim Fulton) Date: Fri, 9 Feb 2007 17:26:17 -0500 Subject: [Distutils] Building an egg from Subversion using zc.buildout In-Reply-To: References: <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <45BE6CAF.3010206@gmx.net> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <1163956881.5763.21.camel@Jane> <5.1.1.6.0.20061119134913.01f3d088@sparrow.telecommunity.com> <45BD22C2.7030403@colorstudy.com> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <5.1.1.6.0.20070128202027.04461008@sparrow.telecommunity.com> <9D0FD265-B323-42C3-80A1-85638A067C9A@zope.com> <45BE6CAF.3010206@gmx.net> <65788D22-5FA7-4AC1-A6FA-08FDA4D173FE@zope.com> <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <5.1.1.6.0.20070129180219.045646e0@sparrow.telecommunity.com> Message-ID: <45244A6B-34EA-4D5F-8BDE-EF6D8E068C0C@zope.com> On Jan 29, 2007, at 6:14 PM, Martin Aspeli wrote: > Phillip J. Eby wrote: >> At 10:37 PM 1/29/2007 +0000, Martin Aspeli wrote: >>> Phillip J. Eby wrote: >>> >>>> Development snapshots of setuptools can be checked out with >>>> "setuptools==dev", and that will always download a new version >>>> because the >>>> installed version is always a version like 0.6c2dev-r51290 or >>>> something. This doesn't match ==dev, thereby forcing >>>> easy_install to >>>> download a fresh copy. >>> Could you explain how the find-link would work with an ==dev version >>> specification? >> >> #egg=archetypes.kss-dev >> >> That is, use 'dev' as the version in the link. The combination of >> an ==dev >> version specification and a #egg=projectname-dev tag means that >> the code >> will be downloaded and rebuilt *every* time, however. > > Alas, this gives me this error: ... > ValueError: ("Couldn't find", Requirement.parse > ('archetypes.kss==dev')) I finally found some time to look into this. Buildout uses setuptools APIs. It uses a PackageIndex to download the distribution and this actually works. It then uses an environment best_match method to get a distribution object and best_match returns None. If I remove the ==dev from the requirement, then best_match is able to find the distribution. This happens because apparently no distributions match a requirement for the "dev" version. Should I view this as a bug in setuptools? Or does buildout need to special case the version "dev"? Martin, If this worked, then buildout would checkout, build, and download these eggs every time it was run in "newest" mode, which is the default mode. I'll make this work on principle, but I suggest that it would be much more efficient and useful to have a recipe that checked out the source and built a develop egg on install and updated it on update. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From optilude at gmx.net Sat Feb 10 00:28:02 2007 From: optilude at gmx.net (Martin Aspeli) Date: Fri, 09 Feb 2007 23:28:02 +0000 Subject: [Distutils] Building an egg from Subversion using zc.buildout In-Reply-To: <45244A6B-34EA-4D5F-8BDE-EF6D8E068C0C@zope.com> References: <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <45BE6CAF.3010206@gmx.net> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <1163956881.5763.21.camel@Jane> <5.1.1.6.0.20061119134913.01f3d088@sparrow.telecommunity.com> <45BD22C2.7030403@colorstudy.com> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <5.1.1.6.0.20070128202027.04461008@sparrow.telecommunity.com> <9D0FD265-B323-42C3-80A1-85638A067C9A@zope.com> <45BE6CAF.3010206@gmx.net> <65788D22-5FA7-4AC1-A6FA-08FDA4D173FE@zope.com> <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <5.1.1.6.0.20070129180219.045646e0@sparrow.telecommunity.com> <45244A6B-34EA-4D5F-8BDE-EF6D8E068C0C@zope.com> Message-ID: <45CD0382.9060407@gmx.net> Jim Fulton wrote: > On Jan 29, 2007, at 6:14 PM, Martin Aspeli wrote: > >> Phillip J. Eby wrote: >>> At 10:37 PM 1/29/2007 +0000, Martin Aspeli wrote: >>>> Phillip J. Eby wrote: >>>> >>>>> Development snapshots of setuptools can be checked out with >>>>> "setuptools==dev", and that will always download a new version >>>>> because the >>>>> installed version is always a version like 0.6c2dev-r51290 or >>>>> something. This doesn't match ==dev, thereby forcing >>>>> easy_install to >>>>> download a fresh copy. >>>> Could you explain how the find-link would work with an ==dev version >>>> specification? >>> #egg=archetypes.kss-dev >>> >>> That is, use 'dev' as the version in the link. The combination of >>> an ==dev >>> version specification and a #egg=projectname-dev tag means that >>> the code >>> will be downloaded and rebuilt *every* time, however. >> Alas, this gives me this error: > > ... > >> ValueError: ("Couldn't find", Requirement.parse >> ('archetypes.kss==dev')) > > I finally found some time to look into this. Thanks Jim! > Buildout uses setuptools APIs. It uses a PackageIndex to download > the distribution and this actually works. It then uses an > environment best_match method to get a distribution object and > best_match returns None. If I remove the ==dev from the requirement, > then best_match is able to find the distribution. This happens > because apparently no distributions match a requirement for the "dev" > version. Should I view this as a bug in setuptools? Or does buildout > need to special case the version "dev"? How come it works with easy_install on the command line? > Martin, > > If this worked, then buildout would checkout, build, and download > these eggs every time it was run in "newest" mode, which is the > default mode. I'll make this work on principle, but I suggest that > it would be much more efficient and useful to have a recipe that > checked out the source and built a develop egg on install and updated > it on update. I think that's what I'd expect, yes. For small eggs and where you're tracking svn closely, it makes sense. We do have a recipe (z2c.recipe.bundlecheckout) which can check out things and then you can list them as development eggs (which you do manually in buildout.cfg). We use this for Plone source code, for example, in "ploneout" the plone 3 buildout. However, if I'm not actually interested in editing the thing, I'd rather not have it in src/. Martin From optilude at gmx.net Sat Feb 10 00:28:02 2007 From: optilude at gmx.net (Martin Aspeli) Date: Fri, 09 Feb 2007 23:28:02 +0000 Subject: [Distutils] Building an egg from Subversion using zc.buildout In-Reply-To: <45244A6B-34EA-4D5F-8BDE-EF6D8E068C0C@zope.com> References: <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <45BE6CAF.3010206@gmx.net> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <1163956881.5763.21.camel@Jane> <5.1.1.6.0.20061119134913.01f3d088@sparrow.telecommunity.com> <45BD22C2.7030403@colorstudy.com> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <5.1.1.6.0.20070128202027.04461008@sparrow.telecommunity.com> <9D0FD265-B323-42C3-80A1-85638A067C9A@zope.com> <45BE6CAF.3010206@gmx.net> <65788D22-5FA7-4AC1-A6FA-08FDA4D173FE@zope.com> <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <5.1.1.6.0.20070129180219.045646e0@sparrow.telecommunity.com> <45244A6B-34EA-4D5F-8BDE-EF6D8E068C0C@zope.com> Message-ID: <45CD0382.9060407@gmx.net> Jim Fulton wrote: > On Jan 29, 2007, at 6:14 PM, Martin Aspeli wrote: > >> Phillip J. Eby wrote: >>> At 10:37 PM 1/29/2007 +0000, Martin Aspeli wrote: >>>> Phillip J. Eby wrote: >>>> >>>>> Development snapshots of setuptools can be checked out with >>>>> "setuptools==dev", and that will always download a new version >>>>> because the >>>>> installed version is always a version like 0.6c2dev-r51290 or >>>>> something. This doesn't match ==dev, thereby forcing >>>>> easy_install to >>>>> download a fresh copy. >>>> Could you explain how the find-link would work with an ==dev version >>>> specification? >>> #egg=archetypes.kss-dev >>> >>> That is, use 'dev' as the version in the link. The combination of >>> an ==dev >>> version specification and a #egg=projectname-dev tag means that >>> the code >>> will be downloaded and rebuilt *every* time, however. >> Alas, this gives me this error: > > ... > >> ValueError: ("Couldn't find", Requirement.parse >> ('archetypes.kss==dev')) > > I finally found some time to look into this. Thanks Jim! > Buildout uses setuptools APIs. It uses a PackageIndex to download > the distribution and this actually works. It then uses an > environment best_match method to get a distribution object and > best_match returns None. If I remove the ==dev from the requirement, > then best_match is able to find the distribution. This happens > because apparently no distributions match a requirement for the "dev" > version. Should I view this as a bug in setuptools? Or does buildout > need to special case the version "dev"? How come it works with easy_install on the command line? > Martin, > > If this worked, then buildout would checkout, build, and download > these eggs every time it was run in "newest" mode, which is the > default mode. I'll make this work on principle, but I suggest that > it would be much more efficient and useful to have a recipe that > checked out the source and built a develop egg on install and updated > it on update. I think that's what I'd expect, yes. For small eggs and where you're tracking svn closely, it makes sense. We do have a recipe (z2c.recipe.bundlecheckout) which can check out things and then you can list them as development eggs (which you do manually in buildout.cfg). We use this for Plone source code, for example, in "ploneout" the plone 3 buildout. However, if I'm not actually interested in editing the thing, I'd rather not have it in src/. Martin From skip at pobox.com Sat Feb 10 00:30:14 2007 From: skip at pobox.com (skip at pobox.com) Date: Fri, 9 Feb 2007 17:30:14 -0600 Subject: [Distutils] Confusion about the effect of eggs on import Message-ID: <17869.1030.405845.96720@montanaro.dyndns.org> (I'm sending this to distutils because I have this vague notion that the problem might have something to do with setuptools or eggs on sys.path.) We have a weird issue trying to import matplotlib at work. Sameer has matplotlib, numpy and scipy installed in his local directory tree: % ls -l /home/titan/sameer/local/lib/python2.4/site-packages total 20 drwxr-xr-x 5 sameer research 2560 Feb 9 13:09 IPython drwxr-xr-x 7 sameer research 2560 Feb 9 14:06 matplotlib drwxr-xr-x 14 sameer research 1024 Feb 9 13:02 numpy -rw-r--r-- 1 sameer research 31 Dec 9 2004 pylab.py -rw-r--r-- 1 sameer research 180 Feb 9 13:46 pylab.pyc drwxr-xr-x 22 sameer research 1024 Feb 9 13:09 scipy If sys.path doesn't contain any eggs matplotlib is imported from Sameer's directory. If it does contain eggs, no matter if they are at the start or the end, his version of matplotlib isn't found. Instead, the centrally installed version is imported. Sameer's versions of numpy and scipy are always imported as expected. Here are three sessions which demonstrate the problem. In the first session I explicitly remove the two eggs from sys.path: % PYTHONPATH=/home/titan/sameer/local/lib/python2.4/site-packages python Python 2.4.2 (#1, Feb 23 2006, 12:48:31) [GCC 3.4.1] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> import sys.path >>> sys.path ['/home/titan/skipm/misc/python/python2', '/home/titan/skipm/misc/python', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/gtk-2.6', '', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/SQLAlchemy-0.3.3-py2.4.egg', '/home/titan/sameer/local/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4', '/opt/app/g++lib6/python-2.4/lib/python2.4/plat-sunos5', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-tk', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-dynload', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/Numeric', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/f2py2e', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/scipy_distutils'] >>> noneggs = [p for p in sys.path if not p.endswith(".egg")] >>> noneggs ['/home/titan/skipm/misc/python/python2', '/home/titan/skipm/misc/python', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/gtk-2.6', '', '/home/titan/sameer/local/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4', '/opt/app/g++lib6/python-2.4/lib/python2.4/plat-sunos5', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-tk', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-dynload', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/Numeric', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/f2py2e', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/scipy_distutils'] >>> sys.path = noneggs >>> import matplotlib >>> matplotlib.__file__ '/home/titan/sameer/local/lib/python2.4/site-packages/matplotlib/__init__.pyc' Matplotlib comes from Sameer's directory, as it should. In the second session I don't mess with sys.path: % PYTHONPATH=/home/titan/sameer/local/lib/python2.4/site-packages python Python 2.4.2 (#1, Feb 23 2006, 12:48:31) [GCC 3.4.1] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path ['/home/titan/skipm/misc/python/python2', '/home/titan/skipm/misc/python', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/gtk-2.6', '', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/SQLAlchemy-0.3.3-py2.4.egg', '/home/titan/sameer/local/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4', '/opt/app/g++lib6/python-2.4/lib/python2.4/plat-sunos5', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-tk', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-dynload', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/Numeric', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/f2py2e', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/scipy_distutils'] >>> import matplotlib Bad key "lines.markeredgecolor" on line 48 in /opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/matplotlib/mpl-data/matplotlibrc. You probably need to get an updated matplotlibrc file from http://matplotlib.sf.net/matplotlibrc or from the matplotlib source distribution Bad key "lines.markerfacecolor" on line 47 in /opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/matplotlib/mpl-data/matplotlibrc. You probably need to get an updated matplotlibrc file from http://matplotlib.sf.net/matplotlibrc or from the matplotlib source distribution >>> matplotlib.__file__ '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/matplotlib/__init__.pyc' Even though Sameer's directory is ahead of the central installation directory, the centrally installed version is imported. Here's the final session. Rearrange sys.path so all eggs are at the end: % PYTHONPATH=/home/titan/sameer/local/lib/python2.4/site-packages python Python 2.4.2 (#1, Feb 23 2006, 12:48:31) [GCC 3.4.1] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path ['/home/titan/skipm/misc/python/python2', '/home/titan/skipm/misc/python', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/gtk-2.6', '', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/SQLAlchemy-0.3.3-py2.4.egg', '/home/titan/sameer/local/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4', '/opt/app/g++lib6/python-2.4/lib/python2.4/plat-sunos5', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-tk', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-dynload', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/Numeric', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/f2py2e', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/scipy_distutils'] >>> noneggs = [p for p in sys.path if not p.endswith(".egg")] >>> eggs = [p for p in sys.path if p.endswith(".egg")] >>> sys.path = noneggs + eggs >>> sys.path ['/home/titan/skipm/misc/python/python2', '/home/titan/skipm/misc/python', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/gtk-2.6', '', '/home/titan/sameer/local/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4', '/opt/app/g++lib6/python-2.4/lib/python2.4/plat-sunos5', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-tk', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-dynload', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/Numeric', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/f2py2e', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/scipy_distutils', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/SQLAlchemy-0.3.3-py2.4.egg'] >>> import matplotlib Bad key "lines.markeredgecolor" on line 48 in /opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/matplotlib/mpl-data/matplotlibrc. You probably need to get an updated matplotlibrc file from http://matplotlib.sf.net/matplotlibrc or from the matplotlib source distribution Bad key "lines.markerfacecolor" on line 47 in /opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/matplotlib/mpl-data/matplotlibrc. You probably need to get an updated matplotlibrc file from http://matplotlib.sf.net/matplotlibrc or from the matplotlib source distribution >>> matplotlib.__file__ '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/matplotlib/__init__.pyc' I get the same result if I unset PYTHONSTARTUP. Any idea what's going on? Thx, -- Skip Montanaro - skip at pobox.com - http://www.webfast.com/~skip/ "The hippies and the hipsters did some great stuff in the sixties, but the geeks pulled their weight too." -- Billy Bragg From pje at telecommunity.com Sat Feb 10 00:49:54 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 09 Feb 2007 18:49:54 -0500 Subject: [Distutils] Building an egg from Subversion using zc.buildout In-Reply-To: <45244A6B-34EA-4D5F-8BDE-EF6D8E068C0C@zope.com> References: <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <45BE6CAF.3010206@gmx.net> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <1163956881.5763.21.camel@Jane> <5.1.1.6.0.20061119134913.01f3d088@sparrow.telecommunity.com> <45BD22C2.7030403@colorstudy.com> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <5.1.1.6.0.20070128202027.04461008@sparrow.telecommunity.com> <9D0FD265-B323-42C3-80A1-85638A067C9A@zope.com> <45BE6CAF.3010206@gmx.net> <65788D22-5FA7-4AC1-A6FA-08FDA4D173FE@zope.com> <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <5.1.1.6.0.20070129180219.045646e0@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070209184841.03d75830@sparrow.telecommunity.com> At 05:26 PM 2/9/2007 -0500, Jim Fulton wrote: > > ValueError: ("Couldn't find", Requirement.parse > > ('archetypes.kss==dev')) > >I finally found some time to look into this. > >Buildout uses setuptools APIs. It uses a PackageIndex to download >the distribution and this actually works. It then uses an >environment best_match method to get a distribution object and >best_match returns None. If I remove the ==dev from the requirement, >then best_match is able to find the distribution. This happens >because apparently no distributions match a requirement for the "dev" >version. Should I view this as a bug in setuptools? The normal usage is "==dev,>=realver" where 'realver' is the real version you want. > Or does buildout >need to special case the version "dev"? I don't think so. From pje at telecommunity.com Sat Feb 10 01:20:31 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 09 Feb 2007 19:20:31 -0500 Subject: [Distutils] Confusion about the effect of eggs on import In-Reply-To: <17869.1030.405845.96720@montanaro.dyndns.org> Message-ID: <5.1.1.6.0.20070209190001.029fdb10@sparrow.telecommunity.com> At 05:30 PM 2/9/2007 -0600, skip at pobox.com wrote: >(I'm sending this to distutils because I have this vague notion that the >problem might have something to do with setuptools or eggs on sys.path.) Their mere presence on sys.path doesn't do anything, if you're not importing pkg_resources or importing anything from inside the eggs. However, matplotlib imports pkg_resources and is a (broken) namespace package. >We have a weird issue trying to import matplotlib at work. I investigated a little, and here's what's happening. Matplotlib declares 'matplotlib.toolkits' as a namespace package, but you can't make a child package a namespace package without the parent being one too. A namespace package combines all packages on sys.path with the same name into a single package. So here's what's happening. When setuptools is on sys.path, then importing matplotlib causes it to declare its namespace, which merges all copies of matplotlib on sys.path into a single super-package -- and invokes all the __init__.py's. When setuptools is not on sys.path, the attempt to declare the namespace fails (it's wrapped in a try/except in matplotlib/__init__.py), so nothing else happens. There are, I think, two problems here. One, is that setuptools shouldn't be executing multiple __init__.py's for a package, but for backward compatibility reasons this isn't being dropped until 0.7a1. Two, is that setuptools currently allows you to declare a child namespace package (like 'matplotlib.toolkits') without explicitly declaring the parent to be a namespace package. So, sometimes people declare a subpackage without realizing the parent will also have to be treated as a namespace package. As a result, they end up thinking that they can include initialization code in the parent package, when in fact there are many circumstances where it simply won't work. In this case, matplotlib/__init__.py contains executable code, which is a no-no for a namespace package. My suggestion would be that matplotlib use a matplotlib_toolkits namespace package, rather than attempting to keep matplotlib.toolkits, since it appears they are relying on a substantial amount of code living in the __init__.py. In short, the problem is a matplotlib bug, but in fairness it's probably due to the sketchy documentation surrounding the proper care and feeding of namespace packages, coupled with the implicit declaration of namespace packages. All of the problems are explained in the setuptools documentation, but unfortunately that's not the same as anybody being able to figure out that the problems will apply to THEM. ;) From skip at pobox.com Sat Feb 10 14:20:49 2007 From: skip at pobox.com (skip at pobox.com) Date: Sat, 10 Feb 2007 07:20:49 -0600 Subject: [Distutils] Confusion about the effect of eggs on import In-Reply-To: <5.1.1.6.0.20070209190001.029fdb10@sparrow.telecommunity.com> References: <17869.1030.405845.96720@montanaro.dyndns.org> <5.1.1.6.0.20070209190001.029fdb10@sparrow.telecommunity.com> Message-ID: <17869.50865.942626.755175@montanaro.dyndns.org> >> (I'm sending this to distutils because I have this vague notion that >> the problem might have something to do with setuptools or eggs on >> sys.path.) Phillip> Their mere presence on sys.path doesn't do anything, if you're Phillip> not importing pkg_resources or importing anything from inside Phillip> the eggs. Phillip> However, matplotlib imports pkg_resources and is a (broken) Phillip> namespace package. ... much useful feedback elided ... Phillip, Thanks for the response. I'm 100% certain I would never have figured that out on my own. I passed your note along to John Hunter (matplotlib author), who I happen to work with. I suspect he'll be able to work things out fairly quickly. Skip From jim at zope.com Sat Feb 10 18:54:54 2007 From: jim at zope.com (Jim Fulton) Date: Sat, 10 Feb 2007 12:54:54 -0500 Subject: [Distutils] Building an egg from Subversion using zc.buildout In-Reply-To: <5.1.1.6.0.20070209184841.03d75830@sparrow.telecommunity.com> References: <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <45BE6CAF.3010206@gmx.net> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <1163956881.5763.21.camel@Jane> <5.1.1.6.0.20061119134913.01f3d088@sparrow.telecommunity.com> <45BD22C2.7030403@colorstudy.com> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <5.1.1.6.0.20070128202027.04461008@sparrow.telecommunity.com> <9D0FD265-B323-42C3-80A1-85638A067C9A@zope.com> <45BE6CAF.3010206@gmx.net> <65788D22-5FA7-4AC1-A6FA-08FDA4D173FE@zope.com> <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <5.1.1.6.0.20070129180219.045646e0@sparrow.telecommunity.com> <5.1.1.6.0.20070209184841.03d75830@sparrow.telecommunity.com> Message-ID: On Feb 9, 2007, at 6:49 PM, Phillip J. Eby wrote: > At 05:26 PM 2/9/2007 -0500, Jim Fulton wrote: >> > ValueError: ("Couldn't find", Requirement.parse >> > ('archetypes.kss==dev')) >> >> I finally found some time to look into this. >> >> Buildout uses setuptools APIs. It uses a PackageIndex to download >> the distribution and this actually works. It then uses an >> environment best_match method to get a distribution object and >> best_match returns None. If I remove the ==dev from the requirement, >> then best_match is able to find the distribution. This happens >> because apparently no distributions match a requirement for the "dev" >> version. Should I view this as a bug in setuptools? > > The normal usage is "==dev,>=realver" where 'realver' is the real > version you want. Is using just ==dev is correct usage? It works with easy_install. I think the user's desire is to get the most recent version. It's not clear to me that they'd know what other revision to specify. >> Or does buildout >> need to special case the version "dev"? > > I don't think so. If ==dev is considered valid and if the environment best_match method isn't going to consider it a match against installed distributions, then I'll need to special case it or use some other API. People using buildout are generally going to expect things that work with easy_install to work with buildout. (Note: Buildout uses PackageIndex to locate and fetch distributions. It then installs distributions itself. if the distribution is already an egg, then this is straightforward. Otherwise, buildout invokes easy_install to do the installation. It can't get a return value from easy_install, so it uses an Environment to find the egg that easy_install created.) Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From pje at telecommunity.com Sat Feb 10 20:16:06 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 10 Feb 2007 14:16:06 -0500 Subject: [Distutils] Building an egg from Subversion using zc.buildout In-Reply-To: References: <5.1.1.6.0.20070209184841.03d75830@sparrow.telecommunity.com> <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <45BE6CAF.3010206@gmx.net> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <1163956881.5763.21.camel@Jane> <5.1.1.6.0.20061119134913.01f3d088@sparrow.telecommunity.com> <45BD22C2.7030403@colorstudy.com> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <5.1.1.6.0.20070128202027.04461008@sparrow.telecommunity.com> <9D0FD265-B323-42C3-80A1-85638A067C9A@zope.com> <45BE6CAF.3010206@gmx.net> <65788D22-5FA7-4AC1-A6FA-08FDA4D173FE@zope.com> <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <5.1.1.6.0.20070129180219.045646e0@sparrow.telecommunity.com> <5.1.1.6.0.20070209184841.03d75830@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070210141348.00a04748@sparrow.telecommunity.com> At 12:54 PM 2/10/2007 -0500, Jim Fulton wrote: >On Feb 9, 2007, at 6:49 PM, Phillip J. Eby wrote: > >>At 05:26 PM 2/9/2007 -0500, Jim Fulton wrote: >>> > ValueError: ("Couldn't find", Requirement.parse >>> > ('archetypes.kss==dev')) >>> >>>I finally found some time to look into this. >>> >>>Buildout uses setuptools APIs. It uses a PackageIndex to download >>>the distribution and this actually works. It then uses an >>>environment best_match method to get a distribution object and >>>best_match returns None. If I remove the ==dev from the requirement, >>>then best_match is able to find the distribution. This happens >>>because apparently no distributions match a requirement for the "dev" >>>version. Should I view this as a bug in setuptools? >> >>The normal usage is "==dev,>=realver" where 'realver' is the real >>version you want. > >Is using just ==dev is correct usage? It works with easy_install. Yes, if you put it on the command line. No, if it's a dependency declared in an egg; it'll have the same problem as you're having with buildout. >I think the user's desire is to get the most recent version. It's >not clear to me that they'd know what other revision to specify. Actually, the reason to use an ==dev (or other similar tag) in a dependency is to allow it to pick up a package that's *only* available via SVN. >(Note: Buildout uses PackageIndex to locate and fetch distributions. >It then installs distributions itself. if the distribution is >already an egg, then this is straightforward. Otherwise, buildout >invokes easy_install to do the installation. It can't get a return >value from easy_install, so it uses an Environment to find the egg >that easy_install created.) Note that easy_install can be told to --record what files it installs, so you could build a WorkingSet or Environment from that list. From jim at zope.com Sat Feb 10 20:20:33 2007 From: jim at zope.com (Jim Fulton) Date: Sat, 10 Feb 2007 14:20:33 -0500 Subject: [Distutils] Building an egg from Subversion using zc.buildout In-Reply-To: <5.1.1.6.0.20070210141348.00a04748@sparrow.telecommunity.com> References: <5.1.1.6.0.20070209184841.03d75830@sparrow.telecommunity.com> <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <45BE6CAF.3010206@gmx.net> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <1163956881.5763.21.camel@Jane> <5.1.1.6.0.20061119134913.01f3d088@sparrow.telecommunity.com> <45BD22C2.7030403@colorstudy.com> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <5.1.1.6.0.20070128202027.04461008@sparrow.telecommunity.com> <9D0FD265-B323-42C3-80A1-85638A067C9A@zope.com> <45BE6CAF.3010206@gmx.net> <65788D22-5FA7-4AC1-A6FA-08FDA4D173FE@zope.com> <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <5.1.1.6.0.20070129180219.045646e0@sparrow.telecommunity.com> <5.1.1.6.0.20070209184841.03d75830@sparrow.telecommunity.com> < 5.1.1.6.0.20070210141348.00a04748@sparrow.telecommunity.com> Message-ID: <203A5F22-38C8-4318-8B1F-B09F754332C9@zope.com> On Feb 10, 2007, at 2:16 PM, Phillip J. Eby wrote: > At 12:54 PM 2/10/2007 -0500, Jim Fulton wrote: ... >> (Note: Buildout uses PackageIndex to locate and fetch distributions. >> It then installs distributions itself. if the distribution is >> already an egg, then this is straightforward. Otherwise, buildout >> invokes easy_install to do the installation. It can't get a return >> value from easy_install, so it uses an Environment to find the egg >> that easy_install created.) > > Note that easy_install can be told to --record what files it > installs, so you could build a WorkingSet or Environment from that > list. I looked at that a while back. I don't remember why I didn't take that approach. At this point, I think it would be easier to special-case ==dev. :) Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From mcfletch at vrplumber.com Sun Feb 11 00:26:45 2007 From: mcfletch at vrplumber.com (Mike C. Fletcher) Date: Sat, 10 Feb 2007 18:26:45 -0500 Subject: [Distutils] Alternates in SetupTools dependencies? Message-ID: <45CE54B5.7090605@vrplumber.com> I'm starting to package OpenGLContext as a series of .eggs but I seem to be missing something key: What's the syntax for declaring "I require numpy or Numeric" I could declare them as "extras", but it's not that they are individually extras, one or the other is *required* for the PyVRML97 library. Similar problem with "PyGame or PyGame-ctypes" they are work-alikes (or closely enough to use either) and we can work with whichever one is installed (preferably choosing the items in order to attempt installation). I don't see any documentation for this (extremely common) case on the setuptools page. Is it just some obvious thing that I'm missing? Thanks all, Mike -- ________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com From mcfletch at vrplumber.com Sun Feb 11 00:40:42 2007 From: mcfletch at vrplumber.com (Mike C. Fletcher) Date: Sat, 10 Feb 2007 18:40:42 -0500 Subject: [Distutils] Best practices for accelerator modules? Message-ID: <45CE57FA.4060601@vrplumber.com> In a number of my OpenGLContext projects I have "accelerator" modules, small, optional C extensions which are optional, and thus likely "extra_requires" candidates. I gather the best practice for these things is to create a separate .egg with some name like "PyVRML97-frustum-accelerators-numpy" with some weird internal package name that won't create a new conflict in the global namespace? Is that a correct understanding? (The situation is actually somewhat more complex in that some of the accelerators are actually numpy/Numeric version dependent (that is, they can only run against the version of numpy/Numeric against which they were built). I'm planning on handling that by having the accelerator module examine the Numeric/numpy version and add that explicit version to its dependencies so that only that version will satisfy the dependency.) i.e. I'm thinking of an egg named PyVRML97-frustum-accelerators-numpy with a package/module named 'PyVRML97-frustum-accelerators-numpy' so that the pollution of the top-level namespace will be minimised with the package's entry points being used to resolve the references without regard to their "unimportability". And that rather convoluted plan makes me think "is this really recommended practice"? Mike -- ________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com From pje at telecommunity.com Sun Feb 11 01:39:05 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 10 Feb 2007 19:39:05 -0500 Subject: [Distutils] Best practices for accelerator modules? In-Reply-To: <45CE57FA.4060601@vrplumber.com> Message-ID: <5.1.1.6.0.20070210193538.0394da00@sparrow.telecommunity.com> At 06:40 PM 2/10/2007 -0500, Mike C. Fletcher wrote: >In a number of my OpenGLContext projects I have "accelerator" modules, >small, optional C extensions which are optional, and thus likely >"extra_requires" candidates. I gather the best practice for these >things is to create a separate .egg with some name like >"PyVRML97-frustum-accelerators-numpy" with some weird internal package >name that won't create a new conflict in the global namespace? Is that >a correct understanding? > >(The situation is actually somewhat more complex in that some of the >accelerators are actually numpy/Numeric version dependent (that is, they >can only run against the version of numpy/Numeric against which they >were built). I'm planning on handling that by having the accelerator >module examine the Numeric/numpy version and add that explicit version >to its dependencies so that only that version will satisfy the dependency.) > >i.e. I'm thinking of an egg named PyVRML97-frustum-accelerators-numpy >with a package/module named 'PyVRML97-frustum-accelerators-numpy' so >that the pollution of the top-level namespace will be minimised with the >package's entry points being used to resolve the references without >regard to their "unimportability". > >And that rather convoluted plan makes me think "is this really >recommended practice"? Mostly, there isn't a good practice for this. See this thread: http://mail.python.org/pipermail/distutils-sig/2007-February/007192.html for some recent discussion on one possible approach that doesn't actually exist yet. :( Your concept of using entry points is rather interesting. I hadn't thought about that as a way to ensure matching versions, but it could work. Still, as you say, it's a convoluted plan. From pje at telecommunity.com Sun Feb 11 01:42:45 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 10 Feb 2007 19:42:45 -0500 Subject: [Distutils] Alternates in SetupTools dependencies? In-Reply-To: <45CE54B5.7090605@vrplumber.com> Message-ID: <5.1.1.6.0.20070210193945.04320ea8@sparrow.telecommunity.com> At 06:26 PM 2/10/2007 -0500, Mike C. Fletcher wrote: >I'm starting to package OpenGLContext as a series of .eggs but I seem to >be missing something key: > > What's the syntax for declaring "I require numpy or Numeric" > >I could declare them as "extras", but it's not that they are >individually extras, one or the other is *required* for the PyVRML97 >library. Similar problem with "PyGame or PyGame-ctypes" they are >work-alikes (or closely enough to use either) and we can work with >whichever one is installed (preferably choosing the items in order to >attempt installation). > >I don't see any documentation for this (extremely common) case on the >setuptools page. Is it just some obvious thing that I'm missing? No, it's a sometimes-requested feature that can't reasonably be implemented in setuptools 0.6. See this recent thread for more details: http://mail.python.org/pipermail/distutils-sig/2007-February/007253.html Interestingly, the numpy vs. Numeric (and/or numarray) use case is the specific pair of alternatives that's most often requested by people asking for this feature. :) From pje at telecommunity.com Sun Feb 11 01:44:04 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 10 Feb 2007 19:44:04 -0500 Subject: [Distutils] Building an egg from Subversion using zc.buildout In-Reply-To: <203A5F22-38C8-4318-8B1F-B09F754332C9@zope.com> References: <5.1.1.6.0.20070210141348.00a04748@sparrow.telecommunity.com> <5.1.1.6.0.20070209184841.03d75830@sparrow.telecommunity.com> <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <45BE6CAF.3010206@gmx.net> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <1163956881.5763.21.camel@Jane> <5.1.1.6.0.20061119134913.01f3d088@sparrow.telecommunity.com> <45BD22C2.7030403@colorstudy.com> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <5.1.1.6.0.20070128202027.04461008@sparrow.telecommunity.com> <9D0FD265-B323-42C3-80A1-85638A067C9A@zope.com> <45BE6CAF.3010206@gmx.net> <65788D22-5FA7-4AC1-A6FA-08FDA4D173FE@zope.com> <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <5.1.1.6.0.20070129180219.045646e0@sparrow.telecommunity.com> <5.1.1.6.0.20070209184841.03d75830@sparrow.telecommunity.com> < 5.1.1.6.0.20070210141348.00a04748@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070210194251.0382f008@sparrow.telecommunity.com> At 02:20 PM 2/10/2007 -0500, Jim Fulton wrote: >On Feb 10, 2007, at 2:16 PM, Phillip J. Eby wrote: > >>At 12:54 PM 2/10/2007 -0500, Jim Fulton wrote: >... > >>>(Note: Buildout uses PackageIndex to locate and fetch distributions. >>>It then installs distributions itself. if the distribution is >>>already an egg, then this is straightforward. Otherwise, buildout >>>invokes easy_install to do the installation. It can't get a return >>>value from easy_install, so it uses an Environment to find the egg >>>that easy_install created.) >> >>Note that easy_install can be told to --record what files it >>installs, so you could build a WorkingSet or Environment from that >>list. > >I looked at that a while back. I don't remember why I didn't take >that approach. > >At this point, I think it would be easier to special-case ==dev. :) Ah, but setuptools has a ==dev06, and I think other people have used other tags to denote various SVN branches. The name used to denote the version is incidental, IOW, so that way lies N special cases, all different. :) From sjmachin at lexicon.net Sun Feb 11 02:59:11 2007 From: sjmachin at lexicon.net (John Machin) Date: Sun, 11 Feb 2007 12:59:11 +1100 Subject: [Distutils] ez_setup.py -U setuptools does TWO downloads, TWO installs Message-ID: <45CE786F.2040400@lexicon.net> Is this usual? The ez_setup.py is fresh off the setuptools website. Environment: Python 2.5; Windows XP Pro SP2 Console log follows. Cheers, John ====================================================================== C:\sw\setuptools>\python25\python ez_setup.py -U setuptools Downloading http://cheeseshop.python.org/packages/2.5/s/setuptools/setuptools-0.6c5-py2.5.egg Searching for setuptools Reading http://www.python.org/pypi/setuptools/ Reading http://cheeseshop.python.org/pypi/setuptools Reading http://www.python.org/pypi/setuptools/0.6c5 Best match: setuptools 0.6c5 Downloading http://cheeseshop.python.org/packages/2.5/s/setuptools/setuptools-0.6c5-py2.5.egg#md5=a8d3f61494ccaa8714dfed37bccd3d5d Processing setuptools-0.6c5-py2.5.egg Moving setuptools-0.6c5-py2.5.egg to c:\python25\lib\site-packages Removing setuptools 0.6c3 from easy-install.pth file Adding setuptools 0.6c5 to easy-install.pth file Installing easy_install-script.py script to C:\python25\Scripts Installing easy_install.exe script to C:\python25\Scripts Installing easy_install-2.5-script.py script to C:\python25\Scripts Installing easy_install-2.5.exe script to C:\python25\Scripts Installed c:\python25\lib\site-packages\setuptools-0.6c5-py2.5.egg Processing dependencies for setuptools Processing setuptools-0.6c5-py2.5.egg Removing c:\python25\lib\site-packages\setuptools-0.6c5-py2.5.egg Copying setuptools-0.6c5-py2.5.egg to c:\python25\lib\site-packages setuptools 0.6c5 is already the active version in easy-install.pth Installing easy_install-script.py script to C:\python25\Scripts Installing easy_install.exe script to C:\python25\Scripts Installing easy_install-2.5-script.py script to C:\python25\Scripts Installing easy_install-2.5.exe script to C:\python25\Scripts Installed c:\python25\lib\site-packages\setuptools-0.6c5-py2.5.egg Processing dependencies for setuptools==0.6c5 C:\sw\setuptools> From pje at telecommunity.com Sun Feb 11 05:41:19 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 10 Feb 2007 23:41:19 -0500 Subject: [Distutils] ez_setup.py -U setuptools does TWO downloads, TWO installs In-Reply-To: <45CE786F.2040400@lexicon.net> Message-ID: <5.1.1.6.0.20070210234021.02969e20@sparrow.telecommunity.com> At 12:59 PM 2/11/2007 +1100, John Machin wrote: >Is this usual? Yes. ez_setup.py is a front-end to easy_install, that happens to also download and install the latest version of setuptools. If you want to just install the latest version of setuptools, the '-U setuptools' is redundant, unless you already have setuptools installed and want to update it. From sjmachin at lexicon.net Sun Feb 11 06:02:56 2007 From: sjmachin at lexicon.net (John Machin) Date: Sun, 11 Feb 2007 16:02:56 +1100 Subject: [Distutils] ez_setup.py -U setuptools does TWO downloads, TWO installs In-Reply-To: <5.1.1.6.0.20070210234021.02969e20@sparrow.telecommunity.com> References: <5.1.1.6.0.20070210234021.02969e20@sparrow.telecommunity.com> Message-ID: <45CEA380.5080304@lexicon.net> On 11/02/2007 3:41 PM, Phillip J. Eby wrote: > At 12:59 PM 2/11/2007 +1100, John Machin wrote: >> Is this usual? > > Yes. ez_setup.py is a front-end to easy_install, that happens to also > download and install the latest version of setuptools. If you want to > just install the latest version of setuptools, the '-U setuptools' is > redundant, unless you already have setuptools installed and want to > update it. Can we try again, please? Refer to the console log that I included. It *was* upgrading: """Removing setuptools 0.6c3 from easy-install.pth file""" From pje at telecommunity.com Sun Feb 11 06:46:36 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 11 Feb 2007 00:46:36 -0500 Subject: [Distutils] ez_setup.py -U setuptools does TWO downloads, TWO installs In-Reply-To: <45CEA380.5080304@lexicon.net> References: <5.1.1.6.0.20070210234021.02969e20@sparrow.telecommunity.com> <5.1.1.6.0.20070210234021.02969e20@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070211004444.02969410@sparrow.telecommunity.com> At 04:02 PM 2/11/2007 +1100, John Machin wrote: >On 11/02/2007 3:41 PM, Phillip J. Eby wrote: >>At 12:59 PM 2/11/2007 +1100, John Machin wrote: >>>Is this usual? >>Yes. ez_setup.py is a front-end to easy_install, that happens to also >>download and install the latest version of setuptools. If you want to >>just install the latest version of setuptools, the '-U setuptools' is >>redundant, unless you already have setuptools installed and want to update it. > >Can we try again, please? Refer to the console log that I included. It >*was* upgrading: > >"""Removing setuptools 0.6c3 from easy-install.pth file""" Yes, but you were upgrading using a NEW version of ez_setup.py. If you'd used the already-present ez_setup.py (from when you installed 0.6c3), OR used the new ez_setup.py without the -U setuptools, you'd have only gotten one install/download. Running ez_setup.py simply attempts to download and install the version that it wants. Then it runs easy_install with any supplied command-line arguments. Thus, passing "-U setuptools" to a *new* ez_setup.py is redundant. From sjmachin at lexicon.net Sun Feb 11 07:38:31 2007 From: sjmachin at lexicon.net (John Machin) Date: Sun, 11 Feb 2007 17:38:31 +1100 Subject: [Distutils] ez_setup.py -U setuptools does TWO downloads, TWO installs In-Reply-To: <5.1.1.6.0.20070211004444.02969410@sparrow.telecommunity.com> References: <5.1.1.6.0.20070210234021.02969e20@sparrow.telecommunity.com> <5.1.1.6.0.20070210234021.02969e20@sparrow.telecommunity.com> <5.1.1.6.0.20070211004444.02969410@sparrow.telecommunity.com> Message-ID: <45CEB9E7.7090905@lexicon.net> On 11/02/2007 4:46 PM, Phillip J. Eby wrote: > At 04:02 PM 2/11/2007 +1100, John Machin wrote: >> On 11/02/2007 3:41 PM, Phillip J. Eby wrote: >>> At 12:59 PM 2/11/2007 +1100, John Machin wrote: >>>> Is this usual? >>> Yes. ez_setup.py is a front-end to easy_install, that happens to >>> also download and install the latest version of setuptools. If you >>> want to just install the latest version of setuptools, the '-U >>> setuptools' is redundant, unless you already have setuptools >>> installed and want to update it. >> >> Can we try again, please? Refer to the console log that I included. It >> *was* upgrading: >> >> """Removing setuptools 0.6c3 from easy-install.pth file""" > > Yes, but you were upgrading using a NEW version of ez_setup.py. If > you'd used the already-present ez_setup.py (from when you installed > 0.6c3), OR used the new ez_setup.py without the -U setuptools, you'd > have only gotten one install/download. > > Running ez_setup.py simply attempts to download and install the version > that it wants. Then it runs easy_install with any supplied command-line > arguments. Thus, passing "-U setuptools" to a *new* ez_setup.py is > redundant. Perhaps so, but my recollection is that didn't work that way last time: the very first ez_setup.py chucked a wobbly when I tried to use it to update setuptools :-) From jim at zope.com Sun Feb 11 19:01:41 2007 From: jim at zope.com (Jim Fulton) Date: Sun, 11 Feb 2007 13:01:41 -0500 Subject: [Distutils] Building an egg from Subversion using zc.buildout In-Reply-To: <5.1.1.6.0.20070210194251.0382f008@sparrow.telecommunity.com> References: <5.1.1.6.0.20070210141348.00a04748@sparrow.telecommunity.com> <5.1.1.6.0.20070209184841.03d75830@sparrow.telecommunity.com> <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <45BE6CAF.3010206@gmx.net> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <1163956881.5763.21.camel@Jane> <5.1.1.6.0.20061119134913.01f3d088@sparrow.telecommunity.com> <45BD22C2.7030403@colorstudy.com> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <5.1.1.6.0.20070128202027.04461008@sparrow.telecommunity.com> <9D0FD265-B323-42C3-80A1-85638A067C9A@zope.com> <45BE6CAF.3010206@gmx.net> <65788D22-5FA7-4AC1-A6FA-08FDA4D173FE@zope.com> <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <5.1.1.6.0.20070129180219.045646e0@sparrow.telecommunity.com> < 5.1.1.6.0.20070209184841.03d75830@sparrow.telecommunity.com> < 5.1.1.6.0.20070210141348.00a04748@sparrow.telecommunity.com> <5.1.1.6.0.20070210194251.0382f008@sparrow.telecommunity.com> Message-ID: <856F4418-78DF-496D-B858-4BC78D7BDA60@zope.com> On Feb 10, 2007, at 7:44 PM, Phillip J. Eby wrote: > At 02:20 PM 2/10/2007 -0500, Jim Fulton wrote: > >> On Feb 10, 2007, at 2:16 PM, Phillip J. Eby wrote: >> >>> At 12:54 PM 2/10/2007 -0500, Jim Fulton wrote: >> ... >> >>>> (Note: Buildout uses PackageIndex to locate and fetch >>>> distributions. >>>> It then installs distributions itself. if the distribution is >>>> already an egg, then this is straightforward. Otherwise, buildout >>>> invokes easy_install to do the installation. It can't get a return >>>> value from easy_install, so it uses an Environment to find the egg >>>> that easy_install created.) >>> >>> Note that easy_install can be told to --record what files it >>> installs, so you could build a WorkingSet or Environment from that >>> list. >> >> I looked at that a while back. I don't remember why I didn't take >> that approach. >> >> At this point, I think it would be easier to special-case ==dev. :) > > Ah, but setuptools has a ==dev06, and I think other people have > used other tags to denote various SVN branches. The name used to > denote the version is incidental, IOW, so that way lies N special > cases, all different. :) I really don't understand this feature. So far I have had to resort to experimentation to get a loose grip on it. I think this needs to be much better documented (or dropped). Unfortunately, I think you are the only one who can document it because I think that you are the only one who truly understands it. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From thomas at thomas-lotze.de Mon Feb 12 00:15:39 2007 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Mon, 12 Feb 2007 00:15:39 +0100 Subject: [Distutils] Setuptools: omit namespace package directories? References: <5.1.1.6.0.20070209152759.05074008@sparrow.telecommunity.com> <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> <5.1.1.6.0.20070209152759.05074008@sparrow.telecommunity.com> <48701E33-297D-496D-8398-C7AABBD6E180@zope.com> <5.1.1.6.0.20070209161150.03d72140@sparrow.telecommunity.com> Message-ID: Am Fri, 09 Feb 2007 23:01:20 +0100 schrieb Thomas Lotze: > But - what prevents the > __init__.py file in this directory from setting up any intermediate > namespaces in sys.modules and causing the real package to be looked for in > some convenient place that avoids directory jumping in the source tree? Nothing does. A proof-of-concept example can be found at . The source tree contains foo/__init__.py and baz/__init__.py, the former setting up some stuff in sys.modules using pkg_resources._handle_ns, the latter just printing out __file__ for proof that it has been imported. You can use this project both in develop mode and as an egg; I installed it using easy_install. Right now, a lot of os.path manipulation on __file__ is being done by foo/__init__.py; if this was replaced by a call to some appropriate pkg_resources function, the package_dir information could be used instead in a clean way. Also, the modules stuff gets set up all at once instead of recursively as declare_namespace does. Thus the intermediate namespace package foo.bar doesn't need an __init__.py, and it fact it doesn't even need a directory in the source tree. I can't see anything bad about that right now, but maybe a missing __init__.py does cause trouble somewhere. -- Viele Gr??e, Thomas From pje at telecommunity.com Mon Feb 12 01:05:44 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 11 Feb 2007 19:05:44 -0500 Subject: [Distutils] Building an egg from Subversion using zc.buildout In-Reply-To: <856F4418-78DF-496D-B858-4BC78D7BDA60@zope.com> References: <5.1.1.6.0.20070210194251.0382f008@sparrow.telecommunity.com> <5.1.1.6.0.20070210141348.00a04748@sparrow.telecommunity.com> <5.1.1.6.0.20070209184841.03d75830@sparrow.telecommunity.com> <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <45BE6CAF.3010206@gmx.net> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <1163956881.5763.21.camel@Jane> <5.1.1.6.0.20061119134913.01f3d088@sparrow.telecommunity.com> <45BD22C2.7030403@colorstudy.com> <5.1.1.6.0.20070128174203.042de500@sparrow.telecommunity.com> <5.1.1.6.0.20070128202027.04461008@sparrow.telecommunity.com> <9D0FD265-B323-42C3-80A1-85638A067C9A@zope.com> <45BE6CAF.3010206@gmx.net> <65788D22-5FA7-4AC1-A6FA-08FDA4D173FE@zope.com> <5.1.1.6.0.20070129172214.04469590@sparrow.telecommunity.com> <5.1.1.6.0.20070129180219.045646e0@sparrow.telecommunity.com> < 5.1.1.6.0.20070209184841.03d75830@sparrow.telecommunity.com> < 5.1.1.6.0.20070210141348.00a04748@sparrow.telecommunity.com> <5.1.1.6.0.20070210194251.0382f008@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070211190004.04dd7350@sparrow.telecommunity.com> At 01:01 PM 2/11/2007 -0500, Jim Fulton wrote: >On Feb 10, 2007, at 7:44 PM, Phillip J. Eby wrote: >>At 02:20 PM 2/10/2007 -0500, Jim Fulton wrote: >>>On Feb 10, 2007, at 2:16 PM, Phillip J. Eby wrote: >>>>At 12:54 PM 2/10/2007 -0500, Jim Fulton wrote: >>>... >>> >>>>>(Note: Buildout uses PackageIndex to locate and fetch >>>>>distributions. >>>>>It then installs distributions itself. if the distribution is >>>>>already an egg, then this is straightforward. Otherwise, buildout >>>>>invokes easy_install to do the installation. It can't get a return >>>>>value from easy_install, so it uses an Environment to find the egg >>>>>that easy_install created.) >>>> >>>>Note that easy_install can be told to --record what files it >>>>installs, so you could build a WorkingSet or Environment from that >>>>list. >>> >>>I looked at that a while back. I don't remember why I didn't take >>>that approach. >>> >>>At this point, I think it would be easier to special-case ==dev. :) >> >>Ah, but setuptools has a ==dev06, and I think other people have >>used other tags to denote various SVN branches. The name used to >>denote the version is incidental, IOW, so that way lies N special >>cases, all different. :) > >I really don't understand this feature. That's because there is no feature, at least not the way you're thinking about it. Marking a link with "#egg=projname-version" simply indicates that easy_install should consider that link to be for that project and version, regardless of what the link appears to be pointing to. Using a version of "dev" to denote a subversion repository link is therefore merely a *convention*, not a feature; it has NO special handling in easy_install. Meanwhile, the use of two versions in a requirement string (so that a subversion link tagged "dev" would be accepted for download, and the installed version would appear to meet the requirement) was a hack invented by a user, not me. :) And that hack, along with all the other behavior being seen, is merely an emergent property of the fact that you can label links, and easy_install's normal link-seeking behavior. From pje at telecommunity.com Mon Feb 12 01:16:39 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 11 Feb 2007 19:16:39 -0500 Subject: [Distutils] Setuptools: omit namespace package directories? In-Reply-To: References: <5.1.1.6.0.20070209152759.05074008@sparrow.telecommunity.com> <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> <5.1.1.6.0.20070209152759.05074008@sparrow.telecommunity.com> <48701E33-297D-496D-8398-C7AABBD6E180@zope.com> <5.1.1.6.0.20070209161150.03d72140@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070211190757.027e5910@sparrow.telecommunity.com> At 12:15 AM 2/12/2007 +0100, Thomas Lotze wrote: >Am Fri, 09 Feb 2007 23:01:20 +0100 schrieb Thomas Lotze: > > > But - what prevents the > > __init__.py file in this directory from setting up any intermediate > > namespaces in sys.modules and causing the real package to be looked for in > > some convenient place that avoids directory jumping in the source tree? > >Nothing does. A proof-of-concept example can be found at >. The source tree >contains foo/__init__.py and baz/__init__.py, the former setting up some >stuff in sys.modules using pkg_resources._handle_ns, the latter just >printing out __file__ for proof that it has been imported. You can use >this project both in develop mode and as an egg; I installed it using >easy_install. > >Right now, a lot of os.path manipulation on __file__ is being done by >foo/__init__.py; if this was replaced by a call to some appropriate >pkg_resources function, the package_dir information could be used instead >in a clean way. > >Also, the modules stuff gets set up all at once instead of recursively as >declare_namespace does. Thus the intermediate namespace package foo.bar >doesn't need an __init__.py, and it fact it doesn't even need a directory >in the source tree. I can't see anything bad about that right now, but >maybe a missing __init__.py does cause trouble somewhere. It appears your goals are somewhat... confused. Namespace packages, as I've already said, do not always even have __init__.py files existing, so there is no place to put your example __init__.py that guarantees it will be executed. Remember that by definition a namespace package has no single "owner" project. It is potentially shared across multiple projects. When installed as an egg, each __init__.py is executed, true. But when a project is NOT installed via egg, but rather by a system packager, the __init__.py doesn't exist, and so cannot be executed. Thus, the normal case for a namespace package is to have a number of __init__.py files that is either 0 or >1. Having exactly one __init__.py present for some namespace package is an abnormality. Therefore, your prototype code is horribly broken, as it will corrupt pkg_resources' internal data structures when it is run more than once -- as it would have to be in order to support *each* project containing code under that namespace package. (And I won't even get into the pointlessness of wrapping module-level code in an import lock.) From thomas at thomas-lotze.de Mon Feb 12 08:59:02 2007 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Mon, 12 Feb 2007 08:59:02 +0100 Subject: [Distutils] Setuptools: omit namespace package directories? References: <5.1.1.6.0.20070209152759.05074008@sparrow.telecommunity.com> <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> <5.1.1.6.0.20070209152759.05074008@sparrow.telecommunity.com> <48701E33-297D-496D-8398-C7AABBD6E180@zope.com> <5.1.1.6.0.20070209161150.03d72140@sparrow.telecommunity.com> <5.1.1.6.0.20070211190757.027e5910@sparrow.telecommunity.com> Message-ID: Phillip J. Eby wrote: > It appears your goals are somewhat... confused. I don't think so. I'm sure I still have to learn about setuptools but I think I know quite well what I want to achieve. > Remember that by definition a namespace package has no single "owner" > project. It is potentially shared across multiple projects. When > installed as an egg, each __init__.py is executed, true. But when a > project is NOT installed via egg, but rather by a system packager, the > __init__.py doesn't exist, and so cannot be executed. If we have to take into account distributions of a project as source code bereft of its top-level namespace's __init__.py being expected to just work, I give up. I assumed to always work with either intact source or built eggs, the latter with or without their namespaces' __init__.py files being used and with or without being installed by our egg tools. The built egg has all the subdirectories needed and doesn't look different from other eggs in that respect. It doesn't rely on any namespace cleverness being performed by __init__.py, other than what setuptools already do for namespace packages. If the namespace mangling performed in that file was integrated with the setuptools' declare_namespace, there wouldn't need to be anything special about the __init__.py of a project with remapped package directories, and it wouldn't hurt if it wasn't executed anyway. In fact, the namespace package and all namespace subpackages could be given a standard __init__.py by setup bdist_egg. The package remapping is only needed when running from source, and I don't see a reason why the __init__.py shouldn't exist and be executed in that case. > Therefore, your prototype code is horribly broken, as it will corrupt > pkg_resources' internal data structures when it is run more than once -- I don't see why it couldn't be made to behave properly. As I said, I'm still learning about setuptools, and I don't claim to be the most apt person to integrate any new functionality at this point. > (And I won't even get into the > pointlessness of wrapping module-level code in an import lock.) I'd appreciate it if you did, as I haven't yet found a difference between using the lock in modules and using it in pkg_resources.declare_namespace which is being called by modules. -- Thomas From jim at zope.com Mon Feb 12 17:09:11 2007 From: jim at zope.com (Jim Fulton) Date: Mon, 12 Feb 2007 11:09:11 -0500 Subject: [Distutils] Better error reporting Message-ID: <0548D213-4CFF-4672-AED7-94B151F12707@zope.com> There are places where setuptools exceptions sometimes lack important information. For example, in PackageIndex.open_url, there is: def open_url(self, url): if url.startswith('file:'): return local_open(url) try: return open_with_auth(url) except urllib2.HTTPError, v: return v except urllib2.URLError, v: raise DistutilsError("Download error: %s" % v.reason) Getting a download error without knowing what the error is for is rather frustrating. :) IMO, this should be: def open_url(self, url): if url.startswith('file:'): return local_open(url) try: return open_with_auth(url) except urllib2.HTTPError, v: return v except urllib2.URLError, v: raise DistutilsError("Download error for %r: %s" % (url, v.reason)) Phillip, would you like me to go ahead and make minor changes like this? Or would you rather make them yourself? Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From pje at telecommunity.com Mon Feb 12 17:38:13 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 12 Feb 2007 11:38:13 -0500 Subject: [Distutils] Setuptools: omit namespace package directories? In-Reply-To: References: <5.1.1.6.0.20070209152759.05074008@sparrow.telecommunity.com> <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> <5.1.1.6.0.20070209152759.05074008@sparrow.telecommunity.com> <48701E33-297D-496D-8398-C7AABBD6E180@zope.com> <5.1.1.6.0.20070209161150.03d72140@sparrow.telecommunity.com> <5.1.1.6.0.20070211190757.027e5910@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070212111624.05348dd8@sparrow.telecommunity.com> At 08:59 AM 2/12/2007 +0100, Thomas Lotze wrote: >Phillip J. Eby wrote: > > > It appears your goals are somewhat... confused. > >I don't think so. I'm sure I still have to learn about setuptools but I >think I know quite well what I want to achieve. I meant the operational goals for the code you wrote, not your overall goal. > > Remember that by definition a namespace package has no single "owner" > > project. It is potentially shared across multiple projects. When > > installed as an egg, each __init__.py is executed, true. But when a > > project is NOT installed via egg, but rather by a system packager, the > > __init__.py doesn't exist, and so cannot be executed. > >If we have to take into account distributions of a project as source code >bereft of its top-level namespace's __init__.py being expected to just >work, I give up. I assumed to always work with either intact source or >built eggs, the latter with or without their namespaces' __init__.py files >being used and with or without being installed by our egg tools. Built eggs contain namespace package __init__.py files, system-installed eggs do not. >The package remapping is only needed when running from source, and I don't >see a reason why the __init__.py shouldn't exist and be executed in that >case. Because 1) it won't be the only __init__.py for the package, and 2) how does it help you avoid having subdirectories for namespace packages? You still have to have the directory to put the __init__.py in, and deeply nested namespace packages aren't a very good idea to begin with. If it's only one or two layers deep, it's just as easy to do it the standard way, without package_dirs. > > Therefore, your prototype code is horribly broken, as it will corrupt > > pkg_resources' internal data structures when it is run more than once -- > >I don't see why it couldn't be made to behave properly. Perhaps it could. I just don't see how! In order to set up the __path__'s for a specific project, the __init__.py must have knowledge of *that* project. Therefore, *each* project must contain such a file, *and* it must be executed. Under setuptools 0.6, it's currently the case that all namespace package __init__'s (except those installed as system packages) are executed. However, under setuptools 0.7, *at most one* will be -- which means that only *one* project will have a chance to do this munging, for its own paths. (All of these things I'm saying are documented in the setuptools manual under the "Namespace Packages" heading, by the way.) > > (And I won't even get into the > > pointlessness of wrapping module-level code in an import lock.) > >I'd appreciate it if you did, as I haven't yet found a difference between >using the lock in modules and using it in pkg_resources.declare_namespace >which is being called by modules. Runtime code that requests activation of an egg can currently result in declare_namespace() being called (see Distribution.activate()), and the import lock thus won't be held at that point in time. From pje at telecommunity.com Mon Feb 12 17:40:51 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 12 Feb 2007 11:40:51 -0500 Subject: [Distutils] Better error reporting In-Reply-To: <0548D213-4CFF-4672-AED7-94B151F12707@zope.com> Message-ID: <5.1.1.6.0.20070212113924.0543f178@sparrow.telecommunity.com> At 11:09 AM 2/12/2007 -0500, Jim Fulton wrote: >Phillip, would you like me to go ahead and make minor changes like >this? Or would you rather make them yourself? Just let me know about them, so I won't be taken by surprise when I can't commit due to being out of date. ;) From thomas at thomas-lotze.de Mon Feb 12 20:40:30 2007 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Mon, 12 Feb 2007 20:40:30 +0100 Subject: [Distutils] Setuptools: omit namespace package directories? References: <5.1.1.6.0.20070209152759.05074008@sparrow.telecommunity.com> <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070208165758.02d37ff8@sparrow.telecommunity.com> <5.1.1.6.0.20070209115618.038a7830@sparrow.telecommunity.com> <5.1.1.6.0.20070209152759.05074008@sparrow.telecommunity.com> <48701E33-297D-496D-8398-C7AABBD6E180@zope.com> <5.1.1.6.0.20070209161150.03d72140@sparrow.telecommunity.com> <5.1.1.6.0.20070211190757.027e5910@sparrow.telecommunity.com> <5.1.1.6.0.20070212111624.05348dd8@sparrow.telecommunity.com> Message-ID: Am Mon, 12 Feb 2007 11:38:13 -0500 schrieb Phillip J. Eby: > and 2) how > does it help you avoid having subdirectories for namespace packages? You > still have to have the directory to put the __init__.py in, and deeply > nested namespace packages aren't a very good idea to begin with. If it's > only one or two layers deep, it's just as easy to do it the standard way, > without package_dirs. Well, this is probably subject to personal preferences. I don't find a better answer to this than that I do find one or two levels too few for some things (maybe that's the german in me for all the desire to keep things orderly ;o)), and that I personally do find all the directory jumping this implies annoying. I mean, one top-level namespace for an organization or for a person to avoid stepping on other people's toes in terms of package names, one for a project, and maybe one for a bit of structure within a project, like modules, plugins, recipes, whatever. Seems sensible to me and makes for up to four-part dotted names - YMMV. > Under setuptools 0.6, it's currently the case that all namespace package > __init__'s (except those installed as system packages) are executed. > > However, under setuptools 0.7, *at most one* will be -- which means that > only *one* project will have a chance to do this munging, for its own > paths. Ah, this is the point where my scheme breaks apart. > (All of these things I'm saying are documented in the setuptools manual > under the "Namespace Packages" heading, by the way.) The crucial sentence in the docs is "If the first loaded __init__.py doesn't declare it, it will never be declared, because no other copies will ever be loaded!", which obviously refers to setuptools 0.7 while I was thinking in terms of (and your assertion that "When installed as an egg, each __init__.py is executed" probably referred to) 0.6. > Runtime code that requests activation of an egg can currently result in > declare_namespace() being called (see Distribution.activate()), and the > import lock thus won't be held at that point in time. OK, thanks. -- Viele Gr??e, Thomas From jorge.vargas at gmail.com Mon Feb 12 23:20:52 2007 From: jorge.vargas at gmail.com (Jorge Vargas) Date: Mon, 12 Feb 2007 18:20:52 -0400 Subject: [Distutils] Compiler support for VC++ 2005 In-Reply-To: <79990c6b0702081134s484503g353220e9423298c1@mail.gmail.com> References: <45B0AADD.13806.25AF95@howard.eegsoftware.com> <79990c6b0702020549q7fa212b2s6fc687b4cc3fc9e1@mail.gmail.com> <45C3437B.1020905@voidspace.org.uk> <79990c6b0702020623tb184f4dya5484b40e7285d4e@mail.gmail.com> <45C34929.4010701@voidspace.org.uk> <32822fe60702021943y4233aal24cca20bc7b5537b@mail.gmail.com> <79990c6b0702050333p7c12a227m73e6c314ced74b5f@mail.gmail.com> <32822fe60702071120s4d5ccb75ha726b13aae84a1c6@mail.gmail.com> <79990c6b0702081134s484503g353220e9423298c1@mail.gmail.com> Message-ID: <32822fe60702121420p22dd6e0fr4041f40a6bedaebc@mail.gmail.com> On 2/8/07, Paul Moore wrote: > On 07/02/07, Jorge Vargas wrote: > > yes that's true the problem comes when you need windows.h or other > > file that the mingw people can't distribute. > > Um, windows.h comes with mingw - it's in the win32api package. (Which > doesn't alter the fact that MySQL may not support being compiled by > mingw - but that's not because mingw doesn't supply the Windows > headers and library files). > you may want to check that again the windows.h mingw supplies is not exactly the same as the windows one, other files have this problem too, and of course they are not allowed to redistribute anything that comes with VS. > Paul. > From p.f.moore at gmail.com Mon Feb 12 23:40:54 2007 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 12 Feb 2007 22:40:54 +0000 Subject: [Distutils] Compiler support for VC++ 2005 In-Reply-To: <32822fe60702121420p22dd6e0fr4041f40a6bedaebc@mail.gmail.com> References: <45B0AADD.13806.25AF95@howard.eegsoftware.com> <79990c6b0702020549q7fa212b2s6fc687b4cc3fc9e1@mail.gmail.com> <45C3437B.1020905@voidspace.org.uk> <79990c6b0702020623tb184f4dya5484b40e7285d4e@mail.gmail.com> <45C34929.4010701@voidspace.org.uk> <32822fe60702021943y4233aal24cca20bc7b5537b@mail.gmail.com> <79990c6b0702050333p7c12a227m73e6c314ced74b5f@mail.gmail.com> <32822fe60702071120s4d5ccb75ha726b13aae84a1c6@mail.gmail.com> <79990c6b0702081134s484503g353220e9423298c1@mail.gmail.com> <32822fe60702121420p22dd6e0fr4041f40a6bedaebc@mail.gmail.com> Message-ID: <79990c6b0702121440r2171e189x6a9fd769aa1303fa@mail.gmail.com> On 12/02/07, Jorge Vargas wrote: > you may want to check that again the windows.h mingw supplies is not > exactly the same as the windows one, other files have this problem > too, and of course they are not allowed to redistribute anything that > comes with VS. I certainly didn't mean to imply that it was the same as the VS one, just that it exposed the same API (as far as I am aware). My understanding is that if a documented Windows API is exposed in the VS (or SDK) header but not in the mingw one, that would be accepted as a bug in the win32api package. But if an application relies on features of the MS windows.h which are not documented (and hence are "implementation details") that should be considered as an application bug (or at least, as a portability issue). Anyway, I'm not saying anything new here, so I won't waste people's time continuing this thread any further. Paul. From skip at pobox.com Tue Feb 13 15:43:20 2007 From: skip at pobox.com (skip at pobox.com) Date: Tue, 13 Feb 2007 08:43:20 -0600 Subject: [Distutils] Whose problem is this? Message-ID: <17873.52872.54723.609722@montanaro.dyndns.org> I noticed that some easy_installed packages don't seem to support online help in my environment at work (Solaris 10/x86, Python 2.4.2). Here's an example: % python Python 2.4.2 (#1, Feb 23 2006, 12:48:31) [GCC 3.4.1] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> import sqlalchemy >>> sqlalchemy.__file__ '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/SQLAlchemy-0.3.3-py2.4.egg/sqlalchemy/__init__.pyc' >>> help(sqlalchemy) Traceback (most recent call last): File "", line 1, in ? File "/opt/app/g++lib6/python-2.4/lib/python2.4/site.py", line 328, in __call__ return pydoc.help(*args, **kwds) File "/opt/app/g++lib6/python-2.4/lib/python2.4/pydoc.py", line 1647, in __call__ self.help(request) File "/opt/app/g++lib6/python-2.4/lib/python2.4/pydoc.py", line 1691, in help else: doc(request, 'Help on %s:') File "/opt/app/g++lib6/python-2.4/lib/python2.4/pydoc.py", line 1475, in doc pager(title % desc + '\n\n' + text.document(object, name)) File "/opt/app/g++lib6/python-2.4/lib/python2.4/pydoc.py", line 295, in document if inspect.ismodule(object): return self.docmodule(*args) File "/opt/app/g++lib6/python-2.4/lib/python2.4/pydoc.py", line 1052, in docmodule for file in os.listdir(object.__path__[0]): OSError: [Errno 20] Not a directory: '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/SQLAlchemy-0.3.3-py2.4.egg/sqlalchemy' Online help for setuptools itself works fine though. Is that a pydoc problem, a Python problem, an easy_install problem or what? I get the same result when I try to get help about Elixir. Skip From pje at telecommunity.com Tue Feb 13 17:42:46 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 13 Feb 2007 11:42:46 -0500 Subject: [Distutils] Whose problem is this? In-Reply-To: <17873.52872.54723.609722@montanaro.dyndns.org> Message-ID: <5.1.1.6.0.20070213113844.027b4488@sparrow.telecommunity.com> At 08:43 AM 2/13/2007 -0600, skip at pobox.com wrote: >Online help for setuptools itself works fine though. > >Is that a pydoc problem, a Python problem, an easy_install problem or what? It's a pydoc problem that's fixed in Python 2.5. You can work around it by installing your eggs unzipped, using the -Z option to easy_install. At some point, I intend to release a hotfix version of pydoc, doctest, and a few other fixed module for use with older versions of Python, but haven't gotten around to it yet. The tricky thing is that there are some Python 2.5 features added (or removed) that mean I can't just slap the 2.5 versions in an egg and call it done. I essentially need to cherry-pick the revisions that fix the zip problems. From jnelson at atwcorp.com Tue Feb 13 18:27:28 2007 From: jnelson at atwcorp.com (Jon Nelson) Date: Tue, 13 Feb 2007 11:27:28 -0600 Subject: [Distutils] setuptools 0.6c5 released :( In-Reply-To: <5.1.1.6.0.20070109172139.02782080@sparrow.telecommunity.com> References: <5.1.1.6.0.20070109143332.02863fe8@sparrow.telecommunity.com> <5.1.1.6.0.20070109143332.02863fe8@sparrow.telecommunity.com> <5.1.1.6.0.20070109172139.02782080@sparrow.telecommunity.com> Message-ID: <45D1F500.6030303@atwcorp.com> Phillip J. Eby wrote: > At 03:34 PM 1/9/2007 -0600, Jon Nelson wrote: >> Phillip J. Eby wrote: >> > I found some upload and bdist related issues while uploading 0.6c4, >> so I've >> > fixed them and released 0.6c5. >> >> I found a bug in 0.6c5: >> >> setuptools/command/bdist_rpm.py is missing an import: >> >> from distutils.sysconfig import get_python_version >> >> get_python_version is used but never defined. > > Ugh. I fixed this one back in September on the trunk, but evidently > missed the backport: > > http://svn.python.org/view/sandbox/trunk/setuptools/setuptools/command/bdist_rpm.py?rev=51967&r1=51960&r2=51967 > > > I've copied it and am making preparations for 0.6c6. I'm just curious where 0.6c6 is since 0.6c5 is has a broken bdist_rpm. -- * * Jon Nelson ** Linux Software Developer * * AmbironTrustWave * * 120 N. LaSalle Street, Suite 1250, Chicago, IL, 60602 * * (312) 873-7292 ** / fax: (312) 443-1620 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070213/85db91c4/attachment.html From ronaldoussoren at mac.com Tue Feb 13 23:35:51 2007 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Tue, 13 Feb 2007 14:35:51 -0800 Subject: [Distutils] pkg_resources._macosx_vers without using os.popen Message-ID: Philip, The attached patched replaces pkg_resources._macosx_vers by a function that directly parses the relevant XML file instead of using sw_vers. The patch is an svn diff relative to the trunk. The main reason I prefer this version is that I'm debugging some C code in a project using setuptools using command-line gdb (who needs IDE's?) and I often (but not always) get I/O errors when reading from a pipe to a forked-off process when running python from gdb. With this patch pkg_resources no longer forks of processes and gdb is happy. When you're not using gdb there's little reason to prefer this version over the original one. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: pkg_resources-macosx-vers.diff Type: application/octet-stream Size: 804 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20070213/5774824a/attachment.obj From pje at telecommunity.com Wed Feb 14 01:15:44 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 13 Feb 2007 19:15:44 -0500 Subject: [Distutils] pkg_resources._macosx_vers without using os.popen In-Reply-To: Message-ID: <5.1.1.6.0.20070213191409.0465eb50@sparrow.telecommunity.com> At 02:35 PM 2/13/2007 -0800, Ronald Oussoren wrote: >Philip, > >The attached patched replaces pkg_resources._macosx_vers by a function >that directly parses the relevant XML file instead of using sw_vers. The >patch is an svn diff relative to the trunk. > >The main reason I prefer this version is that I'm debugging some C code in >a project using setuptools using command-line gdb (who needs IDE's?) and I >often (but not always) get I/O errors when reading from a pipe to a >forked-off process when running python from gdb. With this patch >pkg_resources no longer forks of processes and gdb is happy. > >When you're not using gdb there's little reason to prefer this version >over the original one. Your patch uses plistlib, which doesn't appear to be part of the Python standard library. From constant.beta at gmail.com Wed Feb 14 03:41:45 2007 From: constant.beta at gmail.com (=?ISO-8859-2?Q?Micha=B3_Kwiatkowski?=) Date: Wed, 14 Feb 2007 03:41:45 +0100 Subject: [Distutils] Cheesecake service Message-ID: <5e8b0f6b0702131841p69bb1791yec003045f6a85ec5@mail.gmail.com> Hi, On top of PyPI and Cheesecake I've build a Cheesecake service - a service which automatically downloads and scores new package releases. It is available at this address: http://pypi.pycheesecake.org/pypi/ . Probably of most interest here is the easy_installability page (http://pypi.pycheesecake.org/pypi/easy_installability) which shows releases which are listed on PyPI, yet still can't be easy_installed. If this feature proves itself to be reliable it could probably substitute a list of successful installations on http://peak.telecommunity.com/DevCenter/PackageNotes . I'm open to any suggestions on how this service can be improved. You can learn more about Cheesecake principles in this recent blog post: http://mousebender.wordpress.com/2007/02/09/cheesecake-for-all/ . Cheers, mk From pje at telecommunity.com Wed Feb 14 04:03:22 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 13 Feb 2007 22:03:22 -0500 Subject: [Distutils] Cheesecake service In-Reply-To: <5e8b0f6b0702131841p69bb1791yec003045f6a85ec5@mail.gmail.co m> Message-ID: <5.1.1.6.0.20070213215523.04cf2008@sparrow.telecommunity.com> At 03:41 AM 2/14/2007 +0100, =?ISO-8859-2?Q?Micha=B3_Kwiatkowski?= wrote: >On top of PyPI and Cheesecake I've build a Cheesecake service - a >service which automatically downloads and scores new package releases. >It is available at this address: http://pypi.pycheesecake.org/pypi/ . >Probably of most interest here is the easy_installability page >(http://pypi.pycheesecake.org/pypi/easy_installability) which shows >releases which are listed on PyPI, yet still can't be easy_installed. >[snip] >I'm open to any suggestions on how this service can be improved. More information about *how* the installation failed would be useful for package authors to see at a glance what's wrong. Right now, it's too easy to dismiss it as a peculiarity of your installation. Being able to click through and see the log/tracebacks would probably make it more likely that an author spots the problem(s). From ianb at colorstudy.com Wed Feb 14 05:38:15 2007 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 13 Feb 2007 22:38:15 -0600 Subject: [Distutils] Cheesecake service In-Reply-To: <5.1.1.6.0.20070213215523.04cf2008@sparrow.telecommunity.com> References: <5.1.1.6.0.20070213215523.04cf2008@sparrow.telecommunity.com> Message-ID: <45D29237.1060405@colorstudy.com> Phillip J. Eby wrote: > At 03:41 AM 2/14/2007 +0100, =?ISO-8859-2?Q?Micha=B3_Kwiatkowski?= wrote: >> On top of PyPI and Cheesecake I've build a Cheesecake service - a >> service which automatically downloads and scores new package releases. >> It is available at this address: http://pypi.pycheesecake.org/pypi/ . >> Probably of most interest here is the easy_installability page >> (http://pypi.pycheesecake.org/pypi/easy_installability) which shows >> releases which are listed on PyPI, yet still can't be easy_installed. >> [snip] >> I'm open to any suggestions on how this service can be improved. > > More information about *how* the installation failed would be useful for > package authors to see at a glance what's wrong. Right now, it's too easy > to dismiss it as a peculiarity of your installation. Being able to click > through and see the log/tracebacks would probably make it more likely that > an author spots the problem(s). Looks like it's in there, but two links away, and the installation logs specifically are buried in the rest of the logs. Also the logs are served up as a Python script or something, not text/plain. Ideally the logs would be HTML with anchors, and then you could just add a link on the easy_installability to package_log_uri#installability. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From S.Pascoe at rl.ac.uk Wed Feb 14 11:52:07 2007 From: S.Pascoe at rl.ac.uk (Pascoe, S (Stephen)) Date: Wed, 14 Feb 2007 10:52:07 -0000 Subject: [Distutils] Cheesecake service In-Reply-To: <5e8b0f6b0702131841p69bb1791yec003045f6a85ec5@mail.gmail.com> References: <5e8b0f6b0702131841p69bb1791yec003045f6a85ec5@mail.gmail.com> Message-ID: Excellent, I'm a great fan of cheesecake. Can I ask how packages are selected for scoring? There seems a lot fewer packages on the list than everything in PyPI. I will be giving a couple of talks on eggs at PyCon2007 next week, one aimed at beginners using eggs and one at developers. A key messages of the second talk will be how to make packages easy_install compatible. I'd like to publicise this cheesecake service there. Cheers, Stephen. --- Stephen Pascoe 01235 445980 British Atmospheric Data Centre Rutherford Appleton Laboratory, CCLRC -----Original Message----- From: distutils-sig-bounces at python.org [mailto:distutils-sig-bounces at python.org] On Behalf Of Michal Kwiatkowski Sent: 14 February 2007 02:42 To: distutils-sig at python.org Subject: [Distutils] Cheesecake service Hi, On top of PyPI and Cheesecake I've build a Cheesecake service - a service which automatically downloads and scores new package releases. It is available at this address: http://pypi.pycheesecake.org/pypi/ . Probably of most interest here is the easy_installability page (http://pypi.pycheesecake.org/pypi/easy_installability) which shows releases which are listed on PyPI, yet still can't be easy_installed. If this feature proves itself to be reliable it could probably substitute a list of successful installations on http://peak.telecommunity.com/DevCenter/PackageNotes . I'm open to any suggestions on how this service can be improved. You can learn more about Cheesecake principles in this recent blog post: http://mousebender.wordpress.com/2007/02/09/cheesecake-for-all/ . Cheers, mk _______________________________________________ Distutils-SIG maillist - Distutils-SIG at python.org http://mail.python.org/mailman/listinfo/distutils-sig From constant.beta at gmail.com Wed Feb 14 14:50:08 2007 From: constant.beta at gmail.com (=?ISO-8859-2?Q?Micha=B3_Kwiatkowski?=) Date: Wed, 14 Feb 2007 14:50:08 +0100 Subject: [Distutils] Cheesecake service In-Reply-To: References: <5e8b0f6b0702131841p69bb1791yec003045f6a85ec5@mail.gmail.com> Message-ID: <5e8b0f6b0702140550o5a39abcdva9802fa0479a211d@mail.gmail.com> On 2/14/07, Pascoe, S (Stephen) wrote: > I'm a great fan of cheesecake. Can I ask how packages are selected for > scoring? There seems a lot fewer packages on the list than everything > in PyPI. Currently package scoring is invoked when a new release appears on PyPI. The service can also accept list of packages put by hand, so it is possible to score other packages present on PyPI but without a new release since August 10th. I will pull out full list of packages from PyPI and feed the service with it later today. Cheers, mk From constant.beta at gmail.com Wed Feb 14 14:52:50 2007 From: constant.beta at gmail.com (=?ISO-8859-2?Q?Micha=B3_Kwiatkowski?=) Date: Wed, 14 Feb 2007 14:52:50 +0100 Subject: [Distutils] Cheesecake service In-Reply-To: <45D29237.1060405@colorstudy.com> References: <5.1.1.6.0.20070213215523.04cf2008@sparrow.telecommunity.com> <45D29237.1060405@colorstudy.com> Message-ID: <5e8b0f6b0702140552n32cbd775y550bff4c9b8270f4@mail.gmail.com> On 2/14/07, Ian Bicking wrote: > Phillip J. Eby wrote: > > More information about *how* the installation failed would be useful for > > package authors to see at a glance what's wrong. Right now, it's too easy > > to dismiss it as a peculiarity of your installation. Being able to click > > through and see the log/tracebacks would probably make it more likely that > > an author spots the problem(s). > > Looks like it's in there, but two links away, and the installation logs > specifically are buried in the rest of the logs. Also the logs are > served up as a Python script or something, not text/plain. Ideally the > logs would be HTML with anchors, and then you could just add a link on > the easy_installability to package_log_uri#installability. Good idea. I will have to tweak Cheesecake output a bit, but it is certainly possible. I will let the list know when I finish the implementation. Cheers, mk From ronaldoussoren at mac.com Wed Feb 14 20:47:46 2007 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 14 Feb 2007 20:47:46 +0100 Subject: [Distutils] pkg_resources._macosx_vers without using os.popen In-Reply-To: <5.1.1.6.0.20070213191409.0465eb50@sparrow.telecommunity.com> References: <5.1.1.6.0.20070213191409.0465eb50@sparrow.telecommunity.com> Message-ID: On 14 Feb, 2007, at 1:15, Phillip J. Eby wrote: > At 02:35 PM 2/13/2007 -0800, Ronald Oussoren wrote: >> Philip, >> >> The attached patched replaces pkg_resources._macosx_vers by a >> function that directly parses the relevant XML file instead of >> using sw_vers. The patch is an svn diff relative to the trunk. >> >> The main reason I prefer this version is that I'm debugging some C >> code in a project using setuptools using command-line gdb (who >> needs IDE's?) and I often (but not always) get I/O errors when >> reading from a pipe to a forked-off process when running python >> from gdb. With this patch pkg_resources no longer forks of >> processes and gdb is happy. >> >> When you're not using gdb there's little reason to prefer this >> version over the original one. > > Your patch uses plistlib, which doesn't appear to be part of the > Python standard library. It is on osx, the only platform where _macosx_vers is used. Ronald From pogonyshev at gmx.net Thu Feb 15 00:30:10 2007 From: pogonyshev at gmx.net (Paul Pogonyshev) Date: Thu, 15 Feb 2007 01:30:10 +0200 Subject: [Distutils] c extensions: how to rebuild and test quickly Message-ID: <200702150130.10917.pogonyshev@gmx.net> Hi, Distutils create dynamic library for a C extension in `build/...' directory. This makes it impossible to run program without installing it, which I find very important for developing. Ideally, process should look like this: edit Python code -> test (uninstalled); or edit C code -> ./setup.by build -> test (uninstalled). This is all possible if I create a symbolic link from build directory to the sources. Then extension module can be imported normally and everything is like it was with all code limited to Python. Is it possible in some standard (and preferably portable) way with distutils? Is it already done? Paul From sjmachin at lexicon.net Thu Feb 15 00:47:30 2007 From: sjmachin at lexicon.net (John Machin) Date: Thu, 15 Feb 2007 10:47:30 +1100 Subject: [Distutils] c extensions: how to rebuild and test quickly In-Reply-To: <200702150130.10917.pogonyshev@gmx.net> References: <200702150130.10917.pogonyshev@gmx.net> Message-ID: <45D39F92.30202@lexicon.net> On 15/02/2007 10:30 AM, Paul Pogonyshev wrote: > Hi, > > Distutils create dynamic library for a C extension in `build/...' > directory. This makes it impossible to run program without > installing it, Could you not put the build directory on your PYTHONPATH? Possible, if inconvenient. > which I find very important for developing. Ideally, > process should look like this: > edit Python code -> test (uninstalled); > or > edit C code -> ./setup.by build -> test (uninstalled). > This is all possible if I create a symbolic link from build > directory to the sources. Then extension module can be imported > normally and everything is like it was with all code limited to > Python. > > Is it possible in some standard (and preferably portable) way with > distutils? Is it already done? > Is ....setup.py build_ext --inplace what you are looking for? Don't forget ....setup.py --help build_ext HTH, John From pje at telecommunity.com Thu Feb 15 01:04:02 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 14 Feb 2007 19:04:02 -0500 Subject: [Distutils] c extensions: how to rebuild and test quickly In-Reply-To: <200702150130.10917.pogonyshev@gmx.net> Message-ID: <5.1.1.6.0.20070214190100.04fdc008@sparrow.telecommunity.com> At 01:30 AM 2/15/2007 +0200, Paul Pogonyshev wrote: >Hi, > >Distutils create dynamic library for a C extension in `build/...' >directory. This makes it impossible to run program without >installing it, which I find very important for developing. Ideally, >process should look like this: > > edit Python code -> test (uninstalled); > >or > > edit C code -> ./setup.by build -> test (uninstalled). > >This is all possible if I create a symbolic link from build >directory to the sources. Then extension module can be imported >normally and everything is like it was with all code limited to >Python. > >Is it possible in some standard (and preferably portable) way with >distutils? Is it already done? As someone else has mentioned, build_ext --inplace will do the trick. However, I find it a bit awkward to use, as you have to manually rebuild when testing different Python versions (and/or platforms: I sometimes use both Cygwin and Windows Python, for example). So, setuptools' "test" and "develop" commands (as well as its own version of "build_ext --inplace") do the build in the build/ subdirectory, but then *copy* the files to the source directory. Because each build directory has its own platform-specific files, this automatically uses the right files for the Python version you're running the "test" command with. It's still not perfect, but it can be a nice improvement over plain "build_ext -i" usage. From constant.beta at gmail.com Thu Feb 15 03:35:50 2007 From: constant.beta at gmail.com (=?ISO-8859-2?Q?Micha=B3_Kwiatkowski?=) Date: Thu, 15 Feb 2007 03:35:50 +0100 Subject: [Distutils] Cheesecake service In-Reply-To: <5e8b0f6b0702140552n32cbd775y550bff4c9b8270f4@mail.gmail.com> References: <5.1.1.6.0.20070213215523.04cf2008@sparrow.telecommunity.com> <45D29237.1060405@colorstudy.com> <5e8b0f6b0702140552n32cbd775y550bff4c9b8270f4@mail.gmail.com> Message-ID: <5e8b0f6b0702141835y43575a91pf471ac7e882a0a4a@mail.gmail.com> On 2/14/07, Ian Bicking wrote: > Looks like it's in there, but two links away, and the installation logs > specifically are buried in the rest of the logs. Also the logs are > served up as a Python script or something, not text/plain. Ideally the > logs would be HTML with anchors, and then you could just add a link on > the easy_installability to package_log_uri#installability. OK, logs are now served as HTML pages. Easy_installability page links directly to certain parts of log files. I have a question concerning some of the failing packages. As an example let's take DisplayShelf 1.0. Cheesecake fails to install it, because in its setup.py it imports "turbogears.finddata", which isn't installed on pybots.org. Log file for DisplayShelf is here: http://pypi.pycheesecake.org/pypi/log/171-DisplayShelf==1.0-success-5.log#install . Package has defined "TurboGears" in "install_requires", but is using import that will only be working if TurboGears is already there. In fact, whole setup.py is unusable, because it simply fails to run on systems without TurboGears. Where is the problem and how it can be solved? Cheers, mk From ianb at colorstudy.com Thu Feb 15 04:14:48 2007 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 14 Feb 2007 21:14:48 -0600 Subject: [Distutils] Cheesecake service In-Reply-To: <5e8b0f6b0702141835y43575a91pf471ac7e882a0a4a@mail.gmail.com> References: <5.1.1.6.0.20070213215523.04cf2008@sparrow.telecommunity.com> <45D29237.1060405@colorstudy.com> <5e8b0f6b0702140552n32cbd775y550bff4c9b8270f4@mail.gmail.com> <5e8b0f6b0702141835y43575a91pf471ac7e882a0a4a@mail.gmail.com> Message-ID: <45D3D028.2040003@colorstudy.com> Micha? Kwiatkowski wrote: > On 2/14/07, Ian Bicking wrote: >> Looks like it's in there, but two links away, and the installation logs >> specifically are buried in the rest of the logs. Also the logs are >> served up as a Python script or something, not text/plain. Ideally the >> logs would be HTML with anchors, and then you could just add a link on >> the easy_installability to package_log_uri#installability. > > OK, logs are now served as HTML pages. Easy_installability page links > directly to certain parts of log files. If you add some blank space to the bottom of the page the anchors can be a little more clear. Like in what you link to, it shows up about halfway down the page because there's not enough below it to fill the space. Another more complex solution is some Javascript that sees the anchor and highlights the anchor based on that. But a bunch of
's is much easier ;) > I have a question concerning some of the failing packages. As an > example let's take DisplayShelf 1.0. Cheesecake fails to install it, > because in its setup.py it imports "turbogears.finddata", which isn't > installed on pybots.org. Log file for DisplayShelf is here: > http://pypi.pycheesecake.org/pypi/log/171-DisplayShelf==1.0-success-5.log#install > . Package has defined "TurboGears" in "install_requires", but is using > import that will only be working if TurboGears is already there. In > fact, whole setup.py is unusable, because it simply fails to run on > systems without TurboGears. Where is the problem and how it can be > solved? I think that's just a broken package. It shouldn't do that. Actually finddata probably isn't necessary anymore if they use include_package_data=True. Otherwise they should copy that file into their package before using it. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From pje at telecommunity.com Thu Feb 15 05:30:17 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 14 Feb 2007 23:30:17 -0500 Subject: [Distutils] Cheesecake service In-Reply-To: <45D3D028.2040003@colorstudy.com> References: <5e8b0f6b0702141835y43575a91pf471ac7e882a0a4a@mail.gmail.com> <5.1.1.6.0.20070213215523.04cf2008@sparrow.telecommunity.com> <45D29237.1060405@colorstudy.com> <5e8b0f6b0702140552n32cbd775y550bff4c9b8270f4@mail.gmail.com> <5e8b0f6b0702141835y43575a91pf471ac7e882a0a4a@mail.gmail.com> Message-ID: <5.1.1.6.0.20070214232350.02813cb8@sparrow.telecommunity.com> At 09:14 PM 2/14/2007 -0600, Ian Bicking wrote: >I think that's just a broken package. It shouldn't do that. Actually >finddata probably isn't necessary anymore if they use >include_package_data=True. Otherwise they should copy that file into >their package before using it. Well, there's broken and there's broken. Obviously, for most TurboGears users the package isn't going to fail - either they have Turbogears or the egg is going to be downloaded instead of the source tarball, which means that the setup.py won't be an issue. That having been said, what you just said is of course also 100% correct - they should probably be using include_package_data instead. By the way, it's not entirely clear to me what the Cheesecake system actually tests. My initial response to some of the TurboGears packages being listed as not installable was disbelief, because they are in fact usable with easy_install, as long as the author of the package uploaded an egg compatible with your Python version and platform. It's merely the case that building the package from source will fail without TurboGears being present. However, this makes me wonder if there are false successes as well. If Cheesecake merely tests whether "setup.py install" works, this is not the same as easy_install being able to run on the package. For example, Twisted has a working "setup.py install", but it is pretty much impossible to install with easy_install. Twisted isn't listed on the current page, but I wonder what column it would be in, if it were. From theller at ctypes.org Thu Feb 15 07:58:03 2007 From: theller at ctypes.org (Thomas Heller) Date: Thu, 15 Feb 2007 07:58:03 +0100 Subject: [Distutils] c extensions: how to rebuild and test quickly In-Reply-To: <200702150130.10917.pogonyshev@gmx.net> References: <200702150130.10917.pogonyshev@gmx.net> Message-ID: Paul Pogonyshev schrieb: > Hi, > > Distutils create dynamic library for a C extension in `build/...' > directory. This makes it impossible to run program without > installing it, which I find very important for developing. Ideally, > process should look like this: > > edit Python code -> test (uninstalled); > > or > > edit C code -> ./setup.by build -> test (uninstalled). > > This is all possible if I create a symbolic link from build > directory to the sources. Then extension module can be imported > normally and everything is like it was with all code limited to > Python. > > Is it possible in some standard (and preferably portable) way with > distutils? Is it already done? What I do is to create a loader module named .py that 'replaces' itself with the real extension, loading it via imp from the platform specific build directory that distutils creates. Here is the code: """ def _boot(): import sys, os, imp from distutils.util import get_platform plat_specifier = ".%s-%s" % (get_platform(), sys.version[0:3]) build_dir = os.path.join(os.path.dirname(__file__), 'build', 'lib' + plat_specifier) file, path, descr = imp.find_module(__name__, [build_dir]) imp.load_module(__name__, file, path, descr) _boot(); del _boot """ Thomas From meine at informatik.uni-hamburg.de Thu Feb 15 15:05:52 2007 From: meine at informatik.uni-hamburg.de (Hans Meine) Date: Thu, 15 Feb 2007 15:05:52 +0100 Subject: [Distutils] [PATCH] Handling of scripts / substitution of python executable path Message-ID: <200702151505.53603.meine@informatik.uni-hamburg.de> Hi! For a long time, I have been annoyed by distutils behavior concerning "scripts": I always put #!/usr/bin/env python into the first line in order to let the incredibly useful "env" program start the right python version. I know that it is quite evil to hardcode /usr/bin/python or /usr/local/bin/python; I have seen dozens of #! hacks for finding e.g. perl, and I was delighted to find /usr/bin/env, which solves the problem once and for all. (And yes - it is always in /usr/bin/! ;-) ) Now distutils tries to be intelligent and "destroys" my nice #! lines, which can be quite evil in complex setups, e.g. when you share your home directory via NFS (or rsync/unison) between several environments with different python installations. Furthermore, we are using the "module" system here at our university, so that I can dynamically choose between half a dozen python versions ("module" manages your PATH variables). Replacing the python path turns nice, pure python scrips into platform-specific programs as you can see here: meine at kogspc12:~ head -n1 ~/vigra/interactive/build/scripts-2.4/pyterm #!/software/python-2.4.4/SuSE-9.0/bin/python Note the SuSE-9.0 exec-prefix in the path; we are using several Linux and Solaris versions here: meine at kogspc12:~/tmp/vi3build -> ls -1 /software/python-2.4.4/*/bin/python /software/python-2.4.4/SuSE-10.0/bin/python* /software/python-2.4.4/SuSE-9.0/bin/python* /software/python-2.4.4/SunOS-5.8/bin/python* I see that distutils as it is now does the right thing * on Windows systems, * on any system where /usr/bin/env is missing, or * when the source file has a #! with a broken path. What I propose is a minimal invasive change which keeps /usr/bin/env iff it is in the #! line *and* exists on the current system. (A more brave change would be to always use /usr/bin/env if it exists, but I think that is a topic open for discussion.) Attached you'll find a patch which implements this; I did not yet update tests/test_build_scripts.py however. Comments? Ciao, / / /--/ / / ANS -------------- next part -------------- A non-text attachment was scrubbed... Name: distutils_env.diff Type: text/x-diff Size: 2348 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20070215/860f3754/attachment.bin From meine at informatik.uni-hamburg.de Thu Feb 15 17:01:11 2007 From: meine at informatik.uni-hamburg.de (Hans Meine) Date: Thu, 15 Feb 2007 17:01:11 +0100 Subject: [Distutils] Building shared libraries? Message-ID: <200702151701.13175.meine@informatik.uni-hamburg.de> Hi! I know that distutils' purpose is not to re-implement libtool, but.. it seems that it is a common problem that one wants to ship several extension modules which rely on the same code. E.g. we have several image processing functions split across several extension modules (written in C++), and we need them all to know about our PythonImage class. (E.g. PythonImage::subImage(...) or the like.) A natural way to solve this is to compile pythonimage.cxx into a small shared library which the extension modules link against. So far, it seems that all projects which have this problem use Extension(...) to build a shared library or call make / do whatever custom hacks were necessary for them. (One source of information were the answers to David Abrahams question some years ago: http://mail.python.org/pipermail/distutils-sig/2002-October/002966.html and I also asked on C++-sig lately: http://mail.python.org/pipermail/c++-sig/2007-February/011973.html ) Would it be possible to add a distutils.SharedLibrary class? (OK, would someone be willing to provide the code? ;-) ) Ciao, / / /--/ / / ANS From venkatbo at yahoo.com Thu Feb 15 18:15:10 2007 From: venkatbo at yahoo.com (Venkat Bommakanti) Date: Thu, 15 Feb 2007 09:15:10 -0800 (PST) Subject: [Distutils] ctypes setuptools egg Message-ID: <536256.67040.qm@web30505.mail.mud.yahoo.com> Hi folks, I'd like to know if there is a source egg of ctypes (for Python 2.4.x) that can be built and deployed via setuptools... I need to be able to cross compile ctypes from an i686Linux box targeting both i686 and ppc Linux's. I'd like to deploy it with a python app that is also being built/pkg as an egg. I searched the archives, but found no pointers. Thanks, /venkat ____________________________________________________________________________________ Do you Yahoo!? Everyone is raving about the all-new Yahoo! Mail beta. http://new.mail.yahoo.com From sidnei at enfoldsystems.com Thu Feb 15 18:38:43 2007 From: sidnei at enfoldsystems.com (Sidnei da Silva) Date: Thu, 15 Feb 2007 17:38:43 +0000 (UTC) Subject: [Distutils] setuptools and sdist on Mac OS X, tarfile flukes Message-ID: Not sure if anyone reported a similar issue here since I'm not subscribed, but here it goes. Some of the new plone.* eggs are being built on OS X by the developers, they usually do 'setup.py sdist bdist_egg upload'. Sometimes they only do 'setup.py sdist upload' Turns out that if you try to easy_install and there's only the sdist package, but no egg, easy_install uses tarfile to unpack the package, build an egg and install the egg. However, for some reason some of those nasty '._' files that OS X creates end up in the tarball. And tarfile then fails to unpack the tarball. One such package is 'plone.app.controlpanel 1.0a2' (http://cheeseshop.python.org/packages/source/p/plone.app.controlpanel/plone.app.controlpanel-1.0a2.tar.gz#md5=2b2afeaba0d067f6b1a6707c6d69311d), with the following traceback: C:\DOCUME~1\Sidnei\LOCALS~1\Temp>easy_install -f . plone.app.controlpanel Searching for plone.app.controlpanel Best match: plone.app.controlpanel 1.0a2 Processing plone.app.controlpanel-1.0a2.tar.gz Traceback (most recent call last): File "c:\Python24\Scripts\easy_install-script.py", line 7, in ? sys.exit( File "c:\python24\lib\site-packages\setuptools-0.6c5-py2.4.egg\setuptools\comm and\easy_install.py", line 1670, in main with_ei_usage(lambda: File "c:\python24\lib\site-packages\setuptools-0.6c5-py2.4.egg\setuptools\comm and\easy_install.py", line 1659, in with_ei_usage return f() File "c:\python24\lib\site-packages\setuptools-0.6c5-py2.4.egg\setuptools\comm and\easy_install.py", line 1674, in distclass=DistributionWithoutHelpCommands, **kw File "C:\Python24\lib\distutils\core.py", line 149, in setup dist.run_commands() File "C:\Python24\lib\distutils\dist.py", line 946, in run_commands self.run_command(cmd) File "C:\Python24\lib\distutils\dist.py", line 966, in run_command cmd_obj.run() File "c:\python24\lib\site-packages\setuptools-0.6c5-py2.4.egg\setuptools\comm and\easy_install.py", line 211, in run self.easy_install(spec, not self.no_deps) File "c:\python24\lib\site-packages\setuptools-0.6c5-py2.4.egg\setuptools\comm and\easy_install.py", line 446, in easy_install return self.install_item(spec, dist.location, tmpdir, deps) File "c:\python24\lib\site-packages\setuptools-0.6c5-py2.4.egg\setuptools\comm and\easy_install.py", line 471, in install_item dists = self.install_eggs(spec, download, tmpdir) File "c:\python24\lib\site-packages\setuptools-0.6c5-py2.4.egg\setuptools\comm and\easy_install.py", line 626, in install_eggs unpack_archive(dist_filename, tmpdir, self.unpack_progress) File "c:\python24\lib\site-packages\setuptools-0.6c5-py2.4.egg\setuptools\arch ive_util.py", line 67, in unpack_archive driver(filename, extract_dir, progress_filter) File "c:\python24\lib\site-packages\setuptools-0.6c5-py2.4.egg\setuptools\arch ive_util.py", line 192, in unpack_tarfile tarobj._extract_member(member,dst) # XXX Ugh File "C:\Python24\lib\tarfile.py", line 1440, in _extract_member self.utime(tarinfo, targetpath) File "C:\Python24\lib\tarfile.py", line 1572, in utime raise ExtractError, "could not change modification time" tarfile.ExtractError: could not change modification time I'm left wondering if the issue is: a) Windows b) The nasty OS X file c) The Python 'tarfile' module d) setuptools e) None of the above :) Thoughts? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214 From theller at ctypes.org Thu Feb 15 19:27:57 2007 From: theller at ctypes.org (Thomas Heller) Date: Thu, 15 Feb 2007 19:27:57 +0100 Subject: [Distutils] ctypes setuptools egg In-Reply-To: <536256.67040.qm@web30505.mail.mud.yahoo.com> References: <536256.67040.qm@web30505.mail.mud.yahoo.com> Message-ID: Venkat Bommakanti schrieb: > Hi folks, > > I'd like to know if there is a source egg of ctypes > (for Python 2.4.x) that can be built and deployed via > setuptools... > > I need to be able to cross compile ctypes from an > i686Linux box targeting both i686 and ppc Linux's. I'd > like to deploy it with a python app that is also being > built/pkg as an egg. > Please see the reply to this message on the ctypes-users list. Thomas From constant.beta at gmail.com Thu Feb 15 19:48:13 2007 From: constant.beta at gmail.com (=?ISO-8859-2?Q?Micha=B3_Kwiatkowski?=) Date: Thu, 15 Feb 2007 19:48:13 +0100 Subject: [Distutils] Cheesecake service In-Reply-To: <45D3D028.2040003@colorstudy.com> References: <5.1.1.6.0.20070213215523.04cf2008@sparrow.telecommunity.com> <45D29237.1060405@colorstudy.com> <5e8b0f6b0702140552n32cbd775y550bff4c9b8270f4@mail.gmail.com> <5e8b0f6b0702141835y43575a91pf471ac7e882a0a4a@mail.gmail.com> <45D3D028.2040003@colorstudy.com> Message-ID: <5e8b0f6b0702151048n579fee5eldc5bd4002e304a64@mail.gmail.com> On 2/15/07, Ian Bicking wrote: > Another more complex solution is some Javascript that sees the anchor > and highlights the anchor based on that. But a bunch of
's is much > easier ;) OK, done. Tested on Firefox and Konqueror. Cheers, mk From pogonyshev at gmx.net Thu Feb 15 20:40:35 2007 From: pogonyshev at gmx.net (Paul Pogonyshev) Date: Thu, 15 Feb 2007 21:40:35 +0200 Subject: [Distutils] c extensions: how to rebuild and test quickly In-Reply-To: <45D39F92.30202@lexicon.net> References: <200702150130.10917.pogonyshev@gmx.net> <45D39F92.30202@lexicon.net> Message-ID: <200702152140.35928.pogonyshev@gmx.net> John Machin wrote: > On 15/02/2007 10:30 AM, Paul Pogonyshev wrote: > > Hi, > > > > Distutils create dynamic library for a C extension in `build/...' > > directory. This makes it impossible to run program without > > installing it, > > Could you not put the build directory on your PYTHONPATH? > Possible, if inconvenient. Not really, because I want users (package is a library) to be able to run unit tests. They should be able to do this without any extra steps. > > [...] > Is > ....setup.py build_ext --inplace > what you are looking for? No, for the same reason. Paul From pogonyshev at gmx.net Thu Feb 15 20:41:44 2007 From: pogonyshev at gmx.net (Paul Pogonyshev) Date: Thu, 15 Feb 2007 21:41:44 +0200 Subject: [Distutils] c extensions: how to rebuild and test quickly In-Reply-To: <5.1.1.6.0.20070214190100.04fdc008@sparrow.telecommunity.com> References: <5.1.1.6.0.20070214190100.04fdc008@sparrow.telecommunity.com> Message-ID: <200702152141.45101.pogonyshev@gmx.net> Phillip J. Eby wrote: > At 01:30 AM 2/15/2007 +0200, Paul Pogonyshev wrote: > >Hi, > > > >Distutils create dynamic library for a C extension in `build/...' > >directory. This makes it impossible to run program without > >installing it, which I find very important for developing. Ideally, > >process should look like this: > [...] > > As someone else has mentioned, build_ext --inplace will do the trick. > > However, I find it a bit awkward to use, as you have to manually rebuild > when testing different Python versions (and/or platforms: I sometimes use > both Cygwin and Windows Python, for example). > > So, setuptools' "test" and "develop" commands (as well as its own version > of "build_ext --inplace") do the build in the build/ subdirectory, but then > *copy* the files to the source directory. Because each build directory has > its own platform-specific files, this automatically uses the right files > for the Python version you're running the "test" command with. > > It's still not perfect, but it can be a nice improvement over plain > "build_ext -i" usage. Yeah, I will do that if there is no better way. Paul From pogonyshev at gmx.net Thu Feb 15 20:43:34 2007 From: pogonyshev at gmx.net (Paul Pogonyshev) Date: Thu, 15 Feb 2007 21:43:34 +0200 Subject: [Distutils] c extensions: how to rebuild and test quickly In-Reply-To: References: <200702150130.10917.pogonyshev@gmx.net> Message-ID: <200702152143.34418.pogonyshev@gmx.net> Thomas Heller wrote: > Paul Pogonyshev schrieb: > > Hi, > > > > Distutils create dynamic library for a C extension in `build/...' > > directory. This makes it impossible to run program without > > installing it, which I find very important for developing. Ideally, > > process should look like this: > > [...] > > What I do is to create a loader module named .py that > 'replaces' itself with the real extension, loading it via imp > from the platform specific build directory that distutils creates. This looks clever. However, how do you make sure the module doesn't get itself installed? I.e. I'd not want this hack make it into the installed copy, while it is perfectly OK for quick in-place builds and tests. Paul From pje at telecommunity.com Thu Feb 15 21:04:38 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 15 Feb 2007 15:04:38 -0500 Subject: [Distutils] ctypes setuptools egg In-Reply-To: <536256.67040.qm@web30505.mail.mud.yahoo.com> Message-ID: <5.1.1.6.0.20070215150202.046ee4a0@sparrow.telecommunity.com> At 09:15 AM 2/15/2007 -0800, Venkat Bommakanti wrote: >Hi folks, > >I'd like to know if there is a source egg of ctypes There is no such thing as a "source egg". If you want to fetch an editable source distribution, simply use: easy_install -eb. ctypes And you'll get a ctypes directory in the current directory with the source. If you want to build an egg for your current platform and deposit it in the current directory, without installing scripts or dependencies, use: easy_install -zmxNd. ctypes >I need to be able to cross compile ctypes from an distutils simply doesn't support cross-compiling; I'm afraid you're on your own there. From pje at telecommunity.com Thu Feb 15 21:15:03 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 15 Feb 2007 15:15:03 -0500 Subject: [Distutils] setuptools and sdist on Mac OS X, tarfile flukes In-Reply-To: Message-ID: <5.1.1.6.0.20070215150738.05224e48@sparrow.telecommunity.com> At 05:38 PM 2/15/2007 +0000, Sidnei da Silva wrote: >Not sure if anyone reported a similar issue here since I'm not subscribed, but >here it goes. > >Some of the new plone.* eggs are being built on OS X by the developers, they >usually do 'setup.py sdist bdist_egg upload'. Sometimes they only do 'setup.py >sdist upload' > >Turns out that if you try to easy_install and there's only the sdist package, >but no egg, easy_install uses tarfile to unpack the package, build an egg and >install the egg. However, for some reason some of those nasty '._' >files that OS X creates end up in the tarball. And tarfile then fails to >unpack >the tarball. One such package is 'plone.app.controlpanel 1.0a2' >(http://cheeseshop.python.org/packages/source/p/plone.app.controlpanel/plone.app.controlpanel-1.0a2.tar.gz#md5=2b2afeaba0d067f6b1a6707c6d69311d), >with the following traceback: >... >I'm left wondering if the issue is: > > a) Windows > b) The nasty OS X file > c) The Python 'tarfile' module > d) setuptools > e) None of the above :) It appears to be a combination of "a" and "c". Windows Python 2.5 does not have an issue with the above file, nor does Cygwin Python 2.3. Windows Python 2.3 and 2.4 do have the problem. >Thoughts? Dunno. It sounds like the people making these distributions have an over-eager MANIFEST.in or an over-eager source control system. They should probably add a MANIFEST.in rule to exclude the files from the source release. From thomas at thomas-lotze.de Thu Feb 15 21:19:36 2007 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Thu, 15 Feb 2007 21:19:36 +0100 Subject: [Distutils] Buildout: Recipes with extras Message-ID: Automatic installation of eggs required by an egg extra doesn't seem to work for a recipe used with zc.buildout as documented for eggs in general: Suppose myrecipe has two entry points, the "fancy" one employing yourrecipe. Now if I declare yourrecipe as an extra requirement only to be installed if the "fancy" entry point is used to install any buildout part: myrecipe/setup.py: ... setup(... entry_points={"zc.buildout": [ "default = myrecipe.foo:Recipe", "fancy = myrecipe.bar:Recipe [bar_extra]", ]}, extras_require={"bar_extra": ["yourrecipe"]}, ... ) and use myrecipe I get a pkg_resources.DistributionNotFound error concerning yourrecipe. It sort of works if I manually copy the yourrecipe egg to the eggs directory beforehand, though. Is this a real bug, or are recipes supposed to be so simple as to not need extras? -- Viele Gr??e, Thomas From theller at ctypes.org Thu Feb 15 21:35:40 2007 From: theller at ctypes.org (Thomas Heller) Date: Thu, 15 Feb 2007 21:35:40 +0100 Subject: [Distutils] c extensions: how to rebuild and test quickly In-Reply-To: <200702152143.34418.pogonyshev@gmx.net> References: <200702150130.10917.pogonyshev@gmx.net> <200702152143.34418.pogonyshev@gmx.net> Message-ID: Paul Pogonyshev schrieb: > Thomas Heller wrote: >> Paul Pogonyshev schrieb: >> > Hi, >> > >> > Distutils create dynamic library for a C extension in `build/...' >> > directory. This makes it impossible to run program without >> > installing it, which I find very important for developing. Ideally, >> > process should look like this: >> > [...] >> >> What I do is to create a loader module named .py that >> 'replaces' itself with the real extension, loading it via imp >> from the platform specific build directory that distutils creates. > > This looks clever. However, how do you make sure the module doesn't > get itself installed? I.e. I'd not want this hack make it into the > installed copy, while it is perfectly OK for quick in-place builds > and tests. I'm not sure this trick works for every case, but it works for the ctypes separate distribution. The point is that ctypes is a package, and the ctypes package uses the _ctypes.pyd/so and _ctypes_test.pyd/.so extension modules. The extension modules are not inside the package, they are top-level modules. Here is the source layout in the SVN repository: ./setup.py ./_ctypes.py # the loader module ./_ctypes_test.py # the other loader module ./ctypes/__init__.py # the ctypes package ./ctypes/... # other parts of the package Now, when you call 'setup.py build_ext', the extension modules are built and put into this directory: ./build/lib.win32-2.4/_ctypes.pyd ./build/lib.win32-2.4/_ctypes_test.pyd The setup script contains these lines: setup(..., extensions = [Extension(_ctypes, ...), Extension(_ctypes_test, ...)], packages = ["ctypes", "ctypes.test", "ctypes.macholib"], ) so the loader script will not be included in the distributions. I am not sure if this recipe can be extended to support extensions that are inside packages; but for now I find it nicer than the setuptools 'develop' approach. Thomas From pje at telecommunity.com Thu Feb 15 21:40:36 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 15 Feb 2007 15:40:36 -0500 Subject: [Distutils] Buildout: Recipes with extras In-Reply-To: Message-ID: <5.1.1.6.0.20070215153439.05193730@sparrow.telecommunity.com> At 09:19 PM 2/15/2007 +0100, Thomas Lotze wrote: >Automatic installation of eggs required by an egg extra doesn't seem to >work for a recipe used with zc.buildout as documented for eggs in general: > >Suppose myrecipe has two entry points, the "fancy" one employing >yourrecipe. Now if I declare yourrecipe as an extra requirement only to be >installed if the "fancy" entry point is used to install any buildout part: > >myrecipe/setup.py: > >... > >setup(... > entry_points={"zc.buildout": [ > "default = myrecipe.foo:Recipe", > "fancy = myrecipe.bar:Recipe [bar_extra]", > ]}, > extras_require={"bar_extra": ["yourrecipe"]}, > ... > ) > >and use myrecipe I get a pkg_resources.DistributionNotFound error >concerning yourrecipe. It sort of works if I manually copy the yourrecipe >egg to the eggs directory beforehand, though. > >Is this a real bug, or are recipes supposed to be so simple as to not need >extras? Distributions required by entry points aren't automatically installed; the code loading the entry point has to pass an 'installer' function to the loading API in order to support installation-on-demand. Presumably zc.buildout doesn't do this. An example from setuptools' own code: for ep in pkg_resources.iter_entry_points('distutils.setup_keywords'): value = getattr(self,ep.name,None) if value is not None: ep.require(installer=self.fetch_build_egg) ep.load()(self, ep.name, value) This code loads entry points for any setup() keywords that match the entry point name, and in the process it will install any extras required by the entry point. In setuptools 0.7, I plan to add a feature to make it easier to do this, at least for pure-Python built eggs found on the CheeseShop. pkg_resources will have some class or function you'll be able to pass as the 'installer', and it will use CheesesShop XML-RPC to find and download the eggs to the PYTHON_EGG_CACHE directory. That way, people who want to support an easy and (relatively) safe install-on-demand feature can do so without having to write their own installer package. It won't offer all the bells-and-whistles of easy_install, but that's actually an advantage for this use case, methinks. From pogonyshev at gmx.net Thu Feb 15 22:01:57 2007 From: pogonyshev at gmx.net (Paul Pogonyshev) Date: Thu, 15 Feb 2007 23:01:57 +0200 Subject: [Distutils] c extensions: how to rebuild and test quickly In-Reply-To: References: <200702150130.10917.pogonyshev@gmx.net> <200702152143.34418.pogonyshev@gmx.net> Message-ID: <200702152301.57880.pogonyshev@gmx.net> Thomas Heller wrote: > Paul Pogonyshev schrieb: > > Thomas Heller wrote: > >> Paul Pogonyshev schrieb: > >> > Hi, > >> > > >> > Distutils create dynamic library for a C extension in `build/...' > >> > directory. This makes it impossible to run program without > >> > installing it, which I find very important for developing. Ideally, > >> > process should look like this: > >> > [...] > >> > >> What I do is to create a loader module named .py that > >> 'replaces' itself with the real extension, loading it via imp > >> from the platform specific build directory that distutils creates. > > > > This looks clever. However, how do you make sure the module doesn't > > get itself installed? I.e. I'd not want this hack make it into the > > installed copy, while it is perfectly OK for quick in-place builds > > and tests. > > I'm not sure this trick works for every case, but it works for the ctypes > separate distribution. > > The point is that ctypes is a package, and the ctypes package uses the > _ctypes.pyd/so and _ctypes_test.pyd/.so extension modules. The extension > modules are not inside the package, they are top-level modules. Here > is the source layout in the SVN repository: > > ./setup.py > ./_ctypes.py # the loader module > ./_ctypes_test.py # the other loader module > ./ctypes/__init__.py # the ctypes package > ./ctypes/... # other parts of the package > > Now, when you call 'setup.py build_ext', the extension modules > are built and put into this directory: > > ./build/lib.win32-2.4/_ctypes.pyd > ./build/lib.win32-2.4/_ctypes_test.pyd > > The setup script contains these lines: > > setup(..., > extensions = [Extension(_ctypes, ...), Extension(_ctypes_test, ...)], > packages = ["ctypes", "ctypes.test", "ctypes.macholib"], > ) > > so the loader script will not be included in the distributions. > > I am not sure if this recipe can be extended to support extensions > that are inside packages; but for now I find it nicer than the setuptools > 'develop' approach. > > Thomas Hm, looks too complicated. Thanks anyway. I ended up with custom build_ext command: from distutils.command.build_ext import build_ext as _build_ext class build_ext (_build_ext): def build_extension (self, extension): _build_ext.build_extension (self, extension) if not self.inplace and os.name == 'posix': filename = self.get_ext_filename (extension.name) try: os.remove (filename) except: # Ignore all errors. pass try: os.symlink (os.path.join (os.pardir, self.build_lib, filename), os.path.join ('notify', os.path.basename (filename))) except: # Ignore all errors. pass Maybe I should copy on non-posix systems. Paul From jim at zope.com Thu Feb 15 21:56:31 2007 From: jim at zope.com (Jim Fulton) Date: Thu, 15 Feb 2007 15:56:31 -0500 Subject: [Distutils] Buildout: Recipes with extras In-Reply-To: <5.1.1.6.0.20070215153439.05193730@sparrow.telecommunity.com> References: <5.1.1.6.0.20070215153439.05193730@sparrow.telecommunity.com> Message-ID: On Feb 15, 2007, at 3:40 PM, Phillip J. Eby wrote: > At 09:19 PM 2/15/2007 +0100, Thomas Lotze wrote: >> Automatic installation of eggs required by an egg extra doesn't >> seem to >> work for a recipe used with zc.buildout as documented for eggs in >> general: >> >> Suppose myrecipe has two entry points, the "fancy" one employing >> yourrecipe. Now if I declare yourrecipe as an extra requirement >> only to be >> installed if the "fancy" entry point is used to install any >> buildout part: >> >> myrecipe/setup.py: >> >> ... >> >> setup(... >> entry_points={"zc.buildout": [ >> "default = myrecipe.foo:Recipe", >> "fancy = myrecipe.bar:Recipe [bar_extra]", >> ]}, >> extras_require={"bar_extra": ["yourrecipe"]}, >> ... >> ) >> >> and use myrecipe I get a pkg_resources.DistributionNotFound error >> concerning yourrecipe. It sort of works if I manually copy the >> yourrecipe >> egg to the eggs directory beforehand, though. >> >> Is this a real bug, or are recipes supposed to be so simple as to >> not need >> extras? > > Distributions required by entry points aren't automatically > installed; the > code loading the entry point has to pass an 'installer' function to > the > loading API in order to support installation-on-demand. Presumably > zc.buildout doesn't do this. Thomas, I suggest you report this as a buildout bug at: https://bugs.launchpad.net/zc.buildout/+bugs Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From meine at informatik.uni-hamburg.de Fri Feb 16 11:51:38 2007 From: meine at informatik.uni-hamburg.de (Hans Meine) Date: Fri, 16 Feb 2007 11:51:38 +0100 Subject: [Distutils] Old bug still unfixed? [ python-Patches-1183712 ] package_data chops off first char of default package Message-ID: <200702161151.40363.meine@informatik.uni-hamburg.de> Hi! Yesterday, a student came to me complaining about an old distutils bug which bit him: http://mail.python.org/pipermail/patches/2006-August/020642.html The patch is quite trivial and cannot possibly break something AFAICS, so could someone please apply it? (Hmm, now that I write this; we are still using python 2.4.4 here, so maybe it has been fixed in between?) Ciao, / / /--/ / / ANS -------------- next part -------------- A non-text attachment was scrubbed... Name: distutils-rootpackage.diff Type: text/x-diff Size: 501 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20070216/87cf079e/attachment-0001.bin From thomas at thomas-lotze.de Fri Feb 16 18:15:13 2007 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Fri, 16 Feb 2007 18:15:13 +0100 Subject: [Distutils] Buildout: Recipes with extras References: <5.1.1.6.0.20070215153439.05193730@sparrow.telecommunity.com> Message-ID: Am Thu, 15 Feb 2007 15:56:31 -0500 schrieb Jim Fulton: > Thomas, I suggest you report this as a buildout bug at: > > https://bugs.launchpad.net/zc.buildout/+bugs Done. -- Viele Gr??e, Thomas From theller at ctypes.org Fri Feb 16 19:04:20 2007 From: theller at ctypes.org (Thomas Heller) Date: Fri, 16 Feb 2007 19:04:20 +0100 Subject: [Distutils] easy_install feature request Message-ID: The more I manage the third-party packages I have installed with easy_install, the more I like that I never have to fiddle around in the file system. (Except when I want to free the space that a no-longer needed package consumes, "easy_install -m " works fine). However (since I'm playing around with support for eggs in py2exe), it is not possible to reinstall an already installed (by it multi-version or not) package in zipped or unzipped form by using easy_install. I always have to delete the package in the file-system and then call 'easy_install -z|-Z package'. Thomas From pje at telecommunity.com Fri Feb 16 19:28:49 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 16 Feb 2007 13:28:49 -0500 Subject: [Distutils] easy_install feature request In-Reply-To: Message-ID: <5.1.1.6.0.20070216132542.02a821c8@sparrow.telecommunity.com> At 07:04 PM 2/16/2007 +0100, Thomas Heller wrote: >The more I manage the third-party packages I have installed with easy_install, >the more I like that I never have to fiddle around in the file >system. (Except >when I want to free the space that a no-longer needed package consumes, >"easy_install -m " works fine). > >However (since I'm playing around with support for eggs in py2exe), it is not >possible to reinstall an already installed (by it multi-version or not) >package >in zipped or unzipped form by using easy_install. I always have to delete >the package >in the file-system and then call 'easy_install -z|-Z package'. Patches welcome. :) Have a look at the 'install_egg' method of the easy_install class in setuptools.command.easy_install, which currently checks to see if the source and destination files are the same, in which case it does nothing. You would need to have it confirm that it actually wants to change something, and then rename the old file or something. It'd also need to actually have some code to zip up the original directory, since there is currently only code there to unpack. From thomas at thomas-lotze.de Sat Feb 17 01:01:46 2007 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Sat, 17 Feb 2007 01:01:46 +0100 Subject: [Distutils] Buildout: Recipes with extras References: <5.1.1.6.0.20070215153439.05193730@sparrow.telecommunity.com> Message-ID: Am Thu, 15 Feb 2007 15:40:36 -0500 schrieb Phillip J. Eby: > Distributions required by entry points aren't automatically installed; I see. > the > code loading the entry point has to pass an 'installer' function to the > loading API in order to support installation-on-demand. Presumably > zc.buildout doesn't do this. But neither does easy_install if I got this right: It's possible to install a distribution without its extra dependencies so that this distribution's scripts which need an extra fail when run. Why does easy_install install these scripts at all when it is to install the distribution without the respective necessary extras? > It won't offer all the bells-and-whistles of easy_install, but that's > actually an advantage for this use case, methinks. I wonder why all of the easy_install functionality isn't formulated as a package that others, including zc.buildout and your setuptools 0.7 installer function, can use. I wouldn't be surprised if the answer is that I should read some more of the sources, though ;o) -- Viele Gr??e, Thomas From pje at telecommunity.com Sat Feb 17 01:14:31 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 16 Feb 2007 19:14:31 -0500 Subject: [Distutils] Buildout: Recipes with extras In-Reply-To: References: <5.1.1.6.0.20070215153439.05193730@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070216191039.00a04748@sparrow.telecommunity.com> At 01:01 AM 2/17/2007 +0100, Thomas Lotze wrote: >It's possible to >install a distribution without its extra dependencies so that this >distribution's scripts which need an extra fail when run. Why does >easy_install install these scripts at all when it is to install the >distribution without the respective necessary extras? Because if it were to install the extras, this would mean they're not "extra" any more, so they should be in the project's main dependencies. It's not a perfect solution, but it seemed the least-damaging default. >I wonder why all of the easy_install functionality isn't formulated as a >package that others, including zc.buildout and your setuptools 0.7 >installer function, can use. Because it started out as an extremely short and relatively simple script and only gradually evolved into the gigantic mess it is today. 0.7 is when the refactoring back into reusable pieces will happen. From thomas at thomas-lotze.de Sat Feb 17 01:22:57 2007 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Sat, 17 Feb 2007 01:22:57 +0100 Subject: [Distutils] Buildout: Recipes with extras References: <5.1.1.6.0.20070215153439.05193730@sparrow.telecommunity.com> <5.1.1.6.0.20070216191039.00a04748@sparrow.telecommunity.com> Message-ID: Am Fri, 16 Feb 2007 19:14:31 -0500 schrieb Phillip J. Eby: > Because if it were to install the extras, this would mean they're not > "extra" any more, so they should be in the project's main dependencies. > It's not a perfect solution, but it seemed the least-damaging default. No, my suggestion was not to install extras by default (which of course would lead them ad absurdum) but to not install those scripts that depend on extras not being installed. > Because it started out as an extremely short and relatively simple script > and only gradually evolved into the gigantic mess it is today. 0.7 is > when the refactoring back into reusable pieces will happen. Good thing :o) -- Viele Gr??e, Thomas From matt at matt-good.net Sat Feb 17 18:00:28 2007 From: matt at matt-good.net (Matt Good) Date: Sat, 17 Feb 2007 12:00:28 -0500 Subject: [Distutils] is_python should catch TypeError Message-ID: <1171731628.6069.110.camel@nny> I just got an error trying to upgrade SilverCity from easy_install: File "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/setuptools/command/easy_install.py", line 1462, in is_python compile(text, filename, 'exec') TypeError: compile() expected string without null bytes You can reproduce it easily: >>> from setuptools.command.easy_install import is_python >>> is_python('\0') The fix is simply to add TypeError to the except clause along with SyntaxError. -- Matt Good From pje at telecommunity.com Sat Feb 17 18:57:06 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 17 Feb 2007 12:57:06 -0500 Subject: [Distutils] is_python should catch TypeError In-Reply-To: <1171731628.6069.110.camel@nny> Message-ID: <5.1.1.6.0.20070217125649.045f3c98@sparrow.telecommunity.com> At 12:00 PM 2/17/2007 -0500, Matt Good wrote: >I just got an error trying to upgrade SilverCity from easy_install: > > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/setuptools/command/easy_install.py", > line 1462, in is_python > compile(text, filename, 'exec') >TypeError: compile() expected string without null bytes > >You can reproduce it easily: > >>> from setuptools.command.easy_install import is_python > >>> is_python('\0') > >The fix is simply to add TypeError to the except clause along with >SyntaxError. Added for 0.6c6. Thanks! From ianb at colorstudy.com Tue Feb 20 18:10:01 2007 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 20 Feb 2007 11:10:01 -0600 Subject: [Distutils] Setuptools installed as zip file Message-ID: <45DB2B69.1070102@colorstudy.com> I've received a few reports of setuptools getting installed (via ez_setup.py) as a zip file (maybe just on Python 2.5). Is this something new? Should I expect this possibility? (Or I can just use --always-unzip with ez_setup.py to make sure it's always installed the same way.) -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From hinsen at cnrs-orleans.fr Tue Feb 20 17:09:46 2007 From: hinsen at cnrs-orleans.fr (Konrad Hinsen) Date: Tue, 20 Feb 2007 17:09:46 +0100 Subject: [Distutils] C extensions with non-uniform compiler options Message-ID: <16771503-51EB-4244-ABCE-E7A2B0570134@cnrs-orleans.fr> I need to build a C extension consisting of several C code files, one of which needs to be compiled with different compiler options. There doesn't seem to be a standard way to do this. As a workaround I added a dummy extension that uses that file, and specified the necessary compiler options for that extension. Then I removed the source file from the file list of the real extension and replaced it by an entry in extra_objects. The idea is the re-use the version compiled for the dummy extension module. However, there are two problems: 1) The documentation for extra_objects states that "These files must not have extensions, as the default extension for the compiler is used." That would be nice, but isn't true: on my Mac, the required ".o" is not added. 2) The object file I wish to add has been created in "build/ temp.macosx-10.3-fat-2.5", but this directory is machine-specific. I need to construct this path in a portable way - but how? Did anyone already solve these problems, or find another solution? Thanks in advance, Konrad. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2745 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20070220/20c10a02/attachment.bin From pje at telecommunity.com Tue Feb 20 18:51:13 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 20 Feb 2007 12:51:13 -0500 Subject: [Distutils] Setuptools installed as zip file In-Reply-To: <45DB2B69.1070102@colorstudy.com> Message-ID: <5.1.1.6.0.20070220124820.028aed28@sparrow.telecommunity.com> At 11:10 AM 2/20/2007 -0600, Ian Bicking wrote: >I've received a few reports of setuptools getting installed (via >ez_setup.py) as a zip file (maybe just on Python 2.5). Is this >something new? It's installed as a zipfile under Python 2.5, becuase Python 2.5 supports "-m" being used with zipped modules. > Should I expect this possibility? Yes, since it's the standard behavior. :) Actually, it should probably do this under Python 2.3 as well, since -m isn't relevant there. 2.4 is probably the only version where it shouldn't be installed zipped. > (Or I can just use >--always-unzip with ez_setup.py to make sure it's always installed the >same way.) Probably. From ianb at colorstudy.com Tue Feb 20 23:04:47 2007 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 20 Feb 2007 16:04:47 -0600 Subject: [Distutils] workingenv/setuptools compatibility Message-ID: <45DB707F.8050005@colorstudy.com> Hi again. I mentioned this a long time ago, but I'd like to talk about it again: I'd really like to remove workingenv's monkeypatching of setuptools. There's still just two things: * Removing easy_install's check of site.py (easy_install.install_site_py). * Changing the way scripts are created (using python -S and then changing the path). An option I could put in distutils.cfg for changing script creation would work great (since workingenv installs its own distutils.cfg). Maybe something that points to a function, and that function is passed the normal script text and then modifies it. Or just wraps get_script_header's text (that's what I'm currently doing). For setuptools' site.py, it would be nice if I could put a bit of text in there that indicates that it does everything setuptools' site.py does (at least as much as I choose to do -- no finding the system site.py in my case). Maybe something like __setuptools_setup_version__ = '0.6c6', or 'setuptools_site_py_hash = "xxxxx"', where that's an md5 hash of the setuptools' site.py that it was based on. Or something. This would substantially reduce the amount of annoying hacks in workingenv. And the script stuff would be useful in zc.buildout as well. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From pje at telecommunity.com Tue Feb 20 23:42:22 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 20 Feb 2007 17:42:22 -0500 Subject: [Distutils] workingenv/setuptools compatibility In-Reply-To: <45DB707F.8050005@colorstudy.com> Message-ID: <5.1.1.6.0.20070220173401.0295e790@sparrow.telecommunity.com> At 04:04 PM 2/20/2007 -0600, Ian Bicking wrote: >Hi again. I mentioned this a long time ago, but I'd like to talk about >it again: I'd really like to remove workingenv's monkeypatching of >setuptools. Then remove it. :) You can trivially replace any setuptools built-in commands by simply ensuring that you have corresponding 'distutils.commands' entry points in the working set ahead of setuptools. An easy way to do this in workingenv would be to place your own egg in the easy-install.pth, and removing setuptools' egg from there (leaving it in setuptools.pth). This will then let you override the 'easy_install' and 'develop' commands, to do whatever you want them to. The only pieces this won't affect are the way setup_requires and tests_require get handled (as they use direct imports from setuptools.commands.easy_install) and the way --single-version-externally-managed installs get their scripts generated (which also uses direct imports). However, if I understand correctly, you don't need either of these pieces. Anyway, to do all this, you will need to have an egg that lists entry points for 'easy_install' and 'develop' under the 'distutils.commands' group. (Actually, you might want to include an entry point for 'install' as well, and override the do_egg_install method, which currently imports easy_install directly instead of using self.distribution.get_command_class('easy_install'). I'll fix that in 0.6c6.) From ianb at colorstudy.com Wed Feb 21 19:52:44 2007 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 21 Feb 2007 12:52:44 -0600 Subject: [Distutils] easy_install: one more log message Message-ID: <45DC94FC.3040909@colorstudy.com> I realize if you add a message to the end of easy_install.process_distribution that says that the package is finished installing, you can figure out just what exactly is requiring what else. This is hugely useful to debugging just why different pieces get installed, and if there is a conflict. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From ianb at colorstudy.com Wed Feb 21 21:38:05 2007 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 21 Feb 2007 14:38:05 -0600 Subject: [Distutils] --always-copy, development eggs Message-ID: <45DCADAD.6020303@colorstudy.com> So, I'm trying to achieve a couple different things with workingenv, and I'm not sure how to make it all work... I don't want one workingenv to have unintentional links to package in another environment. Sometimes easy_install will find a package somewhere else on the system, which is fine, but I want it to copy it into the workingenv. I thought the easy thing was to add "[easy_install] always_copy = True" to distutils.cfg. And this works. *But* always_copy doesn't work with develop eggs. So specifically there's a problem if I want to install both distribution A and B as editable/develop eggs, and B depends on A. I can install (via setup.py develop) A, and then install B, but B fails with something like error: Could not find suitable distribution for Requirement.parse('A==0.1') (--always-copy skips system and development eggs) Note that 'A==0.1' is something easy_install/setuptools came up with, B only requires 'A'. So, it seems having always_copy on really gets in the way of development eggs. Is there another way I can avoid sharing packages without always_copy? Or... something, I'm not sure. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From newsgroups at debain.org Fri Feb 23 13:30:48 2007 From: newsgroups at debain.org (newsgroups at debain.org) Date: Fri, 23 Feb 2007 13:30:48 +0100 Subject: [Distutils] Package name != filename, and: Python version information Message-ID: <20070223123048.178AAB3BDC@mail.speedpartner.de> Hi, With the following in my setup script: ------------- from distutils.core import setup setup(name = 'Spiff WikiMarkup', [...] requires = ['Plex'], provides = ['WikiMarkup'], [...] ------------- 1. Distutils generates a tarball named "Spiff WikiMarkup-0.1.tar.gz". How can I replace the space in the filename by an underscore? 2. In Python cheese shop, there is a field that shows the Python version that is required by a listed package. I have not found this covered in the documentation: How do you add the Python version information into the setup script of a package? Also, is there a complete keyword list somewhere? This list: http://docs.python.org/dist/meta-data.html seems to contain only few of the keywords available (for example, the "provides" and "requires" keywords are not listed, even though covered in the previous chapters of the documentation). Thanks, -Samuel From meine at informatik.uni-hamburg.de Fri Feb 23 14:46:46 2007 From: meine at informatik.uni-hamburg.de (Hans Meine) Date: Fri, 23 Feb 2007 14:46:46 +0100 Subject: [Distutils] Old bug still unfixed? [ python-Patches-1183712 ] package_data chops off first char of default package In-Reply-To: <200702161151.40363.meine@informatik.uni-hamburg.de> References: <200702161151.40363.meine@informatik.uni-hamburg.de> Message-ID: <200702231446.47791.meine@informatik.uni-hamburg.de> Hi! Am Freitag, 16. Februar 2007 11:51 schrieb Hans Meine: > The patch is quite trivial and cannot possibly break something AFAICS, so > could someone please apply it? (Hmm, now that I write this; we are still > using python 2.4.4 here, so maybe it has been fixed in between?) I wonder why nobody replied to any of my mails - I sent this simple patch and another slightly more complex one for handling the she-bang in script files: http://mail.python.org/pipermail/distutils-sig/2007-February/007306.html Finally, there was the shared library question, but I can very well imagine that no one had a solution for that one.. -- Ciao, / / /--/ / / ANS From lars at ibp.de Fri Feb 23 15:11:33 2007 From: lars at ibp.de (Lars Immisch) Date: Fri, 23 Feb 2007 15:11:33 +0100 Subject: [Distutils] Old bug still unfixed? [ python-Patches-1183712 ] package_data chops off first char of default package In-Reply-To: <200702231446.47791.meine@informatik.uni-hamburg.de> References: <200702161151.40363.meine@informatik.uni-hamburg.de> <200702231446.47791.meine@informatik.uni-hamburg.de> Message-ID: <45DEF615.1020409@ibp.de> Hi Hans, > Am Freitag, 16. Februar 2007 11:51 schrieb Hans Meine: >> The patch is quite trivial and cannot possibly break something AFAICS, so >> could someone please apply it? (Hmm, now that I write this; we are still >> using python 2.4.4 here, so maybe it has been fixed in between?) > > I wonder why nobody replied to any of my mails - I sent this simple patch and > another slightly more complex one for handling the she-bang in script files: > http://mail.python.org/pipermail/distutils-sig/2007-February/007306.html > > Finally, there was the shared library question, but I can very well imagine > that no one had a solution for that one.. Well, this is the distutils group more by name than by content; distutils is a bit neglected. Anyway, your patch looks good to me. I'd suggest taking the issue to python-dev: you need to find someone with cvs write access to get your patch in. - Lars From pje at telecommunity.com Fri Feb 23 17:27:16 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 23 Feb 2007 11:27:16 -0500 Subject: [Distutils] Package name != filename, and: Python version information In-Reply-To: <20070223123048.178AAB3BDC@mail.speedpartner.de> Message-ID: <5.1.1.6.0.20070223112415.02900c48@sparrow.telecommunity.com> At 01:30 PM 2/23/2007 +0100, newsgroups at debain.org wrote: >Hi, > >With the following in my setup script: > >------------- >from distutils.core import setup >setup(name = 'Spiff WikiMarkup', > [...] > requires = ['Plex'], > provides = ['WikiMarkup'], > [...] >------------- > >1. Distutils generates a tarball named "Spiff WikiMarkup-0.1.tar.gz". How >can I >replace the space in the filename by an underscore? Setuptools fixes this, and various other problems with distribution filenames. Unfortunately, changing this behavior in the distutils would result in backward incompatibilities for people who are relying on its current behavior. >2. In Python cheese shop, there is a field that shows the Python version >that is required by a listed package. > >I have not found this covered in the documentation: How do you add the Python >version information into the setup script of a package? > >Also, is there a complete keyword list somewhere? This list: > >http://docs.python.org/dist/meta-data.html > >seems to contain only few of the keywords available (for example, the >"provides" >and "requires" keywords are not listed, even though covered in the previous >chapters of the documentation). I don't know the answer to these questions, but perhaps someone else here might. From theller at ctypes.org Fri Feb 23 20:44:12 2007 From: theller at ctypes.org (Thomas Heller) Date: Fri, 23 Feb 2007 20:44:12 +0100 Subject: [Distutils] slow Python startup with a couple of easy_install-ed packages Message-ID: Well, easy_install makes it snap to install packages that I may or may not use. Nothing is easier than to 'easy_install something-recently-announced', even if I never will use it (after a preliminary look). Often I forget to uninstall the packages again, or promise to do that later. Disk space is cheap. However, I found that with the ~20 eggs that I have installed the startup time of the Python interpreter (Python2.5, WinXP, measured with 'timeit python -c ""') rises considerably - 2.5 times slower. This only occurs when the eggs are listed in the easy_install.pth file; if they are moved into a 'normal' pth file this slowdown does not occur, although I can still import the eggs fine. I assume the additional time is spent by setuptools looking up entry points or whatever? And that this additional processing is not needed for the eggs that do not provide entry points? Or are not *using* setuptools themselves? If this is true, wouldn't it be possible to skip the processing for those eggs? IMO this is a critical problem of easy_install... (Imaging how the installed and possibly never used eggs pile up). Thomas From newsgroups at debain.org Tue Feb 20 08:52:52 2007 From: newsgroups at debain.org (Samuel Abels) Date: Tue, 20 Feb 2007 08:52:52 +0100 Subject: [Distutils] Package name != filename, and: Python version information In-Reply-To: <5.1.1.6.0.20070223112415.02900c48@sparrow.telecommunity.com> References: <5.1.1.6.0.20070223112415.02900c48@sparrow.telecommunity.com> Message-ID: <1171957972.5356.72.camel@localhost.localdomain> On Fri, 2007-02-23 at 11:27 -0500, Phillip J. Eby wrote: > Setuptools fixes this, and various other problems with distribution > filenames. Unfortunately, changing this behavior in the distutils would > result in backward incompatibilities for people who are relying on its > current behavior. Thanks, I was not aware of setuptools. Unfortunately, my script has to run on a hosted server where I can not install this package. Is there any workaround that I could use? -Samuel From pje at telecommunity.com Fri Feb 23 21:31:59 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 23 Feb 2007 15:31:59 -0500 Subject: [Distutils] --always-copy, development eggs In-Reply-To: <45DCADAD.6020303@colorstudy.com> Message-ID: <5.1.1.6.0.20070223135401.04bcca30@sparrow.telecommunity.com> At 02:38 PM 2/21/2007 -0600, Ian Bicking wrote: >So, I'm trying to achieve a couple different things with workingenv, and >I'm not sure how to make it all work... > >I don't want one workingenv to have unintentional links to package in >another environment. Sometimes easy_install will find a package >somewhere else on the system, which is fine, but I want it to copy it >into the workingenv. I think the problem is that you haven't sufficiently specified what you mean by "copy it into the workingenv". For example, do you want to install a snapshot by creating an egg, or do you want to copy an .egg-link file? For the use case --always-copy was created for, the former is what's desired, and it may someday implement that by running the setup.py. The problem is that right now there's no way to know for sure *which* setup.py created the development egg. One can certainly *guess* that the first setup.py found in the same or a higher-level directory than that containing the .egg-info is the correct one, but it may not always be the case. Some packages have a separate setup.py (that's not *named* setup.py) that would need to be used. I could put something in the .egg-info directory, but it would need to *not* be installed by system packaging tools, because then it would no longer be pointing to the right place. So, if I were to store a link to the setup.py, it would need to be stored in the .egg-link file itself, where easy_install could potentially get to it. As an experiment, I just tried changing the way "develop" and package_index work so that setup scripts can be found automatically, and the appropriate bdist_egg step run. It seems to work okay, except that I'm not sure if we really *want* development eggs to be automatically snapshotted like this, without including some sort of extra "local builds okay" flag. So, the trunk and 0.6 branch now include a --local-snapshots-ok flag you can use to enable this behavior. I don't know if it's exactly what you're looking for, but it at least provides you with one way of handling it. From pje at telecommunity.com Fri Feb 23 21:42:45 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 23 Feb 2007 15:42:45 -0500 Subject: [Distutils] slow Python startup with a couple of easy_install-ed packages In-Reply-To: Message-ID: <5.1.1.6.0.20070223153228.04dc89f0@sparrow.telecommunity.com> At 08:44 PM 2/23/2007 +0100, Thomas Heller wrote: >Well, easy_install makes it snap to install packages that I may or may >not use. Nothing is easier than to 'easy_install >something-recently-announced', >even if I never will use it (after a preliminary look). Often I forget to >uninstall the packages again, or promise to do that later. Disk space is >cheap. > >However, I found that with the ~20 eggs that I have installed the startup time >of the Python interpreter (Python2.5, WinXP, measured with 'timeit python >-c ""') >rises considerably - 2.5 times slower. > >This only occurs when the eggs are listed in the easy_install.pth file; if >they >are moved into a 'normal' pth file this slowdown does not occur, although >I can >still import the eggs fine. > >I assume the additional time is spent by setuptools looking up entry points or >whatever? No. What's happening is that easy-install.pth puts the eggs at the *beginning* of sys.path, in order to ensure that eggs override non-egg packages; this prevents a *huge* amount of complication in sys.path management with respect to detecting conflicts, as well as allowing eggs to be installed that override the standard library (e.g. to allow hotfixing it). The downside is that this means eggs are searched before the stdlib, slowing down the import of stdlib modules at startup. The easy fix is to use the "-m" option, which doesn't put the eggs in any .pth file at all -- and removes them if they're there. >IMO this is a critical problem of easy_install... (Imaging how the >installed and >possibly never used eggs pile up). How many of these eggs are directories, vs. how many are zipfiles? Zipfiles shouldn't be having so much effect. Meanwhile, in setuptools 0.7 there will be a completely different way of managing installation locations, and it will not depend on .pth files or on putting eggs first. Instead, it will use direct old-style installation, combined with .egg-info directories for metadata and for installation manifests to support automated uninstallation, upgrades, and integrity verification. From pje at telecommunity.com Fri Feb 23 21:43:32 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 23 Feb 2007 15:43:32 -0500 Subject: [Distutils] easy_install: one more log message In-Reply-To: <45DC94FC.3040909@colorstudy.com> Message-ID: <5.1.1.6.0.20070223154324.0776d6c0@sparrow.telecommunity.com> At 12:52 PM 2/21/2007 -0600, Ian Bicking wrote: >I realize if you add a message to the end of >easy_install.process_distribution that says that the package is finished >installing, you can figure out just what exactly is requiring what else. Done. From pje at telecommunity.com Fri Feb 23 21:47:41 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 23 Feb 2007 15:47:41 -0500 Subject: [Distutils] Package name != filename, and: Python version information In-Reply-To: <1171957972.5356.72.camel@localhost.localdomain> References: <5.1.1.6.0.20070223112415.02900c48@sparrow.telecommunity.com> <5.1.1.6.0.20070223112415.02900c48@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070223154351.0775c2f8@sparrow.telecommunity.com> At 08:52 AM 2/20/2007 +0100, Samuel Abels wrote: >On Fri, 2007-02-23 at 11:27 -0500, Phillip J. Eby wrote: > > Setuptools fixes this, and various other problems with distribution > > filenames. Unfortunately, changing this behavior in the distutils would > > result in backward incompatibilities for people who are relying on its > > current behavior. > >Thanks, I was not aware of setuptools. > >Unfortunately, my script has to run on a hosted server where I can not >install this package. setuptools can be installed to any PYTHONPATH directory; see: http://peak.telecommunity.com/DevCenter/EasyInstall#custom-installation-locations Especially the sections on "Creating a Virtual Python" and "Traditional PYTHONPATH-based installation". The "Creating a Virtual Python" may be especially useful to you, as it creates a virtual Python installation that gives you complete access to site-packages and the ability to install whatever packages you want, whether they're setuptools-based or not. > Is there any workaround that I could use? For the file-naming problem, I don't know of any, no. For the "can't install packages on a hosted server" problem, see the above for a workaround. :) From newsgroups at debain.org Tue Feb 20 09:39:36 2007 From: newsgroups at debain.org (Samuel Abels) Date: Tue, 20 Feb 2007 09:39:36 +0100 Subject: [Distutils] Package name != filename, and: Python version information In-Reply-To: <5.1.1.6.0.20070223154351.0775c2f8@sparrow.telecommunity.com> References: <5.1.1.6.0.20070223112415.02900c48@sparrow.telecommunity.com> <5.1.1.6.0.20070223112415.02900c48@sparrow.telecommunity.com> <5.1.1.6.0.20070223154351.0775c2f8@sparrow.telecommunity.com> Message-ID: <1171960776.5356.77.camel@localhost.localdomain> On Fri, 2007-02-23 at 15:47 -0500, Phillip J. Eby wrote: > setuptools can be installed to any PYTHONPATH directory Alright, I installed setuptools and it works so far. Except, when I build a package using python2.5 setup.py sdist the filename still contains the space character. Perhaps I missed it in the documentation, but I have not found a way to specify the filename separately from the name. Any hints? -Samuel From pje at telecommunity.com Fri Feb 23 22:38:58 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 23 Feb 2007 16:38:58 -0500 Subject: [Distutils] Package name != filename, and: Python version information In-Reply-To: <1171960776.5356.77.camel@localhost.localdomain> References: <5.1.1.6.0.20070223154351.0775c2f8@sparrow.telecommunity.com> <5.1.1.6.0.20070223112415.02900c48@sparrow.telecommunity.com> <5.1.1.6.0.20070223112415.02900c48@sparrow.telecommunity.com> <5.1.1.6.0.20070223154351.0775c2f8@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070223163746.07760820@sparrow.telecommunity.com> At 09:39 AM 2/20/2007 +0100, Samuel Abels wrote: >On Fri, 2007-02-23 at 15:47 -0500, Phillip J. Eby wrote: > > setuptools can be installed to any PYTHONPATH directory > >Alright, I installed setuptools and it works so far. Except, when I >build a package using > >python2.5 setup.py sdist > >the filename still contains the space character. Perhaps I missed it in >the documentation, but I have not found a way to specify the filename >separately from the name. Any hints? Oops. It appears I mis-spoke. For backward-compatibility reasons, setuptools only fixes this filename issue when building eggs, not source distributions. I got confused. Sorry for leading you on a wild-goose chase. :( From newsgroups at debain.org Tue Feb 20 10:03:56 2007 From: newsgroups at debain.org (Samuel Abels) Date: Tue, 20 Feb 2007 10:03:56 +0100 Subject: [Distutils] Package name != filename, and: Python version information In-Reply-To: <5.1.1.6.0.20070223163746.07760820@sparrow.telecommunity.com> References: <5.1.1.6.0.20070223154351.0775c2f8@sparrow.telecommunity.com> <5.1.1.6.0.20070223112415.02900c48@sparrow.telecommunity.com> <5.1.1.6.0.20070223112415.02900c48@sparrow.telecommunity.com> <5.1.1.6.0.20070223154351.0775c2f8@sparrow.telecommunity.com> <5.1.1.6.0.20070223163746.07760820@sparrow.telecommunity.com> Message-ID: <1171962236.5356.81.camel@localhost.localdomain> On Fri, 2007-02-23 at 16:38 -0500, Phillip J. Eby wrote: > Oops. It appears I mis-spoke. For backward-compatibility reasons, > setuptools only fixes this filename issue when building eggs, not source > distributions. I am still unsure; does this mean that there is no way for uploading packages whose name have a space in them to PyPi? -Samuel From pje at telecommunity.com Fri Feb 23 22:54:42 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 23 Feb 2007 16:54:42 -0500 Subject: [Distutils] Package name != filename, and: Python version information In-Reply-To: <1171962236.5356.81.camel@localhost.localdomain> References: <5.1.1.6.0.20070223163746.07760820@sparrow.telecommunity.com> <5.1.1.6.0.20070223154351.0775c2f8@sparrow.telecommunity.com> <5.1.1.6.0.20070223112415.02900c48@sparrow.telecommunity.com> <5.1.1.6.0.20070223112415.02900c48@sparrow.telecommunity.com> <5.1.1.6.0.20070223154351.0775c2f8@sparrow.telecommunity.com> <5.1.1.6.0.20070223163746.07760820@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070223165307.04dd3628@sparrow.telecommunity.com> At 10:03 AM 2/20/2007 +0100, Samuel Abels wrote: >On Fri, 2007-02-23 at 16:38 -0500, Phillip J. Eby wrote: > > Oops. It appears I mis-spoke. For backward-compatibility reasons, > > setuptools only fixes this filename issue when building eggs, not source > > distributions. > >I am still unsure; does this mean that there is no way for uploading >packages whose name have a space in them to PyPi? Did you try "setup.py sdist upload"? I don't know if PyPI restricts filenames or not. By the way, your mail client seems to think that today is the 20th - you might want to check your clock settings. From newsgroups at debain.org Fri Feb 23 23:04:31 2007 From: newsgroups at debain.org (Samuel Abels) Date: Fri, 23 Feb 2007 23:04:31 +0100 Subject: [Distutils] Package name != filename, and: Python version information In-Reply-To: <5.1.1.6.0.20070223165307.04dd3628@sparrow.telecommunity.com> References: <5.1.1.6.0.20070223163746.07760820@sparrow.telecommunity.com> <5.1.1.6.0.20070223154351.0775c2f8@sparrow.telecommunity.com> <5.1.1.6.0.20070223112415.02900c48@sparrow.telecommunity.com> <5.1.1.6.0.20070223112415.02900c48@sparrow.telecommunity.com> <5.1.1.6.0.20070223154351.0775c2f8@sparrow.telecommunity.com> <5.1.1.6.0.20070223163746.07760820@sparrow.telecommunity.com> <5.1.1.6.0.20070223165307.04dd3628@sparrow.telecommunity.com> Message-ID: <1172268271.5356.88.camel@localhost.localdomain> On Fri, 2007-02-23 at 16:54 -0500, Phillip J. Eby wrote: > At 10:03 AM 2/20/2007 +0100, Samuel Abels wrote: > >On Fri, 2007-02-23 at 16:38 -0500, Phillip J. Eby wrote: > > > Oops. It appears I mis-spoke. For backward-compatibility reasons, > > > setuptools only fixes this filename issue when building eggs, not source > > > distributions. > > > >I am still unsure; does this mean that there is no way for uploading > >packages whose name have a space in them to PyPi? > > Did you try "setup.py sdist upload"? I don't know if PyPI restricts > filenames or not. Well, the file is made available (with the whitespace character included), but things like the Cheesecake service (and, according to the Cheesecake service, easy_install) won't work. Will there be a way to change the filename in future releases? > By the way, your mail client seems to think that today is the 20th - you > might want to check your clock settings. Sorry, I fixed it. -Samuel From pje at telecommunity.com Fri Feb 23 23:46:00 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 23 Feb 2007 17:46:00 -0500 Subject: [Distutils] Package name != filename, and: Python version information In-Reply-To: <1172268271.5356.88.camel@localhost.localdomain> References: <5.1.1.6.0.20070223165307.04dd3628@sparrow.telecommunity.com> <5.1.1.6.0.20070223163746.07760820@sparrow.telecommunity.com> <5.1.1.6.0.20070223154351.0775c2f8@sparrow.telecommunity.com> <5.1.1.6.0.20070223112415.02900c48@sparrow.telecommunity.com> <5.1.1.6.0.20070223112415.02900c48@sparrow.telecommunity.com> <5.1.1.6.0.20070223154351.0775c2f8@sparrow.telecommunity.com> <5.1.1.6.0.20070223163746.07760820@sparrow.telecommunity.com> <5.1.1.6.0.20070223165307.04dd3628@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070223174424.04a0fe60@sparrow.telecommunity.com> At 11:04 PM 2/23/2007 +0100, Samuel Abels wrote: >On Fri, 2007-02-23 at 16:54 -0500, Phillip J. Eby wrote: > > At 10:03 AM 2/20/2007 +0100, Samuel Abels wrote: > > >On Fri, 2007-02-23 at 16:38 -0500, Phillip J. Eby wrote: > > > > Oops. It appears I mis-spoke. For backward-compatibility reasons, > > > > setuptools only fixes this filename issue when building eggs, not > source > > > > distributions. > > > > > >I am still unsure; does this mean that there is no way for uploading > > >packages whose name have a space in them to PyPi? > > > > Did you try "setup.py sdist upload"? I don't know if PyPI restricts > > filenames or not. > >Well, the file is made available (with the whitespace character >included), but things like the Cheesecake service (and, according to the >Cheesecake service, easy_install) won't work. Have you actually tried easy_install on it? You will need to include quote marks on your command line, but it should work nonetheless. If that doesn't work, it's an easy_install bug; please send what you typed and the full easy_install output. But if it *does* work, then it's a Cheesecake bug that it says it doesn't work. :) >Will there be a way to change the filename in future releases? I don't have any plans to do so, no. From constant.beta at gmail.com Sat Feb 24 02:41:10 2007 From: constant.beta at gmail.com (=?ISO-8859-2?Q?Micha=B3_Kwiatkowski?=) Date: Sat, 24 Feb 2007 02:41:10 +0100 Subject: [Distutils] Package name != filename, and: Python version information In-Reply-To: <5.1.1.6.0.20070223174424.04a0fe60@sparrow.telecommunity.com> References: <5.1.1.6.0.20070223112415.02900c48@sparrow.telecommunity.com> <5.1.1.6.0.20070223154351.0775c2f8@sparrow.telecommunity.com> <5.1.1.6.0.20070223163746.07760820@sparrow.telecommunity.com> <5.1.1.6.0.20070223165307.04dd3628@sparrow.telecommunity.com> <1172268271.5356.88.camel@localhost.localdomain> <5.1.1.6.0.20070223174424.04a0fe60@sparrow.telecommunity.com> Message-ID: <5e8b0f6b0702231741v5498f58ch2cea5f0981485149@mail.gmail.com> On 2/23/07, Phillip J. Eby wrote: > At 11:04 PM 2/23/2007 +0100, Samuel Abels wrote: > >Well, the file is made available (with the whitespace character > >included), but things like the Cheesecake service (and, according to the > >Cheesecake service, easy_install) won't work. > > Have you actually tried easy_install on it? You will need to include > quote marks on your command line, but it should work nonetheless. If > that doesn't work, it's an easy_install bug; please send what you typed > and the full easy_install output. But if it *does* work, then it's a > Cheesecake bug that it says it doesn't work. :) Let me jump in here. $ sudo easy_install -v "Spiff WikiMarkup" error: Not a URL, existing file, or requirement spec: 'Spiff WikiMarkup' $ sudo easy_install -v Spiff\ WikiMarkup error: Not a URL, existing file, or requirement spec: 'Spiff WikiMarkup' $ sudo easy_install -v 'Spiff WikiMarkup' error: Not a URL, existing file, or requirement spec: 'Spiff WikiMarkup' $ sudo easy_install -v Spiff_WikiMarkup Searching for Spiff-WikiMarkup Reading http://cheeseshop.python.org/pypi/Spiff_WikiMarkup/ Reading http://cheeseshop.python.org/pypi/Spiff-WikiMarkup/ Couldn't find index page for 'Spiff_WikiMarkup' (maybe misspelled?) Scanning index of all packages (this may take a while) Reading http://cheeseshop.python.org/pypi/ Reading http://cheeseshop.python.org/pypi/Spiff%20WikiMarkup/0.1 Reading http://code.google.com/p/spiff/ No local packages or download links found for Spiff-WikiMarkup error: Could not find suitable distribution for Requirement.parse('Spiff-WikiMarkup') So it seems to be easy_install bug. And again: easy_install have no --version option. Please fix this. I'm using setuptools-0.6c5. Cheers, mk From ziade.tarek at gmail.com Sat Feb 24 21:47:32 2007 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 24 Feb 2007 14:47:32 -0600 Subject: [Distutils] unicode bug in distutils Message-ID: <94bdd2610702241247t568a942dw2fe1b10883b62d20@mail.gmail.com> Hello, I have created a setup.py file for distirbution and I bumped into a small bug when i tried to set my name in the contact field (Tarek Ziad?) Using string (utf8 file): setup( maintainer="Tarek Ziad?" ) leads to: File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/command/register.py", line 162, in send_metadata auth) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/command/register.py", line 257, in post_to_server value = unicode(value).encode("utf-8") UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 10: ordinal not in range(128) Using unicode: setup( maintainer=u"Tarek Ziad?" ) leads to: File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 1094, in write_pkg_file file.write('Author: %s\n' % self.get_contact() ) UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 18: ordinal not in range(128) I would propose a patch for this problem but i don't know what would be the best input (i guess unicode for names) Regards Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070224/802b4927/attachment.htm From ziade.tarek at gmail.com Sat Feb 24 21:51:43 2007 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 24 Feb 2007 14:51:43 -0600 Subject: [Distutils] registration errors, no feedback Message-ID: <94bdd2610702241251v71c526erfa3b338ad6893c60@mail.gmail.com> Hello when I register a package on PyPi using "setup.py register" There's no feedback at all in case of failure when the classifiers are wrong (let's say, a small tipo in the License name for example) On the other hand, doing the same thing from the website forms returns an accurate feedback that tells what's wrong. It could be really nice to add this feature to setuptools, in order to get the same feedback from the prompt Regards Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070224/1dba001e/attachment.html From pogonyshev at gmx.net Sat Feb 24 22:09:45 2007 From: pogonyshev at gmx.net (Paul Pogonyshev) Date: Sat, 24 Feb 2007 23:09:45 +0200 Subject: [Distutils] unicode bug in distutils In-Reply-To: <94bdd2610702241247t568a942dw2fe1b10883b62d20@mail.gmail.com> References: <94bdd2610702241247t568a942dw2fe1b10883b62d20@mail.gmail.com> Message-ID: <200702242309.46022.pogonyshev@gmx.net> Tarek Ziad? wrote: > I have created a setup.py file for distirbution and I bumped into > a small bug when i tried to set my name in the contact field (Tarek Ziad?) > > Using string (utf8 file): > > setup( > maintainer="Tarek Ziad?" > ) How about maintainer = u"Tarek Ziad?"? Paul From ziade.tarek at gmail.com Sat Feb 24 22:06:43 2007 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 24 Feb 2007 15:06:43 -0600 Subject: [Distutils] unicode bug in distutils In-Reply-To: <200702242309.46022.pogonyshev@gmx.net> References: <94bdd2610702241247t568a942dw2fe1b10883b62d20@mail.gmail.com> <200702242309.46022.pogonyshev@gmx.net> Message-ID: <94bdd2610702241306q60b1a10rb91dff4919fdae13@mail.gmail.com> On 2/24/07, Paul Pogonyshev wrote: > > Tarek Ziad? wrote: > > I have created a setup.py file for distirbution and I bumped into > > a small bug when i tried to set my name in the contact field (Tarek > Ziad?) > > > > Using string (utf8 file): > > > > setup( > > maintainer="Tarek Ziad?" > > ) > > How about maintainer = u"Tarek Ziad?"? see second part of my message: Using unicode: setup( maintainer=u"Tarek Ziad?" ) leads to: File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 1094, in write_pkg_file file.write('Author: %s\n' % self.get_contact() ) UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 18: ordinal not in range(128) Tarek Paul > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070224/e04b4665/attachment.htm From tseaver at palladion.com Sat Feb 24 23:45:38 2007 From: tseaver at palladion.com (Tres Seaver) Date: Sat, 24 Feb 2007 17:45:38 -0500 Subject: [Distutils] unicode bug in distutils In-Reply-To: <94bdd2610702241306q60b1a10rb91dff4919fdae13@mail.gmail.com> References: <94bdd2610702241247t568a942dw2fe1b10883b62d20@mail.gmail.com> <200702242309.46022.pogonyshev@gmx.net> <94bdd2610702241306q60b1a10rb91dff4919fdae13@mail.gmail.com> Message-ID: <45E0C012.7090801@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tarek Ziad? wrote: > On 2/24/07, Paul Pogonyshev wrote: >> Tarek Ziad? wrote: >>> I have created a setup.py file for distirbution and I bumped into >>> a small bug when i tried to set my name in the contact field (Tarek >> Ziad?) >>> Using string (utf8 file): >>> >>> setup( >>> maintainer="Tarek Ziad?" >>> ) >> How about maintainer = u"Tarek Ziad?"? > > > see second part of my message: > > Using unicode: > > setup( > maintainer=u"Tarek Ziad?" > ) > > leads to: > > File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", > line 1094, in write_pkg_file > file.write('Author: %s\n' % self.get_contact() ) > UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position > 18: ordinal not in range(128) Don't you still need to tell Python about the encoding of your string literals [1] [2] ? E.g.:: # -*- coding: utf-8 -*- #vim:fileencoding=utf-8 -*- print "Tarek Ziad?" [1] http://docs.python.org/ref/encodings.html#encodings [2] http://docs.python.org/ref/strings.html Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF4MAS+gerLs4ltQ4RAn2FAJ0XLaZOuOaAjLpOygBXx3MmxWnY5ACgpWlw xTllHr/4WeIq1fU74rXcuqg= =Cam9 -----END PGP SIGNATURE----- From tseaver at palladion.com Sat Feb 24 23:45:38 2007 From: tseaver at palladion.com (Tres Seaver) Date: Sat, 24 Feb 2007 17:45:38 -0500 Subject: [Distutils] unicode bug in distutils In-Reply-To: <94bdd2610702241306q60b1a10rb91dff4919fdae13@mail.gmail.com> References: <94bdd2610702241247t568a942dw2fe1b10883b62d20@mail.gmail.com> <200702242309.46022.pogonyshev@gmx.net> <94bdd2610702241306q60b1a10rb91dff4919fdae13@mail.gmail.com> Message-ID: <45E0C012.7090801@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tarek Ziad? wrote: > On 2/24/07, Paul Pogonyshev wrote: >> Tarek Ziad? wrote: >>> I have created a setup.py file for distirbution and I bumped into >>> a small bug when i tried to set my name in the contact field (Tarek >> Ziad?) >>> Using string (utf8 file): >>> >>> setup( >>> maintainer="Tarek Ziad?" >>> ) >> How about maintainer = u"Tarek Ziad?"? > > > see second part of my message: > > Using unicode: > > setup( > maintainer=u"Tarek Ziad?" > ) > > leads to: > > File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", > line 1094, in write_pkg_file > file.write('Author: %s\n' % self.get_contact() ) > UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position > 18: ordinal not in range(128) Don't you still need to tell Python about the encoding of your string literals [1] [2] ? E.g.:: # -*- coding: utf-8 -*- #vim:fileencoding=utf-8 -*- print "Tarek Ziad?" [1] http://docs.python.org/ref/encodings.html#encodings [2] http://docs.python.org/ref/strings.html Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF4MAS+gerLs4ltQ4RAn2FAJ0XLaZOuOaAjLpOygBXx3MmxWnY5ACgpWlw xTllHr/4WeIq1fU74rXcuqg= =Cam9 -----END PGP SIGNATURE----- From pje at telecommunity.com Sun Feb 25 02:45:52 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 24 Feb 2007 20:45:52 -0500 Subject: [Distutils] unicode bug in distutils In-Reply-To: <45E0C012.7090801@palladion.com> References: <94bdd2610702241306q60b1a10rb91dff4919fdae13@mail.gmail.com> <94bdd2610702241247t568a942dw2fe1b10883b62d20@mail.gmail.com> <200702242309.46022.pogonyshev@gmx.net> <94bdd2610702241306q60b1a10rb91dff4919fdae13@mail.gmail.com> Message-ID: <5.1.1.6.0.20070224203115.0270a5a8@sparrow.telecommunity.com> At 02:47 PM 2/24/2007 -0600, Tarek Ziad? wrote: >I have created a setup.py file for distirbution and I bumped into >a small bug when i tried to set my name in the contact field (Tarek Ziad?) > >Using string (utf8 file): > >setup( > maintainer="Tarek Ziad?" >) > >leads to: > > File ".../lib/python2.5/distutils/command/register.py", line 162, in > send_metadata > auth) > File ".../lib/python2.5/distutils/command/register.py", line 257, in > post_to_server > value = unicode(value).encode("utf-8") >UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 10: >ordinal not in range(128) > > >Using unicode: > >setup( > maintainer=u"Tarek Ziad?" >) > >leads to: > > File ".../lib/python2.5/distutils/dist.py", line 1094, in write_pkg_file > file.write('Author: %s\n' % self.get_contact() ) >UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in >position 18: ordinal not in range(128) > >I would propose a patch for this problem but i don't know what would be >the best input (i guess unicode > for names) At 05:45 PM 2/24/2007 -0500, Tres Seaver wrote: >Don't you still need to tell Python about the encoding of your string >literals [1] [2] ? E.g.:: That's not the problem, it's that the code that writes the PKG-INFO file doesn't handle Unicode. See distutils.dist.DistributionMetadata.write_pkg_info(). It needs to use a file with encoding support, if it's doing unicode However, there's currently no standard, as far as I know, for what encoding the PKG-INFO file should use. Meanwhile, the 'register' command accepts Unicode, but is broken in handling it. Essentially, the problem is that Python 2.5 broke this by adding a unicode *requirement* to the "register" command. Previously, register simply sent whatever you gave it, and the PKG-INFO writing code still does. Unfortunately, this means that there is no longer any one value that you can use for your name that will be accepted by both "register" and anything that writes a PKG-INFO file. Both register and write_pkg_info() are arguably broken here, and should be able to work with either strings or unicode, and degrade gracefully in the event of non-ASCII characters in a string. (Because even though "register" is only run by the package's author, users may run other commands that require a PKG-INFO, so a package prepared using Python <2.5 must still be usable with Python 2.5 distutils, and Python <2.5 allows 8-bit maintainer names.) Unfortunately, this isn't fixable until there's a new 2.5.x release. For previous Python versions, both register and write_pkg_info() accepted 8-bit strings and passed them on as-is, so the only workaround for this issue at the moment is to revert to Python 2.4 or less. From nathan at yergler.net Tue Feb 27 02:17:45 2007 From: nathan at yergler.net (Nathan R. Yergler) Date: Mon, 26 Feb 2007 20:17:45 -0500 Subject: [Distutils] easy_install prefers source? Message-ID: At the PyCon sprints Jim and I were working with easy_install and working_env under Windows and noticed some odd behavior. When within an activated working environment, attempting to do something like: c:\...>easy_install lxml==1.1 attempts to retrieve the source distribution rather than the available Windows binary egg for PyPI. Is this the expected behavior? I would have expected easy_install to prefer binary eggs. Thanks, Nathan From pje at telecommunity.com Tue Feb 27 16:17:37 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 27 Feb 2007 10:17:37 -0500 Subject: [Distutils] easy_install prefers source? In-Reply-To: Message-ID: <5.1.1.6.0.20070227101447.0528ccc0@sparrow.telecommunity.com> At 08:17 PM 2/26/2007 -0500, Nathan R. Yergler wrote: >At the PyCon sprints Jim and I were working with easy_install and >working_env under Windows and noticed some odd behavior. When within >an activated working environment, attempting to do something like: > >c:\...>easy_install lxml==1.1 > >attempts to retrieve the source distribution rather than the available >Windows binary egg for PyPI. What windows binary egg for PyPI? If you mean the one for 1.1, the problem is that easy_install isn't reading that page, because it's a "hidden" version. See: $ python -m easy_install -nvv lxml==1.1 Searching for lxml==1.1 Reading http://cheeseshop.python.org/pypi/lxml/ Reading http://codespeak.net/lxml Found link: http://codespeak.net/lxml/lxml-1.2.tgz Found link: http://codespeak.net/lxml/lxml-1.1.2.tgz Found link: http://codespeak.net/lxml/lxml-1.1.1.tgz Found link: http://codespeak.net/lxml/lxml-1.1.tgz Found link: http://codespeak.net/lxml/lxml-1.0.4.tgz Found link: http://codespeak.net/lxml/lxml-1.0.3.tgz Found link: http://codespeak.net/lxml/lxml-1.0.2.tgz Found link: http://codespeak.net/lxml/lxml-1.0.1.tgz Found link: http://codespeak.net/lxml/lxml-1.0.tgz Found link: http://codespeak.net/lxml/lxml-0.9.2.tgz Found link: http://codespeak.net/lxml/lxml-0.9.1.tgz Found link: http://codespeak.net/lxml/lxml-0.9.tgz Found link: http://codespeak.net/lxml/lxml-0.8.tgz Found link: http://codespeak.net/lxml/lxml-0.7.tgz Found link: http://codespeak.net/lxml/lxml-0.6.tgz Found link: http://codespeak.net/lxml/lxml-0.5.1.tgz Found link: http://codespeak.net/lxml/lxml-0.5.tgz Found link: http://cheeseshop.python.org/packages/source/l/lxml/lxml-1.2.1.tar.gz Found link: http://codespeak.net/svn/lxml/trunk#egg=lxml-dev Found link: http://codespeak.net/svn/lxml/branch/lxml-1.2#egg=lxml-1.2bugfix Reading http://cheeseshop.python.org/pypi/lxml/1.2.1 Found link: http://cheeseshop.python.org/packages/source/l/lxml/lxml-1.2.1.tar.gz Best match: lxml 1.1 Downloading http://codespeak.net/lxml/lxml-1.1.tgz > Is this the expected behavior? I would >have expected easy_install to prefer binary eggs. It only prefers what it finds, and currently it won't find Cheeseshop files for "hidden" versions of a pacakge unless you include a -f to that page, e.g. -f http://cheeseshop.python.org/pypi/lxml/1.1 in this case. From Felix.Wiemann at ososo.de Tue Feb 27 18:45:31 2007 From: Felix.Wiemann at ososo.de (Felix Wiemann) Date: Tue, 27 Feb 2007 11:45:31 -0600 Subject: [Distutils] ez_setup.py Message-ID: <45E46E3B.2060007@ososo.de> Hi, I would like to see two things changed about ez_setup.py: The "#!python" at the top should read "#!/bin/env python". And we'd need to have a license header at the top of the file. Public domain is the most pain-free license for us (and probably everyone), but anything Python-compatible should work. Best wishes, Felix From nathan at yergler.net Tue Feb 27 23:36:59 2007 From: nathan at yergler.net (Nathan R. Yergler) Date: Tue, 27 Feb 2007 17:36:59 -0500 Subject: [Distutils] easy_install prefers source? In-Reply-To: References: <5.1.1.6.0.20070227101447.0528ccc0@sparrow.telecommunity.com> Message-ID: Forwarding to the list because I wasn't smart enough to "reply-all"... ---------- Forwarded message ---------- From: Nathan R. Yergler Date: Feb 27, 2007 5:36 PM Subject: Re: [Distutils] easy_install prefers source? To: "Phillip J. Eby" > >c:\...>easy_install lxml==1.1 > > > >attempts to retrieve the source distribution rather than the available > >Windows binary egg for PyPI. > > What windows binary egg for PyPI? If you mean the one for 1.1, the problem > is that easy_install isn't reading that page, because it's a "hidden" > version. See: Yes, I meant the one for 1.1; I assumed (incorrectly, obviously) that by specifying the version, easy_install would look at that particular version page in PyPI if it existed. > > $ python -m easy_install -nvv lxml==1.1 > Searching for lxml==1.1 [snip] > Downloading http://codespeak.net/lxml/lxml-1.1.tgz > > > > Is this the expected behavior? I would > >have expected easy_install to prefer binary eggs. > > It only prefers what it finds, and currently it won't find Cheeseshop files > for "hidden" versions of a pacakge unless you include a -f to that page, > e.g. -f http://cheeseshop.python.org/pypi/lxml/1.1 in this case. Prefering what it finds makes sense, although *I* would prefer that it found the page on PyPI with the egg ;). Is the "hiding" of versions a Cheeseshop thing? Is there a good reason *not* to look for the requested version page on Cheeseshop? (I'm guessing the answer to this is "we don't look for pages, we use an RPC interface [for some definition of RPC]", but I'll ask anyway) Thanks, Nathan From constant.beta at gmail.com Wed Feb 28 02:01:00 2007 From: constant.beta at gmail.com (=?ISO-8859-2?Q?Micha=B3_Kwiatkowski?=) Date: Wed, 28 Feb 2007 02:01:00 +0100 Subject: [Distutils] easy_install prefers source? In-Reply-To: References: <5.1.1.6.0.20070227101447.0528ccc0@sparrow.telecommunity.com> Message-ID: <5e8b0f6b0702271701l30937a0eif9a31de02c702474@mail.gmail.com> On 2/27/07, Nathan R. Yergler wrote: > > It only prefers what it finds, and currently it won't find Cheeseshop > > files for "hidden" versions of a pacakge unless you include a -f to > > that page, e.g. -f http://cheeseshop.python.org/pypi/lxml/1.1 in this > > case. > > Prefering what it finds makes sense, although *I* would prefer that it > found the page on PyPI with the egg ;). Is the "hiding" of versions a > Cheeseshop thing? Is there a good reason *not* to look for the > requested version page on Cheeseshop? (I'm guessing the answer to this > is "we don't look for pages, we use an RPC interface [for some > definition of RPC]", but I'll ask anyway) Upon registering a new version of a given package previous one is being hidden by default. This is a distutils thing and PyPI simply implements this behaviour (by not listing hidden versions). I also think setuptools should look at the appropriate version page of a package, which is the right thing to do from the usability point of view. Since user requested this version he shall have it. In general case, if a package isn't listed on the project homepage nothing will be find and easy_install for this specific version will fail, which is another bad symptom of this implementation. You just got (un)lucky enough that lxml developers list version requested by you on their homepage. If you requested a Cheesecake 0.6 you'll get an error, because 0.6.1 is the current version and 0.6 is hidden: $ sudo easy_install -vv Cheesecake==0.6 Searching for Cheesecake==0.6 Reading http://cheeseshop.python.org/pypi/Cheesecake/ Reading http://pycheesecake.org/ Found link: http://darcs.idyll.org/~t/projects/twill-0.7.4.tar.gz Found link: http://svn.pycheesecake.org/trunk#egg=Cheesecake-dev Found link: http://cheeseshop.python.org/packages/2.4/C/Cheesecake/Cheesecake-0.6.1-py2.4.egg#md5=a3bd1eb4d01eacf731a53359ba940829 Found link: http://cheeseshop.python.org/packages/2.3/C/Cheesecake/Cheesecake-0.6.1-py2.3.egg#md5=6a19de9385bf0a0e61d5820f5b4be748 Found link: http://cheeseshop.python.org/packages/source/C/Cheesecake/cheesecake-0.6.1.tar.gz#md5=f9e19123281f812e27674140bfb19b7e Reading http://cheeseshop.python.org/pypi/Cheesecake/0.6.1 No local packages or download links found for Cheesecake==0.6 error: Could not find suitable distribution for Requirement.parse('Cheesecake==0.6') "Hidden" should mean "hidden from the user *by default*", but should not get into way of a user that *explicitly* requested a release. Cheers, mk From bhendrix at enthought.com Wed Feb 28 19:46:07 2007 From: bhendrix at enthought.com (Bryce Hendrix) Date: Wed, 28 Feb 2007 12:46:07 -0600 Subject: [Distutils] How does distuils/setuptools determine script directory? Message-ID: <45E5CDEF.4080809@enthought.com> I have a python Windows install I built from source, which when installed does not create a Scripts directory. When I install setuptools the easy_install executable is put in Lib/site-packages rather than Scripts. All other eggs also put their scripts in the site-packages directory. How does distutils and setuptools decide where to install scripts to by default? Bryce