From bignose+hates-spam at benfinney.id.au Tue Jan 1 01:03:31 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Tue, 01 Jan 2008 11:03:31 +1100 Subject: [Distutils] Distutils confusion - please give me a hand References: Message-ID: <87y7babeek.fsf@benfinney.id.au> Donn Ingle writes: > I confess to being absolutely lost here. I have a few > sub-directories inside my package that contain help stuff and locale > stuff which I want distutils to include and install. It always > worked before, until I invented the help/common/en directory and now > it just won't work any more. While waiting, use your time constructively with the time-honoured debugging technique of "make the simplest example that shows the problem". In other words, try implementing a tiny project with distutils, and gradually work toward your problematic one, running distutils commands each time you make a change, until it produces the same problematic behaviour. Then, try simplifying it by removing things that don't seem to be related to the problem, until the problem stops happening. Repeat until you have the simplest implementation of the problem you can devise that you can't make simpler without losing the unexpected behaviour. Then, post that problem implementation here, with a description that will no doubt be improved by the above process. -- \ "Don't be afraid of missing opportunities. Behind every failure | `\ is an opportunity somebody wishes they had missed." -- Jane | _o__) Wagner, via Lily Tomlin | Ben Finney From donn.ingle at gmail.com Tue Jan 1 08:01:49 2008 From: donn.ingle at gmail.com (Donn Ingle) Date: Tue, 01 Jan 2008 09:01:49 +0200 Subject: [Distutils] Distutils confusion - please give me a hand References: <87y7babeek.fsf@benfinney.id.au> Message-ID: > While waiting, use your time constructively with the time-honoured > debugging technique of "make the simplest example that shows the > problem". I'd thought my post was mostly doing that already - not runnable, but not vague. Do you mean, given it's contents, that the error I get is an unusual one? If so, then I'd have to hack an example. If not, then I'm hoping for some quick expert advice for 'housing' directory trees full of data under the package or Distribution-root directories. I have spent too many hours trying to get distutils to 'see' my tree and I have actually run out of dev-time for my project now. \d From robin at alldunn.com Tue Jan 1 21:18:57 2008 From: robin at alldunn.com (Robin Dunn) Date: Tue, 01 Jan 2008 12:18:57 -0800 Subject: [Distutils] Distutils confusion - please give me a hand In-Reply-To: References: <87y7babeek.fsf@benfinney.id.au> Message-ID: <477AA031.8090503@alldunn.com> Donn Ingle wrote: >> While waiting, use your time constructively with the time-honoured >> debugging technique of "make the simplest example that shows the >> problem". > I'd thought my post was mostly doing that already - not runnable, but not > vague. > > Do you mean, given it's contents, that the error I get is an unusual one? If > so, then I'd have to hack an example. If not, then I'm hoping for some > quick expert advice for 'housing' directory trees full of data under the > package or Distribution-root directories. > > I have spent too many hours trying to get distutils to 'see' my tree and I > have actually run out of dev-time for my project now. wxPython has some examples of specifying in-package data files that you can probably build upon. See where DATA_FILES is built up, the wx_smart_install_data command class, and the find_data_files function. http://svn.wxwidgets.org/viewvc/wx/wxPython/branches/WX_2_8_BRANCH/ Look at both setup.py and config.py. -- Robin Dunn Software Craftsman http://wxPython.org Java give you jitters? Relax with wxPython! From donn.ingle at gmail.com Tue Jan 1 21:50:00 2008 From: donn.ingle at gmail.com (Donn Ingle) Date: Tue, 01 Jan 2008 22:50:00 +0200 Subject: [Distutils] Distutils confusion - please give me a hand References: <87y7babeek.fsf@benfinney.id.au> <477AA031.8090503@alldunn.com> Message-ID: > wxPython has some examples of specifying in-package data files that you > can probably build upon. Thanks Robin, I'll go poke around. \d From ziade.tarek at gmail.com Wed Jan 2 15:37:22 2008 From: ziade.tarek at gmail.com (=?UTF-8?Q?Tarek_Ziad=C3=A9?=) Date: Wed, 2 Jan 2008 06:37:22 -0800 (PST) Subject: [Distutils] verify status in register.py Message-ID: <14579392.post@talk.nabble.com> Hello I was wondering what is the status of the "--verify" option in register. It looks like it has been deprecated in favor of --dry-run (http://mail.python.org/pipermail/python-checkins/2003-April/035067.html) but I can still see it in the boolean_options in the trunk: http://svn.python.org/view/python/trunk/Lib/distutils/command/register.py?rev=56543&view=markup Shouldn't it be removed ? Regards Tarek -- View this message in context: http://www.nabble.com/verify-status-in-register.py-tp14579392p14579392.html Sent from the Python - distutils-sig mailing list archive at Nabble.com. From carl at personnelware.com Thu Jan 3 19:03:40 2008 From: carl at personnelware.com (Carl Karsten) Date: Thu, 03 Jan 2008 12:03:40 -0600 Subject: [Distutils] installing scripts into the user's PATH Message-ID: <477D237C.2070505@personnelware.com> "Scripts are files containing Python source code, intended to be started from the command line." http://docs.python.org/dist/node11.html I want to do that. I am hoping that once it is done, it will be in a dir that is in the user's OS PATH. (if that sounds restrictive, I didn't say it right. at this point, I don't care what dir it is in, as long as the user doesn't have to type it. like /usr/bin on linux. not sure where the right place would be on mac/win.) Is this supported? Carl K From pje at telecommunity.com Thu Jan 3 19:37:37 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 03 Jan 2008 13:37:37 -0500 Subject: [Distutils] installing scripts into the user's PATH In-Reply-To: <477D237C.2070505@personnelware.com> References: <477D237C.2070505@personnelware.com> Message-ID: <20080103183723.657FC3A40A2@sparrow.telecommunity.com> At 12:03 PM 1/3/2008 -0600, Carl Karsten wrote: >"Scripts are files containing Python source code, intended to be started from >the command line." http://docs.python.org/dist/node11.html > >I want to do that. I am hoping that once it is done, it will be in >a dir that >is in the user's OS PATH. (if that sounds restrictive, I didn't say >it right. >at this point, I don't care what dir it is in, as long as the user >doesn't have >to type it. like /usr/bin on linux. not sure where the right place >would be on >mac/win.) > >Is this supported? The default locations on everything but Windows will generally have this work. For Windows, there isn't much you can do except tell people to make sure the Python "Scripts" directory is added to their PATH. From interstar at gmail.com Thu Jan 3 20:09:12 2008 From: interstar at gmail.com (phil jones) Date: Thu, 3 Jan 2008 17:09:12 -0200 Subject: [Distutils] installing scripts into the user's PATH In-Reply-To: <20080103183723.657FC3A40A2@sparrow.telecommunity.com> References: <477D237C.2070505@personnelware.com> <20080103183723.657FC3A40A2@sparrow.telecommunity.com> Message-ID: I take it there's nothing in distutils that can actually update the user's OS PATH? On Jan 3, 2008 4:37 PM, Phillip J. Eby wrote: > At 12:03 PM 1/3/2008 -0600, Carl Karsten wrote: > >"Scripts are files containing Python source code, intended to be started from > >the command line." http://docs.python.org/dist/node11.html > > > >I want to do that. I am hoping that once it is done, it will be in > >a dir that > >is in the user's OS PATH. (if that sounds restrictive, I didn't say > >it right. > >at this point, I don't care what dir it is in, as long as the user > >doesn't have > >to type it. like /usr/bin on linux. not sure where the right place > >would be on > >mac/win.) > > > >Is this supported? > > The default locations on everything but Windows will generally have > this work. For Windows, there isn't much you can do except tell > people to make sure the Python "Scripts" directory is added to their PATH. > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From lxander.m at gmail.com Thu Jan 3 20:22:21 2008 From: lxander.m at gmail.com (Alexander Michael) Date: Thu, 3 Jan 2008 14:22:21 -0500 Subject: [Distutils] bugfix for GUI.exe launcher In-Reply-To: <4779654F.4020502@alldunn.com> References: <4779654F.4020502@alldunn.com> Message-ID: <525f23e80801031122m14141873l948c63255bba7a1f@mail.gmail.com> On Dec 31, 2007 4:55 PM, Robin Dunn wrote: > I see in the archives that this has been reported before, and I can > verify that the attached patch does fix the problem for me. Basically > what is happening is that the launcher is trying to execute python with > the "#!" still in the name. I verified this by rebuilding gui.exe > without -mwindows arg so it would still be a console app so I would see > the failure message. It printed this: > > "Could not exec #!c:\TOOLS\python25\pythonw.exe" > > > > -- > Robin Dunn > Software Craftsman > http://wxPython.org Java give you jitters? Relax with wxPython! > > > Index: launcher.c > =================================================================== > --- launcher.c (revision 59634) > +++ launcher.c (working copy) > @@ -231,8 +231,8 @@ > > if (is_gui) { > /* Use exec, we don't need to wait for the GUI to finish */ > - execv(python, (const char * const *)(newargs)); > - return fail("Could not exec %s", python); /* shouldn't get here! */ > + execv(ptr, (const char * const *)(newargs)); > + return fail("Could not exec %s", ptr); /* shouldn't get here! */ > } > > /* We *do* need to wait for a CLI to finish, so use spawn */ Robin, I raised this issue in the beginning of December [1]_ but haven't received a response yet (nor have I seen it corrected in the SVN). I would like to see this corrected, but I was under the impression that bug reports for setuptools should be sent to the distutils-sig mailling list. Do you know how we should next proceed to get this fixed? Thanks, Alex .. [1] From carl at personnelware.com Thu Jan 3 21:17:17 2008 From: carl at personnelware.com (Carl Karsten) Date: Thu, 03 Jan 2008 14:17:17 -0600 Subject: [Distutils] installing scripts into the user's PATH In-Reply-To: <20080103183723.657FC3A40A2@sparrow.telecommunity.com> References: <477D237C.2070505@personnelware.com> <20080103183723.657FC3A40A2@sparrow.telecommunity.com> Message-ID: <477D42CD.4020607@personnelware.com> Phillip J. Eby wrote: > At 12:03 PM 1/3/2008 -0600, Carl Karsten wrote: >> "Scripts are files containing Python source code, intended to be >> started from >> the command line." http://docs.python.org/dist/node11.html >> >> I want to do that. I am hoping that once it is done, it will be in a >> dir that >> is in the user's OS PATH. (if that sounds restrictive, I didn't say >> it right. >> at this point, I don't care what dir it is in, as long as the user >> doesn't have >> to type it. like /usr/bin on linux. not sure where the right place >> would be on >> mac/win.) >> >> Is this supported? > > The default locations on everything but Windows will generally have this > work. For Windows, there isn't much you can do except tell people to > make sure the Python "Scripts" directory is added to their PATH. > so if I do: scripts=['foo.py', 'bar.py'] where will they be copied? Carl K From pje at telecommunity.com Thu Jan 3 22:12:14 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 03 Jan 2008 16:12:14 -0500 Subject: [Distutils] bugfix for GUI.exe launcher In-Reply-To: <525f23e80801031122m14141873l948c63255bba7a1f@mail.gmail.co m> References: <4779654F.4020502@alldunn.com> <525f23e80801031122m14141873l948c63255bba7a1f@mail.gmail.com> Message-ID: <20080103211200.DD51D3A40A4@sparrow.telecommunity.com> At 02:22 PM 1/3/2008 -0500, Alexander Michael wrote: >I raised this issue in the beginning of December [1]_ but haven't >received a response yet (nor have I seen it corrected in the SVN). I >would like to see this corrected, but I was under the impression that >bug reports for setuptools should be sent to the distutils-sig >mailling list. Do you know how we should next proceed to get this >fixed? I've been having difficulty reproducing the error, so it's hard for me to confirm that the patch fixes it. If somebody wants to extend their patch to include a test that fails on their system without the patch but succeeds with it, I'd be happy to go ahead and include it. See setuptools/tests/win_script_wrapper.txt for the existing test code, which currently only covers the CLI wrapper, not the GUI one. From pje at telecommunity.com Thu Jan 3 22:16:29 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 03 Jan 2008 16:16:29 -0500 Subject: [Distutils] installing scripts into the user's PATH In-Reply-To: <477D42CD.4020607@personnelware.com> References: <477D237C.2070505@personnelware.com> <20080103183723.657FC3A40A2@sparrow.telecommunity.com> <477D42CD.4020607@personnelware.com> Message-ID: <20080103211614.40CFE3A40A2@sparrow.telecommunity.com> At 02:17 PM 1/3/2008 -0600, Carl Karsten wrote: >Phillip J. Eby wrote: >>At 12:03 PM 1/3/2008 -0600, Carl Karsten wrote: >>>"Scripts are files containing Python source code, intended to be >>>started from >>>the command line." http://docs.python.org/dist/node11.html >>> >>>I want to do that. I am hoping that once it is done, it will be >>>in a dir that >>>is in the user's OS PATH. (if that sounds restrictive, I didn't >>>say it right. >>>at this point, I don't care what dir it is in, as long as the user >>>doesn't have >>>to type it. like /usr/bin on linux. not sure where the right >>>place would be on >>>mac/win.) >>> >>>Is this supported? >>The default locations on everything but Windows will generally have >>this work. For Windows, there isn't much you can do except tell >>people to make sure the Python "Scripts" directory is added to their PATH. > >so if I do: > scripts=['foo.py', 'bar.py'] > >where will they be copied? On unixes, they will typically go to the same directory where the `python` executable itself lives. On Windows, they'll go to the Scripts subdirectory of the Python installation directory. From pje at telecommunity.com Thu Jan 3 22:16:37 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 03 Jan 2008 16:16:37 -0500 Subject: [Distutils] installing scripts into the user's PATH In-Reply-To: References: <477D237C.2070505@personnelware.com> <20080103183723.657FC3A40A2@sparrow.telecommunity.com> Message-ID: <20080103211622.657D23A40A4@sparrow.telecommunity.com> At 05:09 PM 1/3/2008 -0200, phil jones wrote: >I take it there's nothing in distutils that can actually update the >user's OS PATH? Nope. From lxander.m at gmail.com Thu Jan 3 23:49:38 2008 From: lxander.m at gmail.com (Alexander Michael) Date: Thu, 3 Jan 2008 17:49:38 -0500 Subject: [Distutils] bugfix for GUI.exe launcher In-Reply-To: <20080103211200.DD51D3A40A4@sparrow.telecommunity.com> References: <4779654F.4020502@alldunn.com> <525f23e80801031122m14141873l948c63255bba7a1f@mail.gmail.com> <20080103211200.DD51D3A40A4@sparrow.telecommunity.com> Message-ID: <525f23e80801031449v4fe8c7fcjd6c130c0c5ce6f7@mail.gmail.com> On Jan 3, 2008 4:12 PM, Phillip J. Eby wrote: > At 02:22 PM 1/3/2008 -0500, Alexander Michael wrote: > >I raised this issue in the beginning of December [1]_ but haven't > >received a response yet (nor have I seen it corrected in the SVN). I > >would like to see this corrected, but I was under the impression that > >bug reports for setuptools should be sent to the distutils-sig > >mailling list. Do you know how we should next proceed to get this > >fixed? > > I've been having difficulty reproducing the error, so it's hard for > me to confirm that the patch fixes it. If somebody wants to extend > their patch to include a test that fails on their system without the > patch but succeeds with it, I'd be happy to go ahead and include it. > > See setuptools/tests/win_script_wrapper.txt for the existing test > code, which currently only covers the CLI wrapper, not the GUI one. I've added a test for the GUI wrapper to the referenced file, which I've attached. Running doctest in verbose mode on it produces: ********************************************************************** File "win_script_wrapper.txt", line 129, in win_script_wrapper.txt Failed example: print output.read() Expected: Got: Could not exec #!C:\Python25\python.exe ********************************************************************** File "win_script_wrapper.txt", line 131, in win_script_wrapper.txt Failed example: print open(os.path.join(sample_directory, 'test_output.txt'), 'rb').read() Exception raised: Traceback (most recent call last): File "C:\Python25\lib\doctest.py", line 1212, in __run compileflags, 1) in test.globs File "", line 1, in print open(os.path.join(sample_directory, 'test_output.txt'), 'rb').read() IOError: [Errno 2] No such file or directory: 'c:\\docume~1\\alex\\locals~1\\temp\\tmpccuxgd\\test_output.txt' ********************************************************************** 1 items had failures: 2 of 28 in win_script_wrapper.txt ***Test Failed*** 2 failures. This test passes with the modifications: *** launcher.c.orig Thu Jan 3 17:19:54 2008 --- launcher.c Tue Dec 18 09:37:21 2007 *************** int run(int argc, char **argv, int is_gu *** 231,238 **** if (is_gui) { /* Use exec, we don't need to wait for the GUI to finish */ ! execv(python, (const char * const *)(newargs)); ! return fail("Could not exec %s", python); /* shouldn't get here! */ } /* We *do* need to wait for a CLI to finish, so use spawn */ --- 231,238 ---- if (is_gui) { /* Use exec, we don't need to wait for the GUI to finish */ ! execv(ptr, (const char * const *)(newargs)); ! return fail("Could not exec %s", ptr); /* shouldn't get here! */ } /* We *do* need to wait for a CLI to finish, so use spawn */ Thank you! Alex -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: win_script_wrapper.txt Url: http://mail.python.org/pipermail/distutils-sig/attachments/20080103/ebe1bba8/attachment-0001.txt From dpeterson at enthought.com Fri Jan 4 00:00:34 2008 From: dpeterson at enthought.com (Dave Peterson) Date: Thu, 03 Jan 2008 17:00:34 -0600 Subject: [Distutils] installing scripts into the user's PATH In-Reply-To: <20080103183723.657FC3A40A2@sparrow.telecommunity.com> References: <477D237C.2070505@personnelware.com> <20080103183723.657FC3A40A2@sparrow.telecommunity.com> Message-ID: <477D6912.1030304@enthought.com> Phillip J. Eby wrote: > At 12:03 PM 1/3/2008 -0600, Carl Karsten wrote: > >> "Scripts are files containing Python source code, intended to be started from >> the command line." http://docs.python.org/dist/node11.html >> >> I want to do that. I am hoping that once it is done, it will be in >> a dir that >> is in the user's OS PATH. (if that sounds restrictive, I didn't say >> it right. >> at this point, I don't care what dir it is in, as long as the user >> doesn't have >> to type it. like /usr/bin on linux. not sure where the right place >> would be on >> mac/win.) >> >> Is this supported? >> > > The default locations on everything but Windows will generally have > this work. For Windows, there isn't much you can do except tell > people to make sure the Python "Scripts" directory is added to their PATH. > Just curious, but is there a reason setuptools uses 'Scripts' on Windows instead of the 'Tools' dir? Unless I'm mistaken, a a new install of Python 2.5 puts scripts in 'Tools'. -- Dave From dpeterson at enthought.com Fri Jan 4 00:22:58 2008 From: dpeterson at enthought.com (Dave Peterson) Date: Thu, 03 Jan 2008 17:22:58 -0600 Subject: [Distutils] installing scripts into the user's PATH In-Reply-To: <20080103211622.657D23A40A4@sparrow.telecommunity.com> References: <477D237C.2070505@personnelware.com> <20080103183723.657FC3A40A2@sparrow.telecommunity.com> <20080103211622.657D23A40A4@sparrow.telecommunity.com> Message-ID: <477D6E52.9070801@enthought.com> Phillip J. Eby wrote: > At 05:09 PM 1/3/2008 -0200, phil jones wrote: > >> I take it there's nothing in distutils that can actually update the >> user's OS PATH? >> > > Nope. > This may not help in the general case, but if you have a more specific, say customer delivery, issue then you might look at Enthought's open-source Enstaller app -- which is a wrapper around easy_install. It does support running of post-install scripts that do things like this, and goes even further on Windows where it can add start menu icons and registry entries. Unfortunately, 'Enstaller' is still somewhat immature and the current version is way, way too bloated but we at Enthought are continuing to invest in it and we do use it already. -- Dave From pje at telecommunity.com Fri Jan 4 00:48:52 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 03 Jan 2008 18:48:52 -0500 Subject: [Distutils] bugfix for GUI.exe launcher In-Reply-To: <525f23e80801031449v4fe8c7fcjd6c130c0c5ce6f7@mail.gmail.com > References: <4779654F.4020502@alldunn.com> <525f23e80801031122m14141873l948c63255bba7a1f@mail.gmail.com> <20080103211200.DD51D3A40A4@sparrow.telecommunity.com> <525f23e80801031449v4fe8c7fcjd6c130c0c5ce6f7@mail.gmail.com> Message-ID: <20080103234843.A0C0E3A40A2@sparrow.telecommunity.com> At 05:49 PM 1/3/2008 -0500, Alexander Michael wrote: >On Jan 3, 2008 4:12 PM, Phillip J. Eby wrote: > > At 02:22 PM 1/3/2008 -0500, Alexander Michael wrote: > > >I raised this issue in the beginning of December [1]_ but haven't > > >received a response yet (nor have I seen it corrected in the SVN). I > > >would like to see this corrected, but I was under the impression that > > >bug reports for setuptools should be sent to the distutils-sig > > >mailling list. Do you know how we should next proceed to get this > > >fixed? > > > > I've been having difficulty reproducing the error, so it's hard for > > me to confirm that the patch fixes it. If somebody wants to extend > > their patch to include a test that fails on their system without the > > patch but succeeds with it, I'd be happy to go ahead and include it. > > > > See setuptools/tests/win_script_wrapper.txt for the existing test > > code, which currently only covers the CLI wrapper, not the GUI one. > >I've added a test for the GUI wrapper to the referenced file, which I've >attached. SVN is now updated; thanks for the help. From pje at telecommunity.com Fri Jan 4 00:51:49 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 03 Jan 2008 18:51:49 -0500 Subject: [Distutils] installing scripts into the user's PATH In-Reply-To: <477D6912.1030304@enthought.com> References: <477D237C.2070505@personnelware.com> <20080103183723.657FC3A40A2@sparrow.telecommunity.com> <477D6912.1030304@enthought.com> Message-ID: <20080103235134.28E8A3A40A2@sparrow.telecommunity.com> At 05:00 PM 1/3/2008 -0600, Dave Peterson wrote: >Just curious, but is there a reason setuptools uses 'Scripts' on Windows >instead of the 'Tools' dir? Unless I'm mistaken, a a new install of >Python 2.5 puts scripts in 'Tools'. It puts Python-supplied scripts there, yes, but the convention of installing user scripts in the Scripts subdirectory goes back at least a decade. Which doesn't mean it's right, but it's certainly not *new*. :) (Oh, and it's *distutils* that chooses that location, not setuptools.) From noah.gift at gmail.com Fri Jan 4 00:58:30 2008 From: noah.gift at gmail.com (Noah Gift) Date: Thu, 3 Jan 2008 18:58:30 -0500 Subject: [Distutils] virtual env w/ python2.4.4 and python2.5.1 on OS X Leopard, each with different virtual setup tools problem In-Reply-To: <477D6912.1030304@enthought.com> References: <477D237C.2070505@personnelware.com> <20080103183723.657FC3A40A2@sparrow.telecommunity.com> <477D6912.1030304@enthought.com> Message-ID: <2A3BF7DF-7F0C-47EC-97D1-992C9EE008F7@gmail.com> I am attaching a thread I started privately with Ian. The question is multi-part and somewhat complicated so please be patient. Background: I am running python2.5.1, the system python, i.e., default on Leopard: /System/Library/Frameworks/Python.framework/Versions/2.5/bin/ python2.5 I have installed virtualenv via easy_install. I use virtualenv where an entry point was setup in /usr/local/bin/ virtualenv to create a separate virtualenv. This is the buffer from my terminal: (note I use virtualenv to create a groknoahgift virtual env and then run easy_install grokproject egg from there). > The main problem I have is this traceback: > [ngift at Macintosh-6][H:10944][J:0]> virtualenv groknoahgift > New python executable in groknoahgift/bin/python > Installing setuptools............done. > [ngift at Macintosh-6][H:10945][J:0]> groknoahgift > [ngift at Macintosh-6][H:10946][J:0]> ll > total 0 > drwxr-xr-x 7 ngift staff 238 Jan 3 09:46 bin > drwxr-xr-x 3 ngift staff 102 Jan 3 09:46 lib > [ngift at Macintosh-6][H:10947][J:0]> bin/easy_install grokproject > Searching for grokproject > Best match: grokproject 0.6 > Processing grokproject-0.6-py2.5.egg > Adding grokproject 0.6 to easy-install.pth file > Installing grokproject script to /Users/ngift/src/groknoahgift/bin > Using /Library/Python/2.5/site-packages/grokproject-0.6-py2.5.egg > Processing dependencies for grokproject > Finished processing dependencies for grokproject > [ngift at Macintosh-6][H:10948][J:0]> pwd > /Users/ngift/src/groknoahgift > [ngift at Macintosh-6][H:10949][J:0]> ll > total 0 > drwxr-xr-x 8 ngift staff 272 Jan 3 09:46 bin > drwxr-xr-x 3 ngift staff 102 Jan 3 09:46 lib > [ngift at Macintosh-6][H:10950][J:0]> bin/grokproject noahgift > Traceback (most recent call last): > File "bin/grokproject", line 5, in > from pkg_resources import load_entry_point > File "build/bdist.linux-i686/egg/pkg_resources.py", line 2561, in > > File "build/bdist.linux-i686/egg/pkg_resources.py", line 626, in > require > File "build/bdist.linux-i686/egg/pkg_resources.py", line 524, in > resolve > pkg_resources.DistributionNotFound: PasteScript>=1.3 In trying to track down this problem, I then realized that for Grok, I need to use python2.4.4, so I did this to get around the problem, as Ian suggested: [ngift at Macintosh-6][H:10996][J:0]> curl http://svn.colorstudy.com/virtualenv/trunk/virtualenv.py > virtualenv-2.4 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 35516 100 35516 0 0 45505 0 --:--:-- --:--:-- --:--:-- 80395 [ngift at Macintosh-6][H:10997][J:0]> python2.4 virtualenv-2.4 src/py24ENV New python executable in src/py24ENV/bin/python Installing setuptools.................done. [ngift at Macintosh-6][H:10998][J:0]> cd src/py24ENV [ngift at Macintosh-6][H:10999][J:0]> bin/easy_install grokproject Question: Does it make sense two follow the easy-install example and make an entrypoint that corresponds to the python path? From dpeterson at enthought.com Fri Jan 4 01:01:08 2008 From: dpeterson at enthought.com (Dave Peterson) Date: Thu, 03 Jan 2008 18:01:08 -0600 Subject: [Distutils] installing scripts into the user's PATH In-Reply-To: <20080103235134.28E8A3A40A2@sparrow.telecommunity.com> References: <477D237C.2070505@personnelware.com> <20080103183723.657FC3A40A2@sparrow.telecommunity.com> <477D6912.1030304@enthought.com> <20080103235134.28E8A3A40A2@sparrow.telecommunity.com> Message-ID: <477D7744.6060405@enthought.com> Phillip J. Eby wrote: > At 05:00 PM 1/3/2008 -0600, Dave Peterson wrote: >> Just curious, but is there a reason setuptools uses 'Scripts' on Windows >> instead of the 'Tools' dir? Unless I'm mistaken, a a new install of >> Python 2.5 puts scripts in 'Tools'. > > It puts Python-supplied scripts there, yes, but the convention of > installing user scripts in the Scripts subdirectory goes back at least > a decade. > > Which doesn't mean it's right, but it's certainly not *new*. :) > > (Oh, and it's *distutils* that chooses that location, not setuptools.) Right, sorry for the confusion on which package was doing it. I just wish either distutils or the .msi installer was changed because it is confusing to describe what to do to users (especially those new to Python and/or Windows) when the Python .msi on windows doesn't even create the "Scripts" sub-dir, much less add it to a path environment variable for you. :-( -- Dave From ianb at colorstudy.com Fri Jan 4 01:31:32 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 03 Jan 2008 18:31:32 -0600 Subject: [Distutils] easy_install -f confusion Message-ID: <477D7E64.8070504@colorstudy.com> Jeff Hammel and myself were going over a problem with -f seemingly being ignored in easy_install, and it took us a while to realize that you have to do something like "easy_install -f 'url1 url2 ...'", and you can't do "easy_install -f url1 -f url2". It's literally been years that I've been unaware of this, and confused by the results. I think easy_install should give an error if you provide multiple -f's (or make it work if you do so). -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From pje at telecommunity.com Fri Jan 4 02:44:03 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 03 Jan 2008 20:44:03 -0500 Subject: [Distutils] easy_install -f confusion In-Reply-To: <477D7E64.8070504@colorstudy.com> References: <477D7E64.8070504@colorstudy.com> Message-ID: <20080104014349.5A0243A40A2@sparrow.telecommunity.com> At 06:31 PM 1/3/2008 -0600, Ian Bicking wrote: >Jeff Hammel and myself were going over a problem with -f seemingly being >ignored in easy_install, and it took us a while to realize that you have >to do something like "easy_install -f 'url1 url2 ...'", and you can't do >"easy_install -f url1 -f url2". > >It's literally been years that I've been unaware of this, and confused >by the results. I think easy_install should give an error if you >provide multiple -f's (or make it work if you do so). easy_install inherits this issue from the distutils -- i.e., it uses distutils argument parsing, which doesn't support this. It's a bit of a hack that it supports positional arguments at all. :( In some alternate universe in which I have the funding (and therefore time) to actually work on setuptools 0.7 and "nest", this and many other issues would certainly be fixable. From jeff at taupro.com Fri Jan 4 04:15:45 2008 From: jeff at taupro.com (Jeff Rush) Date: Thu, 03 Jan 2008 21:15:45 -0600 Subject: [Distutils] easy_install -f confusion In-Reply-To: <20080104014349.5A0243A40A2@sparrow.telecommunity.com> References: <477D7E64.8070504@colorstudy.com> <20080104014349.5A0243A40A2@sparrow.telecommunity.com> Message-ID: <477DA4E1.3070209@taupro.com> Phillip J. Eby wrote: > > In some alternate universe in which I have the funding (and therefore > time) to actually work on setuptools 0.7 and "nest", this and many > other issues would certainly be fixable. Sometimes you just have to ask. The PSF is looking for worthy projects to fund, and setuptools has come up in conversation several times as a candidate important to the Python community overall. You'd have to write up a detailed proposal of what you can get accomplished for X dollars, but if you get between projects, you really ought to consider it. -Jeff From donn.ingle at gmail.com Fri Jan 4 17:09:26 2008 From: donn.ingle at gmail.com (Donn Ingle) Date: Fri, 04 Jan 2008 18:09:26 +0200 Subject: [Distutils] Distutils confusion - please give me a hand References: Message-ID: FWIW, I have updated a tutorial I did on the Python wiki to include the solution (thanks Robin) to this problem: http://wiki.python.org/moin/DistUtilsTutorial \d From ianb at colorstudy.com Fri Jan 4 18:19:41 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 04 Jan 2008 11:19:41 -0600 Subject: [Distutils] easy_install -f confusion In-Reply-To: <20080104014349.5A0243A40A2@sparrow.telecommunity.com> References: <477D7E64.8070504@colorstudy.com> <20080104014349.5A0243A40A2@sparrow.telecommunity.com> Message-ID: <477E6AAD.2040700@colorstudy.com> Phillip J. Eby wrote: > At 06:31 PM 1/3/2008 -0600, Ian Bicking wrote: >> Jeff Hammel and myself were going over a problem with -f seemingly being >> ignored in easy_install, and it took us a while to realize that you have >> to do something like "easy_install -f 'url1 url2 ...'", and you can't do >> "easy_install -f url1 -f url2". >> >> It's literally been years that I've been unaware of this, and confused >> by the results. I think easy_install should give an error if you >> provide multiple -f's (or make it work if you do so). > > easy_install inherits this issue from the distutils -- i.e., it uses > distutils argument parsing, which doesn't support this. It's a bit of a > hack that it supports positional arguments at all. :( > > In some alternate universe in which I have the funding (and therefore > time) to actually work on setuptools 0.7 and "nest", this and many other > issues would certainly be fixable. The attached patch gives a helpful error message if you give multiple -f options. I used sys.argv; I imagine there might be a more proper way to get the unparsed arguments, but I admit I didn't look too closely. Obviously there are more correct solutions to this, but this addresses what I think is a substantial usability issue. -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org -------------- next part -------------- A non-text attachment was scrubbed... Name: easy_install_find_links.patch Type: text/x-patch Size: 1327 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20080104/eadc8e21/attachment.bin From noah.gift at gmail.com Fri Jan 4 18:31:43 2008 From: noah.gift at gmail.com (Noah Gift) Date: Fri, 4 Jan 2008 12:31:43 -0500 Subject: [Distutils] easy_install -f confusion In-Reply-To: <477DA4E1.3070209@taupro.com> References: <477D7E64.8070504@colorstudy.com> <20080104014349.5A0243A40A2@sparrow.telecommunity.com> <477DA4E1.3070209@taupro.com> Message-ID: <36335BF3-9293-45D6-A38B-5E9F40FBBEEB@gmail.com> On Jan 3, 2008, at 10:15 PM, Jeff Rush wrote: > Phillip J. Eby wrote: >> >> In some alternate universe in which I have the funding (and therefore >> time) to actually work on setuptools 0.7 and "nest", this and many >> other issues would certainly be fixable. > > Sometimes you just have to ask. The PSF is looking for worthy > projects to > fund, and setuptools has come up in conversation several times as a > candidate > important to the Python community overall. You'd have to write up a > detailed > proposal of what you can get accomplished for X dollars, but if you > get > between projects, you really ought to consider it. +1 > > > -Jeff > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig From pje at telecommunity.com Fri Jan 4 20:15:42 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 04 Jan 2008 14:15:42 -0500 Subject: [Distutils] easy_install -f confusion In-Reply-To: <477E6AAD.2040700@colorstudy.com> References: <477D7E64.8070504@colorstudy.com> <20080104014349.5A0243A40A2@sparrow.telecommunity.com> <477E6AAD.2040700@colorstudy.com> Message-ID: <20080104191529.834923A40AE@sparrow.telecommunity.com> At 11:19 AM 1/4/2008 -0600, Ian Bicking wrote: >The attached patch gives a helpful error message if you give >multiple -f options. I used sys.argv; I imagine there might be a >more proper way to get the unparsed arguments, but I admit I didn't >look too closely. Unfortunately, that's not an acceptable approach. sys.argv may have zero to do with the arguments being parsed: consider zc.buildout, for example, let alone setuptools' internal uses of easy_install. These are far from the only programs that internally invoke easy_install while running from a different command line. It might be better to fix distutils parsing so it detects duplicated options in general. If it can be done in a way that setuptools can monkeypatch the fix in, even better. From mikko at redinnovation.com Mon Jan 7 04:13:33 2008 From: mikko at redinnovation.com (Mikko Ohtamaa) Date: Mon, 7 Jan 2008 03:13:33 +0000 (UTC) Subject: [Distutils] =?utf-8?q?BUG=3A_easy=5Finstall=2Eexe_fails_under_C?= =?utf-8?q?=3A=5CProgram_files?= Message-ID: I guess this is due to a space in the path. C:\Documents and Settings\X\Desktop>"c:\Program Files\Plone 3\Python\Scripts\easy_install.exe" Cannot find Python executable C:\Program Ps. Could you please mention how to report easy_install/setuptools bugs in the manual. From mikko at redinnovation.com Mon Jan 7 04:19:53 2008 From: mikko at redinnovation.com (Mikko Ohtamaa) Date: Mon, 7 Jan 2008 03:19:53 +0000 (UTC) Subject: [Distutils] BUG: Python version confusion leads to cryptic errors on Windows Message-ID: When you try to install Python 2.5 built egg on Python 2.4 on Windows, the install starts fine, but is aborted due to "dependency issue" ("error: Could not find suitable distribution for Requirement.parse") in the package you are trying to install. The error message makes the user think there is something wrong in the package files. However, it would be very friendly to report that the package you are trying to install is not compatible with Python version. A sample case below. Trying to install Python 2.5 egg when running Python 2.4. C:\Documents and Settings\cx\Desktop>"c:\Program Files\Plone 3\Python\Pyth on.exe" "c:\Program Files\Plone 3\Python\Scripts\easy_install-script.py" ./aura. theme-1.0.0-py2.5.egg Processing aura.theme-1.0.0-py2.5.egg creating c:\program files\plone 3\python\lib\site-packages\aura.theme-1.0.0-py2. 5.egg Extracting aura.theme-1.0.0-py2.5.egg to c:\program files\plone 3\python\lib\sit e-packages File "c:\program files\plone 3\python\lib\site-packages\aura.theme-1.0.0-py2.5 .egg\aura\theme\skins\aura_theme_custom_templates\createContent.py", line 232 return run(context) SyntaxError: 'return' outside function File "c:\program files\plone 3\python\lib\site-packages\aura.theme-1.0.0-py2.5 .egg\aura\theme\skins\aura_theme_custom_templates\getSection.py", line 14 return iter.Title() SyntaxError: 'return' outside function File "c:\program files\plone 3\python\lib\site-packages\aura.theme-1.0.0-py2.5 .egg\aura\theme\skins\aura_theme_styles\plone.css.py", line 26 return printed SyntaxError: 'return' outside function Adding aura.theme 1.0.0 to easy-install.pth file Installed c:\program files\plone 3\python\lib\site-packages\aura.theme-1.0.0-py2 .5.egg Processing dependencies for aura.theme==1.0.0 Searching for aura.theme==1.0.0 Reading http://pypi.python.org/simple/aura.theme/ Couldn't find index page for 'aura.theme' (maybe misspelled?) Scanning index of all packages (this may take a while) Reading http://pypi.python.org/simple/ No local packages or download links found for aura.theme==1.0.0 error: Could not find suitable distribution for Requirement.parse('aura.theme==1 .0.0') From pje at telecommunity.com Mon Jan 7 04:36:51 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 06 Jan 2008 22:36:51 -0500 Subject: [Distutils] BUG: easy_install.exe fails under C :\Program files In-Reply-To: References: Message-ID: <20080107033634.004653A406C@sparrow.telecommunity.com> At 03:13 AM 1/7/2008 +0000, Mikko Ohtamaa wrote: >I guess this is due to a space in the path. > >C:\Documents and Settings\X\Desktop>"c:\Program Files\Plone >3\Python\Scripts\easy_install.exe" >Cannot find Python executable C:\Program What version are you using? The version on my PC works just fine, even if I add a space to the Scripts directory name: C:\Documents and Settings\pje-2>"c:\python23\scrip ts\easy_install" error: No urls, filenames, or requirements specified (see --help) >Ps. Could you please mention how to report easy_install/setuptools bugs in the >manual. http://peak.telecommunity.com/DevCenter/setuptools#mailing-list From mikko at redinnovation.com Mon Jan 7 05:28:38 2008 From: mikko at redinnovation.com (Mikko Ohtamaa) Date: Mon, 7 Jan 2008 06:28:38 +0200 Subject: [Distutils] BUG: easy_install.exe fails under C :\Program files In-Reply-To: <20080107033634.004653A406C@sparrow.telecommunity.com> References: <20080107033634.004653A406C@sparrow.telecommunity.com> Message-ID: <7b5b293c0801062028h7e0e7b52g9194a57fd556e4e5@mail.gmail.com> > > What version are you using? The version on my PC works just fine, > even if I add a space to the Scripts directory name: Python 2.4.4 (#71, Oct 18 2006, 08:34:43) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. easy_install-2.4.py (--help nor --version don't give additional version information). C:\Documents and Settings\pje-2>"c:\python23\scrip ts\easy_install" > error: No urls, filenames, or requirements specified (see --help) Your python is still c:\python23, not "c:\python 23". >Ps. Could you please mention how to report easy_install/setuptools bugs in > the > >manual. Thanks! This is quite well hidden information, you should advertise yourself a bit more. Also, people usually expect find to a real bug tracker, not just a discussion list as notable open source project as setuptools is. http://peak.telecommunity.com/DevCenter/setuptools#mailing-list > > Cheers, Mikko -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20080107/2b9970be/attachment.htm From carl at personnelware.com Mon Jan 7 18:32:21 2008 From: carl at personnelware.com (Carl Karsten) Date: Mon, 07 Jan 2008 11:32:21 -0600 Subject: [Distutils] installing scripts into the user's PATH In-Reply-To: <477D237C.2070505@personnelware.com> References: <477D237C.2070505@personnelware.com> Message-ID: <47826225.6040600@personnelware.com> Speaking of PATH, here is a wad of code I wrote to add a path in Win. If such a thing already exists, I couldn't find it. pretty much the 2 key lines are: SetValueEx(h, 'path', 0, REG_EXPAND_SZ, NewPath ) rc, dwReturnValue = win32gui.SendMessageTimeout(win32con.HWND_BROADCAST,.... Add it to the reg, use it now (otherwise it doesn't get used till you reboot.) Carl K # addtopath.py """ Adds the passed parameter to the windows PATH Checks to make sure the dir exists and is not already in the path. """ import sys import os from _winreg import * import win32gui, win32con if len(sys.argv)==1: # add the python dir to the path DirToAdd = os.path.dirname( sys.executable ) else: DirToAdd = os.path.abspath(sys.argv[1]) if not os.path.exists(DirToAdd): print "dir %s does not exist." % DirToAdd sys.exit(1) hive = HKEY_LOCAL_MACHINE key=r"SYSTEM\CurrentControlSet\Control\Session Manager\Environment" subkey = 'Path' h = OpenKeyEx( hive, key, 0, KEY_READ ) curPath = QueryValueEx(h, subkey)[0] CloseKey(h) if DirToAdd in curPath.split(';'): print "dir %s already in current path:" % DirToAdd print curPath sys.exit(2) print "adding dir %s" % DirToAdd h = OpenKeyEx( hive, key, 0, KEY_SET_VALUE ) NewPath = curPath + ';' + DirToAdd SetValueEx(h, 'path', 0, REG_EXPAND_SZ, NewPath ) CloseKey(h) print "Path set to", NewPath rc, dwReturnValue = win32gui.SendMessageTimeout(win32con.HWND_BROADCAST, win32con.WM_SETTINGCHANGE, 0, "Environment", win32con.SMTO_ABORTIFHUNG, 5000) sys.exit(0) From pje at telecommunity.com Mon Jan 7 20:46:27 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 07 Jan 2008 14:46:27 -0500 Subject: [Distutils] BUG: easy_install.exe fails under C :\Program files In-Reply-To: <7b5b293c0801062028h7e0e7b52g9194a57fd556e4e5@mail.gmail.co m> References: <20080107033634.004653A406C@sparrow.telecommunity.com> <7b5b293c0801062028h7e0e7b52g9194a57fd556e4e5@mail.gmail.com> Message-ID: <20080107194611.00B903A406C@sparrow.telecommunity.com> At 06:28 AM 1/7/2008 +0200, Mikko Ohtamaa wrote: >What version are you using? The version on my PC works just fine, >even if I add a space to the Scripts directory name: > > >Python 2.4.4 (#71, Oct 18 2006, 08:34:43) [MSC v.1310 32 bit (Intel)] on win32 >Type "help", "copyright", "credits" or "license" for more information. > >easy_install-2.4.py (--help nor --version don't give >additional version information). Would you take a look at the "easy_install-2.4-script.py" file and see if the path after the #! is enclosed in quotes? For that matter, what Python executable is it referring to? From ziade.tarek at gmail.com Tue Jan 8 11:51:46 2008 From: ziade.tarek at gmail.com (=?UTF-8?Q?Tarek_Ziad=C3=A9?=) Date: Tue, 8 Jan 2008 02:51:46 -0800 (PST) Subject: [Distutils] zc.buildout : more infos on download errors Message-ID: <14686910.post@talk.nabble.com> Hello, When a server that is supposed to provide an agg is timing out (it happens sometime, for some libs like PIL, pysqlite, etc) zc.buildout returns a warning like this: Download error: (104, 'Connection reset by peer') -- Some packages may not be found! In the flow of actions, it is very hard to tell which url is down. I had to patch urlopen to catch it. I think zc.buildout could be more verbose here, and tell which package or url failed Regards Tarek -- View this message in context: http://www.nabble.com/zc.buildout-%3A-more-infos-on-download-errors-tp14686910p14686910.html Sent from the Python - distutils-sig mailing list archive at Nabble.com. From jim at zope.com Tue Jan 8 15:09:40 2008 From: jim at zope.com (Jim Fulton) Date: Tue, 8 Jan 2008 09:09:40 -0500 Subject: [Distutils] zc.buildout : more infos on download errors In-Reply-To: <14686910.post@talk.nabble.com> References: <14686910.post@talk.nabble.com> Message-ID: <3D1F048B-4696-4A8C-95C0-35467B43D856@zope.com> On Jan 8, 2008, at 5:51 AM, Tarek Ziad? wrote: > > Hello, > > When a server that is supposed to provide an agg is timing out (it > happens > sometime, > for some libs like PIL, pysqlite, etc) zc.buildout returns a warning > like > this: > > Download error: (104, 'Connection reset by peer') -- Some packages > may not > be found! > > In the flow of actions, it is very hard to tell which url is down. I > had to > patch urlopen > to catch it. > > I think zc.buildout could be more verbose here, and tell which > package or > url failed Agreed. You should submit a bug report. Note that buildout is using setuptools, which uses urllib2 which is maddeningly uninformative in it's error reporting. Jim -- Jim Fulton Zope Corporation From ianb at colorstudy.com Wed Jan 9 01:47:22 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 08 Jan 2008 18:47:22 -0600 Subject: [Distutils] virtualenv, OS X, and GUIs In-Reply-To: <47840BC7.8010801@gmail.com> References: <47840BC7.8010801@gmail.com> Message-ID: <4784199A.6040805@colorstudy.com> Robert Kern wrote: > Is there an appropriate mailing list for talking about virtualenv? I've > run into a problem where virtualenvs on OS X cannot access the screen to > run GUIs. I've found a solution that appears to work and am going to > write a patch to let virtualenv do it. I'd like to get some feedback > from other OS X virtualenv users, though. I've thought maybe the distutils list is the best place for discussion; at least for the moment (better than private email, where no one else will see it). If activity increases much perhaps a separate mailing list will make sense (not everyone who would want to discuss virtualenv will want to be on the distutils list). I think Ronald Oussoren is on that list, but just in case I'll copy him since he's done the Mac specific stuff in virtualenv so far. -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From noah.gift at gmail.com Wed Jan 9 01:54:02 2008 From: noah.gift at gmail.com (Noah) Date: Tue, 8 Jan 2008 19:54:02 -0500 Subject: [Distutils] virtualenv, OS X, and GUIs In-Reply-To: <4784199A.6040805@colorstudy.com> References: <47840BC7.8010801@gmail.com> <4784199A.6040805@colorstudy.com> Message-ID: I think a list just for virtualenv makes a lot of sense. I also think we need a wiki for virtualenv recipes. I also think Ian needs a PR firm just to follow him around all day a to promote his code:) Our local user group is going to have a small talk/ screencast on virtualenv so it would be nice to have a link to it On Jan 8, 2008, at 7:47 PM, Ian Bicking wrote: > Robert Kern wrote: >> Is there an appropriate mailing list for talking about virtualenv? >> I've >> run into a problem where virtualenvs on OS X cannot access the >> screen to >> run GUIs. I've found a solution that appears to work and am going to >> write a patch to let virtualenv do it. I'd like to get some feedback >> from other OS X virtualenv users, though. > > I've thought maybe the distutils list is the best place for > discussion; > at least for the moment (better than private email, where no one else > will see it). If activity increases much perhaps a separate mailing > list will make sense (not everyone who would want to discuss > virtualenv > will want to be on the distutils list). > > I think Ronald Oussoren is on that list, but just in case I'll copy > him > since he's done the Mac specific stuff in virtualenv so far. > > -- > Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig From ronaldoussoren at mac.com Wed Jan 9 07:53:43 2008 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 9 Jan 2008 07:53:43 +0100 Subject: [Distutils] virtualenv, OS X, and GUIs In-Reply-To: <4784199A.6040805@colorstudy.com> References: <47840BC7.8010801@gmail.com> <4784199A.6040805@colorstudy.com> Message-ID: <6E81D276-04C5-4266-9A01-B279A27122E6@mac.com> On 9 Jan, 2008, at 1:47, Ian Bicking wrote: > Robert Kern wrote: >> Is there an appropriate mailing list for talking about virtualenv? >> I've run into a problem where virtualenvs on OS X cannot access the >> screen to run GUIs. I've found a solution that appears to work and >> am going to write a patch to let virtualenv do it. I'd like to get >> some feedback from other OS X virtualenv users, though. What's the solution? It is currently not possible to use GUI api's from the commandline using a virtualenv interpreter, but that's just basic OSX lameness. It works in the system install because that uses some trickery to let the system think that the interpreter is inside an application bundle (at the cost of an extra fork(2)). >> > > I've thought maybe the distutils list is the best place for > discussion; at least for the moment (better than private email, > where no one else will see it). If activity increases much perhaps > a separate mailing list will make sense (not everyone who would want > to discuss virtualenv will want to be on the distutils list). > > I think Ronald Oussoren is on that list, but just in case I'll copy > him since he's done the Mac specific stuff in virtualenv so far. I'm on the distutils list, but don't read that daily. I don't think the virtualenv list is the right place for discussion of virtualenv though. Ronald > > > -- > Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20080109/93297523/attachment.bin From robert.kern at gmail.com Wed Jan 9 08:14:39 2008 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 09 Jan 2008 01:14:39 -0600 Subject: [Distutils] virtualenv, OS X, and GUIs In-Reply-To: <6E81D276-04C5-4266-9A01-B279A27122E6@mac.com> References: <47840BC7.8010801@gmail.com> <4784199A.6040805@colorstudy.com> <6E81D276-04C5-4266-9A01-B279A27122E6@mac.com> Message-ID: Ronald Oussoren wrote: > > On 9 Jan, 2008, at 1:47, Ian Bicking wrote: > >> Robert Kern wrote: >>> Is there an appropriate mailing list for talking about virtualenv? >>> I've run into a problem where virtualenvs on OS X cannot access the >>> screen to run GUIs. I've found a solution that appears to work and am >>> going to write a patch to let virtualenv do it. I'd like to get some >>> feedback from other OS X virtualenv users, though. > > What's the solution? > > It is currently not possible to use GUI api's from the commandline using > a virtualenv interpreter, but that's just basic OSX lameness. It works > in the system install because that uses some trickery to let the system > think that the interpreter is inside an application bundle (at the cost > of an extra fork(2)). Basically, I replicated this procedure for the ~/virtualenv/. I copied the Python.app/ into the ~/virtualenv/, used the install_name_tool to point it ~/virtualenv/.Python, and then compiled pythonw.c with a -D flag to point it to the executable in the copied Python.app. It's not particularly pretty, but it worked for me with python.org's Python 2.5 on Leopard. That's good enough for me, but I'm wondering how robust that is going to be for others. Which framework builds do not have a Python.app? I notice that Leopard's 2.3 doesn't, so I'm guessing that Tiger's didn't either. Were there releases of 2.4 that didn't? Should I bother trying to support Python 2.3? It doesn't look like virtualenv tries to. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ianb at colorstudy.com Wed Jan 9 08:47:50 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 09 Jan 2008 01:47:50 -0600 Subject: [Distutils] virtualenv group Message-ID: <47847C26.1000605@colorstudy.com> I've created a Google Group for virtualenv discussion: http://groups.google.com/group/python-virtualenv/ I also set up a bug tracker on Launchpad: https://bugs.launchpad.net/virtualenv/ -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From ziade.tarek at gmail.com Wed Jan 9 17:42:40 2008 From: ziade.tarek at gmail.com (=?UTF-8?Q?Tarek_Ziad=C3=A9?=) Date: Wed, 9 Jan 2008 08:42:40 -0800 (PST) Subject: [Distutils] zc.buildout : an offline bootstrap.py (was: Re: zc.buildout : more infos on download errors) In-Reply-To: <3D1F048B-4696-4A8C-95C0-35467B43D856@zope.com> References: <14686910.post@talk.nabble.com> <3D1F048B-4696-4A8C-95C0-35467B43D856@zope.com> Message-ID: <14715814.post@talk.nabble.com> Done. I have another problem I would like to express: the boostrap.py file is not behaving as expected when the buildout is used in offline mode. If the user is not connected to the internet and does not have setuptools installed, bootstrap.py will fail because it will try to import it then to get it through the web. The right behavior and order would be : 1. try to see if setuptools and zc.buildout are present in eggs/ 2. try to see if they are available in the environment 3. try to easy_install it That makes a buildout runnable without internet, with just a few elements: - the prepared eggs folder - the prepared download-cache folder - buildout.cfg - bootstrap.py We did a small change to be able to have this behavior: https://ingeniweb.svn.sourceforge.net/svnroot/ingeniweb/IngeniSkel/trunk/ingeniskel/templates/iw_plone_project/buildout/bootstrap.py And I think this principle could be applied in zc.buildout as well Regards Tarek Jim Fulton wrote: > > > On Jan 8, 2008, at 5:51 AM, Tarek Ziad? wrote: > >> >> Hello, >> >> When a server that is supposed to provide an agg is timing out (it >> happens >> sometime, >> for some libs like PIL, pysqlite, etc) zc.buildout returns a warning >> like >> this: >> >> Download error: (104, 'Connection reset by peer') -- Some packages >> may not >> be found! >> >> In the flow of actions, it is very hard to tell which url is down. I >> had to >> patch urlopen >> to catch it. >> >> I think zc.buildout could be more verbose here, and tell which >> package or >> url failed > > > Agreed. You should submit a bug report. Note that buildout is using > setuptools, which uses urllib2 which is maddeningly uninformative in > it's error reporting. > > Jim > > -- > Jim Fulton > Zope Corporation > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > > -- View this message in context: http://www.nabble.com/zc.buildout-%3A-more-infos-on-download-errors-tp14686910p14715814.html Sent from the Python - distutils-sig mailing list archive at Nabble.com. From stephenemslie at gmail.com Thu Jan 10 18:20:11 2008 From: stephenemslie at gmail.com (stephen emslie) Date: Thu, 10 Jan 2008 17:20:11 +0000 Subject: [Distutils] reStructuredText whitespace lost when writing pkg-info Message-ID: <51f97e530801100920h5b8f7b1an50c57271fbbba466@mail.gmail.com> I've noticed that setuptools seems to flatten any indentation in fields when generating the PKG-INFO metadata file. I noticed this because PEP345 [1] suggests that the long description field can be written in reStructuredText (which would be excellent for my current project), but unfortunately whitespace is being lost that breaks things like literal blocks that require indentation. The trouble seems to be that distutils.util.rfc822_escape strips all excess whitespace and adds 8 spaces of padding to the start of every line. For example: >>> rest = """ ... a literal python block:: ... >>> import this ... """ >>> print distutils.util.rfc822_escape(rest) a literal python block:: >>> import this which generates a few warnings and no literal block when docutils tried to render it. Can anyone tell me if this behaviour is necessary to be rfc822/MIME compliant, or is distutils being over-cautious in flattening out all whitespace? The w3c discussion on multiple lines in rfc822 [2] seems to suggest that whitespace can be 'unfolded' safely, so it seems a shame to be throwing it away when it can have important meaning. [1] http://python.cofman.dk/peps/pep-0345.html [2] http://www.w3.org/Protocols/rfc822/3_Lexical.html Stephen Emslie From hervold at gmail.com Thu Jan 10 19:02:07 2008 From: hervold at gmail.com (kieran hervold) Date: Thu, 10 Jan 2008 10:02:07 -0800 Subject: [Distutils] Passing compiler options via the pydistutils.cfg file Message-ID: <6fcd10f70801101002g3339a9cdw2aca791500b7fff2@mail.gmail.com> I've run into a problem using mod_python under OS X Leopard: because the apache2 process runs in 64-bit mode, the python interpreter is forced into 64-bit mode, which breaks compatibility with any Python library with underlying C code. I suspect that adding a compiler flag specifying "-arch x86_64" to any gcc invocation would fix the problem, but I'd rather not do this package by package: I'd like to set one global parameter and be done with it. Apparently, I can add an entry to the global pydistutils.cfg file. However, I can't seem to find the appropriate parameter. The pydistutils.cfg file follows the nomenclature of command-line setup.py invocation, but I haven't found a C compiler pass-through parameter for setup.py. Does such a parameter exist? If I want to add a flag ("-arch x86_64", in this case) to the C compiler invocation, how do I pass that to setup.py? Thanks for any help, kieran hervold From pje at telecommunity.com Thu Jan 10 19:18:53 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 10 Jan 2008 13:18:53 -0500 Subject: [Distutils] Passing compiler options via the pydistutils.cfg file In-Reply-To: <6fcd10f70801101002g3339a9cdw2aca791500b7fff2@mail.gmail.co m> References: <6fcd10f70801101002g3339a9cdw2aca791500b7fff2@mail.gmail.com> Message-ID: <20080110181855.DC5A53A406C@sparrow.telecommunity.com> At 10:02 AM 1/10/2008 -0800, kieran hervold wrote: >I've run into a problem using mod_python under OS X Leopard: because >the apache2 process runs in 64-bit mode, the python interpreter is >forced into 64-bit mode, which breaks compatibility with any Python >library with underlying C code. > >I suspect that adding a compiler flag specifying "-arch x86_64" to any >gcc invocation would fix the problem, but I'd rather not do this >package by package: I'd like to set one global parameter and be done >with it. > >Apparently, I can add an entry to the global pydistutils.cfg file. >However, I can't seem to find the appropriate parameter. The >pydistutils.cfg file follows the nomenclature of command-line setup.py >invocation, but I haven't found a C compiler pass-through parameter >for setup.py. > >Does such a parameter exist? Apparently not: $ python setup.py build_ext --help ... Options for 'build_ext' command: --build-lib (-b) directory for compiled extension modules --build-temp (-t) directory for temporary files (build by-products) --inplace (-i) ignore build-lib and put compiled extensions into the source directory alongside your pure Python modules --include-dirs (-I) list of directories to search for header files (separated by ';') --define (-D) C preprocessor macros to define --undef (-U) C preprocessor macros to undefine --libraries (-l) external C libraries to link with --library-dirs (-L) directories to search for external C libraries (separated by ';') --rpath (-R) directories to search for shared C libraries at runtime --link-objects (-O) extra explicit link objects to include in the link --debug (-g) compile/link with debugging information --force (-f) forcibly build everything (ignore file timestamps) --compiler (-c) specify the compiler type --swig-cpp make SWIG create C++ files (default is C) --swig-opts list of SWIG command line options --swig path to the SWIG executable --help-compiler list available compilers > If I want to add a flag ("-arch x86_64", >in this case) to the C compiler invocation, how do I pass that to >setup.py? There's probably an environment variable you can use, like CFLAGS or some such; check the distutils.sysconfig module to see if you can find something useful. Otherwise, you may have to have apache run your setup.py files to do the building. From michael.dunstan at gmail.com Sat Jan 12 00:50:34 2008 From: michael.dunstan at gmail.com (Michael Dunstan) Date: Sat, 12 Jan 2008 12:50:34 +1300 Subject: [Distutils] zc.buildout and PYTHON_EGG_CACHE Message-ID: <99b561f50801111550m1f1c0889tebb810fc3b8d60c1@mail.gmail.com> Is there a sane way to configure a common PYTHON_EGG_CACHE using zc.buildout? By default PYTHON_EGG_CACHE is a path relative to the current users home directory which can be a permissions hassle if the built-out processes run as a restricted user. There are some details on how to configure PYTHON_EGG_CACHE outside of buildout: http://mail.python.org/pipermail/distutils-sig/2006-November/006909.html And an example egg that ends up being extracted to PYTHON_EGG_CACHE is simplejson. -- Michael Dunstan From jim at zope.com Sat Jan 12 00:54:49 2008 From: jim at zope.com (Jim Fulton) Date: Fri, 11 Jan 2008 18:54:49 -0500 Subject: [Distutils] zc.buildout and PYTHON_EGG_CACHE In-Reply-To: <99b561f50801111550m1f1c0889tebb810fc3b8d60c1@mail.gmail.com> References: <99b561f50801111550m1f1c0889tebb810fc3b8d60c1@mail.gmail.com> Message-ID: <0248B37B-5A4B-4AD6-8B20-A635B025A6A0@zope.com> I wasn't aware that this API existed. I'll look into this. Do you have any suggestion for how you might want to configure this in buildout? Jim On Jan 11, 2008, at 6:50 PM, Michael Dunstan wrote: > Is there a sane way to configure a common PYTHON_EGG_CACHE using > zc.buildout? > > By default PYTHON_EGG_CACHE is a path relative to the current users > home directory which can be a permissions hassle if the built-out > processes run as a restricted user. There are some details on how to > configure PYTHON_EGG_CACHE outside of buildout: > > http://mail.python.org/pipermail/distutils-sig/2006-November/006909.html > > And an example egg that ends up being extracted to PYTHON_EGG_CACHE is > simplejson. > > -- > Michael Dunstan > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- Jim Fulton Zope Corporation From michael.dunstan at gmail.com Sat Jan 12 00:58:47 2008 From: michael.dunstan at gmail.com (Michael Dunstan) Date: Sat, 12 Jan 2008 12:58:47 +1300 Subject: [Distutils] zc.buildout and PYTHON_EGG_CACHE In-Reply-To: <0248B37B-5A4B-4AD6-8B20-A635B025A6A0@zope.com> References: <99b561f50801111550m1f1c0889tebb810fc3b8d60c1@mail.gmail.com> <0248B37B-5A4B-4AD6-8B20-A635B025A6A0@zope.com> Message-ID: <99b561f50801111558h118b69fma591af076bd6d010@mail.gmail.com> How about something in the buildout section? So that could look like: [buildout] python-egg-cache = /some/where/here/or/there On Jan 12, 2008 12:54 PM, Jim Fulton wrote: > I wasn't aware that this API existed. I'll look into this. Do you > have any suggestion for how you might want to configure this in > buildout? > > Jim > > > On Jan 11, 2008, at 6:50 PM, Michael Dunstan wrote: > > > Is there a sane way to configure a common PYTHON_EGG_CACHE using > > zc.buildout? > > > > By default PYTHON_EGG_CACHE is a path relative to the current users > > home directory which can be a permissions hassle if the built-out > > processes run as a restricted user. There are some details on how to > > configure PYTHON_EGG_CACHE outside of buildout: > > > > http://mail.python.org/pipermail/distutils-sig/2006-November/006909.html > > > > And an example egg that ends up being extracted to PYTHON_EGG_CACHE is > > simplejson. > > > > -- > > Michael Dunstan > > _______________________________________________ > > Distutils-SIG maillist - Distutils-SIG at python.org > > http://mail.python.org/mailman/listinfo/distutils-sig > > -- > Jim Fulton > Zope Corporation > > > -- Michael Dunstan From jim at zope.com Sat Jan 12 01:04:23 2008 From: jim at zope.com (Jim Fulton) Date: Fri, 11 Jan 2008 19:04:23 -0500 Subject: [Distutils] zc.buildout and PYTHON_EGG_CACHE In-Reply-To: <99b561f50801111558h118b69fma591af076bd6d010@mail.gmail.com> References: <99b561f50801111550m1f1c0889tebb810fc3b8d60c1@mail.gmail.com> <0248B37B-5A4B-4AD6-8B20-A635B025A6A0@zope.com> <99b561f50801111558h118b69fma591af076bd6d010@mail.gmail.com> Message-ID: <40748809-C7DA-47A9-B97A-478E01A9D8C2@zope.com> On Jan 11, 2008, at 6:58 PM, Michael Dunstan wrote: > How about something in the buildout section? So that could look like: > > [buildout] > python-egg-cache = /some/where/here/or/there OK, what happens if this option isn't present? Jim -- Jim Fulton Zope Corporation From michael.dunstan at gmail.com Sat Jan 12 01:08:14 2008 From: michael.dunstan at gmail.com (Michael Dunstan) Date: Sat, 12 Jan 2008 13:08:14 +1300 Subject: [Distutils] zc.buildout and PYTHON_EGG_CACHE In-Reply-To: <40748809-C7DA-47A9-B97A-478E01A9D8C2@zope.com> References: <99b561f50801111550m1f1c0889tebb810fc3b8d60c1@mail.gmail.com> <0248B37B-5A4B-4AD6-8B20-A635B025A6A0@zope.com> <99b561f50801111558h118b69fma591af076bd6d010@mail.gmail.com> <40748809-C7DA-47A9-B97A-478E01A9D8C2@zope.com> Message-ID: <99b561f50801111608i2975b27n4e724b17c6a9df87@mail.gmail.com> On Jan 12, 2008 1:04 PM, Jim Fulton wrote: > > On Jan 11, 2008, at 6:58 PM, Michael Dunstan wrote: > > > How about something in the buildout section? So that could look like: > > > > [buildout] > > python-egg-cache = /some/where/here/or/there > > > OK, what happens if this option isn't present? I'm not too sure. My first reaction is just business as usual - leave that untouched and PYTHON_EGG_CACHE will default to ~/.python-eggs -- Michael Dunstan From noah.gift at gmail.com Sat Jan 12 01:36:57 2008 From: noah.gift at gmail.com (Noah) Date: Fri, 11 Jan 2008 19:36:57 -0500 Subject: [Distutils] zc.buildout and PYTHON_EGG_CACHE In-Reply-To: <99b561f50801111608i2975b27n4e724b17c6a9df87@mail.gmail.com> References: <99b561f50801111550m1f1c0889tebb810fc3b8d60c1@mail.gmail.com> <0248B37B-5A4B-4AD6-8B20-A635B025A6A0@zope.com> <99b561f50801111558h118b69fma591af076bd6d010@mail.gmail.com> <40748809-C7DA-47A9-B97A-478E01A9D8C2@zope.com> <99b561f50801111608i2975b27n4e724b17c6a9df87@mail.gmail.com> Message-ID: <26E7087F-519D-4E4E-AFC5-347DD33C7B1A@gmail.com> +1 I want to change my grokproject egg cache. And was wondering how. On Jan 11, 2008, at 7:08 PM, "Michael Dunstan" wrote: > On Jan 12, 2008 1:04 PM, Jim Fulton wrote: >> >> On Jan 11, 2008, at 6:58 PM, Michael Dunstan wrote: >> >>> How about something in the buildout section? So that could look >>> like: >>> >>> [buildout] >>> python-egg-cache = /some/where/here/or/there >> >> >> OK, what happens if this option isn't present? > > I'm not too sure. My first reaction is just business as usual - leave > that untouched and PYTHON_EGG_CACHE will default to ~/.python-eggs > > > -- > Michael Dunstan > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig From pje at telecommunity.com Sat Jan 12 04:00:45 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 11 Jan 2008 22:00:45 -0500 Subject: [Distutils] zc.buildout and PYTHON_EGG_CACHE In-Reply-To: <99b561f50801111608i2975b27n4e724b17c6a9df87@mail.gmail.com > References: <99b561f50801111550m1f1c0889tebb810fc3b8d60c1@mail.gmail.com> <0248B37B-5A4B-4AD6-8B20-A635B025A6A0@zope.com> <99b561f50801111558h118b69fma591af076bd6d010@mail.gmail.com> <40748809-C7DA-47A9-B97A-478E01A9D8C2@zope.com> <99b561f50801111608i2975b27n4e724b17c6a9df87@mail.gmail.com> Message-ID: <20080112030045.ED88B3A40A2@sparrow.telecommunity.com> At 01:08 PM 1/12/2008 +1300, Michael Dunstan wrote: >On Jan 12, 2008 1:04 PM, Jim Fulton wrote: > > > > On Jan 11, 2008, at 6:58 PM, Michael Dunstan wrote: > > > > > How about something in the buildout section? So that could look like: > > > > > > [buildout] > > > python-egg-cache = /some/where/here/or/there > > > > > > OK, what happens if this option isn't present? > >I'm not too sure. My first reaction is just business as usual - leave >that untouched and PYTHON_EGG_CACHE will default to ~/.python-eggs Or more precisely, the cache directory will default to PYTHON_EGG_CACHE, if it isn't explicitly set via the API. From pjenvey at groovie.org Sat Jan 12 04:57:29 2008 From: pjenvey at groovie.org (Philip Jenvey) Date: Fri, 11 Jan 2008 19:57:29 -0800 Subject: [Distutils] setuptools on jython Message-ID: <36A10A2D-4103-45D5-BBCF-E8DE60B49880@groovie.org> Jython trunk has made a lot of progress on setuptools support. The gory details are here: http://wiki.python.org/jython/SetuptoolsOnJython The final few issues require some changes to setuptools itself: o usage of os.chmod. most uses are actually wrapped in an os.name == 'posix', or except AttributeError checks, but one was missed (Jython lacks os.chmod) o a unit test assumed CPython dict ordering I've attached a patch against the 0.6 branch for both of these issues. Though there's one more issue that'll be more difficult to deal with: setuptools uses marshal.load to read variables (via co_names and co_consts) from a module's bytecode (pyc/o) without importing it. Jython and IronPython don't compile to .pyc, nor do their code objects support co_names and co_consts, so this isn't portable. marshal.load is used in a couple places: o setuptools.depends.extract_constant, which is only used in setuptools by Require.get_version(). I don't know when this is used or what it's used for o to find un-zip_safe variables/calls when a distribution doesn't specify zip_safetyness. Probably the easiest way to deal with this is to just fallback to zip_safe=False on these platforms -------------- next part -------------- A non-text attachment was scrubbed... Name: jython_compat-r59690.diff Type: application/octet-stream Size: 2254 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20080111/84191a15/attachment.obj -------------- next part -------------- -- Philip Jenvey From pje at telecommunity.com Sat Jan 12 05:51:42 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 11 Jan 2008 23:51:42 -0500 Subject: [Distutils] setuptools on jython In-Reply-To: <36A10A2D-4103-45D5-BBCF-E8DE60B49880@groovie.org> References: <36A10A2D-4103-45D5-BBCF-E8DE60B49880@groovie.org> Message-ID: <20080112045144.439313A40A2@sparrow.telecommunity.com> At 07:57 PM 1/11/2008 -0800, Philip Jenvey wrote: >Jython trunk has made a lot of progress on setuptools support. The >gory details are here: > >http://wiki.python.org/jython/SetuptoolsOnJython By the way, the reason setuptools uses -E for the subprocess is so that Python will ignore any environment variables such as PYTHONPATH et al. It's not clear to me whether making -E a no-op for Jython will actually make this work correctly. I also have some concerns about how the $py.class stuff will work with distutils in general as well as setuptools in particular, but I've tried as much as possible to make setuptools follow the distutils rather than do its own thing. So hopefully most of the relevant things will be taken care of by Jython-level distutils patches. >The final few issues require some changes to setuptools itself: > >o usage of os.chmod. most uses are actually wrapped in an os.name == >'posix', or except AttributeError checks, but one was missed (Jython >lacks os.chmod) > >o a unit test assumed CPython dict ordering > >I've attached a patch against the 0.6 branch for both of these issues. Thanks! >Though there's one more issue that'll be more difficult to deal with: >setuptools uses marshal.load to read variables (via co_names and >co_consts) from a module's bytecode (pyc/o) without importing it. >Jython and IronPython don't compile to .pyc, nor do their code >objects support co_names and co_consts, so this isn't portable. > >marshal.load is used in a couple places: > >o setuptools.depends.extract_constant, which is only used in >setuptools by Require.get_version(). I don't know when this is used >or what it's used for setuptools.depends is essentially an experimental pre-egg feature that shouldn't be used by anything that's supported or documented at the moment. >o to find un-zip_safe variables/calls when a distribution doesn't >specify zip_safetyness. Probably the easiest way to deal with this is >to just fallback to zip_safe=False on these platforms That's probably reasonable. An alternative would be to use the tokenize module to read the source code, since all it really does is look at name tokens and string constants, which can both be identified via tokenization without any higher-level parsing. From pwang at enthought.com Sat Jan 12 16:47:02 2008 From: pwang at enthought.com (Peter Wang) Date: Sat, 12 Jan 2008 09:47:02 -0600 Subject: [Distutils] That setuptools/Pyrex problem with swig_sources Message-ID: Hi everyone, I ran into this problem (traceback below) the other day with setuptools 0.6c7 and Pyrex 0.9.6.4 on python 2.5. I saw Phillip's post from October regarding this: http://lists.copyleft.no/pipermail/pyrex/2007-October/002867.html How do I get this setuptools fix? Moreover, what is the best recommendation for others who are trying to build my project? Is there some monkeypatching I can do in my setup.py to work around this problem? (I see that the problem itself results from a bunch of monkeypatching...) Any help would be greatly appreciated. Pyrex is required to build Chaco on the Mac, and all the Enthought projects use eggs heavily, so this is sort of a fundamental problem for us... Thanks, Peter --------------------------- (repotest)pwang at hyperion ~/src/Chaco_3.0.0b1/Chaco_3.0.0b1$ python setup.py bdist_egg running bdist_egg running egg_info writing requirements to Chaco.egg-info/requires.txt writing Chaco.egg-info/PKG-INFO writing namespace_packages to Chaco.egg-info/namespace_packages.txt writing top-level names to Chaco.egg-info/top_level.txt writing dependency_links to Chaco.egg-info/dependency_links.txt writing manifest file 'Chaco.egg-info/SOURCES.txt' installing library code to build/bdist.macosx-10.3-fat/egg running install_lib running build_py running build_ext building 'enthought.chaco2.contour.contour' extension Traceback (most recent call last): File "setup.py", line 77, in zip_safe = False, File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/distutils/core.py", line 151, in setup dist.run_commands() File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/distutils/dist.py", line 974, in run_commands self.run_command(cmd) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/Users/pwang/envs/repotest/lib/python2.5/site-packages/ setuptools-0.6c7-py2.5.egg/setuptools/command/bdist_egg.py", line 174, in run File "/Users/pwang/envs/repotest/lib/python2.5/site-packages/ setuptools-0.6c7-py2.5.egg/setuptools/command/bdist_egg.py", line 161, in call_command File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/Users/pwang/envs/repotest/lib/python2.5/site-packages/ setuptools-0.6c7-py2.5.egg/setuptools/command/install_lib.py", line 20, in run File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/distutils/command/install_lib.py", line 112, in build self.run_command('build_ext') File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/Users/pwang/envs/repotest/lib/python2.5/site-packages/ setuptools-0.6c7-py2.5.egg/setuptools/command/build_ext.py", line 46, in run File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/distutils/command/build_ext.py", line 290, in run self.build_extensions() File "/Users/pwang/envs/repotest/lib/python2.5/site-packages/ Pyrex-0.9.6.4-py2.5.egg/Pyrex/Distutils/build_ext.py", line 82, in build_extensions self.build_extension(ext) File "/Users/pwang/envs/repotest/lib/python2.5/site-packages/ setuptools-0.6c7-py2.5.egg/setuptools/command/build_ext.py", line 175, in build_extension File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/distutils/command/build_ext.py", line 453, in build_extension sources = self.swig_sources(sources, ext) File "/Users/pwang/envs/repotest/lib/python2.5/site-packages/ setuptools-0.6c7-py2.5.egg/setuptools/command/build_ext.py", line 77, in swig_sources TypeError: swig_sources() takes exactly 3 arguments (2 given) From pjenvey at groovie.org Sun Jan 13 02:57:48 2008 From: pjenvey at groovie.org (Philip Jenvey) Date: Sat, 12 Jan 2008 17:57:48 -0800 Subject: [Distutils] setuptools on jython In-Reply-To: <20080112045144.439313A40A2@sparrow.telecommunity.com> References: <36A10A2D-4103-45D5-BBCF-E8DE60B49880@groovie.org> <20080112045144.439313A40A2@sparrow.telecommunity.com> Message-ID: <123A3D33-41DE-4837-907F-FB062279E4D5@groovie.org> On Jan 11, 2008, at 8:51 PM, Phillip J. Eby wrote: > At 07:57 PM 1/11/2008 -0800, Philip Jenvey wrote: >> Jython trunk has made a lot of progress on setuptools support. The >> gory details are here: >> >> http://wiki.python.org/jython/SetuptoolsOnJython > > By the way, the reason setuptools uses -E for the subprocess is so > that Python will ignore any environment variables such as > PYTHONPATH et al. It's not clear to me whether making -E a no-op > for Jython will actually make this work correctly. Right, Jython doesn't support environment variables (a historical thing) -- its equivalent is the 'registry', a java properties file. I actually held off on having -E disable the registry because I have a feeling there could be evil repercussions, like jython not working at all in some environments. I have to investigate more before going that route. > I also have some concerns about how the $py.class stuff will work > with distutils in general as well as setuptools in particular, but > I've tried as much as possible to make setuptools follow the > distutils rather than do its own thing. So hopefully most of the > relevant things will be taken care of by Jython-level distutils > patches. > distutils support was just recently added, specifically for setuptools =]. It probably has some rough edges still -- but I'm thinking setuptools will be in pretty good shape after these two patches. > >> o to find un-zip_safe variables/calls when a distribution doesn't >> specify zip_safetyness. Probably the easiest way to deal with this is >> to just fallback to zip_safe=False on these platforms > > That's probably reasonable. An alternative would be to use the > tokenize module to read the source code, since all it really does > is look at name tokens and string constants, which can both be > identified via tokenization without any higher-level parsing. > There's always the rare case of an egg containing only byte code, though. Not that I've ever seen eggs like this, but I've heard talk of them before. Attached is a simple patch to disable the zip_safe scan on Jython/ IronPython. Maybe in the future we can use tokenize when all of the source code is present in an archive. -------------- next part -------------- A non-text attachment was scrubbed... Name: jython_zip_safe-r59934.diff Type: application/octet-stream Size: 978 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20080112/235e8f96/attachment.obj -------------- next part -------------- -- Philip Jenvey From billiejoex at gmail.com Mon Jan 14 02:14:05 2008 From: billiejoex at gmail.com (Giampaolo Rodola') Date: Mon, 14 Jan 2008 02:14:05 +0100 Subject: [Distutils] pypi and easy_install In-Reply-To: <20071220165342.3F72A3A410C@sparrow.telecommunity.com> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <20071220034213.315F63A40A4@sparrow.telecommunity.com> <729626cc0712191947s18683928s71ec609dfe562166@mail.gmail.com> <20071220045912.806693A40A4@sparrow.telecommunity.com> <729626cc0712200526q216165d4h5b2d55d59f3fb7db@mail.gmail.com> <20071220165342.3F72A3A410C@sparrow.telecommunity.com> Message-ID: <729626cc0801131714l39547a35ld75bdd8cde411d23@mail.gmail.com> > (This is a distutils feature and doesn't require setuptools, although > the distutils are a bit trickier to get the tarball to include > everything, as you have to explicitly list in MANIFEST.in anything > that distutils doesn't automatically pick up. Sorry for replying so late. Gently, I would be interested in knowing how can I have distutils to include everything. Currently I've tried to use setup.py sdist but CREDITS, HISTORY, INSTALL and LICENSE files aren't included in the tarball. 2007/12/20, Phillip J. Eby : > At 02:26 PM 12/20/2007 +0100, Giampaolo Rodola' wrote: > >Oh, I'm sorry, > >yestarday night it was very late (5:00 AM) and I completely > >misunderstood what you said. > >Thanks for your help, I'm going to rename the tarball. > > It will be better for you in general if you just use "setup.py sdist" > to generate your tarballs, as they will then have the right name, and > also will include the distutils PKG-INFO. > > (This is a distutils feature and doesn't require setuptools, although > the distutils are a bit trickier to get the tarball to include > everything, as you have to explicitly list in MANIFEST.in anything > that distutils doesn't automatically pick up. Setuptools on the > other hand will automatically include anything that's under revision > control in your project, unless you use MANIFEST.in to tell it *not* > to. CVS+SVN support is built in, plugins for various other revision > control systems can be found on PyPI.) > > From pje at telecommunity.com Mon Jan 14 03:29:07 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 13 Jan 2008 21:29:07 -0500 Subject: [Distutils] pypi and easy_install In-Reply-To: <729626cc0801131714l39547a35ld75bdd8cde411d23@mail.gmail.co m> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <20071220034213.315F63A40A4@sparrow.telecommunity.com> <729626cc0712191947s18683928s71ec609dfe562166@mail.gmail.com> <20071220045912.806693A40A4@sparrow.telecommunity.com> <729626cc0712200526q216165d4h5b2d55d59f3fb7db@mail.gmail.com> <20071220165342.3F72A3A410C@sparrow.telecommunity.com> <729626cc0801131714l39547a35ld75bdd8cde411d23@mail.gmail.com> Message-ID: <20080114022910.6D0D33A4077@sparrow.telecommunity.com> At 02:14 AM 1/14/2008 +0100, Giampaolo Rodola' wrote: > > (This is a distutils feature and doesn't require setuptools, although > > the distutils are a bit trickier to get the tarball to include > > everything, as you have to explicitly list in MANIFEST.in anything > > that distutils doesn't automatically pick up. > >Sorry for replying so late. >Gently, I would be interested in knowing how can I have distutils to >include everything. >Currently I've tried to use setup.py sdist but CREDITS, HISTORY, >INSTALL and LICENSE files aren't included in the tarball. http://python.org/doc/2.4.1/dist/manifest.html From p.f.moore at gmail.com Mon Jan 14 12:24:37 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 14 Jan 2008 11:24:37 +0000 Subject: [Distutils] Finding my data_files Message-ID: <79990c6b0801140324i1ff79105pfe572e4d1f21d9fa@mail.gmail.com> If I have a module which installs some files using the data_files=[...] argument to setup.py, how do I then (reliably) find those files from my code? The first question is, is there a way of obtaining the distutils data_files parameter value at runtime? If not, I guess I have to code the same information in my code as is in the setup.py script. Not ideal, but OK... The second question is, if my files are installed using a relative path, then the documentation says (section 2.7) "If directory is a relative path, it is interpreted relative to the installation prefix (Python's sys.prefix for pure-Python packages, sys.exec_prefix for packages that contain extension modules)." That's OK, but again it sounds like I need to code knowledge of whether the setup includes extension modules or not, into my code. OK, if I must, then I must, but I'd rather not do so and find out I missed something obvious... One final question, is it possible (barring obvious hacks like checking if sys.prefix or sys.exec_prefix is a prefix of module.__file__) to determine if I'm running from an installed or a development copy of my module? I have a horrible feeling that the answer to most of this will be "use setuptools", but I don't have that option, as this is not my code (it's MoinMoin, for what it's worth). Thanks, Paul. From iv at lantic.net Mon Jan 14 11:40:39 2008 From: iv at lantic.net (Iwan Vosloo) Date: Mon, 14 Jan 2008 12:40:39 +0200 Subject: [Distutils] Egg depency order Message-ID: <1200307240.728.33.camel@localhost.localdomain> Hi there, I need to get a list of eggs in 'dependency' order: a list with eggs (Distribution objects?) which is sorted so that earlier entries will never depend on entries later in the list. [1] I spotted the _dep_map attribute on Distribution objects and it seems to have something like that already. Exactly what is in _dep_map? Could I use it to compute what I need? - Iwan [1] In fact, this is an intermediary result - I want to compute all the entry points of a tree of dependencies in such a way that the entry points of eggs deeper in the tree will always be earlier in the list than entry points for those shallower in the tree. From pje at telecommunity.com Mon Jan 14 15:42:25 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 14 Jan 2008 09:42:25 -0500 Subject: [Distutils] Finding my data_files In-Reply-To: <79990c6b0801140324i1ff79105pfe572e4d1f21d9fa@mail.gmail.co m> References: <79990c6b0801140324i1ff79105pfe572e4d1f21d9fa@mail.gmail.com> Message-ID: <20080114144230.91E803A4077@sparrow.telecommunity.com> At 11:24 AM 1/14/2008 +0000, Paul Moore wrote: >If I have a module which installs some files using the >data_files=[...] argument to setup.py, how do I then (reliably) find >those files from my code? You can't. Use package_data instead. It's available in the distutils as of Python 2.4. From pje at telecommunity.com Mon Jan 14 15:45:29 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 14 Jan 2008 09:45:29 -0500 Subject: [Distutils] Egg depency order In-Reply-To: <1200307240.728.33.camel@localhost.localdomain> References: <1200307240.728.33.camel@localhost.localdomain> Message-ID: <20080114144529.8A13C3A4077@sparrow.telecommunity.com> At 12:40 PM 1/14/2008 +0200, Iwan Vosloo wrote: >Hi there, > >I need to get a list of eggs in 'dependency' order: a list with eggs >(Distribution objects?) which is sorted so that earlier entries will >never depend on entries later in the list. [1] > >I spotted the _dep_map attribute on Distribution objects and it seems to >have something like that already. > >Exactly what is in _dep_map? Could I use it to compute what I need? No. Use the .requires() method of a distribution instead, to get its immediate requirements, and then you'll need to do your own sort. Please see http://peak.telecommunity.com/DevCenter/PkgResources for the full pkg_resources API. There's no need for you to mess with private attributes for this stuff. >[1] In fact, this is an intermediary result - I want to compute all the >entry points of a tree of dependencies in such a way that the entry >points of eggs deeper in the tree will always be earlier in the list >than entry points for those shallower in the tree. Why? From ionel.mc at gmail.com Sun Jan 13 10:05:19 2008 From: ionel.mc at gmail.com (Ionel Maries Cristian) Date: Sun, 13 Jan 2008 11:05:19 +0200 Subject: [Distutils] setuptools, bdist_wininst and dependecies Message-ID: I'm using setuptools for my setup script. when installing a .exe package dependencies aren't fetched. should it work this way? are there any ways getting the dependencies downloaded and still have a bdist_wininst package ? -- ionel. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20080113/44214925/attachment.htm From pje at telecommunity.com Mon Jan 14 16:12:43 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 14 Jan 2008 10:12:43 -0500 Subject: [Distutils] setuptools, bdist_wininst and dependecies In-Reply-To: References: Message-ID: <20080114151242.BCB313A4077@sparrow.telecommunity.com> At 11:05 AM 1/13/2008 +0200, Ionel Maries Cristian wrote: >I'm using setuptools for my setup script. > >when installing a .exe package dependencies aren't fetched. >should it work this way? >are there any ways getting the dependencies downloaded and still >have a bdist_wininst package ? You could put something in the post-install script to run easy_install, I suppose. :) From ionel.mc at gmail.com Mon Jan 14 16:16:07 2008 From: ionel.mc at gmail.com (Ionel Maries Cristian) Date: Mon, 14 Jan 2008 17:16:07 +0200 Subject: [Distutils] setuptools, bdist_wininst and dependecies In-Reply-To: <20080114151242.BCB313A4077@sparrow.telecommunity.com> References: <20080114151242.BCB313A4077@sparrow.telecommunity.com> Message-ID: That is like scratching your right ear with your left hand. Regardless, is this a bug or a missing feature? On Jan 14, 2008 5:12 PM, Phillip J. Eby wrote: > At 11:05 AM 1/13/2008 +0200, Ionel Maries Cristian wrote: > >I'm using setuptools for my setup script. > > > >when installing a .exe package dependencies aren't fetched. > >should it work this way? > >are there any ways getting the dependencies downloaded and still > >have a bdist_wininst package ? > > You could put something in the post-install script to run > easy_install, I suppose. :) > > -- http://code.google.com/p/cogen/ ionel. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20080114/21a28723/attachment.htm From p.f.moore at gmail.com Mon Jan 14 16:32:21 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 14 Jan 2008 15:32:21 +0000 Subject: [Distutils] Finding my data_files In-Reply-To: <20080114144230.91E803A4077@sparrow.telecommunity.com> References: <79990c6b0801140324i1ff79105pfe572e4d1f21d9fa@mail.gmail.com> <20080114144230.91E803A4077@sparrow.telecommunity.com> Message-ID: <79990c6b0801140732i4547c120u9c791c7043b04514@mail.gmail.com> On 14/01/2008, Phillip J. Eby wrote: > At 11:24 AM 1/14/2008 +0000, Paul Moore wrote: > >If I have a module which installs some files using the > >data_files=[...] argument to setup.py, how do I then (reliably) find > >those files from my code? > > You can't. Use package_data instead. It's available in the > distutils as of Python 2.4. I thought that might be the answer. Unfortunately, MoinMoin still preserves compatibility with 2.3 :-( Paul. From pje at telecommunity.com Mon Jan 14 16:47:27 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 14 Jan 2008 10:47:27 -0500 Subject: [Distutils] Finding my data_files In-Reply-To: <79990c6b0801140732i4547c120u9c791c7043b04514@mail.gmail.co m> References: <79990c6b0801140324i1ff79105pfe572e4d1f21d9fa@mail.gmail.com> <20080114144230.91E803A4077@sparrow.telecommunity.com> <79990c6b0801140732i4547c120u9c791c7043b04514@mail.gmail.com> Message-ID: <20080114154727.BBCC63A4077@sparrow.telecommunity.com> At 03:32 PM 1/14/2008 +0000, Paul Moore wrote: >On 14/01/2008, Phillip J. Eby wrote: > > At 11:24 AM 1/14/2008 +0000, Paul Moore wrote: > > >If I have a module which installs some files using the > > >data_files=[...] argument to setup.py, how do I then (reliably) find > > >those files from my code? > > > > You can't. Use package_data instead. It's available in the > > distutils as of Python 2.4. > >I thought that might be the answer. Unfortunately, MoinMoin still >preserves compatibility with 2.3 :-( So does setuptools. :) From pje at telecommunity.com Mon Jan 14 16:48:26 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 14 Jan 2008 10:48:26 -0500 Subject: [Distutils] setuptools, bdist_wininst and dependecies In-Reply-To: References: <20080114151242.BCB313A4077@sparrow.telecommunity.com> Message-ID: <20080114154825.612C23A4077@sparrow.telecommunity.com> At 05:16 PM 1/14/2008 +0200, Ionel Maries Cristian wrote: >That is like scratching your right ear with your left hand. >Regardless, is this a bug or a missing feature? I don't really see it as either. setuptools provides compatibility with bdist_wininst (both by installing them and being able to make them), but it doesn't magically turn them into something other than what they are. From ionel.mc at gmail.com Mon Jan 14 18:16:39 2008 From: ionel.mc at gmail.com (Ionel Maries Cristian) Date: Mon, 14 Jan 2008 19:16:39 +0200 Subject: [Distutils] setuptools, bdist_wininst and dependecies In-Reply-To: <20080114154825.612C23A4077@sparrow.telecommunity.com> References: <20080114151242.BCB313A4077@sparrow.telecommunity.com> <20080114154825.612C23A4077@sparrow.telecommunity.com> Message-ID: however, i have another problem: for example if i install a egg of a given package and then I install a newer version using a .exe then i will have in site-packages: example-0.1-py2.5.egg\ example-0.2-py2.5.egg-info\ and somehow example-0.1-py2.5.egg will take precedence and import example will give me the old package On Jan 14, 2008 5:48 PM, Phillip J. Eby wrote: > At 05:16 PM 1/14/2008 +0200, Ionel Maries Cristian wrote: > >That is like scratching your right ear with your left hand. > >Regardless, is this a bug or a missing feature? > > I don't really see it as either. setuptools provides compatibility > with bdist_wininst (both by installing them and being able to make > them), but it doesn't magically turn them into something other than > what they are. > > -- http://code.google.com/p/cogen/ ionel. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20080114/63d3f4e1/attachment-0001.htm From ionel.mc at gmail.com Mon Jan 14 18:19:31 2008 From: ionel.mc at gmail.com (Ionel Maries Cristian) Date: Mon, 14 Jan 2008 19:19:31 +0200 Subject: [Distutils] setuptools, bdist_wininst and dependecies In-Reply-To: References: <20080114151242.BCB313A4077@sparrow.telecommunity.com> <20080114154825.612C23A4077@sparrow.telecommunity.com> Message-ID: looks like my new version doesn't get in easy-install.pth any idea how to get my old version out of easy-install.pth ? On Jan 14, 2008 7:16 PM, Ionel Maries Cristian wrote: > however, i have another problem: > for example if i install a egg of a given package and then I install a > newer version using a .exe then i will have in site-packages: > example-0.1-py2.5.egg\ > example-0.2-py2.5.egg-info\ > and somehow example-0.1-py2.5.egg will take precedence and import example > will give me the old package > > On Jan 14, 2008 5:48 PM, Phillip J. Eby wrote: > > > At 05:16 PM 1/14/2008 +0200, Ionel Maries Cristian wrote: > > >That is like scratching your right ear with your left hand. > > >Regardless, is this a bug or a missing feature? > > > > I don't really see it as either. setuptools provides compatibility > > with bdist_wininst (both by installing them and being able to make > > them), but it doesn't magically turn them into something other than > > what they are. > > > > > > > -- > http://code.google.com/p/cogen/ > ionel. > -- http://code.google.com/p/cogen/ ionel. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20080114/879d1a46/attachment.htm From pje at telecommunity.com Mon Jan 14 18:32:42 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 14 Jan 2008 12:32:42 -0500 Subject: [Distutils] setuptools, bdist_wininst and dependecies In-Reply-To: References: <20080114151242.BCB313A4077@sparrow.telecommunity.com> <20080114154825.612C23A4077@sparrow.telecommunity.com> Message-ID: <20080114173338.9EC313A40A2@sparrow.telecommunity.com> At 07:19 PM 1/14/2008 +0200, Ionel Maries Cristian wrote: >looks like my new version doesn't get in easy-install.pth >any idea how to get my old version out of easy-install.pth ? http://peak.telecommunity.com/DevCenter/EasyInstall#uninstalling-packages From ionel.mc at gmail.com Mon Jan 14 18:37:57 2008 From: ionel.mc at gmail.com (Ionel Maries Cristian) Date: Mon, 14 Jan 2008 19:37:57 +0200 Subject: [Distutils] setuptools, bdist_wininst and dependecies In-Reply-To: <20080114173338.9EC313A40A2@sparrow.telecommunity.com> References: <20080114151242.BCB313A4077@sparrow.telecommunity.com> <20080114154825.612C23A4077@sparrow.telecommunity.com> <20080114173338.9EC313A40A2@sparrow.telecommunity.com> Message-ID: that doesn't really work for me i don't want to impose users to work around package manager quirks any solution doing it from a postinstall script ? On Jan 14, 2008 7:32 PM, Phillip J. Eby wrote: > At 07:19 PM 1/14/2008 +0200, Ionel Maries Cristian wrote: > >looks like my new version doesn't get in easy-install.pth > >any idea how to get my old version out of easy-install.pth ? > > http://peak.telecommunity.com/DevCenter/EasyInstall#uninstalling-packages > > -- http://code.google.com/p/cogen/ ionel. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20080114/b821bb8c/attachment.htm From pje at telecommunity.com Mon Jan 14 19:17:55 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 14 Jan 2008 13:17:55 -0500 Subject: [Distutils] setuptools, bdist_wininst and dependecies In-Reply-To: References: <20080114151242.BCB313A4077@sparrow.telecommunity.com> <20080114154825.612C23A4077@sparrow.telecommunity.com> <20080114173338.9EC313A40A2@sparrow.telecommunity.com> Message-ID: <20080114181757.C21DF3A4077@sparrow.telecommunity.com> At 07:37 PM 1/14/2008 +0200, Ionel Maries Cristian wrote: >i don't want to impose users to work around package manager quirks Then don't use bdist_wininst as your installation method. Or at the very least, don't mix two different installation methods for the platform. From sklein at cpcug.org Tue Jan 15 00:56:31 2008 From: sklein at cpcug.org (Stanley A. Klein) Date: Mon, 14 Jan 2008 18:56:31 -0500 (EST) Subject: [Distutils] Craziness in dependency checking of PyXML version Message-ID: <39373.207.188.248.157.1200354991.squirrel@www.cpcug.org> I'm trying to fix something running under ZSI that broke because of a needed change in the WS-Addressing schema version. I ran into a problem that I've had before, where Python can't recognize that I have a proper version of PyXML installed. I get the following traceback: [stan at localhost wsdl]$ wsdl2py -help Traceback (most recent call last): File "/usr/bin/wsdl2py", line 5, in from pkg_resources import load_entry_point File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 2561, in working_set.require(__requires__) File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 626, in require needed = self.resolve(parse_requirements(requirements)) File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 524, in resolve raise DistributionNotFound(req) # XXX put more info here pkg_resources.DistributionNotFound: PyXML>=0.8.3 What I have installed as an rpm is PyXML-0.8.4-6. I've had this problem before with the PyXML version in another context, probably trying to install ZSI as an egg in my user lib directory. Is there an easy fix for this? Thanks. Stan Klein From pje at telecommunity.com Tue Jan 15 01:50:32 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 14 Jan 2008 19:50:32 -0500 Subject: [Distutils] Craziness in dependency checking of PyXML version In-Reply-To: <39373.207.188.248.157.1200354991.squirrel@www.cpcug.org> References: <39373.207.188.248.157.1200354991.squirrel@www.cpcug.org> Message-ID: <20080115005033.679583A4077@sparrow.telecommunity.com> At 06:56 PM 1/14/2008 -0500, Stanley A. Klein wrote: >I'm trying to fix something running under ZSI that broke because of a >needed change in the WS-Addressing schema version. > >I ran into a problem that I've had before, where Python can't recognize >that I have a proper version of PyXML installed. > >I get the following traceback: > >[stan at localhost wsdl]$ wsdl2py -help >Traceback (most recent call last): > File "/usr/bin/wsdl2py", line 5, in > from pkg_resources import load_entry_point > File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 2561, in > > working_set.require(__requires__) > File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 626, in >require > needed = self.resolve(parse_requirements(requirements)) > File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 524, in >resolve > raise DistributionNotFound(req) # XXX put more info here >pkg_resources.DistributionNotFound: PyXML>=0.8.3 > >What I have installed as an rpm is PyXML-0.8.4-6. > >I've had this problem before with the PyXML version in another context, >probably trying to install ZSI as an egg in my user lib directory. > >Is there an easy fix for this? Yes: convince the organization that creates your RPMs not to remove the .egg-info files that the distutils generates, specifically so that system-installed packages can be detected programmatically, without needing to understand every single distribution format in existence. Or, failing that, create an empty "PyXML-0.8.4-py2.5.egg-info" file alongside the top-level modules or packages that were installed by the RPM. :) From sklein at cpcug.org Tue Jan 15 03:03:35 2008 From: sklein at cpcug.org (Stanley A. Klein) Date: Mon, 14 Jan 2008 21:03:35 -0500 (EST) Subject: [Distutils] Craziness in dependency checking of PyXML version In-Reply-To: <20080115005033.679583A4077@sparrow.telecommunity.com> References: <39373.207.188.248.157.1200354991.squirrel@www.cpcug.org> <20080115005033.679583A4077@sparrow.telecommunity.com> Message-ID: <38064.71.178.79.78.1200362615.squirrel@www.cpcug.org> I created the ZSI rpm myself using python setup.py bdist_rpm. The PyXML rpm on my Fedora system was created by Fedora. I will have to create one myself for Centos. BTW, as I said in an earlier email, I found an easier fix. I commented out the statement in the ZSI setup.py that caused Python to check for the dependency. Stan Klein On Mon, January 14, 2008 7:50 pm, Phillip J. Eby wrote: > At 06:56 PM 1/14/2008 -0500, Stanley A. Klein wrote: >>I'm trying to fix something running under ZSI that broke because of a >>needed change in the WS-Addressing schema version. >> >>I ran into a problem that I've had before, where Python can't recognize >>that I have a proper version of PyXML installed. >> >>I get the following traceback: >> >>[stan at localhost wsdl]$ wsdl2py -help >>Traceback (most recent call last): >> File "/usr/bin/wsdl2py", line 5, in >> from pkg_resources import load_entry_point >> File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 2561, >> in >> >> working_set.require(__requires__) >> File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 626, in >>require >> needed = self.resolve(parse_requirements(requirements)) >> File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 524, in >>resolve >> raise DistributionNotFound(req) # XXX put more info here >>pkg_resources.DistributionNotFound: PyXML>=0.8.3 >> >>What I have installed as an rpm is PyXML-0.8.4-6. >> >>I've had this problem before with the PyXML version in another context, >>probably trying to install ZSI as an egg in my user lib directory. >> >>Is there an easy fix for this? > > Yes: convince the organization that creates your RPMs not to remove > the .egg-info files that the distutils generates, specifically so > that system-installed packages can be detected programmatically, > without needing to understand every single distribution format in > existence. > > Or, failing that, create an empty "PyXML-0.8.4-py2.5.egg-info" file > alongside the top-level modules or packages that were installed by the > RPM. :) > -- From pje at telecommunity.com Tue Jan 15 03:51:26 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 14 Jan 2008 21:51:26 -0500 Subject: [Distutils] Craziness in dependency checking of PyXML version In-Reply-To: <38064.71.178.79.78.1200362615.squirrel@www.cpcug.org> References: <39373.207.188.248.157.1200354991.squirrel@www.cpcug.org> <20080115005033.679583A4077@sparrow.telecommunity.com> <38064.71.178.79.78.1200362615.squirrel@www.cpcug.org> Message-ID: <20080115025141.826D03A4077@sparrow.telecommunity.com> At 09:03 PM 1/14/2008 -0500, Stanley A. Klein wrote: >I created the ZSI rpm myself using python setup.py bdist_rpm. > >The PyXML rpm on my Fedora system was created by Fedora. FYI, Fedora version 9 apparently now includes .egg-info for distutils-based packages. From iwan at reahl.org Tue Jan 15 07:11:25 2008 From: iwan at reahl.org (Iwan Vosloo) Date: Tue, 15 Jan 2008 08:11:25 +0200 Subject: [Distutils] Egg depency order In-Reply-To: References: Message-ID: <1200377485.15636.21.camel@localhost.localdomain> On Mon, 14 Jan 2008 09:45:29 -0500 Phillip J. Eby wrote: > >[1] In fact, this is an intermediary result - I want to > compute all the > >entry points of a tree of dependencies in such a way that the > entry > >points of eggs deeper in the tree will always be earlier in > the list > >than entry points for those shallower in the tree. > > Why? > We are writing an app on top of postgresql, using sqlobject. The thing is broken into different re-usable modules - not all of tcanhese modules need to be installed together. The idea is to package a module in an egg, and have the egg advertise all the sqlobject classes (ie, those that are persisted) via entry points. We can then write a seperate generic tool which (given the resource spec of an arbitrary module) can, for example go and create all the database tables necessary. But this is something that needs to happen in a specific order because some tables depend on others. We hope to capture such dependencies using the eggs's dependency mechanism. The tool can do many other things besides creating tables. For example, we'd also like to develop a generic framework on top of sqlobject which will allow us to easily specify the necessary database schema changes, new tables, changed tables and so on (migrations) that are necessary when upgrading from one version of our eggs to another. The idea is that knowledge of classes, tables, and how to migrate old data would be contained in each module's egg. But the generic code to create /change databases would be in its own egg. - Iwan From mikko at redinnovation.com Tue Jan 15 12:00:31 2008 From: mikko at redinnovation.com (Mikko Ohtamaa) Date: Tue, 15 Jan 2008 13:00:31 +0200 Subject: [Distutils] BUG: easy_install.exe fails under C :\Program files In-Reply-To: <20080107194611.00B903A406C@sparrow.telecommunity.com> References: <20080107033634.004653A406C@sparrow.telecommunity.com> <7b5b293c0801062028h7e0e7b52g9194a57fd556e4e5@mail.gmail.com> <20080107194611.00B903A406C@sparrow.telecommunity.com> Message-ID: <7b5b293c0801150300k14fecdfehfc18230a10ebb42@mail.gmail.com> > Would you take a look at the "easy_install-2.4-script.py" file and > see if the path after the #! is enclosed in quotes? For that matter, > what Python executable is it referring to? Here you go: #!C:\Program Files\Plone 3\Python\python.exe # EASY-INSTALL-ENTRY-SCRIPT: 'setuptools==0.6c7 ','console_scripts','easy_install-2.4' Apparently interpreter path with spaces is not liked somewhere. -- Mikko Ohtamaa Managing director, Red Innovation Ltd. +358 40 743 9707 www.redinnovation.com Every problem is solvable if you can throw enough energy drinks at it -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20080115/1c2c6e4e/attachment.htm From pje at telecommunity.com Tue Jan 15 16:28:31 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 Jan 2008 10:28:31 -0500 Subject: [Distutils] Egg depency order In-Reply-To: <1200377485.15636.21.camel@localhost.localdomain> References: <1200377485.15636.21.camel@localhost.localdomain> Message-ID: <20080115152831.91D673A40AE@sparrow.telecommunity.com> At 08:11 AM 1/15/2008 +0200, Iwan Vosloo wrote: >We are writing an app on top of postgresql, using sqlobject. The thing >is broken into different re-usable modules - not all of tcanhese modules >need to be installed together. The idea is to package a module in an >egg, and have the egg advertise all the sqlobject classes (ie, those >that are persisted) via entry points. > >We can then write a seperate generic tool which (given the resource spec >of an arbitrary module) can, for example go and create all the database >tables necessary. But this is something that needs to happen in a >specific order because some tables depend on others. Ah. In that case, just use something roughly like this: def create_tables_for_requirement(req, memo=None): if memo is None: memo = {} for dist in pkg_resources.require(req): create_tables_for_dist(dist, memo) def create_tables_for_dist(dist, memo): if dist not in memo: memo[dist] = 1 for name, ep in dist.get_entry_map('foo.tables'): create_table_for_entry_point(ep, memo) def create_table_for_entry_point(ep, memo): create_tables_for_requirement(ep.dist.requires(ep.extras), memo) # actually create the table here This might be slightly more complex due to the fact that the tables in a given distribution might depend on each other, and there isn't any dependency data in eggs that can tell you about *that*. You might find it simpler to have an entry point for an object that lists the distribution's tables. In fact, the simplest approach of all would be for your table objects to simply import their dependencies and have attributes that you can follow for that. In which case, you don't need *any* of this code, because simply loading the entry point would then suffice for resolving the dependencies. >The tool can do many other things besides creating tables. For example, >we'd also like to develop a generic framework on top of sqlobject which >will allow us to easily specify the necessary database schema changes, >new tables, changed tables and so on (migrations) that are necessary >when upgrading from one version of our eggs to another. > >The idea is that knowledge of classes, tables, and how to migrate old >data would be contained in each module's egg. But the generic code to >create /change databases would be in its own egg. But also note that you don't need to encode everything as egg metadata -- the Python objects referred to by the entry points can have arbitrary complexity. From pje at telecommunity.com Tue Jan 15 16:31:53 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 Jan 2008 10:31:53 -0500 Subject: [Distutils] BUG: easy_install.exe fails under C :\Program files In-Reply-To: <7b5b293c0801150300k14fecdfehfc18230a10ebb42@mail.gmail.com > References: <20080107033634.004653A406C@sparrow.telecommunity.com> <7b5b293c0801062028h7e0e7b52g9194a57fd556e4e5@mail.gmail.com> <20080107194611.00B903A406C@sparrow.telecommunity.com> <7b5b293c0801150300k14fecdfehfc18230a10ebb42@mail.gmail.com> Message-ID: <20080115153156.488A63A40BF@sparrow.telecommunity.com> At 01:00 PM 1/15/2008 +0200, Mikko Ohtamaa wrote: >Would you take a look at the >"easy_install-2.4-script.py " file and >see if the path after the #! is enclosed in quotes? For that matter, >what Python executable is it referring to? > > > >Here you go: > >#!C:\Program Files\Plone 3\Python\python.exe ># EASY-INSTALL-ENTRY-SCRIPT: >'setuptools==0.6c7','console_scripts','easy_install-2.4' > >Apparently interpreter path with spaces is not liked somewhere. Change the first line to: #"!C:\Program Files\Plone 3\Python\python.exe" The path should have had quotes there in the first place; I'm surprised it doesn't. Exactly how did you install setuptools to this location? From pje at telecommunity.com Tue Jan 15 18:56:20 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 Jan 2008 12:56:20 -0500 Subject: [Distutils] That setuptools/Pyrex problem with swig_sources In-Reply-To: References: Message-ID: <20080115175623.3FB703A4077@sparrow.telecommunity.com> At 09:47 AM 1/12/2008 -0600, Peter Wang wrote: >Hi everyone, > >I ran into this problem (traceback below) the other day with >setuptools 0.6c7 and Pyrex 0.9.6.4 on python 2.5. I saw Phillip's >post from October regarding this: >http://lists.copyleft.no/pipermail/pyrex/2007-October/002867.html > >How do I get this setuptools fix? easy_install setuptools==dev06 From sklein at cpcug.org Tue Jan 15 01:11:09 2008 From: sklein at cpcug.org (Stanley A. Klein) Date: Mon, 14 Jan 2008 19:11:09 -0500 (EST) Subject: [Distutils] More re:Craziness in dependency checking of PyXML version Message-ID: <33961.207.188.248.157.1200355869.squirrel@www.cpcug.org> I found a simple fix. I commented out the following line in setup.py: additional_params['install_requires'] = [ "PyXML >= 0.8.3", ] That functionality is seriously broken when on an rpm-based system. Stan Klein From pje at telecommunity.com Tue Jan 15 20:59:42 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 Jan 2008 14:59:42 -0500 Subject: [Distutils] More re:Craziness in dependency checking of PyXML version In-Reply-To: <33961.207.188.248.157.1200355869.squirrel@www.cpcug.org> References: <33961.207.188.248.157.1200355869.squirrel@www.cpcug.org> Message-ID: <20080115195943.44D403A4077@sparrow.telecommunity.com> At 07:11 PM 1/14/2008 -0500, Stanley A. Klein wrote: >I found a simple fix. I commented out the following line in setup.py: > >additional_params['install_requires'] = [ "PyXML >= 0.8.3", ] > >That functionality is seriously broken when on an rpm-based system. How so? A correct installation of PyXML for Python 2.5 and up should include its .egg-info, assuming PyXML is built using the distutils. If it does not do so, the system Python installation is broken -- and that is the case for Fedora version 8. (Fedora 9 fixed the problem.) There isn't a lot that developers can do about distributors removing files without knowing what those files are for. (Note that Fedora 8 specifically *patched the distutils* to prevent .egg-info files from being created or installed, thereby creating this problem.) From billiejoex at gmail.com Wed Jan 16 02:00:46 2008 From: billiejoex at gmail.com (Giampaolo Rodola') Date: Wed, 16 Jan 2008 02:00:46 +0100 Subject: [Distutils] pypi and easy_install In-Reply-To: <20080114022910.6D0D33A4077@sparrow.telecommunity.com> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <20071220034213.315F63A40A4@sparrow.telecommunity.com> <729626cc0712191947s18683928s71ec609dfe562166@mail.gmail.com> <20071220045912.806693A40A4@sparrow.telecommunity.com> <729626cc0712200526q216165d4h5b2d55d59f3fb7db@mail.gmail.com> <20071220165342.3F72A3A410C@sparrow.telecommunity.com> <729626cc0801131714l39547a35ld75bdd8cde411d23@mail.gmail.com> <20080114022910.6D0D33A4077@sparrow.telecommunity.com> Message-ID: <729626cc0801151700q2866b5f7ia2c7d9abf5aeffd7@mail.gmail.com> 2008/1/14, Phillip J. Eby : > At 02:14 AM 1/14/2008 +0100, Giampaolo Rodola' wrote: > > > (This is a distutils feature and doesn't require setuptools, although > > > the distutils are a bit trickier to get the tarball to include > > > everything, as you have to explicitly list in MANIFEST.in anything > > > that distutils doesn't automatically pick up. > > > >Sorry for replying so late. > >Gently, I would be interested in knowing how can I have distutils to > >include everything. > >Currently I've tried to use setup.py sdist but CREDITS, HISTORY, > >INSTALL and LICENSE files aren't included in the tarball. > > http://python.org/doc/2.4.1/dist/manifest.html Is there a way to specify that in setup.py? From pje at telecommunity.com Wed Jan 16 02:48:58 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 Jan 2008 20:48:58 -0500 Subject: [Distutils] pypi and easy_install In-Reply-To: <729626cc0801151700q2866b5f7ia2c7d9abf5aeffd7@mail.gmail.co m> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <20071220034213.315F63A40A4@sparrow.telecommunity.com> <729626cc0712191947s18683928s71ec609dfe562166@mail.gmail.com> <20071220045912.806693A40A4@sparrow.telecommunity.com> <729626cc0712200526q216165d4h5b2d55d59f3fb7db@mail.gmail.com> <20071220165342.3F72A3A410C@sparrow.telecommunity.com> <729626cc0801131714l39547a35ld75bdd8cde411d23@mail.gmail.com> <20080114022910.6D0D33A4077@sparrow.telecommunity.com> <729626cc0801151700q2866b5f7ia2c7d9abf5aeffd7@mail.gmail.com> Message-ID: <20080116014858.D9F273A4077@sparrow.telecommunity.com> At 02:00 AM 1/16/2008 +0100, Giampaolo Rodola' wrote: >2008/1/14, Phillip J. Eby : > > At 02:14 AM 1/14/2008 +0100, Giampaolo Rodola' wrote: > > > > (This is a distutils feature and doesn't require setuptools, although > > > > the distutils are a bit trickier to get the tarball to include > > > > everything, as you have to explicitly list in MANIFEST.in anything > > > > that distutils doesn't automatically pick up. > > > > > >Sorry for replying so late. > > >Gently, I would be interested in knowing how can I have distutils to > > >include everything. > > >Currently I've tried to use setup.py sdist but CREDITS, HISTORY, > > >INSTALL and LICENSE files aren't included in the tarball. > > > > http://python.org/doc/2.4.1/dist/manifest.html > >Is there a way to specify that in setup.py? Not that I'm aware of, no. You just put the MANIFEST.in next to setup.py. The alternative is to use setuptools + source control: setuptools natively picks the files up from SVN or CVS, and there are plugins for several other source control systems including Bazaar, Git, Monotone, and Mercurial, if I recall correctly. (Search PyPI for "setuptools" and the appropriate tool to find the plugin.) (Note that these revision control plugins do NOT require users of your package to have the revision control tool OR the plugin -- it only affects *building* the source distribution, not using it. In fact, it's even possible to make it so your users don't have to have setuptools, although that's not the default setup.) From billiejoex at gmail.com Wed Jan 16 03:05:26 2008 From: billiejoex at gmail.com (Giampaolo Rodola') Date: Wed, 16 Jan 2008 03:05:26 +0100 Subject: [Distutils] pypi and easy_install In-Reply-To: <20080116014858.D9F273A4077@sparrow.telecommunity.com> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <20071220034213.315F63A40A4@sparrow.telecommunity.com> <729626cc0712191947s18683928s71ec609dfe562166@mail.gmail.com> <20071220045912.806693A40A4@sparrow.telecommunity.com> <729626cc0712200526q216165d4h5b2d55d59f3fb7db@mail.gmail.com> <20071220165342.3F72A3A410C@sparrow.telecommunity.com> <729626cc0801131714l39547a35ld75bdd8cde411d23@mail.gmail.com> <20080114022910.6D0D33A4077@sparrow.telecommunity.com> <729626cc0801151700q2866b5f7ia2c7d9abf5aeffd7@mail.gmail.com> <20080116014858.D9F273A4077@sparrow.telecommunity.com> Message-ID: <729626cc0801151805w7f55649crfbb79901916405c6@mail.gmail.com> Thanks for the useful information I wasn't aware of. I'll use MANIFEST file. 2008/1/16, Phillip J. Eby : > At 02:00 AM 1/16/2008 +0100, Giampaolo Rodola' wrote: > >2008/1/14, Phillip J. Eby : > > > At 02:14 AM 1/14/2008 +0100, Giampaolo Rodola' wrote: > > > > > (This is a distutils feature and doesn't require setuptools, although > > > > > the distutils are a bit trickier to get the tarball to include > > > > > everything, as you have to explicitly list in MANIFEST.in anything > > > > > that distutils doesn't automatically pick up. > > > > > > > >Sorry for replying so late. > > > >Gently, I would be interested in knowing how can I have distutils to > > > >include everything. > > > >Currently I've tried to use setup.py sdist but CREDITS, HISTORY, > > > >INSTALL and LICENSE files aren't included in the tarball. > > > > > > http://python.org/doc/2.4.1/dist/manifest.html > > > >Is there a way to specify that in setup.py? > > Not that I'm aware of, no. You just put the MANIFEST.in next to setup.py. > > The alternative is to use setuptools + source control: setuptools > natively picks the files up from SVN or CVS, and there are plugins > for several other source control systems including Bazaar, Git, > Monotone, and Mercurial, if I recall correctly. (Search PyPI for > "setuptools" and the appropriate tool to find the plugin.) > > (Note that these revision control plugins do NOT require users of > your package to have the revision control tool OR the plugin -- it > only affects *building* the source distribution, not using it. In > fact, it's even possible to make it so your users don't have to have > setuptools, although that's not the default setup.) > > From billiejoex at gmail.com Wed Jan 16 04:56:54 2008 From: billiejoex at gmail.com (Giampaolo Rodola') Date: Wed, 16 Jan 2008 04:56:54 +0100 Subject: [Distutils] pypi and easy_install In-Reply-To: <729626cc0801151805w7f55649crfbb79901916405c6@mail.gmail.com> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <729626cc0712191947s18683928s71ec609dfe562166@mail.gmail.com> <20071220045912.806693A40A4@sparrow.telecommunity.com> <729626cc0712200526q216165d4h5b2d55d59f3fb7db@mail.gmail.com> <20071220165342.3F72A3A410C@sparrow.telecommunity.com> <729626cc0801131714l39547a35ld75bdd8cde411d23@mail.gmail.com> <20080114022910.6D0D33A4077@sparrow.telecommunity.com> <729626cc0801151700q2866b5f7ia2c7d9abf5aeffd7@mail.gmail.com> <20080116014858.D9F273A4077@sparrow.telecommunity.com> <729626cc0801151805w7f55649crfbb79901916405c6@mail.gmail.com> Message-ID: <729626cc0801151956j28c577d9g68e7586a3d124b72@mail.gmail.com> I'm sorry, just one more question (hopefully). I work from Windows and I'd like to generate a gzipped tar file (.tar.gz). On documentation I see that: "It requires external utilities: tar and possibly one of gzip, bzip2, or compress" I installed: http://gnuwin32.sourceforge.net/packages/gtar.htm ...and: http://gnuwin32.sourceforge.net/packages/gzip.htm Since apparently there's no way to provide cmd line arguments to tell distutils where to find gtar and gzip executables I created two .bat files in the setup.py directory pointing to original utility executables but after having run setup.py sdist the installation hangs on when arrives to run the tar.exe utility. Is there an easier way for doing what I want? I know that maybe this is more a Windows issue than a distutils issue but I find very uncomfortable the way distutils manage such things. PS - I've never used it but by reading the documentation it seems that the tarfile module would be able to read and write those kind of archives. If that is true it would be much easier using that instead of relying on third party packages like those ones I've tried. 2008/1/16, Giampaolo Rodola' : > Thanks for the useful information I wasn't aware of. > I'll use MANIFEST file. > > 2008/1/16, Phillip J. Eby : > > At 02:00 AM 1/16/2008 +0100, Giampaolo Rodola' wrote: > > >2008/1/14, Phillip J. Eby : > > > > At 02:14 AM 1/14/2008 +0100, Giampaolo Rodola' wrote: > > > > > > (This is a distutils feature and doesn't require setuptools, although > > > > > > the distutils are a bit trickier to get the tarball to include > > > > > > everything, as you have to explicitly list in MANIFEST.in anything > > > > > > that distutils doesn't automatically pick up. > > > > > > > > > >Sorry for replying so late. > > > > >Gently, I would be interested in knowing how can I have distutils to > > > > >include everything. > > > > >Currently I've tried to use setup.py sdist but CREDITS, HISTORY, > > > > >INSTALL and LICENSE files aren't included in the tarball. > > > > > > > > http://python.org/doc/2.4.1/dist/manifest.html > > > > > >Is there a way to specify that in setup.py? > > > > Not that I'm aware of, no. You just put the MANIFEST.in next to setup.py. > > > > The alternative is to use setuptools + source control: setuptools > > natively picks the files up from SVN or CVS, and there are plugins > > for several other source control systems including Bazaar, Git, > > Monotone, and Mercurial, if I recall correctly. (Search PyPI for > > "setuptools" and the appropriate tool to find the plugin.) > > > > (Note that these revision control plugins do NOT require users of > > your package to have the revision control tool OR the plugin -- it > > only affects *building* the source distribution, not using it. In > > fact, it's even possible to make it so your users don't have to have > > setuptools, although that's not the default setup.) > > > > > From billiejoex at gmail.com Wed Jan 16 05:04:49 2008 From: billiejoex at gmail.com (Giampaolo Rodola') Date: Wed, 16 Jan 2008 05:04:49 +0100 Subject: [Distutils] pypi and easy_install In-Reply-To: <729626cc0801151956j28c577d9g68e7586a3d124b72@mail.gmail.com> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <20071220045912.806693A40A4@sparrow.telecommunity.com> <729626cc0712200526q216165d4h5b2d55d59f3fb7db@mail.gmail.com> <20071220165342.3F72A3A410C@sparrow.telecommunity.com> <729626cc0801131714l39547a35ld75bdd8cde411d23@mail.gmail.com> <20080114022910.6D0D33A4077@sparrow.telecommunity.com> <729626cc0801151700q2866b5f7ia2c7d9abf5aeffd7@mail.gmail.com> <20080116014858.D9F273A4077@sparrow.telecommunity.com> <729626cc0801151805w7f55649crfbb79901916405c6@mail.gmail.com> <729626cc0801151956j28c577d9g68e7586a3d124b72@mail.gmail.com> Message-ID: <729626cc0801152004g5c76accfqba28e7e8dc9fa5db@mail.gmail.com> > after having run setup.py sdist the installation hangs > on when arrives to run the tar.exe utility. Sorry, obviously I meant "setup.py sdist --formats=gztar" (instead of "setup.py sdist") that's what I did actually . 2008/1/16, Giampaolo Rodola' : > I'm sorry, just one more question (hopefully). > I work from Windows and I'd like to generate a gzipped tar file (.tar.gz). > On documentation I see that: > > "It requires external utilities: tar and possibly one of gzip, bzip2, > or compress" > > I installed: > http://gnuwin32.sourceforge.net/packages/gtar.htm > ...and: > http://gnuwin32.sourceforge.net/packages/gzip.htm > > Since apparently there's no way to provide cmd line arguments to tell > distutils where to find gtar and gzip executables I created two .bat > files in the setup.py directory pointing to original utility > executables but after having run setup.py sdist the installation hangs > on when arrives to run the tar.exe utility. > Is there an easier way for doing what I want? > I know that maybe this is more a Windows issue than a distutils issue > but I find very uncomfortable the way distutils manage such things. > > PS - I've never used it but by reading the documentation it seems that > the tarfile module would be able to read and write those kind of > archives. If that is true it would be much easier using that instead > of relying on third party packages like those ones I've tried. > > 2008/1/16, Giampaolo Rodola' : > > Thanks for the useful information I wasn't aware of. > > I'll use MANIFEST file. > > > > 2008/1/16, Phillip J. Eby : > > > At 02:00 AM 1/16/2008 +0100, Giampaolo Rodola' wrote: > > > >2008/1/14, Phillip J. Eby : > > > > > At 02:14 AM 1/14/2008 +0100, Giampaolo Rodola' wrote: > > > > > > > (This is a distutils feature and doesn't require setuptools, although > > > > > > > the distutils are a bit trickier to get the tarball to include > > > > > > > everything, as you have to explicitly list in MANIFEST.in anything > > > > > > > that distutils doesn't automatically pick up. > > > > > > > > > > > >Sorry for replying so late. > > > > > >Gently, I would be interested in knowing how can I have distutils to > > > > > >include everything. > > > > > >Currently I've tried to use setup.py sdist but CREDITS, HISTORY, > > > > > >INSTALL and LICENSE files aren't included in the tarball. > > > > > > > > > > http://python.org/doc/2.4.1/dist/manifest.html > > > > > > > >Is there a way to specify that in setup.py? > > > > > > Not that I'm aware of, no. You just put the MANIFEST.in next to setup.py. > > > > > > The alternative is to use setuptools + source control: setuptools > > > natively picks the files up from SVN or CVS, and there are plugins > > > for several other source control systems including Bazaar, Git, > > > Monotone, and Mercurial, if I recall correctly. (Search PyPI for > > > "setuptools" and the appropriate tool to find the plugin.) > > > > > > (Note that these revision control plugins do NOT require users of > > > your package to have the revision control tool OR the plugin -- it > > > only affects *building* the source distribution, not using it. In > > > fact, it's even possible to make it so your users don't have to have > > > setuptools, although that's not the default setup.) > > > > > > > > > From pje at telecommunity.com Wed Jan 16 05:17:09 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 Jan 2008 23:17:09 -0500 Subject: [Distutils] pypi and easy_install In-Reply-To: <729626cc0801151956j28c577d9g68e7586a3d124b72@mail.gmail.co m> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <729626cc0712191947s18683928s71ec609dfe562166@mail.gmail.com> <20071220045912.806693A40A4@sparrow.telecommunity.com> <729626cc0712200526q216165d4h5b2d55d59f3fb7db@mail.gmail.com> <20071220165342.3F72A3A410C@sparrow.telecommunity.com> <729626cc0801131714l39547a35ld75bdd8cde411d23@mail.gmail.com> <20080114022910.6D0D33A4077@sparrow.telecommunity.com> <729626cc0801151700q2866b5f7ia2c7d9abf5aeffd7@mail.gmail.com> <20080116014858.D9F273A4077@sparrow.telecommunity.com> <729626cc0801151805w7f55649crfbb79901916405c6@mail.gmail.com> <729626cc0801151956j28c577d9g68e7586a3d124b72@mail.gmail.com> Message-ID: <20080116041709.26A4E3A4077@sparrow.telecommunity.com> At 04:56 AM 1/16/2008 +0100, Giampaolo Rodola' wrote: >I'm sorry, just one more question (hopefully). >I work from Windows and I'd like to generate a gzipped tar file (.tar.gz). >On documentation I see that: > >"It requires external utilities: tar and possibly one of gzip, bzip2, >or compress" > >I installed: >http://gnuwin32.sourceforge.net/packages/gtar.htm >...and: >http://gnuwin32.sourceforge.net/packages/gzip.htm > >Since apparently there's no way to provide cmd line arguments to tell >distutils where to find gtar and gzip executables I created two .bat >files in the setup.py directory pointing to original utility >executables but after having run setup.py sdist the installation hangs >on when arrives to run the tar.exe utility. >Is there an easier way for doing what I want? >I know that maybe this is more a Windows issue than a distutils issue >but I find very uncomfortable the way distutils manage such things. > >PS - I've never used it but by reading the documentation it seems that >the tarfile module would be able to read and write those kind of >archives. If that is true it would be much easier using that instead >of relying on third party packages like those ones I've tried. Unfortunately the distutils.archive_util module was written before tarfile was available. You should only need to have the "tar" and "gzip" executables in a directory on your PATH in order to have them work, though. From pje at telecommunity.com Wed Jan 16 05:17:39 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 Jan 2008 23:17:39 -0500 Subject: [Distutils] pypi and easy_install In-Reply-To: <729626cc0801151805w7f55649crfbb79901916405c6@mail.gmail.co m> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <20071220034213.315F63A40A4@sparrow.telecommunity.com> <729626cc0712191947s18683928s71ec609dfe562166@mail.gmail.com> <20071220045912.806693A40A4@sparrow.telecommunity.com> <729626cc0712200526q216165d4h5b2d55d59f3fb7db@mail.gmail.com> <20071220165342.3F72A3A410C@sparrow.telecommunity.com> <729626cc0801131714l39547a35ld75bdd8cde411d23@mail.gmail.com> <20080114022910.6D0D33A4077@sparrow.telecommunity.com> <729626cc0801151700q2866b5f7ia2c7d9abf5aeffd7@mail.gmail.com> <20080116014858.D9F273A4077@sparrow.telecommunity.com> <729626cc0801151805w7f55649crfbb79901916405c6@mail.gmail.com> Message-ID: <20080116041738.175B33A4077@sparrow.telecommunity.com> At 03:05 AM 1/16/2008 +0100, Giampaolo Rodola' wrote: >Thanks for the useful information I wasn't aware of. >I'll use MANIFEST file. You probably mean MANIFEST.in -- the MANIFEST file is generated automatically by the distutils, and is even more of a pain to work with. From billiejoex at gmail.com Wed Jan 16 15:48:08 2008 From: billiejoex at gmail.com (Giampaolo Rodola') Date: Wed, 16 Jan 2008 15:48:08 +0100 Subject: [Distutils] pypi and easy_install In-Reply-To: <729626cc0801160646v4f910274t24ec15ce403afc5d@mail.gmail.com> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <20071220165342.3F72A3A410C@sparrow.telecommunity.com> <729626cc0801131714l39547a35ld75bdd8cde411d23@mail.gmail.com> <20080114022910.6D0D33A4077@sparrow.telecommunity.com> <729626cc0801151700q2866b5f7ia2c7d9abf5aeffd7@mail.gmail.com> <20080116014858.D9F273A4077@sparrow.telecommunity.com> <729626cc0801151805w7f55649crfbb79901916405c6@mail.gmail.com> <729626cc0801151956j28c577d9g68e7586a3d124b72@mail.gmail.com> <20080116041709.26A4E3A4077@sparrow.telecommunity.com> <729626cc0801160646v4f910274t24ec15ce403afc5d@mail.gmail.com> Message-ID: <729626cc0801160648h4d764612qeb40692b260d7e5f@mail.gmail.com> > > Unfortunately the distutils.archive_util module was written before > > tarfile was available. Are there chances to see this implemented? 2008/1/16, Giampaolo Rodola' : > > Unfortunately the distutils.archive_util module was written before > > tarfile was available. > > Are there chances to see this implemented? > > 2008/1/16, Phillip J. Eby : > > At 04:56 AM 1/16/2008 +0100, Giampaolo Rodola' wrote: > > >I'm sorry, just one more question (hopefully). > > >I work from Windows and I'd like to generate a gzipped tar file (.tar.gz). > > >On documentation I see that: > > > > > >"It requires external utilities: tar and possibly one of gzip, bzip2, > > >or compress" > > > > > >I installed: > > >http://gnuwin32.sourceforge.net/packages/gtar.htm > > >...and: > > >http://gnuwin32.sourceforge.net/packages/gzip.htm > > > > > >Since apparently there's no way to provide cmd line arguments to tell > > >distutils where to find gtar and gzip executables I created two .bat > > >files in the setup.py directory pointing to original utility > > >executables but after having run setup.py sdist the installation hangs > > >on when arrives to run the tar.exe utility. > > >Is there an easier way for doing what I want? > > >I know that maybe this is more a Windows issue than a distutils issue > > >but I find very uncomfortable the way distutils manage such things. > > > > > >PS - I've never used it but by reading the documentation it seems that > > >the tarfile module would be able to read and write those kind of > > >archives. If that is true it would be much easier using that instead > > >of relying on third party packages like those ones I've tried. > > > Unfortunately the distutils.archive_util module was written before > tarfile was available. > > Could we hope > > > You should only need to have the "tar" and "gzip" executables in a > > directory on your PATH in order to have them work, though. > > > > > From billiejoex at gmail.com Wed Jan 16 15:49:36 2008 From: billiejoex at gmail.com (Giampaolo Rodola') Date: Wed, 16 Jan 2008 15:49:36 +0100 Subject: [Distutils] pypi and easy_install In-Reply-To: <729626cc0801160648ib42c98dj9c63b63d411fb5@mail.gmail.com> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <729626cc0712200526q216165d4h5b2d55d59f3fb7db@mail.gmail.com> <20071220165342.3F72A3A410C@sparrow.telecommunity.com> <729626cc0801131714l39547a35ld75bdd8cde411d23@mail.gmail.com> <20080114022910.6D0D33A4077@sparrow.telecommunity.com> <729626cc0801151700q2866b5f7ia2c7d9abf5aeffd7@mail.gmail.com> <20080116014858.D9F273A4077@sparrow.telecommunity.com> <729626cc0801151805w7f55649crfbb79901916405c6@mail.gmail.com> <20080116041738.175B33A4077@sparrow.telecommunity.com> <729626cc0801160648ib42c98dj9c63b63d411fb5@mail.gmail.com> Message-ID: <729626cc0801160649v7da3edc4p94060f9ab1a8b69a@mail.gmail.com> 2008/1/16, Phillip J. Eby : > > At 03:05 AM 1/16/2008 +0100, Giampaolo Rodola' wrote: > > >Thanks for the useful information I wasn't aware of. > > >I'll use MANIFEST file. > > > > You probably mean MANIFEST.in -- the MANIFEST file is generated > > automatically by the distutils, and is even more of a pain to work with. Yes, sorry: I meant MANIFEST.in. From pje at telecommunity.com Wed Jan 16 18:27:01 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 16 Jan 2008 12:27:01 -0500 Subject: [Distutils] pypi and easy_install In-Reply-To: <729626cc0801160648h4d764612qeb40692b260d7e5f@mail.gmail.co m> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <20071220165342.3F72A3A410C@sparrow.telecommunity.com> <729626cc0801131714l39547a35ld75bdd8cde411d23@mail.gmail.com> <20080114022910.6D0D33A4077@sparrow.telecommunity.com> <729626cc0801151700q2866b5f7ia2c7d9abf5aeffd7@mail.gmail.com> <20080116014858.D9F273A4077@sparrow.telecommunity.com> <729626cc0801151805w7f55649crfbb79901916405c6@mail.gmail.com> <729626cc0801151956j28c577d9g68e7586a3d124b72@mail.gmail.com> <20080116041709.26A4E3A4077@sparrow.telecommunity.com> <729626cc0801160646v4f910274t24ec15ce403afc5d@mail.gmail.com> <729626cc0801160648h4d764612qeb40692b260d7e5f@mail.gmail.com> Message-ID: <20080116172701.5141E3A40A2@sparrow.telecommunity.com> At 03:48 PM 1/16/2008 +0100, Giampaolo Rodola' wrote: > > > Unfortunately the distutils.archive_util module was written before > > > tarfile was available. > >Are there chances to see this implemented? Are you volunteering? :) If so, then perhaps you can get a patch in for Python 2.6. From billiejoex at gmail.com Wed Jan 16 18:39:21 2008 From: billiejoex at gmail.com (Giampaolo Rodola') Date: Wed, 16 Jan 2008 18:39:21 +0100 Subject: [Distutils] pypi and easy_install In-Reply-To: <20080116172701.5141E3A40A2@sparrow.telecommunity.com> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <20080114022910.6D0D33A4077@sparrow.telecommunity.com> <729626cc0801151700q2866b5f7ia2c7d9abf5aeffd7@mail.gmail.com> <20080116014858.D9F273A4077@sparrow.telecommunity.com> <729626cc0801151805w7f55649crfbb79901916405c6@mail.gmail.com> <729626cc0801151956j28c577d9g68e7586a3d124b72@mail.gmail.com> <20080116041709.26A4E3A4077@sparrow.telecommunity.com> <729626cc0801160646v4f910274t24ec15ce403afc5d@mail.gmail.com> <729626cc0801160648h4d764612qeb40692b260d7e5f@mail.gmail.com> <20080116172701.5141E3A40A2@sparrow.telecommunity.com> Message-ID: <729626cc0801160939s1c9563f2i58313d8c467f4bc8@mail.gmail.com> 2008/1/16, Phillip J. Eby : > At 03:48 PM 1/16/2008 +0100, Giampaolo Rodola' wrote: > > > > Unfortunately the distutils.archive_util module was written before > > > > tarfile was available. > > > >Are there chances to see this implemented? > > Are you volunteering? :) If so, then perhaps you can get a patch in > for Python 2.6. Uhm... why not? :-) If someone could give me some pointers on nailing down the issue I could give it a try (note: never seen distutils internals before). My only concern is that it's not rare that patches provided by users are often ignored and they remains stagnant on the Python bug tracker. I just wouldn't spend time on that for nothing. From pje at telecommunity.com Wed Jan 16 20:02:09 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 16 Jan 2008 14:02:09 -0500 Subject: [Distutils] pypi and easy_install In-Reply-To: <729626cc0801160939s1c9563f2i58313d8c467f4bc8@mail.gmail.co m> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <20080114022910.6D0D33A4077@sparrow.telecommunity.com> <729626cc0801151700q2866b5f7ia2c7d9abf5aeffd7@mail.gmail.com> <20080116014858.D9F273A4077@sparrow.telecommunity.com> <729626cc0801151805w7f55649crfbb79901916405c6@mail.gmail.com> <729626cc0801151956j28c577d9g68e7586a3d124b72@mail.gmail.com> <20080116041709.26A4E3A4077@sparrow.telecommunity.com> <729626cc0801160646v4f910274t24ec15ce403afc5d@mail.gmail.com> <729626cc0801160648h4d764612qeb40692b260d7e5f@mail.gmail.com> <20080116172701.5141E3A40A2@sparrow.telecommunity.com> <729626cc0801160939s1c9563f2i58313d8c467f4bc8@mail.gmail.com> Message-ID: <20080116190209.CE3703A40A2@sparrow.telecommunity.com> At 06:39 PM 1/16/2008 +0100, Giampaolo Rodola' wrote: >2008/1/16, Phillip J. Eby : > > At 03:48 PM 1/16/2008 +0100, Giampaolo Rodola' wrote: > > > > > Unfortunately the distutils.archive_util module was written before > > > > > tarfile was available. > > > > > >Are there chances to see this implemented? > > > > Are you volunteering? :) If so, then perhaps you can get a patch in > > for Python 2.6. > >Uhm... why not? :-) >If someone could give me some pointers on nailing down the issue I >could give it a try (note: never seen distutils internals before). See the distutils.archive_util module - the disutils uses that for all its archive creation. >My only concern is that it's not rare that patches provided by users >are often ignored and they remains stagnant on the Python bug tracker. >I just wouldn't spend time on that for nothing. Well, I can review the patch and commit it, if I like it. :) From billiejoex at gmail.com Thu Jan 17 00:56:00 2008 From: billiejoex at gmail.com (Giampaolo Rodola') Date: Thu, 17 Jan 2008 00:56:00 +0100 Subject: [Distutils] pypi and easy_install In-Reply-To: <20080116190209.CE3703A40A2@sparrow.telecommunity.com> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <20080116014858.D9F273A4077@sparrow.telecommunity.com> <729626cc0801151805w7f55649crfbb79901916405c6@mail.gmail.com> <729626cc0801151956j28c577d9g68e7586a3d124b72@mail.gmail.com> <20080116041709.26A4E3A4077@sparrow.telecommunity.com> <729626cc0801160646v4f910274t24ec15ce403afc5d@mail.gmail.com> <729626cc0801160648h4d764612qeb40692b260d7e5f@mail.gmail.com> <20080116172701.5141E3A40A2@sparrow.telecommunity.com> <729626cc0801160939s1c9563f2i58313d8c467f4bc8@mail.gmail.com> <20080116190209.CE3703A40A2@sparrow.telecommunity.com> Message-ID: <729626cc0801161556o26f06a18t44e283b9fc4bdf77@mail.gmail.com> 2008/1/16, Phillip J. Eby : > At 06:39 PM 1/16/2008 +0100, Giampaolo Rodola' wrote: > >2008/1/16, Phillip J. Eby : > > > At 03:48 PM 1/16/2008 +0100, Giampaolo Rodola' wrote: > > > > > > Unfortunately the distutils.archive_util module was written before > > > > > > tarfile was available. > > > > > > > >Are there chances to see this implemented? > > > > > > Are you volunteering? :) If so, then perhaps you can get a patch in > > > for Python 2.6. > > > >Uhm... why not? :-) > >If someone could give me some pointers on nailing down the issue I > >could give it a try (note: never seen distutils internals before). > > See the distutils.archive_util module - the disutils uses that for > all its archive creation. > > > >My only concern is that it's not rare that patches provided by users > >are often ignored and they remains stagnant on the Python bug tracker. > >I just wouldn't spend time on that for nothing. > > Well, I can review the patch and commit it, if I like it. :) Ok. I should find some time for trying to work on this this week. I'll keep you updated. Thanks. From noah.gift at gmail.com Thu Jan 17 10:16:48 2008 From: noah.gift at gmail.com (Noah Gift) Date: Thu, 17 Jan 2008 04:16:48 -0500 Subject: [Distutils] docs on using setuptools in a release cycle with RPMS Message-ID: <2ED54CE8-3A03-4B9B-A296-2CC5CB44F168@gmail.com> I am trying to get egg generation entry points integrated with building RPM's, and hopefully debian packages too. I haven't had much luck finding any documentation on this though. My specific question is: Can I use: python2.5 ./setup.py bdist_rpm --binary-only --release=py25 --python=/ usr/local/bin/python2.5 and expect that an console script entry point I define in setup.py will install to the bin directory for an rpm as well? If someone has some documents or a build script I could look at, I would appreciate it. Noah From ziade.tarek at gmail.com Thu Jan 17 13:40:22 2008 From: ziade.tarek at gmail.com (=?UTF-8?Q?Tarek_Ziad=C3=A9?=) Date: Thu, 17 Jan 2008 04:40:22 -0800 (PST) Subject: [Distutils] multiple servers handling in .pypirc Message-ID: <14917917.post@talk.nabble.com> Hello, like discussed in catalog-sig, and explained here: http://wiki.python.org/moin/EnhancedPyPI I am submitting a patch for distutils so .pypirc can handle several servers: http://bugs.python.org/issue1858 It allows having a .pypirc that handles several servers, using the -r option together with several couples of username/password. I am not quite sure about the acceptation process for such a patch, since it was built through a discussion in catalog-sig, so I am presenting it here for review, feedback, etc. Regards Tarek -- View this message in context: http://www.nabble.com/multiple-servers-handling-in-.pypirc-tp14917917p14917917.html Sent from the Python - distutils-sig mailing list archive at Nabble.com. From pje at telecommunity.com Thu Jan 17 15:32:48 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 17 Jan 2008 09:32:48 -0500 Subject: [Distutils] docs on using setuptools in a release cycle with RPMS In-Reply-To: <2ED54CE8-3A03-4B9B-A296-2CC5CB44F168@gmail.com> References: <2ED54CE8-3A03-4B9B-A296-2CC5CB44F168@gmail.com> Message-ID: <20080117143247.191983A4077@sparrow.telecommunity.com> At 04:16 AM 1/17/2008 -0500, Noah Gift wrote: >I am trying to get egg generation entry points integrated with >building RPM's, and hopefully debian packages too. I haven't had much >luck finding any documentation on this though. My specific question is: > >Can I use: > >python2.5 ./setup.py bdist_rpm --binary-only --release=py25 --python=/ >usr/local/bin/python2.5 > > >and expect that an console script entry point I define in setup.py >will install to the bin directory for an rpm as well? Give it a try and see. :) (Yes, setuptools does support bdist_rpm.) From noah.gift at gmail.com Thu Jan 17 15:54:06 2008 From: noah.gift at gmail.com (Noah Gift) Date: Thu, 17 Jan 2008 09:54:06 -0500 Subject: [Distutils] docs on using setuptools in a release cycle with RPMS In-Reply-To: <20080117143247.191983A4077@sparrow.telecommunity.com> References: <2ED54CE8-3A03-4B9B-A296-2CC5CB44F168@gmail.com> <20080117143247.191983A4077@sparrow.telecommunity.com> Message-ID: <150EB5B7-20E2-4109-BC5D-13EA0A2FDDCD@gmail.com> On Jan 17, 2008, at 9:32 AM, Phillip J. Eby wrote: > At 04:16 AM 1/17/2008 -0500, Noah Gift wrote: >> I am trying to get egg generation entry points integrated with >> building RPM's, and hopefully debian packages too. I haven't had >> much >> luck finding any documentation on this though. My specific >> question is: >> >> Can I use: >> >> python2.5 ./setup.py bdist_rpm --binary-only --release=py25 -- >> python=/ >> usr/local/bin/python2.5 >> >> >> and expect that an console script entry point I define in setup.py >> will install to the bin directory for an rpm as well? > > Give it a try and see. :) (Yes, setuptools does support bdist_rpm.) I knew you were going to say that :) I was actually struggling a bit early this morning trying to figure out exactly how it works. I was hoping I wouldn't have to actually give a traceback and reveal a stupid mistake :) My, for now, build cycle, until I automate it, is that I create a build/lib directory: Step 1: mkdir -p build/lib Step 2: Place script inside of build/lib cp script.py build/lib Step 3: In current directory run: python2.4 ./setup.py bdist_rpm --binary-only --release=py24 --python=/ usr/local/bin/python2.4 Step 4: Grab rpm out of newly created dist directory: This process works just fine if I substitute: python2.4 ./setup.py bdist_egg In my setup.py I have an entry point as follows: entry_points=""" [console_scripts] liten = liten:main """, ) I can easy install a python2.4 or a python 2.5 egg without a glitch, but when I install the rpm I created I am able to run my script, but I get this traceback: [robroy at giftcsllc02 ~]$ liten Traceback (most recent call last): File "/usr/bin/liten", line 7, in ? sys.exit( File "/usr/lib/python2.4/site-packages/setuptools-0.6c7-py2.4.egg/ pkg_resources.py", line 277, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File "/usr/lib/python2.4/site-packages/setuptools-0.6c7-py2.4.egg/ pkg_resources.py", line 2179, in load_entry_point return ep.load() File "/usr/lib/python2.4/site-packages/setuptools-0.6c7-py2.4.egg/ pkg_resources.py", line 1912, in load entry = __import__(self.module_name, globals(),globals(), ['__name__']) ImportError: No module named liten My guess is that somehow I am not creating the RPM properly, and I need to have a different directly structure then when creating an egg? Thanks, Noah From python at jwp.name Thu Jan 17 17:33:59 2008 From: python at jwp.name (James William Pye) Date: Thu, 17 Jan 2008 09:33:59 -0700 Subject: [Distutils] multiple servers handling in .pypirc In-Reply-To: <14917917.post@talk.nabble.com> References: <14917917.post@talk.nabble.com> Message-ID: <20080117163359.GB97097@lit.jwp.name> FWIW, On Thu, Jan 17, 2008 at 04:40:22AM -0800, Tarek Ziad? wrote: > like discussed in catalog-sig, and explained here: > http://wiki.python.org/moin/EnhancedPyPI distutils/setuptools does need something like this, but I'm not terribly fond of how that file is laid out. :( In the EnhancedPyPI page, the .pypirc file starts to take on two roles: a keychain and a list of index servers. Perhaps it would be best to keep these things separate. A global file(and a user relative file) listing index servers or an environment variable would be handy, IMO. Also, it would be preferable if we could just use the URL/URI/IRI syntax: http://user:pass at example.com/pypi just my random thoughts as a user. nice work regardless of how you take my preferences. ;) From pje at telecommunity.com Thu Jan 17 17:47:53 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 17 Jan 2008 11:47:53 -0500 Subject: [Distutils] docs on using setuptools in a release cycle with RPMS In-Reply-To: <150EB5B7-20E2-4109-BC5D-13EA0A2FDDCD@gmail.com> References: <2ED54CE8-3A03-4B9B-A296-2CC5CB44F168@gmail.com> <20080117143247.191983A4077@sparrow.telecommunity.com> <150EB5B7-20E2-4109-BC5D-13EA0A2FDDCD@gmail.com> Message-ID: <20080117164755.EF2273A4077@sparrow.telecommunity.com> At 09:54 AM 1/17/2008 -0500, Noah Gift wrote: >On Jan 17, 2008, at 9:32 AM, Phillip J. Eby wrote: > >>At 04:16 AM 1/17/2008 -0500, Noah Gift wrote: >>>I am trying to get egg generation entry points integrated with >>>building RPM's, and hopefully debian packages too. I haven't had >>>much >>>luck finding any documentation on this though. My specific >>>question is: >>> >>>Can I use: >>> >>>python2.5 ./setup.py bdist_rpm --binary-only --release=py25 -- python=/ >>>usr/local/bin/python2.5 >>> >>> >>>and expect that an console script entry point I define in setup.py >>>will install to the bin directory for an rpm as well? >> >>Give it a try and see. :) (Yes, setuptools does support bdist_rpm.) > >I knew you were going to say that :) I was actually struggling a bit >early this morning trying to figure out exactly how it works. I was >hoping I wouldn't have to actually give a traceback and reveal a >stupid mistake :) > >My, for now, build cycle, until I automate it, is that I create a >build/lib directory: > >Step 1: > >mkdir -p build/lib > >Step 2: > >Place script inside of build/lib > >cp script.py build/lib Why are you doing these steps? I don't understand. >Step 3: > >In current directory run: > >python2.4 ./setup.py bdist_rpm --binary-only --release=py24 >--python=/ usr/local/bin/python2.4 > >Step 4: > >Grab rpm out of newly created dist directory: > >This process works just fine if I substitute: > >python2.4 ./setup.py bdist_egg > > >In my setup.py I have an entry point as follows: > > entry_points=""" > [console_scripts] > liten = liten:main > """, > ) What's the rest of your setup.py? >My guess is that somehow I am not creating the RPM properly, and I >need to have a different directly structure then when creating an egg? My guess is that your 'liten' module isn't listed in the py_modules argument to setup(). Is it? From noah.gift at gmail.com Thu Jan 17 19:07:46 2008 From: noah.gift at gmail.com (Noah Gift) Date: Thu, 17 Jan 2008 13:07:46 -0500 Subject: [Distutils] docs on using setuptools in a release cycle with RPMS In-Reply-To: <20080117164755.EF2273A4077@sparrow.telecommunity.com> References: <2ED54CE8-3A03-4B9B-A296-2CC5CB44F168@gmail.com> <20080117143247.191983A4077@sparrow.telecommunity.com> <150EB5B7-20E2-4109-BC5D-13EA0A2FDDCD@gmail.com> <20080117164755.EF2273A4077@sparrow.telecommunity.com> Message-ID: <1A6BB450-CBB0-45BA-B705-CB28A157AD9C@gmail.com> >> >> >> cp script.py build/lib > > Why are you doing these steps? I don't understand. In order to create an egg, to my knowledge, you create a build/lib directory and place your scripts or library inside, then run setup.py bdist_egg. Those were the steps I took, to successfully create a python 2.4 and a python 2.5 egg. > >> Step 3: >> >> In current directory run: >> >> python2.4 ./setup.py bdist_rpm --binary-only --release=py24 -- >> python=/ usr/local/bin/python2.4 >> >> Step 4: >> >> Grab rpm out of newly created dist directory: >> >> This process works just fine if I substitute: >> >> python2.4 ./setup.py bdist_egg >> >> >> In my setup.py I have an entry point as follows: >> >> entry_points=""" >> [console_scripts] >> liten = liten:main >> """, >> ) > > What's the rest of your setup.py? Here is the rest: from setuptools import Extension, find_packages, setup setup(name='liten', version='0.1.3', description='a de-duplication command line tool', long_description="This command line tool will examine a file system and \ report back duplicates using a md5 checksum hash algorithm.", author='Noah Gift', author_email='noah.gift at gmail.com', url='http://code.google.com/p/liten/', license='MIT', packages=find_packages(), zip_safe=False, entry_points=""" [console_scripts] liten = liten:main """, ) > > >> My guess is that somehow I am not creating the RPM properly, and I >> need to have a different directly structure then when creating an >> egg? > > My guess is that your 'liten' module isn't listed in the py_modules > argument to setup(). Is it? That is correct. I am able to create eggs with entry points that work just fine without specifying my module, but are you saying that in order to use the bdist_rpm command that I also need to include: py_modules=['liten'] Of course, I am going to try adding this right now. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20080117/9c62ce9c/attachment.htm From pje at telecommunity.com Thu Jan 17 19:58:50 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 17 Jan 2008 13:58:50 -0500 Subject: [Distutils] docs on using setuptools in a release cycle with RPMS In-Reply-To: <1A6BB450-CBB0-45BA-B705-CB28A157AD9C@gmail.com> References: <2ED54CE8-3A03-4B9B-A296-2CC5CB44F168@gmail.com> <20080117143247.191983A4077@sparrow.telecommunity.com> <150EB5B7-20E2-4109-BC5D-13EA0A2FDDCD@gmail.com> <20080117164755.EF2273A4077@sparrow.telecommunity.com> <1A6BB450-CBB0-45BA-B705-CB28A157AD9C@gmail.com> Message-ID: <20080117185850.7379D3A4077@sparrow.telecommunity.com> At 01:07 PM 1/17/2008 -0500, Noah Gift wrote: >>>cp script.py build/lib >> >>Why are you doing these steps? I don't understand. > >In order to create an egg, to my knowledge, you create a build/lib >directory and place your scripts or library inside, then run >setup.py bdist_egg. Where did you get that knowledge? We should correct whatever source you got that from, because that is absolutely *not* the way to do it, and is in fact a good way to lose your work. (E.g. "setup.py clean" will delete it.) >That is correct. I am able to create eggs with entry points that >work just fine without specifying my module, but are you saying that >in order to use the bdist_rpm command that I also need to include: > >py_modules=['liten'] No, what I'm saying is that you should have that line in your setup.py, period, and you should not mess with the contents of build/lib or really *anything* under build. The build/ directory is used by the distutils to build things. Your source code goes directly alongside setup.py, generally speaking. You may want to review the "Distributing Python Modules" documentation at: http://docs.python.org/dist/ As you'll find there, there is no mention of putting things into build/ - you put things in the directory where your setup.py is, and the distutils (or setuptools) are responsible for copying them into build/ and other places. There are absolutely no user-serviceable parts in build/. What you've been doing is a bit like taking a power drill and then turning it by the handle instead of plugging it in and pulling the trigger. In some cases it may seem to be work, but mostly you are just making things harder than they need to be. :) From noah.gift at gmail.com Thu Jan 17 21:04:04 2008 From: noah.gift at gmail.com (Noah Gift) Date: Thu, 17 Jan 2008 15:04:04 -0500 Subject: [Distutils] docs on using setuptools in a release cycle with RPMS In-Reply-To: <20080117185850.7379D3A4077@sparrow.telecommunity.com> References: <2ED54CE8-3A03-4B9B-A296-2CC5CB44F168@gmail.com> <20080117143247.191983A4077@sparrow.telecommunity.com> <150EB5B7-20E2-4109-BC5D-13EA0A2FDDCD@gmail.com> <20080117164755.EF2273A4077@sparrow.telecommunity.com> <1A6BB450-CBB0-45BA-B705-CB28A157AD9C@gmail.com> <20080117185850.7379D3A4077@sparrow.telecommunity.com> Message-ID: >>> >> In order to create an egg, to my knowledge, you create a build/lib >> directory and place your scripts or library inside, then run >> setup.py bdist_egg. > > Where did you get that knowledge? We should correct whatever source > you got that from, because that is absolutely *not* the way to do > it, and is in fact a good way to lose your work. (E.g. "setup.py > clean" will delete it.) I may have just assumed this, by looking at examples and reading output from stdout. This may have been a trial and error solution, so I am glad I asked and found out this was incorrect. Of course, I come from a R/D and Systems Administration background so I am pretty used to doing really, really bad things I am not supposed to do :) > >> That is correct. I am able to create eggs with entry points that >> work just fine without specifying my module, but are you saying >> that in order to use the bdist_rpm command that I also need to >> include: >> >> py_modules=['liten'] > > No, what I'm saying is that you should have that line in your > setup.py, period, and you should not mess with the contents of build/ > lib or really *anything* under build. The build/ directory is used > by the distutils to build things. Your source code goes directly > alongside setup.py, generally speaking. Ok, this is good to know. That is much more intuitive. I may have run into some issue that made me think creating a build/lib directory would help. > You may want to review the "Distributing Python Modules" > documentation at: > > http://docs.python.org/dist/ > > As you'll find there, there is no mention of putting things into > build/ - you put things in the directory where your setup.py is, and > the distutils (or setuptools) are responsible for copying them into > build/ and other places. There are absolutely no user-serviceable > parts in build/. > > What you've been doing is a bit like taking a power drill and then > turning it by the handle instead of plugging it in and pulling the > trigger. In some cases it may seem to be work, but mostly you are > just making things harder than they need to be. :) > Believe it or not, I am writing a chapter on Eggs, Buildout, and Virtualenv for our O'Reilly book. I hope to do a good job of explaining how to use eggs/buildout/virtualenv and build RPM's and other packages such as AIX/HP-UX, Debian/Ubuntu. I might be asking a few more dumb questions, so I will try to read as much as I can to limit the total number. I think a large source of confusion for me, was thinking that distutils was somehow deprecated, and that I don't even need to read the docs. I have heard so many horror stories about distutils, that I have mistakenly thought they are somehow broken badly, and it is not worth reading, and that they are replaced by setuptools. I suppose there is going to be some confusion for me in trying to understand the intersection between disutils and setuptools, as I am not sure what I am expected to know from distutils and what setuptools replaces. I was operating under, obviously a false assumption, that setuptools was the was forward and was completely ignoring distutils. ok, off to read the distutils docs.... From pje at telecommunity.com Thu Jan 17 22:06:20 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 17 Jan 2008 16:06:20 -0500 Subject: [Distutils] docs on using setuptools in a release cycle with RPMS In-Reply-To: References: <2ED54CE8-3A03-4B9B-A296-2CC5CB44F168@gmail.com> <20080117143247.191983A4077@sparrow.telecommunity.com> <150EB5B7-20E2-4109-BC5D-13EA0A2FDDCD@gmail.com> <20080117164755.EF2273A4077@sparrow.telecommunity.com> <1A6BB450-CBB0-45BA-B705-CB28A157AD9C@gmail.com> <20080117185850.7379D3A4077@sparrow.telecommunity.com> Message-ID: <20080117210619.696173A4077@sparrow.telecommunity.com> At 03:04 PM 1/17/2008 -0500, Noah Gift wrote: >I suppose there is going to be some confusion for me in >trying to understand the intersection between disutils and setuptools, >as I am not sure what I am expected to know from distutils and what >setuptools replaces. The setuptools docs really only document the differences between distutils and setuptools. So if you read them looking for what setuptools claims you *don't* have to do, then you can skip those parts of the distutils. For example, if you are using a revision control system to manage your source files, then you don't need to know the details of how to create a MANIFEST.in file -- something that's mandatory with the distutils in order to create a source distribution or something like an RPM. Regarding your eggs/buildout/etc. chapter, please feel free to email me an early copy for technical review and I'll promptly inform you about any errors. (Unless it's late next week or any time the following week, as I will be busy and not responding to virtually any email whatsoever :) .) From bgnori at gmail.com Fri Jan 18 02:29:56 2008 From: bgnori at gmail.com (=?ISO-2022-JP?B?GyRCSl06ZEhPOVQbKEI=?=) Date: Fri, 18 Jan 2008 10:29:56 +0900 Subject: [Distutils] [Enthought-dev] Building Fedora RPMs Message-ID: <315ac72e0801171729p66a17dbai71b50d592bc9375d@mail.gmail.com> > I tried to do that and build the package, but I ran into problems of svn > and setuptools remembering what was there before and I got worse errors. > :-( I've had problems like that before. clean /var/tmp with rm -rf /var/tmp/* -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20080118/352c405e/attachment.htm From viktor at chalmers.se Fri Jan 18 11:34:13 2008 From: viktor at chalmers.se (Viktor Fougstedt) Date: Fri, 18 Jan 2008 11:34:13 +0100 Subject: [Distutils] Experience of setuptools' cache design Message-ID: Hello. Being nothing but an innocent bystander, I have not thuroughly searched the archives of this list, and might be writing about something that is "old news" since long ago. If so, I apologize. This is a small story about what I, being an end user that did not know what "setuptools" was, experienced when I installed the latest MySQLdb module (and I am aware that there might very well be issues with how it was installed). My hope is that this is useful to the developers of setuptools in determining if they meet the design criteria they have set up. A colleague of mine installed the latest MySQLdb module, which uses eggs and setuptools. We use Python and MySQL heavily on our mailserver. In that environment, user names and home directories do not always match, and indeed most users running the programs are system users and do not have home directories at all. This showed to be a problem with the setuptools-based MySQLdb module. The reason is the .python-eggs directory. Our configuration daemon runs as a system user "graald", a user without a home directory (that user does not have a vaild shell either). After updating MySQLdb to the setuptools-based version, the Python server code no longer started, because setuptools tried to write to the .python-eggs directory in the user's $HOME. And $HOME is not a valid directory (or, often, the home directory of the person starting the script rather than the account it runs as). Ok, so we can set the PYTHON_EGG_CACHE environment variable, which solves the problem (but in an unclean way). Then comes the Python script that runs as "virtmail" and tries to connect to MySQL, "virtmail" being another user without a home directory. And of course it does not start, and cannot have the same PYTHON_EGG_CACHE directory, since both these users cannot write to the same directory. And, not having a home directory, the "virtmail" user does not have a .bashrc or similar where we can set that user's own cache directory. We cannot set PYTHON _EGG_CACHE in the script itself either, as we cannot know what to set it to. Because if two different system users run the same script, they need different cache directories. Uh-oh. We need to implement code to direct the cache directory depending on the user that's running. And when I tried to run a script under my own user, with a proper home directory, nothing worked as expected either. The reason? I have previously run a Python script as root (but without changing my $HOME at that time). So now I have a .python-eggs in my own home directory owned by root:root. My thoughts (perhaps useful, perhaps not): * Primarly, I think it is unfortunate that an "import foo" starts creating files in the file system - it is not what I personally expect from doing an "import"! * On our user system, with some 20.000 active users, there will be up to 20.000 copies of a .python-eggs directory if someone installs a program that uses a Python Egg (but does not have access to site- packages or does not know how to detect what eggs are used and how to install them there). * I, personally, think it would be better if I explicitly have to _request_ a per-user cache directory being made, rather than needing to implement solutions to _prevent_ that from happening. * If the default is to remain to create files on "import", I would like error checking and fall backs. If the cache directory cannot be created in $HOME, I would like the code to create it somewhere else (or not at all) instead of giving me an exception. As end user I did not request the cache-directory to be made, and therefore do not want to be given an exception caused by it not being created. Especially as I do not know what to do with such an exception. Perhaps creating it in /tmp/python-eggs-$USERNAME, for example. Thanks for listening. /Viktor From pje at telecommunity.com Fri Jan 18 15:49:13 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 18 Jan 2008 09:49:13 -0500 Subject: [Distutils] Experience of setuptools' cache design In-Reply-To: References: Message-ID: <20080118144913.47EEF3A40A2@sparrow.telecommunity.com> At 11:34 AM 1/18/2008 +0100, Viktor Fougstedt wrote: >My thoughts (perhaps useful, perhaps not): > >* Primarly, I think it is unfortunate that an "import foo" starts >creating files in the file system - it is not what I personally >expect from doing an "import"! Note that normal imports will also create .pyc or .pyo files alongside the source, if a valid compiled version of the source isn't available. >* On our user system, with some 20.000 active users, there will be up >to 20.000 copies of a .python-eggs directory if someone installs a >program that uses a Python Egg (but does not have access to site- >packages or does not know how to detect what eggs are used and how to >install them there). If you want eggs to always be installed unzipped, you can add an --always-unzip option to your site-wide distutils.cfg. Then, by default this will unzip an egg at installation time rather than extracting libraries at run time. >* I, personally, think it would be better if I explicitly have to >_request_ a per-user cache directory being made, rather than needing >to implement solutions to _prevent_ that from happening. > >* If the default is to remain to create files on "import", I would >like error checking and fall backs. If the cache directory cannot be >created in $HOME, I would like the code to create it somewhere else >(or not at all) instead of giving me an exception. As end user I did >not request the cache-directory to be made, and therefore do not want >to be given an exception caused by it not being created. Especially >as I do not know what to do with such an exception. Perhaps creating >it in /tmp/python-eggs-$USERNAME, for example. That seems like a reasonable fallback, and I'll take a look at implementing it in a future release. Thanks for the idea! From jim at zope.com Fri Jan 18 16:06:30 2008 From: jim at zope.com (Jim Fulton) Date: Fri, 18 Jan 2008 10:06:30 -0500 Subject: [Distutils] Experience of setuptools' cache design In-Reply-To: References: Message-ID: <680A348C-518A-4491-B3AF-8F75A4F2E2F1@zope.com> On Jan 18, 2008, at 5:34 AM, Viktor Fougstedt wrote: ... > My thoughts (perhaps useful, perhaps not): > > * Primarly, I think it is unfortunate that an "import foo" starts > creating files in the file system - it is not what I personally > expect from doing an "import"! > > * On our user system, with some 20.000 active users, there will be up > to 20.000 copies of a .python-eggs directory if someone installs a > program that uses a Python Egg (but does not have access to site- > packages or does not know how to detect what eggs are used and how to > install them there). > > * I, personally, think it would be better if I explicitly have to > _request_ a per-user cache directory being made, rather than needing > to implement solutions to _prevent_ that from happening. > > * If the default is to remain to create files on "import", I would > like error checking and fall backs. If the cache directory cannot be > created in $HOME, I would like the code to create it somewhere else > (or not at all) instead of giving me an exception. As end user I did > not request the cache-directory to be made, and therefore do not want > to be given an exception caused by it not being created. Especially > as I do not know what to do with such an exception. Perhaps creating > it in /tmp/python-eggs-$USERNAME, for example. I couldn't agree more. And, AFAICT, everyone I know, except Phillip, agrees that this is a bad feature. IMO, any package including extensions should be treated as zip- unsafe. I intend to add this policy to buildout. Then the only way to be bitten is to use one of the resource APIs that causes files to be extracted from zipped eggs. I'm happy to consider calls to these APIs to be bugs. As a work around, when installing packages that have this problem, use the -Z option to easy_install to cause them to be installed unzipped (or unzip them yourself). Jim -- Jim Fulton Zope Corporation From jim at zope.com Fri Jan 18 16:08:51 2008 From: jim at zope.com (Jim Fulton) Date: Fri, 18 Jan 2008 10:08:51 -0500 Subject: [Distutils] Experience of setuptools' cache design In-Reply-To: <20080118144913.47EEF3A40A2@sparrow.telecommunity.com> References: <20080118144913.47EEF3A40A2@sparrow.telecommunity.com> Message-ID: <63AE0A59-F7F7-443C-A3E2-E2D741B42364@zope.com> On Jan 18, 2008, at 9:49 AM, Phillip J. Eby wrote: > At 11:34 AM 1/18/2008 +0100, Viktor Fougstedt wrote: >> My thoughts (perhaps useful, perhaps not): >> >> * Primarly, I think it is unfortunate that an "import foo" starts >> creating files in the file system - it is not what I personally >> expect from doing an "import"! > > Note that normal imports will also create .pyc or .pyo files > alongside the source, if a valid compiled version of the source isn't > available. Only if the process has write access to the directory. No error occurs if the pyc files can't be written. Jim -- Jim Fulton Zope Corporation From python at venix.com Fri Jan 18 16:12:20 2008 From: python at venix.com (Lloyd Kvam) Date: Fri, 18 Jan 2008 10:12:20 -0500 Subject: [Distutils] Experience of setuptools' cache design In-Reply-To: <20080118144913.47EEF3A40A2@sparrow.telecommunity.com> References: <20080118144913.47EEF3A40A2@sparrow.telecommunity.com> Message-ID: <1200669140.22070.48.camel@localhost.localdomain> On Fri, 2008-01-18 at 09:49 -0500, Phillip J. Eby wrote: > >* On our user system, with some 20.000 active users, there will be up > >to 20.000 copies of a .python-eggs directory if someone installs a > >program that uses a Python Egg (but does not have access to site- > >packages or does not know how to detect what eggs are used and how to > >install them there). > > If you want eggs to always be installed unzipped, you can add an > --always-unzip option to your site-wide distutils.cfg. Then, by > default this will unzip an egg at installation time rather than > extracting libraries at run time. With my distutils distutils.__version__=='2.5.1' I had to use this: [easy_install] zip_ok = False --always-unzip was not recognized -- Lloyd Kvam Venix Corp DLSLUG/GNHLUG library http://www.librarything.com/catalog/dlslug http://www.librarything.com/profile/dlslug http://www.librarything.com/rsshtml/recent/dlslug From jim at zope.com Fri Jan 18 16:13:47 2008 From: jim at zope.com (Jim Fulton) Date: Fri, 18 Jan 2008 10:13:47 -0500 Subject: [Distutils] Experience of setuptools' cache design In-Reply-To: <20080118144913.47EEF3A40A2@sparrow.telecommunity.com> References: <20080118144913.47EEF3A40A2@sparrow.telecommunity.com> Message-ID: On Jan 18, 2008, at 9:49 AM, Phillip J. Eby wrote: ... >> * If the default is to remain to create files on "import", I would >> like error checking and fall backs. If the cache directory cannot be >> created in $HOME, I would like the code to create it somewhere else >> (or not at all) instead of giving me an exception. As end user I did >> not request the cache-directory to be made, and therefore do not want >> to be given an exception caused by it not being created. Especially >> as I do not know what to do with such an exception. Perhaps creating >> it in /tmp/python-eggs-$USERNAME, for example. > > That seems like a reasonable fallback, and I'll take a look at > implementing it in a future release. Thanks for the idea! I'm not sure which of the ideas above you are referring to but I think implicitly creating files at run time outside the installed package is a bad idea. I would want some way to prevent it, especially in a production environment. Jim -- Jim Fulton Zope Corporation From pje at telecommunity.com Fri Jan 18 18:32:00 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 18 Jan 2008 12:32:00 -0500 Subject: [Distutils] Experience of setuptools' cache design In-Reply-To: References: <20080118144913.47EEF3A40A2@sparrow.telecommunity.com> Message-ID: <20080118173159.692FD3A40AE@sparrow.telecommunity.com> At 10:13 AM 1/18/2008 -0500, Jim Fulton wrote: >On Jan 18, 2008, at 9:49 AM, Phillip J. Eby wrote: >... >>>* If the default is to remain to create files on "import", I would >>>like error checking and fall backs. If the cache directory cannot be >>>created in $HOME, I would like the code to create it somewhere else >>>(or not at all) instead of giving me an exception. As end user I did >>>not request the cache-directory to be made, and therefore do not want >>>to be given an exception caused by it not being created. Especially >>>as I do not know what to do with such an exception. Perhaps creating >>>it in /tmp/python-eggs-$USERNAME, for example. >> >>That seems like a reasonable fallback, and I'll take a look at >>implementing it in a future release. Thanks for the idea! > > >I'm not sure which of the ideas above you are referring to but I think >implicitly creating files at run time outside the installed package is >a bad idea. I'm referring to the idea of using a temporary directory as a cache. > I would want some way to prevent it, especially in a >production environment. The only way to absolutely prevent it is to install things unzipped. And if you're going to do that, you might as well use .egg-info style eggs, since that will give better runtime performance. That is, for an egg "foo-1.2-py2.5-platform.egg", you would unzip its contents to the target directory and then rename the extracted EGG-INFO directory to "foo-1.2-py2.5-platform.egg-info". >Jim > >-- >Jim Fulton >Zope Corporation > From jim at zope.com Fri Jan 18 18:50:38 2008 From: jim at zope.com (Jim Fulton) Date: Fri, 18 Jan 2008 12:50:38 -0500 Subject: [Distutils] Experience of setuptools' cache design In-Reply-To: <20080118173159.692FD3A40AE@sparrow.telecommunity.com> References: <20080118144913.47EEF3A40A2@sparrow.telecommunity.com> <20080118173159.692FD3A40AE@sparrow.telecommunity.com> Message-ID: <5029B225-3E4A-4561-BB13-3FF52BA6ED3A@zope.com> On Jan 18, 2008, at 12:32 PM, Phillip J. Eby wrote: > At 10:13 AM 1/18/2008 -0500, Jim Fulton wrote: ... >> I would want some way to prevent it, especially in a >> production environment. > > The only way to absolutely prevent it is to install things > unzipped. And if you're going to do that, you might as well > use .egg-info style eggs, since that will give better runtime > performance. That is, for an egg "foo-1.2-py2.5-platform.egg", you > would unzip its contents to the target directory and then rename the > extracted EGG-INFO directory to "foo-1.2-py2.5-platform.egg-info". Thanks. Can you briefly explain or provide a link to something that explains the performance improvement? JIm -- Jim Fulton Zope Corporation From pje at telecommunity.com Fri Jan 18 19:04:43 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 18 Jan 2008 13:04:43 -0500 Subject: [Distutils] Experience of setuptools' cache design In-Reply-To: <5029B225-3E4A-4561-BB13-3FF52BA6ED3A@zope.com> References: <20080118144913.47EEF3A40A2@sparrow.telecommunity.com> <20080118173159.692FD3A40AE@sparrow.telecommunity.com> <5029B225-3E4A-4561-BB13-3FF52BA6ED3A@zope.com> Message-ID: <20080118180442.8D6A63A4109@sparrow.telecommunity.com> At 12:50 PM 1/18/2008 -0500, Jim Fulton wrote: >On Jan 18, 2008, at 12:32 PM, Phillip J. Eby wrote: > >>At 10:13 AM 1/18/2008 -0500, Jim Fulton wrote: >... >>> I would want some way to prevent it, especially in a >>>production environment. >> >>The only way to absolutely prevent it is to install things >>unzipped. And if you're going to do that, you might as well >>use .egg-info style eggs, since that will give better runtime >>performance. That is, for an egg "foo-1.2-py2.5-platform.egg", you >>would unzip its contents to the target directory and then rename the >>extracted EGG-INFO directory to "foo-1.2-py2.5-platform.egg-info". > > >Thanks. > >Can you briefly explain or provide a link to something that explains >the performance improvement? Fewer directories on sys.path = better import performance, compared to individually putting a series of .egg directories on sys.path. This doesn't apply so much for zipped eggs, because the zip directories get cached. Overall, the tradeoffs are: zipped .egg files = faster imports + some startup overhead to read the zip directories .egg directories = slower imports due to lots of stat-ing .egg-info = "normal" imports + no extra startup overhead Zipped .egg files have the fastest import times overall, assuming you don't have so many eggs to read that the overhead wipes out your gains. Extracting all eggs to the same directory is second fastest, with .egg directories (i.e. --always-unzip) being the slowest thing you could possibly do. From jim at zope.com Fri Jan 18 19:31:30 2008 From: jim at zope.com (Jim Fulton) Date: Fri, 18 Jan 2008 13:31:30 -0500 Subject: [Distutils] Experience of setuptools' cache design In-Reply-To: <20080118180442.8D6A63A4109@sparrow.telecommunity.com> References: <20080118144913.47EEF3A40A2@sparrow.telecommunity.com> <20080118173159.692FD3A40AE@sparrow.telecommunity.com> <5029B225-3E4A-4561-BB13-3FF52BA6ED3A@zope.com> <20080118180442.8D6A63A4109@sparrow.telecommunity.com> Message-ID: <891A27C5-3DAA-469D-B707-27E08A7EA90F@zope.com> On Jan 18, 2008, at 1:04 PM, Phillip J. Eby wrote: > At 12:50 PM 1/18/2008 -0500, Jim Fulton wrote: > >> On Jan 18, 2008, at 12:32 PM, Phillip J. Eby wrote: >> >>> At 10:13 AM 1/18/2008 -0500, Jim Fulton wrote: >> ... >>>> I would want some way to prevent it, especially in a >>>> production environment. >>> >>> The only way to absolutely prevent it is to install things >>> unzipped. And if you're going to do that, you might as well >>> use .egg-info style eggs, since that will give better runtime >>> performance. That is, for an egg "foo-1.2-py2.5-platform.egg", you >>> would unzip its contents to the target directory and then rename the >>> extracted EGG-INFO directory to "foo-1.2-py2.5-platform.egg-info". >> >> >> Thanks. >> >> Can you briefly explain or provide a link to something that explains >> the performance improvement? > > Fewer directories on sys.path = better import performance, compared > to individually putting a series of .egg directories on sys.path. Ah, I didn't understand what you meant by "unzip the contents to the target directory". So the idea is that you'd merge the contents of multiple zip files into a single "target" directory. This seems rather messy to me. It doesn't appear to be compatible with multi-version installs. Also, if a newer egg version removes a file, the removed file will be left installed after an upgrade. If two eggs provide the same file, files will be overridden. Admittedly, this is a somewhat pathological case, but the overriding seems to compound the pathology. > This doesn't apply so much for zipped eggs, because the zip > directories get cached. Overall, the tradeoffs are: > > zipped .egg files = faster imports + some startup overhead to read > the zip directories > .egg directories = slower imports due to lots of stat-ing > .egg-info = "normal" imports + no extra startup overhead > > Zipped .egg files have the fastest import times overall, assuming > you don't have so many eggs to read that the overhead wipes out your > gains. Extracting all eggs to the same directory is second fastest, > with .egg directories (i.e. --always-unzip) being the slowest thing > you could possibly do. I wonder what the performance impacts really are. I'm looking forward to measuring this some day for our projects. We've put off making our eggs zip safe so far, as this hasn't been a high priority. :/ As described in a separate thread, I'm going to add an option to buildout so buildout users can explicitly define a directory to use as a setuptools cache. In that case, zip-safe eggs can remain zipped even if they have extensions. Jim -- Jim Fulton Zope Corporation From viktor at chalmers.se Fri Jan 18 19:38:34 2008 From: viktor at chalmers.se (Viktor Fougstedt) Date: Fri, 18 Jan 2008 19:38:34 +0100 Subject: [Distutils] Experience of setuptools' cache design In-Reply-To: <20080118180442.8D6A63A4109@sparrow.telecommunity.com> References: <20080118144913.47EEF3A40A2@sparrow.telecommunity.com> <20080118173159.692FD3A40AE@sparrow.telecommunity.com> <5029B225-3E4A-4561-BB13-3FF52BA6ED3A@zope.com> <20080118180442.8D6A63A4109@sparrow.telecommunity.com> Message-ID: On 18 jan 2008, at 19.04, Phillip J. Eby wrote: > At 12:50 PM 1/18/2008 -0500, Jim Fulton wrote: >> >> Can you briefly explain or provide a link to something that explains >> the performance improvement? > > Fewer directories on sys.path = better import performance, compared > to individually putting a series of .egg directories on sys.path. Hi again, and thanks for the quick and interesting responses. For me as a naive end user, the import performance is not critical on any system I run - my site-packages directories rarely contains more than five to ten packages, and have lots of CPU and I/O to use (I even believe frequently recurring stat():s are cached by the OS). I am, personally, more concerned with a packaging system being scalable to many different computer systems, where user-id:s and home directories do not match (and many user-id:s, indeed, do not have home directories), where there are tens of thousands of users, or where there are two or three separate installations of libraries and binaries and the modules need to use the right ones depending on which python interpreter is run. Being scalable to handle thousands of packages in site-packages is not on my personal list of priorities. It's all about which dimension you want the packaging system to be flexible in, I guess. My little vote goes to the "many users and strange systems" dimension. :-) But who knows - many CPAN installations actually _have_ hundreds of dependencies installed. And with a packaging system that makes it as easy to install Python packages, perhaps the same will happen for us. Regards, /Viktor From pje at telecommunity.com Fri Jan 18 19:55:32 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 18 Jan 2008 13:55:32 -0500 Subject: [Distutils] Experience of setuptools' cache design In-Reply-To: <891A27C5-3DAA-469D-B707-27E08A7EA90F@zope.com> References: <20080118144913.47EEF3A40A2@sparrow.telecommunity.com> <20080118173159.692FD3A40AE@sparrow.telecommunity.com> <5029B225-3E4A-4561-BB13-3FF52BA6ED3A@zope.com> <20080118180442.8D6A63A4109@sparrow.telecommunity.com> <891A27C5-3DAA-469D-B707-27E08A7EA90F@zope.com> Message-ID: <20080118185531.DDD073A40A2@sparrow.telecommunity.com> At 01:31 PM 1/18/2008 -0500, Jim Fulton wrote: >On Jan 18, 2008, at 1:04 PM, Phillip J. Eby wrote: > >>At 12:50 PM 1/18/2008 -0500, Jim Fulton wrote: >> >>>On Jan 18, 2008, at 12:32 PM, Phillip J. Eby wrote: >>> >>>>At 10:13 AM 1/18/2008 -0500, Jim Fulton wrote: >>>... >>> >>>Can you briefly explain or provide a link to something that explains >>>the performance improvement? >> >>Fewer directories on sys.path = better import performance, compared >>to individually putting a series of .egg directories on sys.path. > >Ah, I didn't understand what you meant by "unzip the contents to the >target directory". > >So the idea is that you'd merge the contents of multiple zip files >into a single "target" directory. Correct. >This seems rather messy to me. Well, it's what we all used to do before eggs came along, and what most packaging systems still do now. :) > It doesn't appear to be compatible >with multi-version installs. It *can* be compatible, with certain limitations. > Also, if a newer egg version removes a >file, the removed file will be left installed after an upgrade. If >two eggs provide the same file, files will be overridden. Admittedly, >this is a somewhat pathological case, but the overriding seems to >compound the pathology. Again, this is all true without eggs now. Clearly, eggs have spoiled you tremendously. ;-) Seriously, though, if buildout is tracking what files get installed or overwritten during unzipping, you can manage all of this, just as you presumably do for any other sort of installation recipe, no? >As described in a separate thread, I'm going to add an option to >buildout so buildout users can explicitly define a directory to use as >a setuptools cache. In that case, zip-safe eggs can remain zipped >even if they have extensions. If you want to be really safe/careful, you can pre-extract the eggs to the cache, thereby avoiding any runtime permission problems. From jim at zope.com Fri Jan 18 20:08:45 2008 From: jim at zope.com (Jim Fulton) Date: Fri, 18 Jan 2008 14:08:45 -0500 Subject: [Distutils] Experience of setuptools' cache design In-Reply-To: <20080118185531.DDD073A40A2@sparrow.telecommunity.com> References: <20080118144913.47EEF3A40A2@sparrow.telecommunity.com> <20080118173159.692FD3A40AE@sparrow.telecommunity.com> <5029B225-3E4A-4561-BB13-3FF52BA6ED3A@zope.com> <20080118180442.8D6A63A4109@sparrow.telecommunity.com> <891A27C5-3DAA-469D-B707-27E08A7EA90F@zope.com> <20080118185531.DDD073A40A2@sparrow.telecommunity.com> Message-ID: <2010A64B-8F70-4D37-B270-53FAE2AD5886@zope.com> On Jan 18, 2008, at 1:55 PM, Phillip J. Eby wrote: ... >> It doesn't appear to be compatible >> with multi-version installs. > > It *can* be compatible, with certain limitations. I don't see how, unless you mean that only one version is handled this way. >> Also, if a newer egg version removes a >> file, the removed file will be left installed after an upgrade. If >> two eggs provide the same file, files will be overridden. Admittedly, >> this is a somewhat pathological case, but the overriding seems to >> compound the pathology. > > Again, this is all true without eggs now. Clearly, eggs have > spoiled you tremendously. ;-) Absolutely. No kidding. Eggs are great! > Seriously, though, if buildout is tracking what files get installed > or overwritten during unzipping, you can manage all of this, just as > you presumably do for any other sort of installation recipe, no? It isn't possible if different parts need different versions. If I ignore multi-version requirements, then I could keep track of files installed. In addition, a nice feature of buildouts is that you can share egg directories between buildouts. This is very handy, but makes multi- version support even more important. >> As described in a separate thread, I'm going to add an option to >> buildout so buildout users can explicitly define a directory to use >> as >> a setuptools cache. In that case, zip-safe eggs can remain zipped >> even if they have extensions. > > If you want to be really safe/careful, you can pre-extract the eggs > to the cache, thereby avoiding any runtime permission problems. Yup, except I don't see a way to enumerate the resources. After all, resources aren't listed in the egg meta data. I could make an educated guess and extract all of the non-py files. Or I guess I could just extract everything. :) Disk space is pretty cheap, so this is really quite practical IMO. JIm -- Jim Fulton Zope Corporation From zooko at zooko.com Fri Jan 18 21:13:25 2008 From: zooko at zooko.com (zooko) Date: Fri, 18 Jan 2008 13:13:25 -0700 Subject: [Distutils] Experience of setuptools' cache design In-Reply-To: <680A348C-518A-4491-B3AF-8F75A4F2E2F1@zope.com> References: <680A348C-518A-4491-B3AF-8F75A4F2E2F1@zope.com> Message-ID: <5927B3AA-48FC-4FE6-8AE5-CE30CBE194F4@zooko.com> On Jan 18, 2008, at 8:06 AM, Jim Fulton wrote: > I couldn't agree more. And, AFAICT, everyone I know, except Phillip, > agrees that this is a bad feature. For what it is worth, this is one of (several) blockers preventing Twisted from using setuptools: http://twistedmatrix.com/trac/ticket/2308#comment:6 (Note: I am not a Twisted devel, and even if the issue described in ticket #230, comment:6 were solved, there are probably still a lot of other reasons why the Twisted devels wouldn't use setuptools. Regards, Zooko From pje at telecommunity.com Fri Jan 18 22:33:46 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 18 Jan 2008 16:33:46 -0500 Subject: [Distutils] Experience of setuptools' cache design In-Reply-To: <2010A64B-8F70-4D37-B270-53FAE2AD5886@zope.com> References: <20080118144913.47EEF3A40A2@sparrow.telecommunity.com> <20080118173159.692FD3A40AE@sparrow.telecommunity.com> <5029B225-3E4A-4561-BB13-3FF52BA6ED3A@zope.com> <20080118180442.8D6A63A4109@sparrow.telecommunity.com> <891A27C5-3DAA-469D-B707-27E08A7EA90F@zope.com> <20080118185531.DDD073A40A2@sparrow.telecommunity.com> <2010A64B-8F70-4D37-B270-53FAE2AD5886@zope.com> Message-ID: <20080118214619.555853A40A2@sparrow.telecommunity.com> At 02:08 PM 1/18/2008 -0500, Jim Fulton wrote: >On Jan 18, 2008, at 1:55 PM, Phillip J. Eby wrote: >... >>> It doesn't appear to be compatible >>>with multi-version installs. >> >>It *can* be compatible, with certain limitations. > >I don't see how, unless you mean that only one version is handled this >way. That would be one of the "certain limitations", yes. :) The other is that the version installed this way can't be overridden except by scripts. That is, if you start Python without a script that initializes one of the alternate versions to be on sys.path, then you will get the "single-version" install as the version on sys.path. From pje at telecommunity.com Fri Jan 18 22:54:37 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 18 Jan 2008 16:54:37 -0500 Subject: [Distutils] setuptools on jython In-Reply-To: <123A3D33-41DE-4837-907F-FB062279E4D5@groovie.org> References: <36A10A2D-4103-45D5-BBCF-E8DE60B49880@groovie.org> <20080112045144.439313A40A2@sparrow.telecommunity.com> <123A3D33-41DE-4837-907F-FB062279E4D5@groovie.org> Message-ID: <20080118215435.848E03A40A2@sparrow.telecommunity.com> At 05:57 PM 1/12/2008 -0800, Philip Jenvey wrote: >Attached is a simple patch to disable the zip_safe scan on Jython/ >IronPython. Maybe in the future we can use tokenize when all of the >source code is present in an archive. I've now incorporated comparable changes for bytecode scanning and chmod in the trunk and 0.6 branches. For various reasons (including consistency of logging) I made somewhat more extensive revisions to chmod usage than was in your patch. I also implemented platform detection a bit differently. But the results should be the same. Please let me know if you have any further issues with setuptools on Jython. (I'm rather excited that it works there at all...) From pje at telecommunity.com Fri Jan 18 22:55:47 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 18 Jan 2008 16:55:47 -0500 Subject: [Distutils] Patch for minor easy_install problem In-Reply-To: <47042189.4020408@gmail.com> References: <47042189.4020408@gmail.com> Message-ID: <20080118215545.AB31A3A40A2@sparrow.telecommunity.com> At 04:11 PM 10/3/2007 -0700, Toshio Kuratomi wrote: >It looks like this fix to easy_isntall: >''' >0.6c5 > * Fixed .dll files on Cygwin not having executable > permisions when an egg is installed unzipped. >''' > >introduced a minor bug. From 0.6c5 on, installing an egg unzipped >makes all the files executable. > >Attaching a patch that only makes .dll's executable. > >-Toshio FYI, something similar is now implemented in the trunk and the 0.6 branch, although it is also applied to .so and .py files. From pje at telecommunity.com Sat Jan 19 03:57:05 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 18 Jan 2008 21:57:05 -0500 Subject: [Distutils] Package in both setup_requires and install_requires will not be installed In-Reply-To: References: Message-ID: <20080119025707.C3FF83A40A2@sparrow.telecommunity.com> At 11:23 PM 9/15/2007 +0100, Michael Hoffman wrote: >Joshua Boverhof previously reported this in >. > >If you specify a package in setup_requires it will be built in the >current directory. But even if it is in install_requires as well, it >won't be installed because the requirement is already satisfied at setup >time by the package in the build directory. > >Is there a workaround for this? It is now fixed in the trunk and 0.6 branch. I expect to be making a new release in a few weeks, at which time you'll be able to update your ez_setup.py version to require a version of setuptools that doesn't have this problem any more. From billiejoex at gmail.com Sat Jan 19 04:44:53 2008 From: billiejoex at gmail.com (Giampaolo Rodola') Date: Sat, 19 Jan 2008 04:44:53 +0100 Subject: [Distutils] pypi and easy_install In-Reply-To: <729626cc0801161556o26f06a18t44e283b9fc4bdf77@mail.gmail.com> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <729626cc0801151805w7f55649crfbb79901916405c6@mail.gmail.com> <729626cc0801151956j28c577d9g68e7586a3d124b72@mail.gmail.com> <20080116041709.26A4E3A4077@sparrow.telecommunity.com> <729626cc0801160646v4f910274t24ec15ce403afc5d@mail.gmail.com> <729626cc0801160648h4d764612qeb40692b260d7e5f@mail.gmail.com> <20080116172701.5141E3A40A2@sparrow.telecommunity.com> <729626cc0801160939s1c9563f2i58313d8c467f4bc8@mail.gmail.com> <20080116190209.CE3703A40A2@sparrow.telecommunity.com> <729626cc0801161556o26f06a18t44e283b9fc4bdf77@mail.gmail.com> Message-ID: <729626cc0801181944r2e305f0ap7d9b4c2ea4c5e09b@mail.gmail.com> Ok, I tried to implement this and it is seems I made it. The change permit to use the distutils "--formats=tar,gztar,bztar" option without need of having tar/gzip/bzip2 utilities installed on the system. Don't know if distutils test suite includes tests for the part of code I modified but Lib/test/test_distutils.py run successfully on Python 2.5.1. The only compression format currently left out is ztar. I may be wrong but I haven't seen modules able to work with that format in the Python stdlib. I'll wait for your review. Tell me if you want me to open a ticket on the Python bug tracker to let also other users review the patch, modify something or throw the whole thing away. :-) Giampaolo 2008/1/17, Giampaolo Rodola' : > 2008/1/16, Phillip J. Eby : > > At 06:39 PM 1/16/2008 +0100, Giampaolo Rodola' wrote: > > >2008/1/16, Phillip J. Eby : > > > > At 03:48 PM 1/16/2008 +0100, Giampaolo Rodola' wrote: > > > > > > > Unfortunately the distutils.archive_util module was written before > > > > > > > tarfile was available. > > > > > > > > > >Are there chances to see this implemented? > > > > > > > > Are you volunteering? :) If so, then perhaps you can get a patch in > > > > for Python 2.6. > > > > > >Uhm... why not? :-) > > >If someone could give me some pointers on nailing down the issue I > > >could give it a try (note: never seen distutils internals before). > > > > See the distutils.archive_util module - the disutils uses that for > > all its archive creation. > > > > > > >My only concern is that it's not rare that patches provided by users > > >are often ignored and they remains stagnant on the Python bug tracker. > > >I just wouldn't spend time on that for nothing. > > > > Well, I can review the patch and commit it, if I like it. :) > > Ok. I should find some time for trying to work on this this week. > I'll keep you updated. > > Thanks. > -------------- next part -------------- A non-text attachment was scrubbed... Name: archive_util.diff Type: application/octet-stream Size: 2706 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20080119/20f3a932/attachment-0001.obj -------------- next part -------------- A non-text attachment was scrubbed... Name: archive_util.py Type: text/x-python Size: 7329 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20080119/20f3a932/attachment-0001.py From b3i4old02 at sneakemail.com Sat Jan 19 12:04:44 2008 From: b3i4old02 at sneakemail.com (Michael Hoffman) Date: Sat, 19 Jan 2008 11:04:44 +0000 Subject: [Distutils] Package in both setup_requires and install_requires will not be installed In-Reply-To: <20080119025707.C3FF83A40A2@sparrow.telecommunity.com> References: <20080119025707.C3FF83A40A2@sparrow.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 11:23 PM 9/15/2007 +0100, Michael Hoffman wrote: >> If you specify a package in setup_requires it will be built in the >> current directory. But even if it is in install_requires as well, it >> won't be installed because the requirement is already satisfied at setup >> time by the package in the build directory. > > It is now fixed in the trunk and 0.6 branch. I expect to be making a > new release in a few weeks, at which time you'll be able to update > your ez_setup.py version to require a version of setuptools that > doesn't have this problem any more. Excellent. Thanks! From pje at telecommunity.com Sat Jan 19 19:46:58 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 19 Jan 2008 13:46:58 -0500 Subject: [Distutils] pypi and easy_install In-Reply-To: <729626cc0801181944r2e305f0ap7d9b4c2ea4c5e09b@mail.gmail.co m> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <729626cc0801151805w7f55649crfbb79901916405c6@mail.gmail.com> <729626cc0801151956j28c577d9g68e7586a3d124b72@mail.gmail.com> <20080116041709.26A4E3A4077@sparrow.telecommunity.com> <729626cc0801160646v4f910274t24ec15ce403afc5d@mail.gmail.com> <729626cc0801160648h4d764612qeb40692b260d7e5f@mail.gmail.com> <20080116172701.5141E3A40A2@sparrow.telecommunity.com> <729626cc0801160939s1c9563f2i58313d8c467f4bc8@mail.gmail.com> <20080116190209.CE3703A40A2@sparrow.telecommunity.com> <729626cc0801161556o26f06a18t44e283b9fc4bdf77@mail.gmail.com> <729626cc0801181944r2e305f0ap7d9b4c2ea4c5e09b@mail.gmail.com> Message-ID: <20080119184656.C28B53A4077@sparrow.telecommunity.com> At 04:44 AM 1/19/2008 +0100, Giampaolo Rodola' wrote: >Ok, I tried to implement this and it is seems I made it. >The change permit to use the distutils "--formats=tar,gztar,bztar" >option without need of having tar/gzip/bzip2 utilities installed on >the system. >Don't know if distutils test suite includes tests for the part of code >I modified but Lib/test/test_distutils.py run successfully on Python >2.5.1. > >The only compression format currently left out is ztar. >I may be wrong but I haven't seen modules able to work with that >format in the Python stdlib. > >I'll wait for your review. >Tell me if you want me to open a ticket on the Python bug tracker to >let also other users review the patch, modify something or throw the >whole thing away. :-) That would probably be a good idea. In the meantime, my only comment is that for consistency with make_zipfile, the code should work even if gzip, tarfile, or bz2 aren't importable. That is, falling back to the old spawn-based methods if the relevant library or libraries are not available. The comment at the top, after all, says "This module should be kept compatible with Python 2.1," and the tarfile and bz2 modules weren't added until 2.3. (I'm not sure when gzip was added.) From billiejoex at gmail.com Sat Jan 19 20:03:41 2008 From: billiejoex at gmail.com (Giampaolo Rodola') Date: Sat, 19 Jan 2008 20:03:41 +0100 Subject: [Distutils] pypi and easy_install In-Reply-To: <20080119184656.C28B53A4077@sparrow.telecommunity.com> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <20080116041709.26A4E3A4077@sparrow.telecommunity.com> <729626cc0801160646v4f910274t24ec15ce403afc5d@mail.gmail.com> <729626cc0801160648h4d764612qeb40692b260d7e5f@mail.gmail.com> <20080116172701.5141E3A40A2@sparrow.telecommunity.com> <729626cc0801160939s1c9563f2i58313d8c467f4bc8@mail.gmail.com> <20080116190209.CE3703A40A2@sparrow.telecommunity.com> <729626cc0801161556o26f06a18t44e283b9fc4bdf77@mail.gmail.com> <729626cc0801181944r2e305f0ap7d9b4c2ea4c5e09b@mail.gmail.com> <20080119184656.C28B53A4077@sparrow.telecommunity.com> Message-ID: <729626cc0801191103m5f408f41g30b0029c870427c9@mail.gmail.com> Ok then. I'll work on it as soon as possible and open a ticket on the bug tracker. 2008/1/19, Phillip J. Eby : > At 04:44 AM 1/19/2008 +0100, Giampaolo Rodola' wrote: > >Ok, I tried to implement this and it is seems I made it. > >The change permit to use the distutils "--formats=tar,gztar,bztar" > >option without need of having tar/gzip/bzip2 utilities installed on > >the system. > >Don't know if distutils test suite includes tests for the part of code > >I modified but Lib/test/test_distutils.py run successfully on Python > >2.5.1. > > > >The only compression format currently left out is ztar. > >I may be wrong but I haven't seen modules able to work with that > >format in the Python stdlib. > > > >I'll wait for your review. > >Tell me if you want me to open a ticket on the Python bug tracker to > >let also other users review the patch, modify something or throw the > >whole thing away. :-) > > That would probably be a good idea. In the meantime, my only comment > is that for consistency with make_zipfile, the code should work even > if gzip, tarfile, or bz2 aren't importable. That is, falling back to > the old spawn-based methods if the relevant library or libraries are > not available. The comment at the top, after all, says "This module > should be kept compatible with Python 2.1," and the tarfile and bz2 > modules weren't added until 2.3. (I'm not sure when gzip was added.) > > From pje at telecommunity.com Sat Jan 19 20:08:20 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 19 Jan 2008 14:08:20 -0500 Subject: [Distutils] pypi and easy_install In-Reply-To: <20080119184656.C28B53A4077@sparrow.telecommunity.com> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <729626cc0801151805w7f55649crfbb79901916405c6@mail.gmail.com> <729626cc0801151956j28c577d9g68e7586a3d124b72@mail.gmail.com> <20080116041709.26A4E3A4077@sparrow.telecommunity.com> <729626cc0801160646v4f910274t24ec15ce403afc5d@mail.gmail.com> <729626cc0801160648h4d764612qeb40692b260d7e5f@mail.gmail.com> <20080116172701.5141E3A40A2@sparrow.telecommunity.com> <729626cc0801160939s1c9563f2i58313d8c467f4bc8@mail.gmail.com> <20080116190209.CE3703A40A2@sparrow.telecommunity.com> <729626cc0801161556o26f06a18t44e283b9fc4bdf77@mail.gmail.com> <729626cc0801181944r2e305f0ap7d9b4c2ea4c5e09b@mail.gmail.com> <20080119184656.C28B53A4077@sparrow.telecommunity.com> Message-ID: <20080119190819.1AA2B3A4077@sparrow.telecommunity.com> At 01:46 PM 1/19/2008 -0500, Phillip J. Eby wrote: >At 04:44 AM 1/19/2008 +0100, Giampaolo Rodola' wrote: > >Ok, I tried to implement this and it is seems I made it. > >The change permit to use the distutils "--formats=tar,gztar,bztar" > >option without need of having tar/gzip/bzip2 utilities installed on > >the system. > >Don't know if distutils test suite includes tests for the part of code > >I modified but Lib/test/test_distutils.py run successfully on Python > >2.5.1. > > > >The only compression format currently left out is ztar. > >I may be wrong but I haven't seen modules able to work with that > >format in the Python stdlib. > > > >I'll wait for your review. > >Tell me if you want me to open a ticket on the Python bug tracker to > >let also other users review the patch, modify something or throw the > >whole thing away. :-) > >That would probably be a good idea. In the meantime, my only comment >is that for consistency with make_zipfile, the code should work even >if gzip, tarfile, or bz2 aren't importable. That is, falling back to >the old spawn-based methods if the relevant library or libraries are >not available. The comment at the top, after all, says "This module >should be kept compatible with Python 2.1," and the tarfile and bz2 >modules weren't added until 2.3. (I'm not sure when gzip was added.) Oh, and if tarfile is present, you can open a tarfile for writing with 'w:gz' or 'w:bz2' to have it apply the compression while building, which is probably more efficient. However, if the appropriate module isn't available, it'll raise a tarfile.CompressionError at creation time. So the code could be something like: try: import tarfile except ImportError: tarfile = None if tarfile is not None: ext = compress_ext.get(compress, '') try: tar = tarfile.open(archive_name+ext, "w"+ext.replace('.',':')) except tarfile.CompressionError: tar = tarfile.open(archive_name, "w") tar.add(base_dir) tar.close() if tar.name == archive_name+ext: return tar.name # we're done # ... fall through to old compression code else: # old spawn() code to create tar # everything from here on is just the old archive_util code... if compress: ... From billiejoex at gmail.com Sat Jan 19 20:10:22 2008 From: billiejoex at gmail.com (Giampaolo Rodola') Date: Sat, 19 Jan 2008 20:10:22 +0100 Subject: [Distutils] pypi and easy_install In-Reply-To: <729626cc0801191103m5f408f41g30b0029c870427c9@mail.gmail.com> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <729626cc0801160646v4f910274t24ec15ce403afc5d@mail.gmail.com> <729626cc0801160648h4d764612qeb40692b260d7e5f@mail.gmail.com> <20080116172701.5141E3A40A2@sparrow.telecommunity.com> <729626cc0801160939s1c9563f2i58313d8c467f4bc8@mail.gmail.com> <20080116190209.CE3703A40A2@sparrow.telecommunity.com> <729626cc0801161556o26f06a18t44e283b9fc4bdf77@mail.gmail.com> <729626cc0801181944r2e305f0ap7d9b4c2ea4c5e09b@mail.gmail.com> <20080119184656.C28B53A4077@sparrow.telecommunity.com> <729626cc0801191103m5f408f41g30b0029c870427c9@mail.gmail.com> Message-ID: <729626cc0801191110o57906041qfcb74b75466a559@mail.gmail.com> Just another thing. I was wondering what do you think would be better to do as default. Using Python modules as default and spawn() as fall back or doing the contrary? Maybe using spawn() as default would be better since original utilities should be faster than python modules. 2008/1/19, Giampaolo Rodola' : > Ok then. I'll work on it as soon as possible and open a ticket on the > bug tracker. > > 2008/1/19, Phillip J. Eby : > > At 04:44 AM 1/19/2008 +0100, Giampaolo Rodola' wrote: > > >Ok, I tried to implement this and it is seems I made it. > > >The change permit to use the distutils "--formats=tar,gztar,bztar" > > >option without need of having tar/gzip/bzip2 utilities installed on > > >the system. > > >Don't know if distutils test suite includes tests for the part of code > > >I modified but Lib/test/test_distutils.py run successfully on Python > > >2.5.1. > > > > > >The only compression format currently left out is ztar. > > >I may be wrong but I haven't seen modules able to work with that > > >format in the Python stdlib. > > > > > >I'll wait for your review. > > >Tell me if you want me to open a ticket on the Python bug tracker to > > >let also other users review the patch, modify something or throw the > > >whole thing away. :-) > > > > That would probably be a good idea. In the meantime, my only comment > > is that for consistency with make_zipfile, the code should work even > > if gzip, tarfile, or bz2 aren't importable. That is, falling back to > > the old spawn-based methods if the relevant library or libraries are > > not available. The comment at the top, after all, says "This module > > should be kept compatible with Python 2.1," and the tarfile and bz2 > > modules weren't added until 2.3. (I'm not sure when gzip was added.) > > > > > From pje at telecommunity.com Sat Jan 19 20:14:18 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 19 Jan 2008 14:14:18 -0500 Subject: [Distutils] pypi and easy_install In-Reply-To: <20080119190819.1AA2B3A4077@sparrow.telecommunity.com> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <729626cc0801151805w7f55649crfbb79901916405c6@mail.gmail.com> <729626cc0801151956j28c577d9g68e7586a3d124b72@mail.gmail.com> <20080116041709.26A4E3A4077@sparrow.telecommunity.com> <729626cc0801160646v4f910274t24ec15ce403afc5d@mail.gmail.com> <729626cc0801160648h4d764612qeb40692b260d7e5f@mail.gmail.com> <20080116172701.5141E3A40A2@sparrow.telecommunity.com> <729626cc0801160939s1c9563f2i58313d8c467f4bc8@mail.gmail.com> <20080116190209.CE3703A40A2@sparrow.telecommunity.com> <729626cc0801161556o26f06a18t44e283b9fc4bdf77@mail.gmail.com> <729626cc0801181944r2e305f0ap7d9b4c2ea4c5e09b@mail.gmail.com> <20080119184656.C28B53A4077@sparrow.telecommunity.com> <20080119190819.1AA2B3A4077@sparrow.telecommunity.com> Message-ID: <20080119191418.3CCD63A40A2@sparrow.telecommunity.com> At 02:08 PM 1/19/2008 -0500, Phillip J. Eby wrote: >Oh, and if tarfile is present, you can open a tarfile for writing >with 'w:gz' or 'w:bz2' to have it apply the compression while >building, which is probably more efficient. However, if the >appropriate module isn't available, it'll raise a >tarfile.CompressionError at creation time. So the code could be >something like: > > try: > import tarfile > except ImportError: > tarfile = None > > if tarfile is not None: > ext = compress_ext.get(compress, '') > try: > tar = tarfile.open(archive_name+ext, "w"+ext.replace('.',':')) > except tarfile.CompressionError: > tar = tarfile.open(archive_name, "w") > tar.add(base_dir) > tar.close() > if tar.name == archive_name+ext: > return tar.name # we're done Actually, this bit above is wrong, as it turns out tar.name will never include the extra extension. :( So, a flag is needed in place of using tar.name. From pje at telecommunity.com Sat Jan 19 20:16:15 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 19 Jan 2008 14:16:15 -0500 Subject: [Distutils] pypi and easy_install In-Reply-To: <729626cc0801191110o57906041qfcb74b75466a559@mail.gmail.com > References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <729626cc0801160646v4f910274t24ec15ce403afc5d@mail.gmail.com> <729626cc0801160648h4d764612qeb40692b260d7e5f@mail.gmail.com> <20080116172701.5141E3A40A2@sparrow.telecommunity.com> <729626cc0801160939s1c9563f2i58313d8c467f4bc8@mail.gmail.com> <20080116190209.CE3703A40A2@sparrow.telecommunity.com> <729626cc0801161556o26f06a18t44e283b9fc4bdf77@mail.gmail.com> <729626cc0801181944r2e305f0ap7d9b4c2ea4c5e09b@mail.gmail.com> <20080119184656.C28B53A4077@sparrow.telecommunity.com> <729626cc0801191103m5f408f41g30b0029c870427c9@mail.gmail.com> <729626cc0801191110o57906041qfcb74b75466a559@mail.gmail.com> Message-ID: <20080119191614.240893A4077@sparrow.telecommunity.com> At 08:10 PM 1/19/2008 +0100, Giampaolo Rodola' wrote: >Just another thing. >I was wondering what do you think would be better to do as default. >Using Python modules as default and spawn() as fall back or doing >the contrary? >Maybe using spawn() as default would be better since original >utilities should be faster than python modules. I suppose that's a possibility, although the tarfile module doing the compression might be faster than separate tar and compress steps. The other argument for using spawn() first would be that it's the least invasive and most backward-compatible change. From billiejoex at gmail.com Sat Jan 19 20:20:38 2008 From: billiejoex at gmail.com (Giampaolo Rodola') Date: Sat, 19 Jan 2008 20:20:38 +0100 Subject: [Distutils] pypi and easy_install In-Reply-To: <20080119191614.240893A4077@sparrow.telecommunity.com> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <20080116172701.5141E3A40A2@sparrow.telecommunity.com> <729626cc0801160939s1c9563f2i58313d8c467f4bc8@mail.gmail.com> <20080116190209.CE3703A40A2@sparrow.telecommunity.com> <729626cc0801161556o26f06a18t44e283b9fc4bdf77@mail.gmail.com> <729626cc0801181944r2e305f0ap7d9b4c2ea4c5e09b@mail.gmail.com> <20080119184656.C28B53A4077@sparrow.telecommunity.com> <729626cc0801191103m5f408f41g30b0029c870427c9@mail.gmail.com> <729626cc0801191110o57906041qfcb74b75466a559@mail.gmail.com> <20080119191614.240893A4077@sparrow.telecommunity.com> Message-ID: <729626cc0801191120y4ef5920fn967ab48830122cb4@mail.gmail.com> > The other argument for using spawn() first would be that it's the > least invasive and most backward-compatible change. Yeah, I agree. It would be probably better using spawn() as default. 2008/1/19, Phillip J. Eby : > At 08:10 PM 1/19/2008 +0100, Giampaolo Rodola' wrote: > >Just another thing. > >I was wondering what do you think would be better to do as default. > >Using Python modules as default and spawn() as fall back or doing > >the contrary? > >Maybe using spawn() as default would be better since original > >utilities should be faster than python modules. > > I suppose that's a possibility, although the tarfile module doing the > compression might be faster than separate tar and compress steps. > > The other argument for using spawn() first would be that it's the > least invasive and most backward-compatible change. > > From billiejoex at gmail.com Sat Jan 19 21:21:32 2008 From: billiejoex at gmail.com (Giampaolo Rodola') Date: Sat, 19 Jan 2008 21:21:32 +0100 Subject: [Distutils] pypi and easy_install In-Reply-To: <729626cc0801191120y4ef5920fn967ab48830122cb4@mail.gmail.com> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <729626cc0801160939s1c9563f2i58313d8c467f4bc8@mail.gmail.com> <20080116190209.CE3703A40A2@sparrow.telecommunity.com> <729626cc0801161556o26f06a18t44e283b9fc4bdf77@mail.gmail.com> <729626cc0801181944r2e305f0ap7d9b4c2ea4c5e09b@mail.gmail.com> <20080119184656.C28B53A4077@sparrow.telecommunity.com> <729626cc0801191103m5f408f41g30b0029c870427c9@mail.gmail.com> <729626cc0801191110o57906041qfcb74b75466a559@mail.gmail.com> <20080119191614.240893A4077@sparrow.telecommunity.com> <729626cc0801191120y4ef5920fn967ab48830122cb4@mail.gmail.com> Message-ID: <729626cc0801191221pea32eb6ybc4d4bf142cfbb0@mail.gmail.com> As I suspected there's a bug in the current distutils code. Steps to reproduce the problem: 1: Run "python setup.py sdist --formats=gztar,tar". 2. dist directory now contains "module.gztar" and "module.tar" files as expected. 3: Run "python setup.py sdist --formats=tar,gztar". 4. dist directory now contains only "module.gztar" file because "module.tar" has been overwritten when archive_util.make_tarball() function was called for the second time. I'll try to fix this while I'm working on the other issue. 2008/1/19, Giampaolo Rodola' : > > The other argument for using spawn() first would be that it's the > > least invasive and most backward-compatible change. > > Yeah, I agree. It would be probably better using spawn() as default. > > 2008/1/19, Phillip J. Eby : > > At 08:10 PM 1/19/2008 +0100, Giampaolo Rodola' wrote: > > >Just another thing. > > >I was wondering what do you think would be better to do as default. > > >Using Python modules as default and spawn() as fall back or doing > > >the contrary? > > >Maybe using spawn() as default would be better since original > > >utilities should be faster than python modules. > > > > I suppose that's a possibility, although the tarfile module doing the > > compression might be faster than separate tar and compress steps. > > > > The other argument for using spawn() first would be that it's the > > least invasive and most backward-compatible change. > > > > > From pje at telecommunity.com Sun Jan 20 00:45:01 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 19 Jan 2008 18:45:01 -0500 Subject: [Distutils] pypi and easy_install In-Reply-To: <729626cc0801191221pea32eb6ybc4d4bf142cfbb0@mail.gmail.com> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <729626cc0801160939s1c9563f2i58313d8c467f4bc8@mail.gmail.com> <20080116190209.CE3703A40A2@sparrow.telecommunity.com> <729626cc0801161556o26f06a18t44e283b9fc4bdf77@mail.gmail.com> <729626cc0801181944r2e305f0ap7d9b4c2ea4c5e09b@mail.gmail.com> <20080119184656.C28B53A4077@sparrow.telecommunity.com> <729626cc0801191103m5f408f41g30b0029c870427c9@mail.gmail.com> <729626cc0801191110o57906041qfcb74b75466a559@mail.gmail.com> <20080119191614.240893A4077@sparrow.telecommunity.com> <729626cc0801191120y4ef5920fn967ab48830122cb4@mail.gmail.com> <729626cc0801191221pea32eb6ybc4d4bf142cfbb0@mail.gmail.com> Message-ID: <20080119234500.377083A4077@sparrow.telecommunity.com> At 09:21 PM 1/19/2008 +0100, Giampaolo Rodola' wrote: >As I suspected there's a bug in the current distutils code. >Steps to reproduce the problem: > >1: Run "python setup.py sdist --formats=gztar,tar". >2. dist directory now contains "module.gztar" and "module.tar" files >as expected. >3: Run "python setup.py sdist --formats=tar,gztar". >4. dist directory now contains only "module.gztar" file because >"module.tar" has been overwritten when archive_util.make_tarball() >function was called for the second time. > >I'll try to fix this while I'm working on the other issue. Well, using the integrated compression in the tarfile module would certainly fix it for the case where tarfile is being used. :) It actually seems like a good argument for having tarfile as the primary generation method, with spawning as the backup. From pjenvey at groovie.org Sun Jan 20 02:19:34 2008 From: pjenvey at groovie.org (Philip Jenvey) Date: Sat, 19 Jan 2008 17:19:34 -0800 Subject: [Distutils] setuptools on jython In-Reply-To: <20080118215435.848E03A40A2@sparrow.telecommunity.com> References: <36A10A2D-4103-45D5-BBCF-E8DE60B49880@groovie.org> <20080112045144.439313A40A2@sparrow.telecommunity.com> <123A3D33-41DE-4837-907F-FB062279E4D5@groovie.org> <20080118215435.848E03A40A2@sparrow.telecommunity.com> Message-ID: <30334F78-93E4-4B23-8F2D-D2733A012D61@groovie.org> On Jan 18, 2008, at 1:54 PM, Phillip J. Eby wrote: > > I've now incorporated comparable changes for bytecode scanning and > chmod in the trunk and 0.6 branches. For various reasons > (including consistency of logging) I made somewhat more extensive > revisions to chmod usage than was in your patch. I also > implemented platform detection a bit differently. But the results > should be the same. Please let me know if you have any further > issues with setuptools on Jython. (I'm rather excited that it > works there at all...) > Your patch looks good, thanks! -- Philip Jenvey From ronaldoussoren at mac.com Sun Jan 20 11:52:03 2008 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sun, 20 Jan 2008 11:52:03 +0100 Subject: [Distutils] Experience of setuptools' cache design In-Reply-To: <680A348C-518A-4491-B3AF-8F75A4F2E2F1@zope.com> References: <680A348C-518A-4491-B3AF-8F75A4F2E2F1@zope.com> Message-ID: <73A46E0E-C69C-4E3E-B9E2-61DE47D3BC1C@mac.com> On 18 Jan, 2008, at 16:06, Jim Fulton wrote: > > > IMO, any package including extensions should be treated as zip- > unsafe. +1 there. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20080120/dc85c3fe/attachment.bin From billiejoex at gmail.com Mon Jan 21 22:32:50 2008 From: billiejoex at gmail.com (Giampaolo Rodola') Date: Mon, 21 Jan 2008 22:32:50 +0100 Subject: [Distutils] pypi and easy_install In-Reply-To: <20080119234500.377083A4077@sparrow.telecommunity.com> References: <729626cc0712191709g769da316nd0b177455b0c813f@mail.gmail.com> <729626cc0801161556o26f06a18t44e283b9fc4bdf77@mail.gmail.com> <729626cc0801181944r2e305f0ap7d9b4c2ea4c5e09b@mail.gmail.com> <20080119184656.C28B53A4077@sparrow.telecommunity.com> <729626cc0801191103m5f408f41g30b0029c870427c9@mail.gmail.com> <729626cc0801191110o57906041qfcb74b75466a559@mail.gmail.com> <20080119191614.240893A4077@sparrow.telecommunity.com> <729626cc0801191120y4ef5920fn967ab48830122cb4@mail.gmail.com> <729626cc0801191221pea32eb6ybc4d4bf142cfbb0@mail.gmail.com> <20080119234500.377083A4077@sparrow.telecommunity.com> Message-ID: <729626cc0801211332g413febd8o86e97e2432d66c44@mail.gmail.com> I provided a patch for both issues which are now reported on the bug tracker: http://bugs.python.org/issue1885 http://bugs.python.org/issue1886 2008/1/20, Phillip J. Eby : > At 09:21 PM 1/19/2008 +0100, Giampaolo Rodola' wrote: > >As I suspected there's a bug in the current distutils code. > >Steps to reproduce the problem: > > > >1: Run "python setup.py sdist --formats=gztar,tar". > >2. dist directory now contains "module.gztar" and "module.tar" files > >as expected. > >3: Run "python setup.py sdist --formats=tar,gztar". > >4. dist directory now contains only "module.gztar" file because > >"module.tar" has been overwritten when archive_util.make_tarball() > >function was called for the second time. > > > >I'll try to fix this while I'm working on the other issue. > > Well, using the integrated compression in the tarfile module would > certainly fix it for the case where tarfile is being used. :) It > actually seems like a good argument for having tarfile as the primary > generation method, with spawning as the backup. > > From zooko at zooko.com Mon Jan 21 21:05:34 2008 From: zooko at zooko.com (zooko) Date: Mon, 21 Jan 2008 13:05:34 -0700 Subject: [Distutils] wish: a setuptools plugin that does freshmeat-submit Message-ID: Folks: Freshmeat.net now offers an XML-RPC API to submit project metadata and release announcements. There is a client implementation by ESR in Python. http://freshmeat.net/articles/view/1048/ My wish is that I could install a setuptools plugin, and subsequently do "./setup.py upload freshmeat-submit" to announce new versions to both PyPI and freshmeat.net. Regards, Zooko From ziade.tarek at gmail.com Mon Jan 21 23:35:28 2008 From: ziade.tarek at gmail.com (=?UTF-8?Q?Tarek_Ziad=C3=A9?=) Date: Mon, 21 Jan 2008 14:35:28 -0800 (PST) Subject: [Distutils] multiple servers handling in .pypirc In-Reply-To: <20080117163359.GB97097@lit.jwp.name> References: <14917917.post@talk.nabble.com> <20080117163359.GB97097@lit.jwp.name> Message-ID: <15008218.post@talk.nabble.com> James William Pye wrote: > > In the EnhancedPyPI page, the .pypirc file starts to take on two roles: a > keychain and a list of index servers. Perhaps it would be best to keep > these things separate. A global file(and a user relative file) listing > index > servers or an environment variable would be handy, IMO. > Yeas maybe we could look in the user's home then in some global place. but how this could be done so it works on all platforms ? I can think of /etc in Linux, but what about Windows, is there a common pattern we can use ? James William Pye wrote: > > Also, it would be > preferable if we could just use the > URL/URI/IRI syntax: http://user:pass at example.com/pypi > I am not found of this because for the default server (pypi) you will need to provide explicit username/password if you want to omit the PyPI url James William Pye wrote: > > just my random thoughts as a user. nice work regardless of how you take my > preferences. ;) > Well thanks a lot. At this point I have added a patch proposal, but it can still be enhanced. I will really like to see it in 2.6 because by that time, there will be many PyPI-compatible web sites -- View this message in context: http://www.nabble.com/multiple-servers-handling-in-.pypirc-tp14917917p15008218.html Sent from the Python - distutils-sig mailing list archive at Nabble.com. From pje at telecommunity.com Tue Jan 22 00:01:05 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 21 Jan 2008 18:01:05 -0500 Subject: [Distutils] wish: a setuptools plugin that does freshmeat-submit In-Reply-To: References: Message-ID: <20080121230103.DBAD33A40A2@sparrow.telecommunity.com> At 01:05 PM 1/21/2008 -0700, zooko wrote: >Folks: > >Freshmeat.net now offers an XML-RPC API to submit project metadata >and release announcements. > >There is a client implementation by ESR in Python. > >http://freshmeat.net/articles/view/1048/ > >My wish is that I could install a setuptools plugin, and subsequently >do "./setup.py upload freshmeat-submit" to announce new versions to >both PyPI and freshmeat.net. So what's stopping you? :) http://peak.telecommunity.com/DevCenter/setuptools#adding-commands From python at jwp.name Tue Jan 22 04:08:49 2008 From: python at jwp.name (James William Pye) Date: Mon, 21 Jan 2008 20:08:49 -0700 Subject: [Distutils] multiple servers handling in .pypirc In-Reply-To: <15008218.post@talk.nabble.com> References: <14917917.post@talk.nabble.com> <20080117163359.GB97097@lit.jwp.name> <15008218.post@talk.nabble.com> Message-ID: <20080122030849.GA1111@lit.jwp.name> On Mon, Jan 21, 2008 at 02:35:28PM -0800, Tarek Ziad? wrote: > Yeas maybe we could look in the user's home then in some global > place. but how this could be done so it works on all platforms ? > I can think of /etc in Linux, but what about Windows, is there > a common pattern we can use ? Not sure. Perhaps someone more familiar with windows can speak up about this. > > URL/URI/IRI syntax: http://user:pass at example.com/pypi > > I am not found of this because for the default server (pypi) you will need > to provide explicit username/password if you want to omit the PyPI url Granted, the built-in default case would be strange, but, arguably, you wouldn't need to define the default pypi creds in that form as the current .pypirc already handles that case. *shrug*. From mhammond at skippinet.com.au Tue Jan 22 11:05:57 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Tue, 22 Jan 2008 21:05:57 +1100 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <46512B09.1080304@v.loewis.de> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> Message-ID: <022401c85cde$678bcf10$36a36d30$@com.au> Hi Martin, Way back on Monday, 21 May 2007, you wrote: > This is an issue to be discussed for Python 2.6. I'm personally > hesitant to have the "official" build infrastructure deviate > from the layout that has been in-use for so many years, as a lot > of things depend on it. > > I don't find the need to have separate object directories convincing: > For building the Win32/Win64 binaries, I have separate checkouts > *anyway*, since all the add-on libraries would have to support > multi-arch builds, but I think they don't. ... > > * Re the x64 directory used by the PCBuild8 process. IMO, it makes > > sense to generate x64 binaries to their own directory - my expectation > > is that cross-compiling between platforms is a reasonable use-case, > > and we should support multiple achitectures for the same compiler > > version. > > See above; I disagree. [For additional context; the question was regarding where the output binaries were created for each platform, and specifically if x64 build should use something like 'PCBuild/x64'] I'm bringing this topic up again as I noticed the AMD64 builds for Python 2.6 create their output in the PCBuild/x64 directory, not directly into the PCBuild directory. When I raised this with Christian, he also expressed a desire to be able to cross-compile in the same directory structure and was unaware of this discussion (but I made him aware of it :) You made excellent points about how tools currently recognize the PCBuild directory, and we can't break them, and I agree. I'd like to propose a compromise that might be able to keep everyone happy. The main problem I see with all platforms using 'PCBuild' is that in a cross-compilation scenario, it is impossible for the distutils to determine the location of the correct Python libraries to link with (stuff in its own PYTHONHOME is no good; it's for the wrong platform). Obviously, we can change the distutils so that the location of the correct libraries can be specified, but that implies all other build systems that currently assume PCBuild must also change to work in a cross-compilation scenario. While those external tools *will* work today in a native build on x64, eventually they may need to be upgraded to deal with cross-compilation - and this means that all such tools will also be unable to automatically determine the location of the libraries. They will all need to take the library dir as a user-specified option, which seems a pain (eg, buildbots would seem to need site-specific configuration information to make this work). If eventually all such tools are likely to upgrade, it would be ideal if we can offer them a way to upgrade so the correct libraries can be determined automatically, as they can now for native builds. My compromise proposal is this: Each platform builds in its own directory (ie, PCBuild/x86_64, PCBuild/x86). Every single build configuration has a post-build step that copies the result of the build to the PCBuild directory. We then add an option to the distutils so that a cross-compile platform can be specified and when it is, to use 'PCBuild/{platform}" instead of PCBuild, and we encourage other tools to use the same directory should they want to support "configure-less" cross-compilation (if distutils on the trunk is not still trying to maintain b/w compat, it should just *always* look in PCBuild/{platform}, and I'd suggest py3k not bother with PCBuild at all; build systems will be touched to upgrade to py3k, so that change isn't painful. But I digress :) The main advantage of the compromise is that it allows for the status quo to remain in place - just continue to use separate source trees for each platform, and only ever build a single platform in each tree, and tools are free to have ways of specifying which directory should be used. The official build process need not change. However, it also supports a way to do cross-compilation in a way that build tools can *automatically* locate the correct platform's libraries, and this will be particularly useful for extension authors. Would something like that be acceptable? Thanks, Mark From zooko at zooko.com Tue Jan 22 17:35:19 2008 From: zooko at zooko.com (zooko) Date: Tue, 22 Jan 2008 09:35:19 -0700 Subject: [Distutils] Python version numbers and jython, IronPython, PyPy, and ShedSkin (was: setuptools on jython) In-Reply-To: <30334F78-93E4-4B23-8F2D-D2733A012D61@groovie.org> References: <36A10A2D-4103-45D5-BBCF-E8DE60B49880@groovie.org> <20080112045144.439313A40A2@sparrow.telecommunity.com> <123A3D33-41DE-4837-907F-FB062279E4D5@groovie.org> <20080118215435.848E03A40A2@sparrow.telecommunity.com> <30334F78-93E4-4B23-8F2D-D2733A012D61@groovie.org> Message-ID: <17449818-67BB-4B25-9D85-8B7D42077228@zooko.com> If setuptools works on jython, what does it use for the "pyX.Y" part of the egg names? And in the future hopefully IronPython, PyPy, and ShedSkin will join the setuptools party. Does this revive our interest in supporting "100% Pure Python" eggs that do not have an X.Y Python version number in their names? (Since Python code without native extension modules is a lot more likely to work on various versions of various implementations of Python than Python code with native extension modules.) Or will jython, IronPython, PyPy, and ShedSkin specify some X.Y version of CPython that they support? Regards, Zooko From tseaver at palladion.com Tue Jan 22 18:13:02 2008 From: tseaver at palladion.com (Tres Seaver) Date: Tue, 22 Jan 2008 12:13:02 -0500 Subject: [Distutils] Python version numbers and jython, IronPython, PyPy, and ShedSkin In-Reply-To: <17449818-67BB-4B25-9D85-8B7D42077228@zooko.com> References: <36A10A2D-4103-45D5-BBCF-E8DE60B49880@groovie.org> <20080112045144.439313A40A2@sparrow.telecommunity.com> <123A3D33-41DE-4837-907F-FB062279E4D5@groovie.org> <20080118215435.848E03A40A2@sparrow.telecommunity.com> <30334F78-93E4-4B23-8F2D-D2733A012D61@groovie.org> <17449818-67BB-4B25-9D85-8B7D42077228@zooko.com> Message-ID: <4796241E.8010106@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 zooko wrote: > If setuptools works on jython, what does it use for the "pyX.Y" part > of the egg names? > > And in the future hopefully IronPython, PyPy, and ShedSkin will join > the setuptools party. > > Does this revive our interest in supporting "100% Pure Python" eggs > that do not have an X.Y Python version number in their names? (Since > Python code without native extension modules is a lot more likely to > work on various versions of various implementations of Python than > Python code with native extension modules.) What is the advantage of such a 'noarch' egg over just distributing 'sdist' tarballs (AKA "source eggs")? Setuptools / easy_install is happy to consume them, and I would much rather work with them. > Or will jython, IronPython, PyPy, and ShedSkin specify some X.Y > version of CPython that they support? I can't imagine any of the three being able to use non-source eggs compiled for CPython. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHliQe+gerLs4ltQ4RAqw3AJwPCwlCe41m9M6GR8ih7IYORWhkZQCgliAh SRchfLIlsDEbwrqhbrEpS2Y= =l0oA -----END PGP SIGNATURE----- From zooko at zooko.com Tue Jan 22 18:27:50 2008 From: zooko at zooko.com (zooko) Date: Tue, 22 Jan 2008 10:27:50 -0700 Subject: [Distutils] Python version numbers and jython, IronPython, PyPy, and ShedSkin In-Reply-To: <4796241E.8010106@palladion.com> References: <36A10A2D-4103-45D5-BBCF-E8DE60B49880@groovie.org> <20080112045144.439313A40A2@sparrow.telecommunity.com> <123A3D33-41DE-4837-907F-FB062279E4D5@groovie.org> <20080118215435.848E03A40A2@sparrow.telecommunity.com> <30334F78-93E4-4B23-8F2D-D2733A012D61@groovie.org> <17449818-67BB-4B25-9D85-8B7D42077228@zooko.com> <4796241E.8010106@palladion.com> Message-ID: On Jan 22, 2008, at 10:13 AM, Tres Seaver wrote: > What is the advantage of such a 'noarch' egg over just distributing > 'sdist' tarballs (AKA "source eggs")? Setuptools / easy_install is > happy to consume them, and I would much rather work with them. That's an excellent point. Honestly, the only reason I ever notice this arch question is because the setuptools egg itself (for bootstrapping with ez_setup.py) has a pyX.Y in it, and this makes it cost an extra 600 KB if I want to bundle setuptools with my app. I can solve this easily enough by hacking my copy of ez_setup.py to load a setuptools egg without an X.Y in its name, though... ;-) So I withdraw my interest in no-arch eggs, because you've reminded me that I strongly prefer source eggs. Regards, Zooko From ziade.tarek at gmail.com Tue Jan 22 23:03:56 2008 From: ziade.tarek at gmail.com (=?UTF-8?Q?Tarek_Ziad=C3=A9?=) Date: Tue, 22 Jan 2008 14:03:56 -0800 (PST) Subject: [Distutils] multiple servers handling in .pypirc In-Reply-To: <20080122030849.GA1111@lit.jwp.name> References: <14917917.post@talk.nabble.com> <20080117163359.GB97097@lit.jwp.name> <15008218.post@talk.nabble.com> <20080122030849.GA1111@lit.jwp.name> Message-ID: <15029591.post@talk.nabble.com> James William Pye wrote: > > Granted, the built-in default case would be strange, but, arguably, you > wouldn't need to define the default pypi creds in that form as the current > .pypirc already handles that case. *shrug*. > Yes but that would end up with a file that has two syntax because you need to provide the username and password for pypi somewhere -- View this message in context: http://www.nabble.com/multiple-servers-handling-in-.pypirc-tp14917917p15029591.html Sent from the Python - distutils-sig mailing list archive at Nabble.com. From matt at matt-good.net Wed Jan 23 03:32:08 2008 From: matt at matt-good.net (Matt Good) Date: Tue, 22 Jan 2008 18:32:08 -0800 Subject: [Distutils] Python version numbers and jython, IronPython, PyPy, and ShedSkin (was: setuptools on jython) In-Reply-To: <17449818-67BB-4B25-9D85-8B7D42077228@zooko.com> References: <36A10A2D-4103-45D5-BBCF-E8DE60B49880@groovie.org> <20080112045144.439313A40A2@sparrow.telecommunity.com> <123A3D33-41DE-4837-907F-FB062279E4D5@groovie.org> <20080118215435.848E03A40A2@sparrow.telecommunity.com> <30334F78-93E4-4B23-8F2D-D2733A012D61@groovie.org> <17449818-67BB-4B25-9D85-8B7D42077228@zooko.com> Message-ID: <99EDEF0B-2888-4B39-A8C2-DE64BB537553@matt-good.net> On Jan 22, 2008, at 8:35 AM, zooko wrote: > If setuptools works on jython, what does it use for the "pyX.Y" part > of the egg names? I would assume/hope that it uses py2.2 since that's the Python version they're currently compatible with. > Or will jython, IronPython, PyPy, and ShedSkin specify some X.Y > version of CPython that they support? Jython does already: $ jython Jython 2.2a0 on java1.5.0_13 (JIT: null) >>> import sys >>> sys.version_info (2, 2, 0, 'alpha', 0) They should use the CPython version numbers to indicate what version of the language specification they implement. If they support compiled platform-specific extensions in eggs they could append an identifier like the CPython eggs do now for compiled extensions (e.g. Foo-1.1-py2.2-jython). -- Matt From pjenvey at groovie.org Thu Jan 24 21:12:57 2008 From: pjenvey at groovie.org (Philip Jenvey) Date: Thu, 24 Jan 2008 12:12:57 -0800 Subject: [Distutils] Python version numbers and jython, IronPython, PyPy, and ShedSkin (was: setuptools on jython) In-Reply-To: <17449818-67BB-4B25-9D85-8B7D42077228@zooko.com> References: <36A10A2D-4103-45D5-BBCF-E8DE60B49880@groovie.org> <20080112045144.439313A40A2@sparrow.telecommunity.com> <123A3D33-41DE-4837-907F-FB062279E4D5@groovie.org> <20080118215435.848E03A40A2@sparrow.telecommunity.com> <30334F78-93E4-4B23-8F2D-D2733A012D61@groovie.org> <17449818-67BB-4B25-9D85-8B7D42077228@zooko.com> Message-ID: On Jan 22, 2008, at 8:35 AM, zooko wrote: > If setuptools works on jython, what does it use for the "pyX.Y" > part of the egg names? > > And in the future hopefully IronPython, PyPy, and ShedSkin will > join the setuptools party. > > Does this revive our interest in supporting "100% Pure Python" eggs > that do not have an X.Y Python version number in their names? > (Since Python code without native extension modules is a lot more > likely to work on various versions of various implementations of > Python than Python code with native extension modules.) > > Or will jython, IronPython, PyPy, and ShedSkin specify some X.Y > version of CPython that they support? > You bring up a good point actually; setuptools bdist_egg hardcodes pyX.Y for egg filenames, so Jython currently creates eggs tagged as - py2.3. This is going to be odd because eggs created from Jython don't contain .pyc files, instead they contain Jython $py.class files. In the case of a zip_safe=False egg, easy_install actually re- compiles the eggs' .py files to byte code during their installation. So a zip_safe=False egg created from Jython would end up with .py, .pyc and $py.class when installed on CPython. Though it looks like no byte compiling is done for zip_safe=True eggs. These eggs aren't extracted and recompiled, they're just installed as is. In that case an egg created from Jython would end up installed on CPython with no .pyc files. CPython would fallback to using the .py file in that case, but that's not ideal. Is it worth it to use a different filename pattern than pyX.Y on Jython? -- Philip Jenvey From david at ar.media.kyoto-u.ac.jp Fri Jan 25 07:06:17 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Fri, 25 Jan 2008 15:06:17 +0900 Subject: [Distutils] Uploading an egg behind a proxy Message-ID: <47997C59.8080409@ar.media.kyoto-u.ac.jp> Hi, I would like to upload an egg to the cheeseshop, but I have a problem because my network is behind a proxy. My configuration is unusual in the sense that my network has no DNS server, and everything is done through the proxy: proxy has to be used to access an url: Submitting dist/numscons-0.3.1-py2.5.egg to http://www.python.org/pypi (-2, 'Name or service not known') I know for sure I can access pypi, and that this is a proxy related problem (other python projects relying on urllib2 had similar problems, e.g. bzr). cheers, David From david at ar.media.kyoto-u.ac.jp Fri Jan 25 07:27:38 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Fri, 25 Jan 2008 15:27:38 +0900 Subject: [Distutils] How to make an egg easy_installable ? Message-ID: <4799815A.6010100@ar.media.kyoto-u.ac.jp> Hi, I wanted to make an egg from my package, so that people install it with easy_install. I managed to make an egg, which works fine, but I have problems with the cheeseshop, and how easy_install retrieve the egg. First, I uploaded the egg using register and upload command, and I can see the egg on the cheeseshop. But when trying to use easy_install, it does not work: it tries to look for packages in at the URL I gave in url argument, and do not take the egg from the cheeseshop. As a temporary "fix", I removed url and url_download arguments, and re-upload my package: this time, easy_install get the egg, but fails installing it: Creating /usr/media/misc/local/lib/python2.5/site-packages/site.py Searching for numscons Reading http://cheeseshop.python.org/pypi/numscons/ Reading http://cheeseshop.python.org/pypi/numscons/0.3.1 Best match: numscons 0.3.1 Downloading http://pypi.python.org/packages/2.5/n/numscons/numscons-0.3.1-py2.5.egg#md5=081b0cfc46d0571b85020a289c7b3c2d Processing numscons-0.3.1-py2.5.egg creating /usr/media/misc/local/lib/python2.5/site-packages/numscons-0.3.1-py2.5.egg Extracting numscons-0.3.1-py2.5.egg to /usr/media/misc/local/lib/python2.5/site-packages Traceback (most recent call last): File "/usr/bin/easy_install", line 8, in load_entry_point('setuptools==0.6c6', 'console_scripts', 'easy_install')() File "/usr/lib/python2.5/site-packages/setuptools/command/easy_install.py", line 1670, in main with_ei_usage(lambda: File "/usr/lib/python2.5/site-packages/setuptools/command/easy_install.py", line 1659, in with_ei_usage return f() File "/usr/lib/python2.5/site-packages/setuptools/command/easy_install.py", line 1674, in distclass=DistributionWithoutHelpCommands, **kw File "distutils/core.py", line 151, in setup File "distutils/dist.py", line 974, in run_commands File "distutils/dist.py", line 994, in run_command File "/usr/lib/python2.5/site-packages/setuptools/command/easy_install.py", line 211, in run self.easy_install(spec, not self.no_deps) File "/usr/lib/python2.5/site-packages/setuptools/command/easy_install.py", line 446, in easy_install return self.install_item(spec, dist.location, tmpdir, deps) File "/usr/lib/python2.5/site-packages/setuptools/command/easy_install.py", line 471, in install_item dists = self.install_eggs(spec, download, tmpdir) File "/usr/lib/python2.5/site-packages/setuptools/command/easy_install.py", line 619, in install_eggs return [self.install_egg(dist_filename, tmpdir)] File "/usr/lib/python2.5/site-packages/setuptools/command/easy_install.py", line 696, in install_egg return self.egg_distribution(destination) File "/usr/lib/python2.5/site-packages/setuptools/command/easy_install.py", line 661, in egg_distribution metadata = EggMetadata(zipimport.zipimporter(egg_path)) zipimport.ZipImportError: not a Zip file And at this point, I am lost :) cheers, David From lists at zopyx.com Fri Jan 25 07:53:07 2008 From: lists at zopyx.com (Andreas Jung) Date: Fri, 25 Jan 2008 07:53:07 +0100 Subject: [Distutils] Uploading an egg behind a proxy In-Reply-To: <47997C59.8080409@ar.media.kyoto-u.ac.jp> References: <47997C59.8080409@ar.media.kyoto-u.ac.jp> Message-ID: <67F0E8DEDCA4AA0C49F0B76C@[192.168.0.24]> --On 25. Januar 2008 15:06:17 +0900 David Cournapeau wrote: > Hi, > > I would like to upload an egg to the cheeseshop, but I have a > problem because my network is behind a proxy. My configuration is > unusual in the sense that my network has no DNS server, and everything > is done through the proxy: proxy has to be used to access an url: > > Submitting dist/numscons-0.3.1-py2.5.egg to http://www.python.org/pypi > (-2, 'Name or service not known') > > I know for sure I can access pypi, and that this is a proxy related > problem (other python projects relying on urllib2 had similar problems, > e.g. bzr). > Not an answer but you can upload your files to PyPI manually using the web interface. Andreas -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 186 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20080125/55e3a442/attachment.pgp From david at ar.media.kyoto-u.ac.jp Sat Jan 26 12:52:09 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Sat, 26 Jan 2008 20:52:09 +0900 Subject: [Distutils] Uploading an egg behind a proxy In-Reply-To: <67F0E8DEDCA4AA0C49F0B76C@[192.168.0.24]> References: <47997C59.8080409@ar.media.kyoto-u.ac.jp> <67F0E8DEDCA4AA0C49F0B76C@[192.168.0.24]> Message-ID: <479B1EE9.7060101@ar.media.kyoto-u.ac.jp> Andreas Jung wrote: > > > --On 25. Januar 2008 15:06:17 +0900 David Cournapeau > wrote: > >> Hi, >> >> I would like to upload an egg to the cheeseshop, but I have a >> problem because my network is behind a proxy. My configuration is >> unusual in the sense that my network has no DNS server, and everything >> is done through the proxy: proxy has to be used to access an url: >> >> Submitting dist/numscons-0.3.1-py2.5.egg to http://www.python.org/pypi >> (-2, 'Name or service not known') >> >> I know for sure I can access pypi, and that this is a proxy related >> problem (other python projects relying on urllib2 had similar problems, >> e.g. bzr). >> > Not an answer but you can upload your files to PyPI manually using the > web > interface. > > Andreas Yes, sure :) This and ssh tunneling work, but that's not ideal. thanks, David From python at jwp.name Sat Jan 26 20:35:21 2008 From: python at jwp.name (James William Pye) Date: Sat, 26 Jan 2008 12:35:21 -0700 Subject: [Distutils] Uploading an egg behind a proxy In-Reply-To: <479B1EE9.7060101@ar.media.kyoto-u.ac.jp> References: <47997C59.8080409@ar.media.kyoto-u.ac.jp> <67F0E8DEDCA4AA0C49F0B76C@[192.168.0.24]> <479B1EE9.7060101@ar.media.kyoto-u.ac.jp> Message-ID: <20080126193521.GB1263@lit.jwp.name> On Sat, Jan 26, 2008 at 08:52:09PM +0900, David Cournapeau wrote: > Yes, sure :) This and ssh tunneling work, but that's not ideal. Have you tried the ``http_proxy`` environment variable? (Didn't test, so I dunno. Just poked around the urllib source a bit and found a blurb about _proxy environment variables) From david at ar.media.kyoto-u.ac.jp Sun Jan 27 06:06:40 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Sun, 27 Jan 2008 14:06:40 +0900 Subject: [Distutils] Uploading an egg behind a proxy In-Reply-To: <20080126193521.GB1263@lit.jwp.name> References: <47997C59.8080409@ar.media.kyoto-u.ac.jp> <67F0E8DEDCA4AA0C49F0B76C@[192.168.0.24]> <479B1EE9.7060101@ar.media.kyoto-u.ac.jp> <20080126193521.GB1263@lit.jwp.name> Message-ID: <479C1160.6010509@ar.media.kyoto-u.ac.jp> James William Pye wrote: > On Sat, Jan 26, 2008 at 08:52:09PM +0900, David Cournapeau wrote: > >> Yes, sure :) This and ssh tunneling work, but that's not ideal. >> > > Have you tried the ``http_proxy`` environment variable? > > (Didn't test, so I dunno. Just poked around the urllib source a > bit and found a blurb about _proxy environment variables) > > > That's not the problem (and I have the variable defined to a correct value). The problem is that most of the time, the proxy is used to get a connection, but not to translate the url into an IP. In my case, this is needed. For example, ping google.com does not work on my machine: not because of ICMP blocking, but because google.com is not resolved to an IP. Only the proxy can do that cheers, David From python at jwp.name Sun Jan 27 08:54:30 2008 From: python at jwp.name (James William Pye) Date: Sun, 27 Jan 2008 00:54:30 -0700 Subject: [Distutils] Uploading an egg behind a proxy In-Reply-To: <479C1160.6010509@ar.media.kyoto-u.ac.jp> References: <47997C59.8080409@ar.media.kyoto-u.ac.jp> <67F0E8DEDCA4AA0C49F0B76C@[192.168.0.24]> <479B1EE9.7060101@ar.media.kyoto-u.ac.jp> <20080126193521.GB1263@lit.jwp.name> <479C1160.6010509@ar.media.kyoto-u.ac.jp> Message-ID: <20080127075430.GA34687@lit.jwp.name> On Sun, Jan 27, 2008 at 02:06:40PM +0900, David Cournapeau wrote: > ... yeah, that's right. Looking at the upload command in 2.5's branch, httplib is used directly, so the http_proxy environment variable isn't even being referenced on ``./setup.py upload``. Thus the name resolution error. From pje at telecommunity.com Sun Jan 27 20:51:47 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 27 Jan 2008 14:51:47 -0500 Subject: [Distutils] How to make an egg easy_installable ? In-Reply-To: <4799815A.6010100@ar.media.kyoto-u.ac.jp> References: <4799815A.6010100@ar.media.kyoto-u.ac.jp> Message-ID: <20080127195143.58F0C3A4065@sparrow.telecommunity.com> At 03:27 PM 1/25/2008 +0900, David Cournapeau wrote: >Hi, > > I wanted to make an egg from my package, so that people install it >with easy_install. I managed to make an egg, which works fine, Precisely how did you make it? I downloaded it, and there are some peculiarities, not the least of which is that the EGG-INFO directory is named "egg-info" (lowercase) instead. From zooko at zooko.com Mon Jan 28 05:17:49 2008 From: zooko at zooko.com (zooko) Date: Sun, 27 Jan 2008 21:17:49 -0700 Subject: [Distutils] wanted: a catalog of setuptools plugins Message-ID: <8E7A32F1-9C65-401B-A424-E590D6ECCD0E@zooko.com> Folks: The setuptools plugins architecture may eventually supplant Makefiles in my projects. It's very nice that people can re-use other people's development hacks as packages containing setuptools plugins instead of by cutting and pasting snippets of code into their Makefiles. It would be nice if I could find a listing of such plugins that I might want to use in my own development. Here is a list of setuptools plugins that I just found by browsing the Python Package Index: setuptools_bzr, setuptools_darcs, setuptools_mtn, setuptools_qt4, darcsver, eggchecker, iw.releaser, apydia, bbfreeze (oh I wish I had known about bbfreeze a month ago!), py2app, mkcode, eggtestinfo, easydeb, buildutils, hg.setuptools, setuptools-git Here is one that I wish someone would convert into a plugin (it runs unit tests using Twisted's "trial" unit test framework): https://code.fluendo.com/elisa/trac/browser/trunk/setup.py So my question is: how can we make it easier for people to find these? I was unaware of most of these until just now and there are probably other useful plugins that I'm still unaware of. One idea that I had was to run a query on the Python Package Index database to find all packages whose setup.py's define entry points in the "distutils.commands" group name. Another was to put the word out (i.e. in the setuptools documentation) that if your package is a setuptools plugin, that you should name your package "setuptools_blah_blah" and register it on the Python Package Index. Regards, Zooko P.S. Here is the trac ticket where the allmydata.org project is discussing moving more of our build infrastructure into setuptools plugins: http://allmydata.org/trac/tahoe/ticket/179 -- "rewrite our Makefile in Python instead of GNUmake" From zooko at zooko.com Mon Jan 28 15:41:37 2008 From: zooko at zooko.com (zooko) Date: Mon, 28 Jan 2008 07:41:37 -0700 Subject: [Distutils] wanted: a catalog of setuptools plugins In-Reply-To: <8E7A32F1-9C65-401B-A424-E590D6ECCD0E@zooko.com> References: <8E7A32F1-9C65-401B-A424-E590D6ECCD0E@zooko.com> Message-ID: <4F152FFE-C3B7-46B2-8A37-544D50C0A494@zooko.com> Following up to my own post: On Jan 27, 2008, at 9:17 PM, zooko wrote: > One idea that I had was to run a query on the Python Package Index > database to find all packages whose setup.py's define entry points in > the "distutils.commands" group name. > > Another was to put the word out (i.e. in the setuptools > documentation) that if your package is a setuptools plugin, that you > should name your package "setuptools_blah_blah" and register it on > the Python Package Index. And this morning I just noticed the "keywords" field in the metadata. Perhaps the setuptools docs could encourage authors of setuptools plugins to put the keyword "plugin" into their keywords? Regards, Zooko From dpeterson at enthought.com Mon Jan 28 16:19:47 2008 From: dpeterson at enthought.com (Dave Peterson) Date: Mon, 28 Jan 2008 09:19:47 -0600 Subject: [Distutils] wanted: a catalog of setuptools plugins In-Reply-To: <4F152FFE-C3B7-46B2-8A37-544D50C0A494@zooko.com> References: <8E7A32F1-9C65-401B-A424-E590D6ECCD0E@zooko.com> <4F152FFE-C3B7-46B2-8A37-544D50C0A494@zooko.com> Message-ID: <479DF293.4080905@enthought.com> zooko wrote: > Following up to my own post: > > On Jan 27, 2008, at 9:17 PM, zooko wrote: > >> One idea that I had was to run a query on the Python Package Index >> database to find all packages whose setup.py's define entry points in >> the "distutils.commands" group name. >> >> Another was to put the word out (i.e. in the setuptools >> documentation) that if your package is a setuptools plugin, that you >> should name your package "setuptools_blah_blah" and register it on >> the Python Package Index. >> > > And this morning I just noticed the "keywords" field in the > metadata. Perhaps the setuptools docs could encourage authors of > setuptools plugins to put the keyword "plugin" into their keywords? > I think something along the latter lines is better than imposing a specific name format on projects. For example, we have some projects that include entry points for 'distutils.commands' but they also do other things so we wouldn't want to imply they only do setuptools by having a name like 'setuptools_blah_blah'. I'd also point out that there are other types of plugins (Trac, Enthought's Envisage, etc.) so perhaps the keyword should be 'distutils_plugin' or 'setuptools_plugin'? -- Dave From fdrake at acm.org Mon Jan 28 16:28:25 2008 From: fdrake at acm.org (Fred Drake) Date: Mon, 28 Jan 2008 10:28:25 -0500 Subject: [Distutils] wanted: a catalog of setuptools plugins In-Reply-To: <4F152FFE-C3B7-46B2-8A37-544D50C0A494@zooko.com> References: <8E7A32F1-9C65-401B-A424-E590D6ECCD0E@zooko.com> <4F152FFE-C3B7-46B2-8A37-544D50C0A494@zooko.com> Message-ID: On Jan 28, 2008, at 9:41 AM, zooko wrote: > And this morning I just noticed the "keywords" field in the > metadata. Perhaps the setuptools docs could encourage authors of > setuptools plugins to put the keyword "plugin" into their keywords? Another option, for which there's precedent, is to define a classifier for the setuptools framework, similar to the classifiers used for the various web frameworks. -Fred -- Fred Drake From david at ar.media.kyoto-u.ac.jp Tue Jan 29 04:48:36 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 29 Jan 2008 12:48:36 +0900 Subject: [Distutils] How to make an egg easy_installable ? In-Reply-To: <20080127195143.58F0C3A4065@sparrow.telecommunity.com> References: <4799815A.6010100@ar.media.kyoto-u.ac.jp> <20080127195143.58F0C3A4065@sparrow.telecommunity.com> Message-ID: <479EA214.9020507@ar.media.kyoto-u.ac.jp> Phillip J. Eby wrote: > > Precisely how did you make it? I downloaded it, and there are some > peculiarities, not the least of which is that the EGG-INFO directory > is named "egg-info" (lowercase) instead. > > > I simply used the bdist_egg command from setuptools. I rebuilt an egg from my last version (0.3.4), and when unzipping it, I have EGG-INFO directory this time. How does easy_install decide where to look for packages ? cheers, David From jeff at taupro.com Tue Jan 29 15:52:48 2008 From: jeff at taupro.com (Jeff Rush) Date: Tue, 29 Jan 2008 08:52:48 -0600 Subject: [Distutils] Question re zc.buildout Name Expansion Rules Message-ID: <479F3DC0.8000309@taupro.com> In using buildout, I've come across a couple of limitations and I'm wondering if there are workarounds. Suppose I have something like: [FreeTDS_installation] recipe = zc.recipe.cmmi url = blah odbcinst_ini = [FreeTDS] Driver = ${FreeTDS_installation:location}/lib/libtdsodbc.so 1) Since the variable expansion is referring to an attribute of its own .cfg section, is there a way to omit the section name? Something like (but this doesn't work): Driver = ${location}/lib/libtdsodbc.so It just seems extra work to keep putting my own section name in, and is dangerous if I cut/paste lines to move about. 2) The initial case above fails to parse because at the time the ${} expression is reached during read-in time, the part section has not yet been added to the global registry of part sections. Hence I get a "FreeTDS_installation" undefined error. It just seems there ought to be a way for a part section to refer to its own location attribute. Thanks for any tips, -Jeff From jim at zope.com Tue Jan 29 16:22:53 2008 From: jim at zope.com (Jim Fulton) Date: Tue, 29 Jan 2008 10:22:53 -0500 Subject: [Distutils] Question re zc.buildout Name Expansion Rules In-Reply-To: <479F3DC0.8000309@taupro.com> References: <479F3DC0.8000309@taupro.com> Message-ID: <481CE7E2-FFAB-4C9A-B222-C142738409A1@zope.com> On Jan 29, 2008, at 9:52 AM, Jeff Rush wrote: > In using buildout, I've come across a couple of limitations and I'm > wondering > if there are workarounds. > > Suppose I have something like: > > [FreeTDS_installation] > recipe = zc.recipe.cmmi > url = blah > odbcinst_ini = > [FreeTDS] > Driver = ${FreeTDS_installation:location}/lib/libtdsodbc.so > > 1) Since the variable expansion is referring to an attribute of its > own .cfg section, is there a way to omit the section name? > Something like (but this doesn't work): > > Driver = ${location}/lib/libtdsodbc.so > > It just seems extra work to keep putting my own section name > in, and is dangerous if I cut/paste lines to move about. Yup. I do plan to add a syntax for this. > 2) The initial case above fails to parse because at the time > the ${} expression is reached during read-in time, the part > section has not yet been added to the global registry of > part sections. Hence I get a "FreeTDS_installation" > undefined error. > > It just seems there ought to be a way for a part section > to refer to its own location attribute. I agree that this would be helpful. (This is going to be tricky to implement, but that's a separate issue.) Would you mind adding a feature request at: https://blueprints.launchpad.net/zc.buildout/ Jim -- Jim Fulton Zope Corporation From python at jwp.name Tue Jan 29 19:59:19 2008 From: python at jwp.name (James William Pye) Date: Tue, 29 Jan 2008 11:59:19 -0700 Subject: [Distutils] Publishing Documentation Message-ID: <20080129185919.GA28892@lit.jwp.name> I use PyPI to publish my Python modules, and it is great. I love just running 'upload' and being more or less done with distribution. However, the current pain point I have with releasing is the associated documentation. It's not that I can't publish the information somewhere, it's just that it's inconvenient at best. (yes, I know, "my problem") My question is what thoughts have people had about extending PyPI and distutils/setuptools to manage the publication of package documentation in addition to package releases? I see some fantastic advantages to having the publication of documentation being handled by distutils/setuptools and PyPI: Covenience. A `publish` command pushing the documentation into the repository and being done with it would be a bit of a dream to me. Most of my documentation is targeted at developers so a doc-string based mechanism similar to pydoc--but on PyPI--would suite my and most developers needs. This also encourages the production of better modules by removing mundane hurdles involved in providing quality Python modules. Cross-package integration. A painful aspect of serving documentation is that it is desireable to have complete references to superclasses and other referenced objects. With a setuptools based `publish` command, I imagine that documentation could be extracted in a way that allows class references to be associated with the packages that they come from, opening the door for PyPI package based external-reference resolution: Package A and B providing module m1 and m2 with classes c1 and c2: B.m2.c2 inherits from A.m1.c1 Package B depends on package A. [ultimately for class c1] `publish` realizes this link while walking package B's modules and denotes that the link should be created to package A.m1.c1 for describing B.m1.c2. Easy cross-references. With PyPI managing the documentation, cross-references to Python object documentation should be easy to implement. A stylesheet PI could probably render the appropriate XML. Okay, so epydoc and others can make documentation, and you're one rsync away from having it published. It's still not convenient and it creates an interface inconsistency across packages that makes "taking in" multiple projects more difficult than an integrated solution. I think I should also mention that I've seen some packages basically publish their documentation in the package's long_description... Seems like it would be a good idea to provide a more appropriate place for this information. From bignose+hates-spam at benfinney.id.au Wed Jan 30 06:04:31 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Wed, 30 Jan 2008 16:04:31 +1100 Subject: [Distutils] Publishing Documentation References: <20080129185919.GA28892@lit.jwp.name> Message-ID: <87y7a7dhuo.fsf@benfinney.id.au> James William Pye writes: > I think I should also mention that I've seen some packages basically > publish their documentation in the package's long_description... > Seems like it would be a good idea to provide a more appropriate > place for this information. The long_description, in turn, is often generated directly from the (top-level) module docstring. It's entirely appropriate for that docstring to be a guide to using the package, especially if it also contains doctest tests. -- \ "Intellectual property is to the 21st century what the slave | `\ trade was to the 16th." ?David Mertz | _o__) | Ben Finney From python at jwp.name Wed Jan 30 17:12:14 2008 From: python at jwp.name (James William Pye) Date: Wed, 30 Jan 2008 09:12:14 -0700 Subject: [Distutils] Publishing Documentation In-Reply-To: <20080129185919.GA28892@lit.jwp.name> References: <20080129185919.GA28892@lit.jwp.name> Message-ID: <20080130161214.GA39096@lit.jwp.name> On Tue, Jan 29, 2008 at 11:59:19AM -0700, James William Pye wrote: > My question is what thoughts have people had about extending PyPI and > distutils/setuptools to manage the publication of package documentation > in addition to package releases? patches welcome?... =\ From python at jwp.name Thu Jan 31 04:39:49 2008 From: python at jwp.name (James William Pye) Date: Wed, 30 Jan 2008 20:39:49 -0700 Subject: [Distutils] Publishing Documentation In-Reply-To: <20080130161214.GA39096@lit.jwp.name> References: <20080129185919.GA28892@lit.jwp.name> <20080130161214.GA39096@lit.jwp.name> Message-ID: <20080131033949.GA44744@lit.jwp.name> On Wed, Jan 30, 2008 at 09:12:14AM -0700, James William Pye wrote: > patches welcome?... =\ I'm going to send this to catalog-sig. It seems like a more appropriate channel for some or all of this.. Sorry for the noise. From mhammond at skippinet.com.au Thu Jan 31 07:25:18 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Thu, 31 Jan 2008 17:25:18 +1100 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <022401c85cde$678bcf10$36a36d30$@com.au> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <022401c85cde$678bcf10$36a36d30$@com.au> Message-ID: <083301c863d2$12dc2b40$389481c0$@com.au> I'm wondering if anyone has any comment on this issue? Its currently impossible to use distutils as documented to build x64 extensions, either natively or cross-compiled using the trunk and while I'm keen to help fix this, I'd like agreement on the direction. Can I assume silence means general agreement on the compromise proposal I outlined? Thanks, Mark > -----Original Message----- > From: python-dev-bounces+mhammond=keypoint.com.au at python.org > [mailto:python-dev-bounces+mhammond=keypoint.com.au at python.org] On > Behalf Of Mark Hammond > Sent: Tuesday, 22 January 2008 9:06 PM > To: '"Martin v. L?wis"' > Cc: distutils-sig at python.org; python-dev at python.org > Subject: Re: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows > > Hi Martin, > > Way back on Monday, 21 May 2007, you wrote: > > > This is an issue to be discussed for Python 2.6. I'm personally > > hesitant to have the "official" build infrastructure deviate > > from the layout that has been in-use for so many years, as a lot > > of things depend on it. > > > > I don't find the need to have separate object directories convincing: > > For building the Win32/Win64 binaries, I have separate checkouts > > *anyway*, since all the add-on libraries would have to support > > multi-arch builds, but I think they don't. > ... > > > * Re the x64 directory used by the PCBuild8 process. IMO, it makes > > > sense to generate x64 binaries to their own directory - my > expectation > > > is that cross-compiling between platforms is a reasonable use-case, > > > and we should support multiple achitectures for the same compiler > > > version. > > > > See above; I disagree. > > [For additional context; the question was regarding where the output > binaries were created for each platform, and specifically if x64 build > should use something like 'PCBuild/x64'] > > I'm bringing this topic up again as I noticed the AMD64 builds for > Python > 2.6 create their output in the PCBuild/x64 directory, not directly into > the > PCBuild directory. When I raised this with Christian, he also > expressed a > desire to be able to cross-compile in the same directory structure and > was > unaware of this discussion (but I made him aware of it :) > > You made excellent points about how tools currently recognize the > PCBuild > directory, and we can't break them, and I agree. I'd like to propose a > compromise that might be able to keep everyone happy. > > The main problem I see with all platforms using 'PCBuild' is that in a > cross-compilation scenario, it is impossible for the distutils to > determine > the location of the correct Python libraries to link with (stuff in its > own > PYTHONHOME is no good; it's for the wrong platform). Obviously, we can > change the distutils so that the location of the correct libraries can > be > specified, but that implies all other build systems that currently > assume > PCBuild must also change to work in a cross-compilation scenario. > While > those external tools *will* work today in a native build on x64, > eventually > they may need to be upgraded to deal with cross-compilation - and this > means > that all such tools will also be unable to automatically determine the > location of the libraries. They will all need to take the library dir > as a > user-specified option, which seems a pain (eg, buildbots would seem to > need > site-specific configuration information to make this work). If > eventually > all such tools are likely to upgrade, it would be ideal if we can offer > them > a way to upgrade so the correct libraries can be determined > automatically, > as they can now for native builds. > > My compromise proposal is this: Each platform builds in its own > directory > (ie, PCBuild/x86_64, PCBuild/x86). Every single build configuration > has a > post-build step that copies the result of the build to the PCBuild > directory. We then add an option to the distutils so that a cross- > compile > platform can be specified and when it is, to use 'PCBuild/{platform}" > instead of PCBuild, and we encourage other tools to use the same > directory > should they want to support "configure-less" cross-compilation (if > distutils > on the trunk is not still trying to maintain b/w compat, it should just > *always* look in PCBuild/{platform}, and I'd suggest py3k not bother > with > PCBuild at all; build systems will be touched to upgrade to py3k, so > that > change isn't painful. But I digress :) > > The main advantage of the compromise is that it allows for the status > quo to > remain in place - just continue to use separate source trees for each > platform, and only ever build a single platform in each tree, and tools > are > free to have ways of specifying which directory should be used. The > official build process need not change. However, it also supports a > way to > do cross-compilation in a way that build tools can *automatically* > locate > the correct platform's libraries, and this will be particularly useful > for > extension authors. > > Would something like that be acceptable? > > Thanks, > > Mark > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python- > dev/mhammond%40keypoint.com.au