[Distutils] using python generated by buildout to run buildout
Łukasz Nowak
email at lnowak.com
Wed Oct 6 12:15:04 CEST 2010
Hello,
I am playing with zc.buildout for some time. I experienced issues with
system provided pythons (even virutalenved), so I switched to build my
own python. Of course as I am lazy, I am using buildout to configure
and compile various python versions on my machine.
Quite often I need to provide "self contained" buildout profile, which
would be able to run using python compiled by itself.
So my example profile looks like this:
[buildout]
parts =
rebootstrap2.4
realrun
find-links = http://www.nexedi.org/static/packages/source/
[python2.4]
prefix = ${buildout:parts-directory}/${:_buildout_section_name_}
executable = ${:prefix}/bin/python2.4
# this is just wrapper for hexagonit.recipe.cmmi which makes it not
returning python part
recipe = erp5.recipe.cmmisafe
url =
http://python.org/ftp/python/2.4.6/Python-2.4.6.tgz
configure-options =
--enable-unicode=ucs4
--with-threads
--with-readline
--with-dbm
--with-zlib
--with-ssl
--with-bz2
[rebootstrap2.4]
recipe = zc.recipe.egg
eggs = zc.buildout
python = python2.4
[realrun]
recipe = plone.recipe.command
command =
echo Running with python ${buildout:executable}
update-command = ${:command}
So first step for me is to bootstrap builodut:
curl -s http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py
| python -S -
So now my bin/buildout is referring to "system python":
head -n1 bin/buildout
#!/home/luke/bin/python -S
Then I need to rebootstrap it:
bin/buildout install python2.4 rebootstrap2.4
python2.4* parts are installing and compiling python, rebootstrap2.4
part is reinstalling bin/buildout using this python.
Cool, my bin/buildout is using python it compiled:
head -n1 bin/buildout
#!/home/luke/pybuildout/parts/python2.4/bin/python2.4
So I can run again, this time usually. python will be compiled, as
buildout signature changed, but this is not an issue (ccache + distcc
makes it really fast). Buildout is using python he provided,
chicken-and-egg issue solved by hand:
bin/buildout
...
realrun: Running '
echo Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4'
Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4
Unused options for realrun: 'update-command'.
Re running builodut does not affect python anymore:
bin/buildout
Updating python2.4-dbm-patch.
Updating python2.4.
Updating rebootstrap2.4.
Updating realrun.
realrun: Running
echo Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4
Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4
And so far I am happy, I am using correct python version.
What I'd like to develop is to make those steps automatically, so that
after running:
curl -s http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py
| python -S -
bin/buildout
Such steps would happen automagically:
1 normal bootstrap with system python
2 compile python as needed
3 install new bin/buildout
4 re-run bin/buildout
5 compile python as signature change
6 install new bin/buildout (which is the same)
7 re-run bin/buildout
8 do not compile python, as signature had not changed, continue...
Theoretically steps 5,6 and 7 could be avoided, but lets say I am
purist -- I do not trust that python compiled by buildout which is
running python I do not trust is bad :)
Easy question: Is there ability for buildout to do smart rebootstrap
in one run, with selecting part which contains python?
If no I am ready to develop such thing. So I read a bit about
extensions - I can hook before and after buildout is run. Extensions
have access to buildout object, I think they can play with buildout
externals a lot, including re-running buildout (I saw in output that
buildout re-runs itself after upgrading himself).
Are extension the way to go with such thing?
If needed I am even ready to play a bit more with zc.buildout
internals code to extend if required, but according to my
understanding how buildout works there are enough places I can "hook"
to.
Any other comments? References?
Regards,
Luke
More information about the Distutils-SIG
mailing list