[Python-ideas] Rewriting the build system (part 2)

[Andrew Barnert]

Something nobody's mentioned yet in this thread:

The autotools stuff doesn't just handle different platforms, it also handles local differences (not every FooLinux x.y system has Tcl/Tk; if yours doesn't, you can still build Python, you just won't get tkinter) and user options (disabling features, using a specific OpenSSL instead of the older system one that would get autodetected, etc.).

Autoconf makes this relatively easy (not that anything in autoconf is easy, but if you can write the OpenSSL detection, it's not significantly harder to write OpenSSL detection with a flag to override the path). Is the same thing true for other build configuration systems?


Sent from my iPhone

On Mar 16, 2015, at 9:27 AM, Chris Barker <chris.barker at noaa.gov> wrote:

>> >> That written in Python, so Python would require Python to build which
>> >> would require Python to build which would require...
> 
> A bit more here:
> 
> It seems there are two cases:
> 
> 1) Development on and for a common, mature platform:
> 
> In this case finding a functional python with which to run the build tool sis trivial. We're all set, no need to exclude python-based tools.
> 
> 2) Development for / porting to a new and/or obscure platform:
> 
> In this case, then there is no existing python with which to run the build tools. And there are two ways you might go about this (in general):
> 
>   a) cross-compiling -- run the compiler on a existing mature platform, and generate a compiled python for the new platform. IN this case, the build tools only need to run on the development platform, so we are back to (1), and all set. Note that the hard part of cross-compiling is still done by the C compiler, not really the build/configuration tools, The build tools need to be able to set up the cross-compile -- but presumable a new tool is chosen be cause it's easier to do things like that with it...
> 
>   b) Native compiling -- compiling python on the new platform. This is the tricky one. In this case, the entire goal is to get python building/running on the new platform. Clearly you can't just use the nifty python-based build system to do that. But as that's the goal, you'll still need to do the tricky parts anyway. So you could cross-compile the first version -- with maybe as few modules as possible to get started. Another option is a build system that needs python for the configuration stage, but not the compilation stage -- kind of like autotools: autotools is used in a three-step process:
> 
>   i) run autoconf, etc, to generate a configure script -- this is often distributed with the source tarball, so that end users don't need the autoconf tools
> 
>  ii) run the configure script to generate Makefiles, etc. configured for the platform at hand.
> 
>  iii) run make to do the actual build.
> 
> Usually the last two steps are run natively -- but it's not too much of a stretch to run i and ii on a known build machine, and then only ned to run make on the final target. That would require the build tools to have a cross-configure ability.
> 
> I have no idea if any of the known tools can do this, but if so, it would be pretty usable.
> 
> And also worth keeping in mind that truly new platforms are pretty rare -- this kind of bootstrapping would not have to be done often --not a killer if it takes some hand-work.
> 
> But the grain of salt : I have neer and am unlikely ever, to try to port a significant project to a new platform. Folks that actually need to do this should make these decisions.
> 
>> Will this affect the commands required to build Python from source, or
>> is all this just for actual development?
> 
> well, it may or may not be both:
> 
> If the system supported a three-step process I outlined above, then makefiles (or the like) could be shipped for the common platforms, though probably easier to run the configure step on the final target. But the vast majority of final targets do have a python ready to run the configure/build machinery.
> 
>>  someone
>> who has little knowledge of Python's internals, and just wants to
>> build the latest version (either from hg or from a source tarball).
> 
> That's the assumption I'm following -- the user wants the latest version -- so probably already has easy access to an older version with which to run the build tools...
> 
>> But in the second case, it'd be much safer to minimize the
>> requirements. If you can untar the source and just "./configure; make;
>> sudo make install" your way to having the latest Python, that's a LOT
>> easier than I've just been doing trying to get my Dad onto a newer
>> Mozilla Thunderbird. (I don't understand what it's doing all along
>> there, but it pulls a whole bunch of separate repositories when I say
>> update, and there's a separate bootstrap step, and ... I dunno.)
> 
> requiring _some_, not very recent, python is not exactly
> 
>  """a whole bunch of separate repositories when I say update, and there's a separate bootstrap step"""
> 
> If there is an already build python (like OS-X, any general purpose linux distro, Windows if you don't mind point and clicking the python.org installer, etc, then it's not really a "bootstrap".
> 
> In short -- using a python-based build system would make porting to a brand-new platform trickier -- but not really affect most other building.
> 
> -CHB
> 
> 
> -- 
> 
> Christopher Barker, Ph.D.
> Oceanographer
> 
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
> 
> Chris.Barker at noaa.gov
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150316/725f7aa4/attachment-0001.html>