[Python-ideas] Rewriting the build system (part 2)

Tue Mar 17 18:06:52 CET 2015

On Mon, Mar 16, 2015 at 10:38 PM, Andrew Barnert <abarnert at yahoo.com> wrote:

> On Mar 16, 2015, at 12:19 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>
> Most due let you do that easily. For instance, fbuild lets you just trap
> the raised error and move on.
>
>
> How does trapping an error and moving on give you a user-configurable
> option to select which OpenSSL to use, whether he wants a fat i386/x86_64
> build or just x86_64, whether he wants a framework bundle or a traditional
> layout...? (These are the first three flags that Homebrew passes to
> Python's configure aside from the install prefix, which I'd hope everything
> can handle.)
>
>
Just mix a command-line argument with a try-catch block. It's actually much
easier/less-hackish than it sounds.

The nice thing is that fbuild allows multiple builders, so separating
target/host stuff and command-line args is super-easy.

> On Mon, Mar 16, 2015 at 1:05 PM, Andrew Barnert <
> abarnert at yahoo.com.dmarc.invalid> wrote:
>
>> Something nobody's mentioned yet in this thread:
>>
>> The autotools stuff doesn't just handle different platforms, it also
>> handles local differences (not every FooLinux x.y system has Tcl/Tk; if
>> yours doesn't, you can still build Python, you just won't get tkinter) and
>> user options (disabling features, using a specific OpenSSL instead of the
>> older system one that would get autodetected, etc.).
>>
>> Autoconf makes this relatively easy (not that anything in autoconf is
>> easy, but if you can write the OpenSSL detection, it's not significantly
>> harder to write OpenSSL detection with a flag to override the path). Is the
>> same thing true for other build configuration systems?
>>
>>
>> Sent from my iPhone
>>
>> On Mar 16, 2015, at 9:27 AM, Chris Barker <chris.barker at noaa.gov> wrote:
>>
>> >> That written in Python, so Python would require Python to build which
>>> >> would require Python to build which would require...
>>>
>>
>> A bit more here:
>>
>> It seems there are two cases:
>>
>> 1) Development on and for a common, mature platform:
>>
>> In this case finding a functional python with which to run the build tool
>> sis trivial. We're all set, no need to exclude python-based tools.
>>
>> 2) Development for / porting to a new and/or obscure platform:
>>
>> In this case, then there is no existing python with which to run the
>> build tools. And there are two ways you might go about this (in general):
>>
>>   a) cross-compiling -- run the compiler on a existing mature platform,
>> and generate a compiled python for the new platform. IN this case, the
>> build tools only need to run on the development platform, so we are back to
>> (1), and all set. Note that the hard part of cross-compiling is still done
>> by the C compiler, not really the build/configuration tools, The build
>> tools need to be able to set up the cross-compile -- but presumable a new
>> tool is chosen be cause it's easier to do things like that with it...
>>
>>   b) Native compiling -- compiling python on the new platform. This is
>> the tricky one. In this case, the entire goal is to get python
>> building/running on the new platform. Clearly you can't just use the nifty
>> python-based build system to do that. But as that's the goal, you'll still
>> need to do the tricky parts anyway. So you could cross-compile the first
>> version -- with maybe as few modules as possible to get started. Another
>> option is a build system that needs python for the configuration stage, but
>> not the compilation stage -- kind of like autotools: autotools is used in a
>> three-step process:
>>
>>   i) run autoconf, etc, to generate a configure script -- this is often
>> distributed with the source tarball, so that end users don't need the
>> autoconf tools
>>
>>  ii) run the configure script to generate Makefiles, etc. configured for
>> the platform at hand.
>>
>>  iii) run make to do the actual build.
>>
>> Usually the last two steps are run natively -- but it's not too much of a
>> stretch to run i and ii on a known build machine, and then only ned to run
>> make on the final target. That would require the build tools to have a
>> cross-configure ability.
>>
>> I have no idea if any of the known tools can do this, but if so, it would
>> be pretty usable.
>>
>> And also worth keeping in mind that truly new platforms are pretty rare
>> -- this kind of bootstrapping would not have to be done often --not a
>> killer if it takes some hand-work.
>>
>> But the grain of salt : I have neer and am unlikely ever, to try to port
>> a significant project to a new platform. Folks that actually need to do
>> this should make these decisions.
>>
>> Will this affect the commands required to build Python from source, or
>>> is all this just for actual development?
>>>
>>
>> well, it may or may not be both:
>>
>> If the system supported a three-step process I outlined above, then
>> makefiles (or the like) could be shipped for the common platforms, though
>> probably easier to run the configure step on the final target. But the vast
>> majority of final targets do have a python ready to run the configure/build
>> machinery.
>>
>>  someone
>>> who has little knowledge of Python's internals, and just wants to
>>> build the latest version (either from hg or from a source tarball).
>>>
>>
>> That's the assumption I'm following -- the user wants the latest version
>> -- so probably already has easy access to an older version with which to
>> run the build tools...
>>
>> But in the second case, it'd be much safer to minimize the
>>>
>> requirements. If you can untar the source and just "./configure; make;
>>> sudo make install" your way to having the latest Python, that's a LOT
>>> easier than I've just been doing trying to get my Dad onto a newer
>>> Mozilla Thunderbird. (I don't understand what it's doing all along
>>> there, but it pulls a whole bunch of separate repositories when I say
>>> update, and there's a separate bootstrap step, and ... I dunno.)
>>>
>>
>> requiring _some_, not very recent, python is not exactly
>>
>>  """a whole bunch of separate repositories when I say update, and there's
>> a separate bootstrap step"""
>>
>> If there is an already build python (like OS-X, any general purpose linux
>> distro, Windows if you don't mind point and clicking the python.org
>> installer, etc, then it's not really a "bootstrap".
>>
>> In short -- using a python-based build system would make porting to a
>> brand-new platform trickier -- but not really affect most other building.
>>
>> -CHB
>>
>>
>> --
>>
>> Christopher Barker, Ph.D.
>> Oceanographer
>>
>> Emergency Response Division
>> NOAA/NOS/OR&R            (206) 526-6959   voice
>> 7600 Sand Point Way NE   (206) 526-6329   fax
>> Seattle, WA  98115       (206) 526-6317   main reception
>>
>> Chris.Barker at noaa.gov
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
>
> --
> Ryan
> [ERROR]: Your autotools build scripts are 200 lines longer than your
> program. Something’s wrong.
> http://kirbyfan64.github.io/
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>

-- 
Ryan
[ERROR]: Your autotools build scripts are 200 lines longer than your
program. Something’s wrong.
http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150317/92378334/attachment-0001.html>