[Python-ideas] Rewriting the build system (part 2)

[Ned Deily]

In article 
<CALGmxEJLmU6Jy1LiRo9qbZs09WSsvK2yhGuWWQid3Q=UJC-8tg at mail.gmail.com>,
 Chris Barker <chris.barker at noaa.gov> wrote:
> This doesn't seem to me to be a winner takes all proposition -- there is no
> reason someone couldn't set up a new build system for Python, get it all
> working nicely on their platform(s) of choice, persuade others how
> wonderful it is, get contributions, etc.
> 
> If it turns out to really be better/easier etc. than autoconf, then , and
> only then, would it make any sense to talk about replacing the existing
> official build system. And even then, the two could live in parallel for
> quite some time...

Sorry, that's just not realistic.  We already have two different build 
systems, one for Windows and one for all other platforms, and it's 
difficult enough to keep both of them running.  Adding yet another 
system in parallel is not in the cards without a major infusion of 
resources.  There are frequent minor changes to the current build 
systems as part of new Python features or bug fixes and those run the 
risk of breaking things today and too often do.  Major changes to build 
systems impact most parts of the Python ecosphere, for example, 
development, automated testing, manual testing, release management, 
downstream distributors of CPython (many of them managing their own sets 
of local modifications to the current processes), etc etc.  To be even 
able to evaluate the merits of a major build system migration, we would 
first need to understand what we support today, what are the actual 
shortcomings of what we have today, and what we want to support in the 
future.  Then, given that, we'd need to decide what are the options and 
what are their costs.  One prerequisite to that is having a much more 
comprehensive test build environment so we could make even substantial 
incremental changes with a reasonable degree of confidence that we're 
not breaking support of a defined set of platforms and configurations.  
We don't have that today.  Working towards that would be a necessary and 
important step.

Also, note that, in all this talk about autoconf, be aware the CPython 
build processes don't even make full use of current automake/autoconf 
features.  Much of the current Python build process predates the 
addition of features to those tools.  Just bringing the current 
configure scripts and Makefiles up-to-date to use all the current 
features would be a big task that would imply a big testing effort on 
resources we don't currently have.

Also keep in mind that builds and installations are areas that few core 
CPython developers are all that interested in and with good reason.  
It's usually tedious and generally unrewarding work especially when 
operating in a vacuum.  Let's first work towards properly identifying 
and testing current important environments (for example, from typical 
installed Python configurations, not build directories and, as noted 
before, the almost totally undocumented and tested world of 
cross-building) and then we can start incrementally improving things 
with some degree of confidence that we're not going to break things for 
us and our users.

-- 
 Ned Deily,
 nad at acm.org