[Python-Dev] Script(s) for building Python on Windows

[Jason R. Coombs]

The current scripts for building Python lack some things to be desired.

 

The first thing I notice when I try to build Python on Windows is the
scripts expect to be run inside of a Visual Studio environment, the
environment of which is only defined inside of a cmd.exe context. This means
the scripts can't be executed from within Powershell (my preferred shell on
Windows). One must first shell out to cmd.exe, which disables any
Powershell-specific features the developer might have installed (aliases,
functions, etc).

 

The second thing I notice is the scripts assume Visual Studio 2008. And
while I recognize that Python is specifically built against Visual Studio
2008 for the official releases and that Visual Studio 2008 may be the only
officially-supported build environment, later releases, such as Visual
Studio 2010 are also adequate for testing purposes. I've been developing
Python against Visual Studio 2010 for quite a while and it seems to be more
than adequate. And while it's not the responsibility of the scripts to
accommodate such environments, if the scripts could allow for such
environments, that would be nice. Furthermore, having scripts that codify
the process to upgrade will facilitate the migration should someone make the
decision to officially upgrade to Visual Studio 2010.

 

The third thing that I notice is that the command-line argument handling by
the batch scripts is clumsy (compared to argparse, for example). This
clumsiness is not a criticism of the authors, who have done well with the
tools they had. However, batch programming is probably one of the least
powerful ways to automate builds these days.

 

So to ease my experience, I've developed my own library of functions and
commands to facilitate building Python that aren't subject to the above
limitations. Of course, I built these in Python, so they do require Python
to build Python (not a huge burden, but worth mentioning). All of these
modules are open-source and part of the jaraco.develop package
<http://pypi.python.org/pypi/jaraco.develop> .

 

The first of these modules is jaraco.develop.vstudio
<https://bitbucket.org/jaraco/jaraco.develop/src/b7263c9d9c93/jaraco/develop
/vstudio.py> . It exposes a class for locating Visual Studio in the usual
locations, loading the environment for that instance of Visual Studio, and
upgrading a project or solution file to that version. This class in
particular enables running Visual Studio commands (including msbuild) from
within a Visual Studio environment without actually requiring a cmd.exe
context with that environment.

 

Another module is jaraco.develop.python
<https://bitbucket.org/jaraco/jaraco.develop/src/b7263c9d9c93/jaraco/develop
/python.py> , which includes build_python, a function (and command) to build
Python using whatever version of Visual Studio can be found (9 or 10
required). It has no environmental requirements except that Visual Studio be
installed. Simply run build-python (part of jaraco.develop's console
scripts) and it will build PCbuild.sln from the current directory to
whatever targets are specified (or all of them if none are specified). The
builder currently makes some assumptions (such as always building the 64-bit
Release targets), but those could easily be customized using argparse
parameters.

 

This package and these modules have been tested and run on Python 2.7+.
These tools solve the three shortcomings I mentioned above and make the
development process so much smoother, IMO. If these modules were built into
the repository, building Python could be as simple as "hg clone; cd
cpython/pcbuild; ./build.py" (assuming only Visual Studio and Python
available).

 

I'd like to propose migrating this functionality (mainly these two modules)
into the CPython heads for Python 2.7, 3.1, 3.2, and default as
PCbuild/build.py (or similar). This functionality doesn't necessarily need
to supersede the existing scripts (env, build_env, build), though it
certainly could (and would as far as my usage is concerned).

 

If there are no objections, I'll work to extract the aforementioned
functionality from the jaraco.develop modules and into a portable script and
put together a proof-of-concept in the default branch. The build script
should not interfere with any build bots or other existing build processes,
but should enable another more powerful technique for producing builds.

 

I look forward to your comments and feedback.

 

Regards,

Jason

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120116/09d496a9/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6662 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120116/09d496a9/attachment-0001.bin>