[Numpy-discussion] Compiling NumPy on Windows for Python 3.3

[Peter Cock]

On Mon, Nov 12, 2012 at 9:20 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>> ...
>> Found executable C:\cygwin\usr\bin\gcc.exe
>> g++ -mno-cygwin _configtest.o -lmsvcr100 -o _configtest.exe
>> Could not locate executable g++
>> Executable g++ does not exist
>
>
> A C++ compiler shouldn't be needed for numpy, so it shouldn't try to find
> one. On Linux it doesn't try this. It looks like on Windows (with MinGW at
> least) it does. I looked at a build log from a compile with MinGW and g++
> available
> (http://projects.scipy.org/numpy/attachment/ticket/1909/dc6b601-builld.log),
> and g++ is used exactly once: for the linking stage of
> numpy.core._dummy.pyd.
>
> Looking at core/setup.py that is related to __STDC_FORMAT_MACROS. That led
> me to https://github.com/numpy/numpy/commit/64423fdb18e8. So the easy way
> forward is to install g++. The hard way is figuring out why that define was
> necessary and find a way around it.
>
> Ralf

Very strange (wanting a C++ compiler).

I had some trouble updating my cygwin installation (probably in part
due to issues stemming from a change in Windows username) and
gave up and reinstalled cygwin.

This presented a new issue that gcc and g++ wouldn't work from
the Windows cmd prompt (due to being symlinks), so I deleted
the symlinks /usr/bin/gcc[.exe] and /usr/bin/g++[.exe] and just
replaced them with copies of the binaries /usr/bin/gcc-3.exe and
/usr/bin/g++-3.exe - which seems to work. [I don't recall having to
do that before, perhaps a relatively recent regression in cygwin?]

Now, back to numpy, ...

C:\repositories\numpy>c:\python33\python setup.py build --compiler=mingw32
<snip>
gcc -mno-cygwin -O2 -Wall -Wstrict-prototypes
-DNPY_NEEDS_MINGW_TIME_WORKAROUND -DNPY_MINGW_USE_CUSTOM_MSVCR
-D__MSVCRT_VERSION__=0x1000 -Inumpy\core\include
-Ibuild\src.win32-3.3\numpy\core\include/numpy
-Inumpy\core\src\private -Inumpy\core\src -Inumpy\core
-Inumpy\core\src\npymath -Inumpy\core\src\multiarray
-Inumpy\core\src\umath -Inumpy\core\src\npysort -Inumpy\core\include
-Ic:\python33\include -Ic:\python33\include
-Ibuild\src.win32-3.3\numpy\core\src\multiarray
-Ibuild\src.win32-3.3\numpy\core\src\umath -c
numpy\random\mtrand\distributions.c -o
build\temp.win32-3.3\Release\numpy\random\mtrand\distributions.o
g++ -mno-cygwin -shared
build\temp.win32-3.3\Release\numpy\random\mtrand\mtrand.o
build\temp.win32-3.3\Release\numpy\random\mtrand\randomkit.o
build\temp.win32-3.3\Release\numpy\random\mtrand\initarray.o
build\temp.win32-3.3\Release\numpy\random\mtrand\distributions.o
-Lc:\python33\libs -Lc:\python33\PCbuild -Lbuild\temp.win32-3.3
-lpython33 -lmsvcr100 -o build\lib.win32-3.3\numpy\random\mtrand.pyd
running scons
running build_scripts
creating build\scripts.win32-3.3
Creating build\scripts.win32-3.3\f2py.py
  adding 'build\scripts.win32-3.3\f2py.py' to scripts

C:\repositories\numpy>

Horray! It appears to work. I guess I need nose for the tests now...

However, what is the correct way to now install this build? I have a
workaround (given below), but the obvious routes failed:

-------------------------------------------------------------

C:\repositories\numpy>c:\python33\python setup.py install --compiler=mingw32
Converting to Python3 via 2to3...
Running from numpy source directory.
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: option --compiler not recognized

-------------------------------------------------------------

Yet the only other likely choice of leaving out the compiler option also fails:

C:\repositories\numpy>c:\python33\python setup.py install
Converting to Python3 via 2to3...
Running from numpy source directory.
non-existing path in 'numpy\\distutils': 'site.cfg'
F2PY Version 2
numpy\core\setup_common.py:86: MismatchCAPIWarning: API mismatch detected, the C
 API version numbers have to be updated. Current C api version is 8, with checks
um f4362353e2d72f889fda0128aa015037, but recorded checksum for C API version 8 i
n codegen_dir/cversions.txt is 17321775fc884de0b1eda478cd61c74b. If functions we
re added in the C API, you have to update C_API_VERSION  in numpy\core\setup_com
mon.py.
  MismatchCAPIWarning)
blas_opt_info:
<snip>
running install
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands
--compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands
--fcompiler options
running build_src
build_src
building py_modules sources
building library "npymath" sources
No module named 'numpy.distutils.msvccompiler' in numpy.distutils;
trying from distutils
Unable to find productdir in registry
Checking environ VS100COMNTOOLS
error: Unable to find vcvarsall.bat

C:\repositories\numpy>

-------------------------------------------------------------

I went with plan (B), creating C:\Python33\Lib\distutils\distutils.cfg
containing:

[build]
compiler=mingw32

Now I don't have to give a compiler switch and distutils will
default to using mingw32 with Python 3.3, and build and
install seem to work.

I've not yet run the numpy tests yet, but I think this means
my github branches are worth merging:

https://github.com/peterjc/numpy/commits/msvc10

Regards,

Peter

P.S. I am really looking forward to official Windows installers for
NumPy on Python 3.3 on Windows...