[Cython] Suggested action for "Python.h not found"

[Elizabeth A. Fischer]

On Thu, Jun 9, 2016 at 2:49 AM, Nathaniel Smith <njs at vorpus.org> wrote:

> On Wed, Jun 8, 2016 at 10:37 PM, Elizabeth A. Fischer
> <elizabeth.fischer at columbia.edu> wrote:
> > Spack gives you complete control over your compiler.  This is important
> if
> > you're  building Python extensions, especially C++-based extensions,
> which
> > must be built with the same compilers use to build Python.
>
> Just to hopefully avoid confusing people -- you mostly only need to
> worry about this on Windows. On OS X and Linux, I can't think of any
> situation where using a different compiler for your extension and for
> Python will cause a problem.
>

It could cause a problem if your extension is a wrapper around a library
that you wrote in  C++ or Fortran --- whether you wrote your extension by
hand, or especially if you used the Cython variant that generates  C++ code
(a logical choice, if you're interfacing to a C++ library).  If your C++
Cython extension requires a different compiler than that used to build
Cython, you will also find that standard  Cython ways to building your
extension break.

Maybe you're right --- maybe the extension itself is built in C, and all
 C++ elements are safely "encapsulated" away from Python.  Maybe you can
coax Cython into building extensions with your chosen compiler.  But who
wants to spend time finding out exactly which heterogeneous compilers work
together, and how to avoid the standard Cython build paths?

The problem is more general than compilers --- it can actually bite you ANY
time you use two different versions of a shared library in a build that you
ultimately link together.  Suppose your Python stack was built with NetCDF4
but you built your extension with NetCDF3?  In a software stack of 50
packages, the possibilities for this kind of problem are endless --- or
more precisely, there are 50 opportunities for this kind of problem.  And
they are almost impossible to prevent if you're doing things by hand.
Sometimes you can get away with it, and sometimes these problems result in
strange, time-sucking bugs.  Spack ensures that only ONE version of each
package is linked in your final binary.

and the
> interface between the libraries uses std::string or std::list, and
> you're on Linux, then you need to be careful about the GCC 5 / C++11
> ABI transition. The short version is that if you consistently use
> versions of GCC <5 OR consistently use versions of GCC >=5, then you
> should be fine; the long version can be found by googling :-).
>

That has never been my experience.  My actual experience is every time I
upgraded my MacPorts, I had to specifically re-build all my C++ libraries.
This was after running into strange, time-sucking bugs.  Advise on GCC's
C++ supports this:

http://stackoverflow.com/questions/23895081/can-you-mix-c-compiled-with-different-versions-of-the-same-compiler

And then there's Fortran... in which the .mod files are incompatible
between every release.

In theory, theory and practice are the same.  In practice, the only things
that work are those than are used commonly.  People test that software
stacks build and work together with a single compiler.  They don't test
random heterogeneous combinations of compilers, libraries, etc.

At the end of the day, the advice is simple: build your entire software
stack with the same compiler (and same set of libraries too).  When you
upgrade compilers, re-build your entire software stack.  If you use an
auto-builder like Spack or MacPorts, this is easier than upgrading to
Windows 10 ;-).

-- Elizabeth
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20160609/43cb24c2/attachment.html>