[Wheel-builders] Problem with finding python library via CMake

[Jean-Christophe Fillion-Robin]

Hi Matt,

(1) On linux, you should not link against the library. For that reason, it
is not available in manylinux.

To provide some background:

-----------------
Since the python interpreter exports its symbol (see [1]), python modules
should not link against any python libraries.
To ensure it is not the case, we configure the project using an empty file
as python library.

 [1] "Note that libpythonX.Y.so.1 is not on the list of libraries that a
manylinux1 extension is allowed to link to. Explicitly linking to
libpythonX.Y.so.1 is unnecessary in almost all cases: the way ELF linking
works, extension modules that are loaded into the interpreter automatically
get access to all of the interpreter's symbols, regardless of whether or
not the extension itself is explicitly linked against libpython. [...]"

Source: https://www.python.org/dev/peps/pep-0513/#libpythonx-y-so-1
-----------------



(2) Updated CMake

While this would not solve this particular problem, there are python wheels
available for CMake.
See https://pypi.org/pypi/cmake/

And also for ninja build tool (quite usual for automatic parallelization of
the build). Note that the version distributed on pypi is patched
<https://github.com/kitware/ninja#readme> version that include the
jobserver capability as well as as fortran support .
See https://pypi.org/project/ninja/


(3) Streamlined build

I suggest you look into scikit-build, this provide a nice integration of
CMake and python setuptools.
See http://scikit-build.readthedocs.io/


(4) CI and upload to PyPI

For an example of project, consider looking at
https://github.com/amueller/word_cloud


Hth
Jc






On Wed, Jan 23, 2019 at 9:41 AM Clarkson, Matt <m.clarkson at ucl.ac.uk> wrote:

> Hi there,
>
> If someone could help me, I would be eternally grateful.
>
> I'm currently creating:
> https://github.com/MattClarkson/CMakeCatchTemplate
>
> as a template project, from which others can generate their own C++ CMake,
> Catch, CTest project, and I'm providing support for building Boost.Python,
> PyBind11, and hence wheels, via Appveyor, and Travis, via manylinux.
>
> I'm currently on Issue #45, and branch: 45-pypi, and the Travis log looks
> like it's working:
> https://travis-ci.com/MattClarkson/CMakeCatchTemplate/builds/98215507
>
> However, if you look at a Linux build, for example:
> https://travis-ci.com/MattClarkson/CMakeCatchTemplate/jobs/172197399
>
> and line 599, we see that CMake is finding the right python executable,
> and the right python include dir, but picking up /usr/lib64/libpython2.4
> instead of the required version 3.5.5. This means, when I pip install on
> another independent machine, downloading the wheel from pypi, I get a
> library not found error. I should say at this point, that I'm actually
> using the opencv-python docker image provided here:
> quay.io/skvark/manylinux, but this is essentially the same as manylinux,
> with these additions:
> https://github.com/skvark/opencv-python/tree/master/docker, but from a
> python perspective, its the same.
>
> So, my question (as I currently know almost nothing about docker):
>
> a) Does the docker image have all python libraries installed? If so, where
> are they? Is it worth trying to coax cmake 3.9 to find the right one? At
> the moment I do not yet know (and have limited time today), if I can run or
> log into the docker container to find the libraries.
>
> b) Or do we just create our own docker image, with a more up to date
> version of cmake?
>
> If you have any advice, I'd be grateful, as I'm new to wheel building.
>
> Thanks
>
> Matt
>
>
> _______________________________________________
> Wheel-builders mailing list
> Wheel-builders at python.org
> https://mail.python.org/mailman/listinfo/wheel-builders
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/wheel-builders/attachments/20190123/5a003a04/attachment-0001.html>