From jpatton at cs.wisc.edu  Mon Oct  9 16:44:51 2017
From: jpatton at cs.wisc.edu (Jason Patton)
Date: Mon, 09 Oct 2017 15:44:51 -0500
Subject: [C++-sig] Boost.Python and Python limited API
Message-ID: <CAFyOkSUEfd6jTf5p9_T+ETR7ZJ0WVi09V+rfux8qr32O7N=NEg@mail.gmail.com>

Apologies on resurrecting a 7.5 year old topic (
https://mail.python.org/pipermail/cplusplus-sig/2010-March/015390.html
) but does anyone else have experience with building Boost.Python with
the limited API (see: https://docs.python.org/3/c-api/stable.html and
https://www.python.org/dev/peps/pep-0384/ )? My experience suggests
that Boost.Python is not compatible, but I'd like to make sure that
I'm not missing something with my build process:


$ ./bootstrap.sh --with-python=/opt/python/3.6.3/bin/python3.6
--prefix=/opt/boost/1.65.1
Building Boost.Build engine with toolset gcc...
tools/build/src/engine/bin.linuxx86_64/b2
Detecting Python version... 3.6
Detecting Python root... /opt/python/3.6.3
<snip>

$ ./b2 --with-python # compiles with no errors
<snip>

$ ./b2 clean
<snip>
$ ./b2 --with-python define="Py_LIMITED_API=0x03020000"
<snip>
gcc.compile.c++
bin.v2/libs/python/build/gcc-4.8.5/release/threading-multi/list.o
In file included from ./boost/python/object/make_ptr_instance.hpp:8:0,
                 from ./boost/python/to_python_indirect.hpp:11,
                 from ./boost/python/converter/arg_to_python.hpp:10,
                 from ./boost/python/call.hpp:15,
                 from ./boost/python/object_core.hpp:14,
                 from ./boost/python/object.hpp:9,
                 from ./boost/python/list.hpp:10,
                 from libs/python/src/list.cpp:5:
./boost/python/object/make_instance.hpp: In static member function
'static PyObject* boost::python::objects::make_instance_impl<T,
Holder, Derived>::execute(Arg&)':
./boost/python/object/make_instance.hpp:35:36: error: invalid use of
incomplete type 'PyTypeObject {aka struct _typeobject}'
         PyObject* raw_result = type->tp_alloc(
                                    ^
In file included from /opt/python/3.6.3/include/python3.6/Python.h:68:0,
                 from ./boost/python/detail/wrap_python.hpp:151,
                 from ./boost/python/detail/prefix.hpp:13,
                 from ./boost/python/list.hpp:8,
                 from libs/python/src/list.cpp:5:
/opt/python/3.6.3/include/python3.6/object.h:109:12: error: forward
declaration of 'PyTypeObject {aka struct _typeobject}'
     struct _typeobject *ob_type;
            ^
In file included from ./boost/python/cast.hpp:12:0,
                 from ./boost/python/refcount.hpp:9,
                 from ./boost/python/to_python_indirect.hpp:19,
                 from ./boost/python/converter/arg_to_python.hpp:10,
                 from ./boost/python/call.hpp:15,
                 from ./boost/python/object_core.hpp:14,
                 from ./boost/python/object.hpp:9,
                 from ./boost/python/list.hpp:10,
                 from libs/python/src/list.cpp:5:
./boost/python/base_type_traits.hpp: At global scope:
./boost/python/base_type_traits.hpp:36:25: error: 'PyMethodObject' was
not declared in this scope
 struct base_type_traits<PyMethodObject>
                         ^
./boost/python/base_type_traits.hpp:36:39: error: template argument 1 is invalid
 struct base_type_traits<PyMethodObject>
                                       ^
In file included from ./boost/python/converter/object_manager.hpp:10:0,
                 from ./boost/python/to_python_value.hpp:19,
                 from ./boost/python/detail/invoke.hpp:19,
                 from ./boost/python/detail/caller.hpp:16,
                 from ./boost/python/object/function_handle.hpp:8,
                 from ./boost/python/converter/arg_to_python.hpp:19,
                 from ./boost/python/call.hpp:15,
                 from ./boost/python/object_core.hpp:14,
                 from ./boost/python/object.hpp:9,
                 from ./boost/python/list.hpp:10,
                 from libs/python/src/list.cpp:5:
./boost/python/converter/pyobject_traits.hpp:31:40: error:
'PyListObject' was not declared in this scope
     template <> struct pyobject_traits<Py##T##Object>           \
                                        ^
./boost/python/converter/pyobject_traits.hpp:36:1: note: in expansion
of macro 'BOOST_PYTHON_BUILTIN_OBJECT_TRAITS'
 BOOST_PYTHON_BUILTIN_OBJECT_TRAITS(List);
 ^
./boost/python/converter/pyobject_traits.hpp:31:53: error: template
argument 1 is invalid
     template <> struct pyobject_traits<Py##T##Object>           \
                                                     ^
./boost/python/converter/pyobject_traits.hpp:36:1: note: in expansion
of macro 'BOOST_PYTHON_BUILTIN_OBJECT_TRAITS'
 BOOST_PYTHON_BUILTIN_OBJECT_TRAITS(List);
 ^
./boost/python/converter/pyobject_traits.hpp:32:25: error:
'PyListObject' was not declared in this scope
         : pyobject_type<Py##T##Object, &Py##T##_Type> {}
                         ^
./boost/python/converter/pyobject_traits.hpp:36:1: note: in expansion
of macro 'BOOST_PYTHON_BUILTIN_OBJECT_TRAITS'
 BOOST_PYTHON_BUILTIN_OBJECT_TRAITS(List);
 ^
./boost/python/converter/pyobject_traits.hpp:32:53: error: template
argument 1 is invalid
         : pyobject_type<Py##T##Object, &Py##T##_Type> {}
                                                     ^
./boost/python/converter/pyobject_traits.hpp:36:1: note: in expansion
of macro 'BOOST_PYTHON_BUILTIN_OBJECT_TRAITS'
 BOOST_PYTHON_BUILTIN_OBJECT_TRAITS(List);
 ^
./boost/python/converter/pyobject_traits.hpp:31:40: error:
'PyDictObject' was not declared in this scope
     template <> struct pyobject_traits<Py##T##Object>           \
                                        ^
./boost/python/converter/pyobject_traits.hpp:41:1: note: in expansion
of macro 'BOOST_PYTHON_BUILTIN_OBJECT_TRAITS'
 BOOST_PYTHON_BUILTIN_OBJECT_TRAITS(Dict);
 ^
./boost/python/converter/pyobject_traits.hpp:31:53: error: template
argument 1 is invalid
     template <> struct pyobject_traits<Py##T##Object>           \
                                                     ^
./boost/python/converter/pyobject_traits.hpp:41:1: note: in expansion
of macro 'BOOST_PYTHON_BUILTIN_OBJECT_TRAITS'
 BOOST_PYTHON_BUILTIN_OBJECT_TRAITS(Dict);
 ^
./boost/python/converter/pyobject_traits.hpp:32:25: error:
'PyDictObject' was not declared in this scope
         : pyobject_type<Py##T##Object, &Py##T##_Type> {}
                         ^
./boost/python/converter/pyobject_traits.hpp:41:1: note: in expansion
of macro 'BOOST_PYTHON_BUILTIN_OBJECT_TRAITS'
 BOOST_PYTHON_BUILTIN_OBJECT_TRAITS(Dict);
 ^
./boost/python/converter/pyobject_traits.hpp:32:53: error: template
argument 1 is invalid
         : pyobject_type<Py##T##Object, &Py##T##_Type> {}
                                                     ^
./boost/python/converter/pyobject_traits.hpp:41:1: note: in expansion
of macro 'BOOST_PYTHON_BUILTIN_OBJECT_TRAITS'
 BOOST_PYTHON_BUILTIN_OBJECT_TRAITS(Dict);
 ^
./boost/python/converter/pyobject_traits.hpp:31:40: error:
'PyTupleObject' was not declared in this scope
     template <> struct pyobject_traits<Py##T##Object>           \
                                        ^
./boost/python/converter/pyobject_traits.hpp:42:1: note: in expansion
of macro 'BOOST_PYTHON_BUILTIN_OBJECT_TRAITS'
 BOOST_PYTHON_BUILTIN_OBJECT_TRAITS(Tuple);
 ^
./boost/python/converter/pyobject_traits.hpp:31:53: error: template
argument 1 is invalid
     template <> struct pyobject_traits<Py##T##Object>           \
                                                     ^
./boost/python/converter/pyobject_traits.hpp:42:1: note: in expansion
of macro 'BOOST_PYTHON_BUILTIN_OBJECT_TRAITS'
 BOOST_PYTHON_BUILTIN_OBJECT_TRAITS(Tuple);
 ^
./boost/python/converter/pyobject_traits.hpp:32:25: error:
'PyTupleObject' was not declared in this scope
         : pyobject_type<Py##T##Object, &Py##T##_Type> {}
                         ^
./boost/python/converter/pyobject_traits.hpp:42:1: note: in expansion
of macro 'BOOST_PYTHON_BUILTIN_OBJECT_TRAITS'
 BOOST_PYTHON_BUILTIN_OBJECT_TRAITS(Tuple);
 ^
In file included from ./boost/python/object/function_handle.hpp:8:0,
                 from ./boost/python/converter/arg_to_python.hpp:19,
                 from ./boost/python/call.hpp:15,
                 from ./boost/python/object_core.hpp:14,
                 from ./boost/python/object.hpp:9,
                 from ./boost/python/list.hpp:10,
                 from libs/python/src/list.cpp:5:
./boost/python/detail/caller.hpp: In function 'PyObject*
boost::python::detail::get(mpl_::int_<BOOST_PP_ITERATION_0>, PyObject*
const&)':
./boost/python/detail/caller.hpp:48:36: error: there are no arguments
to 'PyTuple_GET_ITEM' that depend on a template parameter, so a
declaration of 'PyTuple_GET_ITEM' must be available [-fpermissive]
     return PyTuple_GET_ITEM(args_,N);
                                    ^
./boost/python/detail/caller.hpp:48:36: note: (if you use
'-fpermissive', G++ will accept your code, but allowing the use of an
undeclared name is deprecated)
./boost/python/detail/caller.hpp: In function 'unsigned int
boost::python::detail::arity(PyObject* const&)':
./boost/python/detail/caller.hpp:53:34: error: 'PyTuple_GET_SIZE' was
not declared in this scope
     return PyTuple_GET_SIZE(args_);
                                  ^

    "g++"   -O3 -finline-functions -Wno-inline -Wall -pthread -fPIC
-m64  -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DNDEBUG
-DPy_LIMITED_API=0x03020000  -I"."
-I"/opt/python/3.6.3/include/python3.6" -c -o
"bin.v2/libs/python/build/gcc-4.8.5/release/threading-multi/list.o"
"libs/python/src/list.cpp"

...failed gcc.compile.c++
bin.v2/libs/python/build/gcc-4.8.5/release/threading-multi/list.o...
<snip a bunch more errors>


And so libboost_python3 is never built, e.g.:
...skipped <pbin.v2/libs/python/build/gcc-4.8.5/release/link-static/threading-multi>libboost_python3.a
for lack of <pbin.v2/libs/python/build/gcc-4.8.5/release/link-static/threading-multi>list.o...


I can attach the rest of the output if anyone finds some use, let me know.

Thanks

Jason Patton

From stefan at seefeld.name  Tue Oct 10 23:51:08 2017
From: stefan at seefeld.name (Stefan Seefeld)
Date: Tue, 10 Oct 2017 23:51:08 -0400
Subject: [C++-sig] Boost.Python and Python limited API
In-Reply-To: <CAFyOkSUEfd6jTf5p9_T+ETR7ZJ0WVi09V+rfux8qr32O7N=NEg@mail.gmail.com>
References: <CAFyOkSUEfd6jTf5p9_T+ETR7ZJ0WVi09V+rfux8qr32O7N=NEg@mail.gmail.com>
Message-ID: <6abe1680-cd1b-c36c-a511-7f24dce64b4e@seefeld.name>

On 09.10.2017 16:44, Jason Patton wrote:
> Apologies on resurrecting a 7.5 year old topic (
> https://mail.python.org/pipermail/cplusplus-sig/2010-March/015390.html
> ) but does anyone else have experience with building Boost.Python with
> the limited API (see: https://docs.python.org/3/c-api/stable.html and
> https://www.python.org/dev/peps/pep-0384/ )? My experience suggests
> that Boost.Python is not compatible, but I'd like to make sure that
> I'm not missing something with my build process:

[...]

I'm not sure what you mean by "compatible", but indeed, Boost.Python
right now doesn't provide a build variant that generates bindings for
that stable ABI/API.
That being said, I'd be happy to assist anyone who wants to work on
patches towards that goal, if there is enough interest. Please open an
issue on https://github.com/boostorg/python/issues so we can discuss
details.

Regards,
Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20171010/04dc542f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.png
Type: image/png
Size: 3642 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20171010/04dc542f/attachment-0001.png>