[pypy-commit] pypy issue2444: merge default into branch
mattip
pypy.commits at gmail.com
Sun Dec 18 23:21:36 EST 2016
Author: Matti Picus <matti.picus at gmail.com>
Branch: issue2444
Changeset: r89174:b3b99a26e2b9
Date: 2016-12-18 23:42 +0200
http://bitbucket.org/pypy/pypy/changeset/b3b99a26e2b9/
Log: merge default into branch
diff too long, truncating to 2000 out of 15848 lines
diff --git a/.hgignore b/.hgignore
--- a/.hgignore
+++ b/.hgignore
@@ -77,3 +77,5 @@
^.hypothesis/
^release/
^rpython/_cache$
+
+pypy/module/cppyy/.+/*\.pcm
diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
--- a/pypy/config/pypyoption.py
+++ b/pypy/config/pypyoption.py
@@ -190,6 +190,12 @@
"make sure that all calls go through space.call_args",
default=False),
+ BoolOption("disable_entrypoints",
+ "Disable external entry points, notably the"
+ " cpyext module and cffi's embedding mode.",
+ default=False,
+ requires=[("objspace.usemodules.cpyext", False)]),
+
OptionDescription("std", "Standard Object Space Options", [
BoolOption("withtproxy", "support transparent proxies",
default=True),
diff --git a/pypy/doc/contributor.rst b/pypy/doc/contributor.rst
--- a/pypy/doc/contributor.rst
+++ b/pypy/doc/contributor.rst
@@ -1,3 +1,9 @@
+#encoding utf-8
+
+Contributors
+------------
+::
+
Armin Rigo
Maciej Fijalkowski
Carl Friedrich Bolz
@@ -307,7 +313,7 @@
Mads Kiilerich
Antony Lee
Jason Madden
- Daniel Neuh�user
+ Daniel Neuhäuser
reubano at gmail.com
Yaroslav Fedevych
Jim Hunziker
diff --git a/pypy/doc/cppyy.rst b/pypy/doc/cppyy.rst
--- a/pypy/doc/cppyy.rst
+++ b/pypy/doc/cppyy.rst
@@ -1,145 +1,61 @@
cppyy: C++ bindings for PyPy
============================
-The cppyy module creates, at run-time, Python-side classes and functions for
-C++, by querying a C++ reflection system.
-The default system used is `Reflex`_, which extracts the needed information
-from C++ header files.
-Another current backend is based on `CINT`_, and yet another, more important
-one for the medium- to long-term will be based on `cling`_.
-The latter sits on top of `llvm`_'s `clang`_, and will therefore allow the use
-of C++11.
-The work on the cling backend has so far been done only for CPython, but
-bringing it to PyPy is a lot less work than developing it in the first place.
+The cppyy module delivers dynamic Python-C++ bindings.
+It is designed for automation, high performance, scale, interactivity, and
+handling all of modern C++ (11, 14, etc.).
+It is based on `Cling`_ which, through `LLVM`_/`clang`_, provides C++
+reflection and interactivity.
+Reflection information is extracted from C++ header files.
+Cppyy itself is built into PyPy (an alternative exists for CPython), but
+it requires a `backend`_, installable through pip, to interface with Cling.
-.. _Reflex: https://root.cern.ch/how/how-use-reflex
-.. _CINT: https://root.cern.ch/introduction-cint
-.. _cling: https://root.cern.ch/cling
-.. _llvm: http://llvm.org/
+.. _Cling: https://root.cern.ch/cling
+.. _LLVM: http://llvm.org/
.. _clang: http://clang.llvm.org/
-
-This document describes the version of cppyy that lives in the main branch of
-PyPy.
-The development of cppyy happens in the "reflex-support" branch.
-
-
-Motivation
-----------
-
-To provide bindings to another language in CPython, you program to a
-generic C-API that exposes many of the interpreter features.
-With PyPy, however, there is no such generic C-API, because several of the
-interpreter features (e.g. the memory model) are pluggable and therefore
-subject to change.
-Furthermore, a generic API does not allow any assumptions about the calls
-into another language, forcing the JIT to behave conservatively around these
-calls and with the objects that cross language boundaries.
-In contrast, cppyy does not expose an API, but expects one to be implemented
-by a backend.
-It makes strong assumptions about the semantics of the API that it uses and
-that in turn allows the JIT to make equally strong assumptions.
-This is possible, because the expected API is only for providing C++ language
-bindings, and does not provide generic programmability.
-
-The cppyy module further offers two features, which result in improved
-performance as well as better functionality and cross-language integration.
-First, cppyy itself is written in RPython and therefore open to optimizations
-by the JIT up until the actual point of call into C++.
-This means for example, that if variables are already unboxed by the JIT, they
-can be passed through directly to C++.
-Second, a backend such as Reflex (and cling far more so) adds dynamic features
-to C++, thus greatly reducing impedance mismatches between the two languages.
-For example, Reflex is dynamic enough to allow writing runtime bindings
-generation in python (as opposed to RPython) and this is used to create very
-natural "pythonizations" of the bound code.
-As another example, cling allows automatic instantiations of templates.
-
-See this description of the `cppyy architecture`_ for further details.
-
-.. _cppyy architecture: http://morepypy.blogspot.com/2012/06/architecture-of-cppyy.html
+.. _backend: https://pypi.python.org/pypi/PyPy-cppyy-backend
Installation
------------
-There are two ways of using cppyy, and the choice depends on how pypy-c was
-built: the backend can be builtin, or dynamically loadable.
-The former has the disadvantage of requiring pypy-c to be linked with external
-C++ libraries (e.g. libReflex.so), but has the advantage of being faster in
-some cases.
-That advantage will disappear over time, however, with improvements in the
-JIT.
-Therefore, this document assumes that the dynamically loadable backend is
-chosen (it is, by default).
-See the :doc:`backend documentation <cppyy_backend>`.
+This assumes PyPy2.7 v5.7 or later; earlier versions use a Reflex-based cppyy
+module, which is no longer supported.
+Both the tooling and user-facing Python codes are very backwards compatible,
+however.
+Further dependencies are cmake (for general build), Python2.7 (for LLVM), and
+a modern C++ compiler (one that supports at least C++11).
-A standalone version of Reflex that also provides the dynamically loadable
-backend is available for `download`_. Note this is currently the only way to
-get the dynamically loadable backend, so use this first.
+Assuming you have a recent enough version of PyPy installed, use pip to
+complete the installation of cppyy::
-That version, as well as any other distribution of Reflex (e.g. the one that
-comes with `ROOT`_, which may be part of your Linux distribution as part of
-the selection of scientific software) will also work for a build with the
-builtin backend.
+ $ MAKE_NPROCS=4 pypy-c -m pip install --verbose PyPy-cppyy-backend
-.. _download: http://cern.ch/wlav/reflex-2014-10-20.tar.bz2
-.. _ROOT: http://root.cern.ch/
+Set the number of parallel builds ('4' in this example, through the MAKE_NPROCS
+environment variable) to a number appropriate for your machine.
+The building process may take quite some time as it includes a customized
+version of LLVM as part of Cling, which is why --verbose is recommended so that
+you can see the build progress.
-Besides Reflex, you probably need a version of `gccxml`_ installed, which is
-most easily provided by the packager of your system.
-If you read up on gccxml, you will probably notice that it is no longer being
-developed and hence will not provide C++11 support.
-That's why the medium term plan is to move to cling.
-Note that gccxml is only needed to generate reflection libraries.
-It is not needed to use them.
-
-.. _gccxml: http://www.gccxml.org
-
-To install the standalone version of Reflex, after download::
-
- $ tar jxf reflex-2014-10-20.tar.bz2
- $ cd reflex-2014-10-20
- $ ./build/autogen
- $ ./configure <usual set of options such as --prefix>
- $ make && make install
-
-The usual rules apply: <prefix>/bin needs to be added to the ``PATH`` and
-<prefix>/lib to the ``LD_LIBRARY_PATH`` environment variable.
-For convenience, this document will assume that there is a ``REFLEXHOME``
-variable that points to <prefix>.
-If you downloaded or built the whole of ROOT, ``REFLEXHOME`` should be equal
-to ``ROOTSYS``.
-
-The following is optional, and is only to show how pypy-c can be build
-:doc:`from source <build>`, for example to get at the main development branch of cppyy.
-The :doc:`backend documentation <cppyy_backend>` has more details on the backend-specific
-prerequisites.
-
-Then run the translation to build ``pypy-c``::
-
- $ hg clone https://bitbucket.org/pypy/pypy
- $ cd pypy
- $ hg up reflex-support # optional
-
- # This example shows python, but using pypy-c is faster and uses less memory
- $ python rpython/bin/rpython --opt=jit pypy/goal/targetpypystandalone --withmod-cppyy
-
-This will build a ``pypy-c`` that includes the cppyy module, and through that,
-Reflex support.
-Of course, if you already have a pre-built version of the ``pypy`` interpreter,
-you can use that for the translation rather than ``python``.
-If not, you may want :ref:`to obtain a binary distribution <prebuilt-pypy>` to speed up the
-translation step.
+The default installation will be under
+$PYTHONHOME/site-packages/cppyy_backend/lib,
+which needs to be added to your dynamic loader path (LD_LIBRARY_PATH).
+If you need the dictionary and class map generation tools (used in the examples
+below), you need to add $PYTHONHOME/site-packages/cppyy_backend/bin to your
+executable path (PATH).
Basic bindings example
----------------------
-Now test with a trivial example whether all packages are properly installed
-and functional.
-First, create a C++ header file with some class in it (note that all functions
-are made inline for convenience; a real-world example would of course have a
-corresponding source file)::
+These examples assume that cppyy_backend is pointed to by the environment
+variable CPPYYHOME, and that CPPYYHOME/lib is added to LD_LIBRARY_PATH and
+CPPYYHOME/bin to PATH.
+
+Let's first test with a trivial example whether all packages are properly
+installed and functional.
+Create a C++ header file with some class in it (all functions are made inline
+for convenience; if you have out-of-line code, link with it as appropriate)::
$ cat MyClass.h
class MyClass {
@@ -153,11 +69,11 @@
int m_myint;
};
-Then, generate the bindings using ``genreflex`` (part of ROOT), and compile the
-code::
+Then, generate the bindings using ``genreflex`` (installed under
+cppyy_backend/bin in site_packages), and compile the code::
$ genreflex MyClass.h
- $ g++ -fPIC -rdynamic -O2 -shared -I$REFLEXHOME/include MyClass_rflx.cpp -o libMyClassDict.so -L$REFLEXHOME/lib -lReflex
+ $ g++ -std=c++11 -fPIC -rdynamic -O2 -shared -I$CPPYYHOME/include MyClass_rflx.cpp -o libMyClassDict.so -L$CPPYYHOME/lib -lCling
Next, make sure that the library can be found through the dynamic lookup path
(the ``LD_LIBRARY_PATH`` environment variable on Linux, ``PATH`` on Windows),
@@ -209,7 +125,7 @@
For example::
$ genreflex MyClass.h --rootmap=libMyClassDict.rootmap --rootmap-lib=libMyClassDict.so
- $ g++ -fPIC -rdynamic -O2 -shared -I$REFLEXHOME/include MyClass_rflx.cpp -o libMyClassDict.so -L$REFLEXHOME/lib -lReflex
+ $ g++ -std=c++11 -fPIC -rdynamic -O2 -shared -I$CPPYYHOME/include MyClass_rflx.cpp -o libMyClassDict.so -L$CPPYYHOME/lib -lCling
where the first option (``--rootmap``) specifies the output file name, and the
second option (``--rootmap-lib``) the name of the reflection library where
@@ -311,7 +227,7 @@
Now the reflection info can be generated and compiled::
$ genreflex MyAdvanced.h --selection=MyAdvanced.xml
- $ g++ -fPIC -rdynamic -O2 -shared -I$REFLEXHOME/include MyAdvanced_rflx.cpp -o libAdvExDict.so -L$REFLEXHOME/lib -lReflex
+ $ g++ -std=c++11 -fPIC -rdynamic -O2 -shared -I$CPPYYHOME/include MyAdvanced_rflx.cpp -o libAdvExDict.so -L$CPPYYHOME/lib -lCling
and subsequently be used from PyPy::
@@ -370,7 +286,7 @@
bound using::
$ genreflex example.h --deep --rootmap=libexampleDict.rootmap --rootmap-lib=libexampleDict.so
- $ g++ -fPIC -rdynamic -O2 -shared -I$REFLEXHOME/include example_rflx.cpp -o libexampleDict.so -L$REFLEXHOME/lib -lReflex
+ $ g++ -std=c++11 -fPIC -rdynamic -O2 -shared -I$CPPYYHOME/include example_rflx.cpp -o libexampleDict.so -L$CPPYYHOME/lib -lCling
* **abstract classes**: Are represented as python classes, since they are
needed to complete the inheritance hierarchies, but will raise an exception
@@ -666,13 +582,10 @@
Templates
---------
-A bit of special care needs to be taken for the use of templates.
-For a templated class to be completely available, it must be guaranteed that
-said class is fully instantiated, and hence all executable C++ code is
-generated and compiled in.
-The easiest way to fulfill that guarantee is by explicit instantiation in the
-header file that is handed to ``genreflex``.
-The following example should make that clear::
+Templates can be automatically instantiated, assuming the appropriate header
+files have been loaded or are accessible to the class loader.
+This is the case for example for all of STL.
+For example::
$ cat MyTemplate.h
#include <vector>
@@ -686,68 +599,10 @@
int m_i;
};
- #ifdef __GCCXML__
- template class std::vector<MyClass>; // explicit instantiation
- #endif
-
-If you know for certain that all symbols will be linked in from other sources,
-you can also declare the explicit template instantiation ``extern``.
-An alternative is to add an object to an unnamed namespace::
-
- namespace {
- std::vector<MyClass> vmc;
- } // unnamed namespace
-
-Unfortunately, this is not always enough for gcc.
-The iterators of vectors, if they are going to be used, need to be
-instantiated as well, as do the comparison operators on those iterators, as
-these live in an internal namespace, rather than in the iterator classes.
-Note that you do NOT need this iterators to iterator over a vector.
-You only need them if you plan to explicitly call e.g. ``begin`` and ``end``
-methods, and do comparisons of iterators.
-One way to handle this, is to deal with this once in a macro, then reuse that
-macro for all ``vector`` classes.
-Thus, the header above needs this (again protected with
-``#ifdef __GCCXML__``), instead of just the explicit instantiation of the
-``vector<MyClass>``::
-
- #define STLTYPES_EXPLICIT_INSTANTIATION_DECL(STLTYPE, TTYPE) \
- template class std::STLTYPE< TTYPE >; \
- template class __gnu_cxx::__normal_iterator<TTYPE*, std::STLTYPE< TTYPE > >; \
- template class __gnu_cxx::__normal_iterator<const TTYPE*, std::STLTYPE< TTYPE > >;\
- namespace __gnu_cxx { \
- template bool operator==(const std::STLTYPE< TTYPE >::iterator&, \
- const std::STLTYPE< TTYPE >::iterator&); \
- template bool operator!=(const std::STLTYPE< TTYPE >::iterator&, \
- const std::STLTYPE< TTYPE >::iterator&); \
- }
-
- STLTYPES_EXPLICIT_INSTANTIATION_DECL(vector, MyClass)
-
-Then, still for gcc, the selection file needs to contain the full hierarchy as
-well as the global overloads for comparisons for the iterators::
-
- $ cat MyTemplate.xml
- <lcgdict>
- <class pattern="std::vector<*>" />
- <class pattern="std::vector<*>::iterator" />
- <function name="__gnu_cxx::operator=="/>
- <function name="__gnu_cxx::operator!="/>
-
- <class name="MyClass" />
- </lcgdict>
-
Run the normal ``genreflex`` and compilation steps::
$ genreflex MyTemplate.h --selection=MyTemplate.xml
- $ g++ -fPIC -rdynamic -O2 -shared -I$REFLEXHOME/include MyTemplate_rflx.cpp -o libTemplateDict.so -L$REFLEXHOME/lib -lReflex
-
-Note: this is a dirty corner that clearly could do with some automation,
-even if the macro already helps.
-Such automation is planned.
-In fact, in the Cling world, the backend can perform the template
-instantations and generate the reflection info on the fly, and none of the
-above will any longer be necessary.
+ $ g++ -std=c++11 -fPIC -rdynamic -O2 -shared -I$CPPYYHOME/include MyTemplate_rflx.cpp -o libTemplateDict.so -L$CPPYYHOME/lib -lCling
Subsequent use should be as expected.
Note the meta-class style of "instantiating" the template::
@@ -764,8 +619,6 @@
1 2 3
>>>>
-Other templates work similarly, but are typically simpler, as there are no
-similar issues with iterators for e.g. ``std::list``.
The arguments to the template instantiation can either be a string with the
full list of arguments, or the explicit classes.
The latter makes for easier code writing if the classes passed to the
@@ -775,95 +628,40 @@
The fast lane
-------------
-The following is an experimental feature of cppyy.
-It mostly works, but there are some known issues (e.g. with return-by-value).
-Soon it should be the default mode, however.
+By default, cppyy will use direct function pointers through `CFFI`_ whenever
+possible. If this causes problems for you, you can disable it by setting the
+CPPYY_DISABLE_FASTPATH environment variable.
-With a slight modification of Reflex, it can provide function pointers for
-C++ methods, and hence allow PyPy to call those pointers directly, rather than
-calling C++ through a Reflex stub.
+.. _CFFI: https://cffi.readthedocs.io/en/latest/
-The standalone version of Reflex `provided`_ has been patched, but if you get
-Reflex from another source (most likely with a ROOT distribution), locate the
-file `genreflex-methptrgetter.patch`_ in pypy/module/cppyy and apply it to
-the genreflex python scripts found in ``$ROOTSYS/lib``::
-
- $ cd $ROOTSYS/lib
- $ patch -p2 < genreflex-methptrgetter.patch
-
-With this patch, ``genreflex`` will have grown the ``--with-methptrgetter``
-option.
-Use this option when running ``genreflex``, and add the
-``-Wno-pmf-conversions`` option to ``g++`` when compiling.
-The rest works the same way: the fast path will be used transparently (which
-also means that you can't actually find out whether it is in use, other than
-by running a micro-benchmark or a JIT test).
-
-.. _provided: http://cern.ch/wlav/reflex-2014-10-20.tar.bz2
-.. _genreflex-methptrgetter.patch: https://bitbucket.org/pypy/pypy/src/default/pypy/module/cppyy/genreflex-methptrgetter.patch
CPython
-------
-Most of the ideas in cppyy come originally from the `PyROOT`_ project.
-Although PyROOT does not support Reflex directly, it has an alter ego called
-"PyCintex" that, in a somewhat roundabout way, does.
-If you installed ROOT, rather than just Reflex, PyCintex should be available
-immediately if you add ``$ROOTSYS/lib`` to the ``PYTHONPATH`` environment
-variable.
+Most of the ideas in cppyy come originally from the `PyROOT`_ project, which
+contains a CPython-based cppyy.py module (with similar dependencies as the
+one that comes with PyPy).
+A standalone pip-installable version is planned, but for now you can install
+ROOT through your favorite distribution installer (available in the science
+section).
.. _PyROOT: https://root.cern.ch/pyroot
-There are a couple of minor differences between PyCintex and cppyy, most to do
-with naming.
-The one that you will run into directly, is that PyCintex uses a function
-called ``loadDictionary`` rather than ``load_reflection_info`` (it has the
-same rootmap-based class loader functionality, though, making this point
-somewhat moot).
-The reason for this is that Reflex calls the shared libraries that contain
-reflection info "dictionaries."
-However, in python, the name `dictionary` already has a well-defined meaning,
-so a more descriptive name was chosen for cppyy.
-In addition, PyCintex requires that the names of shared libraries so loaded
-start with "lib" in their name.
-The basic example above, rewritten for PyCintex thus goes like this::
-
- $ python
- >>> import PyCintex
- >>> PyCintex.loadDictionary("libMyClassDict.so")
- >>> myinst = PyCintex.gbl.MyClass(42)
- >>> print myinst.GetMyInt()
- 42
- >>> myinst.SetMyInt(33)
- >>> print myinst.m_myint
- 33
- >>> myinst.m_myint = 77
- >>> print myinst.GetMyInt()
- 77
- >>> help(PyCintex.gbl.MyClass) # shows that normal python introspection works
-
-Other naming differences are such things as taking an address of an object.
-In PyCintex, this is done with ``AddressOf`` whereas in cppyy the choice was
-made to follow the naming as in ``ctypes`` and hence use ``addressof``
-(PyROOT/PyCintex predate ``ctypes`` by several years, and the ROOT project
-follows camel-case, hence the differences).
-
-Of course, this is python, so if any of the naming is not to your liking, all
-you have to do is provide a wrapper script that you import instead of
-importing the ``cppyy`` or ``PyCintex`` modules directly.
-In that wrapper script you can rename methods exactly the way you need it.
-
-In the cling world, all these differences will be resolved.
+There are a couple of minor differences between the two versions of cppyy
+(the CPython version has a few more features).
+Work is on-going to integrate the nightly tests of both to make sure their
+feature sets are equalized.
Python3
-------
-To change versions of CPython (to Python3, another version of Python, or later
-to the `Py3k`_ version of PyPy), the only part that requires recompilation is
-the bindings module, be it ``cppyy`` or ``libPyROOT.so`` (in PyCintex).
-Although ``genreflex`` is indeed a Python tool, the generated reflection
-information is completely independent of Python.
+The CPython version of cppyy supports Python3, assuming your packager has
+build the backend for it.
+The cppyy module has not been tested with the `Py3k`_ version of PyPy.
+Note that the generated reflection information (from ``genreflex``) is fully
+independent of Python, and does not need to be rebuild when switching versions
+or interpreters.
.. _Py3k: https://bitbucket.org/pypy/pypy/src/py3k
@@ -871,5 +669,4 @@
.. toctree::
:hidden:
- cppyy_backend
cppyy_example
diff --git a/pypy/doc/cppyy_backend.rst b/pypy/doc/cppyy_backend.rst
deleted file mode 100644
--- a/pypy/doc/cppyy_backend.rst
+++ /dev/null
@@ -1,45 +0,0 @@
-Backends for cppyy
-==================
-
-The cppyy module needs a backend to provide the C++ reflection information on
-which the Python bindings are build.
-The backend is called through a C-API, which can be found in the PyPy sources
-in: :source:`pypy/module/cppyy/include/capi.h`.
-There are two kinds of API calls: querying about reflection information, which
-are used during the creation of Python-side constructs, and making the actual
-calls into C++.
-The objects passed around are all opaque: cppyy does not make any assumptions
-about them, other than that the opaque handles can be copied.
-Their definition, however, appears in two places: in the C code (in capi.h),
-and on the RPython side (in :source:`capi_types.py <pypy/module/cppyy/capi/capi_types.py>`), so if they are changed, they
-need to be changed on both sides.
-
-There are two places where selections in the RPython code affect the choice
-(and use) of the backend.
-The first is in :source:`pypy/module/cppyy/capi/__init__.py`::
-
- # choose C-API access method:
- from pypy.module.cppyy.capi.loadable_capi import *
- #from pypy.module.cppyy.capi.builtin_capi import *
-
-The default is the loadable C-API.
-Comment it and uncomment the builtin C-API line, to use the builtin version.
-
-Next, if the builtin C-API is chosen, the specific backend needs to be set as
-well (default is Reflex).
-This second choice is in :source:`pypy/module/cppyy/capi/builtin_capi.py`::
-
- import reflex_capi as backend
- #import cint_capi as backend
-
-After those choices have been made, built pypy-c as usual.
-
-When building pypy-c from source, keep the following in mind.
-If the loadable_capi is chosen, no further prerequisites are needed.
-However, for the build of the builtin_capi to succeed, the ``ROOTSYS``
-environment variable must point to the location of your ROOT (or standalone
-Reflex in the case of the Reflex backend) installation, or the ``root-config``
-utility must be accessible through ``$PATH`` (e.g. by adding ``$ROOTSYS/bin``
-to ``PATH``).
-In case of the former, include files are expected under ``$ROOTSYS/include``
-and libraries under ``$ROOTSYS/lib``.
diff --git a/pypy/doc/extending.rst b/pypy/doc/extending.rst
--- a/pypy/doc/extending.rst
+++ b/pypy/doc/extending.rst
@@ -12,7 +12,7 @@
* Write them in pure Python and use ctypes_.
-* Write them in C++ and bind them through Reflex_.
+* Write them in C++ and bind them through :doc:`cppyy <cppyy>` using Cling.
* Write them in as `RPython mixed modules`_.
@@ -61,11 +61,11 @@
.. _libffi: http://sourceware.org/libffi/
-Reflex
-------
+Cling and cppyy
+---------------
The builtin :doc:`cppyy <cppyy>` module uses reflection information, provided by
-`Reflex`_ (which needs to be `installed separately`_), of C/C++ code to
+`Cling`_ (which needs to be `installed separately`_), of C/C++ code to
automatically generate bindings at runtime.
In Python, classes and functions are always runtime structures, so when they
are generated matters not for performance.
@@ -76,11 +76,14 @@
The :doc:`cppyy <cppyy>` module is written in RPython, thus PyPy's JIT is able to remove
most cross-language call overhead.
-:doc:`Full details <cppyy>` are `available here <cppyy>`.
+:doc:Full details are `available here <cppyy>`.
-.. _installed separately: http://cern.ch/wlav/reflex-2013-08-14.tar.bz2
-.. _Reflex: https://root.cern.ch/how/how-use-reflex
+.. _installed separately: https://pypi.python.org/pypi/PyPy-cppyy-backend
+.. _Cling: https://root.cern.ch/cling
+.. toctree::
+
+ cppyy
RPython Mixed Modules
---------------------
@@ -94,7 +97,3 @@
This is how the numpy module is being developed.
-.. toctree::
- :hidden:
-
- cppyy
diff --git a/pypy/doc/index-of-release-notes.rst b/pypy/doc/index-of-release-notes.rst
--- a/pypy/doc/index-of-release-notes.rst
+++ b/pypy/doc/index-of-release-notes.rst
@@ -59,6 +59,7 @@
.. toctree::
+ release-pypy3.3-v5.5.0.rst
release-pypy3.3-v5.2-alpha1.rst
CPython 3.2 compatible versions
diff --git a/pypy/doc/whatsnew-head.rst b/pypy/doc/whatsnew-head.rst
--- a/pypy/doc/whatsnew-head.rst
+++ b/pypy/doc/whatsnew-head.rst
@@ -45,3 +45,22 @@
Assign ``tp_doc`` to the new TypeObject's type dictionary ``__doc__`` key
so it will be picked up by app-level objects of that type
+
+.. branch: cling-support
+
+Module cppyy now uses cling as its backend (Reflex has been removed). The
+user-facing interface and main developer tools (genreflex, selection files,
+class loader, etc.) remain the same. A libcppyy_backend.so library is still
+needed but is now available through PyPI with pip: PyPy-cppyy-backend.
+
+The Cling-backend brings support for modern C++ (11, 14, etc.), dynamic
+template instantations, and improved integration with CFFI for better
+performance. It also provides interactive C++ (and bindings to that).
+
+.. branch: better-PyDict_Next
+
+Improve the performance of ``PyDict_Next``. When trying ``PyDict_Next`` on a
+typedef dict, the test exposed a problem converting a ``GetSetProperty`` to a
+``PyGetSetDescrObject``. The other direction seem to be fully implemented.
+This branch made a minimal effort to convert the basic fields to avoid
+segfaults, but trying to use the ``PyGetSetDescrObject`` will probably fail.
diff --git a/pypy/goal/targetpypystandalone.py b/pypy/goal/targetpypystandalone.py
--- a/pypy/goal/targetpypystandalone.py
+++ b/pypy/goal/targetpypystandalone.py
@@ -83,12 +83,18 @@
return 1
return exitcode
+ return entry_point, get_additional_entrypoints(space, w_initstdio)
+
+
+def get_additional_entrypoints(space, w_initstdio):
# register the minimal equivalent of running a small piece of code. This
# should be used as sparsely as possible, just to register callbacks
-
from rpython.rlib.entrypoint import entrypoint_highlevel
from rpython.rtyper.lltypesystem import rffi, lltype
+ if space.config.objspace.disable_entrypoints:
+ return {}
+
@entrypoint_highlevel('main', [rffi.CCHARP, rffi.INT],
c_name='pypy_setup_home')
def pypy_setup_home(ll_home, verbose):
@@ -188,11 +194,11 @@
return -1
return 0
- return entry_point, {'pypy_execute_source': pypy_execute_source,
- 'pypy_execute_source_ptr': pypy_execute_source_ptr,
- 'pypy_init_threads': pypy_init_threads,
- 'pypy_thread_attach': pypy_thread_attach,
- 'pypy_setup_home': pypy_setup_home}
+ return {'pypy_execute_source': pypy_execute_source,
+ 'pypy_execute_source_ptr': pypy_execute_source_ptr,
+ 'pypy_init_threads': pypy_init_threads,
+ 'pypy_thread_attach': pypy_thread_attach,
+ 'pypy_setup_home': pypy_setup_home}
# _____ Define and setup target ___
diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
--- a/pypy/interpreter/baseobjspace.py
+++ b/pypy/interpreter/baseobjspace.py
@@ -428,6 +428,8 @@
make_finalizer_queue(W_Root, self)
self._code_of_sys_exc_info = None
+ self._builtin_functions_by_identifier = {'': None}
+
# can be overridden to a subclass
self.initialize()
diff --git a/pypy/interpreter/function.py b/pypy/interpreter/function.py
--- a/pypy/interpreter/function.py
+++ b/pypy/interpreter/function.py
@@ -247,16 +247,15 @@
def descr_function_repr(self):
return self.getrepr(self.space, 'function %s' % (self.name,))
- # delicate
- _all = {'': None}
def _cleanup_(self):
+ # delicate
from pypy.interpreter.gateway import BuiltinCode
if isinstance(self.code, BuiltinCode):
# we have been seen by other means so rtyping should not choke
# on us
identifier = self.code.identifier
- previous = Function._all.get(identifier, self)
+ previous = self.space._builtin_functions_by_identifier.get(identifier, self)
assert previous is self, (
"duplicate function ids with identifier=%r: %r and %r" % (
identifier, previous, self))
@@ -264,10 +263,10 @@
return False
def add_to_table(self):
- Function._all[self.code.identifier] = self
+ self.space._builtin_functions_by_identifier[self.code.identifier] = self
- def find(identifier):
- return Function._all[identifier]
+ def find(space, identifier):
+ return space._builtin_functions_by_identifier[identifier]
find = staticmethod(find)
def descr_function__reduce__(self, space):
diff --git a/pypy/interpreter/gateway.py b/pypy/interpreter/gateway.py
--- a/pypy/interpreter/gateway.py
+++ b/pypy/interpreter/gateway.py
@@ -671,10 +671,10 @@
return space.newtuple([builtin_code,
space.newtuple([space.wrap(self.identifier)])])
- def find(indentifier):
+ @staticmethod
+ def find(space, identifier):
from pypy.interpreter.function import Function
- return Function._all[indentifier].code
- find = staticmethod(find)
+ return Function.find(space, identifier).code
def signature(self):
return self.sig
diff --git a/pypy/interpreter/pyframe.py b/pypy/interpreter/pyframe.py
--- a/pypy/interpreter/pyframe.py
+++ b/pypy/interpreter/pyframe.py
@@ -4,6 +4,7 @@
import sys
from rpython.rlib import jit
from rpython.rlib.debug import make_sure_not_resized, check_nonneg
+from rpython.rlib.debug import ll_assert_not_none
from rpython.rlib.jit import hint
from rpython.rlib.objectmodel import instantiate, specialize, we_are_translated
from rpython.rlib.rarithmetic import intmask, r_uint
@@ -298,7 +299,13 @@
# stack manipulation helpers
def pushvalue(self, w_object):
depth = self.valuestackdepth
- self.locals_cells_stack_w[depth] = w_object
+ self.locals_cells_stack_w[depth] = ll_assert_not_none(w_object)
+ self.valuestackdepth = depth + 1
+
+ def pushvalue_none(self):
+ depth = self.valuestackdepth
+ # the entry is already None, and remains None
+ assert self.locals_cells_stack_w[depth] is None
self.valuestackdepth = depth + 1
def _check_stack_index(self, index):
@@ -311,6 +318,9 @@
return index >= stackstart
def popvalue(self):
+ return ll_assert_not_none(self.popvalue_maybe_none())
+
+ def popvalue_maybe_none(self):
depth = self.valuestackdepth - 1
assert self._check_stack_index(depth)
assert depth >= 0
@@ -385,6 +395,9 @@
def peekvalue(self, index_from_top=0):
# NOTE: top of the stack is peekvalue(0).
# Contrast this with CPython where it's PEEK(-1).
+ return ll_assert_not_none(self.peekvalue_maybe_none(index_from_top))
+
+ def peekvalue_maybe_none(self, index_from_top=0):
index_from_top = hint(index_from_top, promote=True)
index = self.valuestackdepth + ~index_from_top
assert self._check_stack_index(index)
@@ -396,7 +409,7 @@
index = self.valuestackdepth + ~index_from_top
assert self._check_stack_index(index)
assert index >= 0
- self.locals_cells_stack_w[index] = w_object
+ self.locals_cells_stack_w[index] = ll_assert_not_none(w_object)
@jit.unroll_safe
def dropvaluesuntil(self, finaldepth):
diff --git a/pypy/module/_cffi_backend/__init__.py b/pypy/module/_cffi_backend/__init__.py
--- a/pypy/module/_cffi_backend/__init__.py
+++ b/pypy/module/_cffi_backend/__init__.py
@@ -1,6 +1,6 @@
import sys
from pypy.interpreter.mixedmodule import MixedModule
-from rpython.rlib import rdynload, clibffi, entrypoint
+from rpython.rlib import rdynload, clibffi
from rpython.rtyper.lltypesystem import rffi
VERSION = "1.9.1"
@@ -68,9 +68,14 @@
if has_stdcall:
interpleveldefs['FFI_STDCALL'] = 'space.wrap(%d)' % FFI_STDCALL
- def startup(self, space):
- from pypy.module._cffi_backend import embedding
- embedding.glob.space = space
+ def __init__(self, space, *args):
+ MixedModule.__init__(self, space, *args)
+ #
+ if not space.config.objspace.disable_entrypoints:
+ # import 'embedding', which has the side-effect of registering
+ # the 'pypy_init_embedded_cffi_module' entry point
+ from pypy.module._cffi_backend import embedding
+ embedding.glob.space = space
def get_dict_rtld_constants():
@@ -85,11 +90,3 @@
for _name, _value in get_dict_rtld_constants().items():
Module.interpleveldefs[_name] = 'space.wrap(%d)' % _value
-
-
-# write this entrypoint() here, to make sure it is registered early enough
- at entrypoint.entrypoint_highlevel('main', [rffi.INT, rffi.VOIDP],
- c_name='pypy_init_embedded_cffi_module')
-def pypy_init_embedded_cffi_module(version, init_struct):
- from pypy.module._cffi_backend import embedding
- return embedding.pypy_init_embedded_cffi_module(version, init_struct)
diff --git a/pypy/module/_cffi_backend/embedding.py b/pypy/module/_cffi_backend/embedding.py
--- a/pypy/module/_cffi_backend/embedding.py
+++ b/pypy/module/_cffi_backend/embedding.py
@@ -1,4 +1,5 @@
import os
+from rpython.rlib import entrypoint
from rpython.rtyper.lltypesystem import lltype, rffi
from rpython.translator.tool.cbuild import ExternalCompilationInfo
@@ -46,6 +47,8 @@
glob = Global()
+ at entrypoint.entrypoint_highlevel('main', [rffi.INT, rffi.VOIDP],
+ c_name='pypy_init_embedded_cffi_module')
def pypy_init_embedded_cffi_module(version, init_struct):
# called from __init__.py
name = "?"
diff --git a/pypy/module/_pickle_support/maker.py b/pypy/module/_pickle_support/maker.py
--- a/pypy/module/_pickle_support/maker.py
+++ b/pypy/module/_pickle_support/maker.py
@@ -77,7 +77,7 @@
def builtin_code(space, identifier):
from pypy.interpreter import gateway
try:
- return gateway.BuiltinCode.find(identifier)
+ return gateway.BuiltinCode.find(space, identifier)
except KeyError:
raise oefmt(space.w_RuntimeError,
"cannot unpickle builtin code: %s", identifier)
@@ -86,7 +86,7 @@
def builtin_function(space, identifier):
from pypy.interpreter import function
try:
- return function.Function.find(identifier)
+ return function.Function.find(space, identifier)
except KeyError:
raise oefmt(space.w_RuntimeError,
"cannot unpickle builtin function: %s", identifier)
diff --git a/pypy/module/_ssl/test/test_ssl.py b/pypy/module/_ssl/test/test_ssl.py
--- a/pypy/module/_ssl/test/test_ssl.py
+++ b/pypy/module/_ssl/test/test_ssl.py
@@ -169,8 +169,8 @@
}
def setup_method(self, method):
- # https://www.verisign.net/
- ADDR = "www.verisign.net", 443
+ # https://gmail.com/
+ ADDR = "gmail.com", 443
self.w_s = self.space.appexec([self.space.wrap(ADDR)], """(ADDR):
import socket
diff --git a/pypy/module/cppyy/__init__.py b/pypy/module/cppyy/__init__.py
--- a/pypy/module/cppyy/__init__.py
+++ b/pypy/module/cppyy/__init__.py
@@ -14,7 +14,6 @@
'_set_class_generator' : 'interp_cppyy.set_class_generator',
'_set_function_generator': 'interp_cppyy.set_function_generator',
'_register_class' : 'interp_cppyy.register_class',
- '_is_static' : 'interp_cppyy.is_static',
'_get_nullptr' : 'interp_cppyy.get_nullptr',
'CPPInstanceBase' : 'interp_cppyy.W_CPPInstance',
'addressof' : 'interp_cppyy.addressof',
diff --git a/pypy/module/cppyy/bench/Makefile b/pypy/module/cppyy/bench/Makefile
--- a/pypy/module/cppyy/bench/Makefile
+++ b/pypy/module/cppyy/bench/Makefile
@@ -26,4 +26,4 @@
bench02Dict_reflex.so: bench02.h bench02.cxx bench02.xml
$(genreflex) bench02.h $(genreflexflags) --selection=bench02.xml -I$(ROOTSYS)/include
- g++ -o $@ bench02.cxx bench02_rflx.cpp -I$(ROOTSYS)/include -shared -lReflex -lHistPainter `root-config --libs` $(cppflags) $(cppflags2)
+ g++ -o $@ bench02.cxx bench02_rflx.cpp -I$(ROOTSYS)/include -shared -std=c++11 -lHistPainter `root-config --libs` $(cppflags) $(cppflags2)
diff --git a/pypy/module/cppyy/capi/builtin_capi.py b/pypy/module/cppyy/capi/builtin_capi.py
--- a/pypy/module/cppyy/capi/builtin_capi.py
+++ b/pypy/module/cppyy/capi/builtin_capi.py
@@ -1,12 +1,11 @@
from rpython.rtyper.lltypesystem import rffi, lltype
+from rpython.rlib.rarithmetic import intmask
from rpython.rlib import jit
-import reflex_capi as backend
-#import cint_capi as backend
+import cling_capi as backend
from pypy.module.cppyy.capi.capi_types import C_SCOPE, C_TYPE, C_OBJECT,\
- C_METHOD, C_INDEX, C_INDEX_ARRAY, WLAVC_INDEX,\
- C_METHPTRGETTER, C_METHPTRGETTER_PTR
+ C_METHOD, C_INDEX, C_INDEX_ARRAY, WLAVC_INDEX, C_FUNC_PTR
identify = backend.identify
pythonize = backend.pythonize
@@ -52,13 +51,6 @@
compilation_info=backend.eci)
def c_get_scope_opaque(space, name):
return _c_get_scope_opaque(name)
-_c_get_template = rffi.llexternal(
- "cppyy_get_template",
- [rffi.CCHARP], C_TYPE,
- releasegil=ts_reflect,
- compilation_info=backend.eci)
-def c_get_template(space, name):
- return _c_get_template(name)
_c_actual_class = rffi.llexternal(
"cppyy_actual_class",
[C_TYPE, C_OBJECT], C_TYPE,
@@ -154,6 +146,13 @@
compilation_info=backend.eci)
def c_call_d(space, cppmethod, cppobject, nargs, args):
return _c_call_d(cppmethod, cppobject, nargs, args)
+_c_call_ld = rffi.llexternal(
+ "cppyy_call_ld",
+ [C_METHOD, C_OBJECT, rffi.INT, rffi.VOIDP], rffi.LONGDOUBLE,
+ releasegil=ts_call,
+ compilation_info=backend.eci)
+def c_call_ld(space, cppmethod, cppobject, nargs, args):
+ return _c_call_ld(cppmethod, cppobject, nargs, args)
_c_call_r = rffi.llexternal(
"cppyy_call_r",
@@ -164,11 +163,17 @@
return _c_call_r(cppmethod, cppobject, nargs, args)
_c_call_s = rffi.llexternal(
"cppyy_call_s",
- [C_METHOD, C_OBJECT, rffi.INT, rffi.VOIDP], rffi.CCHARP,
+ [C_METHOD, C_OBJECT, rffi.INT, rffi.VOIDP, rffi.SIZE_TP], rffi.CCHARP,
releasegil=ts_call,
compilation_info=backend.eci)
def c_call_s(space, cppmethod, cppobject, nargs, args):
- return _c_call_s(cppmethod, cppobject, nargs, args)
+ length = lltype.malloc(rffi.SIZE_TP.TO, 1, flavor='raw')
+ try:
+ cstr = _c_call_s(cppmethod, cppobject, nargs, args, length)
+ cstr_len = intmask(length[0])
+ finally:
+ lltype.free(length, flavor='raw')
+ return cstr, cstr_len
_c_constructor = rffi.llexternal(
"cppyy_constructor",
@@ -185,15 +190,14 @@
def c_call_o(space, method, cppobj, nargs, args, cppclass):
return _c_call_o(method, cppobj, nargs, args, cppclass.handle)
-_c_get_methptr_getter = rffi.llexternal(
- "cppyy_get_methptr_getter",
- [C_SCOPE, C_INDEX], C_METHPTRGETTER_PTR,
+_c_get_function_address = rffi.llexternal(
+ "cppyy_get_function_address",
+ [C_SCOPE, C_INDEX], C_FUNC_PTR,
releasegil=ts_reflect,
compilation_info=backend.eci,
- elidable_function=True,
random_effects_on_gcobjs=False)
-def c_get_methptr_getter(space, cppscope, index):
- return _c_get_methptr_getter(cppscope.handle, index)
+def c_get_function_address(space, cppscope, index):
+ return _c_get_function_address(cppscope.handle, index)
# handling of function argument buffer ---------------------------------------
_c_allocate_function_args = rffi.llexternal(
@@ -215,8 +219,8 @@
[], rffi.SIZE_T,
releasegil=ts_memory,
compilation_info=backend.eci,
- elidable_function=True,
random_effects_on_gcobjs=False)
+ at jit.elidable
def c_function_arg_sizeof(space):
return _c_function_arg_sizeof()
_c_function_arg_typeoffset = rffi.llexternal(
@@ -224,8 +228,8 @@
[], rffi.SIZE_T,
releasegil=ts_memory,
compilation_info=backend.eci,
- elidable_function=True,
random_effects_on_gcobjs=False)
+ at jit.elidable
def c_function_arg_typeoffset(space):
return _c_function_arg_typeoffset()
@@ -237,6 +241,20 @@
compilation_info=backend.eci)
def c_is_namespace(space, scope):
return _c_is_namespace(scope)
+_c_is_template = rffi.llexternal(
+ "cppyy_is_template",
+ [rffi.CCHARP], rffi.INT,
+ releasegil=ts_reflect,
+ compilation_info=backend.eci)
+def c_is_template(space, name):
+ return _c_is_template(name)
+_c_is_abstract = rffi.llexternal(
+ "cppyy_is_abstract",
+ [C_SCOPE], rffi.INT,
+ releasegil=ts_reflect,
+ compilation_info=backend.eci)
+def c_is_abstract(space, cpptype):
+ return _c_is_abstract(cpptype)
_c_is_enum = rffi.llexternal(
"cppyy_is_enum",
[rffi.CCHARP], rffi.INT,
@@ -286,9 +304,8 @@
[C_TYPE, C_TYPE], rffi.INT,
releasegil=ts_reflect,
compilation_info=backend.eci,
- elidable_function=True,
random_effects_on_gcobjs=False)
- at jit.elidable_promote('2')
+ at jit.elidable
def c_is_subtype(space, derived, base):
if derived == base:
return 1
@@ -296,12 +313,11 @@
_c_base_offset = rffi.llexternal(
"cppyy_base_offset",
- [C_TYPE, C_TYPE, C_OBJECT, rffi.INT], rffi.SIZE_T,
+ [C_TYPE, C_TYPE, C_OBJECT, rffi.INT], rffi.LONG, # actually ptrdiff_t
releasegil=ts_reflect,
compilation_info=backend.eci,
- elidable_function=True,
random_effects_on_gcobjs=False)
- at jit.elidable_promote('1,2,4')
+ at jit.elidable
def c_base_offset(space, derived, base, address, direction):
if derived == base:
return 0
@@ -340,7 +356,7 @@
i += 1
py_indices.append(index)
index = indices[i]
- c_free(rffi.cast(rffi.VOIDP, indices)) # c_free defined below
+ c_free(space, rffi.cast(rffi.VOIDP, indices)) # c_free defined below
return py_indices
_c_method_name = rffi.llexternal(
@@ -474,7 +490,7 @@
return charp2str_free(space, _c_datamember_type(cppscope.handle, datamember_index))
_c_datamember_offset = rffi.llexternal(
"cppyy_datamember_offset",
- [C_SCOPE, rffi.INT], rffi.SIZE_T,
+ [C_SCOPE, rffi.INT], rffi.LONG, # actually ptrdiff_t
releasegil=ts_reflect,
compilation_info=backend.eci)
def c_datamember_offset(space, cppscope, datamember_index):
@@ -519,27 +535,29 @@
compilation_info=backend.eci)
def c_strtoull(space, svalue):
return _c_strtoull(svalue)
-c_free = rffi.llexternal(
+_c_free = rffi.llexternal(
"cppyy_free",
[rffi.VOIDP], lltype.Void,
releasegil=ts_memory,
compilation_info=backend.eci)
+def c_free(space, voidp):
+ return _c_free(voidp)
def charp2str_free(space, charp):
string = rffi.charp2str(charp)
voidp = rffi.cast(rffi.VOIDP, charp)
- c_free(voidp)
+ _c_free(voidp)
return string
_c_charp2stdstring = rffi.llexternal(
"cppyy_charp2stdstring",
- [rffi.CCHARP], C_OBJECT,
+ [rffi.CCHARP, rffi.SIZE_T], C_OBJECT,
releasegil=ts_helper,
compilation_info=backend.eci)
-def c_charp2stdstring(space, svalue):
- with rffi.scoped_view_charp(svalue) as charp:
- result = _c_charp2stdstring(charp)
- return result
+def c_charp2stdstring(space, pystr, sz):
+ with rffi.scoped_view_charp(pystr) as cstr:
+ cppstr = _c_charp2stdstring(cstr, sz)
+ return cppstr
_c_stdstring2stdstring = rffi.llexternal(
"cppyy_stdstring2stdstring",
[C_OBJECT], C_OBJECT,
@@ -547,3 +565,26 @@
compilation_info=backend.eci)
def c_stdstring2stdstring(space, cppobject):
return _c_stdstring2stdstring(cppobject)
+
+_c_stdvector_valuetype = rffi.llexternal(
+ "cppyy_stdvector_valuetype",
+ [rffi.CCHARP], rffi.CCHARP,
+ releasegil=ts_helper,
+ compilation_info=backend.eci)
+def c_stdvector_valuetype(space, pystr):
+ cstr = rffi.str2charp(pystr)
+ result = _c_stdvector_valuetype(cstr)
+ rffi.free_charp(cstr)
+ if result:
+ return charp2str_free(space, result)
+ return ""
+_c_stdvector_valuesize = rffi.llexternal(
+ "cppyy_stdvector_valuesize",
+ [rffi.CCHARP], rffi.SIZE_T,
+ releasegil=ts_helper,
+ compilation_info=backend.eci)
+def c_stdvector_valuesize(space, pystr):
+ cstr = rffi.str2charp(pystr)
+ result = _c_stdvector_valuesize(cstr)
+ rffi.free_charp(cstr)
+ return result
diff --git a/pypy/module/cppyy/capi/capi_types.py b/pypy/module/cppyy/capi/capi_types.py
--- a/pypy/module/cppyy/capi/capi_types.py
+++ b/pypy/module/cppyy/capi/capi_types.py
@@ -18,5 +18,4 @@
C_INDEX_ARRAY = rffi.LONGP
WLAVC_INDEX = rffi.LONG
-C_METHPTRGETTER = lltype.FuncType([C_OBJECT], rffi.VOIDP)
-C_METHPTRGETTER_PTR = lltype.Ptr(C_METHPTRGETTER)
+C_FUNC_PTR = rffi.VOIDP
diff --git a/pypy/module/cppyy/capi/cint_capi.py b/pypy/module/cppyy/capi/cint_capi.py
deleted file mode 100644
--- a/pypy/module/cppyy/capi/cint_capi.py
+++ /dev/null
@@ -1,437 +0,0 @@
-import py, os, sys
-
-from pypy.interpreter.error import OperationError
-from pypy.interpreter.gateway import interp2app, unwrap_spec
-from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.baseobjspace import W_Root
-
-from rpython.translator.tool.cbuild import ExternalCompilationInfo
-from rpython.rtyper.lltypesystem import rffi, lltype
-from rpython.rlib import libffi, rdynload
-from rpython.tool.udir import udir
-
-from pypy.module.cppyy.capi.capi_types import C_OBJECT
-
-
-__all__ = ['identify', 'std_string_name', 'eci', 'c_load_dictionary']
-
-pkgpath = py.path.local(__file__).dirpath().join(os.pardir)
-srcpath = pkgpath.join("src")
-incpath = pkgpath.join("include")
-
-if os.environ.get("ROOTSYS"):
- import commands
- (stat, incdir) = commands.getstatusoutput("root-config --incdir")
- if stat != 0:
- rootincpath = [os.path.join(os.environ["ROOTSYS"], "include"), py.path.local(udir)]
- rootlibpath = [os.path.join(os.environ["ROOTSYS"], "lib64"), os.path.join(os.environ["ROOTSYS"], "lib")]
- else:
- rootincpath = [incdir, py.path.local(udir)]
- rootlibpath = commands.getoutput("root-config --libdir").split()
-else:
- rootincpath = [py.path.local(udir)]
- rootlibpath = []
-
-def identify():
- return 'CINT'
-
-ts_reflect = True
-ts_call = True
-ts_memory = False
-ts_helper = False
-
-std_string_name = 'string'
-
-# force loading in global mode of core libraries, rather than linking with
-# them as PyPy uses various version of dlopen in various places; note that
-# this isn't going to fly on Windows (note that locking them in objects and
-# calling dlclose in __del__ seems to come too late, so this'll do for now)
-with rffi.scoped_str2charp('libCint.so') as ll_libname:
- _cintdll = rdynload.dlopen(ll_libname, rdynload.RTLD_GLOBAL | rdynload.RTLD_NOW)
-with rffi.scoped_str2charp('libCore.so') as ll_libname:
- _coredll = rdynload.dlopen(ll_libname, rdynload.RTLD_GLOBAL | rdynload.RTLD_NOW)
-with rffi.scoped_str2charp('libHist.so') as ll_libname:
- _coredll = rdynload.dlopen(ll_libname, rdynload.RTLD_GLOBAL | rdynload.RTLD_NOW)
-
-eci = ExternalCompilationInfo(
- separate_module_files=[srcpath.join("cintcwrapper.cxx")],
- include_dirs=[incpath] + rootincpath,
- includes=["cintcwrapper.h"],
- library_dirs=rootlibpath,
- libraries=["Hist", "Core", "Cint"],
- use_cpp_linker=True,
-)
-
-_c_load_dictionary = rffi.llexternal(
- "cppyy_load_dictionary",
- [rffi.CCHARP], rdynload.DLLHANDLE,
- releasegil=False,
- compilation_info=eci)
-
-def c_load_dictionary(name):
- result = _c_load_dictionary(name)
- # ignore result: libffi.CDLL(name) either returns a handle to the already
- # open file, or will fail as well and produce a correctly formatted error
- return libffi.CDLL(name)
-
-
-# CINT-specific pythonizations ===============================================
-_c_charp2TString = rffi.llexternal(
- "cppyy_charp2TString",
- [rffi.CCHARP], C_OBJECT,
- releasegil=ts_helper,
- compilation_info=eci)
-def c_charp2TString(space, svalue):
- with rffi.scoped_view_charp(svalue) as charp:
- result = _c_charp2TString(charp)
- return result
-_c_TString2TString = rffi.llexternal(
- "cppyy_TString2TString",
- [C_OBJECT], C_OBJECT,
- releasegil=ts_helper,
- compilation_info=eci)
-def c_TString2TString(space, cppobject):
- return _c_TString2TString(cppobject)
-
-def _get_string_data(space, w_obj, m1, m2 = None):
- from pypy.module.cppyy import interp_cppyy
- obj = space.interp_w(interp_cppyy.W_CPPInstance, w_obj)
- w_1 = obj.space.call_method(w_obj, m1)
- if m2 is None:
- return w_1
- return obj.space.call_method(w_1, m2)
-
-### TF1 ----------------------------------------------------------------------
-class State(object):
- def __init__(self, space):
- self.tfn_pyfuncs = []
- self.tfn_callbacks = []
-
-_create_tf1 = rffi.llexternal(
- "cppyy_create_tf1",
- [rffi.CCHARP, rffi.ULONG, rffi.DOUBLE, rffi.DOUBLE, rffi.INT], C_OBJECT,
- releasegil=False,
- compilation_info=eci)
-
- at unwrap_spec(args_w='args_w')
-def tf1_tf1(space, w_self, args_w):
- """Pythonized version of TF1 constructor:
- takes functions and callable objects, and allows a callback into them."""
-
- from pypy.module.cppyy import interp_cppyy
- tf1_class = interp_cppyy.scope_byname(space, "TF1")
-
- # expected signature:
- # 1. (char* name, pyfunc, double xmin, double xmax, int npar = 0)
- argc = len(args_w)
-
- try:
- if argc < 4 or 5 < argc:
- raise TypeError("wrong number of arguments")
-
- # first argument must be a name
- funcname = space.str_w(args_w[0])
-
- # last (optional) argument is number of parameters
- npar = 0
- if argc == 5: npar = space.int_w(args_w[4])
-
- # second argument must be a callable python object
- w_callable = args_w[1]
- if not space.is_true(space.callable(w_callable)):
- raise TypeError("2nd argument is not a valid python callable")
-
- # generate a pointer to function
- from pypy.module._cffi_backend import newtype, ctypefunc, func
-
- c_double = newtype.new_primitive_type(space, 'double')
- c_doublep = newtype.new_pointer_type(space, c_double)
-
- # wrap the callable as the signature needs modifying
- w_ifunc = interp_cppyy.get_interface_func(space, w_callable, npar)
-
- w_cfunc = ctypefunc.W_CTypeFunc(space, [c_doublep, c_doublep], c_double, False)
- w_callback = func.callback(space, w_cfunc, w_ifunc, None)
- funcaddr = rffi.cast(rffi.ULONG, w_callback.get_closure())
-
- # so far, so good; leaves on issue: CINT is expecting a wrapper, but
- # we need the overload that takes a function pointer, which is not in
- # the dictionary, hence this helper:
- newinst = _create_tf1(space.str_w(args_w[0]), funcaddr,
- space.float_w(args_w[2]), space.float_w(args_w[3]), npar)
-
- # w_self is a null-ptr bound as TF1
- from pypy.module.cppyy.interp_cppyy import W_CPPInstance, memory_regulator
- cppself = space.interp_w(W_CPPInstance, w_self, can_be_None=False)
- cppself._rawobject = newinst
- memory_regulator.register(cppself)
-
- # tie all the life times to the TF1 instance
- space.setattr(w_self, space.wrap('_callback'), w_callback)
-
- # by definition for __init__
- return None
-
- except (OperationError, TypeError, IndexError) as e:
- newargs_w = args_w[1:] # drop class
-
- # return control back to the original, unpythonized overload
- ol = tf1_class.get_overload("TF1")
- return ol.call(None, newargs_w)
-
-### TTree --------------------------------------------------------------------
-_ttree_Branch = rffi.llexternal(
- "cppyy_ttree_Branch",
- [rffi.VOIDP, rffi.CCHARP, rffi.CCHARP, rffi.VOIDP, rffi.INT, rffi.INT], C_OBJECT,
- releasegil=False,
- compilation_info=eci)
-
- at unwrap_spec(args_w='args_w')
-def ttree_Branch(space, w_self, args_w):
- """Pythonized version of TTree::Branch(): takes proxy objects and by-passes
- the CINT-manual layer."""
-
- from pypy.module.cppyy import interp_cppyy
- tree_class = interp_cppyy.scope_byname(space, "TTree")
-
- # sigs to modify (and by-pass CINT):
- # 1. (const char*, const char*, T**, Int_t=32000, Int_t=99)
- # 2. (const char*, T**, Int_t=32000, Int_t=99)
- argc = len(args_w)
-
- # basic error handling of wrong arguments is best left to the original call,
- # so that error messages etc. remain consistent in appearance: the following
- # block may raise TypeError or IndexError to break out anytime
-
- try:
- if argc < 2 or 5 < argc:
- raise TypeError("wrong number of arguments")
-
- tree = space.interp_w(interp_cppyy.W_CPPInstance, w_self, can_be_None=True)
- if (tree is None) or (tree.cppclass != tree_class):
- raise TypeError("not a TTree")
-
- # first argument must always always be cont char*
- branchname = space.str_w(args_w[0])
-
- # if args_w[1] is a classname, then case 1, else case 2
- try:
- classname = space.str_w(args_w[1])
- addr_idx = 2
- w_address = args_w[addr_idx]
- except (OperationError, TypeError):
- addr_idx = 1
- w_address = args_w[addr_idx]
-
- bufsize, splitlevel = 32000, 99
- if addr_idx+1 < argc: bufsize = space.c_int_w(args_w[addr_idx+1])
- if addr_idx+2 < argc: splitlevel = space.c_int_w(args_w[addr_idx+2])
-
- # now retrieve the W_CPPInstance and build other stub arguments
- space = tree.space # holds the class cache in State
- cppinstance = space.interp_w(interp_cppyy.W_CPPInstance, w_address)
- address = rffi.cast(rffi.VOIDP, cppinstance.get_rawobject())
- klassname = cppinstance.cppclass.full_name()
- vtree = rffi.cast(rffi.VOIDP, tree.get_rawobject())
-
- # call the helper stub to by-pass CINT
- vbranch = _ttree_Branch(vtree, branchname, klassname, address, bufsize, splitlevel)
- branch_class = interp_cppyy.scope_byname(space, "TBranch")
- w_branch = interp_cppyy.wrap_cppobject(space, vbranch, branch_class)
- return w_branch
- except (OperationError, TypeError, IndexError):
- pass
-
- # return control back to the original, unpythonized overload
- ol = tree_class.get_overload("Branch")
- return ol.call(w_self, args_w)
-
-def activate_branch(space, w_branch):
- w_branches = space.call_method(w_branch, "GetListOfBranches")
- for i in range(space.r_longlong_w(space.call_method(w_branches, "GetEntriesFast"))):
- w_b = space.call_method(w_branches, "At", space.wrap(i))
- activate_branch(space, w_b)
- space.call_method(w_branch, "SetStatus", space.wrap(1))
- space.call_method(w_branch, "ResetReadEntry")
-
-c_ttree_GetEntry = rffi.llexternal(
- "cppyy_ttree_GetEntry",
- [rffi.VOIDP, rffi.LONGLONG], rffi.LONGLONG,
- releasegil=False,
- compilation_info=eci)
-
- at unwrap_spec(args_w='args_w')
-def ttree_getattr(space, w_self, args_w):
- """Specialized __getattr__ for TTree's that allows switching on/off the
- reading of individual branchs."""
-
- from pypy.module.cppyy import interp_cppyy
- tree = space.interp_w(interp_cppyy.W_CPPInstance, w_self)
-
- space = tree.space # holds the class cache in State
-
- # prevent recursion
- attr = space.str_w(args_w[0])
- if attr and attr[0] == '_':
- raise OperationError(space.w_AttributeError, args_w[0])
-
- # try the saved cdata (for builtin types)
- try:
- w_cdata = space.getattr(w_self, space.wrap('_'+attr))
- from pypy.module._cffi_backend import cdataobj
- cdata = space.interp_w(cdataobj.W_CData, w_cdata, can_be_None=False)
- return cdata.convert_to_object()
- except OperationError:
- pass
-
- # setup branch as a data member and enable it for reading
- w_branch = space.call_method(w_self, "GetBranch", args_w[0])
- if not space.is_true(w_branch):
- raise OperationError(space.w_AttributeError, args_w[0])
- activate_branch(space, w_branch)
-
- # figure out from where we're reading
- entry = space.r_longlong_w(space.call_method(w_self, "GetReadEntry"))
- if entry == -1:
- entry = 0
-
- # setup cache structure
- w_klassname = space.call_method(w_branch, "GetClassName")
- if space.is_true(w_klassname):
- # some instance
- klass = interp_cppyy.scope_byname(space, space.str_w(w_klassname))
- w_obj = klass.construct()
- # 0x10000 = kDeleteObject; reset because we own the object
- space.call_method(w_branch, "ResetBit", space.wrap(0x10000))
- space.call_method(w_branch, "SetObject", w_obj)
- space.call_method(w_branch, "GetEntry", space.wrap(entry))
- space.setattr(w_self, args_w[0], w_obj)
- return w_obj
- else:
- # builtin data
- w_leaf = space.call_method(w_self, "GetLeaf", args_w[0])
- space.call_method(w_branch, "GetEntry", space.wrap(entry))
-
- # location
- w_address = space.call_method(w_leaf, "GetValuePointer")
- buf = space.getarg_w('s*', w_address)
- from pypy.module._rawffi import buffer
- assert isinstance(buf, buffer.RawFFIBuffer)
- address = rffi.cast(rffi.CCHARP, buf.datainstance.ll_buffer)
-
- # placeholder
- w_typename = space.call_method(w_leaf, "GetTypeName" )
- from pypy.module.cppyy import capi
- typename = capi.c_resolve_name(space, space.str_w(w_typename))
- if typename == 'bool': typename = '_Bool'
- w_address = space.call_method(w_leaf, "GetValuePointer")
- from pypy.module._cffi_backend import cdataobj, newtype
- cdata = cdataobj.W_CData(space, address, newtype.new_primitive_type(space, typename))
-
- # cache result
- space.setattr(w_self, space.wrap('_'+attr), space.wrap(cdata))
- return space.getattr(w_self, args_w[0])
-
-class W_TTreeIter(W_Root):
- def __init__(self, space, w_tree):
- from pypy.module.cppyy import interp_cppyy
- tree = space.interp_w(interp_cppyy.W_CPPInstance, w_tree)
- self.vtree = rffi.cast(rffi.VOIDP, tree.get_cppthis(tree.cppclass))
- self.w_tree = w_tree
-
- self.current = 0
- self.maxentry = space.r_longlong_w(space.call_method(w_tree, "GetEntriesFast"))
-
- space = self.space = tree.space # holds the class cache in State
- space.call_method(w_tree, "SetBranchStatus", space.wrap("*"), space.wrap(0))
-
- def iter_w(self):
- return self.space.wrap(self)
-
- def next_w(self):
- if self.current == self.maxentry:
- raise OperationError(self.space.w_StopIteration, self.space.w_None)
- # TODO: check bytes read?
- c_ttree_GetEntry(self.vtree, self.current)
- self.current += 1
- return self.w_tree
-
-W_TTreeIter.typedef = TypeDef(
- 'TTreeIter',
- __iter__ = interp2app(W_TTreeIter.iter_w),
- next = interp2app(W_TTreeIter.next_w),
-)
-
-def ttree_iter(space, w_self):
- """Allow iteration over TTree's. Also initializes branch data members and
- sets addresses, if needed."""
- w_treeiter = W_TTreeIter(space, w_self)
- return w_treeiter
-
-# setup pythonizations for later use at run-time
-_pythonizations = {}
-def register_pythonizations(space):
- "NOT_RPYTHON"
-
- allfuncs = [
-
- ### TF1
- tf1_tf1,
-
- ### TTree
- ttree_Branch, ttree_iter, ttree_getattr,
- ]
-
- for f in allfuncs:
- _pythonizations[f.__name__] = space.wrap(interp2app(f))
-
-def _method_alias(space, w_pycppclass, m1, m2):
- space.setattr(w_pycppclass, space.wrap(m1),
- space.getattr(w_pycppclass, space.wrap(m2)))
-
-# callback coming in when app-level bound classes have been created
-def pythonize(space, name, w_pycppclass):
-
- if name == "TCollection":
- _method_alias(space, w_pycppclass, "append", "Add")
- _method_alias(space, w_pycppclass, "__len__", "GetSize")
-
- elif name == "TF1":
- space.setattr(w_pycppclass, space.wrap("__init__"), _pythonizations["tf1_tf1"])
-
- elif name == "TFile":
- _method_alias(space, w_pycppclass, "__getattr__", "Get")
-
- elif name == "TObjString":
- _method_alias(space, w_pycppclass, "__str__", "GetName")
- _method_alias(space, w_pycppclass, "_cppyy_as_builtin", "GetString")
-
- elif name == "TString":
- _method_alias(space, w_pycppclass, "__str__", "Data")
- _method_alias(space, w_pycppclass, "__len__", "Length")
- _method_alias(space, w_pycppclass, "__cmp__", "CompareTo")
- _method_alias(space, w_pycppclass, "_cppyy_as_builtin", "Data")
-
- elif name == "TTree":
- _method_alias(space, w_pycppclass, "_unpythonized_Branch", "Branch")
-
- space.setattr(w_pycppclass, space.wrap("Branch"), _pythonizations["ttree_Branch"])
- space.setattr(w_pycppclass, space.wrap("__iter__"), _pythonizations["ttree_iter"])
- space.setattr(w_pycppclass, space.wrap("__getattr__"), _pythonizations["ttree_getattr"])
-
- elif name[0:8] == "TVectorT": # TVectorT<> template
- _method_alias(space, w_pycppclass, "__len__", "GetNoElements")
-
-# destruction callback (needs better solution, but this is for CINT
-# only and should not appear outside of ROOT-specific uses)
-from pypy.module.cpyext.api import cpython_api, CANNOT_FAIL
-
- at cpython_api([rffi.VOIDP], lltype.Void, error=CANNOT_FAIL)
-def _Py_cppyy_recursive_remove(space, cppobject):
- from pypy.module.cppyy.interp_cppyy import memory_regulator
- from pypy.module.cppyy.capi import C_OBJECT, C_NULL_OBJECT
-
- obj = memory_regulator.retrieve(rffi.cast(C_OBJECT, cppobject))
- if obj is not None:
- memory_regulator.unregister(obj)
- obj._rawobject = C_NULL_OBJECT
diff --git a/pypy/module/cppyy/capi/cling_capi.py b/pypy/module/cppyy/capi/cling_capi.py
--- a/pypy/module/cppyy/capi/cling_capi.py
+++ b/pypy/module/cppyy/capi/cling_capi.py
@@ -1,8 +1,17 @@
import py, os
+from pypy.objspace.std.iterobject import W_AbstractSeqIterObject
+
+from pypy.interpreter.error import OperationError
+from pypy.interpreter.gateway import interp2app
+
from rpython.translator.tool.cbuild import ExternalCompilationInfo
-from rpython.rtyper.lltypesystem import rffi
-from rpython.rlib import libffi, rdynload
+from rpython.rtyper.lltypesystem import rffi, lltype
+from rpython.rlib.rarithmetic import intmask
+from rpython.rlib import jit, libffi, rdynload
+
+from pypy.module._rawffi.array import W_ArrayInstance
+from pypy.module.cppyy.capi.capi_types import C_OBJECT
__all__ = ['identify', 'std_string_name', 'eci', 'c_load_dictionary']
@@ -16,7 +25,8 @@
if os.environ.get("ROOTSYS"):
if config_stat != 0: # presumably Reflex-only
rootincpath = [os.path.join(os.environ["ROOTSYS"], "interpreter/cling/include"),
- os.path.join(os.environ["ROOTSYS"], "interpreter/llvm/inst/include")]
+ os.path.join(os.environ["ROOTSYS"], "interpreter/llvm/inst/include"),
+ os.path.join(os.environ["ROOTSYS"], "include"),]
rootlibpath = [os.path.join(os.environ["ROOTSYS"], "lib64"), os.path.join(os.environ["ROOTSYS"], "lib")]
else:
rootincpath = [incdir]
@@ -39,13 +49,21 @@
std_string_name = 'std::basic_string<char>'
+# force loading (and exposure) of libCore symbols
+with rffi.scoped_str2charp('libCore.so') as ll_libname:
+ _coredll = rdynload.dlopen(ll_libname, rdynload.RTLD_GLOBAL | rdynload.RTLD_NOW)
+
+# require local translator path to pickup common defs
+from rpython.translator import cdir
+translator_c_dir = py.path.local(cdir)
+
eci = ExternalCompilationInfo(
separate_module_files=[srcpath.join("clingcwrapper.cxx")],
- include_dirs=[incpath] + rootincpath,
+ include_dirs=[incpath, translator_c_dir] + rootincpath,
includes=["clingcwrapper.h"],
library_dirs=rootlibpath,
libraries=["Cling"],
- compile_extra=["-fno-strict-aliasing"],
+ compile_extra=["-fno-strict-aliasing", "-std=c++11"],
use_cpp_linker=True,
)
@@ -59,11 +77,120 @@
pch = _c_load_dictionary(name)
return pch
+_c_stdstring2charp = rffi.llexternal(
+ "cppyy_stdstring2charp",
+ [C_OBJECT, rffi.SIZE_TP], rffi.CCHARP,
+ releasegil=ts_helper,
+ compilation_info=eci)
+def c_stdstring2charp(space, cppstr):
+ sz = lltype.malloc(rffi.SIZE_TP.TO, 1, flavor='raw')
+ try:
+ cstr = _c_stdstring2charp(cppstr, sz)
+ cstr_len = intmask(sz[0])
+ finally:
+ lltype.free(sz, flavor='raw')
+ return rffi.charpsize2str(cstr, cstr_len)
-# Cling-specific pythonizations
+# TODO: factor these out ...
+# pythonizations
+
+#
+# std::string behavior
+def stdstring_c_str(space, w_self):
+ """Return a python string taking into account \0"""
+
+ from pypy.module.cppyy import interp_cppyy
+ cppstr = space.interp_w(interp_cppyy.W_CPPInstance, w_self, can_be_None=False)
+ return space.wrap(c_stdstring2charp(space, cppstr._rawobject))
+
+#
+# std::vector behavior
+class W_STLVectorIter(W_AbstractSeqIterObject):
+ _immutable_fields_ = ['overload', 'len']#'data', 'converter', 'len', 'stride', 'vector']
+
+ def __init__(self, space, w_vector):
+ W_AbstractSeqIterObject.__init__(self, w_vector)
+ # TODO: this should live in rpythonize.py or something so that the
+ # imports can move to the top w/o getting circles
+ from pypy.module.cppyy import interp_cppyy
+ assert isinstance(w_vector, interp_cppyy.W_CPPInstance)
+ vector = space.interp_w(interp_cppyy.W_CPPInstance, w_vector)
+ self.overload = vector.cppclass.get_overload("__getitem__")
+
+ from pypy.module.cppyy import capi
+ v_type = capi.c_stdvector_valuetype(space, vector.cppclass.name)
+ v_size = capi.c_stdvector_valuesize(space, vector.cppclass.name)
+
+ if not v_type or not v_size:
+ raise NotImplementedError # fallback on getitem
+
+ w_arr = vector.cppclass.get_overload("data").call(w_vector, [])
+ arr = space.interp_w(W_ArrayInstance, w_arr, can_be_None=True)
+ if not arr:
+ raise OperationError(space.w_StopIteration, space.w_None)
+
+ self.data = rffi.cast(rffi.VOIDP, space.uint_w(arr.getbuffer(space)))
+
+ from pypy.module.cppyy import converter
+ self.converter = converter.get_converter(space, v_type, '')
+ self.len = space.uint_w(vector.cppclass.get_overload("size").call(w_vector, []))
+ self.stride = v_size
+
+ def descr_next(self, space):
+ if self.w_seq is None:
+ raise OperationError(space.w_StopIteration, space.w_None)
+ if self.len <= self.index:
+ self.w_seq = None
+ raise OperationError(space.w_StopIteration, space.w_None)
+ try:
+ from pypy.module.cppyy import capi # TODO: refector
+ offset = capi.direct_ptradd(rffi.cast(C_OBJECT, self.data), self.index*self.stride)
+ w_item = self.converter.from_memory(space, space.w_None, space.w_None, offset)
+ except OperationError as e:
+ self.w_seq = None
+ if not e.match(space, space.w_IndexError):
+ raise
+ raise OperationError(space.w_StopIteration, space.w_None)
+ self.index += 1
+ return w_item
+
+def stdvector_iter(space, w_self):
+ return W_STLVectorIter(space, w_self)
+
+# setup pythonizations for later use at run-time
+_pythonizations = {}
def register_pythonizations(space):
"NOT_RPYTHON"
- pass
+
+ allfuncs = [
+
+ ### std::string
+ stdstring_c_str,
+
+ ### std::vector
+ stdvector_iter,
+
+ ]
+
+ for f in allfuncs:
+ _pythonizations[f.__name__] = space.wrap(interp2app(f))
+
+def _method_alias(space, w_pycppclass, m1, m2):
+ space.setattr(w_pycppclass, space.wrap(m1),
+ space.getattr(w_pycppclass, space.wrap(m2)))
def pythonize(space, name, w_pycppclass):
- pass
+ if name == "string":
+ space.setattr(w_pycppclass, space.wrap("c_str"), _pythonizations["stdstring_c_str"])
+ _method_alias(space, w_pycppclass, "_cppyy_as_builtin", "c_str")
+ _method_alias(space, w_pycppclass, "__str__", "c_str")
+
+ if "vector" in name[:11]: # len('std::vector') == 11
+ from pypy.module.cppyy import capi
+ v_type = capi.c_stdvector_valuetype(space, name)
+ if v_type:
+ space.setattr(w_pycppclass, space.wrap("value_type"), space.wrap(v_type))
+ v_size = capi.c_stdvector_valuesize(space, name)
+ if v_size:
+ space.setattr(w_pycppclass, space.wrap("value_size"), space.wrap(v_size))
+ space.setattr(w_pycppclass, space.wrap("__iter__"), _pythonizations["stdvector_iter"])
diff --git a/pypy/module/cppyy/capi/loadable_capi.py b/pypy/module/cppyy/capi/loadable_capi.py
--- a/pypy/module/cppyy/capi/loadable_capi.py
+++ b/pypy/module/cppyy/capi/loadable_capi.py
@@ -1,14 +1,18 @@
from rpython.rtyper.lltypesystem import rffi, lltype
+from rpython.rlib.rarithmetic import intmask
from rpython.rlib import jit, jit_libffi, libffi, rdynload, objectmodel
from rpython.rlib.rarithmetic import r_singlefloat
from rpython.tool import leakfinder
+from pypy.interpreter.gateway import interp2app
from pypy.interpreter.error import oefmt
from pypy.module._cffi_backend import ctypefunc, ctypeprim, cdataobj, misc
+from pypy.module._cffi_backend import newtype
+from pypy.module.cppyy import ffitypes
from pypy.module.cppyy.capi.capi_types import C_SCOPE, C_TYPE, C_OBJECT,\
- C_METHOD, C_INDEX, C_INDEX_ARRAY, WLAVC_INDEX, C_METHPTRGETTER_PTR
+ C_METHOD, C_INDEX, C_INDEX_ARRAY, WLAVC_INDEX, C_FUNC_PTR
reflection_library = 'libcppyy_backend.so'
@@ -21,11 +25,32 @@
class _Arg: # poor man's union
_immutable_ = True
- def __init__(self, h = 0, l = -1, s = '', vp = rffi.cast(rffi.VOIDP, 0)):
+ def __init__(self, tc, h = 0, l = -1, s = '', p = rffi.cast(rffi.VOIDP, 0)):
+ self.tc = tc
self._handle = h
self._long = l
self._string = s
- self._voidp = vp
+ self._voidp = p
+
+class _ArgH(_Arg):
+ _immutable_ = True
+ def __init__(self, val):
+ _Arg.__init__(self, 'h', h = val)
+
+class _ArgL(_Arg):
+ _immutable_ = True
+ def __init__(self, val):
+ _Arg.__init__(self, 'l', l = val)
+
+class _ArgS(_Arg):
+ _immutable_ = True
+ def __init__(self, val):
+ _Arg.__init__(self, 's', s = val)
+
+class _ArgP(_Arg):
+ _immutable_ = True
+ def __init__(self, val):
+ _Arg.__init__(self, 'p', p = val)
# For the loadable CAPI, the calls start and end in RPython. Therefore, the standard
# _call of W_CTypeFunc, which expects wrapped objects, does not quite work: some
@@ -55,14 +80,18 @@
argtype = self.fargs[i]
# the following is clumsy, but the data types used as arguments are
# very limited, so it'll do for now
- if isinstance(argtype, ctypeprim.W_CTypePrimitiveSigned):
+ if obj.tc == 'l':
+ assert isinstance(argtype, ctypeprim.W_CTypePrimitiveSigned)
misc.write_raw_signed_data(data, rffi.cast(rffi.LONG, obj._long), argtype.size)
- elif isinstance(argtype, ctypeprim.W_CTypePrimitiveUnsigned):
+ elif obj.tc == 'h':
+ assert isinstance(argtype, ctypeprim.W_CTypePrimitiveUnsigned)
misc.write_raw_unsigned_data(data, rffi.cast(rffi.ULONG, obj._handle), argtype.size)
- elif obj._voidp != rffi.cast(rffi.VOIDP, 0):
+ elif obj.tc == 'p':
+ assert obj._voidp != rffi.cast(rffi.VOIDP, 0)
data = rffi.cast(rffi.VOIDPP, data)
data[0] = obj._voidp
else: # only other use is sring
+ assert obj.tc == 's'
n = len(obj._string)
assert raw_string == rffi.cast(rffi.CCHARP, 0)
# XXX could use rffi.get_nonmovingbuffer_final_null()
@@ -89,35 +118,36 @@
self.library = None
self.capi_calls = {}
- import pypy.module._cffi_backend.newtype as nt
+ nt = newtype # module from _cffi_backend
+ state = space.fromcache(ffitypes.State) # factored out common types
# TODO: the following need to match up with the globally defined C_XYZ low-level
# types (see capi/__init__.py), but by using strings here, that isn't guaranteed
- c_opaque_ptr = nt.new_primitive_type(space, 'unsigned long')
+ c_opaque_ptr = state.c_ulong
- c_scope = c_opaque_ptr
- c_type = c_scope
- c_object = c_opaque_ptr
- c_method = c_opaque_ptr
- c_index = nt.new_primitive_type(space, 'long')
+ c_scope = c_opaque_ptr
+ c_type = c_scope
+ c_object = c_opaque_ptr
+ c_method = c_opaque_ptr
+ c_index = state.c_long
+ c_index_array = state.c_voidp
- c_void = nt.new_void_type(space)
- c_char = nt.new_primitive_type(space, 'char')
- c_uchar = nt.new_primitive_type(space, 'unsigned char')
- c_short = nt.new_primitive_type(space, 'short')
- c_int = nt.new_primitive_type(space, 'int')
- c_long = nt.new_primitive_type(space, 'long')
- c_llong = nt.new_primitive_type(space, 'long long')
- c_ullong = nt.new_primitive_type(space, 'unsigned long long')
- c_float = nt.new_primitive_type(space, 'float')
- c_double = nt.new_primitive_type(space, 'double')
+ c_void = state.c_void
+ c_char = state.c_char
+ c_uchar = state.c_uchar
+ c_short = state.c_short
+ c_int = state.c_int
+ c_long = state.c_long
+ c_llong = state.c_llong
+ c_ullong = state.c_ullong
+ c_float = state.c_float
+ c_double = state.c_double
+ c_ldouble = state.c_ldouble
- c_ccharp = nt.new_pointer_type(space, c_char)
- c_index_array = nt.new_pointer_type(space, c_void)
+ c_ccharp = state.c_ccharp
+ c_voidp = state.c_voidp
- c_voidp = nt.new_pointer_type(space, c_void)
c_size_t = nt.new_primitive_type(space, 'size_t')
-
c_ptrdiff_t = nt.new_primitive_type(space, 'ptrdiff_t')
self.capi_call_ifaces = {
@@ -127,7 +157,6 @@
'resolve_name' : ([c_ccharp], c_ccharp),
'get_scope' : ([c_ccharp], c_scope),
- 'get_template' : ([c_ccharp], c_type),
'actual_class' : ([c_type, c_object], c_type),
# memory management
@@ -146,14 +175,16 @@
'call_ll' : ([c_method, c_object, c_int, c_voidp], c_llong),
'call_f' : ([c_method, c_object, c_int, c_voidp], c_float),
'call_d' : ([c_method, c_object, c_int, c_voidp], c_double),
+ 'call_ld' : ([c_method, c_object, c_int, c_voidp], c_ldouble),
'call_r' : ([c_method, c_object, c_int, c_voidp], c_voidp),
- 'call_s' : ([c_method, c_object, c_int, c_voidp], c_ccharp),
+ # call_s actually takes an size_t* as last parameter, but this will do
+ 'call_s' : ([c_method, c_object, c_int, c_voidp, c_voidp], c_ccharp),
'constructor' : ([c_method, c_object, c_int, c_voidp], c_object),
'call_o' : ([c_method, c_object, c_int, c_voidp, c_type], c_object),
- 'get_methptr_getter' : ([c_scope, c_index], c_voidp), # TODO: verify
+ 'get_function_address' : ([c_scope, c_index], c_voidp), # TODO: verify
# handling of function argument buffer
'allocate_function_args' : ([c_int], c_voidp),
@@ -163,6 +194,8 @@
# scope reflection information
'is_namespace' : ([c_scope], c_int),
+ 'is_template' : ([c_ccharp], c_int),
+ 'is_abstract' : ([c_type], c_int),
'is_enum' : ([c_ccharp], c_int),
# type/class reflection information
@@ -216,8 +249,14 @@
'strtoull' : ([c_ccharp], c_ullong),
'free' : ([c_voidp], c_void),
- 'charp2stdstring' : ([c_ccharp], c_object),
+ 'charp2stdstring' : ([c_ccharp, c_size_t], c_object),
+ #stdstring2charp actually takes an size_t* as last parameter, but this will do
+ 'stdstring2charp' : ([c_object, c_voidp], c_ccharp),
'stdstring2stdstring' : ([c_object], c_object),
+
+ 'stdvector_valuetype' : ([c_ccharp], c_ccharp),
+ 'stdvector_valuesize' : ([c_ccharp], c_size_t),
+
}
# size/offset are backend-specific but fixed after load
@@ -277,87 +316,99 @@
ptr = w_cdata.unsafe_escaping_ptr()
return rffi.cast(rffi.VOIDP, ptr)
+def _cdata_to_ccharp(space, w_cdata):
+ ptr = _cdata_to_ptr(space, w_cdata) # see above ... something better?
+ return rffi.cast(rffi.CCHARP, ptr)
+
def c_load_dictionary(name):
return libffi.CDLL(name)
# name to opaque C++ scope representation ------------------------------------
def c_num_scopes(space, cppscope):
- return space.int_w(call_capi(space, 'num_scopes', [_Arg(h=cppscope.handle)]))
+ return space.int_w(call_capi(space, 'num_scopes', [_ArgH(cppscope.handle)]))
def c_scope_name(space, cppscope, iscope):
- args = [_Arg(h=cppscope.handle), _Arg(l=iscope)]
+ args = [_ArgH(cppscope.handle), _ArgL(iscope)]
return charp2str_free(space, call_capi(space, 'scope_name', args))
def c_resolve_name(space, name):
- return charp2str_free(space, call_capi(space, 'resolve_name', [_Arg(s=name)]))
+ return charp2str_free(space, call_capi(space, 'resolve_name', [_ArgS(name)]))
def c_get_scope_opaque(space, name):
- return rffi.cast(C_SCOPE, space.uint_w(call_capi(space, 'get_scope', [_Arg(s=name)])))
-def c_get_template(space, name):
- return rffi.cast(C_TYPE, space.uint_w(call_capi(space, 'get_template', [_Arg(s=name)])))
+ return rffi.cast(C_SCOPE, space.uint_w(call_capi(space, 'get_scope', [_ArgS(name)])))
def c_actual_class(space, cppclass, cppobj):
- args = [_Arg(h=cppclass.handle), _Arg(h=cppobj)]
More information about the pypy-commit
mailing list