[pypy-dev] [ANN] Python compilers workshop at SciPy this year

[John Camara]

It turns out there is some work in progress in the Spark project to share
its memory with non JVM programs. See
https://issues.apache.org/jira/browse/SPARK-10399.  Once this is completed
it should be fairly trivial to expose it to Python and then maybe JIT
integration could be discussed at that time.  This is a huge step forward
over sharing Java objects.  From the title of the ticket it appears it
would be a c++ interface but looking at the pull request it looks like it
will be a c interface.

In the end the blocker may just come down to PyPy having complete support
for Numpy. Without Numpy the success of this would be somewhat limited
based on user expectations and without PyPy it maybe to slow for many
applications.

On Thu, Mar 24, 2016 at 1:11 PM, John Camara <john.m.camara at gmail.com>
wrote:

> Hi Armin,
>
> At a minimum tighter execution is required as well as sharing memory.  But
> on the other hand you have raised the bar so high with cffi, having a clean
> and unbloated interface, that it would be nice if a library with a similar
> spirit existed for java. Having support in PyPy's JIT to remove all the
> marshalling types would be a big plus on top of the shared memory as well
> as some integration between the 2 GCs would likely be required.
>
> Maybe the best approach would be a combination of existing libraries and a
> new interface that allows for sharing of memory.  Maybe similar to numpy
> arrays with a better API that avoids the pit falls of numpy relying on
> CPython semantics/implementation details.  After all the only thing that
> needs to be eliminated is the copying/serialization of large data
> arrays/structures.
>
> John
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20160324/cca9759c/attachment.html>