[pypy-dev] JVM integration

[Antonio Cuni]

Paul deGrandis wrote:
> Anto, Niko, and All,

HI Paul,

> I have begun working on PyPy again and am starting on jvm integration.
>  After talking briefly with Carl on IRC, I looked at the code in CLI.
> 
> I have a few questions:
> It appears as though the general approach is to identify entry points
> into CLI code (via dlls), use reflection, and make calls.  This is
> also later used to make (r)python modules into dlls themselves
> (allowing for bidirectional integration).  Is this correct?  

I'm not sure to understand what you mean, so I briefly explain how 
things work for CLI:

   1. (interp-level) you can use .NET libraries from RPython; this means 
that in theory you can do in RPython everything you do in C# (modulo the 
fact that the support is not complete right now, but it could be in the 
future)

   2. (app-level) you can use .NET libraries from pypy-cli; to do this, 
you go through the clr module: part of such a module is written in 
RPython and contains calls to .NET reflection API, which are used to 
dynamically access .NET libraries from python

Obviously, point 2 depends on point 1 to work.

In the source code, point 1 is implemented by translator/cli/dotnet.py, 
while point 2 is implemented by module/clr/*.py.

In both cases, probably a lot of code can be shared between the CLI and 
JVM versions; so, the first thing to do would be to split dotnet.py into 
a backend-independent part (probably to be put in translator/oosupport/) 
and the CLI specific part, then write the JVM specific code. The same 
for the code in module/clr/.

 > Also, how
 > does the notion of PythonNet play into this (ie: where do you get the
 > CLR module from)?

Look again at the point (1) above; ok, we can use .NET libraries in 
RPython; consider this RPython snippet:

def main():
     System.Console.WriteLine("Hello World")

when we translate it trough gencli, we get a .NET exe whose bytecode 
contains a call to System.Console::WriteLine, and everything is fine.

On the other hand, one of the greatest features of RPython is that we 
can test RPython programs directly in CPython, without the need to 
translate them; as you probably already know, if you type e.g. py.py you 
run the PyPy interpreter on top of CPython, which is extremely good for 
testing.

Now, suppose you want to run the snippet above on top of CPython, i.e. 
*without* translating it; how can it work? CPython can't call 
System.Console.WriteLine, can it?

This is where PythonNet come to the action: it allows CPython to access 
.NET libraries; so, even if we can't run that snippet directly on 
CPython, we can run it on PythonNet (which is nothing else than a thin 
wrapper of CPython).

To be clearer, the only role that PythonNet plays is to allow testing 
without the need of a translation. If you look at 
translator/cli/test/test_dotnet.py, you can see the TestPythonnet class: 
this class runs all the tests in TestDotnetRtyping to ensure that every 
snippet behave the same when translated and when interpreted.

There are equivalent of PythonNet for JVM; I think the most promising is 
JPype, which Carl Friedrich tried some time ago.

> If this is not the current approach, is there a document that someone
> can point me to that talks about this in further detail?
> Lastly, as work for integration on the JVM begins, a larger goal is
> the generalize the approach.  Has any work been done in the CLI/CLR to
> generalize the approach so far?

I wrote that code with JVM in mind, so I think it wouldn't be too hard 
to split the CLR specific parts from the general purpose parts.


ciao Anto