[IronPython] CPython access

[Jim Hugunin]

Timothy Fitz wrote:
> Where there be any plans for allowing IronPython to work with an
> instance of CPython directly? There are quite a few instances where
> this would be extremely handy (SWIG-wrapped libraries such as
> wxPython, or tightly optimized pieces that deal directly with
> operating system concepts such as Twisted reactors where rewriting in
> IronPython or converting may not be trivial enough)

There are several options for supporting existing C-based Python
modules.

1. Rewrite the modules in C# using .NET libraries. This will produce the
smallest and best integrated modules at the end of the day, but requires
substantial work per module. This is what has been done for Jython.  For
modules with deep integration with the runtime, like __builtins__, sys,
gc, thread there isn't any other good answer.  Other modules like
socket, re, xml, etc. have close counterparts in the .NET space and it's
very tempting to provide these as small adapters over the existing
libraries. For the large external libraries like wxPython, opengl, etc.
there is often a .NET wrapper already provided that could make it simple
once again to provide a small adapter to match the Python module's API. 

2. Build a bridge like you suggest using the existing CPython dll.
There's a substantial amount of upfront work involved in writing the
bridge, but once it was done it should be easy (no work at all
hopefully) to bring in existing Python modules to IronPython.  This
would be great from the perspective of making tons of existing Python
modules just work in IronPython, but there are some significant
drawbacks.

The bridge would need to translate between CPython and IronPython
objects and this would never be a completely seamless translation.
There would be a considerable per-call performance overhead so that this
would probably be a poor choice for APIs with small functions that get
called a lot where perf matters, Numeric comes to mind.  The fact that
you now have two separate memory managers means you can wind up with
uncollectible cycles, but that might be rare in practice.  The bridge
would also give up the advantages of a unified cross-language debugger,
the ability to have purely managed verifiable .NET assemblies to run in
secure contexts, and some other benefits of tight integration with the
platform.  On the CPython side, modules that used deep engine features
would be hard/impossible to bridge so there would be a group of modules
that couldn't work here.

3. A crazy idea is to arrange to port the modules to managed C++
combined with a fake set of include files that makes the modules really
link with IronPython rather than CPython. In theory, this could let the
modules work well with IronPython with only a recompile required. This
could be really cool if it worked out, but I'm the least confident that
this option is truly possible.

Thanks - Jim