[pypy-dev] python 2 and python 3 sharing an interpreter?

[Bengt Richter]

On 12/09/2011 01:09 PM Martijn Faassen wrote:
> Hey Armin,
>
> Thanks for your response!
>
> On Fri, Dec 9, 2011 at 12:31 PM, Armin Rigo<arigo at tunes.org>  wrote:
>
>> On Thu, Dec 8, 2011 at 20:12, Martijn Faassen<faassen at startifact.com>  wrote:
>>> Anyway, separate from the binding, I'm wondering about the challenge of two
>>> Pythons in one runtime - how hard would it be to generate this with the
>>> current PyPy? (imagining two different Python interpreters were available)
>>
>> Getting two completely separate interpreters in one process is trivial
>> in PyPy, quite unlike CPython (blame C).  This works even with the GC
>> (you get one for both) and the JIT (you get one JIT that can
>> "meta-trace" starting from either interpreter).  That means that maybe
>> PyPy is better than CPython as a starting point.
>
> Most definitely, I'd say!
>
>> Of course someone
>> still has a serious amount of work to do to integrate the two
>> interpreters to the point where it becomes something more than, well,
>> two unrelated interpreters in one process.
>
> Yes, that's the challenge. But that sounds like a feasible challenge -
> PyPy also has support for transparent proxies, so my hope is that you
> could write a Python 2 to Python 3 object proxy and one that does it
> the other way around. If you have that and some import namespace
> separation hackery you'd already be quite far.
>
>> [ More generally, you can translate several interpreters for different
>> languages and getting a single unified JIT, capable of inlining across
>> languages.  It is an interesting challenge that explores some unique
>> benefits of PyPy --- worth a paper, most probably :-) ]
>
> That is indeed a major benefit. It would allow a much more seamless
> upgrade process between multiple incompatible versions of programming
> languages. In the Python 2 and Python 3 case, thanks to the unified
> JIT you might pay only very little penalty for the integration,
> depending on how the JIT and proxies would interact.
>
> Of course all this would have to wait for a Python 3 implementation in
> PyPy too. I was unenthusiastic about that before, as I believe PyPy is
> in a unique competitive position right now compared to CPython as an
> actively developed Python 2 implementation. But having one would make
> this integration possible, which I believe would solve endless issues
> the community has now with the discontinuity between the two
> languages.
>
> In the mean time it should be possible to experiment with just two
> Python 2 interpreters in the same process space, and work on a system
> to share modules between them. It would not provide any real benefit
> to just a single Python 2 as far as I can tell, but it could serve as
> a proof of concept.
>
> Is anyone interested in working with me on the latter experiment?
>
> Regards,
>
> Martijn

PMJI, but I was thinking that if you had both py272 and py3k modules it would seem
natural to keep them in separate directories, so I was wondering if
__init__.py could do some useful work in effecting translations containing
both kinds of sources? And maybe a command line option (-py2x or -py3x) could
tell what kind of main to expect? Polyglot packages? ;-)

Maybe __init__.py could also invoke some kind of annotation
("handwave" ;-) file processing?

Regards,
Bengt Richter