On-topic: alternate Python implementations

[MRAB]

On 04/08/2012 20:06, Stefan Behnel wrote:
> Paul Rubin, 04.08.2012 20:18:
>> Stefan Behnel writes:
>>>> C is pretty poor as a compiler target: how would you translate Python
>>>> generators into C, for example?
>>> Depends. If you have CPython available, that'd be a straight forward
>>> extension type.
>>
>> Calling CPython hardly counts as compiling Python into C.
>
> CPython is written in C, though. So anything that CPython does can be done
> in C. It's not like the CPython project used a completely unusual way of
> writing C code.
>
> Besides, I find your above statement questionable. You will always need
> some kind of runtime infrastructure when you "compile Python into C", so
> you can just as well use CPython for that instead of reimplementing it
> completely from scratch. Both Cython and Nuitka do exactly that, and one of
> the major advantages of that approach is that they can freely interact with
> arbitrary code (Python or not) that was written for CPython, regardless of
> its native dependencies. What good would it be to throw all of that away,
> just for the sake of having "pure C code generation"?
>
>
>>> For the yielding, you can use labels and goto. Given that you generate
>>> the code, that's pretty straight forward as well.
>>
>> You're going to compile the whole Python program into a single C
>> function so that you can do gotos inside of it?  What happens if the
>> program imports a generator?
>
> No, you are going to compile only the generator function into a function
> that uses gotos, maybe with an additional in-out struct parameter that
> holds its state. Then, on entry, you read the label (or its ID) from the
> previous state, reset local variables and jump to the label. On exit, you
> store the state back end return. Cython does it that way. Totally straight
> forward, as I said.
>
>
>>>> How would you handle garbage collection?
>>> CPython does it automatically for us at least.
>>
>> You mean you're going to have all the same INCREF/DECREF stuff on every
>> operation in compiled data?  Ugh.
>
> If you don't like that, you can experiment with anything from a dedicated
> GC to transactional memory.
>
>
>>> Lacking that, you'd use one of the available garbage collection
>>> implementations,
>>
>> What implementations would those be?  There's the Boehm GC which is
>> useful for some purposes but not really suitable at large scale, from
>> what I can tell.  Is there something else?
>
> No idea - I'll look it up when I need one. Last I heard, PyPy had a couple
> of GCs to choose from, but I don't know how closely the are tied into its
> infrastructure.
>
>
>>> or provide none at all.
>>
>> You're going to let the program just leak memory until it crashes??
>
> Well, it's not like CPython leaks memory until it crashes, now does it? And
> it's written in C. So there must be ways to handle this also in C.
>
> Remember that CPython didn't even have a GC before something around 2.0,
> IIRC. That worked quite ok in most cases and simply left the tricky cases
> to the programmers. It really depends on what your requirements are. Small
> embedded systems, time critical code and real-time systems are often much
> better off without garbage collection. It's pure convenience, after all.
>
[snip]
CPython relied entirely on reference counting, so memory could leak you 
if inadvertently created a cycle of memory references. That problem was
fixed when a mark-and-sweep mechanism was added (it's called
occasionally to collect any unreachable cycles).