[Cython] PEP 3135 -- New Super

[Vitja Makarov]

2011/7/5 Stefan Behnel <stefan_ml at behnel.de>:
> Vitja Makarov, 05.07.2011 09:17:
>>
>> 2011/7/5 Stefan Behnel:
>>>
>>> Vitja Makarov, 05.07.2011 08:21:
>>>>
>>>> I was thinking about implementing new super() with no arguments.
>>>
>>> http://trac.cython.org/cython_trac/ticket/696
>>>
>>>> The problem is where to store __class__, I see two options here:
>>>>
>>>> 1. Add func_class member to CyFunction, this way __class__ will be
>>>> private and not visible for inner functions:
>>>> 2. Put it into closure
>>>
>>> The second option has the advantage of requiring the field only when
>>> super()
>>> is used, whereas the first impacts all functions.
>>>
>>> I would expect that programs commonly have a lot more functions than
>>> specifically methods that use a no-argument call to super(), so this may
>>> make a difference.
>>>
>>
>> So, now classes are created the following way:
>>
>> class_dict = {}
>> class_dict.foo = foo_func
>> class = CreateClass(class_dict)
>>
>> So after class is created I should check its dict for CyFunction
>> members (maybe only ones that actually require __class__)
>> and set __class__:
>>
>> for value in class.__dict__.itervalues():
>>    if isinstance(value, CyFunction) and value.func_class is WantClass:
>>        value.func_class = class
>
> Remember that no-args super() can only be used in functions that are
> literally written inside of a class body, so we actually know at compile
> time which functions need this field. We can thus do better than a generic
> loop over all fields. We even have the function object pointers directly
> available in the module init function where we create the class body.
>

Yes, but now cyfunction references are lost as they were in temps.

> BTW, we also need a way to make this work for cdef classes. No idea how
> different that would be.
>

I think super() for cdef classes should be much easy to implement as
we already know cname for a class at compile time. That would be
simple transform.

>
>>> OTOH, not all methods have a closure, so creating one just to store the
>>> "__class__" field is very wasteful, in terms of both runtime and memory
>>> overhead. A lot more wasteful than paying 8 bytes of memory for each
>>> function, with no additional time overhead.
>>
>> Going this way it only requires to initialize closure:
>
> Yes, and that's costly.
>
>
>> Btw, first way requires cyfunction signature change, it would accept
>> cyfunction object as first argument.
>
> We currently pass the binding (i.e. owning) object, right?
>

Right. When we pass CyFunction closure should be moved into its members.

>
>> This also could help to solve default args problem.
>
> And potentially other problems, too. Think of heap allocated modules, for
> example.
>
> http://trac.cython.org/cython_trac/ticket/173
> http://trac.cython.org/cython_trac/ticket/218
>
> Seeing this, I'm all for using a field in CyFunction.
>
>
>> Yeah, I think Cython super with no args should be a little bit faster
>> then classic one.
>
> I think speed isn't really all that important here. Calling Python methods
> is costly enough anyway.
>
> IMO, the main reason for the heuristic is to prevent class objects from
> being kept alive by their methods, except for the single case where super()
> is used. Keeping a class alive just because one of its methods is still used
> somewhere can be very costly, depending on the content of the class dict. It
> also creates a reference cycle, which is another costly thing in CPython as
> it requires a GC run over the whole class dict to get cleaned up.
>
> The situation for modules is at least slightly different, as modules do not
> tend to get unloaded, so there's always a reference to them from
> sys.modules. If that ever gets removed, it's really ok if it takes time to
> clean things up.
>

Yes, but I hope that classes are rarely deleted.
And I'm afraid that we can't avoid cycles when implementing __class__
for pure-python classes.

-- 
vitja.