[Cython] non-virtual methods

Tue Aug 30 18:18:47 CEST 2011

On Tue, Aug 30, 2011 at 9:14 AM, Vitja Makarov <vitja.makarov at gmail.com> wrote:
> 2011/8/30 Robert Bradshaw <robertwb at math.washington.edu>:
>> On Mon, Aug 29, 2011 at 10:57 PM, Vitja Makarov <vitja.makarov at gmail.com> wrote:
>>> 2011/8/30 Robert Bradshaw <robertwb at math.washington.edu>:
>>>> On Sun, Aug 28, 2011 at 4:00 AM, Vitja Makarov <vitja.makarov at gmail.com> wrote:
>>>>> 2011/8/27 Vitja Makarov <vitja.makarov at gmail.com>:
>>>>>> 2011/8/27 Vitja Makarov <vitja.makarov at gmail.com>:
>>>>>>> 2011/8/26 Vitja Makarov <vitja.makarov at gmail.com>:
>>>>>>>> 2011/8/25 Vitja Makarov <vitja.makarov at gmail.com>:
>>>>>>>>> 2011/8/25 Vitja Makarov <vitja.makarov at gmail.com>:
>>>>>>>>>> 2011/8/25 Stefan Behnel <stefan_ml at behnel.de>:
>>>>>>>>>>> Vitja Makarov, 25.08.2011 20:32:
>>>>>>>>>>>>
>>>>>>>>>>>> 2011/8/25 Stefan Behnel<stefan_ml at behnel.de>:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Vitja Makarov, 25.08.2011 18:11:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2011/8/24 Stefan Behnel:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Vitja Makarov, 24.08.2011 21:17:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I tried final classes:
>>>>>>>>>>>>>>>> 2. In this example foo call is done through virtual table
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> cimport cython
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> @cython.final
>>>>>>>>>>>>>>>> cdef class Foo:
>>>>>>>>>>>>>>>>     cdef foo(self):
>>>>>>>>>>>>>>>>         print 'haha'
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> def test():
>>>>>>>>>>>>>>>>     cdef Foo a = Foo()
>>>>>>>>>>>>>>>>     a.foo()
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>   __pyx_t_1 = ((struct __pyx_vtabstruct_3yyy_Foo
>>>>>>>>>>>>>>>> *)__pyx_v_a->__pyx_vtab)->foo(__pyx_v_a); if (unlikely(!__pyx_t_1))
>>>>>>>>>>>>>>>> {__pyx_filename = __pyx_f[0]; __pyx_lineno = 12; __pyx_clineno =
>>>>>>>>>>>>>>>> __LINE__; goto __pyx_L1_error;}
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Right, this is not implemented yet. Feel free to do so. Also see
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://trac.cython.org/cython_trac/ticket/474
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 3. I can't use final decorator for methods (error reported)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://trac.cython.org/cython_trac/ticket/586
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What is the prefered syntax keyword inline or final decorator?
>>>>>>>>>>>>>
>>>>>>>>>>>>> "final" is the right option here.
>>>>>>>>>>>>>
>>>>>>>>>>>>> They are orthogonal concepts. Only because you declare a method "final"
>>>>>>>>>>>>> does
>>>>>>>>>>>>> not mean you want to inline it, and just because you declare it "inline"
>>>>>>>>>>>>> does not (necessarily) mean that you cannot override it. Admittedly, the
>>>>>>>>>>>>> semantics of an overridable inline method may turn out to be somewhat
>>>>>>>>>>>>> obscure and error prone, so I think it's a good idea to let "inline"
>>>>>>>>>>>>> imply
>>>>>>>>>>>>> "final". But not the other way round.
>>>>>>>>>>>>
>>>>>>>>>>>> But both inline and final methods should bypass vtab, right?
>>>>>>>>>>>
>>>>>>>>>>> Yes. But in the "final" case, it's always clear which method implementation
>>>>>>>>>>> to use - it's not overridable, so there is only one choice. In the "inline"
>>>>>>>>>>> case, it could still be overridable and we may have a subtype of the
>>>>>>>>>>> declared type in our hands at runtime, thus choosing the wrong method at
>>>>>>>>>>> compile time. That's why only the "final" case is safe.
>>>>>>>>>>>
>>>>>>>>>>> Note that I'm only talking about the semantics of the qualifier themselves
>>>>>>>>>>> here. If we allow "inline" methods, I think we should force them to be
>>>>>>>>>>> "final" as well. But that's a practical choice, not a semantic implication.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Also I'm not sure about C inline qualifier here.
>>>>>>>>>>>
>>>>>>>>>>> That's what "inline" requests.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> I see three options:
>>>>>>>>>>>>
>>>>>>>>>>>>  - non-virtual: bypass vtab
>>>>>>>>>>>>  - final: non-virtual, non-overridable
>>>>>>>>>>>
>>>>>>>>>>> How would you want to bypass the vtable in the "non-virtual" case if the
>>>>>>>>>>> method is overridable?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>  - inline: non-virtual, C inline qualifier is used
>>>>>>>>>>>
>>>>>>>>>>> Correct.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Ok.
>>>>>>>>>>
>>>>>>>>>> I think it's better to implement final method then user could choose
>>>>>>>>>> to use inline qualifier or not.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I tried it here:
>>>>>>>>> https://github.com/vitek/cython/commit/ddf80a80dc75aced2cd92dc32afa77a7bcf2de02
>>>>>>>>>
>>>>>>>>> There is one problem: vtab bypassing should be enabled if final method
>>>>>>>>> is defined in the same module.
>>>>>>>>> I don't know how to check that final method comes from cimport (it's
>>>>>>>>> okay with pxd, the problem is pyx)
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> https://github.com/vitek/cython/commit/6e91fc257a683018ba6be340d384f9a7c34ef425
>>>>>>>>
>>>>>>>> Here is update version. I've add tree asserts and final method's prototypes.
>>>>>>>>
>>>>>>>> --
>>>>>>>> vitja.
>>>>>>>>
>>>>>>>
>>>>>>> I've created ticket for compiler crash when cython.final is used
>>>>>>> inside pxd file:
>>>>>>>
>>>>>>> http://trac.cython.org/cython_trac/ticket/722
>>>>>>>
>>>>>>> Also I've updated final methods test case (added tree path assertions)
>>>>>>> https://github.com/vitek/cython/commit/92edb09419c9b77a792f7c43e6ddd760b00c4e74
>>>>>>>
>>>>>>>
>>>>>>> About declaration origin detection may be it's a good idea to have a
>>>>>>> flag at scope level something like is_pxd_scope or
>>>>>>> is_declaration_scope?
>>>>>>>
>>>>>>
>>>>>> It seems to me that I found a way to fix pxd/pyx cimport problem. I've
>>>>>> created pull request:
>>>>>>
>>>>>> https://github.com/cython/cython/pull/59
>>>>>>
>>>>>
>>>>> I've add support for inline methods, now you can declare inline method
>>>>> in pxd file:
>>>>>
>>>>> cdef class Foo:
>>>>>   cdef inline foo(self):
>>>>>       return 1
>>>>
>>>> Cool.
>>>>
>>>> I suppose for cross-module calls, non-inline methods vtables are best
>>>> one can do short of actually linking the modules together.
>>>
>>> Sure. This is how it works now. But noitice that final isn't supported
>>> now at pxd scope. So it might not work.
>>>
>>>
>>> https://github.com/cython/cython/pull/59/files#L5R1888
>>>
>>> For subclassing I check that both 3parent and children are in the same scope.
>>> When pxd is cimported vtable bypassing should be disabled in case it's
>>> cimported from anpther module.
>>> Not sure how to implement this. I think we should first fix final at pxd.
>>>
>>>> One
>>>> question about your code--I'm not seeing how you're disallowing
>>>> overriding final cpdef methods from Python. (Should we even allow
>>>> final cpdef methods on non-final classes?)
>>>>
>>>
>>> I think it's hard and tricky to disallow cpdef overriding but I think
>>> it's better to have cpdef final methods.
>>> And I wouldn't argue if you say that final cpdef methods shouldn't be
>>> allowed at all.
>>
>> Yes, disallowing it with an explicit compile-time error was what I was
>> thinking.
>>
>
> What about final classes with cpdef methods?
>
> @cython.final
> class Foo:
>    cpdef bar(self):
>        pass
>
> Should that raise an error?

That should be perfectly fine.

- Robert