[Cython] Speedup module-level lookup

[mark florisson]

On 19 January 2012 08:00, Robert Bradshaw <robertwb at math.washington.edu> wrote:
> On Wed, Jan 18, 2012 at 11:53 PM, Vitja Makarov <vitja.makarov at gmail.com> wrote:
>> 2012/1/19 Vitja Makarov <vitja.makarov at gmail.com>:
>>> 2012/1/19 Robert Bradshaw <robertwb at math.washington.edu>:
>>>> I think the right thing to do here is make all module-level globals
>>>> into "cdef public" attributes, i.e. C globals with getters and setters
>>>> for Python space. I'm not sure whether this would best be done by
>>>> creating a custom dict or module subclass, but it would probably be
>>>> cleaner and afford much more than a 1.6x speedup.
>>>>
>>>> - Robert
>>>>
>>>> On Wed, Jan 18, 2012 at 12:30 PM, Vitja Makarov <vitja.makarov at gmail.com> wrote:
>>>>> I tried to optimize module lookups (__pyx_m) by caching internal PyDict state.
>>>>>
>>>>> In this example bar() is 1.6 time faster (500us against 842us):
>>>>>
>>>>> C = 123
>>>>> def foo(a):
>>>>>     return C * adef bar():
>>>>>     for i in range(10000):        foo(i)
>>>>> Here is proof of
>>>>> concept:https://github.com/vitek/cython/commit/1d134fe54a74e6fc6d39d09973db499680b2a8d9
>>>>>
>>>>> So the question is: does it worth it?
>>>>>
>>>
>>> Yes, nice idea.
>>> It's possible to subclass PyModuleObject and I didn't find any use of
>>> PyModule_CheckExact() in CPython's sources:
>>>
>>> import types
>>> import sys
>>>
>>> global_foo = 1234
>>>
>>> class CustomModule(types.ModuleType):
>>>    def __init__(self, name):
>>>        types.ModuleType.__init__(self, name)
>>>        sys.modules[name] = self
>>>
>>>    @property
>>>    def foo(self):
>>>        return global_foo
>>>
>>>    @foo.setter
>>>    def foo(self, value):
>>>        global global_foo
>>>        global_foo = value
>>>
>>> CustomModule('foo')
>>>
>>> import foo
>>> print foo.foo
>>>
>>
>> But this seems to break globals().
>
> How so? We have to hack globals() to get it to work for a Cython
> module anyways. (I wonder if this must return a dict, or would any
> mapping (or subclass of dict) be sufficient...)
>
> - Robert
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel

You'd also want this to work from python space using module.__dict__
or vars(module). I think the custom dict could solve this. Or would
you make __dict__ a property as well? (I don't know if vars() would
still break).