[Cython] Speedup module-level lookup

[Robert Bradshaw]

On Thu, Jan 19, 2012 at 12:18 AM, Chris Colbert <sccolbert at gmail.com> wrote:
> If it doesn't pass PyDict_CheckExact you won't be able to use it as the
> globals to eval or exec.

:(. I wonder how many other places have similar restrictions, perhaps
even implicitly. In particular, this would mean that an eval statement
modifying globals() would be difficult to efficiently detect. Still,
if this can be done at all, the massive speedup for the common case
could make it worth it.

> That's assuming you hack the module type so you can
> change its __dict__.

I don't think that's near as big of a hurdle (from C).

> On Jan 19, 2012 2:01 AM, "Robert Bradshaw" <robertwb at math.washington.edu>
> wrote:
>>
>> On Wed, Jan 18, 2012 at 11:53 PM, Vitja Makarov <vitja.makarov at gmail.com>
>> wrote:
>> > 2012/1/19 Vitja Makarov <vitja.makarov at gmail.com>:
>> >> 2012/1/19 Robert Bradshaw <robertwb at math.washington.edu>:
>> >>> I think the right thing to do here is make all module-level globals
>> >>> into "cdef public" attributes, i.e. C globals with getters and setters
>> >>> for Python space. I'm not sure whether this would best be done by
>> >>> creating a custom dict or module subclass, but it would probably be
>> >>> cleaner and afford much more than a 1.6x speedup.
>> >>>
>> >>> - Robert
>> >>>
>> >>> On Wed, Jan 18, 2012 at 12:30 PM, Vitja Makarov
>> >>> <vitja.makarov at gmail.com> wrote:
>> >>>> I tried to optimize module lookups (__pyx_m) by caching internal
>> >>>> PyDict state.
>> >>>>
>> >>>> In this example bar() is 1.6 time faster (500us against 842us):
>> >>>>
>> >>>> C = 123
>> >>>> def foo(a):
>> >>>>     return C * adef bar():
>> >>>>     for i in range(10000):        foo(i)
>> >>>> Here is proof of
>> >>>>
>> >>>> concept:https://github.com/vitek/cython/commit/1d134fe54a74e6fc6d39d09973db499680b2a8d9
>> >>>>
>> >>>> So the question is: does it worth it?
>> >>>>
>> >>
>> >> Yes, nice idea.
>> >> It's possible to subclass PyModuleObject and I didn't find any use of
>> >> PyModule_CheckExact() in CPython's sources:
>> >>
>> >> import types
>> >> import sys
>> >>
>> >> global_foo = 1234
>> >>
>> >> class CustomModule(types.ModuleType):
>> >>    def __init__(self, name):
>> >>        types.ModuleType.__init__(self, name)
>> >>        sys.modules[name] = self
>> >>
>> >>    @property
>> >>    def foo(self):
>> >>        return global_foo
>> >>
>> >>    @foo.setter
>> >>    def foo(self, value):
>> >>        global global_foo
>> >>        global_foo = value
>> >>
>> >> CustomModule('foo')
>> >>
>> >> import foo
>> >> print foo.foo
>> >>
>> >
>> > But this seems to break globals().
>>
>> How so? We have to hack globals() to get it to work for a Cython
>> module anyways. (I wonder if this must return a dict, or would any
>> mapping (or subclass of dict) be sufficient...)
>>
>> - Robert
>> _______________________________________________
>> cython-devel mailing list
>> cython-devel at python.org
>> http://mail.python.org/mailman/listinfo/cython-devel
>
>
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>