[Cython] Core dump when Python is built without debug symbols

[Marin Atanasov Nikolov]

On Thu, May 30, 2013 at 3:08 PM, Nikita Nemkin <nikita at nemkin.ru> wrote:
>
>
> In pkg-db.pxi, this line:
>
>         return Pkg(<object>pkg)
>
> and in pkg-pkg.pxi this line:
>
>     def __cinit__(self, pkg):
>         self._pkg = <c_pkg.pkg *>pkg
>
> are the problem.
>
>
Thanks, Nikita! That was the problem indeed :)

The funny part is that I was looking for a way to actually pass a C pointer
to __cinit__ and doing it that way the compiler didn't complain at all. In
fact it used to work, until I've removed Python debug symbols and then I
started seeing surprises :)


> You are trying to pass a C pointer to __cinit__. What actually
> happens is that, because of the <object> cast, Cython treats pkg as
> as Python object _itself_ and tries to do some object operations on it.
>
> Two ways to solve this (really common) problem:
>
> 1) Pass your pointer as a real python object containing the integer value
>    of the pointer:
>
>     def __cinit__(self, pkg):
>         self._pkg = <c_pkg.pkg *><Py_ssize_t>pkg  # or Py_intptr_t
>
>     return Pkg(<object><Py_ssize_t>pkg)
>
>     Note the <Py_ssize_t> cast. When used on a C pointer, it makes a C
>     integer that Cython convert to python int object. And when <Py_ssize_t>
>     is used on a python int object, Cyton extracts its integer value
>     as a C-level value, so it is castable to a C-level pointer.
>
>     Using Py_ssize_t (or Py_intptr_t or your platform intptr_t) guarantees
>     that the pointer will fit.
>
> 2) Don't use __cinit__. Declare your own cdef init method and call that.
>    Since it's cdef method, it can take any C-level arguments, pointerw,
>    whatever without the need to convert to python objects and back.
>
>    @cython.final  # small optimization
>    cdef _init(self, c_pkg.pkg * pkg):
>         self._pkg = pkg
>
>    # cdef __cinit__ not needed. Maybe use it to set default
>    # attribute values.
>
>    cdef Pkg pkg = Pkg.__new__(Pkg)  # fast object creation. Pkg() also
> works.
>    pkg._init(pkg)
>
>    (When you use the __new__ trick, __cnit__ is called without
>    arguments and __init__ is not called at all. The benefit to
>    this ugly syntax is that __new__ is faster than creating the object
>    with traditional Pyhton syntax.)
>
> Also, if all your initialization is pointer assigment, you can
> just do it directly:
>
>    cdef Pkg pkg = Pkg.__new__(Pkg)  # fast object creation
>    pkg._pkg = pkg
>
>
>
I've chosen to stick with 2) -- I guess I like it better and having an
additional _init() is fine with me, makes things clearer. Thanks again!


> There is a problem related to cython-dev. I think the code you
> wrote should have never compiled in the first place. <PyObject*>
> cast must be required if you want to reinterpret a C pointer
> as object or vice versa.
>
>
Actually the code compiles just fine and also works as expected :) The only
condition is to have Python with debug symbols installed.

Any idea why having Python with debug symbols actually overcomes this issue?

Thanks again Nikita!

Best regards,
Marin


>
> Best regards,
> Nikita Nemkin
> ______________________________**_________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/**mailman/listinfo/cython-devel<http://mail.python.org/mailman/listinfo/cython-devel>
>



-- 
Marin Atanasov Nikolov

dnaeon AT gmail DOT com
http://www.unix-heaven.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20130530/3feec5b4/attachment-0001.html>