[Python-Dev] Use C extensions compiled in release mode on a Python compiled in debug mode

[Victor Stinner]

You have my support is you work on removing static types :-)

Here are my notes on the current C APIs to define a type:
https://pythoncapi.readthedocs.io/type_object.html

IMHO static types should go away in the long term. They are causing
too many practical issues.

Victor


Le mar. 30 avr. 2019 à 02:01, Neil Schemenauer
<nas-python at arctrix.com> a écrit :
>
> On 2019-04-27, Nathaniel Smith wrote:
> > For Py_TRACE_REFS specifically, IIUC the only goal is to be able to produce
> > a list of all live objects on demand. If that's the goal, then static type
> > objects aren't a huge deal. You can't add extra data into the type objects
> > themselves, but since there's a fixed set of them and they're immortal, you
> > can just build a static list of all of them in PyType_Ready.
>
> As far as I understand, we have a similar problem already for
> gc.get_objects() because those static type objects don't have a
> PyGC_Head.  My 2-cent proposal for fixing things in the long term
> would be to introduce a function like PyType_Ready that returns a
> pointer to the new type.  The argument to it would be what is the
> current static type structure.  The function would copy things from
> the static type structure into a newly allocated type structure.
>
> We have a kind of solution already with PyType_FromSpec, etc.
> However, I think it is harder to convert existing extension module
> source code to use that API.  We want to make it very easy for
> people to fix source code.
>
> If we can remove static types, that would allow us to kill off
> Py_TYPE(o)->tp_is_gc(o).  I understand why that exists but I think
> it is quite an ugly detail of the current GC implementation.  I
> wonder about the performance impact of it given current memory
> latencies.  When we do a full GC run, we call PyObject_IS_GC() on
> many objects.  I fear having to lookup and call tp_is_gc could be
> quite expensive.
>
> I've been playing with the idea of using memory bitmaps rather then
> the PyGC_Head.  That idea seems to depend on removing static type
> objects.  Initially I was thinking of it as reducing the memory
> overhead for GC types.  Now I think the memory overhead doesn't
> matter too much but perhaps the bitmaps would be much faster due to
> memory latency.  There is an interesting Youtube video that compares
> vector traversals vs linked list traversals in C++.  Linked lists on
> modern machines are really terrible.
>
> Regards,
>
>   Neil



-- 
Night gathers, and now my watch begins. It shall not end until my death.