[Python-Dev] Using PEP384 Stable ABI for the lzma extension module

[Nick Coghlan]

(My comments are based on the assumption Amaury started with
http://hg.python.org/sandbox/nvawda/file/09d984063fca/Modules/_lzmamodule.c)

On Tue, Oct 4, 2011 at 12:18 PM, Amaury Forgeot d'Arc
<amauryfa at gmail.com> wrote:
> - Py_LIMITED_API is incompatible with --with-pydebug, and compilation stops.
>  I skipped the check to continue.

That seems like an odd (and undesirable) restriction. If different
Python versions are going to expose the same ABI, it seems strange of
debug and release versions can't do the same.

> - I replaced PyBytes_GET_SIZE() with Py_SIZE(), which is OK,
> and PyBytes_AS_STRING() with PyBytes_AsString(), which may
> have a slight performance impact.

Yes, the price of using the stable ABI is that performance tricks that
depend on exact memory layouts are no longer available.

> - I replaced
>      Py_TYPE(self)->tp_free((PyObject *)self);
>  with PyObject_Del(self), I hope this is the same thing
>  (for a non-GC object)

That looks right in this particular case, but problematic in general.

The stable ABI probably needs a better solution for tp_new slots
invoking tp_alloc and tp_dealloc slots invoking tp_free. In fact, a
systematic review of the slot documentation is probably needed,
pointing out the stable ABI alternatives to all of the recommended
"cross slot" invocations (and creating them if they don't already
exist).

> - _PyBytes_Resize() is missing; I moved it under a Py_LIMITED_API
>  section.

No, that's not valid. Bytes are officially immutable - mutating them
when the reference count is only 1 is a private for a reason. The
correct way to do this without relying on that implementation detail
is to use a byte array instead.

> - For the "y*" argument spec, the Py_buffer structure is required
>  (only for two fields: buf and len), as well as PyBuffer_Release()

Yeah, PEP 3118 support will eventually appear in the stable ABI, but
we need to fix it first (see issue 10181).

> - PyType_FromSpec() does not call PyType_Ready(), which caused
>  crashes in __new__.

That sounds like it may just be a bug.

Although looking at the C API docs, PEP 384 documentation appears to
be basically non-existent...

> Now the module seems to work correctly and passes tests... at least on
> Linux in a standard environment.  I will do other tests on Windows.
>
> What do you think about using the stable ABI even in shipped extensions?

It's probably not a bad idea, otherwise we may compilation without
realising it. This is especially so for extension modules that *don't*
need access to any of the interpreter internals.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia