[Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0

[M.-A. Lemburg]

On 2008-06-02 01:30, Gregory P. Smith wrote:
> On Fri, May 30, 2008 at 1:37 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>> Sorry, I probably wasn't clear enough:
>>
>> Why can't we have both PyString *and* PyBytes exposed as C
>> APIs (ie. visible in code and in the linker) in 2.x, with one redirecting
>> to the other ?
>>
>>>> * Why should the 2.x code base turn to hacks, just because 3.x wants
>>>> to restructure itself ?
>>> With the better explanation from Greg of what the checked in approach
>>> achieves (i.e. preserving exact ABI compatibility for PyString_*, while
>>> allowing PyBytes_* to be used at the source code level), I don't see what
>>> has been done as being any more of a hack than the possibly more common
>>> "#define <oldname> <newname>" (which *would* break binary compatibility).
>>>
>>> The only things that I think would tidy it up further would be to:
>>> - include an explanation of the approach and its effects on API and ABI
>>> backward and forward compatibility within 2.x and between 2.x and 3.x in
>>> stringobject.h
>>> - expose the PyBytes_* functions to the linker in 2.6 as well as 3.0
>> Which is what I was suggesting all along; sorry if I wasn't
>> clear enough on that.
>>
>> The standard approach is that you provide #define redirects from the
>> old APIs to the new ones (which are then picked up by the compiler)
>> *and* add function wrappers to the same affect (to make linkers,
>> dynamic load APIs such ctypes and debuggers happy).
>>
>>
>> Example from pythonrun.h|c:
>> ---------------------------
>>
>> /* Use macros for a bunch of old variants */
>> #define PyRun_String(str, s, g, l) PyRun_StringFlags(str, s, g, l, NULL)
>>
>> /* Deprecated C API functions still provided for binary compatiblity */
>>
>> #undef PyRun_String
>> PyAPI_FUNC(PyObject *)
>> PyRun_String(const char *str, int s, PyObject *g, PyObject *l)
>> {
>>        return PyRun_StringFlags(str, s, g, l, NULL);
>> }
>>
> 
> Okay, how about this?  http://codereview.appspot.com/1521
> 
> Using that patch, both PyString_ and PyBytes_ APIs are available using
> function stubs similar to the above.  I opted to define the stub
> functions right next to the ones they were stubbing rather than
> putting them all at the end of the file or in another file but they
> could be moved if someone doesn't like them that way.

Thanks. I was working on a similar patch. Looks like you beat
me to it.

The only thing I'm not sure about is having the wrappers in the
same file - this is likely to cause merge conflicts when doing
direct merging and even with an automated renaming approach,
the extra code would be easier to remove if it were e.g. at
the end of the file or even better: in a separate file.

My patch worked slightly differently: it adds wrappers PyString*
that forward calls to the PyBytes* APIs and they all live in
stringobject.c. stringobject.h then also provides aliases
so that recompiled extensions pick up the new API names.

While working on my patch I ran into an issue that I haven't
been able to resolve: the wrapper functions got optimized away
by the linker and even though they appear in the libpython2.6.a,
they don't end up in the python binary itself.

As a result, importing Python 2.5 in the resulting 2.6
binary still fails with a unresolved PyString symbol.

Please check whether that's the case for your patch as well.

>> I still believe that we should *not* make "easy of merging" the
>> primary motivation for backporting changes in 3.x to 2.x. Software
>> design should not be guided by restrictions in the tool chain,
>> if not absolutely necessary.
>>
>> The main argument for a backport needs to be general usefulness
>> to the 2.x users, IMHO... just like any other feature that
>> makes it into 2.x.
>>
>> If merging is difficult then this needs to be addressed, but
>> there are more options to that than always going back to the
>> original 2.x trunk code. I've given a few suggestions on how
>> this could be approached in other emails on this thread.
> 
> I am not the one doing the merging or working on merge tools so I'll
> leave this up to those that are.

I'm not sure whether there are any specific merge tools around -
apart from the 2to3.py script.

There also doesn't seem to be any documentation on the merge
process itself (at least nothing that Google can find in the
PEPs), so it's difficult to make any suggestions.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 02 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-07-07: EuroPython 2008, Vilnius, Lithuania            34 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611