[Python-Dev] Preventing 1.5 extensions crashing under 1.6/2.0 Python

[James C. Ahlstrom]

I just got back from vacation, so I am catching up.  The following
concerns Windows only, not *.so files on Unix.

Background:  Existing foo.pyd Windows extensions will crash
the new Python because foo.pyd will load python15.dll,
and python.exe will load python20.dll.  MarkH explained why:

[Mark]
> The 1.5 module is linked against Python15.dll.  When Python.exe (linked
> against Python16.dll) starts up, it initializes the Python data structures
> in Python16.dll.  When Python then imports foo.pyd (linked against
> Python15.dll), it immediately tries to call the functions in that DLL (such
> as getting the thread state).  As Python has not been initialized in that
> DLL, we immediately die.

This is really bad.  Normally under Windows, DLL names do not change
unless the API exported is changed in a major way.  For example,
there have been several versions of mfc42.dll without changing the
name.

[Mark continues]
> Ironically, if Python1x.dll was simply named "Python.dll", there is a
> _reasonable_ chance it would work fine.  The cost of doing this tho, is
> that you can not have Python 1.5 and Python 1.6 "side by side" on the same
> machine.

What this means is that if the new Python 2.0 (1.6?) simply names
its DLL python15.dll just as before, everything Just Works almost.
That is, it Just Works provided that foo.pyd only calls Python
C functions whose interface has not changed.

You can support Python 1.5 and 1.6 by removing the python*.dll
from the Windows directories, and putting it in the directory
of python.exe so it goes with the proper executable.

Guido van Rossum wrote:

> There are endless variations on this theme.  What's missing at the
> moment is a bit of requirements analysis.  What's the problem we're
> trying to solve here?  So far, the only problem that came up was that
> on Windows, you get an obscure crash when trying to load an extension
> built for Python 1.5(.x).

Yes, and this is a major problem, and is un-Windows-like.

> Do we really need a whole new version
> negotiating architecture?

No, just keep the old name "python15.dll".  Old foo.pyd will
still work if they use only Python C functions with the same
interface.

Individual *.pyd authors will have to advise users whether
they need to upgrade to new *.pyd versions or not.

> You can already get the version string by
> looking at Py_GetVersion() -- this also answers the question of how a
> version is represented.

True.  And on Windows, there is a standard Version resource
string which gives the DLL version, and can be read without
loading the DLL.  This string is referenced by the Wise (and
other) installers to decide whether a DLL is newer and should
be replaced.

> On Windows, link-time compatibility is broken whenever we change the
> second digit of the version number, e.g. 15, 16, 20.

Yikes, lets not let that happen!

JimA