[Python-Dev] Preventing 1.5 extensions crashing under 1.6/2.0 Python

[Gordon McMillan]

Barry Scott wrote:

> I've been looking at the code to see how hard it is to
> implement my two function solution to the mismatched
> extensions crash.
> 
> I can do the work to generate a patch if the following
> analysis and design is acceptable.
> 
> Requirements as I see it are:
> 
> 1. Python extensions must match the python core to work correctly

Not necessarily.

> 2. Python must only load an extension that is compatible with its
> API 
> 3. Extensions written against python15 do not have to work
>    against python16 etc.
> 4. No Python Core resources available for implementing backward
>    compatibility for API changes.
> 5. Extension writers prefer new version of Python to refuse to
> load
>    old extensions rather the risk subtle bugs through
>    compatibility code in core.
> 
> I'll use XXX to stand in for the extension name.
> 
> The changes require that pointers to two functions are
> returned rather the one to initXXX.

You may be making things more complex than they need to 
be. On (most?) *nixen, a dynamic load is basically a runtime 
link. If the link fails (a symbol has disappeared), the module 
won't load. IOW, I think it's really only when an API call has 
changed semantics that you need to worry.

[Some unixen still don't support dynamic loading; others, eg 
AIX, do it sufficiently differently to cause havoc with portability; 
but I think it's fair to say it's a "runtime link".]

Further, it's now common practice to have xxx.so be a link to 
xxx.major.minor.shoesize.so, so diagnosis is relatively easy.

In other words, aren't we *only* worried about Windows?

If not, then:

> int PythonExtensionApiVersionXXX()
>  {
>  return PYTHON_API_VERSION;
>  }

is rather restrictive, since you can't write a module at 2.1 time 
that declares itself compatible with 2.0. Better to pass in the 
running version, and return yes / no.

Now on Windows, since a good chunk of the "link" was done 
at compile time, the only possible answer is 
  return arg == PYTHON_API_VERSION;

But if we're only worried about Windows, I think hacking the 
Windows load code to turn
 import kjbuckets
into a LoadLibraryEx of kjbuckets+2digitversion+extension 
does the same thing with less pain.
 
OTOH, I'm willing to live with a procedural solution:
 - the installer creates a fresh DLLs subdirectory
 - users are warned not to move extensions into the new DLLs 
directory, but to get new ones
 - extension writers who don't use the DLLs directory can
   - be glad they coded defensively
   - be sorry they didn't.


- Gordon