New versions breaking extensions, etc.

[Nick Coghlan]

Jive wrote:
> Can someone explain to me why Python 2.4 on MS Windows has these backward
> compatibility problems?  What am I missing?

The problem is the Python C/API. At the moment, it exposes things directly (like 
data structures) that may change size between major version releases. The other 
issue is that the interpreter and the extensions may be linked to different 
versions of the Microsoft runtime.

This is a solvable problem, but it would require:
   1. Major additions to the C/API - a "Python Major Version Agnostic" API that 
hides all data structures behind opaque pointers, never passes file descriptors 
between the interpreter and the extension and never gets Python to delete memory 
allocated by the extension (or vice-versa)
   2. Rewriting extensions to use the new API functions instead of the current 
major version specific ones.

Such an API is likely to be both harder to work with and slower, simply due to 
the limitations of C.

To this date, no-one has cared enough about the problem to put in the effort 
required to make the C API version agnostic. Given that the API almost always 
remains *source* compatible, within a month or two of a new Python release, the 
extension developers have generally tweaked their build environments to also 
produce binaries for the new release.

In the current case (Python 2.4), the transition from msvcrt to msvcrt71, and 
the poor quality of the free development tools offered by Microsoft has 
exacerbated the problem on Windows. The Python developers are looking at 
improving the documentation and support for building extensions with Windows 
compilers other than Visual Studio .Net 2003 (mainly MinGW and the free MS tools).

Although key extension developers could always try asking the PSF to buy them a 
Visual Studio license ;)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at email.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.skystorm.net