[Python-Dev] [Python-checkins] PyCapsule backport

[Antoine Pitrou]

Hello Larry,

> You're right, my changes aren't backwards compatible.  I thought it was 
> reasonable for four reasons:
> 
> 1. The CObject API isn't safe.  It's easy to crash Python 2.6 in just a 
> few lines by mixing and matching CObjects.  Switching Python to capsules 
> prevents a class of exploits.  I've included a script at the bottom of 
> this message that demonstrates three such crashes.  The script runs in 
> Python 2 and 3, but 3.1 doesn't crash because it's using capsules.

Ok, but this is IMHO not a good enough reason to break the API. No prior
announcement (prior to 2.7) has been made that PyCObject APIs would be replaced
by their PyCapsule equivalents. You must leave people time to adapt.

> 2. As I just mentioned, Python 3.1 already uses capsules everywhere 
> instead of CObjects.  Since part of the purpose of Python 2.7 is to 
> prepare developers for the to upgrade to 3.1, getting them to switch to 
> capsules now is just one more way they are prepared.

Not /forcing them/ to switch, though. If you agree to put a bit more work into
it, and make the PyCObject APIs coexist with the corresponding PyCapsule APIs in
2.7, then I think it would be fine.

The point where we force people to switch their APIs is the 2.x -> 3.x
transition. That's the whole point of breaking compatibility at one defined
point. If we force them to switch when migrating to 2.7 instead, it becomes
messy and quite unfriendly to them.

> 3. Because CObject is unsafe, I want to deprecate it in 2.7, and if we 
> ever made a 2.8 I want to remove it completely.

I don't think there is a point in removing it in 2.8. If 2.8 ever exists, its
purpose will be to serve users who don't /want/ to switch to 3.x yet, so
breaking compatibility for them kind of ruins the purpose.

> First, we could do as Antoine Pitrou suggests on the bug (issue 7992): 
> wherever the CObject used to be published as a module attribute to 
> expose an API, we could provide both a CObject and a capsule; internally 
> Python would only use the capsules. [...]
> 
> Second, we could make CObject internally support unpacking capsules.  If 
> you gave a capsule to PyCObject_AsVoidPtr() it would unpack it and 
> return the pointer within. [...]
> 
> Third, I've been pondering writing a set of preprocessor macros, shipped 
> in their own header file distributed independently of Python and 
> released to the public domain, that would make it easy to use either 
> CObjects or capsules depending on what version of Python you were 
> compiling against.  Obviously, using these macros would require a source 
> code change in the third-party library.  But these macros would make it 
> a five-minute change.  This could compliment the first or second approaches.
> 
> Fourth, we could back out of the changes to published APIs and convert 
> them back to CObjects.  -1.

I think solution #1 would be the best one.
#2 looks too complicated.
#3 is not much friendlier than the current compatibility breakage.
#4 is the necessary fallback if you don't want to implement #1.

Regards

Antoine.