[Python-Dev] Proposal: explicitly disallow function/class mismatches in accelerator modules

[Steven D'Aprano]

On Mon, Jul 11, 2016 at 01:32:32PM +1000, Chris Angelico wrote:
> On Mon, Jul 11, 2016 at 1:26 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> > (1) How much extra effort are we going to *mandate* that core devs put
> > in to hide the differences between C and Python code, for the benefit of
> > a small minority that will notice them?
> >
> 
> The subject line is raising one specific difference: the use of a
> function in one version and a class in the other. I think it's not
> unreasonable to stipulate one specific incompatibility that mustn't be
> permitted.

Not so fast. Have you read the bug tracker issue that started this?

There are certain things, such as functools.partial() objects, which are 
most naturally implemented as a closure (i.e. a function) in Python, and 
as a class in C. Whether partial() happens to be a class or a function 
is an implementation detail. There are advantages to being able to write 
the simplest code that works, and prohibiting that mismatch means that 
you cannot do that.

If a core developer wishes to extend the API of partial objects to 
include such things as subclassing, isinstance tests, pickling etc, then 
it is reasonable to insist that both the C and the Python implementation 
both support the same API and both use a class. But at the moment I 
don't think any of those things are promised or supported[1], except by 
accident, so removing the discrepency between them is not a bug fix, but 
adding new features.

The more I think about it, the more I feel that it is *unreasonable* to 
mandate that devs must ensure that alternate implementations mirror each 
others *unsupported and accidental features*. Mirror the supported API? 
Absolutely! But not accidental features.

I think its worth reading the issue on the tracker:

http://bugs.python.org/issue27137

This isn't an actual problem that occurred in real code, it's a 
theoretical issue that Emanuel discovered, and by his own admission 
feels that he was doing something dubious ("It may not be the best idea 
to subclass something that is meant to be final" -- ya think?). Raymond 
Hettinger makes some good points about the costs of feature creep needed 
to support these accidental implementation features, and is against it.

But on the other hand, Serhiy also makes some good points about the 
usefulness of pickling partial objects. So as far as this *specific* 
issue goes, perhaps it is justified to make sure the Python 
implementation supports pickling.

(Aside: why can't closures be pickled?)

But generalising this to all possibly mismatches between a C class 
implementation and a Python function implementation doesn't necessarily 
follow. Raymond's general point about simplicity versus feature creep 
still stands, even if in this case adding pickling is useful.



[1] If I'm wrong about this, and these features are supported, then 
Emanuel has found a hole in the functools test suite and a weakness in 
our testing: it's too hard to ensure that *both* the Python and C code 
is tested.


-- 
Steve