[Python-Dev] splitext('.cshrc')

[Andrew Bennetts]

Hi Martin,

"Martin v. Löwis" wrote:
> glyph at divmod.com schrieb:
[...]
> > The use-cases being discussed here would be better served by having new 
> > APIs that do particular things and don't change existing semantics, 
> > though.  For example, a "guess_mime_type(path)" function which could 
> > examine a path and figure out its mime type based on its extension 
> > (among other things).
> 
> I disagree. The current behavior is incorrect (and always has been);
> the new behavior is clearly correct.

Glyph's proposing that rather than risk breaking existing code (and in the worst
possible way: silently, giving wrong answers rather than exceptions), we examine
what benefits changing splitext would bring, and see if there's a way to get
those benefits without that risk.  I don't think that's an idea to dismiss out
of hand.

You silently omitted this part of glyph's objection:

glyph at divmod.com wrote:
[...]
> 
> Incompatible changes may be *acceptable* for feature releases, but that doesn't
> mean they are desirable.  The cost of incompatibility should be considered for
> every change.  This cost is particularly bad when the incompatibility is of the
> "silent breakage" variety - the change being discussed here would not be the
> sort of thing that one could, say, produce a warning about or gently deprecate;
> anything relying on the old behavior would suddenly be incorrect, and any
> software wishing to straddle the 2.5<->2.6 boundary would be better off just
> implementing their own splitext() than relying on the stdlib.

At the moment you seem to be saying that because you're possibly fixing some
currently buggy code, it's ok to possibly break some currently working code.
I'm not sure that's a good idea, and I expect it will create headaches for
someone trying to be compatible with both 2.5 and 2.6.

If the existing behaviour was clearly buggy by the existing documentation,
rather than undefined, this would be less of a concern.

The point about silent failures is an important one too.  This will cause some
code that was working correctly to give wrong answers, rather than an exception
or any indication that this previously correct use is now incorrect and causing
bugs.

If "splitext" was removed and replaced with, say, a "split_ext" that behaves as
you (and basically everyone, myself included) agrees it should behave, then code
won't silently do the wrong thing.  It will immediately and visibly fail, in an
understandable way.  Similarly, if split_ext was added side-by-side with
splitext, and splitext changed to emit DeprecationWarnings for a release,
there's minimal risk that existing code that works would break.

See also the comment at the bottom of
http://sourcefrog.net/weblog/software/aesthetics/interface-levels.html about
"perverse interface design".

I wonder if "First, do no harm" should be an explicit guideline when it comes
evaluating incompatible changes, even if they fix other things?

There seems to be the problem that all existing users of splitext that may
encounter '.foo' will need to audit their code to see if they are relying on
previously undefined behaviour that will change.  I think it would be much more
helpful for python to somehow help users discover this fact, than simply define
the previously undefined behaviour to be different to what they might have been
relying on.

All that said, I don't think I've ever used splitext myself, so I don't care so
much about what happens to it.  But I do worry about the general approach to
backwards compatibility.

-Andrew.