Does Python really follow its philosophy of "Readability counts"?

[Steven D'Aprano]

On Fri, 23 Jan 2009 21:36:59 -0500, Luis Zarrabeitia wrote:

> Quoting Steven D'Aprano <steve at REMOVE-THIS-cybersource.com.au>:
> 
>> On Fri, 23 Jan 2009 13:07:55 -0500, Luis Zarrabeitia wrote:
>> 
>> > It should be in _our_ power as the team of all participant coders on
>> > _our_ project to decide if we should mess with the internals or not.
>> > 
>> > What makes no sense is that it should be in the original author's
>> > power to decide, if he is not part of _our_ team.
>> 
>> Makes *no* sense? There's *no* good reason *at all* for the original
>> author to hide or protect internals?
> 
> My bad, sorry.
> It makes sense... if the original author is an egotist who believes he
> must control how I use that library.

Then I guess Guido must be such an egotist, because there's plenty of 
internals in Python that you can't (easy) mess with.


> Or, if external forces make him do
> it (maybe like, 'oh, if I change python, then I'm not using python
> anymore').

That parenthesised comment makes no sense to me. Python has changed 
significantly since it was first released. Recently, print became a 
function instead of a statement, and one of the motivations for this was 
to allow people to change the behaviour of Python's print simply by 
defining a new function. "Shadowing built-ins", as they call it, is a 
feature, not a bug. I can't see any good reason for thinking that if you 
change (say) the way Python prints, you don't have Python any more.

Even more fundamental changes have occurred, e.g. new style classes, 
ABCs, nested scopes.


[...]
> If a variable is marked as... I don't like 'private', I'll call it
> 'implementation detail', I would not use it without good reason. Not
> even by subclassing it. Why do you assume that I'd change list._length
> if I could? I wouldn't.

I didn't say you would change it on a whim. I said that *if* it were 
exposed to Python code, you *could* change it. You might change it 
because you thought you had a good reason to. You might change it by 
accident. You might not realise the consequences of changing it. Who 
knows? It doesn't matter what your motives are.

My point is that you claimed that there is no good reason at all for 
hiding implementation details. Python is full of implementation details 
which are quite effectively hidden from Python programmers. So there are 
two possibilities:

(1) you are right that it "makes no sense" (your words) for the original 
author (in this case, Guido) to hide those implementation details from 
Python programmers; or

(2) you are wrong that it "makes no sense", because there is at least one 
case where the original author (Guido again) did a sensible thing by 
hiding implementation details.

In an effort to avoid going round and round in circles, let me explicitly 
say that option (2) does not imply that it always makes sense to hide 
implementation details.



> Anyway, did you notice that your "counter-example" was a radical
> change-the-way-python-works scenario? 

No, my scenario is merely extending what you can already do with pure-
Python classes to built-in classes written in C. It would have a radical 
effect (pure Python code could core dump easily) but it wouldn't be a 
radical change. It might take as little as one new function.


[...]
>> So what you're saying is that the fundamental design of Python -- to be
>> a high-level  language that manages memory for you while avoiding
>> common programming errors such as buffer overflows -- makes "no sense".
>> Is that what you intended?
> 
> Yes, that's what I intended, obviously. I'd like to have buffer
> overflows in python. In case you don't understand irony: don't go
> putting words in my mouth. I'm not putting words in yours.

And neither am I. I'm pointing out the logical implications of your 
position. If you find those implications unpleasant, then perhaps you 
should modify your position to be less extreme and more realistic.


>> As I see it, you have two coherent positions. On the one hand, you
>> could be like Mark Wooding, and say that Yes you want to risk buffer
>> overflows by messing with the internals -- in which case I'm not sure
>> what you see in Python, which protects so many internals from you. Or
>> you can say that you made a mistake, that there are *some* good reasons
>> to protect/hide internals from external access.
> 
> Or, I could have a third option: assume that I am a grownup who knows
> what he is doing.

This is totally orthogonal to what we're discussing. Whether you are a 
grownup or a child, whether you have good reasons or bad reasons, you can 
still make either of the two choices.


> After all, even with all those "protections" in list,
> I could just create an extension module to shoot me in the foot anyway,
> if I really wanted to.

Yes you could, and you could hack the OS to manipulate data behind the 
scenes, and you could build a hardware device to inject whatever data you 
want directly into the memory. You can do any of those things. So what?

Data hiding isn't about some sort of mythical 100% certainty against any 
imaginable failure mode. Data hiding is a tool like any other, and like 
all tools, it has uses and misuses, and it works under some circumstances 
and not others. Wrenches are excellent for tightening bolts even though 
they don't work in weightlessness (the astronaut spins around instead), 
and hammers are good for hammering nails even though they won't work on 
the surface of Pluto (the metal will become brittle and shatter). Data 
hiding is no different. 

If you don't get 100% certainty that there will never be a failure no 
matter what, what do you get? Just off the top of my head, it:

* makes it easier for an optimising compiler to give fast code if it 
doesn't have to assume internal details can be changed;

* makes it easier to separate interface from implementation when you can 
trust that the implementation actually isn't being used;

* gives the developer more freedom to change the implementation;

* makes it possible for meaningful correctness proofs;

* reduces the amount of interconnections between different parts of your 
program by ensuring that all interaction goes through the interface 
instead of the implementation;

* which in turn reduces the amount of testing you need to do;

and possibly others.


[...]
> I don't think this is going anywhere. Now you are trying to push me to
> the extremes, changing what I _said_ for your exaggerated interpretation
> of it just so you could shoot it down, or force me to say that I want
> buffer overflows in python. I believe that's called "strawman".

No, I'm not changing anything you said. I'm pointing out the implications 
of what you said. Don't blame me for seeing what logical consequences 
following from your statement.


> I stand by my words - but not by your "interpretation" of them:
> 
>> > What makes no sense is that it should be in the original author's
>> > power to decide, if he is not part of _our_ team.
> 
> Do you _really_ read from that sentence that I should dislike python
> because it makes it a bit harder to get a buffer overflow with their
> native types?

Well, you tell me: does it make sense for Guido to have decided to make 
it hard for pure Python developers to cause buffer overflows?

If your answer is Yes, it makes sense, then obviously your earlier 
statement that it makes no sense is *wrong*, at least under some 
circumstances. Then we can make progress: data hiding isn't *always* evil 
and anti-freedom and useless, it's okay when Guido does it. Then we can 
act like grownups and discuss under what other circumstances it is or 
isn't good to use data hiding, instead of making sweeping generalisations 
that it is never good and always useless.

If your answer is No, it makes no sense, Guido was wrong to hide 
implementation details from Python developers, then I can't imagine what 
you get out of the stifling, unpleasant B&D language Python. Perhaps you 
like the syntax?



-- 
Steven