Encapsulation in Python

[Steven D'Aprano]

On Fri, 11 Mar 2016 12:41 am, Ben Mezger wrote:

> Hi all,
> 
> I've been studying Object Oriented Theory using Java. Theoretically, all
> attributes should be private, meaning no one except the methods itself
> can access the attribute;

(Note: in the following, when I say "encapsulation", I am actually referring
to *data hiding*.)

That's one theory. Another theory is: no they shouldn't, all attributes
should be public. That most accurately models actual physical objects and
maximises the usefulness of the attribute.

People over-use private, and if you google, you will find many people
asking "how can I access private variables and methods?". You will also
learn that encapsulation is the enemy of test-drive development:

https://jasonmbaker.wordpress.com/2009/01/08/enemies-of-test-driven-development-part-i-encapsulation/

and also of good user-interfaces:

https://ayende.com/blog/4375/encapsulation-is-the-enemy-of-the-user-interface

Some might say that encapsulation is a waste of time:

http://c2.com/cgi/wiki?EncapsulationIsaWasteOfTime

See also:

http://c2.com/cgi/wiki?MethodsShouldBePublic
http://c2.com/cgi/wiki?ForgetAboutWritingAccessors


Python offers a middle ground: where encapsulation is important for safety,
use it, otherwise rely on trust and a gentleman's agreement. We're all
adults here: if you use my internals, that's your choice, and you can live
with the consequences.

In practice this means that internal details of classes written in C are
completely hidden from Python code unless explicitly made public. Why?
Because if the caller messes with the internal data of a C class, they can
cause a segmentation fault, which can execute arbitrary code. This is a
serious problem, and Python has a philosophy of Absolutely No Seg Faults.
It should be impossible to cause the Python interpreter to seg fault or
dump core (except via ctypes, which is special). That means that classes
written in C are hide their internals.

But what about Python classes? You can't cause a seg fault in Python code.
So Python's philosophy is:

- trying to *enforce* private data is a waste of time, people will find a
way around it, even if it is a horrible nasty hack;

- so instead, rely on trust: we mark "private" attributes with a leading
underscore, and trust that users won't abuse them;

- if they do, the consequences are on their head, not ours: you have no
responsibility to users who misuse your private attributes.

Furthermore, are you *sure* that you need private attributes? That will, of
course, depend on the size of the project you are working on. If it is a
small class in a small program, then it is probably a waste of time and
effort: You Ain't Going To Need It.

Python gives you five levels of enforcement of encapsulation:

1. None what so ever. YAGNI, so just use a regular attribute and 
   don't over-engineer your simple class.

2. Trust the user to obey the naming convention: name the 
   attribute with a leading underscore to make it private by
   convention.

3. If you need to offer a public interface which differs from 
   your internal implementation, you can use properties or 
   custom-made descriptors to implement getters and setters 
   which look like ordinary attribute access. (This often 
   relies on #2 as well.)

4. The encapsulation offered by closures is even harder to 
   break, but it applies to functions, not classes.

5. For the strongest level of encapsulation, move your code
   into a C extension class.



For most users, 1 through 3 is more than enough.



> public class Foo {
>     private int bar;
>     ...
> 
> Normally in Java, we would write getters and setters to set/get the
> attribute bar. However, in Python, we normally create a class like so;
> 
> class Foo(object):
>     bar = 0
>     ...

No, normally we wouldn't. The above makes bar shared by all instances.
Normally, if you want each instance to get their own value of bar, you
would assign it in the constructor/initialiser:

class Foo(object):
    def __init__(self):
        self.bar = 0


For ints, this isn't much of a practical difference, but it makes a very big
difference for mutable classes like lists.


> And we usually don't write any getters/setters (though they exist in
> Python, I have not seen much projects making use of it).

Correct. YAGNI: don't use getters/setters until you have proof that you need
it, not just "well my professor tells me I should do this".

As a Java developer, you might sometimes feel that (by Java standards)
Python code is awfully slap-dash and sloppy. That's okay -- by Python
standards, Java code is awfully "bondage and discipline", strict and
pedantic.

See here:

http://dirtsimple.org/2004/12/python-is-not-java.html
http://dirtsimple.org/2004/12/java-is-not-python-either.html



> We can easily encapsulate (data hiding) Foo's class using the '_'
> (underscore) when creating a new attribute, however, this would require
> all attributes to have a underscore.

Well, if you really want them to be private, that's the way to do it. But
why are they all private? Private attributes have both a benefit and a
cost. Are you sure the benefit is worth it?

Sometimes, rather than make each attribute private, it is easier to just
make the whole class private.


> According to this answer [1], it's acceptable to to expose your
> attribute directly (Foo.bar = 0), so I wonder where the encapsulation
> happens in Python? If I can access the attribute whenever I want (with
> the except of using a underscore), what's the best way to encapsulate a
> class in Python? Why aren't most of the projects not using
> getters/setters and instead they access the variable directly?

Despite what your Java teacher may have said, encapsulation is not the most
important thing about classes. A lot will depend on the size of your
project, of course: in an extremely large project, you will see Python code
written a lot more like Java code. But for small projects, not so much.

In fact, a lot of Python code doesn't even use classes.

Stop Writing Classes: http://www.youtube.com/watch?v=o9pEzgHorH0

http://steve-yegge.blogspot.com.au/2006/03/execution-in-kingdom-of-nouns.html


On the other hand, sometimes we should write more classes:

http://lucumr.pocoo.org/2013/2/13/moar-classes/


> 
> Regards,
> 
> Ben Mezger
> 
> [1] - http://stackoverflow.com/q/4555932

-- 
Steven