A problem with some OO code.

[Steven D'Aprano]

On Sat, 04 Feb 2006 04:21:50 -0800, TPJ wrote:

> Help me please, because I really don't get it. I think it's some stupid
> mistake I make, but I just can't find it. I have been thinking about it
> for three days so far and I still haven't found any solution.
> 
> My code can be downloaded from here:
> http://www.tprimke.net/konto/PyObject-problem.tar.bz2. There are some
> scripts for GNU/Linux system (bash to be precise).

Hint: when asking strangers to help you, at THEIR time and expense, with
no offer to recompense them for their efforts, it is up to YOU to make
their job as easy as possible. 

That means don't expect people to download some untrusted, potentially
dangerous, piece of code from a stranger on Usenet, go to the trouble of
extracting it from an archive, and then debug it for you.

You need to simplify the code as much as possible, cutting away everything
you can, to the SMALLEST amount of code that still experiences the problem.

I've made some comments based on the information in your post. If they
don't help, you will need to spend some time creating a short example that
has the same problem. There is very little point in showing us test code
that works -- your problem isn't with code that works, it is with code
that doesn't work.


> All you need to know is that there are four classes.

No, I'm sure we need to know much more than that.


> (Of course, you
> may generate all the documentation by typing just "./make_docs". All
> you need is Python with Epydoc installed.)

Oh right, well just let me rush off and install Epydoc just for you, okay?



> In the PyObject.py there are two classes: Object and OManager. When
> Object wants to know if some link is safe (in another words: it wants
> to know if some another object exists), 

What is a link in your application? What do you mean "another object"?
Like an int or a string?

This is the normal Python idiom for testing whether an object exists
(strictly speaking, whether a name is bound to an object).

try:
    some_object
except NameError:
    print "some_object does not exist"

Is that what you mean?



> it calls the proper method of
> its objects' manager (the OManager instance). Details are not important
> here, 

I bet the details are absolutely vital.


> so I'll just say, that if the link is safe, the objects' manager
> calls the proper method of the object, that sent him request. This
> method of the object just stores the information about that link (that
> it's safe), in the so-called private field (the fields used to store
> the information about the links are called __ls_existing_links and
> __ls_demanded_links).

This is a terribly involved way of doing it, and I'm not even sure what
"it" is that you are trying to do. Are you trying to do some sort of
dispatch mechanism?

I see you have a line "i = self.__ls_demanded_links.index( s_object )". It
looks like you are storing your data in a chunk of text, and then running
index on that text to find the bit you want to work with. If you wanted to
do that much work, just write your script in bash and be done with it!
Python has lists and dicts and other advanced data structures. Use them.
You will be glad.



> And now the tests. In the test.py file this is a class called Obj. That
> class is just an Object. 

So Obj is an alias for Object? What's an Object? Is it the same as the
Python built-in type "object"?

But later on you say:

"All these methods are the methods of the Object class (defined in
PyObject.py), not of the Obj class (defined in test.py)."

So Object and Obj are not the same.



> This is also a class called System1 - this
> class is used only to create a more complex system composed of many
> objects and one objects' manager.
> 
> You can run all the tests by just typing "./test" in console (or a
> terminal emulator). I'm still getting the same error:
> 
> --------------------------------------------------
> Traceback (most recent call last):
>   File "test.py", line 142, in test04CreateSystems
>     system1 = System1()
>   File "test.py", line 81, in __init__
>     self.objA.demandNewLink( 'B' )
>   File "/home/tpj/opt/Praca/Programy/PyObject/workdir/PyObject.py",
> line 503, in demandNewLink
>     s_object )
>   File "/home/tpj/opt/Praca/Programy/PyObject/workdir/PyObject.py",
> line 800, in _makeNewLink
>     o_srcobj._makeLinkSafe( s_target )
>   File "/home/tpj/opt/Praca/Programy/PyObject/workdir/PyObject.py",
> line 608, in _makeLinkSafe
>     self.__makeLinkSafe( s_object )
>   File "/home/tpj/opt/Praca/Programy/PyObject/workdir/PyObject.py",
> line 582, in __makeLinkSafe
>     i = self.__ls_demanded_links.index( s_object )
> AttributeError: 'Obj' object has no attribute
> '_Object__ls_demanded_links'

[snip]

> Perhaps I'll explain what's going on there. First, the objects' manager
> is created, and then also some objects are created. After doing that,
> the object called "objA" demands a new link to the other object, called
> "objB" (this is the line 81 in the error message above). 

Your description does not match what line 81 actually says. It says:

self.objA.demandNewLink('B')

self (an instance of System1, not Obj or Object) grabs the attribute
called "objA", and calls its method demandNewLink with an argument of
the string 'B', not objB an instance of Obj or Object. 

(Aside: generic names like "Obj" or "Object" are poor practice unless they
genuinely are generic. A good name should be self-documenting if possible,
but more importantly, names which are easy to confuse like Obj and Object
should be avoided.)

If you want to work with Object instances, why not just use them instead
of passing around strings that represent them?



> As the result,
> the "demandNewLink" method of the object "objA" is called. This method
> calls the "_makeNewLink" method of the objects' manager, and this
> method calls the "_makeLinkSafe" method of the object, that demanded a
> new link. The last method, called "__makeNewLink" (so this is a
> so-called private method), is called then. 

This seems like a very brittle way to code.

> This last method is supposed
> to check if the link was really demanded and then make it "safe" by
> placing the information about this link in the proper dictionary
> (__ls_existing_links).

Huh? Do you mean that there is a way for links (whatever they are) to be
requested without calling demandNewLink? How would this happen?


> All these methods are the methods of the Object class (defined in
> PyObject.py), not of the Obj class (defined in test.py). According to
> my knowledge about the OO programming Python should be able to realize,
> that the so-called private fields, that are being referred from these
> Object class methods, are the private fields of the *Obj class
> instance*, not of the *Object class one*.

Some serious confusion here.

What are fields? Do you mean attributes? I'm going to assume that you do.

Would it help if I told you that Python doesn't have private attributes?
It *simulates* private attributes using name mangling.

Thus:

class Parrot:
    __plumage = "red"

actually creates a class attribute "_Parrot__plumage". If you know the
name of the class, you can bypass Python's "private attributes".

PLEASE don't bother to tell us how Python sucks because it doesn't have
"real private attributes", we've heard the arguments (and the abuse) fifty
times before.

Now, the consequences:

When Python does a lookup, it looks in the appropriate namespace for a
name. Except for special rules relating to inheritance, Python doesn't
magically look up attributes in another object's namespace just because
that attribute was defined as a private attribute elsewhere. See below for
the consequences.



> I even wrote a simple program to test the usage of the so-called
> private fields in Python classes:

Good. Now how about getting a simple program that demonstrates your
problem?


> class A( object ):
>   def __init__( self ):
>     self.__a = 1
>   def _add( self ):
>     print "self.__a =", self.__a
>     self.__a += 1
>   def print_a( self ):
>     print self.__a

Do you enjoy all this make-work, writing tedious methods to get or set or
print attributes? I suggest you ditch all the private attributes and just
make them public. You've already spent three days hitting your head
against a brick wall because of excessive data-hiding. Was it worth it?
Python is a language that encourages the attitude "We're all adults here"
-- getters/setters are discouraged.



> class B( A ):
>   def __init__( self ):
>     A.__init__( self )
>     self.__b = 2
>   def print_b( self ):
>     print self.__b
> 
> def test():
>   o = B()
>   o.print_a()
>   o.print_b()
>   o._add()
>   o.print_a()
>   o.print_b()

I don't think this is showing what you think it is showing. Private vs
public attributes just get in the way: they work *exactly* the same in
Python except for name mangling. This is the problem bit:

class B( A ):
    def __init__( self ):
      A.__init__( self )  ### LOOK HERE

You aren't creating an A instance here, you are calling A.__init__ with a
first argument of "self", which is a B instance. __init__ does not create
instances, it runs AFTER the instance is already created.

Perhaps you already know that. But I don't think you understand the
consequences of this, when using so-called private attributes:

The call to A.__init__ creates an attribute __a to self. Remember, self is
an instance of B (not A). Apply the name-mangling rules, which are applied
at compile time: the code in A's method mangles __a to _A__a. So
A.__init__ creates an attribute _A__a in B's namespace. So B has two
attributes, _A__a and _B__b.

That means that B methods that access self.__b will work, because the name
mangling rules work *with you*, and self.__b is mangled to self._B__b. But
B methods that access self.__a will NOT work, because it is mangled to
self._B__a, and that attribute does not exist.

(However, that is not your problem.)


> The code works just as I thought it should work: all is fine here.

Except it is brittle and hard to maintain.

> The
> B class instance (the b object) is able to refer to the so-called
> private field of the A class (through a A-class method, that was
> inherited by the class B).
> 
> Why I can't do the same thing in test.py and in PyObject.py?

Here is your exception again:

> line 582, in __makeLinkSafe
>     i = self.__ls_demanded_links.index( s_object )
> AttributeError: 'Obj' object has no attribute
> '_Object__ls_demanded_links'

This is not making sense to me. If self is an instance of Obj, the
name-mangling rules will convert self.__ls_demanded_links to
self._Obj__ls_demanded_links. The only thing I can guess is that somehow,
somewhere, you have tried to manually set the class of an object.

Something like this perhaps?

>>> class Obj:
...     __bar = "nothing"
...
>>> class Object(Obj):
...     __foo = "something"
...     def foo(self):
...         print self.__foo
...     def bar(self):
...         print self.__bar
...
>>> x = Object()
>>> x.__class__.__name__ = "Obj"
>>> x.foo()
something
>>> x.bar()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "<stdin>", line 6, in bar
AttributeError: Obj instance has no attribute '_Object__bar'

That's the only way I can think of to get such an unusual error.



-- 
Steven.