__del__ problem - more details provided. still stumped though.

[Warren Postma]

Okay, more details, since i've obviously oversimplified the previous attempt
at explaining the problem.

I have four modules here, all used in an embedded Python application on
Windows (using static linking, not the python15.dll, because this is a
deeply embedded system with no DLLs on the target CPU).

ztrdb.py - this is my "application specifics" module, containing a basic
"data definition" of the
            tables used in my database layer.

lwdb.py - lightweight database class (lwdb_base) and some inherited classes
that implement various
            data storage techniques, from a simple "pickle rows to a text
file" to various
            techniques that rely on the BSDDB 2.0's hash and btree file
storage.


Basically, I wanted to separate the Instances of Tables that were
application specific, from my
generic and hopefully reusable data-handling-classes.  I also wanted lwdb.py
to track ALL instances of tables created from ANY class defined in lwdb.py.
In other words, I wanted to be able to do operations on all tables (such as
closing all tables at once, or flushing changes to disk
on all tables at once).

So, in lwdb.py:

lwdb_tables = {}

class lwdb_base:
    def __init__(self,tablename,description, row_class_obj, filename)
         global lwdb_tables
         # rest of constructor snipped because it's not important here
         lwdb_tables[tablename] = self


Next, for reasons which are still not clear even to me, I built ztrdb.py,
which was my own instances. It did things like this:

table1 = lwdb.lwdb_base("tablename","a description", table1rowclassobj,
"tablename.txt")

Now because the Object reference was stored to both lwdb_tables (a strong
reference) and table1 also a strong reference) and there is no easy way to
make a weak-reference, there were now TWO reference counts to each table
object.

The problem I was having was that at shutdown I was getting this exception,
emitted somewhere from the bowels of the BSD Database layer 2.0 module
(written by Robin Dunn, and as far as I know it has been reliable, yet it is
complex enough that if it *was* having a problem, I wouldn't have the
foggiest idea how to fix it.). Here is the text of the exception, which
occurs when the module lwdb is being destroyed, and the lwdb_tables
dictionary, and the table objects it contains  are finally going out of
scope:


Exception except.AttributeError:
"'None' object has no attribute 'time'" in
<method lwdb_base.__del__ of lwdb_hash instance at 5de410>
ignored

Exception except.AttributeError:
"'None' object has no attribute 'delete_DB'" in
 <method DBPtr.__del__ of DB instance at 5dfd00>
ignored


Exception except.AttributeError:
"'None' object has no attribute 'time'" in
<method lwdb_base.__del__ of lwdb_base instance at 5de6d0>
ignored


In this case I have figured out the following:

1. this message indicates that errors during __del__ are masked, so that the
destructor can finish cleaning up.
2. there is one lwdb_base object being destroyed and two lwdb_hash objects
(a subclass of lwdb_base that uses a BSDDB hash file for storage).

I have figured out a workaround, but can't figure out why it works:

1. Don't keep the reference count of the table objects at 2, so you may only
have ONE reference to it. Here's how I did it:

# in ztrdb.py

# the lwdb.lwdb_tables dictionary now has a refcount of 2
#but the contents are still at refcount of 1
tables = lwdb.lwdb_tables

# at the beginning of shutting the system down, before anything
# else shuts down, we delete the table objects:
def call_me_at_shutdown():
    del lwdb.lwdb_tables
    tables=None


Also, in the lwdb_base.__del__ method I inserted a print statement so I
could see when __del__ got called. If All table Objects __del__ methods are
called before my C program reaches the Py_Finalize() call in my C program,
then everything is fine. If however the modules are left to clean up things
in their own preferred orders inside Py_Finalize, I get those errors above.

I found where those errors are originating from, too, but I'm still no
closer to understanding this.  The file db.py, which is included with Robin
Dunn's BSDDB 2.0 package for python, includes this definition:


class DBPtr :
    def __init__(self,this):
        self.this = this
        self.thisown = 0
    def __del__(self):
        if self.thisown == 1 :
            dbc.delete_DB(self.this)  # appears some exceptions being raised
here.
    # rest of class DBPtr snipped


That still leaves the two messages regarding unknown attribute named "time".
the only reference to time in my modules is calls to time.time() in a
function that could be called in the destructor of the table objects, while
updating the table headers one last time before closing the file. Is it
possible that the stuff added to our namespace by the "import time" function
has already been deleted before the objects in the module are deleted?
Shouldn't all instances of objects be deleted before the global namespace of
a module gets trashed? What gives?

Any help sorting these mysteries out would be greatly appreciated.

The code itself is about 10k, perhaps too large to attach here, but
certainly small enough to send to anyone. If it's possibly a fault with the
bsddb 2.0 I'd like to help find it and fix it if I can, and give Robin Dunn
a patch. Unfortunately, I'm stumped!

Warren