[Python-Dev] doctest, exec and __module__

[Martijn Faassen]

Hi there,

I've just witnessed an interesting consequence of the way doctest works.

I ran into an issue when doctesting an aspect of SQLAlchemy, where the 
following guard clause tripped me up:

     # In the normal call flow, a request for any of the 3 basic collection
     # types is transformed into one of our trivial subclasses
     # (e.g. InstrumentedList).  Catch anything else that sneaks in here...
     if cls.__module__ == '__builtin__':
         raise sa_exc.ArgumentError(
             "Can not instrument a built-in type. Use a "
             "subclass, even a trivial one.")

My class, coming in as cls here, was defined in a doctest, like this:

   >>> class Foo(object):
   ...    pass

It turns out that doctest compiles and executes this bit of code using a 
line like this:

          # Don't blink!  This is where the user's code gets run.
         exec compile(example.source, filename, "single",
                      compileflags, 1) in test.globs

This places new key/value pairs into a dictionary, in this case 
test.globs. Unfortunately when the execution results in a class 
definition, it'll have its __module__ attribute set to '__builtin__'. 
Try as I might, I couldn't convince exec to do it any differently.

I can't think of an nice way to work around this problem either. The 
ugly workaround in the doctest itself works:

   >>> Foo.__module__ = 'whatever'

That isn't very nice though. You could also iterate through all the 
values in the dictionary after each exec, and then check whether it's a 
class, and if so, set its __module__ to something else than __builtin__, 
but that doesn't feel very pretty (or efficient) either.

Any ideas? Am I missing something? Is there really no way to control 
this behavior with exec?

I'd like to somehow fix doctest.py so it doesn't set the __module__ to 
'__builtin__' for everything. '__main__' would be nicer, as that's what 
the interpreter shell does, and a doctest example already looks like the 
interpreter shell. While the above SQLAlchemy code is hardly pretty, I 
can't think of any better way to put in a safeguard like that either.

Regards,

Martijn