[py-dev] Backtracking on doctest

[holger krekel]

On Fri, May 13, 2005 at 14:00 -0500, Ian Bicking wrote:
> Well, now I'm starting to backtrack.  After I extended my test while 
> adding new features, I looked back over my doctests and realized they 
> were pretty pointless.  For brevity, most of my code no longer produced 
> output at all, but instead was doing internal inserts.  If I didn't have 
> a wrapper around my application for testing this might not have been 
> feasible, but without a wrapper the testing wouldn't be feasible.

Interesting.  With PyPy we noticed that we have lots of problems
with some doctest/output-comparison related CPython regression 
tests.  Doctests and output-comparisons are pretty sensitive 
to the exact way things are represented as strings.  Nevertheless
i still like the general doctest idea. 

> Anyway, I converted it to a normal py.test test.  It seems much more 
> reasonable.  I'm a little sad that it's not sitting right next to the 
> servlet that it tests -- I liked that the tests were in a docstring 
> (though they were also becoming rather long-winded, so maybe that's not 
> ultimately practical either).  

Did you consider putting tests in the same source file as the code
being tested?  It's probably often not too nice but i am using it
for one-file scripts and simply invoke 'py.test myscript.py'. 

> setup_module turned out to be quite useful; I added this to my conftest, 
> and do a "from conftest import setup_module" in my module:
> 
> def setup_module(module):
>     app = TestApp(server.make_app(CONFIG.current_conf()),
>                   CONFIG.current_conf())
>     module.app = app
>     module.CONFIG = CONFIG
>     fixture.reset_state()

I guess we should start a "recipe" section in the
documentation for nice ideas like this.    

Btw, i once thought about making your above
conftest.setup_module(module) get invoked automagically just
like Collectors/Items are looked up.  An advantage could be
that you could stack setup_module initializations but i am not
sure how useful that is (it might be for larger projects).
It's easy to add this feature should we want it. 

> I think it's likely that other boilerplate stuff will go in there in the 
> future.  I'd still like doctests to work nicely, but it becomes less 
> important to me.
>
> Anyway, no requests here, just thought I'd share my current and always 
> changing thoughts on the matter.

Thanks a lot.  I admit that your "ravioli" term used earlier deeply 
hurt my feelings until i realized that you didn't say Spaghetti :-) 

cheers, 
    
    holger