[py-dev] the seeming need of task-based build-tool properties in py.test

[Ronny Pfannschmidt]

On Wed, 2010-02-17 at 17:13 +0100, Adam wrote:
> Ronny Pfannschmidt wrote:
> 
> > ordering issues indeed get important as soon as one creates test-items
> > that depend on the order and execution of other test items
> > 
> > this is already happening for glashammers acceptance-testsuite
> 
its more or less breaking a big far acceptance test down in reportable
steps

the current solution isnt nice, but works in the current py.test,
it will have to be adapted once we have a full decission on how to
handle steps of acceptance tests (the ideas are still fuzzy)

> Personally, I consider this a broken test setup that really should be
> separated between a cached_setup and the tests. I wouldn't want to leave
> a global state around. However, I don't know hard this is for
> glashammers specific cases. And I guess it is a sort of boilerplate.
> 
one of the inherent properties of those acceptance tests is,
that a step in the middle ma fail without necessaryly breaking the rest,
however the order is important

the same would go for doctest chunks

order dependce, failure indepence

its generally a tricky area to deal with and i expect that we do it
wrong a few times before something thats really acceptable emerges as
the result of feedback

-- Ronny


> > another part of the issue is tests that take long,
> > but are not required to run every time - like for example code
> > validators
> 
> I consider this a separate issue (and would be cool to have).
> 
> --Adam