[Distutils] test command for setup.py

[Berthold Höllmann]

>>>>> "Greg" == Greg Ward <gward@python.net> writes:

    Greg> On 21 September 2000, Berthold Höllmann said:
    >> I think it is easier for the developer to have code and tests
    >> together.

    Greg> Not if the tests are really exhaustive! When I write good
    Greg> tests (or at least what I think are good tests), the test
    Greg> script is roughly the size of the module being tested. I
    Greg> don't want my "if __name__ == '__main__'" block to be as big
    Greg> as the rest of the module. I think the "test" command should
    Greg> support "in-situ" testing, but it's *not* enough for serious
    Greg> tests.

    >> One can leave out the tests from any distribution by using a
    >> Manifest file.

    Greg> Huh? That loses half the point of writing tests, which is to
    Greg> assure people building and installing your modules that they
    Greg> probably work as advertised on their platform.

My Idea here was, to distinguish between test to enshure correct
compilation on the "customers" computer and tests to help the
developer my the Manifest file, but to keep all tests with your code
in one (CVS?)repository.

    >> But for the developer it is helpful to have a set of tests that
    >> one can run after changes have been made to the sources to
    >> enshure that "little" changes have not unexpected side effects.

    Greg> Yes! That's the other half of the reason for writing tests.

    Greg> (You can define "half" here as 40-60, 20-80, whatever.)

    >> ...

    Greg> Oooh, yes, in a previous life I struggled (briefly) with
    Greg> automated testing of numerical algorithms. Not fun. But
    Greg> asking end-users to examine the test output and make sure
    Greg> that the difference between 1.0234324 and 1.0234821 is less
    Greg> than the machine epsilon is surely even less fun.

It begins with the difference between 1.00000E+00 and 1.0000E+000
(where is it?).

    >> Also the underlaying Fortran code makes different decisions on
    >> different platforms. That makes automatic testing nearly
    >> impossible, but I want at least a constistant way to generate
    >> the needed output, and I want to test everything before I
    >> install it.

    Greg> Here's my extreme position:

    Greg>   If a test isn't fully automated -- i.e., if it takes more
    Greg> than epsilon human intelligence to determine if the test
    Greg> passed or failed -- then it's not a proper test, it's just a
    Greg> demo.

Accepted for "functionality" tests, to be executed by the "customer",
but I don't want to distinguish these from heavier tests, to be
executed by the developer.

    Greg> ...

    >> All this can be done with my posted, extremly lightweight
    >> test.py extension for the _command_ subdirectory. The test
    >> scripts can written to implement/use any testframework you
    >> like. Maybe a

    Greg> OK, lightweight is good. (Something I occasionally need to
    Greg> be reminded of...) I'll definitely take a look at it.
Thanks


Greetings

Berthold
-- 
bhoel@starship.python.net / http://starship.python.net/crew/bhoel/
        It is unlawful to use this email address for unsolicited ads
        (USC Title 47 Sec.227). I will assess a US$500 charge for
        reviewing and deleting each unsolicited ad.