[AstroPy] Tests for AstroPy (was POLL: vision for a common Astronomy package)

[Victoria G. Laidler]

Brian Kloppenborg wrote:
>
>
> On 07/06/2011 09:35 AM, Victoria G. Laidler wrote:
>> Ole Streicher wrote:
>>> BTW, are there ideas for regression tests of astropy? Especially in the
>>> case you mentioned, this would be *very* helpful to check that
>>> everything works in the target system.
>> I was thinking about this as well. I suggest that there be minimum
>> testing standards for astropy packages. Packages should come with tests
>> that:
>> - can be run using a standard syntax (what?)
>> - verify that all significant elements of the package can successfully
>> execute
>> - verify that all significant elements of the package, for some minimum
>> set of common use cases, give the right answer when run
>> - come with documentation that in some way describe test coverage (how
>> much of the system do the tests cover) and the source of the test
>> answers (ie, how did you determine the "right answer" used in the tests)
> There is a nice unit testing package in python that could be used for 
> this:
> http://docs.python.org/library/unittest.html
>
> I use it quite frequently on larger projects.  Implementing is as 
> simple as a series of Assert statements whose outcome is known (i.e. 
> hard-coded).
Unittest has recently undergone a major rewrite for Python 2.7. There 
are some backwards incompatibilities, so we'd need to be somewhat 
careful here.

Unittest is also a fairly primitive testing framework (and deliberately 
written to match the Java JUnit package, which isn't always a very 
pythonic way of doing things). There are a variety of test-related 
packages out there, many of them summarized here:

http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy

although I suppose once we go outside the standard library, the issue of 
"which of these tools will astropy ship with to support its internal 
testing" becomes important. :/ Although, another possible approach is to 
segment the tests, so you can run the minimum tests (described above) 
without installing anything extra, and additional tests if you feel like 
also installing whatever the package developer likes for testing.

There's also a very active Testing in Python mailing list, to which you 
can subscribe from the taxonomy page.

Vicki