[Chicago] The current state of Testing Stuff

[Kumar McMillan]

There's no magic bullet and you will get the most mileage by testing
at all levels (with the right balance).  It might help to think of it
like this.  What if you didn't have any tests whatsoever?

- You would have a hard time upgrading 3rd party libs, e.g. Django or
even Python itself
- It would be hard for other people (team mates) to work on your code
because they might not understand what will break when they change
something
- you will have a hard time understanding what will happen to your app
in the face of exceptions or error cases because those are hard to
trigger manually
- you run the risk of building a tightly coupled system
- you won't feel confident in experimenting with new code layouts (refactoring)
- development might feel quicker but it will soon become cumbersome
during the debugging phase

That said, it's easy to write brittle or useless tests.  If you find
yourself deleting massive amounts of test code on each commit don't
get discouraged from testing!  Instead try to figure out why the tests
were brittle.  For people new to testing I'd suggest trying to avoid
unit tests because they are the easiest to mess up.  Start at a higher
level like testing each web request or each command line script.
After that you'll start to see more efficient tests that integrate
only a few components at a time.  This is my 2 cents from many
failures and lots of trial and error.  Pretty soon you won't trust any
code unless it has a test and writing the tests will come naturally --
won't slow you down.

Kumar

On Fri, Dec 10, 2010 at 10:48 AM, Brian Ray <brianhray at gmail.com> wrote:
> This is an intentionally vague topic regarding testing from
> acceptability to unit tests. I am suggesting things have changed and
> perhaps it is time to review testing methodologies.  I am sure many of
> you have dealt with this change already. At least this could be an
> interesting topic for some I hope.
>
> I recall in 2005 ChiPy had a plethora of talks on unit testing, Mock
> Objects and fitness testing. Later we talked a lot about nose tests
> and runner tools. More recently we had talks on great tools like tox
> but it still included things like hudson to run. Somewhere in between
> we talked about Selenium.  There was a link (oh here it is) that lists
> tools http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy . I am
> not sure this test is exhaustive.
>
> So here comes my question.... What did we learn from these different
> efforts in testing; what has changed in testing; what is the best
> modern method to implement large scale testing that covers the whole
> stack? How much testing in development should be made available for
> testing in QA and acceptability?
>
> What is the point of writing test on something that will never fail?
> Have testing tools changed to become more restful? Has anyone ever had
> a test failure from an automated suite that actually pointed to
> something useful? It seems testing the smaller lower level items can
> be covered well with Unit Tests. Higher and middle level items perhaps
> are covered well by fitness and maybe browser level? How do we
> automate useful tests? How does one approach testing more complicated
> things like events and threads?  Now things are becoming so mashed...
> what are people doing to test interpolation with thinks like web
> services that someone else maintains?
>
> I guess I am also looking for life cycle configuration ideas.  I could
> see someone saying something like this:  "We write UnitTests we run
> from Nose for TDD, for acceptability in development we use urllib2 to
> test RESTFul stuff, and then on top of that we use Selenium for
> browser testing..."
>
> --
>
> Brian Ray
> _______________________________________________
> Chicago mailing list
> Chicago at python.org
> http://mail.python.org/mailman/listinfo/chicago
>