Testing python command line apps -- Running from within the projects w/o installing

[Ben Finney]

Göktuğ Kayaalp <self at gkayaalp.com> writes:

> On Fri, Nov 01, 2013 at 10:42:23AM +1100, Ben Finney wrote:
> > Keep the body of “if __name__ == '__main__':” to an absolute
> > minimum. Put all of the set-up and process-end functionality into
> > discrete functions with discrete purposes, clear return values, and
> > explicit parameters.
>
> This is usually the way I write my modules, and I do test my command
> line interfaces in a manner similar to your example.  But what I was
> asking is how to actually run these programs within a command shell
> session and test them in a more *tangible* fashion:

I think by “more tangible fashion” you're asking not about unit testing,
but about full-system testing.

This is called by several names – behaviour testing, integration testing,
acceptance testing – each having different connotations.

Testing at levels of abstraction above the unit is important, but
Python's ‘unittest’ is not a good fit. You'll need a different tool.

For behaviour testing, I recommend Behave, which lets you describe
assertions in English and have them automatically tested
<URL:https://pypi.python.org/pypi/behave/>.

For integration testing, I recommend an automated build system like
Jenkins <URL:http://jenkins-ci.org/> which can get your full source
tree, build it using your build system, and run all your arbitrary test
commands.

For running arbitrary commands as tests, you might be interested in
Atheist <URL:http://arco.esi.uclm.es/~david.villa/atheist/html/>. I
haven't tried it.

> I write a secondary script, 'app_tester', which is similar to 'app',
> but before running app.main.main, it inserts ~/app to the front of
> sys.path.

I think that's a poor solution. It's the worst of both worlds: you have
a special-case tool, but one that doesn't actually test the application
properly since it's not a very accurate representation of how the
application will actually be installed. Better is simply using the build
system to do a real, temporary install, and running the tests in there.

-- 
 \        “Odious ideas are not entitled to hide from criticism behind |
  `\          the human shield of their believers' feelings.” —Richard |
_o__)                                                         Stallman |
Ben Finney