Benchmarking some modules - strange result

[Marko Rauhamaa]

Dan Stromberg <drsalists at gmail.com>:

> BTW, there isn't much else going on on this computer, except some
> possible small cronjobs.

The OS caches the files in RAM. You'd expect the file I/O performance to
get much better on test reruns. On the other hand, OS cache flushing
happens at random times, which affects the results in the other
direction.

Also, I take it you aren't running your tests in a virtual machine or on
the cloud, right?

Finally, you should run each test case in isolation. IOW, restart Python
between each test case so the test cases don't interfere with each
other. The startup overhead could be reduced by starting the test cases
with an os.fork(). However, the performance tests should run long enough
to overwhelm to startup and teardown overhead.


Marko