[Numpy-discussion] Handling named temporary files in tests

[Francesc Alted]

A Monday 09 March 2009, David Cournapeau escrigué:
> Hi,
>
>     While fixing several windows specific unit test failures, I
> encountered some problems I am not sure how to solve. In particular,
> we have a relatively common idiom as follows:
>
> Open file securely with a visible name (using NamedTemporaryFile)
> write some content into it
> open the file with another open call
>
> Of course, this does not work on windows. NamedTemporaryFile is
> basically useless on this platform (windows refuses to let a process
> to reopen a file opened from NamedTemporaryFile). I can see two
> solutions: - using mkstemp + re-opening the file later from the name
> returned by mkstemp: AFAICT, this basically defeats the whole purpose
> of mkstemp - have our own layer to bypass mkstemp on windows, where
> security is not a concern anyway, and use the proper functions on
> sane platforms.
>
> Do people have an opinion on this ? Or maybe a solution to the
> problem altogether ?

We have a similar case of use for the PyTables project, and we ended 
implementing a class that is meant to be used as a mixin from which the 
test classes inherits.  The class is, more or less:

class TempFileMixin:
    def setUp(self):
        """Set ``h5file`` and ``h5fname`` instance attributes."""
        self.h5fname = tempfile.mktemp(suffix='.h5')
        self.h5file = tables.openFile(
            self.h5fname, 'w', title=self._getName())

    def tearDown(self):
        """Close ``h5file`` and remove ``h5fname``."""
        self.h5file.close()
        self.h5file = None
        os.remove(self.h5fname)

    def _reopen(self, mode='r'):
        """Reopen ``h5file`` in the specified ``mode``."""
        self.h5file.close()
        self.h5file = tables.openFile(self.h5fname, mode)
        return True

The advantage is that, by simply inheriting from `TempFileMixin`, the 
developer have avaliable the ``h5file`` (file handler) and ``h5fname`` 
(file name).  The mixin is responsible to open, close and remove the 
temporary file.  In addition, the `_reopen()` method allows you to 
manually close and re-open the temporary, if that is what you want.

An example of use:

# Test for building very large MD columns without defaults
class MDLargeColTestCase(common.TempFileMixin, common.PyTablesTestCase):
    reopen = True

    def test01_create(self):
        "Create a Table with a very large MD column.  Ticket #211."
        N = 2**18
        cols = {'col1': Int8Col(shape=N, dflt=0)}
        tbl = self.h5file.createTable('/', 'test', cols)
        tbl.row.append()   # add a single row
        tbl.flush()
        if self.reopen:
            self._reopen()
            tbl = self.h5file.root.test
        # Check the value
        assert allequal(tbl[0]['col1'], zeros(N, 'i1'))

This proved to be pretty handy in practice.

HTH,

-- 
Francesc Alted