[Tutor] How do I test file operations (Such as opening, reading, writing, etc.)?

[boB Stepp]

On Wed, Jan 27, 2016 at 11:24 PM, Danny Yoo <dyoo at hashcollision.org> wrote:

> You can make the file input/output interface a parameter of your editor.
>
> ###############################
> class Editor(object):
>     def __init__(self, filesystem):
>         self.filesystem = filesystem
>     ...
> ################################
>
> Since it's an explicit parameter, we can pass in either something that
> does it "for real" by using the built-in input output functions, or we
> can "fake it", by providing something that's convenient for unit
> tests.
>
>
> What do we need out of an input/output interface?  Well, maybe a few
> basic operations.  Let's say that we need to be able to do two things:
>
>
>     1. Open files, which returns a "filelike" object.  If it can't
> find the file, let's have it raise an IOError.
>
>     2. Create new files, which returns a "filelike" object.
>
> This is admittedly bare-bones, but let's demonstrate what this might
> look like.  First, let's see what a real implementation might look
> like:
>
> ################################
> class RealFilesystem(object):
>     def __init__(self):
>         pass
>
>     def open(self, filename):
>         return open(filename, 'r')
>
>     def create(self, filename):
>         return open(filename, 'w')
> ################################
>
> where we're just delegating the methods here to use the built-in
> open() function from Python's standard library.
>
> If we need to construct an editor that works with the real file
> system, that's not too bad:
>
>     editor = Editor(filesystem=RealFilesystem())

I was already planning on designing a class to handle my program's
file I/O.  I will probably need to add an append method, too.  Thanks
for giving my a sound starting point!


> Now what about a test-friendly version of this?  This actually isn't
> bad either; we can make judicious use of the StringIO class, which
> represents in-memory streams:
>
> ###############################################
> from StringIO import StringIO
>
> class FakeFilesystem(object):
>     """Simulate a very simple filesystem."""
>     def __init__(self):
>         self.filecontents = {}
>
>     def _open_as_stringio(self, filename):
>         filelike = StringIO(self.filecontents[filename])
>         real_close = filelike.close
>         def wrapped_close():
>             self.filecontents[filename] = filelike.getvalue()
>             real_close()
>         filelike.close = wrapped_close
>         return filelike
>
>     def open(self, filename):
>         if filename in self.filecontents:
>             return self._open_as_stringio(filename)
>         else:
>             raise IOError, "Not found"
>
>     def create(self, filename):
>         self.filecontents[filename] = None
>         return self._open_as_stringio(filename)
> ################################################
>
> (I'm using Python 2.7; if you're on Python 3, substitute the initial
> import statement with "from io import StringIO").

I was just scanning the docs on io.  A note relevant to IOError:

"Changed in version 3.3: Operations that used to raise IOError now
raise OSError, since IOError is now an alias of OSError."


> This is a class that will look approximately like a filesystem,
> because we can "create" and "open" files, and it'll remember.  All of
> this is in-memory, taking advantage of the StringIO library.  The
> "tricky" part about this is that we need to watch when files close
> down, because then we have to record what the file looked like, so
> that next time we open the file, we can recall it.
>
>
> Let's see how this works:
>
> ################################
>>>> fs = FakeFilesystem()
>>>> fs.open('hello')
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "fake_filesystem.py", line 21, in open
>     raise IOError, "Not found"
> IOError: Not found
>>>> h = fs.create('hello')
>>>> h.write('hello world')
>>>> h.close()
>>>> h2 = fs.open('hello')
>>>> h2.read()
> 'hello world'
>>>> h2.close()
> ################################

I will need to read the io docs in detail, but I am wondering if the
"with open ..." context manager is still usable to handle simulated
file closing using this technique?

> So for our own unit tests, now we should be able to say something like this:
>
> ##################################################
>     def test_foobar(self):
>         fs = FakeFileSystem()
>         fs.filecontents['classifiers.txt'] = """
> something here to test what happens when classifiers exists.
> """
>         e = Editor(filesystem=fs)
>         # ... fill me in!
> ##################################################

You have given me a substantial hunk of meat to chew on, Danny!  Thank
you very much for your lucid explanation and examples!!  You have
given me a very solid starting point if I choose this route.  I may
very well need to experiment with all of the approaches mentioned in
this thread.  Much to learn!

-- 
boB