Has anyone released a Python "mock filesystem" for automated testing?

[Remy Blank]

Peter Hansen wrote:
> Googling the archives and the web suggests that only I and
> Remy Blank have done much along these lines.  I don't see
> any sign that anyone has actually released such a beast
> yet, however.

Not yet, not yet...

> For now, in case anyone is interested, I have support
> for basic open, read/write, __iter__, close, "r" and "w"
> modes, a couple of the most basic exceptions, and a
> tiny handful of lesser things.  All well supported by
> their own unit tests, of course, to ensure quality.

The status on my side is that I have the "read-only" portion working
quite well, with support for file attributes (stat) but no access
checking yet. Also supported by a unit test suite.

However, when I started on the "write" code, I stumbeled on a few areas
where I wasn't sure how the system should behave. A few examples:

 - Where do you land if you chdir("/dir1/symlink/../subdir") when
symlink points to /dir2/subdir? If you os.abspath() the path before
interpretation, you land in /dir1/subdir. If you don't, you land in
/dir2/subdir.
 - Who can remove a file?
 - Who can change the mode of a file? Its owner and group?
 - Which operations change atime? mtime? ctime?
 - What exceptions are thrown for various error conditions?

Those are the simple ones I remember, but I know there were other where
I couldn't find the solution easily in manpages. And I didn't want to
have to look everything up in the Linux kernel source code. So the next
idea was to change the test suite so that it could run either on the
real filesystem, or on the mockfs. That way, I could ensure that the
behaviour was identical.

This proved to be more challenging than I thought, because for most
tests (chmod, chown, access control) you need to be root to run the test
suite. And I felt too uncomfortable running a test suite that could "rm
-rf /" by mistake, as root.

So the next idea was to write a framework where you could run test cases
within a chroot()'ed environment, so that (as long as you don't use
chroot() in you test cases) you were pretty safe not to wipe your harddisk.

This has finally worked out better than I had expected. I have a
"JailTestCase" that creates a new directory for every test case,
chroot()s to it in setUp(), runs the test case and exits the chroot jail
in tearDown() (that part was tricky). There is also a preJailEntry()
hook for e.g. unpacking a tar archive into the directory before entering
the chroot jail. The jail directories are never removed automatically,
so they can be examined in case of failure. Obviously, it works only on
*nix type systems.

I have also implemented a FileSystem class that allows the easy creation
of all filesystem entities (files, directories, pipes, sockets, ...).

Here's a class/def description of the library:

def enterJail(path):
def exitJail():

class JailTestCase(unittest.TestCase):
        def setJailDir(path):
        setJailDir = staticmethod(setJailDir)
        def jailDir():
        jailDir = staticmethod(jailDir)
        def __init__(self, methodName='runTest'):
        def preJailEntry(self):
        def setUp(self):
        def tearDown(self):

class RealFilesystem:
        def createFile(self, path, mode=None, uid=None, gid=None,
atime=None, mtime=None, content=None):
        def createDir(self, path, mode=None, uid=None, gid=None,
atime=None, mtime=None):
        def createSymlink(self, path, uid=None, gid=None, atime=None,
mtime=None, target=None):
        def createCharDevice(self, path, major, minor, mode=None,
uid=None, gid=None, atime=None, mtime=None):
        def createBlockDevice(self, path, major, minor, mode=None,
uid=None, gid=None, atime=None, mtime=None):
        def createPipe(self, path, mode=None, uid=None, gid=None,
atime=None, mtime=None):
        def createSocket(self, path, mode=None, uid=None, gid=None,
atime=None, mtime=None):

So I am now at the point where I could resume working on mockfs. Now,
one of the reasons why I wanted a mockfs library (besides having to run
tests as root) was that I was afraid that using the real filesystem
would be too slow. Well, I was proven wrong while implementing this
ChrootJail module: it is actually quite fast. At the moment, I am
wondering if it wouldn't be more efficient to just polish up this
ChrootJail module and use that instead of a mockfs.

If you're interested, I can send you the module with its unit test
suite. Feedback is as always very welcome.

Best regards.
-- Remy


PS: Here's the class/def state of my mockfs (some functions are not yet
fully functional):

class Inode(object):
        def Size(self):
        def IsDirectory(self):
        def IsSymLink(self):
        def AddLink(self):
        def RemoveLink(self):
        def ExtraStatFields(self):
        def Stat(self):

class DirectoryInode(Inode, dict):
        def __setitem__(self, Key, Value):
        def __delitem__(self, Key):
        def clear(self):
        def update(self, Rhs):
        def setdefault(self, Key, Value):
        def pop(self, *Args):
        def popitem(self):

class FileInode(Inode):
class SymLinkInode(Inode):
class PipeInode(Inode):
class SocketInode(Inode):
class DeviceInode(Inode):
class BlockDeviceInode(DeviceInode):
class CharDeviceInode(DeviceInode):

class File(object):
        def __init__(self, FileName, Mode="r", BufSize=-1):
        def close(self):
        def flush(self):
        def read(self, Size=-1):
        def readline(self, Size=-1):
        def readlines(self, SizeHint=None):
        def xreadlines(self):
        def seek(self, Offset, Whence=0):
        def tell(self):
        def truncate(self, Size=None):
        def write(self, Str):
        def writelines(self, Sequence):
        def __iter__(self):
        def next(self):

class MockFilesystem(object):
        def __init__(self):
        def AddDirectory(self, Path, **KwArgs):
        def AddFile(self, Path, **KwArgs):
        def AddSymLink(self, Path, **KwArgs):
        def AddPipe(self, Path, **KwArgs):
        def AddSocket(self, Path, **KwArgs):
        def AddBlockDevice(self, Path, **KwArgs):
        def AddCharDevice(self, Path, **KwArgs):
        def AddHardLink(self, Path, Target):
        # Substitutes for os functions
        def chdir(self, Path):
        def getcwd(self):
        def getcwdu(self):
        def chroot(self, Path):
        def chmod(self, Path, Mode):
        def chown(self, Path, Uid, Gid):
        def lchown(self, Path, Uid, Gid):
        def listdir(self, Path):
        def lstat(self, Path):
        def readlink(self, Path):
        def remove(self, Path):
        def rmdir(self, Path):
        def stat(self, Path):
        def statvfs(self, Path):
        def unlink(self, Path):
        def utime(self, Path, Times):



Remove underscore and suffix in reply address for a timely response.