How to test?

[Manfred Lotz]

On Sat, 25 Apr 2020 18:41:37 +1200
DL Neil <PythonList at DancesWithMice.info> wrote:

> On 25/04/20 5:16 PM, Manfred Lotz wrote:
> > On Fri, 24 Apr 2020 19:12:39 -0300
> > Cholo Lennon <chololennon at hotmail.com> wrote:
> >   
> >> On 24/4/20 15:40, Manfred Lotz wrote:  
> >>> I have a command like application which checks a directory tree
> >>> for certain things. If there are errors then messages will be
> >>> written to stdout.
> >>>
> >>> How to test this in the best way?
> >>>
> >>> One idea was for the error situations to write messages to files
> >>> and then later when running the tests to compare the error
> >>> messages output to the previously saved output.
> >>>
> >>> Is there anything better?
> >>>
> >>>      
> >>
> >> Maybe I am wrong because I don't understand your scenario: If your
> >> application is like a command, it has to return an error code to
> >> the system, a distinct number for each error condition. The error
> >> code is easier to test than the stdout/stderr.
> >>  
> > 
> > Yes, a different error code for each condition is good to test.
> > However, I need to test as well the correct file name and other
> > values belonging to a certain error condition.  
> 
> It is frustrating to be shown only part of the information, and later
> be told our efforts aren't good-enough. 

I found the tips I got here quite good. Actually, each time when I had
a questions here I got competent answers.

> How about respecting our
> (donated) time, and posting some sample code that shows exactly
> what/where the problem lies?
> 

Sorry, perhaps I was too lazy to explain. Hope the following
explanation isn't too bad.


What I do here specifically is to check directory trees' file objects
for user and group ownerships as well as permissions according to a
given policy. There is a general policy and there could be exceptions
which specify a specific policy for a certain file.

The policy file is a JSON file and could have different categories.
Each category defines a policy for a certain directory tree. Comand
line args could overwrite directory names, as well as user and group.
Or if for example a directory is not specified in the JSON file I am
required to specify it via command line. Otherwise no check can take
place.

Here an example of a simple json file:

default":
  {
      "match_perms":  "644",
      "match_permsd":  "755",
      "match_permsx":  "755",
      "owner":  "manfred",
      "group":  "manfred"
  }
}

another simple example:

{
"default":
  {
    "have_all_permsx" : "555",
    "have_all_perms" : "444",
    "have_all_permsd" : "555",
    "owner":  "manfred",
    "group":  "manfred"
  }
}


class Category contains a policy.

Here a simplified version of Category


class Category:
    def __init__(self, cat, jsoncat, dirdict, ownerdict, groupdict):

        self.cat = cat
        self.jsoncat = jsoncat
        self.keep_fs = jsoncat.get('keep_fs')
        self.dir = jsoncat.get('dir')

        if dirdict.get(cat):
            self.dir = os.path.realpath(dirdict[cat])

        self.policy = Policy(jsoncat, ownerdict, groupdict)

    def check(self, fpath):
        self.policy.check(fpath)

It is important to note that ownerdict and groupdict are  overwrites
from the command line, and could be empty if nothing was
specified.

In my main routine I do this

...
  for cat in cats:
        ... 
        for d in scantree(cat.dir):
            # if `keep_fs` was specified then we must
            # make sure the file is on the same device
            if cat.keep_fs and devid != get_devid(d.path):
                continue

            cat.check(d)

...


class Policy contains required user and group owners and a
list of permission objects for files, for executable files and for
directories. 


Those permissions are described like this

class BasePerms:
    def __init__(self, perms: str):
        """Create an object holding a permission which is octal"""
        self.perms = int(perms, 8)

class MatchPerms(BasePerms):
    def check(self, fpath: str, mode: int):
        mode = mode & 0o7777
        if mode != self.perms:
            print("%s perms differ, should have %s has %s" % (fpath, oct(self.perms), oct(mode)))

class HaveAllPerms(BasePerms):
    def check(self, fpath: str, mode: int):
        mode = mode & 0o7777
        if (mode & self.perms) != self.perms:
            print("%s: perms differ, should have all of %s has %s" % (fpath, oct(self.perms), oct(mode)))


After getting some ideas here I want to change my code to return an
error object in check() instead of doing a print(). Or return None
if all is ok. 

Then the error object could have a print-like method and a compare
method so that later on when doing a test I could compare two error
objects.  Perhaps I can combine it with callback. I have to play with
these things.


-- 
Manfred