general module auditing

[Irmen de Jong]

On 4-7-2014 1:09, Rita wrote:
> 
> here is what I am doing now,
> 
> egrep 'from|import' *.py | wc -l which is giving me that. But this does not
> give me the number of times the particular module gets called. I was
> thinking of adding a logging feature to all of my modules so every time
> they get called it will be written to a log file with corresponding host
> and username. Is there an easy way to do that?


Okay I've read up a bit on Python import hooks and came up with the following code.
It hooks into Python's import mechanim by putting a custom loader into sys.meta_path
that logs the time, the user, the machine and the name of the module being imported.
Theoretically you could load this as a startup module (sitecustomize.py?) and make it
log to a central location or something like that.

The code at the end of this message outputs the following on my machine:

$ python audit.py
Hello I'm about to import a module.
2014-07-04 19:01:12,321 [irmen at Neptune] importing /cgi
2014-07-04 19:01:12,323 [irmen at Neptune] importing /urlparse
2014-07-04 19:01:12,328 [irmen at Neptune] importing /mimetools
[... and some more modules...]
Bye.



Code follows:


import sys
import logging
import socket
import os
import getpass


def setup_import_logging():
    class ContextFilter(logging.Filter):
        def filter(self, record):
            record.hostname = socket.gethostname()
            if sys.version_info < (3, 0):
                record.username = getpass.getuser()
            else:
                record.username = os.getlogin()
            return True

    # configure the logger, adapt as desired:
    logging.basicConfig(level=logging.DEBUG, format="%(asctime)s
[%(username)s@%(hostname)s] %(message)s")

    class AuditingImporter(object):
        log = logging.getLogger("auditingimporter")
        log.setLevel(logging.DEBUG)
        log.addFilter(ContextFilter())

        def find_module(self, fullname, path):
            return self.find_spec(fullname, path)

        def find_spec(self, fullname, path, target=None):
            self.log.debug("importing {path}/{fullname}".format(path=path or "",
fullname=fullname))
            return None

    sys.meta_path.insert(0, AuditingImporter())


setup_import_logging()

print("Hello I'm about to import a module.")

import cgi      # will generate a load of logging entries

print("Bye.")