Use of logging module to track TODOs

[Jordi Riera]

Hey list,

I always have issues with TODOs as they stay in the code and most of time forgot.
On that, I tried to find a way to track them and to be able to output them automatically. Automatically is a key as if devs have to do it manually it won't be done.

While I was checking the logging module and it features I thought about a little hack on it to use it to check files and output TODOs.
Logging check files and can have different channels for each level.
_Well, it needs to write another way the TODO as it will be now a new logging level and TODO would be wrote as logger.todo('Comment here')_

Here is my code:
___________________________________________________________________
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# pylint: disable=W0511
# disabled the warning about TODOs
#
""" Adding a new level to the logging.
    the new level will help to trace the TODOs in files.
"""
import logging
import os
import sys
import json
# setting the new level in logging module.
#
new_level = (logging.DEBUG + 1, 'TODO')
logging.addLevelName(*new_level)
logging.TODO = new_level[0]

class DictFormatter(logging.Formatter):
    """ Formatter emitting data in the form of a dictionnary.

        dict Pattern:
            {
                record.name: {
                    record.lineno: {
                        'message': record.getMessage(),
                        'created': self.formatTime(record, self.datefmt),
                        'funcName': record.funcName,
                    },
                },
            }
    """
    def format(self, record):
        """ Save the data in the json file and return the return
            of the parent class.
        """
        message = record.getMessage()

        if record.exc_info:
            # Cache the traceback text to avoid converting it multiple times
            # (it's constant anyway)
            if not record.exc_text:
                record.exc_text = self.formatException(record.exc_info)

        if record.exc_text:
            try:
                message = record.exc_text
            except UnicodeError:
                # Sometimes filenames have non-ASCII chars, which can lead
                # to errors when s is Unicode and record.exc_text is str
                # See issue 8924.
                # We also use replace for when there are multiple
                # encodings, e.g. UTF-8 for the filesystem and latin-1
                # for a script. See issue 13232.
                message = record.exc_text.decode(
                    sys.getfilesystemencoding(),
                    'replace'
                )

        return {
            unicode(record.lineno): {
                u'message': unicode(message),
                u'created': unicode(self.formatTime(record, self.datefmt)),
                u'funcName': unicode(record.funcName),
            }
        }


class SpecificJsonFileHandler(logging.FileHandler):
    """ Specific FileHandler emitting a specific level of message.

        :attribute level_message: logging level of message to consider. (int)
    """
    def __init__(self, level_message, *args, **kwargs):
        super(SpecificJsonFileHandler, self).__init__(*args, **kwargs)
        self.__level_message = level_message

        # To know if it is the first time the handler emit.
        # It meant to know if we need to reset the data for this file in the json file.
        #
        self.__first_emit = True

    def emit(self, record):
        """ Only save in a json for the specific level of message. Skip all other messages."""
        if record.levelno == self.__level_message:
            with open(self.baseFilename, 'r') as json_file:
                try:
                    data = json.load(json_file)
                except ValueError:
                    data = {}

            # Test if it is the fist time the instance will emit for the logger.
            # At this point, the data from the logger is reset to avoid to keep
            # cancelled/resolved todos.
            #
            if self.__first_emit:
                data[record.name] = {}
                self.__first_emit = False

            data[record.name].update(self.format(record))

            with open(self.baseFilename, 'w') as json_file:
                json.dump(data, json_file, sort_keys=True, indent=4)


class TodoLogger(logging.getLoggerClass()):
    """ Logger accepting the Todo parameter."""
    __default_filename = 'log.log'

    def __init__(self, *args, **kwargs):
        super(TodoLogger, self).__init__(*args, **kwargs)
        self.__handler = SpecificJsonFileHandler(
            logging.TODO,
            kwargs.get('todo_filename', self.__default_filename),
        )
        self.__handler.setLevel(logging.TODO)
        self.__handler.setFormatter(
            DictFormatter()
        )

        self.addHandler(self.__handler)

    def todo(self, message, *args, **kwargs):
        """ Log 'msg % args' with severity 'TODO'.

            To pass exception information, use the keyword argument exc_info with
            a true value, e.g.

            logger.debug("Houston, we have a %s", "thorny problem", exc_info=1)

            If the dev environment is detected and even the level is not enabled,
            the log will be saved in the json file to keep the file up to date.
        """
        # Need this test or the message of TODOS will be always displayed.
        #
        if self.isEnabledFor(logging.TODO):
            self._log(logging.TODO, message, args, **kwargs)
        # If the logger is used by a developer, even if it is not enabled
        # we save the todos in the json file.
        #
        elif self.isDevEnv():
            # We will trick the system here to keep the engine running
            # and so stay compatible with any update of logging module.
            # Basically, we will make the system believing there in only
            # the handler linked to the JsonFormater to run.
            #
            # So we save the list of all the handler registered to the logger.
            #
            handlers = self.handlers
            # we register the handler we are interested in.
            #
            self.handlers = [self.__handler]
            # Run the system as nothing particular happened.
            #
            self._log(logging.TODO, message, args, **kwargs)
            # then we register back all the handlers.
            #
            self.handlers = handlers

    # pylint: disable=C0103
    # Disable the warning about the name convention. I am following the name
    # convention of the module.
    #
    @staticmethod
    def isDevEnv():
        """ Test to detect if the environment is a development environment.

            :rtype: bool
        """
        return 'DEVELOPMENT' in os.environ


logging.setLoggerClass(TodoLogger)
___________________________________________________________________
(https://github.com/foutoucour/todologger/blob/master/logger.py)
___________________________________________________________________

The code is a first try. It needs updates and surely have weaknesses. I plan to have a JsonToMd translator pre-push hook. So each time a push is sent, a todo.md will be update with the new version of the json.
fixme, note etc. might be easy added if needed.
I would be interested to know your point of view on this.

How do you do with your TODOs?

Regards,
Jordi