Modifying a built-in function for logging purposes

[Bengt Richter]

On Sat, 14 May 2005 19:19:08 -0700, Robert Kern <rkern at ucsd.edu> wrote:

>qwweeeit at yahoo.it wrote:
>> Hi Greg,
>> thank for your replay, but I didn't succeed in any way. You must
>> consider however that I'm not a Python "expert"...
>> IMHO, it must be a script that change part of the interpreter, and
>> substitute a new module (py) in the place of the standard one (py or
>> pyc). The standard module must be saved in order to be able to undo the
>> changes and go back to the normal behaviour.
>> The problem is that I don't know if the built-in functions like open
>> (or file) are written in Python or in C and, besides that, if they can
>> be modified.
>
>Short answer: if you don't know stuff like this, then you probably 
>shouldn't mess around with the builtins in production code.
>
>Depending on your operating system, there are probably programs that let 
>you list all of the files that have been opened on your system. For 
>example, on OS X, lsof(1) does the trick.
>
>> Other solutions which modify the source to be logged, are not
>> solutions, because it is far simpler to introduce here and there print
>> statements... 
>
Playing with this a little:

----< logopen.py >---------------------------------------
import sys, time

class OpenLogger(object):
    def __init__(self, logfile=sys.stdout):
        self.logfile = logfile
        self.old_open = __builtins__.open
    def open(self, path, *mode):
        openargs = ', '.join(map(repr, mode and [path, mode[0]] or [path]))
        tt = time.time()
        thdr = '%s.%02d' % (time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(tt)), int(tt*100)%100) 
        print >> self.logfile, '%s: open(%s) called from %r in "%s" line %s' % ((thdr, openargs) + 
            [(f.f_code.co_name, f.f_code.co_filename, f.f_lineno) for f in [sys._getframe(1)]][0])
        return self.old_open(path, *mode)
    def on(self):
        __builtins__.open = self.open
    def off(self):
        __builtins__.open = self.old_open

def main(*logfile):
    logger = OpenLogger(*logfile)
    try:
        logger.on()
        script_to_monitor = sys.argv[1]
        sys.argv[0:] = sys.argv[1:]
        xdict = dict(__builtins__=__builtins__, __name__='__main__')
        execfile(script_to_monitor, xdict)
    finally:
        logger.off()

if __name__ == '__main__':
    if not sys.argv[2:]: raise SystemExit, """
        Usage: [python] logopen.py [-log logfile] script_to_monitor [script_to_monitor args]"""
    logfile = sys.argv[1] == '-log' and sys.argv.pop(1) and [open(sys.argv.pop(1), 'a')] or []
    main(*logfile)
---------------------------------------------------------

A script whose opens we can monitor, that opens a file from module and function scope:
----< pnlines.py >---------------------------------------
import sys
print '----< %r' % sys.argv[1]
for i, line in enumerate(open(sys.argv[1], 'r')):
    print '%4s: %s' %(i+1, line.rstrip())

def foo(nlines=1):
    for i, line in enumerate(open(sys.argv[1])):    # test default 'r'
        print '%4s: %s' %(i+1, line.rstrip())
        if i+1 >= nlines: break
print '----< 3 lines of %r' % sys.argv[1]
foo(3)
---------------------------------------------------------

Result (tested only as as far as you see here):

[ 0:23] C:\pywk\clp>del logopen.txt
Could Not Find C:\pywk\clp\logopen.txt

[ 0:23] C:\pywk\clp>py24 logopen.py

        Usage: [python] logopen.py [-log logfile] script_to_monitor [script_to_monitor args]

Ok, so we pass pnlines.py as the file for it itself to print:

[ 0:23] C:\pywk\clp>py24 logopen.py -log logopen.txt pnlines.py pnlines.py
----< 'pnlines.py'
   1: import sys
   2: print '----< %r' % sys.argv[1]
   3: for i, line in enumerate(open(sys.argv[1], 'r')):
   4:     print '%4s: %s' %(i+1, line.rstrip())
   5:
   6: def foo(nlines=1):
   7:     for i, line in enumerate(open(sys.argv[1])):    # test default 'r'
   8:         print '%4s: %s' %(i+1, line.rstrip())
   9:         if i+1 >= nlines: break
  10: print '----< 3 lines of %r' % sys.argv[1]
  11: foo(3)
  12:
----< 3 lines of 'pnlines.py'
   1: import sys
   2: print '----< %r' % sys.argv[1]
   3: for i, line in enumerate(open(sys.argv[1], 'r')):

[ 0:24] C:\pywk\clp>type logopen.txt
2005-05-15 00:24:06.55: open('pnlines.py', 'r') called from '?' in "pnlines.py" line 3
2005-05-15 00:24:06.55: open('pnlines.py') called from 'foo' in "pnlines.py" line 7

Maybe the OP can build on this and contribute back something more useful and tested ;-)

Regards,
Bengt Richter