a problem with writing a generator

[Tim Chase]

Paweł Banyś wrote:
>> Assuming that include directives are like
>>
>>   #include "blahblah"
> 
> Yes, I have already tried the methods related to source code processing
> using Python generators but unfortunately I am dealing with BIND and its
> named.conf files.

(dealing with BIND named.conf files doesn't sound like homework, 
so I'll bite, though this might make a pretty good CS homework 
problem in the right context)

Sounds like a recursive generator could do the trick.  I'd also 
keep track of "files already recursed" to make sure you don't 
have infinite loops.  Something like this (untested):

   import re
   import os
   class ImportException(Exception): pass
   class RecursionException(Exception): pass
   include_re = re.compile(r'^\s*#include\s+"([^"]+"')
   def importer(fname, paths=None,seen=None):
     if not paths: paths = [os.curdir]
     if seen is None: seen = set()
     f = file(fname)
     for i, line in enumerate(f):
       m = include_re.match(line)
       if m: # this line is an "#include"
         recurse_fname = m.group(1)
         if recurse_fname in seen:
           raise RecursionException(
             "[%s:%i] Recursion detected on [%s]" %
             (fname, i, recurse_fname)
             )
         for pth in paths:
           full_path = os.path.join(pth, recurse_fname)
           if os.isfile(full_path):
             seen.add(recurse_fname) # push
             for line in importer(full_path, paths, seen):
               yield line
             seen.remove(recurse_fname) # pop
             break
         else:
           raise ImportException("[%s:%i] could not find [%s]" %
             (fname, i, recurse_fname))
       else: # not an include
         yield line
     f.close()

There might be some bugs lurking in there, and there might be 
some robustness aspects to consider (like using "with" for the 
file to make sure it gets closed, or wrapping it in a try/finally 
if you're using an older version of Python), and you might also 
consider allowing a certain depth of recursion (I'd use a dict of 
fname->recursion-depth to keep tabs on how deep I'd gotten, and 
raise an exception if I go too deep), but otherwise, that should 
give you a pretty straight-forward and easy-to-tweak starting point.

Hope this helps,

-tim