unit testing CGI scripts?

[Robert Brewer]

Jim Hefferon wrote:
> Does anyone have suggestions for unit testing CGI scripts?  For
> instance, how can I set the FieldStorage to have certain values?
> Do I have to go with a dummy FieldStorage class?  And is there a
> better way to check the output than to compare the resulting 
> (sometimes very large) strings?  Changing the HTML slightly means
> that I have to edit the tests and that defeats the purpose of the
> tests, to some extent.  I'd greatly appreciate any tips.

There have been several times when I've needed to not only unit test,
but actually pull out a debugger for a CGI script. Both of these were
made possible by abstracting the UI from the rest of the code, as I
described at:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/225299

Having the UI isolated in this way allows me to write a third UI which
pretends to be a webserver, but which I can load with my own "client
requests" (some details may be different from the (older) recipe, but
you get the idea):

import re
from HTMLParser import HTMLParser
from cgi import parse_qs

import dejavu
from dejavu.html import uihtml


class UserInterfaceHTMLDebug(uihtml.UserInterfaceHTML):
    """Create an interface imitating a CGI interface.
    
    Use this interface to test or debug (step through) dejavu code,
    acting as if the requests were coming from a web browser.
    
    For example:
    
    ns = dejavu.Namespace()
    cfgs = r'C:\data\default.conf, C:\data\myapp.conf'
    ui = UserInterfaceHTMLDebug(cfgs, ns, 'rbre')
    print ui.request('http://localhost/myapp/index.htm', 'ID=300')
    ID = ui.responses[-1].form_elements()['ID']['value']
    """
    
    responses = None
    
    def __init__(self, configFileNames, namespace, user=''):
        
        # Set response hook.
        self.responses = []
        
        # Set standard properties.
        self.protocol = u'https'
        self.port = 80
        self.hostname = u'localhost'
        self.physicalPath = u''
        self.user = user
        self.path = u'/'
        self.script = u''
        
        config = {u'Dispatch': {u'pattern':
r'%(AppURLRoot)s/(.*)\.htm$',
                                u'repl': r'%(\1)s'},
                  u'Dispatches': {u'default' :
u'dejavu.html.default.homepage',
                                  u'-Error':
u'dejavu.html.uihtml.error'},
                  u'ConfigFiles': configFileNames,
                  }
        self.load(config)
    
    def request(self, script, params='', newUser=None):
        """Request a page (calls dispatch())."""
        
        atoms = script.split(u"/")
        self.script = atoms.pop()
        self.path = u"/".join(atoms)
        self.physicalPath = u''
        if newUser is not None:
            self.user = newUser
        
        self.set_params(params)
        retVal = self.dispatch()
        return '%s/%s: %s' % (self.path, self.script, retVal)
        
    def set_params(self, queryString):
        """Parse CGI params into a paramset."""
        self.requestParams.flush()
        inParams = parse_qs(unicode(queryString), True, False)
        for eachName, eachValue in inParams.items():
            for eachSubValue in eachValue:
                self.requestParams.add_param(eachName, eachSubValue)
    
    def write(self, textToOutput, doMarkUp=False):
        if doMarkUp:
            textToOutPut = self.mark_up_for_output(textToOutPut)
        self.responses.append(Response(textToOutput.encode("UTF-8")))


class Response(unicode):
    """A web page response. This is a unicode string with extra
methods."""
    
    def __repr__(self):
        return u'<uidebug Response object>'
    
    def elements(self, parser):
        return parser.get_elements(self)
    
    def form_elements(self):
        elems = FormElementParser('input').get_elements(self)
        elems.extend(FormElementParser('select').get_elements(self))
        return elems


class DejavuParser(HTMLParser):
    elements = {}
    tagType = u''
    
    def __init__(self, tagType=''):
        HTMLParser.__init__(self)
        self.elements = {}
        self.tagType = tagType
    
    def get_elements(self, input):
        self.feed(input)
        self.close()
        return self.elements


class FormElementParser(DejavuParser):
    def handle_starttag(self, tag, attrs):
        if tag == self.tagType:
            attrs = dict(attrs)
            if 'name' in attrs:
                attrTable = dict([(key, value) for key, value
                                  in attrs.iteritems() if key !=
'name'])
                self.elements[attrs['name']] = attrTable


Best investment I ever made, breaking out the UI like that. Hope it at
least gives you some ideas.

To Dutin Lee: the same can be said for database access, only it's more
complicated. I use an abstract StorageManager, with subclasses for ADO,
ODBC, etc.. including a subclass just for debugging. To some extent, it
depends on how large your project is; i.e., whether such an investment
will pay off.


Robert Brewer
MIS
Amor Ministries
fumanchu at amor.org