Advice on optimizing a Python data driven rules engine

Malcolm Greene python at bdurham.com
Thu Aug 11 10:56:13 EDT 2016


Background: I'm building a  rules engine for transforming rows of data being returned by csv DictReader, eg. each row of data is a dict of column name to value mappings. My rules are a list of rule objects whose attributes get referenced by rule specific methods. Each rule has an associated method which gets called based on rule name.

Looking for some advice on how to optimize the BOILERPLATE portions of
the following type of code. There's an awful lot of dot dereferencing
going on. One thought was to pass in the values being dereferenced as
parameters and return value. But this would just move the dereferencing
to another point in my program and would add the overhead of parameter
passage. Is there a technique where I could store a reference to these
values that would make their access more efficient?

def action_strip(self):
    # BOILERPLATE: lookup value being transformed (possible to get a
    # direct 'pointer' to this dic entry???)
    value = self.data[self.rule.target_column]

    # BOILERPLATE: get rule's hard coded parameter
    match = self.rule.value.strip()

    # perform a generic transformation action
    while value.startswith(match):
        value = value.replace(match, '', 1).strip()
    while value.endswith(match):
        value = value[0:len(value) - len(match)].strip()

    # BOILERPLATE: update the value we just transformed
    self.data[self.rule.target_column] = value

Thank you,
Malcolm



More information about the Python-list mailing list