[Baypiggies] Discussion for newbies/beginner night talks - test results

[Shannon -jj Behrens]

On 2/15/07, Chad Netzer <chad.netzer at gmail.com> wrote:
> On 2/15/07, Daniel Yoo <dyoo at cs.wpi.edu> wrote:
>
> > To make this easier, we should change the defintions so that these are
> > actual functions that can be compared functionally.  For example:
> >
> > ##############################################
> > def holding_with_lists():
> >      l = []
> >      for x in range(0,100000):
> >          l.append("abcdef")
> >      return ''.join(l)
> >
> > def holding_naive():
> >      l = ""
> >      for x in range(0,100000):
> >          l += "abcdef"
> >      return l
> >
> > assert holding_with_lists() == holding_naive()
> > ##############################################
> >
> > Otherwise, without making sure these different implemenations return the
> > same result, we risk comparing apples to oranges.  I suspect that's what's
> > happened here.
>
>
> Danny, in addition to your good advice, I'd also suggest not using
> range() in the loop, since Dennis was also intending to look at memory
> footprint.  Use xrange(), or make a simple iterator instead.
>
> Dennis, you said in your example, "The final time.sleep(2000) is to
> allow time for garbage collection to reduce memory usage."  However,
> that certainly will have no effect with CPython, nor do operating
> system memory allocators work that way.  If I'm wrong about this,
> someone please enlighten me.  You could try:
>
> gc.collect()
>
> but these simple examples probably don't have any unreachable memory
> cycles.  In general, memory usage benchmarking can be a tricky
> undertaking, sometimes even for 'simple' examples.

I saw a talk at PyCon two years ago that said that Python would never
give memory back to the OS.  That's probably changed, but I'd check
first.

I think this discussion is a bit of a red herring.  I still think we
should focus on readability.  If Dennis insists on using += instead of
a nice templating engine like Cheetah, and it turns out that
list.append uses less memory than +=, then let's remember that that
StringIO takes care of using a list internally.  I suggest he do the
following since he seems to dislike triple quoted strings (which I
don't agree with):

from cStringIO import StringIO
buf = StringIO()
w = buf.write

w("String")
w("String %s bar." % name)
...

return w.getvalue()

Better yet, wrap that boilerplate in a function and use a "with"
statement to do it automatically ;)

As another contribution to perhaps make him happier, I submit this,
which I use to use before I switched to Cheetah:

===============================================
def evalstr(s, dollar="$"):

    """Evaluate a string with embedded python expressions.

    This is a Python version of the function evalstr, which is used for
    variable interpolation.

    For instance,

        a = 5
        evalStr("I have $a$ dogs.")

    will return

        "I have 5 dogs."

    The code between the $'s can be any valid Python expression.  I'll just
    basically do

        str(eval(expression), yourGlobals, yourLocals)

    Since eval is being used, the expression must return some value.  To escape
    $'s in s, just use $$ (even within expressions).  To use a different
    character than $, pass the desired character in the second (optional)
    argument.  If there are an odd number of $ symbols, a ValueError exception
    will be raised.  Here are some additional examples (assuming a = 5):

    "My dog is $a$ years old." -> "My dog is 5 years old.",
    "I have $$5.00" -> "I have $5.00",
    "My $a$ year old dog has $$5.00" -> "My 5 year old dog has $5.00",
    "My dog has $'$$%s.' % a$" -> "My dog has $5.",
    "My dog has $'$$' + str(a) + '.'" -> ValueError exception,
    "My dog is $a$.$" -> ValueError exception,
    "$$ is a char." -> "$ is a char.",
    "$a$ is a number." -> "5 is a number.",
    "$a$" -> "5",
    "$a" -> ValueError exception,
    "I have a $$" -> "I have a $",
    "" -> "",
    "Spam" -> "Spam",
    "$" -> ValueError exception,
    "$ evalstr('I am #a#.', '#') $" -> "I am 5."

    """

    # By the way, I'm sorry if this stuff is hard to read.  If you
    # can come up with something more elegant, I'd love to see it,
    # but this is the best that I could come up with.

    import string
    import sys
    inEv = 0     # Are we in eval mode (i.e. inside a single $)?
    inDollar = 0 # We just got a dollar.  We might be in an escape sequence.
    ret = ""     # This is the string we're returning.
    tok = ""     # The token we're currently working with.
    stack = []   # All good languages are stack based ;)
    i=j=0        # These are indexes into s.

    # Grab the calling function's namespace.  I can't find a nice
    # way to do this other than manually generating an exception
    # and surfing the traceback.  As for performance, you can do
    # this about 10000 times in a bit less than a second on my
    # PIII 600 mhz system.

    try:
        1/0
    except:
        traceback = sys.exc_info()[2]
        currFrame = traceback.tb_frame
        prevFrame = currFrame.f_back
        prevGlobals = prevFrame.f_globals
        prevLocals = prevFrame.f_locals

    # If they forget to close the $, raise a ValueError exception.

    if string.count(s, dollar) % 2:
        raise ValueError, "unmatched dollar symbol"

    sLen = len(s)
    while 1:

        # Get next token.

        j = i
        if j==sLen:
            tok = ""
        elif s[j] == dollar:
            tok = dollar
            i = j+1
        else:
            while j<sLen and s[j]!=dollar: j=j+1
            if j==sLen:
                tok = s[i:]
                i = sLen
            else:
                tok = s[i:j]
                i = j

        # Deal with receiving $'s (may call continue).

        if tok == dollar:
            if inDollar: inDollar = 0
            else:
                inDollar = 1
                continue

        # Deal with appending stuff to the stack or to ret.

        if inDollar:
            inEv = not inEv
            inDollar = 0
        if inEv: stack = stack + [tok]
        else:
            if len(stack) > 0:
                ret = ret + \
                    str(eval(string.join(stack, ""),
                         prevGlobals,
                         prevLocals))
            ret = ret + tok
            stack = []

        # Deal with end of string.

        if tok == "": break
    return ret

# I also have a C version of this if you want it.
===============================================

Happy Hacking!
-jj

-- 
http://jjinux.blogspot.com/