Python syntax in Lisp and Scheme

Wed Oct 8 00:55:33 EDT 2003

On 07 Oct 2003 14:50:04 +0200, Matthias <no at spam.pls> wrote:

>bokr at oz.net (Bengt Richter) writes:
>
>> On 06 Oct 2003 12:54:30 +0200, Matthias <no at spam.pls> wrote:
>> >1.) Inventing new control structures (implement lazy data structures,
>> You mean like an object olazy with olazy.x and olazy.y, where x might
>> be an integer 123 and y might be the text of the latest bug python report
>> on sourceforge? Not so hard with properties (you'd proabably want to cache
>> the latest for y and not re-check for some reasonable interval, but that
>> doesn't change client code (unless you later decide you want the value to
>> be a tuple of (timestamp, text), etc.)
>
>Actually, I meant more lazy-like-lazy-in-Haskell.  Infinite data
>structures and such.  "primes" being a list representing _all_ prime
>numbers for instance.  You can build this as soon as you have closures
>but making the construction easy to use for the application programmer
>might be a challenge without macros.  But I don't know what
>"properties" are in Python, possibly they are built for exactly that.
No, generators are closer to "exactly that" e.g., in the following function,
"yield" is the keyword that makes it into a generator. The initial call effectively
becomes a factory function call that returns an initialized generator, and calls
to the generators' .next() method resumes execution first at the beginning, running
up to the first yield, where execution is suspended and the yield value returned to
the next() method caller. A subsequent .next() call resumes execution right after the
last yield, and so on forever or until the generator exits without hitting a yield,
which terminates the sequence.

 >>> def lazyprimes(prime_range_top):
 ...     import array
 ...     primes = array.array('l',[2])
 ...     yield 2
 ...     for prime_candidate in xrange(3,prime_range_top,2):
 ...         for p in primes:
 ...             if prime_candidate%p==0: break
 ...         else:
 ...             primes.append(prime_candidate)
 ...             yield prime_candidate
 ...
 >>> primegen = lazyprimes(1000) # primes under 1000
 >>> primegen.next()
 2

Properties allow you to create a class whose instances have a property attribute that
is accessed just like an ordinary attribute, but may invoke arbitrary functions to get
and/or set the apparent state. E.g.,

 >>> class Foo(object):
 ...     def __init__(self, k, ptop):
 ...         self.k = k
 ...         self.pnext = lazyprimes(ptop).next
 ...     p = property(lambda self: self.pnext())
 ...
 >>> foo = Foo(123, 32)

Here k is an ordinary instance attribute and p looks like another in the syntax of the
access in an expression, but it is hooked into a lazy prime generator:

 >>> foo.k
 123
 >>> foo.p
 2
 >>> foo.p, foo.p
 (3, 5)
 >>> foo.k
 123
 >>> [(foo.k, foo.p, c) for c in 'abcd']
 [(123, 7, 'a'), (123, 11, 'b'), (123, 13, 'c'), (123, 17, 'd')]
 >>> foo.pnext()
 19
 >>> foo.pnext()
 23
 >>> foo.p
 29
 >>> foo.p
 31
 >>> foo.p
 Traceback (most recent call last):
   File "<stdin>", line 1, in ?
   File "<stdin>", line 5, in <lambda>
 StopIteration

Well, I should have provided for dealing with the end-of-sequence exception, most likely,
unless reaching it was an error. Not hard.

Getting back to primegen, which last yielded the first prime 2 above,
I'll bind a shorter name to the next method (bound to the particular generator),
for easier typing ;-)

 >>> pn = primegen.next
 >>> pn()
 3
Here we'll enumerate
 >>> for i,p in enumerate(iter(pn,31)): print i,p
 ...
 0 5
 1 7
 2 11
 3 13
 4 17
 5 19
 6 23
 7 29
 >>> pn()  # we used up 31 as the sentinel ending a series of calls to pn() so,
 37
 >>> # here we got the next one after 31
 ...
 >>> for i in xrange(10): print pn(), # 10 more
 ...
 41 43 47 53 59 61 67 71 73 79
 >>> pn() #one more
 83
 >>> primegen.next()
 89
 >>> for p in primegen: print p, # the rest
 ...
 97 101 103 107 109 113 127 131 137 139 149 151 157 163 167 173 179 181 191 193 197 199 211 223 2
 27 229 233 239 241 251 257 263 269 271 277 281 283 293 307 311 313 317 331 337 347 349 353 359 3
 67 373 379 383 389 397 401 409 419 421 431 433 439 443 449 457 461 463 467 479 487 491 499 503 5
 09 521 523 541 547 557 563 569 571 577 587 593 599 601 607 613 617 619 631 641 643 647 653 659 6
 61 673 677 683 691 701 709 719 727 733 739 743 751 757 761 769 773 787 797 809 811 821 823 827 8
 29 839 853 857 859 863 877 881 883 887 907 911 919 929 937 941 947 953 967 971 977 983 991 997
 >>>

(In opportune line wrap on my console screen, but I assume it's clear enough). 

>
>> >  implement declarative control structures, etc.)  
>> You mean like case or such? 
>
>No. I was thinking about Prolog and such.  Or nondeterministic
>programming.  Or multimethod dispatch.
For the latter in Python, see
    http://www-106.ibm.com/developerworks/linux/library/l-pydisp.html
>
>> >2.) Serve as abbreviation of repeating code.  Ever used a code
>> >  generator?  Discovered there was a bug in the generated code?  Had
>> >  to fix it at a zillion places?
>> >  => Macros serve as extremely flexible code generators, and there
>> >  is only one place to fix a bug.
>> >  => Many Design Patterns can be implemented as macros, allowing you
>> >  to have them explicitly in your code.  This makes for better 
>> >  documentation and maintainability.
>> You can generate code many ways in Python. What use case are you thinking of?
>
>I was not talking about generating code /in/ Python but generating
>code /for/ Python /within/ it.  For the Design Pattern use case take a
>look at http://norvig.com/design-patterns/
I  was also talking about "generating codef /for/ Python /within/ it" -- even in
a recrusive routine, see answer to 4. below ;-)

>
>> >3.) Invent pleasant syntax in limited domains.
>> >  => Some people don't like Lips' prefix syntax.  It's changeable if you 
>> >  have macros.
>> >  => This feature can also be misused.
>> You can do this also.
>
>You can change Python's syntax?  Easily?
No, I didn't mean that literally, but just as you can create source dynamically
and compile it for subsequent execution (as with the trivial text source template below),
you could define an app-specific mini language and translate and compile it for use later
in the same program run. If you create a class and populate class variables from values
in a config file, are you doing this? I think so, at a trivial level. If you wrote an
import function that could import a subset of scheme and have it dynamically translated
to something that looks like a python module to the python user, have you changed python's
syntax? No. Have you made other syntax available to the python programmer? Yes.

I had an infatuation with scheme, some years back now. I still think she was sweet ;-)

>
>> >4.) Do computations at compile time instead of at runtime.
>> >  => Have heard about template metaprogramming in the C++ world?
>> >  People do a lot to get fast performance by shifting computation 
>> >  to compile time.  Macros do this effortlessly.
>> This also, but Python has so many possible compile times ;-)
>
>I'm not sure I understand.

 >>> import time
 >>> def multicomp(maxrec=5, level=0):
 ...     fooname = 'foo_level_%s'%level
 ...     source = """
 ... def %s(): print 'I was compiled at level %s on %s.'
 ... """% (fooname, level, time.ctime())
 ...     d={}
 ...     exec source in d
 ...     time.sleep(2) # to guarantee time stamp change
 ...     if level<maxrec: return (d[fooname],)+ multicomp(maxrec,level+1)
 ...     return (d[fooname],)
 ...
 >>> mc = multicomp()
 >>> for f in mc: print 'My name is %r and'%(f.__name__,),; f()
 ...
 My name is 'foo_level_0' and I was compiled at level 0 on Tue Oct 07 21:19:18 2003.
 My name is 'foo_level_1' and I was compiled at level 1 on Tue Oct 07 21:19:20 2003.
 My name is 'foo_level_2' and I was compiled at level 2 on Tue Oct 07 21:19:22 2003.
 My name is 'foo_level_3' and I was compiled at level 3 on Tue Oct 07 21:19:24 2003.
 My name is 'foo_level_4' and I was compiled at level 4 on Tue Oct 07 21:19:26 2003.
 My name is 'foo_level_5' and I was compiled at level 5 on Tue Oct 07 21:19:28 2003.
 >>>

Ok, the recursion was gratuitous, except that it shows compilation happening dynamically,
and you can easily see you could leave such routines compiled at the outer level for
execution any time you wanted, and thus get "many compile times" ;-)

>
>> Python is pretty sharp ;-)
>> I think we need some realistic use cases for your "specific" [categories of]
>> examples in order to compare how problems would be approached.
>
>Well, if you don't want to learn about syntactic abstraction you'll
>probably never miss it during your programming.  Just keep in mind
>that before oo-abstraction became fashionable people didn't miss OOP
>either.

Actually, I would like to learn more about it. I am fascinated by the
interplay between the worlds of concrete representations and abstract entities,
and their interrelated transformations. ISTM macros definitely have a place in the pantheon.
I have yet to grok scheme's hygienic macro stuff, though ;-) One of these days...

Regards,
Bengt Richter