Stream programming

Kiuhnm kiuhnm03.4t.yahoo.it
Fri Mar 23 12:00:23 EDT 2012


I've been writing a little library for handling streams as an excuse for 
doing a little OOP with Python.

I don't share some of the views on readability expressed on this ng. 
Indeed, I believe that a piece of code may very well start as complete 
gibberish and become a pleasure to read after some additional 
information is provided.

I must say that imposed indentation is a pain when one is trying to 
define some sort of DSL (domain specific language). Also, Python's 
operator overloading is a bit limited, but that makes for a more 
rewarding experience in my case.

Here's an example of what you can write:

numbers - push - avrg - 'med' - pop - filter(lt('med'), ge('med'))\
     - ['same', 'same'] - streams(cat) - 'same'

Ok, we're at the "complete gibberish" phase.

Time to give you the "additional information".

I will use "<=>" to mean "is equivalent to". That's not part of the DSL.
A flow has one or more streams:
   1 stream:
     [1,2,3]
   2 streams:
     [1,3,5] | [2,4,6]
Two flows can be concatenated:
   [1,2,3] + [4,5,6] <=> [1,2,3,4,5,6]
   [0] + ([1,2] | [3,4]) + [10] <=> [0,1,2,10] | [0,3,4,10]
   ([1,2] | [10,20]) + ([3,4] | [30,40]) <=> [1,2,3,4] | [10,20,30,40]
A flow can be transformed:
   [1,2] - f <=> [f(1),f(2)]
   ([1,2] | [3,4]) - f <=> [f(1,3),f(2,4)]
   ([1,2] | [3,4]) - [f] <=> [f(1),f(2)] | [f(3),f(4)]
   ([1,2] | [3,4]) - [f,g] <=> [f(1),f(2)] | [g(3),g(4)]
   [1,2] - [f,g] <=> [f(1),f(2)] | [g(1),g(2)]
Some functions are special and almost any function can be made special:
   [1,2,3,4,5] - filter(isprime) <=> [2,3,5]
   [[],(1,2),[3,4,5]] - flatten <=> [1,2,3,4,5]
Note that 'filter' is not really necessary, thanks to 'flatten'.
Flows can be named, remembered and used
   as a value:
     [1,2,3,4,5] - 'flow' + val('flow') <=> [1,2,3,4,5]*2
   as a transformation chain:
     [1,2,3] - skipfirst - 'again' | [4,5,6] - func('again')
       <=> [2,3] | [5,6]
   Recursion is also possible and stops when a function is applied to an 
empty sequence.
Flows can be saved (push) and restored (pop) :
   [1,2,3,4] - push - by(2) - 'double' - pop | val('double')
       <=> [1,2,3,4] | [2,4,6,8]
There are easier ways to achieve the same result, of course:
   [1,2,3,4] - [id, by(2)]

Let's go back to our example. I didn't tell you anything but you should 
be able to understand it anyway.

numbers - push - avrg - 'med' - pop - filter(lt('med'), ge('med'))\
     - ['same', 'same'] - streams(cat) - 'same'

It reads as

"take a list of numbers - save it - compute the average and named it 
'med' - restore the flow - create two streams which have, respect., the 
numbers less than 'med' and those greater or equal to 'med' - do the 
/entire/ 'same' process on each one of the two streams - concat the 
resulting streams - name all this /entire/ process 'same'.
Not readable enough? Replace 'same' with 'qsort'.

Is that readable or am I going crazy? [note: that's a rhetorical 
question whose answer is "That's very readable!"]

Kiuhnm



More information about the Python-list mailing list