[Tutor] Text processing and functional programming?

[Clay Shirky]

> (If we're sadistic, we can even reimplement map() by using list
> comprehensions!
> 
> ###
> def map(function, sequence):
>   return [function(x) for x in sequence]
> ###
> )

But why not just do this in-line, as [ function(x) for x in sequence ] ?
What does the def and return buy you?
 
> If you've seen other FP languages, the above won't look too weird.  But if
> your experience is with something like Pascal, then the above will look
> very weird.

Its perl, actually, that I'm reacting to, where map is a key component of
much unreadable code.
 
> List comprehensions can take in an arbitrary expression, so we can also do
> it this way:
[...]
> In contrast, map() only deals with function objects, and not arbitrary
> expressions.  

So this gets at a question I put badly before: If list comprehensions can
take arbitrary expressions, and can pass any collection of values through
those functions, is the list comprehension construct a superset of what map,
filter, and lambda can do?

Put another way, other than preference for different flavors of syntactic
sugar, would you ever prefer map/filter/lambda to list comprehensions, or
v-v? 

And, relatedly, are list comprehensions FP constructs, or does hiding the
details of the (possibly nested) loops not create the same value as applying
functional equivalents of those loops?

> That's what lambda's used for, to allow us to wrap an arbitrary expression
> as a function.  Nothing fancier than that.  It might have been nice to
> avoid the intimidating keyword "lambda", but oh well.  *grin*

Someone on the python-dev list suggested it should have been called 'anon'.

> Since Python has list comprehensions as part of the language core now,
> it's often preferable to use them instead of the traditional map()
> builtin: most programmers will feel comfortable with list comprehensions
> because the syntax isn't that daunting.

Ah. Thanks. This answers my question above.
 
> def take_all_lengths(L):
>   return [len(x) for x in L]
> 
> def mass_url_reader(L):
>   return [urllib.urlopen(x) for x in L]
> ###
> 
> So it's not hard, but it's just a bit more work.

But, but, but now I'm confused again.

>>>> def take_all_lengths(L):
....    return [ len(x) for x in L ]
>>>> print take_all_lengths( [ 'This', 'is', 'a', 'test' ] )

is the same as
    
>>>> print [ len(x) for x in [ 'This', 'is', 'a', 'test' ] ]

so why do the bit more work in the first place? Right now, it just looks
like another layer of indirection to me.

Is it just that these are examples, and at some point you want to write a
def that performs so many functions that you don't want to write it in-line?
And if so, doesn't this push against the FP idea of narrowly defined
functions?

I believe Alan when he says the studies find that FP leads to more
performant code, but I still don't really understand why you would chose a
map/filter/lambda construction over a list comprehension, or v-v.

Are there times when you can do something with one of those constructs but
not the other?

-clay