[Python-ideas] Statements vs Expressions... why?

[Josiah Carlson]

On Wed, Sep 10, 2008 at 9:34 PM, Cliff Wells <cliff at develix.com> wrote:
> On Wed, 2008-09-10 at 18:30 -0700, Josiah Carlson wrote:
>> On Wed, Sep 10, 2008 at 6:14 PM, Cliff Wells <cliff at develix.com> wrote:
>> > On Wed, 2008-09-10 at 17:16 -0600, Adam Olsen wrote:
>> [snip]
>> >> They're the use-case you need to justify
>> >> against the substantial changes you propose.
>> >
>> > I guess I don't see it as substantial to people who don't wish to use it
>> > (although it's quite substantial to people who do).  Overall, I think
>> > this is why I feel the change doesn't require a huge amount of
>> > justification: you aren't *forced* to use it, but if you need it, it's
>> > huge.  It doesn't impose any significant stylistic change on people who
>> > prefer the imperative style, but it opens vast doors for people wishing
>> > to approach problems from a functional path.
>>
>> Once language syntax is added to, changed, etc., it's very difficult
>> to remove those additions, changes, etc., even when the feature is
>> rarely used, ugly, and generally a bad idea (see back-quotes `x` for
>> repr(x) ).  This may not seem like a big deal to you, because you want
>> this feature, but for the rest of us who have little (arguably no) use
>> for the feature, adding semantics to syntax, or adding syntax =
>> additional mental overhead;
>
> Again I assert the opposite: that Python is currently forced to explain
> the arbitrary distinction that currently exists and that a significant
> amount of extra syntax and language features exist to work around that
> distinction.  People may not clamor for the change but there's quite a
> few newcomers to Python who must have the distinction explained to them.

For people who have gone through the Python tutorial or any one of the
many good 'learning Python' texts (online, paper, free, and paid-for)
the distinction has already been explained.  For those who refuse (for
some reason) to go through one of those resources, that's not an issue
with Python, that's an issue with the user.

As a data point; everyone that I've taught Python over the years
(undergrads and interns who only ever used Java before to 30-year
programming veterans of apl/cobol/fortran), not a single one of them
has ever had a problem with the difference between statements and
expressions.  It is possible that I've only ever worked with gifted
students and coworkers, though this is unlikely.

>> never mind the additional sections in the
>> tutorial where we have to explain why this *particular* special case
>> of a DSL was special enough to break the rules of explicit is better
>> than implicit (why should a multi-line if statement implicitly return
>> something in one place but not in another?)
>
> I respect your work and knowledge of Python, but that's insufficient to
> get me to respect a straw-man argument.  I've not suggested that Python
> add anything for any particular situation.  I'm suggesting a change that
> would make it more logically consistent, expressive, and help reduce the
> language and syntax clutter it's been accruing at a steady pace.  The
> fact that it also helps in a rather broad range of problems is ancillary
> and my particular use case was presented as an example, not as *the*
> problem to be solved.

What I've found (historically) is that any time I find myself trying
to do something that is not available within the syntax of Python,
it's usually because it's a bad idea.  In your case, you want (for
example) the following to be valid Python...

lfcn = lambda x: (
    if x:
        True
    else:
        False)

Similarly...

lfcn = lambda x: (
    for i in xrange(x):
        yield fcn(i, x, ofcn(i))
    )

Why?  Initially it is for a templating language that you have designed
that makes these kinds of things difficult to do during an automatic
translation from working Python source code into compiled source code.
 Presumably something of the form...

html [ body [ a (href='http://python.org') ["Python!"] ] ]

To be converted into something like...

def generate_html(out):
    out.write('''<html>
  <body>
    <a href="http://python.org">Python!</a>
  </body>
</html>''')

Trust me, I understand your desire.  I have used Cheetah and Spitfire
(http://code.google.com/p/spitfire/), and the speedups gained by
translating templates into Python source are not to be laughed at.
And before I used those, I saw nevow.stan (from which your syntax is
effectively identical to), and wrote my own variant
(http://code.activestate.com/recipes/440563/ - I use multi-calling
semantics rather than __getitem__ stuff).  The only reason I didn't
write a compiler (and run into the same issues you have) was because I
didn't care about dynamic content generation, I was using it for
pre-generation of static web pages.

But I digress.  At some point, you acknowledge that your templating
language is constrained by your programming language of choice (as is
the case for systems that use the base programming language for
syntax, because of the convenience of parsing), or really, your
templating language is constrained because you refuse to write your
own parser.  Don't get me wrong, the convenience of writing a
templating language in Python syntax is amazing...but you've noticed
that your compilation steps are hampered by Python's immediate
execution of code.  In fact, your own renderers are a work-around for
something that is /relatively/ natural in Cheetah/Spitfire:

    $for $link in $links:
        <a href="$link.url">$link.label</a>

Which is generally compiled into the following (with a bit of extra
garbage, but you get the idea)...

    for link in links:
        out.write('''<a href="%s">%s</a>'''%(link.url, link.label))

Now, I loathe writing Cheetah/Spitfire because I loathe writing html,
which is why I wrote my own templating system, but there are
work-arounds.  For example, if you want to stick with Python syntax...

For("$link", "$links", "<a href="$link.url">$link.label</a")

Now you just need to write your For factory and flattener, which
produces the right code (easy), and with a little work, you can even
make for loops composable, add ifs, etc.  I know, it's not as elegant
(for you) in this situation as statements becoming expressions, but it
will work today. (note that pie-in-the-sky best-case scenario is 18+
months for Python 3.1, but more likely 15 years for Python 4 ;) )

> Also, I don't know where you got the idea that I suggest any expression
> should return a value sometimes but not others.  I'm suggesting they
> *always* return values, but that you don't always need to worry about
> them unless you plan to use the value in another expression.  Kind of
> how functions already work in Python.

You are right, I misread an earlier email in this thread.

>> I know, "it's just one little change".  I've made the argument myself.
>>  But that doesn't mean that my idea was a good idea (it wasn't), nor
>> does it mean that your current idea is (I think everyone in this
>> thread except you would agree that it's a bad idea).
>
> Sure.  Because everyone who might have agreed with me has already
> migrated to Ruby or started a new language (Logix, Boo, et al).  I just
> happen to be lazy enough to entertain the thin hope that Python could be
> fixed so I don't have to learn a new language or programming
> environment ;-)

That's not necessary.  You've already solved your problem with
renderers, and I've pointed out another method for getting some
behavior in-line (If you are careful, you could probably build all of
Python out of constructs similar to the For above).  Of course there's
always the option of just using some version of the output of the
compiler/ast (I have done as much to pull out class definitions,
function definitions, etc., in the editor that I have written).  You
could even write a tool for doing the conversion ;) .

>> Before continuing on in defending your proposal here, I suggest you
>> try comp.lang.python .  If you can't get a dozen people to agree that
>> it's a good idea, or if (like here) the only replies are negative (or
>> even if the majority of them are), then I would suggest you drop it.
>
> Actually I avoided c.l.py because I saw this list suggested as the
> appropriate place for new or controversial ideas (and the list name
> seemed to suggest the same).
>
> The fact that lambda was so bitterly defended against GvR's strong
> desire to remove it should be hint enough that a significant number of
> Pythonistas are interested in functional and expression-oriented
> programming.

Indeed!  I have a penchant for the use of list comprehensions,
generator expressions, map, etc. - when they are reasonably
applicable.  The question is always: what is reasonably applicable.

> In any case, I actually got the response I expected and ultimately I
> expect the discussion here was probably far more enlightened than I
> would expect on c.l.py.
>
> It seems most Pythoneers work in particular domains where this isn't an
> issue, are satisfied with the workarounds, or simply are unable to see
> there's a vast world of elegant solutions beyond what imperative
> languages can describe.  Unfortunately this only confirms my (rather
> reluctant) expectation that I'll be forced to move to a more expressive
> language to satisfy my programming itches.

Pythoneers do see that there are /alternate/ solutions to what is
currently available within Python, hence why people tend to like to
write parsers in Python ;) .  One thing you may want to look into is
that you aren't using the full expressive power of Python in your
templating language, and because of this, it is actually quite easy to
write a parser for your subset, adding all of the necessary behavior
and translation.  I used DParser in Python for a few years and loved
it, though it seems like the home page for it is no longer available.

On the one hand, yes, you are currently constrained by the limits of
Python's syntax.  And yes, you could move to Ruby, Boo, etc.  Or you
could try alternate constructs (For, If, Def, etc.), write your own
parser (which isn't hard), or shrug and admit to yourself that the
limitations really aren't all that bad.  Inconvenient, sure.  The end
of breve in Python?  Not necessarily.

 - Josiah