Python's "only one way to do it" philosophy isn't good?

[Graham Breed]

Douglas Alan wote:
> Graham Breed <x31equsenet at gmail.com> writes:
>
> > Another way is to decorate functions with their local variables:
>
> >>>> from strict import my
> >>>> @my("item")
> > ... def f(x=1, y=2.5, z=[1,2,4]):
> > ...     x = float(x)
> > ...     w = float(y)
> > ...     return [item+x-y for item in z]
>
> Well, I suppose that's a bit better than the previous suggestion, but
> (1) it breaks the style rule of not declaring variables until you need
> them, and (2) it doesn't catch double initialization.

(1) is a style rule that many style guides explicitly violate.  What
is (2) and why would it be a problem?

A better way that I think is fine syntactically would be

from strict import norebind, set
@norebind
def f(x=1, y=2.5, z=[1.2.4]):
    set(x=float(x))
    set(w=float(y))
    return [item+x-y for item in z]

It won't work because the Python semantics don't allow a function to
alter a nested namespace.  Or for a decorator to get at the locals of
the function it's decorating.  It's an example of Python restricting
flexibility, certainly.

> > The best way to catch false rebindings is to stick a comment with
> > the word "rebound" after every statement where you think you're
> > rebinding a variable.
>
> No, the best way to catch false rebindings is to have the computers
> catch such errors for you.  That's what you pay them for.

How does the computer know which rebindings are false unless you tell
it?

> > Then you can search your code for cases where there's a "rebound"
> > comment but no rebinding.
>
> And how do I easily do that?  And how do I know if I even need to in
> the face of sometimes subtle bugs?

In UNIX, you do it by putting this line in a batch file:

egrep -H 'rebound' $* | egrep -v '^[^:]+:[[:space:]]*([.[:alnum:]]+)
[[:space:]]*=(|.*[^."])\<\1\>'

You don't know you need to do it, of course.  Like you wouldn't know
you needed to use the let and set macros if that were possible.
Automated checks are only useful for problems you know you might have.

> > Assuming you're the kind of person who knows that false rebindings
> > can lead to perplexing bugs, but doesn't check apparent rebindings
> > in a paranoid way every time a perplexing bug comes up, anyway.
> > (They aren't that common in modern python code, after all.)
>
> They're not that uncommon, either.

The 300-odd line file I happened to have open had no examples of the
form x = f(x).  There was one rebinding of an argument, such as:

if something is None:
    something = default_value

but that's not the case you were worried about.  If you've decided it
does worry you after all there may be a decorator/function pattern
that can check that no new variables have been declared up to a
certain point.

I also checked a 400-odd file which has one rebinding that the search
caught.  And also this line:

            m, n = n, m%n

which isn't of the form I was searching for.  Neither would the set()
solution above be valid, or the substitution below.  I'm sure it can
be done with regular expressions, but they'd get complicated.  The
best way would be to use a parser, but unfortunately I don't
understand the current Python grammar for assignments.  I'd certainly
be interested to see how your proposed macros would handle this kind
of thing.

This is important because the Python syntax is complicated enough that
you have to be careful playing around with it.  Getting macros to work
the way you want with results acceptable to the general community
looks like a huge viper pit to me.  That may be why you're being so
vague about the implementation, and why no macro advocates have
managed to get a PEP together.  A preprocessor that can read in
modified Python syntax and output some form of real Python might do
what you want.  It's something you could work on as a third-party
extension and it should be able to do anything macros can.


That aside, the short code sample I give below does have a rebinding
of exactly the form you were worried about.  It's still idiomatic for
text substitutions and so code with a lot of text substitutions will
likely have a lot of rebindings.  You could give each substituted text
a different name.  I think that makes some sense because if you're
changing the text you should give it a name to reflect the changes.
But it's still error prone: you might use the wrong (valid) name
subsequently.  Better is to check for unused variables.

> I've certainly had it happen to me on several occasions, and sometimes
> they've been hard to find as I might not even see the mispeling even
> if I read the code 20 times.

With vim, all you have to do is go to the relevant line and type ^* to
check that the two names are really the same.  I see you use Emacs but
I'm sure that has an equivalent.

> (Like the time I spent all day trying to figure out why my assembly
> code wasn't working when I was a student and finally I decided to ask
> the TA for help, and while talking him through my code so that he
> could tell me what I was doing wrong, I finally noticed the "rO" where
> there was supposed to be an "r0".  It's amazing how useful a TA can
> be, while doing nothing at all!)

Assembly then, not Python.

> > And you're also the kind of person who's troubled by perplexing bugs
> > but doesn't run a fully fledged lint.
>
> Maybe PyLint is better than Lint for C was (hated it!), but my idea of
> RAD does not include wading through piles of useless warning messages
> looking for the needle warning in the warning haystack.  Or running
> any other programs in the midst of my code, run, code, run, ..., loop.

Well, that's the choice you have I'm afraid.  Either the computer
nannies you about spurious bugs in your code or you miss genuine
bugs.   I don't see why it makes a difference for an external tool to
do the nannying.  Checking for unused variables is the obvious way to
check for spelling mistakes in Python.

> > Maybe that's the kind of person who wouldn't put up with anything
> > short of a macro as in the original proposal.  All I know is that
> > it's the kind of person I don't want to second guess.
>
> As it is, I code in Python the way that a normal Python programmer
> would, and when I have a bug, I track it down through sometimes
> painstaking debugging as a normal Python programmer would.  Just as
> any other normal Python programmer, I would not use the alternatives
> suggested so far, as I'd find them cumbersome and inelegant.  I'd
> prefer not to have been bit by the bugs to begin with.  Consequently,
> I'd use let and set statements, if they were provided (or if I could
> implement them), just as I have the equivalents to let and set in
> every other programming language that I commonly program in other than
> Python.

Except that, unlike a normal Python programmer, you don't want to use
Python syntax.

I have, out of interest, written a Python script to convert from a
Python-like language with let and set into real Python.  It's really
not that difficult.  It's full of holes, of course, but I'm not
convinced that macros would be any different.  Here it is:

import sys, re, os

general_match = r"(?m)^(\s*)%s(\s+)(\w+)(\s*=)"
general_insert = r"\1assert '\3' %s in locals()\n\1\3\4"

let_spec = general_match % "let", general_insert % "not"
set_spec = general_match % "set", general_insert % ""
escape_spec = general_match % r"\\([ls]et)", r"\1\2\3\4\5"

inputfile = sys.argv[1]
outputfile = os.path.splitext(inputfile)[0] + os.path.extsep + 'py'

code = open(inputfile).read()

for match, insert in let_spec, set_spec, escape_spec:
    code = re.sub(match, insert, code) # rebound

open(outputfile, 'w').write(code)


                         Graham