"def" vs "sub" (was RE: More random python observations from a perl programmer)

[Tim Peters]

[posted & mailed]

[tchrist, discovers that Python "def" is executable, and notes that
 this is a gotcha for Perl programmers]

[Tim, notes that Perl's "sub" is equally a gotcha for Python programmers]

[back to tchrist]
> You didn't localize your function!  Remember, in Perl things
> are only local if you say they are.

But it doesn't matter, Tom <0.5 wink>:  the "gotchas" you identified in
Python are things that would throw a Perl programmer if they hadn't read and
understood the Python docs first.  Perl is at least equally as surprising to
the Python programmer who hasn't read and understood the Perl docs first (as
the self-proclaimed Pythoneers who flame on c.l.p.m. prove more than often
enough).

[and where I had
    sub inner sub { ... },
 do instead]
> ...
> That could have been like this:
>
>     sub outer1 {
>         my $x = 50;
>         local *inner = sub {
>             print "in the first inner, and here's x: $x\n";
>         };
>         &inner();
>    }

No problem, but that's (symbol table globbing) a *real* stretch for someone
just learning the language.  There are real barriers for native speakers of
either language in picking up the other.

I ended up doing something like this instead:

sub makefac {
    my $fac;
    $fac = sub {
        my $n = shift;
        return 1 if $n < 2;
        return $n * &$fac($n - 1);
    };
    return $fac;
}

my $i;
for ($i = 0; $i < 1000000; ++$i) {
    my $fac = makefac;
    my $n = &$fac(3);
    print "$n ";
}

This one is more interesting than it looks <wink>:  in the last version of
Perl I tried it on, it leaked memory at a prodigious rate.  The reason it's
interesting is because it's faking pretty much what Python would have to do
to support lexical closures, given that a nested "def" acts exactly like an
assignment statement binding a name local to its containing function (and
here an anonymous sub is assigned to a lexical local of *its* containing
function).

In Python and in Perl (at least in the Perl I last tried that under) that
creates cyclic garbage:  makefac's lexicals have to contain a binding for
fac, and fac has to point back to makefac's lexicals in order to resolve
up-level references.  And that's (unrecoverable trash) the reason Python
doesn't have lexical closures today.  If a nested def were-- as my original
nested "sub inner" is in Perl --treated as global, that problem would go
away, and for the same reason it does in Perl.  But it wouldn't be Python
anymore either <wink>.

> ...
> BTW, you should have gotten a bucket of frightened warnings out of
> the compiler about that.

I got strong indications of trouble with -w, yes.

[examples of great msgs produced by
     perl -wc
 and
     perl -Mdiagnostics -wc
]

> ...
> BTW, see what I mean about the importance of good error messages?

No argument from me that Perl's msgs are often very helpful, much more often
pinpointing a problem than Python's (the "runaway multi-line string starting
at line nnn" is one Python should steal!).  On the other hand, I've written
a few Klines of production Perl too, and my experience is that Perl has so
many more syntactic and semantic gotchas than Python that the language would
be nearly unusable without great msgs.  Python's NameError may drive you
nuts, but it's the same as Perl's runtime "use of uninitialized value" --
and Perl is no more help in tracking down the *cause* of those than is
Python in cracking a NameError.  At least Perl's -w warns about names that
appear only once!  You have to grab a copy of PyLint for that (but then you
are a fan of separate tools, right <wink>?).

[various ways to promote warning into errors, "if you feel all
 warnings should kill you"]

Whereas Python has no warnings of any kind -- it it's irked enough at me to
complain at all, it's irked enough to kill me.  In that it's a faithful
reflection of Guido's personality <wink>.  Seriously, I do prefer that.

> ...
> You know, it occurs to me that it might be nice to be able to
> selectively promote only compile-time warnings to fatals.

As a native speaker of Python, -w and "use strict" decorate all my Perl, and
in  the few Official Perl projects I've been on "NO WARNINGS!" was an
absolute rule.  The more fatals the better.

> Perl does a lot of compile-time stuff (including constant
> expression folding, inlining of some subroutines, and folding
> of those results!).
>
> What does Python do at compile-time?

It's too busy desperately scrounging for excuses to kill your program with a
bogus SyntaxError to do anything useful <wink>.  About the only compile-time
optimization is that function locals (in the absence of "exec" and "import
*" statements) are mapped to consecutive indices into a contiguous runtime
vector.

"exec" and "import *" can alter the *set* of local names, so their presence
inhibits that optimization.  That's why "exec" is a stmt rather than a
function:  since no builtin names are resolved at compile-time, and any name
may be bound to any object at runtime, if exec were a function Python
couldn't know that

    exec(s)

does refer to the builtin "exec", or that

    f(s)

*doesn't*, so the local-var optimization would be crippled almost always.
So "exec" is a keyword, the only kind of word whose meaning is known for
sure at compile-time.

Python's default arguments are sometimes used to fake lexical closures, but
they're also used to fake the effect of compile-time binding; e.g., because
def *is* executable, when Python (at runtime) gets to

def somefunc(x, int=int):
    ... = int(...)

the local name "int" gets bound to whatever-the-heck the global name "int"
is bound to at the time the def is executed.  Of course it *is* bound to the
builtin int function, but Python can't know that at compile-time.  This
trick lets the programmer force the issue, and within somefunc the "int"
lookup is very fast (just a C indexed array ref); without this trick each
dynamic instance of "int" in somefunc does a failing lookup in the global
dict first, and then a successful lookup in the builtin dict.

So, no matter what anyone here tells you, Perl doesn't have a monopoly on
being cute & repulsive at the same time <0.8 wink>.

takes-one-to-know-one-ly y'rs  - tim