variable declaration

[Alex Martelli]

Michael Tobis <mt at 3planes.com> wrote:

> Alex Martelli wrote:
> > Michael Tobis <mt at 3planes.com> wrote:
> 
> > he can perfectly
> > well correct his misexpression if he cares to -- not my job to try to
> > read his mind and perform exegesis on his words.
> 
> Well, I hate to try to tell you your job, but it doesn't seem to be to
> be all that great of a marketing strategy to actively chase people
> away... Hey, he might have been a Nutshell customer.

I'd far rather sell one fewer copy of the Nutshell, than sell one more
to somebody who is not _happy_ to use Python.


> > I think it would have been true, but weak and insufficient.  Not only
> > experienced Python users have that opinion: lack of declarations
> didn't
> > faze me even I was a total newbie
> 
> It did me, and it did many others. Perhaps you are unrepresentative.

Maybe; I did have good knowledge of a variety of languages, for example.
However, I have seen many people come upon ideas that were new to them
and meet them with interest and curiosity, even if initially doubtful
about the novelty, rather than with fear and loathing.  I think that
such an attitude is a better predictor of how happy a person will be
with Python (or other technologies he's initially unfamiliar with) than
"previously accumulated knowledge".


> It's one thing to say "no can do, sorry", it's another to say "you
> don't need this anyway and if you think you do you aren't worthy".

To say one is sorry about something that, in fact, makes one deliriously
happy, would be the worst sort of hypocrisy, even though it's a socially
common ``white lie''.  As for "worthy", that's a completely different
viewpoint, and I would appreciate it if you didn't put words into my
mouth.

Believing that somebody might be unhappy using Python (or any other
given technology), at least at this point in their personal development,
due to their ingrained mindset and strongly held opinions against one of
the cornerstones of the language, has nothing to do with "worth".


> In fact, it was your book I spent the most time thumbing through
> looking for the "use strict" equivalent that I was absolutely certain
> must exist. Hell, even Fortran eventually gave in to "IMPLICIT NONE".

...which has nothing to do with the case, since in Fortran, and from day
one, all names had statically determinable types anyway.

> It's practically the only thing I've ever expected to find in Python
> that hasn't vastly exceeded my expectations, aand I'm sure Alexander is
> not the only person to be put off by it.

By Hugol's Law, surely not.  So what?  If he sticks to his misguided
expectations, he won't be happy with Python.  If he outgrows them, he
probably will.  But the best way forwards is to get used to unit-testing
and thereby realize declarations are deadweight, THEN use a language
where they just aren't there.

> In fact, I'd recommend a
> paragraph early in the Nutshell book saying "there are no declarations,

What about a paragraph asking the reader to *READ* the book before
offering advice about it?  On p.32, "Python has no declarations"; on
p.39, under Variables, right at the start of the paragraph, "In Python,
there are no declarations.  The existence of a variable depends on a
statement that <i>binds</i> the variable, or, in other words, that sets
a name to hold a reference to some object".

Not only are these words pretty early, right towards the start of the
chapter giving an overview on the language, but I think the second
snippet has quite a prominent position.  If you're looking for a way to
declare variables, and don't think of looking at the very start of the
section titled "Variables" -- or if the words you find there,
reinforcing the previous assertion to the same effect, still keep you
"thumbing through" the book in search of what I just said isn't there, I
disclaim responsibility.

A book, particularly a quick reference, needs to be considered as a
_cooperative_ effort between writer and reader.  I can fairly be tasked
with expressing every important thing about the language, and I think I
do -- not all of the "sea lawyer"-level quibbles, but all of the aspects
most readers truly need.  But I cannot fairly be asked to *belabor*
every aspect that might faze some reader or other, to ward against the
reader not being careful and not realizing that, in a work where
conciseness is of the essence, every single word is there for a purpose.

If you need repetition and belaboring, get a book that's intended as
slow-paced tutorial.  Demanding tutorial-like repetitiousness of a
*quick reference* is absurd and irresponsible.

> no use strict, no implicit none, sorry, forget it", and an index

This is the second time you ask or suggest that I take an apologetic
attitude (or at least mouth platitudes that sound apologetic without
_meaning_ to be apologetic in the least), and I repeat: forget it.

> listing under "declarations" pointing to a detailed exegesis of their
> nonexistence. It would have saved me some time.

...and would have cost time to careful readers, ones who know that "no
declarations" means (surprise!) *NO* declarations, and are sensible
enough to NOT expect "detailed exegesis" in a *quick reference* work.

No way -- and I'm not apologetic in the least about it, please note.

Having a tightly limited space budget, I carefully allocate it to
materials that I think it will do most good to most readers in the main
target audience: Python programmers.  "I'm not trying to teach Python
here", as I say right at the start of that chapter (it's chapter 4, and
I believe it's just the sample chapter O'Reilly chose to place on their
site, so readers of this thread who don't own the book should be able to
have a look and follow this debate more closely) -- it's certainly
_possible_ to learn Python from the book, but only by reading very
closely and carefully, because the book eschews the repetition and
belaboring that a tutorial work would abound in (unless it be very, very
fast paced for a tutorial, which is also a possible stance; I believe
it's the stance taken by the excellent "Dive Into Python").

To you, it's the lack of declarations (because you can't or won't take
assertions about "no declarations" at face value); to another, it's
significant indentation, issues of typing, limits on recursion, argument
passing, the lack of a switch/case statement, -2**2... I mention each
and every one of these issues, of course, but it would be totally
inappropriate to offer for each the "detailed exegesis" you require.


> It's true that in some sense an assignment is all the declaration you
> need. I think Carl Banks's point (what we think of as assignment as a
> carryover from other languages is really rebinding, and in many cases
> can be avoided) is also helpful.

It's known as an "assignment statement" in Python, too, and its
semantics may be ones of binding or rebinding.  "Assignment statements
are the most common way to bind variables and other references", and the
whole following paragraph about rebinding, Nutshell p. 39.

> But that doesn't make the "epselon" bug go away, and wanting to have a
> way to catch it quickly isn't, to my mind, obviously a criminal act.

Of course not, and test-driven design is the right way to catch quickly
that bug, and many others (including but not limited to ones caused by
other typos).

> Also, based on what DogWalker demonstrates, it's really not that alien
> to Python and should be feasible.

I assume you're referring to the abuse of __slots__ that's so
ridiculously popular?  As I explain on p. 86, __slots__ is meant
strictly to let you save memory.  It does NOT work well to catch typos
in real-life code (as opposed to toy-level examples).  Michele Simionato
has a reasonable recipe on ActiveState's Cookbook (that recipe's also
going to be in the Cookbook's 2nd edition, due out in a couple months)
to show how to use __setattr__ instead (much more sensible) if you're so
tremendously "typo-phobic".

But, like the "leading __" idea, weird metaclasses and decorators that
let you omit the explicit 'self', and quite a few others, these nifty
tricks ARE in fact fully alien to Python "as she's spoken" -- they're
quoted, repeated, and varied upon, with ridiculous frequency, totally
out of proportion to their miniscule importance in the normal and
idiomatic practice of the language, because more and more people come to
Python, from a wide variety of other languages, keen to not change their
habits and "keep coding X in Python" for any value of X you might care
to name.

Pythonistas like showing off Python's power, in particular by mimicking
the different ways in which different languages work, and we may well
think that trying to soothe the worries of people coming to Python with
all the wrong expectations and thereby make Python more converts -- I
know I've been guilty of this particular error in the past.  But I do
think it's an error when it gets overdone the way it's being overdone
these days; that to be really happy with Python, one should use Python
to do *PYTHON* programming, NOT to halfway-mimic Java, Visual Basic,
Fortran, PHP, Eiffel, Scheme, Dylan, Javascript, Ada, and Haskell, all
rolled into one.

As Steve Holden just posted, this may well mean that Python is not in
fact suitable for EVERYbody -- only people who are willing to give a try
to Python AS SUCH, rather than feeling a deep-seated need to mimic other
languages they're used to (perhaps due to unwillingness to use important
and excellent methodologies and tools, from unit-tests onwards).  If so,
then, so be it: yeah, even if it means I sell fewer copies of my books,
let those fewer copies be sold to people who'll APPRECIATE AND ENJOY
both the books and the language, rather than pine for ``variable
declarations'', thumb endlessly looking for detailed exegeses of what
ISN'T there, and so on.  I think the total amount of happiness in the
world will be enhanced thereby, and, in the end, that's what matters.


> > > Also, the assertion that "Python has no declarations whatsoever" is
> no
> > > longer obviously true. In the 2.4 decorator syntax, a decorator
> line is
> > > not executable, but rather a modifier to a subsequent symbol
> binding. I
> > > call it a declaration.
> >
> > You may call it a strawberry, if you wish, but that doesn't mean it
> will
> > taste good with fresh cream.  It's nothing more and nothing less than
> an
> > arguably weird syntax for a perfectly executable statement:
> 
> This may well be true in implementation, but cognitively it is a
> declaration that modifies the reference and not the referent. I see
> that it is a big deal to ask for more of these, but I don't see why.

Because the "cognition" is simply and totally wrong and
counterproductive: if you have the wrong mental model of what "splat
foo" means, you won't be productive in coding and using various new
possibilities for ``foo'' within that syntax.

It's not a matter of implementation, but of SEMANTICS -- what the syntax
form actually DOES, quite independent of HOW it does it, is to call a
HOF once the following def statement is done executing, to ``filter''
the function object to be bound to the name.  No more, no less, and
NOTHING to do with ``declarations'', any more than statements such as
def and class are declarations (thinking of them as such is a serious
conceptual error and gravely hampers programming productivity).

I think your total and utter misconception of decorator syntax as being
a "declaration" is the best possible argument against that syntax; I
also think the syntax was probably worth having anyway, despite that.
After all, somebody sufficiently crazed from a withdrawal crisis from
declarations, as you appear to be, may go around calling "declarations"
anything whatsoever, be it splats, def, class, import, assignments, and
so on -- we can't rip all statements out from Python to make it into a
safe padded cell for declaration-junkies where they won't hurt
themselves and others.

I hope one day to forward this exchange to Guido as part of a dossier
against the latest (and neat!) syntax tweak he's considering for Python
3.0 -- having something like:

def f(x: <expression>):
   <body>

be a syntactic shortcut for

def f(x):
    x = <something>(x, <expression>)
    <body>

where the `<something>' might be the ``adapt'' builtin (per PEP 246) or
some variation thereon.  The argument boils down to: many people will
mis-conceptualize this nifty shortcut as a DECLARATION and thereby get
hopelessly confused and utterly misuse it, completely failing to
understand or accept that it's just a syntax shortcut to promote an
important idiom, just like the splat-syntax for decorators.

He'll probably go ahead anyway, of course, just as he did for decorators
despite the huge flamewars -- if he wasn't such a stubborn guy, Python
would be unlikely to be so unique;-).


> > > Let me add that I remain unconvinced that a language cannot combine
> the
> > > best features of Python with very high performance, which is
> ultimately
> >
> > I'm also unconvinced.  Fortunately, so is the EU, so they have
> approved
> > very substantial financing for the pypy project, which aims in good
> part
> > exactly at probing this issue.
> 
> I hope this works out, but it's hard for me to see how pypy will avoid
> lots of hashing through dictionaries. I'm willing to help it by
> declaring an immutable reference. Here, don't look this up; it always
> points to that.

If such crutches are necessary, I guess that will emerge.  The best
compilers for other dynamic languages, such as the stalin compiler for
scheme, can do without such crutches -- I'm sure it's hard for you to
see how, but if you study modern advanced compiler theory (which IS
hard, of course) then you might perhaps stand a chance.

> I'm guessing that this will also be considered a bad idea, and maybe
> someday I'll understand why. I'm looking for insight, not controversy.

I'm not sure there's much insight to be had: you want some forms of
redundancy, which you can get in other languages; we'd rather avoid such
localized redundancy, as the "end-to-end" checks that testing gives us
makes it so much deadweight -- and we NEED testing anyway, even with the
little redundancy aspects, because no such redundancy will catch many
errors, including typos such as a + where a - was meant, a < where a <=
was meant, and so on.

Introducing such redundancy as "optional" soon makes it practically
mandatory for most people in most situations, for many reasons, such as,
because people who manage programming efforts often like restricting
their employees under the misapprehension that this somehow makes things
better -- the history of all languages which introduced ``optional''
redundancy makes this abundantly clear.  So, in practice, defending
redundancy as ``it's just optional'' is nothing but a figleaf: if it
gets its ugly foot in the door, it WILL be around most people's necks
most of the time foreverafter.

Python is one of the few widespread languages where one is blissfully
free of practically-mandated redundancy, and can thus practice the
healthy programming principle of "Once, and only once" -- not having to
repeat things twice like some magical formula in a ritual.  This lack of
redundancy is a good part of Python's power, in the judgment of many of
us who deliberately chose Python rather than other languages which allow
(and indeed practically mandate) the redundancy.

I, and many others, chose Python over other languages, because (duh!) we
saw Python's differences wrt the others as its _strengths_ -- and ever
since we have had to fight off the well-meaning attempts of ceaseless
waves of newcomers, who are utterly keen to spoil Python by making it
more similar to those other languages we rejected in its favor, i.e.,
sap Python's strengths.  Many old-timers already believe that changes to
Python in recent years have not been an overall good thing -- I
disagree, as I believe the changes so far have mostly strengthened
Python's support for its own design principles and goals, but I
understand their viewpoint.

If you're looking for insight on end-to-end checks as being preferable
to localized ones, the best paper I know is one on networking theory,
<http://mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf> -- for
test-driven design, Beck's book, the already-referenced Robert Martin
blog entry <http://www.artima.com/weblogs/viewpost.jsp?thread=4639>, and
innumerable other entries about "unit testing", "agile programming",
"test driven design", and suchlike, which you can google for.


> I am trying to talk about having expressive power in constraining
> references as well as the referents. Python studiously avoids this, but
> decorators change that.

No it doesn't!  A decorator puts absolutely NO constraint on any
reference whatsoever -- you're perfectly free to rebind every reference
at will, as usual.

A *descriptor*, or any of several other OO mechanisms, may put whatever
constraints you like on ``compound'' references -- attributes and items.
Most simply, you can define __setitem__, etc, in a class, and then
assignments to an indexing on instances of that class will go through
such special methods which may do, *AT RUNTIME*, whatever checks you
like; and similarly for __setattr__ -- and if you can use metaclasses to
put such constraints on classes just like classes put them on instances.

A descriptor lets you ``constrain'' one specific attribute of instances
by having all bindings of that attribute go through the descriptor's
__set__ method (if it defines one, of course -- that's currently known
as a ``data descriptor'' though the terminology is shifting).  Again,
that's entirely a runtime issue -- no ``declaration'' whatsoever.


> I am not deep enough into the mojo as yet to
> have more than a glimmer of an idea about the distinction you are
> making. It's not the one I'm trying to make.

The distinction that seems relevant to me: a *declaration* is about
something you tell the _compiler_, something which has intrinsically
static effects, _preliminary_ to runtime; a *statement* has effects at
runtime _if and when it executes_.

In Python, "global" is a declaration -- a wart -- because it has exactly
these ``preliminary'' effects, independent of runtime and execution.

But, for example, ``class'' definitely isn't: it builds and binds a
separate class object each time it executes, IF it executes.  E.g.:

classes = []
for i in range(3):
    class Foo(object): pass
    classes.append(Foo)

classes[2].zippo = 23
classes[0].zippo = 17
print classes[2].zippo

See?  No declarations whatsoever -- just a few perfectly ordinary
statements, which do perfectly ordinary runtime operations, such as
making objects, binding names, calling methods, ...

If the first statement in the loop body was changed to:

    Foo = type('Foo', (), {})

we'd have exact semantic equivalence.  It's NOT a matter of "mere
implementation": this is how the class statement is DEFINED to work;
it's SEMANTICS.

> decorators may not be implemented as declarations, but they cognitively
> act as declarations, and that's what I care about here.

Once again, with feeling: implementation doesn't really matter.  Their
SEMANTICS are defined in those terms -- perfectly ordinary runtime
executable statements.  In *NO* *WAY* *WHATSOEVER* do the "act as
declarations", and if your cognition tells you differently, then it's
your cognition that is faulty in this case -- you're badly misreading
the whole situation.

In particular, decorators don't put any "constraints on references",
which you seem to somehow mysteriously equate to ``declarations''.  Of
course, you may use a decorator to install a _descriptor_ object (which
may put constraints on compound-references, as above mentioned), just as
you might use an assignment for the same kind of "installing".  But that
doesn't make
   @foo(23)
   def bar( ... :
      ...
any more of ``cognitively a declaration'' than the exact semantic
equivalent:
   def bar( ... :
      ...
   bar = foo(23)(bar)

If this occurs outside a class, there's no connection to compound
references, thus certainly no constraint; if inside a class, and foo(23)
is a callable which returns a data-descriptor object when called with
function object bar as its argument, then this may imply constraints on
access to x.bar where x is an instance of said class.


> > I have nothing against a declarative style _per se_ -- it just
> doesn't
> > fit Python's "everything happens at runtime" overall worldview, and
> that
> > simple and powerful worldview is a good part of what makes Python
> tick
> > SO well.
> 
> I'm glad you have said something something I absolutely agree with. I'm
> alarmed at the suggestions here that class and def blocks are
> declarative. The fact that they're executable is really a core part of
> the beauty of Python.

And the fact that *decorators* are executable is exactly equivalent.


> However, I don't see how an 'import strict' would necessarily violate
> this, nor an "import immutableref", which is something I would find
> useful in trying to wrestle with NumArray, and which a high-performance
> Python could (I think) use to advantage.
> 
> Now I may be wrong; in fact I'd bet against me and in favor of you and
> Frederik if I had to bet. It's just that I don't see why I'm wrong.

What would the effects of such an ``import strict'' be?  Stop any
binding or rebinding of barenames except by some arcane incantations?
Then, by inevitably becoming a "theoretically optional, practically
mandatory" part of Python use, anytime a typical pointy-haired boss has
anything to do with it, it would sabotage programmers' productivity, by
forcing them to use just such redundant incantations and thereby violate
"once and only once".  Otherwise, I don't see how it would avert the
``epselon'' terror you keep waving at us.

If ``import immutableref'' is meant to make Python into a
single-assignment language (specifically and only for the module into
which it gets imported), it probably would not run as high a risk of
becoming mandatory -- single-assignment languages, and other functional
programming languages based on concepts of data being immutable, have
been around for ages but PHBs are terrified by them anyway (reasonably:
it takes a highly mathematical mind to be effective at functional
programming).  I do not understand how this would help you with
numarray, in the least, so perhaps you could help with some examples of
how you would like such a declaration to work.  Presumably the language
thus constrained would not have for loops (which do need to rebind the
control variable over and over), and while loops would also be pretty
iffy, so recursion abilities would have to be strengthened considerably,
for starters.  More generally, I have my doubts that Python can be
usefully constrained to single-assignment semantics without needing some
compensating additions elsewhere.  But maybe single-assignment is not
what you mean by your extremely elliptic mention, so I'll wait for your
examples of how that would help you with numarray in particular.


Alex