[Python-Dev] The baby and the bathwater (Re: Scoping, augmented assignment, 'fast locals' - conclusion)

[Boris Borcic]

Josiah Carlson wrote:

>>
>> [BB]
>>  >> I'd say a first step in convincing me I am wrong would be to show me 
>> examples of
>>  >> object methods of the standard library that are recursive, and cut out for
>>  >> recursion.
>>
>> [JC]
>>  > Actually, I don't believe that is necessary. I've shown that you would
>>  > get better performance with a non-recursive function and passing two
>>  > arguments, than you would passing one argument with scoping tricks to
>>  > get the second.
>>
>> Assuming my aim was purely performance clearly stretches the hints I gave that I 
>> wasn't unconcerned by it. Taking ground on this to unwarrantedly suppose that my 
>> methodology was deficient, rectify it, and then dictate the "necessary" 
>> conclusions I find... er, preposterous seems balanced, "thought police" not 
>> totally unwarranted.
> 
> In my initial and later readings of your messages, I read 2 reasons why
> you were using a closure rather than some other method to perform this
> particular operation:

That wording already doesn't apply. I told you I was working *not* on a program 
but on a *space* of programs of identical function, as of manner of probing 
recent features and modules of Python that I manage to apply to it. Besides 
learning, my purpose is also aesthetic.

Some versions use closures, some don't - of course.

> 1. Minimal code change.

That's a reducive synonym for "being able to compare versions while changing 
only single factors of my choosing". Currently I would add : "without your 
mechanical avatar coming into the field uninvited".

> 2. It was faster than using an object with a particular interface.

No way. PLEASE RENEW YOUR GLASSES.

> 
> If there were other reasons, then I obviously missed them.

Excuse me, my perception is that there has not been a single one of my 
interventions that you took fully at face value. I am the perpetual strawman in 
an argument that is not adressed to myself (as manifested) but to some shadow of 
myself by some projection operator, onto a convenient manifold of ideally 
misguided contradictors.

> However, since no one else has responded substantively to your postings,
> other than Terry and Guido's 'not gonna change',

That's "Terry's 'take the fairy tale' <comma> and Guido's 'not gonna change'", 
to be exact.

> I can only presume that
> others have found that your desired result (augmented assignments for
> names not previously existing in the current scope, but existing in a
> parent lexical scope, be performed on the object bound to a parent scope
> name), is undesireable for Python 2.5

This is again misrepresentation. I definitely did *not* come to pydev to ask for 
a change, I merely intervened in the middle on an ongoing thread at the point it 
was being said that, in effect, I did not exist. Thereupon you intervened with 
no other purpose than to show, in effect, that I did not exist, forcing me into 
an endless chain of rectifications. Now you are telling me I want it included in 
the 2.x series. That's confusing the citizen for the lobbyist.

> 
> In a portion of your message that I've cut out, you described how the
> goal of your current programming project was to write a sudoku solver in
> fewer than 60 lines and that ran in fewer than 10ms for arbitrary
> problems.

No. PLEASE RENEW YOUR GLASSES.

FYI, an universal sudoku solver in distribution python, confortably less than 60 
locs, and solving an average problem in confortably less than 10 ms (on my 
machine) : that was an early result, and indeed a surprise.

The "programming project" I *did* describe was the project of *further* 
exploring, with systematism, the space of nearly equivalent programs - while 
slightly *relaxing* the frame of the initial result, and while exercizing 
varying parts of relatively recent distribution Python or third party modules 
that I did not know, or not know well.

Seeking beauty, if you want. You understand the quest for beauty ? I guess not 
in the manner of someone who started programming when programs had to fit in a 
couple dozen kbytes.

The "compiler feature" under discussion just pissed me off when it popped up to 
veto what was emergeing as the winner of my beauty contest after examining quite 
a few worthy candidates over weeks.

> Great, wonderful, I wish you luck; I will not question or
> "refuse legitimacy, motivation, efficiency or methodological soundness"
> of your approach when using the Python programming language.

What's wrong with chosing a set problem to compare features of the Python 
programming language against, and learn from it ? Please tell.

And what's wrong, when seeking beauty, with picking a stringent constraint and 
then explore in parallel various manners to satisfy it, relaxing it slightly if 
need be ? How do you think nature evolved birds or flowers or anything that lives ?

> However, you are offering a series of reasons as to why the Python
> programming language should be changed.

It's *you* who made me your mirror. I was merely saying : "hey, I find this 
crazy, can someone provide explanations ?".

> I'm questioning your reasoning.

No, you are not questioning *my* reasoning. Never. Nada. You are perpetually 
*making up* a reasoning for me that *very roughly* fits what I say, but has the 
nice feature that you know how to refute it.

> You may find it insufficient, misinformed, prejudiced, nonsense,
> mean-spirited, or even that I'm a loony who should be in your killfile.

Well, at this point you risk an even worse fate than that ;) That is, the 
intersection of the file tagged "arrogant would-be mind-readers", and of the 
file tagged "you are late on the normal trajectory if you did not reproduce my 
own oh-so-significant mistakes", both well-populated already, although I forgot 
who I put in them.

> My purpose here isn't to convince _you_ that you are wrong about this
> feature, as arguing via email has never been my strong suit, but instead
> to offer at least one valid reason why your change could or would hurt
> some Python users, and/or the effectiveness of their code.

The problem is that you fail to be polite about it more often than not. Most of 
the time not really talking to the reasonable person I tend to be.

> 
> I will mention that the lack of _any_ public support in python-dev
> likely says far more about the unliklihood of acceptance than anything
> you or I have probably typed in the last few days.

And what's the purpose of that declaration ? You are the, hum, crazed lobbyist, 
not me. I am just defending that my pov is perfectly legitimate, including when 
it's examined by your criterions - once the fallacious ones among them are 
filtered out.

> 
> 
> [snip]
>> [JC]
>>  > when it was an
>>  > actual error (a not uncommon case),
>>
>> I guess your insistence on the rightful methodology entitles me to enquire in 
>> turn on your choice of tool to establish the relative frequency of said case ?
> 
> My measurement is quite subjective.  Have I ever made that mistake, or
> do I know of anyone who has made that mistake? In this case, I have made
> the mistake I mention, both in personal code, as well as code I was
> writing as I was tutoring.

Thanks for the transparency. You mean code you were writing in real time before 
an audience ? Hum. Surely you take the time to debug example code before 
presenting it ?

The error you mean - do we agree on this ? - is to intend two variables of the 
same name, one in the parent scope, the other shadowing it in the child scope, 
and mutating the variable in the child scope while having forgotten to 
initialize it.

A few questions. First, do we agree or not that gratuitous shadowing is by 
itself a mistake ? Or do you mean it happened to you while the shadowing wasn't 
gratuitous ? Also, do you mean it happened to you in the Python context so that 
Python stopped it at "ref to uninitialized local" - or in some other langiage 
context ? In the Python case, I can only wonder as to what could make it 
memorable - not the debugging, certainly.

In the non-Python case, well, it's not really clear that a tendency manifest in 
another language should spontaneously carry over to Python : for instance, 
Python differs from some languages in that augmented assignments aren't 
value-producing expressions. And some language push towards embedding scopes to 
a level much greater than would Python even without "the javanese watchdog" imo.

My take on the issue, btw, is that the "scope capture" behavior of 
(non-augmented) assignment statement is in Python the residual counterpart of 
variable declarations in other languages; and it is significant that it shares 
the most typical basic form, since in most cases declaration and initialization 
are fused. This doesn't carry over to augmented assignments that are mutation 
operators.

> I'm not aware of significant problems in
> user code, usually because the moment the code is run, the programmer
> fixes it.

Thanks for your honesty.

> 
> Here are two questions for you:
> 
> 1. Do you have any proof showing that the error I claim exists (not
> uncommonly was my description), is in fact so rare, that I am
> essentially the only one who has made the mistake?

Well, my take on an issue of such form is perhaps particular, but - as I already 
suggested with my "hate for whatever resembles a constraining curriculum of 
errors to make" - I really believe, as a matter of ethics, that the burden of 
proof is on your side, not mine.

I can try to explain more, if you want.


> 2. Do lexically nested augmented assignments happen so often as to make
> this problem a "must solve" for any Python revision?

Hum, first of all implementation of the solution is (according to recent 
records), so  trivial that it occurs spontaneously as a bug. Therefore, even 
conceding to you the usage be so rare that the gain is vanishingly small, the 
ratio gain/effort would be one of vanishingly small quantities, which itself is 
a priori not determined to be close to zero.

This of course does not count the cost of the decision. AFAICS, you are yourself 
the most significant factor in determining that cost, and because of that I 
think it would not be fair dialectics that I should take that cost into account 
to answer your own question. Right ?

Second, the assumption of rarity leading to insignificance - does it escape you 
that's a sword that cuts both way - as concerns a sizeable part of your own 
argumentation ?

Third, to me the matter is foremost one of aesthetics. The contentious "feature" 
made me briefly feel like I was confused on the identity of the lady I was 
making love to.

Fourth, of course I can survive without it.

> 
> 
>> [JC]
>>  > which is a bit of a no-no,
>>
>> What I would have believed a /bit/ of a no-no for Python, is to turn away from 
>> the "consenting adults" spirit to go the way of so many other programming languages.
>>
>> Picture of the latter way : - As a matter of course, to place the virtue of 
>> catching errors early, before the virtue of following programmers' legitimate 
>> intentions. - Designing for assumed chronic lusers in need of being protected by 
>> the compiler from shooting themselves in the foot.

It is really denaturing to what I was saying there, that it got disconnected 
from the immediate context of my claim/demo that your analysis secretely 
referred to a situation of cumulated simultaneous mistakes while presenting it 
as a single error.

> 
> Do skilled programmers make errors?  Yes.  Do skilled programmers make
> errors when they are first using a programming language?  Yes.

Of course everybody makes errors, but it doesn't follow from this, that all make 
the same errors, or should. Think of errors as genes of the developping minds, 
and design better for biodiversity, I'd say. My first professional programming 
was on a system that did not support any form of encapsulation and gave me 
access to 300 distinct opaque variable names : quite a training on the matter of 
variable name capture, I tell you.

As concerns Python, the issue is that in most aspects it is carefully crafted so 
that you can often "just try it and it works as you would expect". For this to 
work, consistency is very important. Our whole debate is that according to me 
the "feature" under discussion is not in the same spirit while you (and Terry 
indeed) found preferable not to acknowledge the notion, what also prevented the 
possibility to convince me that the "feature" should exist anyway, for logical 
reasons a reasonable mind could hear.

This was further complicated by your obsession with proving *first* that I could 
not possibly have a legitimate use-case, whatever I said.

> It has
> always been my experience with Python that it seeks to be reasonable to
> use for both newcomers to programming and long-time "skilled"
> programmers.

Nobody forces any school to present closures to beginners. Burn SICP before your 
class.

More seriously, I believe the case you made against closures really should hit 
what incites to the "simultaneous mistake", eg the ease of shadowing builtins 
and globals, inadvertently to start with. A nicely fat set of keyword-like names 
that one can't shadow without special action, would that not be the best of 
schools against gratuitous shadowing ?

Well, maybe that would be too much of a change for Python. OTOH, mhh, I would 
need to study the matter more in depth, but at first sight I see absolutely no 
obstacle to trading a ban on closures shadowing names from the outer local 
scope, against the current ban on augmented assignment of variables from the 
outer scope (even without including straightforward assignment in the bargain).

This would give python closures a new educative role : a context that trains 
against shadowing. If I am not mistaken, it would fulfill the "use case" you 
find to the "feature" I dislike, just as well.

> 
> In my statements regarding possible errors related to augmented
> assignments, I merely believe that people can and will make mistakes.  I
> further believe that if I manage to make a particular mistake, that
> someone else has, because while I'm not the most brilliant programmer
> out there, I'm also certainly not the worst, and I would guess that I
> lack the creativity to generate a completely new class of mistakes.

Whatever, programming or not, I believe the most dangerous (in the sense of most 
difficult to rectify) class of mistakes is the class of diagnostic errors - 
meta-errors, so to say; because of that, I believe a danger with mistakes or 
errors is to anticipate too much on them, and, given their diversity, to 
overestimate the scope of the lessons learned from the sample of them that most 
caught our attention (usually because of their painful consequences).

Besides, I think there are subtle issues with taking errors as building material 
for policies destined to public consomption by new generations. It is vital to 
find a self-standing positive goal to substitute to the negative goal of 
avoiding the repetition of the error, etc.

Btw, how you dealt with my case tells me you don't particularly fear diagnostic 
errors, but your way of presenting closures and classes in Python tells me that 
you probably agree with my later remark.

> 
> You talk about "Designing for assumed chronic lusers",

Again, this has lost its proper context.

> I'm talking about
> designing for human beings, some of whom are new to the language.

This starts resembling the disputes I had with my wife over the education of our 
child. Total divergence as to the adequacy of trying to control the child's POV.

> 
> 
>  - Josiah
> 

Boris
--
"on naît tous les mètres du même monde"