Python paradigms

[Robert W. Cunningham]

Oleg Broytmann wrote:

> On Sun, 9 Apr 2000, Fredrik Lundh wrote:
> > Oleg Broytmann wrote:
> > >    blah = ((foo() and [poo()]) or [boo()])[0]
> > >
> > >    In C, "foo() ? poo() : boo()" will not call boo() if foo() returns
> > true;
> > > but in Python, you'll have a side-effect of calling boo() even if foo()
> > > returns true - what is wrong, usually.
> >
> > did you try running the code?
> >
> > python's "and" and "or" operators doesn't evaluate the
> > second argument unless they really have to...
>
>    Yes, I missed that, sorry.

So, it works just fine, right?

>    Anyway, it is strange (not to say "ugly") construct; I prefer not to see
> it in a code...

I remember a professor who had a mantra he'd repeat just before accepting
programming assignments from his students:

  "All code without comments is UGLY code.  And ugly code gets an 'F', even if it
works perfectly."

Which had an implicit corollary:  "Nasty Code + Comments = Non-Ugly Code"

Not all problems have "nice" solutions, but even the messiest solution can be
made palatable with the addition of suitable comments.

The best example of this I can recall was in an extremely obfuscated MD5 hash
generator that fit in something like two or three lines of code.  The comments
required two pages (to explain both the MD5 algorithm and this implementation),
and did their job quite well.

The "self documenting" version of the code, on the other hand, required only one
page, but even with maximum compiler optimizations it ran 4 times slower than the
obfuscated version.

I'm not saying either of these were ideal pieces of code.  But when the emphasis
is on performance AND maintainability, then nasty code combined with sufficient
comments works extremely well.  When the emphasis is on "readable" or "self
documenting" code, it can be much harder to obtain suitable performance.  It has
also been my experience that performing structural optimizations to code is
somewhat easier when the code is more compact and well documented, than when it
is "self-documented" and spread over a greater area.

In Python, performance quickly slows to a crawl unless a handful of trusty
structural optimizations are used, such as assigning methods to variables before
calling them repeatedly within a loop.  This clearly makes the code less "self
documenting", since it imposes a structural optimization to circumvent a basic
behavior of the Python execution environment.  Surely, someday the Python
interpreter will be able to create such optimizations for us on the fly (just a
little loop invariant migration, right?), but in the meantime these manual
optimization constructs are so prevalent that they have almost become part of the
Python language.  Indeed, they are documented as "necessary" by GvR himself!

I see each and every one of them as a wart on the basic elegance of the Python
language, since they generally serve only to manually optimize (circumvent)
access to namespaces (or other Python "features"), to "break" the most basic
elements of OO implementation!

It is getting to the point that these optimizations and constructs aren't even
documented in most of the code I've looked at, and I'm still a Python newbie.  To
me, even some "guru" code is ugly, ugly, ugly.  And such code is present in the
documents that define the Python language and its use.

So, when I hear Python gurus arguing about the "ugliness" of various coding
constructs, it makes me laugh out loud.  The common use of Python *demands* hand
optimization (aka "uglification") of the code to gain decent performance.  As the
code becomes faster and more usable, it gets uglier and uglier.  Maybe that's too
strong a statement:  IMHO, the "prettiest" "self documenting" Python code is
often unusably slow, especially where loops are involved.

The argument in this case should be about the degree of commenting needed, not
the code itself!  AFAIK, the code is FINE (though possibly not "perfect").  Has
it been tested for performance and side effects against other forms?  Until we
know better, it may be the case that the code we've been discussing to create
something equivalent to C's "?:" construct is OPTIMAL!

Let's assume, for the moment, that this is indeed the case:  Now, obviously, the
only thing left to discuss is the comments!  Right?  Who would toss fast code
merely because it was "ugly"?  Certainly no Python programmer!

And what IS the best way to comment such an obtuse line of code?  I submit my
example that's quoted at the top.  If you understand the "?:" construct from C,
it does the job perfectly (if minimally).  No matter how "ugly" the related
Python code is.

Python is, for now, condemned to use "ugly" code constructs if it is to see
maximum use and performance.  Let's use them when we must, but also pay strict
attention to thoroughly documenting them whenever they are used.

Which brings me to a related thought:  Has anyone looked at post-processing .pyc
files to optimize them?  Or are there plans to include a bytecode optimizer in
the Python interpreter or runtime environment (as a JIT optimizer/compler)?
Without, of course, bringing the entire JVM or JRE into the game...  That might
be overkill.  Any such moves should serve to reduce or eliminate the *need* for
much of the present ugly Python code.


Oh-to-have-loop-invariant-migration-ly yours,

-BobC