Is there a "Large Scale Python Software Design" ?

[Alex Martelli]

Dave Brueck <dave at pythonapocrypha.com> wrote:
   ...
> >>(2) much more common in code written by 
> >>lazy/underexperienced developers who are already considered a liability,
> > 
> > No, I think you're wrong here.  Typos are just as frequent for just
> > about all classes of coders, lazy or eager, experienced or not -- the
> > eager experienced ones often use faster typing (nothing to do with
> > static typing;-).
> 
> Hmm.. when I wrote that I was merely guessing at what "classes of errors"
> he was referring to, and I honestly didn't even consider typos. Instead I
> was thinking

Ah, OK, sorry, it was the first thing that came to mind.

> of cases where the programmer is reading too much into information
> returned from the compiler. Two specific instances I've encountered in the
> past came to mind:
> 
> a) Slopping casting - the attitude that casts are needed only when the
> compiler complains, and warnings/errors due to type mismatches are given a
> cast without much thought - the goal is to silence the compiler.

"The goal is to silence the compiler" is a common mindset, quite
understandable given the popular and reasonable shop rule "-Wall, and
can't be checked in until it given no warnings".  Not exclusive to lazy
or inexperienced coders.  Still, you do have a point here.

> b) Sloppy refactoring - "I added a parameter to the method signature and
> recompiled. In each case where the compiler complained, I added the new
> parameter to the call until all instances were fixed".

OK, nolo contendere -- I _have_ seen this happen.  Without a decent
refactoring browser in the toolset it will keep happening.  Running the
unit tests is a better way to check your refactoring, but it isn't
perfect either -- unit tests typically don't aim at 100% code coverage,
much less 100% branch coverage.


> Basically, they're cases where static typing appears to give an advantage
> in locating problems, but they are sometimes misleading (and therefore can
> _sometimes_ be a disadvantage if relied on inappropriately) and sometimes
> the remedy more involves helping the developer progress & improve (as such
> efforts yield greater fruits anyway).

OK, you've made your point well, thanks.  With all the "sometimes"
there, I can't disagree;-).


> >>development process they must be defined - perhaps that's the root
> >>problem of some static type systems - they make you declare intent and
> >>semantics when you know the _least_ about them! Consider the parallels
> >>to available knowledge in compile-time versus run-time optimizations).
> > 
> > If you mean typesystems such as Haskell's or ML's, allowing extended
> > inference (and, in Haskell's case, the wonder of typeclasses), I think
> > you're being a bit unfair here.  You can refactor your types and
> > typeclasses just as much as any other part of your code, so the "when
> > they must be defined" seems a bit of a red herring to me
> 
> True enough - the "when" bit apples to the static type systems of e.g.
> C/C++ - where up front you have to specify more detail than you may have
> (what set of values might it have, how will it be used, does it ever need
> to behave like objects of a similar type, how do we represent "nothing",
> do we need to be able to differentiate between uninitialied and "not
> present", etc.), and refactoring later on can have interesting
> side-effects (not necessarily due to the typing system directly; rather,
> such a type system encourages programmers to prematurely think about
> things like saving memory by using a short vs an int or making sure a
> structure wastes the minimum possible number of bytes on alignment padding
> - later on when you better understand how the data is used and would like
> to refactor it, it can get messy trying to separate decisions that were
> made due to requirements versus those made for other reasons).

Again a good point, this one about strongly pushing programmers (who are
quite prone to this mistake anyway -- yeah, even the expert ones!-) to
premature optimization.  In theory, optional-typing systems like Dylan's
should be perfect here -- you start typeless (in the static sense) and
only add typing later, and optionally, and selectively, as an
optimization.  But then, such systems will never provide the "error
safety" which many static-typing enthusiasts claim: in such cases,
typing _is_ rather, seen strictly as an optimization-aid hint to the
compiler.  We've seen by the widespread overuse of __slots__ what can
all too easily happen when such "optional hints for optimization only"
are exposed: a substantial minority (at least) of programmers will
overuse them in a way quite different from their design intentions, and
one which they don't even support all that well.

Perhaps we need a broader "hinting system" design -- assert is fine, I
think, but clearly most programmers don't agree.  Anyway, such hints as
could be used for both error-checking and easier (for the compiler)
optimization, will generally check errors at runtime (just like, say,
design-by-contract does) -- only occasionally, not systematically, will
the compiler be able to prove that a certain hint's check is bound to
fail.  So, this isn't really germane to static typing.

Thanks for making several good points clearly and convincingly.


Alex