waterfall (was Re: REPOST: Re: Book "python programming patterns". anybody read this??)

[Peter Milliken]

"Alex Martelli" <aleax at aleax.it> wrote in message
news:mailman.1010095689.2965.python-list at python.org...
> "Peter Milliken" <peter.milliken at gtech.com> wrote in message
> news:a0vvrj$e862 at news1.gtech.com...
>     ...
> > > How do you KNOW that "the customer's requirements are definitive"?
> >
> > How can you BUILD something if you don't know what it is? If you are a
>
> By *finding out* what it is.  The process of "building", in the widest
> sense of the word, is the process of finding out what is being built.
>

And you don't call "finding out" requirements analysise? :-)

I'll drop the rest of the email now......

> > software house then you have to get an agreed contractual basis as to
what
> > you deliver - otherwise you are always changing what you deliver - which
> is
>
> Remember that well over 90% of software development is in-house, so that
the
> specific business models you may have in mind are quite marginal in the
> world of software development as a whole.  Further, a good slice of what
is
> NOT "in house" is development of software to be sold (or rented,
subscribed
> to, included as novelty gift in potato chip packs, etc) "off the shelf",
so
> there is absolutely no "agreed contractual basis" with your customers
until
> AFTER you have the software ready for sale (rent, subscription, etc).
>
>
> > in software otherwise you go out of business backwards. Of course, if
you
> > have the "luxury" of developing for an inhouse part of the company then
> you
> > (the supplier) and the customer can screw around to your hearts content
>
> Except that this is not the customer's interest: they want SOLUTIONS to
> their problems, the latter often being rather ill-defined ones.  So, it's
> in their best interest to work with you to help find out exactly what they
> need, and with what priority in terms of what needs to work first, what
can
> come later, what would "be nice to have" but not vital, and so on.
>
> Further, YOUR interests are also to similarly develop successful, useful,
> usable, software systems -- systems that get used and deliver results.
> That's assuming you're an engineer at heart, rather than a bureaucrat
whose
> key goals are following established procedure and always having somebody
> else to blame when things go wrong, of course.
>
> > So nailing down the requirements is always the first step - agreement
> means
>
> It almost never is.  Use velcro or masking tape, not nails.
>
> The requirements are going to be refined, prioritized, changed in scope,
> refactored, reprioritized, etc, throughout the life of a successful
> software system.  Those nails would get bent out of shape and rust far too
> soon to be any real use.
>
> > failures that I have seen and read about have one underlying common
> thread -
> > the requirements were still being changed at the time the project was
> > cancelled.
>
> I.e., the customer was in the process of finding out what they actually
> needed -- a perfectly normal interactive discovery process -- and
somewhere
> in some decision-making positions were people who did NOT realize this and
> were not prepared for it.  Most likely because they harbored [expletive
> deleted] notions about "nailing down the requirements is always the first
> step".
>
> BTW, you may not read about them as failures, but examples also abound
> of software that WAS painstakingly specified in minute detail before any
> step towards building it was taken, then later delivered -- and never
> used, being totally irrelevant to the business needs.  In my (indirect)
> experience, this tends to happen in public procurement, where laws and
> regulations may indeed mandate such an absurd process.  It also happens
> in development for "off-the-shelf" sale (etc), in which case it tends to
> be more visible since the end product, despite fulfilling its "nailed
down"
> specs perfectly, doesn't sell.
>
> It may not be something you will read about as a failure, but, both as
> a taxpayer AND as an engineer, I consider these disasters, which are far
> from unheard of, every bit as bad as other development disasters.
>
>
> > > Let's say: use waterfall when you are ABSOLUTELY certain that the
> > > customer's requirements are definitive and you have understood them
> > > correctly and completely at once.  To make "absolutely certain" more
> > > concrete, let's say that you will undertake to disembowel yourself
> > > ritually if it turns out you were wrong -- that the requirements were
> > > NOT quite as clear, well-communicated, well-understood, definitive
> and/or
> > > nailed-down as you thought.  If you're not willing to commit to this,
> > > then you're not ABSOLUTELY certain, and you shouldn't use waterfall.
> >
> > I think you are getting too rigid here - all software models are based
> upon
> > the fundamental steps of requirements analyise, design, code and test.
> Even
>
> You may call then "steps" if you wish, and you have missed quite a few
> crucial ones (such as "deploy", "document and/or train", "tune", ...),
> but that doesn't make them anywhere as separate and/or sequential as all
> that.
>
> > models based upon rapid protyping follow the same basic steps (the
> > requirements are very loose obviously, the aim of protyping is to refine
> > them) - the protype is built to a set of understood requirements, the
> > protype has some form of design, there is coding and there is testing.
>
> "There is" understanding, design, coding, testing, and other things yet
> "there are".  But the optimal sequencing, interleaving, merging and
> splitting of the various activities are not optimally rigid -- YOU tell
> ME "I am getting too rigid", I, on the other hand, claim Agile Development
> processes are the very antithesis of rigidity and Waterfall IS rigidity in
> software development -- an inappropriate, inferior and unsuitable
> methodology.
>
>
> > So I am getting confused by your responses - all you seem to do is blast
> > waterfall - I haven't seen any attempt at some constructive response
here,
> a
>
> All I want to do in this thread is make sure waterfall is and stay
blasted,
> since benighted discussors are trying to revive the zombie.  Excellent
> textbooks, articles, courses etc, abound, on many superior alternatives,
> such as Rational Unified Process, Extreme Programming, and so forth.
>
> > proposal for some other model because it fits better into your concepts
of
> > software development or personal situations.
>
> Read anything published in the field in the last few years and you'll
> find nothing but.  Meanwhile, the monster of Waterfall needs to be
> spiked through the heart each and every time it threatens to revive, which
> is basicall what I'm doing here.
>
> > All I have said is that waterfall can and has worked in the past and
>
> It can't and hasn't, in any relevant case (see below).
>
> > shouldn't be dismissed upon the argument that "it hasn't worked in the
> past,
> > so therefore lets not consider it" (my interpretation of your
responses).
> > This seems a nonsense argument at best.
>
> It has never worked, it can never work, it's an absurdity based on a
> total misconception of the nature of programming, of engineering, of
> the human mind, of human society, and of the universe.  There, good
> enough for you?
>
> You're of course driving me to rather extreme assertions, just as I
> would be if (e.g.) debating some other similar absurdity which every
> historical experience and common sense show can't work, yet upon
> which altar blood (true or metaphorical) has been shed aplenty in
> the past and threatens to be shed again until and unless the nightmare
> is driven forever from the sublunar world.
>
> There ARE trivial 'projects' (not worth the name of 'projects') which
> basically consist in doing for the hundredth time much the same thing
> as you've done the last 99 times.  Boring, but true; in this case the
> amount of "interleaving" needed may diminish, potentially down to a
> lower bound of zero.  This is much like saying that "communism" can
> work within a small, close-knit family where everybody trust and love
> each other: in such a scenario there is no need to account for "whose
> money it is", who's earning and who's spending, and so on.  But it
> *JUST DOESN'T SCALE* -- it's no basis for economics, which has to deal
> with the vast majority of interesting cases, where people, far from
> loving each other, may barely _stand_ each other.  Idealizing and
> extrapolating the tiny "loving family" scenario leads to horrors that
> range from endless queues outside of empty shops all the way to rivers
> of blood down the streets.  Wrong-headed software development approaches
> so far have less potential for utter disaster -- but, deaths HAVE resulted
> from "waterfall" development (specifically its hubris-like lack of
> check-backs and continuous verification), as any reader of the Risks
> column will know.
>
> Waterfall proponents traditionally wanted to dismiss such occurrences
> as (paraphrasing) "operator error".  That's a good symptom of a system
> that's not fit for use by human beings, and the rationalization attempts
> for the system.  Sure, there's nothing wrong with communism per se, only
> when you apply it to human beings do problems appear (for all I know,
> Martians may be quite happy with it).  Perfect beings might be able to
> apply Waterfall -- I don't really care about them (being perfect, they'll
> surely be good at picking their methodologies anyway): "Know then thyself,
> presume not God to scan; The proper study of Mankind is Man".
>
> Man is fallible, and has evolved in an environment of trial-and-error,
> continuous feedback, continuous adjustment.  The worst failures come when
> one's pride rears up and proclaims that one HAS fully grasped the issues,
> so no more tentative, feedback, adjustments, etc, are possible -- when one
> denies the fallibility, or fails to accept the indispensable "checks and
> balances" that compensate for it and make it bearable.  "Pride comes
before
> a fall".
>
> On the subject of "operator error", see Perrow, "Normal Accidents"; and to
> some extent Norman, "Design of Everyday Things".  On inflexibility
> (resulting from excessive pride from past successes) as the key aspect in
> the (economic) fall of every Empire, Carlo Cipolla has an excellent short
> article in his delightful, highly mixed book "Le Tre Rivoluzioni" (there's
> probably an English version somewhere too, given that Cipolla has written
> and taught in English even more than in Italian -- I can look for it, if
> you want to try and read it).  Kennedy's "Rise and Fall of the Great
> Powers" is, to some extent, a look at the same issue with wider scope.
>
> Waterfall is highly reminiscent of each of these general issues.
>
>
> > any superior state. It appears more to me that the new genes entering
the
> > pool from colleges every year think they are the next Leonardo Da Vinci
of
>
> A failure, as an engineer, of course -- he applied "waterfall", rather
than
> iteratively testing and checking that he knew what he was doing.  So, none
> of his machines ever really worked well, and some of his incredible works
of
> art (artists can afford far more pride than engineers can) were not as
> lasting as those of his contemporaries (who, more humbly, tried things in
> small increments and kept double-checking with customers...).
>
> > software development, that they can ignore the lessons of the past and
>
> I.e., human fallibility?  If new graduates are so full of misplaced pride
> that they think they can use Waterfall, rather than solid, interactive
> mingled-steps development, I entirely blame their teachers.
>
> > stride out there with supreme confidence that they know what is right -
> > hence people like me (who have been here for several decades now) see
the
> > same mistakes being made time and time again. I am sure that you are
just
>
> Yes, I do see Waterfall being praised -- but more likely by people my
> contemporaries, such as you, rather than by young brilliant graduates (who
> are more likely, at least, to have read recent relevant literature).
>
>
> > confident as I am that you "know what is right". I have seen what works
>
> It's exactly because *I know that I don't know* (and that others don't
know
> either), that I consider it a horror to praise and defend a methodology
> that could only work for perfect (and mind-reading) developers who are
> developing for idealized customers who DO know exactly what they need.
>
> > what doesn't, I don't pretend that I know it all, but what I do know is
> > correct for the circumstances I have experienced :-). So waterfall can
and
> > does work in the right context. So do the other models - there is no
> single
> > model that will work best for all circumstances.
>
> No, but, as I don't care about what works or doesn't for hypothetical
> Martians, I do know there's a model that will work horribly in all
> interesting circumstances: it's called Waterfall.
>
>
> > So, unless you are "absolutely certain" then I hope you are on a time
and
> > materials contract, otherwise there will be a win-win situation and the
>
> That I am absolutely certain that Waterfall doesn't work for human beings
> doesn't mean I already know what does work best for any given situation,
> since there are so many other possibilities; it means I, and the rest of
> the team, _including the customer_, can *find out*, and interactively as
> well as iteratively develop and refine solutions and methods that "work
> well enough" to deliver significant business value, reliably and
repeatably.
>
>
> Alex
>