[Doc-SIG] Improving the documenting process.

[Laura Creighton]

Chad suggested I post this reply to him back to doc-sig, and see
if there are others like me who think that the python documentation
problem centres around 'not enough good documentation being
produced' and a process that is opaque, has bottlenecks, and
excludes the very people who would be best at writing good
documentation, because they aren't python-language developers.

I thought to shorten it, and then thought, what the heck,  maybe
some other people would be interested in what I was doing in the 80s.
Sorry that this will bore some others.

Happy New Year,
Laura
------- Forwarded Message
Laura Creighton wrote:
> In a message of Sat, 31 Dec 2005 22:13:02 EST, Chad Whitacre writes:
> 
>>Laura,
>>
>>Thanks for giving me an opportunity to clarify my proposal.
> 
> 
>>On the other hand, I confess that I don't follow your suggestion that 
>>separation of "logical" and "rendered" information leads to a plain text 
>>base format, one without any markup. Why exclude markup that is semantic 
>>rather than presentational?
> 
> 
> Oh yes.  This may not be well-known.  I keep forgetting.
> 
> Because authoring is an art.  What you need are tools that support
> the process of composing excellent prose. All of the structural
> markup greatly hinders the process.  In exactly the same way that
> all the curly-braces-and-semi-colon junk in languages we know and
> hate, all there for the benefit of the compiler, detract from writing
> well.  The more intrusive the markup, the worse the prose.  (Which
> has been measured in places like the lab I worked in in the 80s.)
> 
> There is still hot debate in some circles as to exactly why this is
> so, but the prevailing theory is that good prose is dependant on using
> the part of your brain that listens.  So most people really have to
> 'hear the words in their heads' as they write in order to create the
> best prose.  (And some do not.  Why this is so is unknown.)  But when
> they start fiddling around with the presentation details at the same
> time as they are writing the text in first place, their attention is
> divided and the texts are significantly worse.
> 
> So, when we stuck people into our labs and measured them, our standard
> for 'how people write' was 'using emacs or vi and the MS macro 
> packages for troff'.  And it was clear that people were producing
> pretty rotten texts.  So we measured how well they wrote if they
> just used vi or emacs, and skipped the markup.  And for most people
> this produced a significant improvement.  At this point, believing I
> was onto something, I went out and founded a computer company, SoftQuad,
> with a bunch of friends of mine in the publishing business.  Device
> Independent Troff was new, laser printers were new,  bitmapped displays
> were new, and Apple was promising this thing called the Macintosh.  We 
> thought that we had the killer app.  A WYSIWYG editor, though I don't
> think we called it that.  Without evidence, we were convinced that we
> knew why it was that people wrote poorer text when using troff, and 
> that had to do with it being hard to use the macro packages.
> 
> And this goes to show that 'it ain't the stuff you don't know but
> the stuff you know that ain't so' that really hurts you.  Because this
> was one of the more spectacular cases of 'completely getting it wrong'.
> We made an editor (actually two -- a moded one which we gave the vi
> bindings and a modeless one which we gave the emacs bindings) and then
> took our usual suspects of university students in the humanities who
> already knew vi or emacs and had them go at it.
> 
> They were very happy to not have to learn the MS macro set.  But to our
> dismay, the prose still ended up rotten.  We'd solved the wrong problem.
> And the very best results were still being had by authors that used
> whatever text editor they liked best to compose text, and then handed
> the work to somebody else -- a compositor if they were dealing with
> our Dead-Tree-Printing Office, or the secretaries of the Political
> Economics department if you were a student or a professor there.
> Making the thing look good on paper -- or on the screen, was then
> somebody else's job.  But even if you composed the text first, and
> then made a second pass after you got the text down correct you would
> end up with a better result than if you composed and composited
> simultaneously.    Next we discovered that most people wrote better
> if they wrote with somebody else. (For non-fiction.  For fiction we
> couldn't come to any conclusions except that we needed to study the
> cognitive process of writing fiction more.) On the other hand, documents
> designed by committee almost never were well written unless the
> committee met, decided on the structure of the document and what
> its content was to be, and then let one or two people actually write
> the thing.
> 
> 
>>Also, you hint that it might be helpful to define the problem itself 
>>more clearly, and I agree. The problem I want to see solved is 
>>Fredrik's: how do we cut the documentation workflow cycle down from 3-6 
>>months to, say, 3-6 days?
> 
> 
> This is a fine goal.  But I think our problem is that we do not have
> enough high-quality documentation, and that the people who find problems
> with the documentation do not have an easy way to discuss this with 
> other people who are interested.  Submitting a bug report only means
> that people who are reading the bug tracker get to be informed.  This
> excludes the bulk of interested people.  
> 
> If you haven't read the Kruger Dunning Report _Unskilled and Unaware of It_
> http://www.apa.org/journals/features/psp7761121.pdf I suggest you go read
> it now.  It applies with particular vengeance to authoring skills, most
> especially in people who are writing in their native language.  Ability
> in language usage it not particularly bell-curved.  Both ends have
> relatively large numbers when compared to the population at large.  
> Excellent writing ability is something that people have selected for
> to such an extent that the right end is greatly increased, while 
> non-native speakers, and people with learning disabilities end up making
> the left end chunkier as well.
> 
> Which sadly means that the it is not just the lowest octile of language 
> users who lack the cognative abilities to judge their performance.  In
> our labs we found it remarkably easy to produce samples of people where
> 80% of the people in the room did not write very well, according to the
> judgement of the top 2%, but nearly all of that 80% believed that they
> wrote 'rather well'.
> 
> This makes the whole process of criticise-and-rewrite even harder than
> it already is, which is hard enough in the first place, and makes a
> bug tracker a particularly difficult place to raise your issues. There
> is no particular good way to say 'could I have a different python
> developer to discuss this with?  This one is not skilled enough to
> understand me'.  The more confused you are, and the harder time you
> have expressing and understanding your confusion, the worse it becomes.
> (It is hard on the unskilled python developer, as well.  He really has
> no way to tell the difference between things he cannot understand because
> he lacks the skills, and things he cannot understand because the person
> who posted the idea lacks the skills, and plain old bad ideas.)
> 
> One of the more frustrating things that I have seen happen again and again
> is where some new person, who doesn't understand how to do something,
> complains about the documentation.  All too often the response that this
> person gets centres around 'explaining to the newbie how the code 
> really works and how the newbie should have solved his problem'.  This
> drives the newbie, who already has solved his problem up the wall.
> He doesn't want help in learning things, he wants different and
> better documentation.
> 
> He may even have a fix.  But the fix is often going to be poor.  It
> may solve this problem, but it leaves things more confusing for others
> in some way.  This does not mean that the existing documentation is
> adequate.  It means that somebody who is both aware of the code
> involved, and who is aware of the newbies problems in comprehension,
> needs to rewrite the documentation.
> 
> Now it may be that such people are rare in general.  I think they are 
> rarer among computer progammers than in the population at large.  But
> I think that our structure exacerbates our problem.  If you re-read
> http://mail.python.org/pipermail/doc-sig/2005-December/003466.html
> you will see Fred Drake assert that:
> 
>  'comments from the release X.Y.Z docs need to be handled before 
>   releasing X.Y.Z+1 or X.Y+1.*, or they aren't being used to improve 
>   the documentation at all'
> 
> which is either trivially true, or the indication of a larger problem.
> This looks to me as if documentation releases are delayed until every
> (a large number? enough?) comments are handled, which strikes me
> as a very large bottleneck.  Wikipedia is proving that you can do
> continuous updates on documentation.  Do we need them?  Would we
> get more people working on improving the documents if the live
> version was a mediawiki?
> 
> I think the question of 'what do we need in order to share and
> collaboratively work on documents more effectively' is the one that
> we need to solve if we want more rapid turnover.  So I would say 
> that our workflow doesn't need _shortening_ so much as _transforming
> altogether_.  And I don't think I understand exactly all the steps
> we have now, so it would be nice if somebody who knows this very well
> would post them.  Then we could analyse them, and see why it is that
> we perform the way we do.
> 
> Laura
------- End of Forwarded Message