Python Productivity Gain?

[Harry George]

"Terry Reedy" <tjreedy at udel.edu> writes:

> "Harry George" <harry.g.george at boeing.com> wrote in message
> news:xqxbrnvbhvk.fsf at cola2.ca.boeing.com...
> > It is common for a ComSci prof or grad student to crank up such a
> > study, using undergrad and grad students as the subjects.  These
> > subjects can generally be coerced to participate ("it is required for
> > the course").  For "novice programmer" research, high school students
> > are often used.  These tend to be self-selected, and ar not
> > representative for the general population.
> 
> The key feature that makes a study/experiment statistically analyzable is
> randomization of subjects to treatments.  So, to compare two languages
> (simplest case), you have everyone write programs in both languages, but
> randomize the order (half one way, the other half the other way) or you
> have one half do language A and the other half B, again randomizing the
> assigment.  Also, if there is any subjectivity in the evaluation of
> results, then the judges should not know the language when judging the
> output.
> 
> Do the studies you speak of meet these criteria?
> 
> Terry J. Reedy
> 
> 

The key is to use a valid experimental model in the first place.  All
the randomizing, DOE, double blind, ANOVA, discriminant analysis, factor
analysis, etc in the world doesn't help if the model is a poor
rendition of the intended target.  

I'm not in this field myself, and only know of it from scanning the
literature on language selection some time ago.  My impression was
that the academic papers using students as subjects did an honest job
of designing the experiment and probably got honest results for their
experimental population.  And they usually discussed their concerns
that the model was not very representative of industry practice.


> 
> 

-- 
harry.g.george at boeing.com
6-6M21 BCA CompArch Design Engineering
Phone: (425) 342-0007