Python syntax in Lisp and Scheme

[Doug Tolton]

Andrew Dalke wrote:
> Doug Tolton:
> 
>>Yes and I have repeatedly stated that I disagree with it.  I simply do
>>not by that allowing expressiveness via high level constructs detracts
>>from the effectiveness of the group.  That argument is plainly
>>ridiculous, if it were true then Python would be worse than Java,
>>because Python is *far* more expressive.
> 
> 
> I disagree with your summary.  Compare:
> 
>   The argument is that expressive power for a single developer can, for
>   a group of developers and especially those comprised of people with
>   different skill sets and mixed expertise, reduce the overall effectiveness
>   of the group.
> 
> Notice the "can".   Now your summary is:
> 
>   ...allowing expressiveness via high level constructs detracts
>   from the effectiveness of the group
> 
> That implies that at least I assert that *all* high level constructs
> detract from group effectiveness, when clearly I am not saying
> that.
> 
> 
> 
>>>If this is indeed the crux, then any justification which says "my brain"
>>>and "I" is suspect, because that explicitly ignores the argument.
>>
>>Apparently you can't read very well.  I simply stated that I believe our
>>point of contention to be that issue, I never stated I believe that
>>because it's some vague theory inside my head.
> 
> 
> Nor can you, because I did not say that.  I said that the arguments you
> use to justify your assertions could be stronger if you were to include
> cases in your history and experience which show that you understand
> the impacts of a language feature on both improving and detracting from
> a group effort.  Since you do have that experience, bring it up.  But
> since your arguments are usually along the lines of "taking tools out
> of your hands", they carry less weight for this topic.
> 
> (Ambiguity clarification: "your hands" is meant as 2nd person singular
> possessive and not 2nd person plural. :)
> 
> 
>>No, what I was referring to wasn't estimation.  Rather I was referring
>>to the study that found that programmers on average write the same
>>number of lines of code per year regardless of the language they write
>>in.
> 
> 
> McConnell's book has the same study, with outliers for assembly
> and APL.  Indeed, I mentioned this in my reply:
> 
>>>... and that LOC has a
>>>good correlation to development time, excluding extremes like APL
>>>and assembly.
> 
> 
>> Therefore the only way to increase productivity is to write
>>software in a language that uses less lines to accomplish something
>>productive.  See Paul Grahams site for a discussion.
> 
> 
> I assume you refer to "Succinctness is Power" at
>   http://www.paulgraham.com/power.html
> 
> It does not make as strong a case as you state here.  It argues
> that "succintness == power" but doesn't make any statement
> about how much more succinct Lisp is over Python.  He doesn't
> like Paul Prescod's statement, but there's nothing to say that
> Python can't be both easier to read and more succinct.  (I am
> not making that claim, only pointing out that that essay is pure
> commentary.)
> 
> Note also that it says nothing about group productivity.
> If it takes me 5% longer to write a program in language X
> then language Y, but where I can more easily use code and
> libraries developed by others then it might be a good choice
> for me to use a slightly less succinct language.
> 
> Why don't people use APL/J/K with it's succinctness?
> 
> I also disagree with Graham's statement:
> 
>>the most accurate measure of the relative power of
>>programming languages might be the percentage of
>>people who know the language who will take any job
>>where they get to use that language, regardless of the
>>application domain.
> 
> 
> I develop software for computational life sciences.  I would
> do so in Perl, C++, Java, even Javascript because I find
> the domain to be very interesting.  I would need to be very
> low on money to work in, say, accounting software, even if
> I had the choice of using Python.
> 
> 
> 
>>You are saying that Python and Perl are similarly compact?!?
>>You have got to be kidding right?
>>Perl is *far* more compact than Python is.  That is just ludicrous.
> 
> 
> Yes.  In this I have a large body of expertise by which to compare
> things.  Perl dominates bioinformatics sofware development, and the
> equivalent Python code is quite comparable in side -- I argue that
> Python is easier to understand, but it's still about the same size.
> 
> 
>>It's always nice just to chuck some arbitrary table into the
>>conversation which conveniently backs some poitn you were trying to
>>make, and also conveniently can't be located for anyone to check the
>>methodology.
> 
> 
> "Can't be located"!?!?!  I gave a full reference to the secondary material,
> included the full quote (with no trimming to bias the table more my way),
> gave the context to describe the headings, and gave you a reference
> to the primary source!  And I made every reasonable effort to find both
> sources online.
> 
> Since you can't be suggesting that I tracked down and destroyed
> every copy of McConnell's book and of the primary literature (to make
> it truely unlocatable) then what's your real complaint?  That things exist
> in the world which aren't accessible via the web?  And how is that my
> fault?
> 
> 
>>If you want some real world numbers on program length check here:
>>http://www.bagley.org/~doug/shootout/
> 
> 
> If I want some real world numbers on program length, I do it myself:
>   http://pleac.sourceforge.net/
> I wrote most of the Python code there
> 
> Still, since you insist, I went to the scorecard page and changed
> the weights to give LOC a multipler of 1 and the others a multiplier
> of 0.  This is your definition of succinctness, yes?  This table
> is sorted (I think) by least LOC to most.
> 
>             SCORES
> Language Implementation Score Missing
> Ocaml       ocaml        584     0
> Ocaml       ocamlb       584     0
> Ruby        ruby         582     0
> Scheme      guile        578     0
> Python      python       559     0
> Pike        pike         556     0
> Perl        perl         556     0
> Common Lisp cmucl        514     0
> Scheme      bigloo       506     1
> Lua         lua          492     2
> Tcl         tcl          478     3
> Java        java         468     0
> Awk         mawk         457     6
> Awk         gawk         457     6
> Forth       gforth       449     2
> Icon        icon         437     7
> C++         g++          435     0
> Lisp        rep          427     3
> Haskell     ghc          413     5
> Javascript  njs          396     5
> Erlang      erlang       369     8
> PHP         php          347     9
> Emacs Lisp  xemacs       331     9
> C           gcc          315     0
> SML         mlton        284     0
> Mercury     mercury      273     8
> Bash        bash         264     14
> Forth       bigforth     264     10
> SML         smlnj        256     0
> Eiffel      se           193     4
> Scheme      stalin       131     17
> 
> So:
>   - Why aren't you using Ocaml?
>   - Why is Scheme at the top *and* bottom of the list?
>   - Python is right up there with the Lisp/Scheme languages
>   -  ... and with Perl.
> 
> Isn't that conclusion in contradiction to your statements
> that 1) "Perl is *far* more compact than Python is" and 2)
> the implicit one that Lisp is significantly more succinct than
> Python?  (As you say, these are small projects .. but you did
> point out this site so implied it had some relevance.)
> 
> 
>>I just don't buy these numbers or the chart from Mcconell on faith.  I
>>would have to see his methodolgy, and understand what his motivation in
>>conducting the test was.
> 
> 
> I invite you to dig up the original paper (which wasn't McConnell)
> and enlighten us.  Until then, I am as free to agree with McConnell --
> more so because his book is quite good and comprehensive with
> sound arguments comparing and contrasting the different
> approaches and with no strong hidden agenda that I can detect.
> 
> 
>>It still wasn't relevant to Macros.  However, because neither of you
>>understand Macros, you of course think it is relevant.
> 
> 
> My lack of knowledge not withstanding, the question I pose to
> you is, in three parts:
>   - is it possible for a language feature to make a single programmer
>        more expressive/powerful while hindering group projects?
Yes I believe this to be the case.  However in my own experience even 
working with language such as Visual Basic and Java (which are far less 
expressive than Python), people give me code that is so obfuscated that 
is could compete in the Perl contenst.

In my experience, it hasn't been expressiveness per se that caused the 
most problems.  It has been lack of familiarity with sound software 
engineering concepts, or more specific lack of experience building real 
world applications.

So the short answer, is that *any* operator / feature used incorrectly 
can cause massive confusion.  I've seen this with simple operators such 
as loop (ever seen seven nested loops doing different things at 
different levels?  It's can be ugly)
>   - can you list three examples of situations where that's occured?
Hmm, does everythime I've read someone elses code count? ;)
In seriousness, I have yet to be on any serious project where someone 
doesn't do something that I disagree with.  Personally though, I haven't 
run across a problem where a cleanly implemented abstraction (ie class, 
macro, HOF or metaclass) has caused a loss of productivity.  In my 
experience it has been quite the opposite.

Most of the development teams that I've worked on have gravitated 
towards two groups.  Those who write utilities and substrates for the 
development framework, and those who consume them.  This has happened 
even if not specified by management, simply because those with the 
ability to write reusable abstractions end up doing it a lot.  I have 
personally seen on numerous occaisions development speed up greatly when 
the proper high level constructs were in place.

>   - can you list one example where the increased flexibility was, in
>        general, a bad idea?  That is, was there a language which would
>        have been better without a language feature.
I don't necessarily believe that to be the case.  Certainly I can list 
cases where utilizing a certain feature for a certain problem has been a 
bad idea.  That doesn't general to the language would be better without 
the feature though.  For that to be the case, IMO, there would have to 
be *no* redeaming value to the feature, or it's use would have to be so 
massively problematic that it nearly always causes problems.

I can't think of any feature off hand where I would say "take it out of 
the language, that's just stupid".  Perhaps there are some, and I'm just 
missing them while I'm thinking about it.

One example of mis-use that caused some serious headaches:
Back in 1999 I was lead on a team building a heavy duty enterprise web 
application.  Management decided that our best choice was to use Visual 
Basic and MTS.  The system had to be scalable, it had to be extremely 
fault tolerant and it had to be very flexible.  The architecture 
initially decided upon was to have three web servers, two application 
servers and a fully fault tolerant sql server.

Based on the initial reports from MS we decided to test DCOM from the 
web servers to the application servers (remember when that was the big 
fad?).  We quickly found out that our performance was terrible, and 
couldn't scale to support our minimum required users.  Switching things 
around we re-configured and went with five web servers each running the 
MTS components locally.

Another problem we ran into was that we decided to test out the XML 
hype.  All of our messaging between objects and between systems was sent 
via XML payloads.  This turned out to be extremely slow, and we ended up 
ripping out most of the XML messaging guts in order to spead up the system.

We also encountered serious problems with people not knowing how to 
efficiently utilize a SQL Server.  For instance they would get a 
recordset from each table (rather than joining) and then loop through 
each recordset comparing the values and constructing their resultset. 
Rewriting the queries to properly utilize joins and where clauses 
yielded several orders of magnitude performance increases.
> 
> Note that I did not at all make reference to macros.  Your statements
> to date suggest that your answer to the first is "no."
> 
That's not exactly my position, rather my position is that just about 
anything can and will be abused in some way shape or fashion.  It's a 
simple fact of working in teams.  However I would rather err on the side 
of abstractability and re-usability than on the side of forced restrictions.