Is Python overhyped (just like Java)?

[David Abrahams]

Alex Martelli <aleax at aleax.it> writes:

> Ajay na wrote:
>
>> Can some-one please explain why one would want to use Python?
>
> One word: *PRODUCTIVITY*.
>
>> What is wrong with C++?
>
> One word: *COMPLEXITY*.
>
>
>> In my opinion, the problem with C++ has nothing to with the language
>> semantics.  The problem is that when people are confronted with a
>> powerful language like C++, they tend to want to optimize everything
>> to death.  That's when they get themselves in trouble.
>
> You're over-generalizing.  Not all users of C++ are so naive as to
> have failed to read Knuth -- "Premature optimization is the root of
> all evil in programming".  But the point is -- by choosing a lower
> level language, like C++, at the start of your project, rather than a
> higher level one, like Python, you ARE optimizing WAY prematurely.

In my experience, it may be even more powerful to begin with hybrid
development from the ground up.  On one project, when I tried to start
with Python and push time-critical code into C++, I found that
although algorithm prototyping had been fast, the Python interface I
had developed for those critical components was not well-suited to the
C++ world.  In my application it was easy to predict where the bulk of
the time would be spent, and had I written the core engine in C++
early on I might have saved myself a redesign effort.  Another effect
of this approach is that you end up with a well-designed core C++
library that can be used by C++-only applications, instead of
something that really only works well with the Python wrapper.

> One example I give is a task for which C++ is quite suited, with its
> standard library -- reading a text file, breaking it into
> whitespace-separated 'words', building an index from each word to
> the line numbers on which it appears, and showing the index with
> words in alphabetical order, one per line, each followed by the line
> numbers on which it appears.

Boy, that sure sounds like a job for Python to me.  On the other hand,
I can't think of any C++-ish job that Python wouldn't be good for as
long as you're willing to spend some more cycles.


> Thanks to the excellent support given for these tasks by the standard
> library, the C++ source is ONLY twice as big as the Python source for
> the same job (a more usual ratio is around 5:1).  This holds for both
> the simplest versions, and the slightly more complicated ones with
> somewhat better optimization.  The runtimes on my box (Linux Mandrake
> 9.0, gcc 3.1, Python 2.3) are, when the programs are run on the 4.4 MB
> of the "King James Bible" in plain ASCII text: 17.4 seconds for the
> simplest C++, going down to 15 with optimizations; 11.2 seconds for
> the simplest Python, going down to 8.1 with optimizations (CPU times
> are in very similar ratios).  Of course, this basically reflects the
> excellence of Python's intrinsics (dictionaries, lists, strings)
> versus the lesser quality of C++'s library implementation (maps,
> vectors, strings) -- with different implementations, you may see
> different ratios.

Really?  Are you sure that time wasn't sunk into I/O?  "Read all the
lines from a file" is a primitive in Python, but not in C++.

Also, it's only fair to point out that C++ doesn't have one library
implementation -- each compiler comes with its own.  Which
implementation(s) were you testing?  Oh, I see it was probably
libstdc++ that comes with gcc 3.1.  Well, IIRC GCC-3.1 was well known
to optimize poorly (it took them some time to figure out how to
integrate all the new optimizations that came in with GCC-3, and they
actually made things slower for a while), and I wouldn't exactly say
that libstdc++ is the fastest implementation.

In other words, there's nothing intrinsic about C++ which says that it
has to be twice as slow as Python at this job.  I don't think it's
fair to talk about "the lesser quality of C++'s library
implementation."

Aren't you glad I'm asking these questions now, instead of at your
talk in Oxford? ;-)

<snip>

> The simplest and most flexible C++ you can write is still way
> bigger, more complicated, and less flexible than the most refined
> and optimized Python code it may make sense to write -- it's as
> simple as this.  

I think when you're comparing the use of the core language I have to
agree.  However, when it comes to functionality provided by libraries,
it really depends on the quality and availability of the library
interface.  That said, I think Python still has way more useful
libraries than C++ does.

> For tasks to which Python is extremely well suited (text processing
> of all kinds, including XML parsing, for example), you may ALSO get
> better running time than C++ with the standard library would give
> you -- in general, C++ lets you develop faster code, but oh what a
> price in terms of productivity you pay for that!
>
> And the funniest thing is, there IS no need to pay that price -- 90%
> of your program's runtime is likely to be taken up by 10% of your
> program's source code, or some such ratio.  By writing Python first,
> you'll often get an application with acceptable performance; if not,
> you profile it, find out the hot-spots, optimize those in Python
> terms, and if that's still not enough, it's EASY to recode the
> hot-spots in faster ways while still leaving MOST of your application
> in Python.  There are many ways to do such recoding (even without
> counting the still experimental 'psyco' selective just-in-time
> optimizer), and among them are ways to integrate C++, such as SciPy's
> "weave" and the Boost Python Library (if you don't know Boost, DO give
> it a look -- it WILL increase your C++ productivity, and not just by
> easing integration of C++ to Python, either).
>
>> people who don't understand C++, are afraid to use the 'virtual'
>> features of that language because it's 'too expensive'.  But that's
>> stupid...because Python's 'virtualness' is even more expensive!
>
> It's quite inconsiderate of you to imply that Python users are "people
> who don't understand C++", when among those users are people like
> Andrew Koenig (author of "Ruminations on C++" and other great C++
> books, as well as a towering figure of C++ development -- the
> algorithm for name lookup in the C++ language is called "Koenig
> lookup" because HE developed it...!), Bruce Eckel (author of
> best-sellers "Thinking in C++" and "Thinking in Java"), and so many
> others whose C++ competence is in all likelihood AT LEAST as good as
> yours.
>
>
>> Nope...I'm not trolling...I want someone to give a solid and
>> intelligent argument about why I should switch from C++ to Python!
>
> You shouldn't *SWITCH*!  If you hadn't already made the huge investment
> to master all the complexity of C++, it might be best to avoid it --
> but, if you HAVE made it, count it as a "sunk cost", as I do, and see
> how best to leverage it.  Learn Python (a trivial investment compared
> to learning C++), and use BOTH Python and C++ in your development.
> You'll find that Python gives you extremely high productivity and
> flexibility, letting you prototype, experiment, refactor whole
> architectures from the inside out and back again -- even if you knew
> the final product MUST be delivered in C++ due to contractual
> requirements, you'd STILL be better off doing the early phases in
> Python ANYWAY.  Once the program is working, benchmark it; most often,
> you'll find you're done -- in a FRACTION of the time.  If the
> performance isn't satisfactory, profile it, and start optimizing --
> including recoding parts in C++, e.g.  with weave or the Boost Python
> Library.  In the end, you'll use both Python and C++ in the same
> program, *each for what it does best*: Python for most of the code, C++
> for the bottlenecks.  It's as simple as this, really!
>
> And your programming productivity will soar...
>
>
> Alex
>

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com