Decline and fall of scripting languages ?

[Paul Rubin]

bokr at oz.net (Bengt Richter) writes:
> >built-in concurrency support.  OCaml seems to crush Haskell and Erlang
> >(and even Java) in performance.  Occam isn't used for much practical
> >any more, but takes a purist approach to concurrency that seems worth
> >studying.
> IIRC, I've seen something about a web server implemented in erlang
> with tremendous performance at high levels of concurrency where other
> implementations bog down. So I would want further details before I
> believe that all OCaml versions "crush" all Erlang versions in performance.

Good point you and Donn make about shootouts.  However, the shootout
figure differences in these particular cases are vast, and nature of
the implementations seems to make that result inevitable.  I've been
looking a little more at the docs of the languages (haven't written
any code yet).  Erlang is sort of like Python: dynamically typed, and
compiled to bytecode that gets interpreted.  Haskell is statically
typed and has apparently has a real compiler, but uses lazy evaluation
for everything, which if done in the obvious way (I don't know how
they actually do it), there's a closure wrapped around every value.
Also, the language itself seems to do a lot of stuff for you behind
the scenes, almost like Prolog.  In that sense it's really a higher
level language than normal imperative languages, so I can see why
people like programming in it.  But it doesn't make for the fastest
machine code.  OCaml seems to be sort of like Lisp with static types,
and syntax geared towards functional-style programming.  In that sense
it's maybe less high-level than Haskell.  But it means standard Lisp
compiler techniques can make excellent output code, seriously
competing with C/C++ compilers.  There's an SML compiler (MLton) that
seems to significantly beat C++ for scientific number crunching speed
a lot of the time.

> I'd suggest finding that Erlang web server writeup.
> Hm, some googling ...
> I think this is the graph I saw:
>     http://www.sics.se/~joe/apachevsyaws.html
> 
> Can't vouch for what it means, but taken at face value
> would seem to warrant a further look into details.

That looks like purely a test of how many simultaneous connections the
server can handle.  They had each client connect to the server, read
one byte, then hold the connection open and do nothing.  Apache uses a
separate process or OS thread for each connection, so it wedged once a
few thousand requests were open.  Erlang apparently uses microthreads,
probably allocating every call frame on the heap like SML/NJ did, so
they showed it with 80,000 connections open.  The key, though, is that
the clients were doing basically nothing.  It just meant that many
small data structures were allocated in memory of one server process.
It doesn't say anything about how fast the server can actually
generate and serve dynamic content.

The Mnesia database looks interesting and I'm going to have to read
about it further and figure out if something like it can be done in
Python.

I realized that my server will spend a lot of cycles in mod_gzip or
the equivalent.  So I think a (maybe not achievable) performance goal
is for the web app to use 50% of the available cycles making html, and
the other 50% go to gzipping the html.  That means that the app should
make dynamic output as fast as gzip can compress it, which is at least
several megabytes per second.  (All static content would be stored
pre-gzipped and served by khttpd (Linux kernel httpd) without the
dynamic app ever seeing the request).  I think that requires compiled
code and use of an in-process database (no remote SQL) for almost all
requests.  It's an order of magnitude faster than the crap that almost
everyone is running on real sites.

> This turned up also
>     http://eddie.sourceforge.net/txt/WP_1.0.html

Oh, this is nice.  I'm thinking of an architecture like OK Web Server
(http://okws.org) but without (ugh!) C++, letting the kernel httpd
serve static content as mentioned.

> I guess this is a home for erlang:
>     http://www.erlang.se/

That's the commercial site, the free stuff is at www.erlang.org.

> Much other stuff, but you know how to google ;-)

Yes, thanks ;-).