Python web development, really

[Karsten W. Rohrbach]

Paul Rubin <phr-n2003b at NOSPAMnightsong.com> wrote:
> I'm not that crazy about threads either.  I don't understand why more
> multi-process servers don't use shared memory instead of disk files,

SHM is an API which might have differences from platform to platform.
File I/O should be quite the same in the POSIX world. Furthermore, files
*are* RAM, in case of enough RAM, so they are pretty fast. On most
platforms you can mmap() them and, together with working file locking,
use them as sahred memory with a pretty similar API but with better
performance (cf. libmm).

> or even (gak) SQL databases and worse.  Sometimes I wonder whether MPI

Abusing a RDBMS for cache data storage dosn't look like a good design
decision.

> (the pseudo-shared memory used in scientific clusters like Beowulf)
> might be an appropriate abstraction for programs like web servers.

MPI implementations are very (versatile|general)(?). In case of web
applications, a large number of problems is solved by separating
network I/O from execution logic from data I/O:
    Apache/cache/proxy[i] -> App Server[j] -> DB Server[k]
where ( j > i > k ), at least in my experience ;-)
In this scenario, state and session information and shared data is held
in the back end DB server, but other mechanisms are possible. Cache data
is held on-disk in the front end Apache systems, which practically
increases the overall amount of cached data proportional to the number
of frontend systems, but this buys us network performance, so I consider
it a Good Thing[tm].

> There are very efficient MPI implementations that let processes on the
> same machine communicate through shared memory, and can also let
> processes on separate machines communicate through a LAN.

It all depends on the "amount of interactivity" you need between your
application's components, and in what stage of processing. If there's
just session IDs, cookies, user names and such (speak: "joe web app"),
it's pretty simple to implement that as a data-centric model in the
backend. For other (pseudo-realtime or interactive) applications, a
networked shared memory model would make sense.

Regards,
/k