Python Web Programming - looking for examples of solid high-traffic sites

[Alex Martelli]

Victor Kryukov <victor.kryukov at gmail.com> wrote:
   ...
> And although http://www.python.org/about/quotes/ lists many big names
> and wonderful examples, be want more details. E.g. our understanding
> is that Google uses python mostly for internal web-sites, and
> performance is far from perfect their. YouTube is an interesting
> example - anybody knows more details about that?

Hmmm, I do, but exactly because I _do_ know a lot more details I cannot
discuss them unless you're under the proper Non-Disclosure Agreement
with Google, Inc.  The best I can do otherwise is to point you to
already existing webpages -- I'm not going to reveal Google-Confidential
information, nor go to the substantial trouble to get such info cleared
by Legal and all other Google stakeholders.

For example, it HAS been published elsewhere that YouTube uses lighttpd,
not Apache: <http://trac.lighttpd.net/trac/wiki/PoweredByLighttpd>.

Fortunately, I managed to convince my YouTube colleagues to publically
present many more details about the evolution of their architecture
which had been discussed in Google-confidential talks and presentations
-- and my wife Anna, who's on the program selection committee of OSCON,
may have helped that talk get accepted (must not have been a hard
job:-).  See:
<http://conferences.oreillynet.com/cs/os2007/view/e_sess/13435>
 
I hope to see you (and anybody else interested in solid technical
details about using Python for websites on YouTube's scale) in Portland,
OR on July 26 -- that will also be your first and best chance to ask
Mike Solomon specific questions that his talk might not directly
address.  Once that's done, maybe somebody can convince the YouTube guys
to contribute a "Python Success Story" so that future querants about
this can be easily pointed to a URL!-)

I would also encourage anybody who's so keenly interested in Python to
visit our jobs-listing web app, e.g., if within the US, at
<http://www.google.com/support/jobs/bin/topic.py?loc_id=1100&dep_id=1173
&by_loc=1>.  Of course, one should *never* have the implementation
language of a web-app show up in the URL, and I believe we've carefully
avoided that error in other external-facing web-apps, such as (one I can
reveal is indeed in Python, because that was mentioned at
<http://www.sauria.com/~twl/conferences/pycon2005/20050325/Python%20at%2
0Google.html>) code.google.com.  Etc, etc.

Performance of web-apps (be they internal or external) depends more on
the architecture than on the implementation language (as long as highly
optimized C or C++, NOT Java or Python or whatever, is used for the few
jobs that are extremely CPU-intensive, such as codecs, of course:-).

So, if some Python-coded internal web-apps at Google perform badly
(which may be the case as you say, though I can't think of any
off-hand), it must be an issue of architecture.  For example, a heavily
used internal web-app at Google is Mondrian,
<http://www.niallkennedy.com/blog/archives/2006/11/google-mondrian.html>
, Guido van Rossum's web-app for code review -- it's got a good, solid
architecture, and its performance is so good that many Googlers, me
included, have switched to it for all the reviews we do (and, believe
me, we do MANY -- _nothing_ is submitted to the global code repository
until it's been OK'd in a code review).  I can't mention other such apps
because, AFAIK, they haven't been previously talked about in public and
so they're Google Confidential by default until otherwise determined.


Alex