Is Python suitable for a huge, enterprise size app?

[Dave Brueck]

Paul Rubin wrote:
> "Kay Schluehr" <kay.schluehr at gmx.net> writes:
>>To answer Your initial question: there is probably no technical reason
>>against Python as a language or the CPython runtime. Both are very
>>stable and mature.
> 
> 
> I'd like to agree with this but I just can't.  Python is a great
> language for prototyping and trying stuff out.  It's fast and pleasant
> to write.  But you're right when you say that constant testing and
> integration is absolutely necessary to develop anything substantial in
> Python.
[snip]
> I'm in the middle of prototyping something in Python and it's going
> pretty well despite occasional snags like those mentioned above.  But
> at the end of the day I don't think what we get will be solid enough
> for use in production.  We're planning to use the Python prototype to
> get the feature set and protocols figured out, then reimplement in Java.  
> The OP might consider that approach.

You bring up some good points and I imagine the bugs you've encountered are 
pretty frustrating, but I'm surprised at how different our experience has been. 
I know my comments about Python probably come across a "total fan boy" - but for 
us Python has been extremely positive.

We currently use Python, in production, in the following ways:

- custom HTTP web servers
- custom HTTP proxies
- behind-the-scenes log processors and other operational tools
- several Zope-based applications, both internally and externally facing
- both the client and server sides of a distributed work cluster
- the client side of a media encoding farm (server is a Zope app)
- a downloadable Windows client that our customer's customers use, that includes 
a Windows COM server (an ActiveX control we implemented using ctypes).

Most of these have been in production for a year or more, with the oldest being 
about 4 years (not a long time, but certainly long enough to be considered 
stable). The toughest pieces initially were the Zope apps and the Windows COM 
stuff - Zope because of documentation and COM because, well, it's COM.

During that time, the biggest problem we had with Python and its standard 
libraries was one issue with socket.recv and memory allocation when doing lots 
of receives. That lost us a few days of hunting for a memory leak, but we were 
able to resolve it by looking at the C source. Another problem was the need to 
call unlink (or something like that) on XML minidom elements in some 
circumstances. Apart from that, we haven't encountered much of anything in terms 
of bugs in the language or the standard library in *years*. In fact, I don't 
recall encountering a bug in the language itself.

We've generally trailed the standard Python releases by about a year (although 
this week I moved the encoding farm to Python 2.4.1, which is half a year 
early). We don't use Tkinter. We don't use much outside the standard library: we 
make heavy use of ctypes, psycopg (Postgres), and lately CherryPy, but that's 
about it. So for us, the number of bugs in Python + stdlib has been really low.

One thing from your experience that did resonate with me is that, except for 
ftplib and occasionally urllib (for basic, one-shot GETs), we don't use any of 
the standard library's "protocol" modules - partly because we had to implement 
our own HTTP libraries for performance and scalability reasons anyway, and 
partly because we had trouble figuring out e.g. all the ins and outs of 
urllib/urllib2/httplib.

Overall it's been such a positive experience for us that nobody in the company - 
from grunt testers up to the CTO - has any reservations about using Python in 
production anymore (even though initially they all did). All of the developers 
have previous experience with using Java in production systems, and none 
seriously consider it for new projects at all.

-Dave