Why isn't Python king of the hill?

[Martijn Faassen]

Geoffrey  Gerrietts <geoff at homegain.com> wrote:

> I've spent a lot of time trying to come up with a response to
> Martijn's comments and questions.

I didn't realize just a few of my curious questions would have this
effect. I'm interested in scalability issues, as I hope to be running
into them eventually. (yeah, I meant that; if I run into them that means
business is going well :).
 
> It's taken me more time than I
> thought it would. My first cut was approaching a novel in length.
> I'm going to try to restrain myself to the salient points this
> time. (Still didn't do all that good a job, did I?)

Read with interest.

> Martijn writes:
>> [ deletia ]
>> How does Zope's ZEO fit in this picture? I would think it'd
>> help scale a Python app across a lot of boxes, right? Zope of
>> course has a persistence toolkit; the ZODB. RPC mechanisms
>> exist for Zope as well; for instance XML-RPC.  How were these
>> not satisfactory? 

> We use ZEO to centralize our actual Zope objects. Business
> objects, the core of our dynamic model, are managed outside Zope.
> In Zope, we have images and DTML content, with a few wrapper
> objects around external entities. We use an authentication scheme
> that uses Zope's ACL mechanisms but stores user records in a SQL
> database.

How does Zope interact with the business objects? Through external
methods?

> I've looked at ZODB and ZEO some for my personal edification. If
> I'm not mistaken, they provide a great mechanism for object
> persistence with key-based retrieval. My evaluation suggested
> that ZODB and ZEO were not very good at handling arbitrary
> retrieval, or even retrieval at the level of a SQL select
> statement.

What do you mean with 'arbitrary retrieval'? I agree SQL select-like
retrieval still isn't doable very well with the ZODB; the relational
paradigm does well here, and accomplishing this kind of thing with the ZODB
seems to be difficult. The ZCatalog can help here, and the underlying
BTree implementation has been rewritten to decrease some bottlenecks
recently, but my experience with it is extremely limited; I can't
'think' Catalog yet when I think about my problems.

> I don't think we should expect them to do that, but
> that's a very real requirement for much or most business data.

Agreed; the ZODB isn't good at lots of writes either, and its paradigm
is so different from the relational one a relational style application
shouldn't be written in it. There may be other approaches with the
ZODB to accomplish the same things, but I rather like the data being
in a relational database, and don't feel up to too much experimenting
with critical data. Eventually the ZODB may get there and I'll be
watching it. Also interesting is recent thought going into improved
integration of Zope with relational databases.

> I don't know a whole lot about Zope and XML-RPC. I've read bits
> and pieces on one site or another. It looks pretty easy to use
> when Zope is your XML-RPC server, and it looks pretty easy to use
> when you're on the client end of an XML-RPC call. 

I've played with this a bit and it's pretty trivial; almost the same
as calling into the ZODB directly, though with some important limitations
(I believe XML-RPC can't do very much with objects, such as object
parameters). SOAP will also be interesting in this area, but 
I don't think Zope speaks this yet and I haven't played with it.

> Developing a
> server seems only slightly more complicated, and looks to be
> something that could be automated or abstracted fairly readily.

> We use ILU in the place where XML-RPC would be. I honestly can't
> give you a compare-contrast as to how the two crack up against
> one another, or why we picked ILU. I suspect that we use ILU
> because it was available and mature when the site was being
> pulled together. Maybe one of the original architects had some
> good reusable code generation tools to use with ILU.

> I think our deployment benefits from ILU's directory-based
> approach. An ILU server publishes where it's running so that
> clients can connect to it; that adds a little fault-tolerance and
> scalability to the design. On the other hand, ILU's pretty rigid
> about a lot of things, so in a few specific spots in our
> application, we've sacrificed some of the freedoms that generally
> come with programming in Python.

I suppose this could be somewhat comparable with running CORBA 
servers, right? I don't have much experience in this area, though
I'm very curious to hear more about how you integrate Zope with 
this.

> Now I've answered the questions (I think fairly), but I'd like to
> take a step back and re-frame my comments.

> I like Python and I like programming in Python. I took this job
> because I would get to program in Python. Some of my peers don't
> share my enthusiasm, which is a shame, but I think they're
> learning to appreciate it.

> When I think about designing a webapp, though, I don't think
> about an app like the one I'm working on now. I think about an
> app I worked on a couple years back, one that really deserved to
> be a webapp but wasn't.

> That app ran out of an office of a half-dozen fairly tech-savvy
> librarians. They knew web stuff, but did not know programming.
> They had to hire consultants to develop for them, and hire
> consultants to do system administration for them.

> When I think about designing a webapp, this is the webapp I think
> about -- the one I never got to write. For the last year or year
> and a half, that mental design has been done in Java. In moments
> of weakness, I think that I could get away with using Jython, but
> in the end I come back to straight Java. The Java toolkit won't
> make development any faster, but it does have other compelling
> features.

> Some of the features are either lacking or hidden to my limited
> vision in Python, notably decimal math and PDF generation.

Python has reportlab for PDF, which seems to work pretty well:

http://www.reportlab.com

I believe there's a PEP about introducing decimal math into the
Python core as the default behavior eventually; there's a thread elsewhere
on the newsgroup. I'm not very familiar with these issues, though.
Do you need decimal math to avoid the subtle rounding errors and such
inherent in floats, when doing currency calculations and such?

> There's a certain draw to being able to take a single webapp and
> deploy it on any of a dozen different app servers that know how
> to handle the content, though it's small.

What do you mean by this, exactly? Are you referring to the whole
Enterprise JavaBeans framework (sorry if I confuse any of the
terminology here) that's been implemented by various vendors? I've
always been skeptical about how well that actually works, but here
I'm skeptical without any actual real knowledge. :)

> But the biggest draw is
> that someone who's proficient with Java is not too hard to find.

Are you sure about this? I mean, sure, lots of people are being cranked
out that can use Java to a certain extent, but is it really true
that it's not hard to find someone *good* at it?

> Someone who's proficient with Python is considerably harder to
> lay hands on.

But I'd think that anyone who's actually good at Java would be 
retrainable to use Python in a matter of days. I was okay with C++
and various other languages at the time, and picked it up in no
time, and from what I've read here over the years I'm sure my story isn't
the only one. 

I'll grant you the use of Zope may be different. :) Then again, I may
be too skeptical about this; I've seen a non-programmer but web-savvy
person go from absolutely no Zope *or* Python *or* SQL knowledge last october
to maintaining and extending a web app I wrote back then, right now.
He's fiddling with the DTML, adding stuff here and there. He's added
a bit of SQL too, and knows how to exploit acquisition. He's now moving
into bits of Python. There's a lot he doesn't understand yet, but
he sure is effective. I wonder if that's a purely isolated case or if this
happens more often. I've seen something similar happen with a few others
though (though they did have more programming experience), so Zope may
not be as confounding as think. :)

> Furthermore, while performance isn't a huge deal to those folks,
> their app did show signs of growing to cover a lot more terrain.
> It's conceivable that once the app was turned loose, it really
> would need to worry about the kind of massive load that starts to
> show off Python's seams.

Are Java's seams that much tighter? I know it's faster, on average,
though it does seem to consume oodles of memory. And then there's the
question about closing the seams; it would seem to me that this would
be easier to do in Python in many situations, because it's just a lot
more flexible.

> I think that part of the issue here is that when I think about a
> webapp, I don't think about an app that will be written and won't
> change. I don't think about working on the same app for the rest
> of my life, or even for more than a couple years. And I think
> about something that's continually growing, and I plan as if it
> will be as cool to the rest of the world as I think it is.

Any larger app will have to grow and change over time, so I consider
this as well for my Zope/Python/relational database apps, which is
why I'm interested in your ideas about what makes Java more suitable
for these purposes. Not so I can go and use Java, but for stealing
some ideas. :)

> For more modest ambitions, the factors that weigh heavy on me may
> seem irrelevant or even overweening. :)

Many people are using Python for rather less modest ambitions, however. 
I'm certainly planning to. :)

> Meanwhile, I'll build all
> my administration tools, glue apps, and prototypes in Python. :)

Good!

Thanks for the interesting posting,

Martijn
-- 
History of the 20th Century: WW1, WW2, WW3?
No, WWW -- Could we be going in the right direction?