Suitability of Python for a Big Application?

[Boudewijn Rempt]

bobyu5 at mailcity.com wrote:

> This application has to run on multiple OSes, access various major
> relational databases, and internet enabled, meaning that it can FTP or
> send e-mails as part of its feature. Plus it has to be very easily
> extensible and modifiable because modification to the requirements will
> occur very frequently. Also, it has to be blazingly fast. Majority of
> the heavy duty programming will be done via database programming but
> tying up those stored procedures as well as supplying a sleek GUI will
> be the job of this application.

If you really need to use a variety of databases, then you probably
can't rely on stored procedures/packages to do the server-side work,
because there isn't a database where those work in the same way. And
even so, support for, say Oracle packages, is skimpy outside Oracles
own tools. I'm going to install Oracle next week to try whether they
can be accessed from Python, though.

You really need some middle man, if you go the various databases
route, and designing decent middleware is difficult, in my experience.

> And the most important of all, it has already been tried to be built
> using VB 5.0 with a miserable result. (2 years of effort has still not
> produced a usable application) All the issues that I have mentionned
> above are more or less critical.

Now that's recognizable... You aren't talking about the project I've
been working on myself for the past year, are you ;-).

<...>

> 1) GUI library: I tried to look at TK library and the look and feel was
> not as sleek as what comes with Windows; plus it felt very slow.

I don't think TK is really suitable - but if your application interface
is simple enough a web interface should suffice - and those can be as
slick as the graphics designer you hire can make it. Take a look at
WxPython or PyQt (that also seems to run under Windows).

> 2) Math operation: there is a possibility that some heavy duty
> calculation would have to be performed on around 100,000 rows of data
> using Python - how slow would this be?

Are you getting the data from the database? Then you want the processing
on the same machine as the data - even so, getting that many rows out
of a database will take a bit of time. A bit of experimentation with
numpy should allow you to determine whether the concept is going to work.

> 3) Heavy duty text processing using regexp (at least 40MB big)- I know
> Perl is really fast in this regard; is Python as fast?

I wouldn't know about this one.

> For 1) I thought I could solve this problem by using Zope - I get
> instantly a GUI that is based upon web browsers. This eliminates those
> annoying installation problems with customized DLLs as I found out using
> VB development approach.

I do find Zope a bit slow - even when I browse a Zope database on a
machine on my local network.

<...>

> Are my assumptions valid? Am I missing anything? Originally we wanted to
> have Outlook like UI - is this kind of UI possible to build using Python
> and its libraries?

I haven't ever seen Outlook, but a basic tree-to-the-left,
data-on-the-right interface isn't too difficult to make. If you are
prepared to get rid of the web interface, you could use WxPython, I
suppose, though that doesn't impress me as being ready for the enterprise
;-). Also, remember that there are very few database-ready widgets -
you will have to write the basics yourself.

I wonder whether you have enough developers who are proficient with
Python and all the other new technologies you want to use - Python is
good, but it isn't a panacea, and trusting a new tool to work miracles
is a recipe for disaster. I'm quite convinced that even VB can produce
good applications, in time, in budget ;-).

I'd say, write a small proof-of-concept app - with three or four
tables, a hundred meg of generated data and a crude interface, and
determine whether the tool will allow you to do what you want.

> Everybody seems to be using C++ or Java for projects of this scope; has
> anybody tried to do something similar using Python, or even Perl?

I've never seen C++ or Java used for large-scale database applications.
Most projects I've seen use Oracle Forms, Visual Basic or Powerbuilder
or things like that. But then, what's large? A large number of tables
and screens, a large amount of data, or both? And then, what's the
domain of the app? Transaction oriented, data warehousing, information
serving?

> Any anecdotes or recommendations would be heartily appreciated!

I'm trying to build a small-to-mid-range multi-user database application
using Python and PyKDE/PyQt as a gui. I figure people who want to use my
application can afford buying another PC to serve out X11 applications
to their legacy desktop systems (Windows and Mac). Besides, I want a
read-only browser interface. The architecture I'm working on includes
a database server, which I connect to using a DB-API II compliant
interface, a SQL translation layer, an application object layer, an
application server that dishes out XML-formatted data to the client gui
or to the client special purpose http server, that serves the various
browser clients. It's a hobby project, though. My bosses don't want
another programming tool in the shop, after the debacle with VB 5.0.

-- 

Boudewijn Rempt  | http://denden.conlang.org