Newbie: Crazy, but Quick

[Quinn Dunkan]

On Fri, 22 Feb 2002 09:23:24 -0800, Cliff Wells
<logiplexsoftware at earthlink.net> wrote:
>For that matter, you could also store the Python objects directly in the
>SQL database.  My own opinion is that SQL is so easy that it's a no-brainer
>to use it whenever possible.  A problem I've seen many times is people
>utilize some shortcut because they feel the program will never need
>anything more, only later to discover that the program has utility far
>beyond what they expected, but the shortcut they've taken cripples it, and
>they end up rewriting (or worse, they don't).

I don't find SQL to be a no-brainer.  You have to download the thing (MySQL at
least is massive... the sunos binary tarball I have is >5MB).  Then you have to
install it (create tables, blah blah, learn MySQL's complicated permission
system, etc.).  Then you have to turn it on (which means either getting root to
edit the system rcs or setting up some hackery to restart it if the server
reboots).  Not to mention that you have to learn enough SQL to do all this.
Oh yes, and if there are no binaries for your platform, I hope you happen to
have a decent C++ compiler lying around.

And even if it is easy to introduce giant complicated support program, should
you?  If linking in a massive shared library (or *any* shared library) is as
easy as tacking on a '-lpiggy', is it a no-brainer to link it in whenever
possible?  I don't think so.  How easy it is for the developer to create
something is only part of the equation (that's what the python "creed" of
legibility and simplicity is about, yes?).


Of course, you're right that you can easily marshal python objects to SQL, I
assume by pickling to a string.  But then you lose the "it's not python
specific" advantage of SQL.  And you still have to scrunch your data into the
relational rows/columns model.  What if it's hierarchical and lumpy?


>What if she decides to maintain historical data?  Then to keep the files
>reasonably sized, she'll have to start juggling files.

I think the *dbm type stuff scales ok up to mediumish files at least (there are
a few > 50MB gdbms lying around, INN uses it for article index or something).
I have no idea if it's happy all the way up to a few gigs.  If I wanted to use
it for that much data I'd test it out.

>Software has a tendency to grow beyond the developer's expectations, so I
>consider it unwise to take shortcuts when a better long-term solution isn't
>much more difficult to implement.

That's one point of view, and a reasonable one.  Another point of view is that
you should only design for what you know you need, instead of increasing
generality and complexity to deal with some hypothetical future situation.
If you think your program has acquired a new requirement, you decide if
it's really a requirement or if it should be done by another program, and if
it is, you modify your program, which hopefully is easy to modify since you've
been trying to keep things simple.

In the shelve case, it would be pretty easy to write an SQL-using backend
that implements the shelve interface.  And maybe you'll like that interface
better than the SQL-ish cur.execute(this_and_that) anyway.

>Not only that, but storing the data in a SQL database separates the data
>storage from the program, so in the future she could use that data from
>some other program, not just the Python one.  Then if she decides she wants
>to access the data from, say, a web server, it's not a big problem. 

You can write CGI scripts in python :)