Database vs Data Structure?

[M.-A. Lemburg]

On 2008-04-18 05:37, erikcw wrote:
> Hi,
> 
> I'm working on a web application where each user will be creating
> several "projects" in there account, each with 1,000-50,000 objects.
> Each object will consist of a unique name, an id, and some meta data.
> 
> The number of objects will grow and shrink as the user works with
> their project.
> 
> I'm trying to decided whether to store the objects in the database
> (each object gets it's own row) or to use some sort of data-structure
> (maybe nested dictionaries or a custom class) and store the pickled
> data-structure in a single row in the database (then unpickle the data
> and query in memory).
> 
> A few requirements:
> -Fast/scalable (web app)
> -able to query objects based on name and id.
> -will play nicely with versioning (undo/redo)
> 
> Any input on the best way to go?

Relational databases offer the best scalability and reliability,
so I'd go for those as backend.

If your data is mostly read-only and usually only touches
a small part of your database (e.g. the data for a day or two),
then you can increase query performance by keeping that part in
an in-memory database (e.g. use sqlite or Oracle TimesTen)
and only update the copy in case something changes in the
backend.

Regards,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 18 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611