(in memory) database

[Fredrik Lundh]

mark wrote:

 > I need to extract data from text files (~4 GB) on this data some
 > operations are performed like avg, max, min, group etc. The result is
 > formated and written in some other text files (some KB).

you could probably do all that with data stream processing, but if you 
haven't worked with such algorithms, just stuffing it all in a database
is probably less work for you (if not for your CPU).

> Unfortunately I have only some knowledge of SQLite which is not an
> option here.

why is sqlite not an option?  it's is bundled with Python these days, 
and should be available (or trivial to install) on all major deployment 
platforms.

</F>