(in memory) database

[mark]

Hi there,

I need to extract data from text files (~4 GB) on this data some
operations are performed like avg, max, min, group etc. The result is
formated and written in some other text files (some KB).

I currently think about database tools might be suitable for this. I
would just write the import from the text files and ... the tool does
the rest. The only problem I can imagine is that this would not be
fast enough. But I would give it a shoot.
Unfortunately I have only some knowledge of SQLite which is not an
option here.

Some additional requirements I can think of are:
- Python (I want to hone my programming skills too)
- Python-only (no C-lib) for simplicity (installation, portability).
Therefore SQLite is not an option
- must be fast
- I like SQL (select a, b from ...) this would be nice (row[..] + ...
is a little hard getting used to)

So far I found PyDBLite, PyTables, Buzhug but they are difficult to
compare for a beginner.

Cheers,
Mark