best way to handle this in Python

[Ian Kelly]

On Fri, Jul 20, 2012 at 4:34 AM, Rita <rmorgan466 at gmail.com> wrote:
> Thats an interesting data structure Dennis. I will actually be running this
> type of query many times preferable in an ad-hoc environment. That makes it
> tough for sqlite3 since there will be several hundred thousand tuples.

Several hundred thousand is not an enormous number.  I think you're
underestimating sqlite3.  I just tried a test with one million tuples,
six colors per tuple (six million rows altogether).  Each row contains
a primary key, a timestamp, a color, and a count, with an index on the
timestamp column.  Building the database from scratch took about a
minute; adding the index took about another minute.  Incremental
updates would of course be much faster.  Queries like "select * from
data where timestamp between 500000 and 600000" return instantly (from
a user perspective).

Cheers,
Ian