From coreybrett at gmail.com  Mon Apr 11 10:07:23 2022
From: coreybrett at gmail.com (Corey Boyle)
Date: Mon, 11 Apr 2022 10:07:23 -0400
Subject: [Flask] temporary database
Message-ID: <CAD=ssw0dhkHd9xb=rKTrgSctw7aO5xLSgLZGhz2GZArO-ty=YQ@mail.gmail.com>

Curious if anyone has any suggestions for the following.

I have a "database" of about 96k rows that I want to dump from an old
legacy business system and import into my Flask app.

The data is approximately 5MB when stored as a CSV.

I would like to do this dump once a day (maybe more often if practical).

Users of the Flask app will only need to read/search the data.

I already have a Postgres DB (using SQLAlchemy) that I could use, but
that doesn't seem like the best option, since I would be clearing the
entire table, and reloading all the data every time I run an update.

Redis seems like a good idea, but if I'm not mistaken, I could only
search the keys, not the values.

Would something like MongoDB be a good fit?

Something where all the data is stored in-memory instead of disk would be ideal.

From ziirish at ziirish.info  Mon Apr 11 10:13:33 2022
From: ziirish at ziirish.info (Benjamin Ziirish SANS)
Date: Mon, 11 Apr 2022 16:13:33 +0200
Subject: [Flask] temporary database
In-Reply-To: <CAD=ssw0dhkHd9xb=rKTrgSctw7aO5xLSgLZGhz2GZArO-ty=YQ@mail.gmail.com>
References: <CAD=ssw0dhkHd9xb=rKTrgSctw7aO5xLSgLZGhz2GZArO-ty=YQ@mail.gmail.com>
Message-ID: <20220411141333.frbkj7jxxk2yoc5m@uphira.ziirish.info>

* On Monday, April 11, 2022 at 10:07 AM -0400, Corey Boyle <coreybrett at gmail.com> wrote:
> Curious if anyone has any suggestions for the following.
> 
> I have a "database" of about 96k rows that I want to dump from an old
> legacy business system and import into my Flask app.
> 
> The data is approximately 5MB when stored as a CSV.
> 
> I would like to do this dump once a day (maybe more often if practical).
> 
> Users of the Flask app will only need to read/search the data.
> 
> I already have a Postgres DB (using SQLAlchemy) that I could use, but
> that doesn't seem like the best option, since I would be clearing the
> entire table, and reloading all the data every time I run an update.
> 
> Redis seems like a good idea, but if I'm not mistaken, I could only
> search the keys, not the values.
> 
> Would something like MongoDB be a good fit?
> 
> Something where all the data is stored in-memory instead of disk would be ideal.

Well, if the data fits in 5MB and you'd like some kind of in-memory database,
you might have a look at SQLite's in-memory databases:
https://www.sqlite.org/inmemorydb.html

If you don't need any RDBMS feature, why not playing with a good old Python dict
that you would populate from your dump above as a JSON or something like that?