Suggestions on storing, caching, querying json

[Sayth Renshaw]

On Thursday, 5 October 2017 15:13:43 UTC+11, Sayth Renshaw  wrote:
> HI
> 
> Looking for suggestions around json libraries. with Python. I am looking for suggestions around a long term solution to store and query json documents across many files.
> 
> I will be accessing an api and downloading approx 20 json files from an api a week. Having downloaded this year I have over 200 files already. So it will grow at a reasonable rate.
> 
> What I have initially done is store them into a mongo db. Now I am wondering if this is useful or prudent since other than querying the json I wont have much use of other mongo features.
> 
> When querying the json files though queries will utilise multiple json files at once, not just retrieving a single record. The usage is for data analysis.
> 
> Is there a good json storage option, with caching and optimal querying etc.
> 
> Regarding querying I did find a library for json searching called ObjectPath written in Python http://objectpath.org/reference.html
> 
> Looking to leverage your experience.
> 
> Cheers
> 
> Sayth

There is a new extension for redis ReJson and redis-py for using redis and python as a json store. http://rejson.io/ and https://github.com/andymccurdy/redis-py. Not sure if this has much more upside than mongo other than having a more fmailiar query language like JsonPath http://rejson.io/path/

Cheers

Sayth