Python database of plain text editable by notepad or vi

[Jon Clements]

On 26 Mar, 09:49, James Harris <james.harri... at googlemail.com> wrote:
> On 25 Mar, 22:56, Jon Clements <jon... at googlemail.com> wrote:
>
>
>
> > On 25 Mar, 22:40, James Harris <james.harri... at googlemail.com> wrote:
>
> > > I am looking to store named pieces of text in a form that can be
> > > edited by a standard editor such as notepad (under Windows) or vi
> > > (under Unix) and then pulled into Python as needed. The usual record
> > > locking and transactions of databases are not required.
>
> > > Another way to look at it is to treat the separate files as entries in
> > > a dictionary. The file name would be the key and the lines of the file
> > > the value.
>
> > > Anyone know of a database (with a Python interface) which will allow
> > > text files to be treated as database fields? If not I can just write
> > > it but I thought it best to ask if there was an existing solution
> > > first.
> ...
> > I could be missing something here, but aren't you basically just
> > talking about an OS's filesystem?
>
> For storage, yes. The files would be marked-up text stored in the
> filesystem. The "dbms" (I use the term loosely!) would provide access
> to them by some full or partial key mechanism yet to be determined.
> Read-only access would do - at least for now.
>
> > glob or listdir somewhere, then create a dict using the file contents
> > would meet your criteria, with very little lines of code -- but I
> > would be interested to know what the use-case was for this... Is it
> > read completely at start up time, or if each file contains a large
> > amount of lines and aren't fixed width (or has no other indexing
> > support without maintenance), then is a complete sequential-scan
> > required each time, or do you just tell the user to not update when
> > running (unless I s'pose something along the lines of a SIGHUP for
> > config files is applicable).
>
> All good questions. For now, at least, the files can be read-only and
> I'd want those on disk to be the master copies at all times. If I was
> writing it myself I'd probably 'cache' some files in memory and stat
> them before use. If newer I would reread the file.
>

It's hard to bore this group :)

>
>
> > Sorry, just don't understand why you'd want this.
>
> I tried to avoid boring folks with the details. I'm toying with some
> ideas for a way to help generate source code (in various languages,
> not just Python). If it goes ahead the text files would be mainly
> marked-up code snippets - with or without symbols that need to be
> replaced.
>
> Rather than write one single monolithic app I thought to split it into
> reusable components. One part being data access could perhaps be an
> existing database (and I'll take a look at jkn's suggestion).
>
> Think of the database as similar to an associative array stored on
> disk. The only difference is I may want to play fast and loose with
> the keys in some ways - e.g. check for partial key matches or return a
> list of part-matched keys. The language name could be part of the key
> but I'd also need to store variants for specific language versions.
> I'm not sure yet how it will all pan out. As I say, just throwing
> around some ideas.
>
> James

Thanks for the explanation.

Who admins and, who's editing this data?

I couldn't 100% guarantee that I could modify a text file and always
put the right
delimiter in the right place and remember to escape the relevant chars
(and I'm
probably not the 'average' user).

Any opposition to just putting it in a 'proper' DB, then 'blobbing'
the values?
(or just integrate a procedure/script/function whatever your chosen
RDBMS calls to choose it).
Or in some systems, 'externally referencing'... loads of DB's have
free front-ends,
and there are lots of Python libraries.

I think perhaps, all I'm saying is, I'd choose a different approach.
I'd provide a front-end, rather than choose to re-write the wheel over
DB's.

Be nice to know how you get on, if you'd be so kind?

Cheers,

Jon.