[Tutor] edit JSON or file in real time

[Dennis Lee Bieber]

On Wed, 29 Sep 2021 17:59:52 +0200, Julius Hamilton
<juliushamilton100 at gmail.com> declaimed the following:

>
>However, I also would like a smooth, convenient way to load that data into
>Python in a data type easy to work with in Python, i.e. lists.

	If it is a tab separated file, the CSV module can read it if you
provide the proper "dialect".

>
>I also would like to make sure that if I edit something in that data
>structure it is saved immediately to the corresponding file. I wouldn't
>want to make an edit to an entry in the data in the live shell and somehow
>lose my progress because it wasn't written to the file.
>
	You can't unless every record has exactly the same number of bytes (if
the data is UTF-8, and you use any characters that are not in the 7-bit
ASCII set, the number of bytes in the row might change). A change in the
number of bytes means you either leave old data (number of bytes got
smaller) OR you overwrite part of the next record (more bytes in record).

	For the case where bytes stays the same (but one record may be
different length than another), you would have to use lower level file I/O
that provides seek/tell functions; use tell() before a record read to save
the start position of the record; read the record, process it, use seek()
to go back to the start of that record, write the record (you may or may
not need to use seek() using the value from tell() + length of record to
ensure you are positioned at the proper start of the next record).
>
>I would rather find some simple mode or mechanism where Python knows the
>list I am editing corresponds to a file, and it understands the
>correspondence. I.e., list(0) is the first entry in the tab-delimited file.
>
	https://docs.python.org/3/library/dbm.html
but you will have to convert the data first -- DBM files act like simple
on-disk dictionaries; each record has a string key (for you, likely the
record number, but the record data would have to be a string -- the TSV row
as is, not split into a list).

	https://docs.python.org/3/library/shelve.html#module-shelve
similar to DBM, but the data value can be a Python list rather than TSV --
the key must still be a string. Problem is that shelves are not really
portable to non-Python applications, so you still need to have
import/export converters to go between portable TSV and internal working
shelf file.

	Neither has "automatic" update of the file. The normal sequence is
		internal_data = dbm_shelf [ key ]	#read record
		dbm_shelf [ key ] = process(internal_data)	#write record
shelf with writeback=True will keep a cache of accessed records, and can
write all records using a single .sync() call.




-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/