Parallel(?) programming with python

[Dennis Lee Bieber]

On Mon, 8 Aug 2022 19:39:27 +0200, Andreas Croci <andrea.croci at gmx.de>
declaimed the following:

>
>Do you mean queues in the sense of deque (the data structure)? I ask 
>because I can see the advantage there when I try to pop data from the 
>front of it, but I don't see the sense of the following statement ("than 

	Most likely this was a reference to the Queue module -- which is used
to pass data from one thread to another. Your "fetch" thread would package
up the "new" data to be processed by the FFT thread. The FFT thread is
blocked waiting for data to appear on the queue -- when it appears, the FFT
thread reads the entire packet of data and proceeds to process it.

	Note that in this scheme, the FFT thread is NOT on a timer -- the fetch
thread controls the timing by when it puts data into the queue.

cf:
https://docs.python.org/3/library/threading.html
https://docs.python.org/3/library/queue.html

>
>That would obviusly save some coding (but would introduce the need to 
>code the interaction with the database), but I'm not sure it would speed 
>up the thing. Would the RDBMS allow to read a table while something else 
>is writing to it? I doubt it and I'm not sure it doesn't flush the cache 
>before letting you read, which would include a normally slow disk access.
>

	Depends upon the RDBMs. Some are "multi-version concurrency" -- they
snapshot the data at the time of the read, while letting new writes
proceed. But if one is doing read/modify/write, this can cause a problem as
the RDBM will detect that a record was modified by someone else and prevent
you from changing it -- you have to reselect the data to get the current
version.

	You will want to treat each of your network fetches as a transaction --
and close the transaction fast. Your FFT process would need to select all
data in the range to be processed, and load it into memory so you can free
that transaction

https://www.sqlite.org/lockingv3.html See section 3.0 and section 5.0



-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/