Parallel(?) programming with python

[Schachner, Joseph (US)]

Why would this application *require* parallel programming?   This could be done in one, single thread program.   Call time to get time and save it as start_time.   Keep a count of the number of 6 hour intervals, initialize it to 0.

Once a second read data an append to list.  At 6 hours after start time, call a function that does an FFT (see comment about scipy below) and increment the count of 6 hour intervals.  Call time and save new start time. Continue execution.

After 28 six hour intervals, save the list and then slice the list to  shorten it as you want.  Reset the count of 6 hour intervals to zero.

The FFT might take a second, even if you use scipy, depending on how long the list is (If you don’t know about numpy and scipy, look them up! You need them.   Your list can be an array in numpy).  
Saving and slicing the list should take less than a second.

This single thread approach avoids thinking about multiprocessing, locking and unlocking data structures, all that stuff that does not contribute to the goal of the program.

--- Joseph S.


Teledyne Confidential; Commercially Sensitive Business Data

-----Original Message-----
From: Andreas Croci <andrea.croci at gmx.de> 
Sent: Monday, August 8, 2022 6:47 AM
To: python-list at python.org
Subject: Parallel(?) programming with python

tI would like to write a program, that reads from the network a fixed amount of bytes and appends them to a list. This should happen once a second.

Another part of the program should take the list, as it has been filled so far, every 6 hours or so, and do some computations on the data (a FFT).

Every so often (say once a week) the list should be saved to a file, shorthened in the front by so many items, and filled further with the data coming fom the network. After the first saving of the whole list, only the new part (the data that have come since the last saving) should be appended to the file. A timestamp is in the data, so it's easy to say what is new and what was already there.

I'm not sure how to do this properly: can I write a part of a program that keeps doing its job (appending data to the list once every second) while another part computes something on the data of the same list, ignoring the new data being written?

Basically the question boils down to wether it is possible to have parts of a program (could be functions) that keep doing their job while other parts do something else on the same data, and what is the best way to do this.