[Tutor] looking but not finding

[Mats Wichmann]

On 7/11/23 15:08, o1bigtenor wrote:

> What I specifically need assistance on is software that enables one to log
> data. What I find mountainous reams of information is on syslog - - -
> useful - - - - but when it comes to logging data - - - - well - - - quite NOT.

...

> The Python info https://docs.python.org/3/howto/logging.html  is quite
> slanted toward syslog although it does not state that. I was thinking that
> someone just might have produced something like pyplot for producing
> graphs or scipy which gets close to what I want in its statistics section
> (for the optimisation routines for a PID controller I think I would find a
> way to call a fortran routine that I've found that is scary good instead of
> using Python as in scipy) but I haven't been able to find anything when
> I include the term data log (logger/logging etc).
> 
> What I'm storing - - -
> 1. item identification
> 2. time/date (routine is called every 0.5 seconds at this point in the planning)
> 3. value (from the weighing sensor system)

Well, a quick glance turns up a fair bit of material.  Here's something 
that sounds like what you're talking about:

http://www.steves-internet-guide.com/simple-python-mqtt-data-logger/

It's written from an IoT prespective, so rather than raw sensor data, 
he's dealing with smart enough "things" that they can report in a 
structured way over a data bus, but it's the logging concepts you were 
asking about. I believe there's a followon article in the series that 
shows using a database, rather than a plain file, for logging.

Here's a project that's more on the analysis side:

https://github.com/whylabs/whylogs

The world is full of tools for data analysis and visualization so I will 
assume you can find those if you don't want to roll your own (matplotlib 
is probably the "gold standard" for plotting - pyplot is just an 
interface to that).