[Tutor] looking but not finding

[o1bigtenor]

On Wed, Jul 12, 2023 at 8:20 AM Mats Wichmann <mats at wichmann.us> wrote:
>
> On 7/11/23 15:08, o1bigtenor wrote:
>
> > What I specifically need assistance on is software that enables one to log
> > data. What I find mountainous reams of information is on syslog - - -
> > useful - - - - but when it comes to logging data - - - - well - - - quite NOT.
>
> ...
>
> > The Python info https://docs.python.org/3/howto/logging.html  is quite
> > slanted toward syslog although it does not state that. I was thinking that
> > someone just might have produced something like pyplot for producing
> > graphs or scipy which gets close to what I want in its statistics section
> > (for the optimisation routines for a PID controller I think I would find a
> > way to call a fortran routine that I've found that is scary good instead of
> > using Python as in scipy) but I haven't been able to find anything when
> > I include the term data log (logger/logging etc).
> >
> > What I'm storing - - -
> > 1. item identification
> > 2. time/date (routine is called every 0.5 seconds at this point in the planning)
> > 3. value (from the weighing sensor system)
>
> Well, a quick glance turns up a fair bit of material.  Here's something
> that sounds like what you're talking about:
>
> http://www.steves-internet-guide.com/simple-python-mqtt-data-logger/

dunno about finding 'a fair bit of material' - - - I seem to have been using
the wrong search terms.
>
> It's written from an IoT prespective, so rather than raw sensor data,
> he's dealing with smart enough "things" that they can report in a
> structured way over a data bus, but it's the logging concepts you were
> asking about. I believe there's a followon article in the series that
> shows using a database, rather than a plain file, for logging.

This is in the right direction but that follow article doesn not address
a number of the points that mr Alan makes. (see my response to his note)
>
> Here's a project that's more on the analysis side:
>
> https://github.com/whylabs/whylogs
>
> The world is full of tools for data analysis and visualization so I will
> assume you can find those if you don't want to roll your own (matplotlib
> is probably the "gold standard" for plotting - pyplot is just an
> interface to that).
>
Agreed - - - this is an area where I have not trouble finding ideas or
techniques
its that mid point of not only grabbing the sensor info but also stashing it!

Thanks for the pointers!