[Flask] Data Logger with flask

[Anthony Ford]

To chime in, Andrea has the right idea.

You want to separate your background logger and your web view. If you
leverage your SQLAlchemy models right, you can share those in a common
package to both applications, or if your SQL is straight-forward, you can
skip it and write your own queries.

I have a project for work that is this exact same architecture. For
continual logging of multiple parameters of the Arecibo Observatory's
Cryogenics system, my background process uses PyModbus to poll the
industrial microcontroller modules for the parameters, dumps them in a
Postgres DB, and my web front-end pulls the DB data and generates live
graphs and JSON structures for the API.

I originally used Supervisor (http://supervisord.org/) to manage the
background process, but I've started to explore using uwsgi to launch it as
a worker process. No real progress on that yet.

For managing the Polling, I use APScheduler (
http://apscheduler.readthedocs.org/en/latest/userguide.html), but that's
cause originally I had a more dynamic polling (adjusting rates throughout
the day to account for cooler nights and hotter days). I now poll  every 5
minutes, and could have easily done it with a loop.

Good Luck!

Anthony Ford,
KF5IBN,
ford.anthonyj at gmail.com

On Tue, Apr 19, 2016 at 3:19 AM, Andrea D'Amore <and.damore at gmail.com>
wrote:

> On 18 April 2016 at 15:11, Andreas Dorfner
> <andreas.dorfner at stud.hs-regensburg.de> wrote:
> > Next, I would like to save that data in a database and create a website
> to edit the informations as a graph or a diagram.
> > For that, I will use flask, but I'm not sure if flask is the right
> framework for this project?
>
> It depends on what you mean for "right", Flask is certainly capable of
> doing that.
> Actually any microframework would be, but with Flask you have a nice
> ecosystem if you want to add anything later on.
> I suggest go with Flask, but this is a Flask mailing list so my advice
> may be biased.
>
> > Is it possible to run the webserver for the website and the Pymodbus
> code in the same Python-File or is it a problem because of the while
> infinite loop? Maybe it's better to split it to two separate files, one for
> read data and save it in a database; the second one for the webserver. But
> how can I link that two files, because I think they have to run
> simultaneously.
>
> You're doing active polling there, while a webserver adheres to a
> client-server paradigm where the client initiates the action while the
> server waits.
> Those two are different in nature, I've never checked werkzeug's
> internals but I'm not sure it won't keep the thread busy. I'd keep
> those two parts separated.
> Running two programs at the same time isn't an issue, in a sh-like
> shell you run:
>
>     ./fetcher.py & ./webserver.py &
>
> or any variation of that, according to your spawner.
> You're likely wanting to display the data from the meter as a system
> service, i.e. you switch on the beagleboard, you wait an adequate
> amount of time and then the system should be working.
> How to implement that depends on the operating system that the BB is
> running.
>
> > Maybe I can use some parts of that tutorial and save the data of the
> energy meter in the database I have initiazized there?
>
> The app looks fairly simple from what you described, you want a single
> view, in which you load a bunch of data from the db and then pass it
> to the template for rendering. I'd use one of the nice js plotting
> library out there to plot the data passed.
>
> Then you either reload the whole page or you could go with a more
> refined solution, implement a specific method returning just the
> updated data and then just reload the chart via AJAX rather than the
> whole page.. It may not be worth the effort it in this case since the
> 10 seconds refresh period is quite long and you're not likely to have
> more than a couple clients at a time.
>
>
> --
> Andrea
> _______________________________________________
> Flask mailing list
> Flask at python.org
> https://mail.python.org/mailman/listinfo/flask
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/flask/attachments/20160419/2fec305a/attachment.html>