From logging to files to a better solution: syslog, Sentry, Logstash, ....

[dieter]

Thomas Güttler <hv at tbz-pariv.de> writes:
> ...
> Why we are unhappy with logging to files:
>
>  - filtering: We don't want to get INFO messages over the VPN.

You can quite easily control at what level messages are logged with
the standard Python logging framework. Each handler has a level
and will ignore messages at a lower level.

>  - Rotating: Rotating files is possible, but somehow cumbersome.

There are standard tools to rotate logfiles.

>  - Support structured logging of values (json) in the future.

Again, the Python logging framework is quite flexible with
respect to the format of logged messages.

> ...
> Which solution could fit for our environment?

I work for a customer with a similar environment (he uses "Zope" instead
of "Django") - and he uses logfiles. The logfiles are automatically
rotated and there are in the order of half a dozen to a dozen logfiles
per day.

When I have to analyse a problem with the help of the logfiles,
I do not copy them via VPN but do the filtering remotely and only
copy the filtered portion, if necessary.