RotatingFileHandler bugs/errors and a general logging question.

[Vinay Sajip]

On May 10, 6:37 pm, nicholas.petre... at gmail.com wrote:
> On May 9, 12:37 am, Vinay Sajip <vinay_sa... at yahoo.co.uk> wrote:
>
> Our biggest concerns with the network solution is having a single
> point of failure and the need for scalability. We could have
> potentially thousands of machines doingloggingacross multiple
> geographic sites. Our fear with the network solution is overwhelming
> the server or group of servers as well as having a single point of
> failure for thelogginginterface. In addition using a server or
> servers would require added support for the logigng server machines.
> Our NFS infrastructure is very well supported and can handle the load
> generated by these machines already (A load which would be many times
> more than what theloggingwould generate) which is why we would like
> to log directly to file system without going through a separate
> server. Also adding in aloggingserver introduces one more level
> where we could potentially have failure. We would like to keep the
> infrastructure for ourloggingas simple as possible as we rely on log
> files to give us critical information when troubleshooting issues.
>
> It sounds like my only option may be using a server in order to handle
> theloggingfrom different hosts. That or possibly having individual
> log files for each host.
>

There are other options - you don't necessarily need a separate
logging server. For example, you could

(a) Have a single network receiver process per host which writes to
disk and avoids the problem of contention for the file. Although this
process could be a point of failure, it's a pretty simple piece of
software and it should be possible to manage the risks.
(b) If you wanted to centralize log information, you could move the
log files from each host onto a central NFS disk using standard tools
such as e.g. rsync, and then manipulate them for reporting purposes
however you want.

Regards,

Vinay Sajip