[issue44399] log rotator cookbook example might waste disk space

[Vinay Sajip]

Vinay Sajip <vinay_sajip at yahoo.co.uk> added the comment:

The cookbook example is (by design) limited in scope and doesn't especially discuss usage in multi-thread or multi-process scenarios - as with other examples in the cookbook which aren't specifically concerned with such scenarios. I don't think this is a problem with the example.

The problem occurs because on POSIX-style platforms, you can have multiple sources pointing to a file, and if deleted it only actually disappears when there are no references to it (such as open file descriptors/handles in other threads/processes). On Windows, for example, you would typically get a "sharing violation" type of error because you can't have multiple concurrent writers to a given file. This is not specifically a logging issue, as you could get the same thing happening with other kinds of files shared between processes and/or threads.

Adding os.truncate() might just serve to hide the problem for a while, but the right answer is to avoid opening the same log file multiple times. In the examples you mention, do multiple threads create handlers pointing to the same file? That might be an anti-pattern. Normally, the main thread configures logging (early in the "if __name__ == '__main__'" clause which creates the relevant handlers) and then any subsequently created threads just log to their loggers, but don't create any new handlers in the thread.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue44399>
_______________________________________