Celery + Django logging and RotateFileHandler

Heron Rossi rossiheron at gmail.com
Thu Jun 1 19:44:03 EDT 2017


I have a Django project setup with Celery and a RotatingFileHandler setting like shown below:

    LOGGING = {
        'version': 1,
        'handlers': {
            'console':{
                'class':'logging.StreamHandler',
                'stream': sys.stdout,
            },
            'celery': {
                'class': 'logging.handlers.RotatingFileHandler',
                'filename': '/var/log/celery/celery.log',
                'formatter': 'verbose',
                'maxBytes': (1024 * 1024 * 10),  # 10 mb
                'backupCount': 10
            },
        },
        'formatters': {
            'verbose': {
                'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
            },
        },
        'loggers': {
            'sm9.views': {
                'handlers':['console'],
                'propagate': True,
                'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'),
            },
            'sm9.helpers': {
                'handlers':['console'],
                'propagate': True,
                'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'),
            },
            'sm9.soap': {
                'handlers':['celery'],
                'propagate': True,
                'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'),
            },
            'sm9.tasks': {
                'handlers':['celery'],
                'propagate': True,
                'level': os.getenv('CELERYD_LOG_LEVEL', 'INFO'),
            },
            'celery': {
                'handlers': ['celery'],
                'level': os.getenv('CELERYD_LOG_LEVEL', 'INFO'),
                'propagate': True
            },
        },
    }

When celery.log reaches 10mb the system is creating 2 more files: celery.log.1, celery.log.2 . All files are beeing populated simultaneously instead of reaching 10mb and opening a new file. Is this the expected behavior?

Celery is running as a daemon with 3 threads.



More information about the Python-list mailing list