[New-bugs-announce] [issue40139] mimetypes module racy

[Uwe Kleine-König]

New submission from Uwe Kleine-König <uwe+python at kleine-koenig.org>:

Hello,

in a project using aiohttp with Python 3.5 as provided by Debian Stretch (3.5.3) I sometimes see a wrong mimetype assigned to .css files. When trying to create a minimal reproduction recipe a colleage and I came up with:

    import asyncio
    import sys
    from mimetypes import guess_type
    
    async def f():
        t = guess_type('foo.css')
    
        return t == ('text/css', None)
    
    
    async def main():
        done, pending = await asyncio.wait([
            asyncio.ensure_future(f()),
            asyncio.ensure_future(f()),
        ])
    
        return all(d.result() for d in done)
    
    if __name__ == '__main__':
        loop = asyncio.get_event_loop()
        if not loop.run_until_complete(main()):
            print("FAIL")
            exit(1)

We didn't see this exact code failing but something very similar and only once. Up to now we only tested on Python 3.5 as this is what is used in production.

By code inspection I found a race: In the module's guess_type function there is:

    if _db is None:
        init()
    return ...

It can happen here that init() is entered twice when the first context entered init() but gets preempted before setting _db.

However I failed to see how this can result in guess_type returning None (which is what we occasionally see in our production code).

Also the code in mimetypes.py is rather convoluted with two different guards for not calling init (_db is None + not inited), init() updating various global variables and instantiating a MimeTypes object that depends on these variables, ...

mimetypes.py changed in master a few times, as I didn't spot the actual problem yet and the issue hardly reproduces I cannot tell if the problem still exists in newer versions of Python.

There are also some bug reports that seem related, I found reading https://bugs.python.org/issue38656 and https://bugs.python.org/issue4963 interesting.

Best regards
Uwe

----------
components: Library (Lib)
messages: 365500
nosy: ukl
priority: normal
severity: normal
status: open
title: mimetypes module racy
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue40139>
_______________________________________