[Python-Dev] AMD64 Windows8.1 Refleaks 3.x buildbot is back to green!

[Victor Stinner]

Hi,

The "AMD64 Windows 8.1 Refleaks 3.x" buildbot (which hunts reference
leaks and memory leaks) was failing on test_asyncio for 1 year:

   https://bugs.python.org/issue32710

It was a leak of a single reference:

   test_aiosend leaked [1, 1, 1] references, sum=3

I tried multiple times since last year (January 2018) to understand
the leak: it didn't make any sense (well, as any bug at the beginning,
no?). I checked several times the complex asyncio code: all transports
are cleanly closed, the event loop is properly closed, etc. The code
looks correct.

After a long sleepness night... I still failed to reproduce the bug
:-( But I succeeded to get a way shorter reproducer script. Thanks to
this script, I was able to loop to get 100 reference leaks instead of
leaking a single reference. Using tracemalloc, I found the faulty
line... but it still didn't make sense to me. After additional several
hours of debug, I found that an overlapped object wasn't released
properly: an asynchronous WSASend().

The problem was when an overlapped WSASend() failed immediately, the
internal buffer was not released, whereas it holds a reference to the
input byte string. **It means that an asyncio send() failure using the
ProactorEventLoop can cause a memory leak**... I'm very surprised that
nobody noticed such **huge** memory leak previously!

The _overlapped bugfix:

https://github.com/python/cpython/commit/a234e148394c2c7419372ab65b773d53a57f3625

Eventually, the "AMD64 Windows 8.1 Refleaks 3.x" buildbot is back to green!

   https://buildbot.python.org/all/#/builders/80

It means that it will be easier and faster to spot reference or memory
leak regressions (specific to Windows, the Gentoo Refleaks buildbot
was already green for several months!).

Since ProactorEventLoop became the default event loop in Python 3.8
(on Windows, it's specific to Windows), I hope that we fixed all most
obvious bugs!

This story also means that any very tiny buildbot failure (a single
test method failure on a single very specific buildbot) can hide a
giant bug ;-) Sadly, we have to fix *all* buildbots failures to find
them... Hopefully, almost all buildbots are green currently.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.