[Python-Dev] AMD64 Windows8.1 Refleaks 3.x buildbot is back to green!

[Andrew Svetlov]

That's great!

I would add that many Windows Proactor bugs were found after implementing
sendfile support.
If sendfile is not available for some reason asyncio uses a fallback to
send a file content chunk-by-chunk reading it in memory.
The fallback code was easy and straightforward, it worked just fine on
Linux but the same code failed on proactor.

Thank you very much, Victor, for finally pinning the problem down.

On Wed, Jan 9, 2019 at 7:51 PM Brett Cannon <brett at python.org> wrote:

> Thanks for tracking that down! Been bugging me as well and the one time I
> tried to figure it out I got no where, so kudos for sticking with it!
>
> On Wed, 9 Jan 2019 at 09:13, Victor Stinner <vstinner at redhat.com> wrote:
>
>> Hi,
>>
>> The "AMD64 Windows 8.1 Refleaks 3.x" buildbot (which hunts reference
>> leaks and memory leaks) was failing on test_asyncio for 1 year:
>>
>>    https://bugs.python.org/issue32710
>>
>> It was a leak of a single reference:
>>
>>    test_aiosend leaked [1, 1, 1] references, sum=3
>>
>> I tried multiple times since last year (January 2018) to understand
>> the leak: it didn't make any sense (well, as any bug at the beginning,
>> no?). I checked several times the complex asyncio code: all transports
>> are cleanly closed, the event loop is properly closed, etc. The code
>> looks correct.
>>
>> After a long sleepness night... I still failed to reproduce the bug
>> :-( But I succeeded to get a way shorter reproducer script. Thanks to
>> this script, I was able to loop to get 100 reference leaks instead of
>> leaking a single reference. Using tracemalloc, I found the faulty
>> line... but it still didn't make sense to me. After additional several
>> hours of debug, I found that an overlapped object wasn't released
>> properly: an asynchronous WSASend().
>>
>> The problem was when an overlapped WSASend() failed immediately, the
>> internal buffer was not released, whereas it holds a reference to the
>> input byte string. **It means that an asyncio send() failure using the
>> ProactorEventLoop can cause a memory leak**... I'm very surprised that
>> nobody noticed such **huge** memory leak previously!
>>
>> The _overlapped bugfix:
>>
>>
>> https://github.com/python/cpython/commit/a234e148394c2c7419372ab65b773d53a57f3625
>>
>> Eventually, the "AMD64 Windows 8.1 Refleaks 3.x" buildbot is back to
>> green!
>>
>>    https://buildbot.python.org/all/#/builders/80
>>
>> It means that it will be easier and faster to spot reference or memory
>> leak regressions (specific to Windows, the Gentoo Refleaks buildbot
>> was already green for several months!).
>>
>> Since ProactorEventLoop became the default event loop in Python 3.8
>> (on Windows, it's specific to Windows), I hope that we fixed all most
>> obvious bugs!
>>
>> This story also means that any very tiny buildbot failure (a single
>> test method failure on a single very specific buildbot) can hide a
>> giant bug ;-) Sadly, we have to fix *all* buildbots failures to find
>> them... Hopefully, almost all buildbots are green currently.
>>
>> Victor
>> --
>> Night gathers, and now my watch begins. It shall not end until my death.
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com
>


-- 
Thanks,
Andrew Svetlov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20190109/e9a0de65/attachment.html>