[issue26259] Memleak when repeated calls to asyncio.queue.Queue.get is performed, without push to queue.

[Guido van Rossum]

Guido van Rossum added the comment:

> The easy solution was to check if the queue was empty and skip the read (get call) if there was nothing in the queue.

There's an API for that -- get_nowait().

I'm happy to review a fix if you come up with one. But I currently
don't have time to devise a fix myself. (Not even a docfix, if you
think that's sufficient, but if you submit one it will swiftly be
applied.)

On Mon, Feb 1, 2016 at 9:28 AM, Jonas Brunsgaard <report at bugs.python.org> wrote:
>
> Jonas Brunsgaard added the comment:
>
> In my particular case, I developed an application close to beanstalkd, but with redis as "engine". I did create a callbackback reader class for users to subclass, the callbackreader is checking every second, on every tube(queue.Object). If new data has arrived for processing (this subroutine is using queue.get with wait_for). Maybe asyncio.Condition would have been the better choice. The easy solution was to check if the queue was empty and skip the read (get call) if there was nothing in the queue.
>
> Before my fix, over a week the program would take up 10 Gigs of memory in our staging environment if nothing was touched, so I was assigned to investigate the cause. I think the current behavior is undesirable and cumbersome to see through, and if not changed there should be some kind of note in the documentation, so other good python folks will have a better chance to understand the behavior without reading the cpython asyncio queue implementation.
>
> ----------
>
> _______________________________________
> Python tracker <report at bugs.python.org>
> <http://bugs.python.org/issue26259>
> _______________________________________

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26259>
_______________________________________