[issue26233] select.epoll.wait() should away calling malloc() each time

[STINNER Victor]

STINNER Victor added the comment:

I dislike allocating large memory block on the stack, we should not abuse it.

I propose instead to store the memory block allocated on the heap in the epoll object. Attached patch implements this idea.

The buffer is stolen from the epoll object when epoll.poll() is called. Then it is stored again inside the epoll object, except if another thread stored a larger buffer in the meanwhile. The largest buffer is also kept.

The buffer exchange is protected by the GIL.

My patch also overallocates the buffer by 50% to avoid calling realloc() to many times. Using the selectors module, maxevents is the number of registered FD. Using asyncio, the number of registered FD changes a lot.

Side effect: the memory block is not released after the call to epoll.poll(). If you can it with an insane max_events, it will bloat your memory until your close the poller.

If the approach is considered interested and efficient, I can work on a similar patch for other pollers.

----------
Added file: http://bugs.python.org/file41744/epoll_buffer.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26233>
_______________________________________