[issue35589] BaseSelectorEventLoop.sock_sendall() performance regression: extra copy of data

[Huazuo Gao]

New submission from Huazuo Gao <gaohuazuo at gmail.com>:

Prior to PR 10419, sock_sendall does not make a copy of the data. PR 10419 introduced an extra copy, which may cause problem for code that send a huge chunk of data simultaneously to many peers. Relevant change is:

https://github.com/python/cpython/pull/10419/files#diff-2d64b02252335b37396e00e56fa66984R443

Bellow is a test that show the regression between 3.7.1 and 3.8-dev

---

import asyncio
import socket
import os
from subprocess import check_output

loop = asyncio.get_event_loop()

def mem_usage():
    pid = str(os.getpid())
    print(check_output(['ps', '-o', 'rss,comm'], text=True))

async def main():
    data = bytearray(10*10**6)
    data = memoryview(data)
    tasks = []
    for i in range(100):
        s1, s2 = socket.socketpair()
        s1.setblocking(False)
        s2.setblocking(False)
        tasks.append(loop.create_task(loop.sock_sendall(s1, data)))
        tasks.append(loop.create_task(loop.sock_recv(s2, 1)))
    await asyncio.sleep(0.1)
    mem_usage()
    for t in tasks:
        t.cancel()
    await asyncio.wait(tasks)

loop.run_until_complete(main())

---

result

3.7.1: 24724
3.8-dev: 979184

----------
components: asyncio
messages: 332534
nosy: Huazuo Gao, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: BaseSelectorEventLoop.sock_sendall() performance regression: extra copy of data
type: resource usage
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue35589>
_______________________________________