[issue22864] Add filter to multiprocessing.Pool

Christian Leichsenring report at bugs.python.org
Thu Oct 1 08:06:16 EDT 2020


Christian Leichsenring <christian.leichsenring at googlemail.com> added the comment:

The main point the OP didn't make is exactly the issue that Pool.map returns a list which is potentially very large given that multiprocessing is used to process large amounts of data.

So IMHO either there should be the possibility to exclude elements from being saved in memory (i.e. Pool.filter) or Pool.map shouldn't return a list but just an iterable.

----------
nosy: +christian.leichsenring

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue22864>
_______________________________________


More information about the Python-bugs-list mailing list