[issue38923] Spurious OSError "Not enough memory resources" when allocating memory using multiprocessing.RawArray

[Brian Kardon]

New submission from Brian Kardon <brian.kardon at gmail.com>:

When I try to create a series of multiprocessing.RawArray objects, I get an "OSError: Not enough memory resources are available to process this command". However, by my calculations, the total amount of memory I'm trying to allocate is just about 1 GB, and my system reports that it has around 20 GB free memory. I can't find any information about any artificial memory limit, and my system seems to have plenty of memory for this, so it seems like a bug to me. This is my first issue report, so I apologize if I'm doing this wrong.

The attached script produces the following output on my Windows 10 64-bit machine with Python 3.7:

Creating buffer # 0
Creating buffer # 1
Creating buffer # 2
Creating buffer # 3
Creating buffer # 4
...etc...
Creating buffer # 276
Creating buffer # 277
Creating buffer # 278
Creating buffer # 279
Creating buffer # 280
Traceback (most recent call last):
  File ".\Cruft\memoryErrorTest.py", line 10, in <module>
    buffers.append(mp.RawArray(imageDataType, imageDataSize))
  File "C:\Users\Brian Kardon\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\context.py", line 129, in RawArray
    return RawArray(typecode_or_type, size_or_initializer)
  File "C:\Users\Brian Kardon\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\sharedctypes.py", line 61, in RawArray
    obj = _new_value(type_)
  File "C:\Users\Brian Kardon\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\sharedctypes.py", line 41, in _new_value
    wrapper = heap.BufferWrapper(size)
  File "C:\Users\Brian Kardon\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\heap.py", line 263, in __init__
    block = BufferWrapper._heap.malloc(size)
  File "C:\Users\Brian Kardon\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\heap.py", line 242, in malloc
    (arena, start, stop) = self._malloc(size)
  File "C:\Users\Brian Kardon\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\heap.py", line 134, in _malloc
    arena = Arena(length)
  File "C:\Users\Brian Kardon\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\heap.py", line 38, in __init__
    buf = mmap.mmap(-1, size, tagname=name)
OSError: [WinError 8] Not enough memory resources are available to process this command

----------
components: Library (Lib), Windows, ctypes
files: memoryErrorTest.py
messages: 357531
nosy: bkardon, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Spurious OSError "Not enough memory resources" when allocating memory using multiprocessing.RawArray
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file48744/memoryErrorTest.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38923>
_______________________________________