[issue35152] too small type for struct.pack/unpack in mutliprocessing.Connection

Oleksandr Buchkovskyi report at bugs.python.org
Sat Nov 3 04:00:06 EDT 2018


New submission from Oleksandr Buchkovskyi <olex.buchkovsky at gmail.com>:

the problem is reproduced on big multiprocessing.Process output
when the size of the output gets bigger than signed int a struct error is raised
```
python3 test.py
Process ForkPoolWorker-1:
Traceback (most recent call last):
  File "/usr/lib64/python3.5/multiprocessing/pool.py", line 125, in worker
    put((job, i, result))
  File "/usr/lib64/python3.5/multiprocessing/queues.py", line 347, in put
    self._writer.send_bytes(obj)
  File "/usr/lib64/python3.5/multiprocessing/connection.py", line 200, in send_bytes
    self._send_bytes(m[offset:offset + size])
  File "/usr/lib64/python3.5/multiprocessing/connection.py", line 393, in _send_bytes
    header = struct.pack("!i", n)
struct.error: 'i' format requires -2147483648 <= number <= 2147483647

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.5/multiprocessing/process.py", line 252, in _bootstrap
    self.run()
  File "/usr/lib64/python3.5/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib64/python3.5/multiprocessing/pool.py", line 130, in worker
    put((job, i, (False, wrapped)))
  File "/usr/lib64/python3.5/multiprocessing/queues.py", line 347, in put
    self._writer.send_bytes(obj)
  File "/usr/lib64/python3.5/multiprocessing/connection.py", line 200, in send_bytes
    self._send_bytes(m[offset:offset + size])
  File "/usr/lib64/python3.5/multiprocessing/connection.py", line 393, in _send_bytes
    header = struct.pack("!i", n)
struct.error: 'i' format requires -2147483648 <= number <= 2147483647
```

this error can be fixed by changing types that are used herу for struct.pack/unpack to !Q

----------
components: Library (Lib)
files: error_on_bug_multiprocessing_output.py
messages: 329189
nosy: ahcub
priority: normal
severity: normal
status: open
title: too small type for struct.pack/unpack in mutliprocessing.Connection
type: crash
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file47903/error_on_bug_multiprocessing_output.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue35152>
_______________________________________


More information about the Python-bugs-list mailing list