[issue18073] pickle.Unpickler may read too many bytes, causing hangs with blocking input stream

[Julien Muchembled]

New submission from Julien Muchembled:

I use pickle over sockets to make 2 processes communicate and I experience hangs when the resulting pickle message is exactly 4097 bytes.

Attached file is a small script reproducing the issue, trying a few values around 4096. It runs on both Python 2.7 & 3.3:

- Python 2.7.3: ok (script exits)
- Python 3.3.2: last iteration hangs

With strace, you can easily see that after reading the 4097 bytes:

[pid 17100] read(3,  <unfinished ...>
[pid 17100] <... read resumed> "\200\2X\367\17\0\0                         "..., 4096) = 4096
[pid 17100] read(3, ".", 4096)          = 1

it reads again without reason:

[pid 17100] read(3, 

No issue with the Python implementation, that you can try with little change:

- Python 2.7.3: ok   (from pickle import Unpickler)
- Python 3.3.2: ok   (from pickle import _Unpickler as Unpickler)

----------
components: Extension Modules
files: test_case.py
messages: 190149
nosy: jm
priority: normal
severity: normal
status: open
title: pickle.Unpickler may read too many bytes, causing hangs with blocking input stream
type: behavior
versions: Python 3.3
Added file: http://bugs.python.org/file30388/test_case.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18073>
_______________________________________