[New-bugs-announce] [issue13503] improved efficiency of bytearray pickling by using bytes type instead of str

[Irmen de Jong]

New submission from Irmen de Jong <irmen at razorvine.net>:

Pickling of bytearrays is quite inefficient right now, because bytearray's __reduce__ encodes the bytes of the bytearray into a str. A pickle made from a bytearray is quite a bit larger than necessary because of this, and it also takes a lot more processing to create it and to convert it back into the actual bytearray when unpickling (because it uses bytearray's str based initializer with encoding).

I've attached a patch (both for the default 3.x branch and the 2.7 branch) that changes this to use the bytes type instead. A pickle made from a bytearray with this patch applied now utilizes the BINBYTES/BINSTRING pickle opcodes which are a lot more efficient than BINUNICODE that is used now. The reconstruction of the bytearray now uses bytearray's initializer that takes a bytes object.

I don't think additional unit tests are needed because test_bytes already performs pickle tests on bytearrays.

A bit more info can be found in my recent post on comp.lang.python about this, see http://mail.python.org/pipermail/python-list/2011-November/1283668.html

----------
components: Interpreter Core
files: bytearray3x.patch
keywords: easy, needs review, patch
messages: 148627
nosy: alexandre.vassalotti, irmen, pitrou
priority: normal
severity: normal
stage: patch review
status: open
title: improved efficiency of bytearray pickling by using bytes type instead of str
type: performance
versions: Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4
Added file: http://bugs.python.org/file23812/bytearray3x.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue13503>
_______________________________________