how to get bytes from bytearray without copying

Ian Kelly ian.g.kelly at gmail.com
Sun Mar 2 20:27:40 EST 2014


On Sun, Mar 2, 2014 at 5:07 PM, Juraj Ivančić <juraj.ivancic at gmail.com> wrote:
> Is it possible to somehow 'steal' bytearray's buffer and make it a read-only
> bytes? I failed to find a way to do this, and would like to make sure.
>
> My use case is, I would expect, fairly common. I read a certain (potentially
> very large) amount of data from the network into a pre-allocated bytearray.
> From that point on, this data is logically read-only. To prevent making
> redundant copies, I wrap it in a memoryview, and then slice and dice it. The
> problem with this memoryview is that it, and its slices, are considered
> writable, and thus cannot be hashed:
>
> ValueError: cannot hash writable memoryview object
>
> The only way (AFAICT) to make this work is to first create a bytes object
> from bytearray, but this copies the data. I don't need this copy, so I'd
> like to avoid it, because of both principle and performance reasons.
>
> Is there any reason why bytearray isn't able to release/convert its buffer
> to bytes? I see that it has a clear() method which... well... clears it. The
> former would be much more useful.
>
> I would also be content if there is some way of making memoryview
> artificially read-only to avoid the above error.

Python 3.3 has a C API function to create a memoryview for a char*,
that can be made read-only.

    http://docs.python.org/3/c-api/memoryview.html#PyMemoryView_FromMemory

I don't see a way to do what you want in pure Python, apart from
perhaps writing an elaborate proxy class that would just be a poor
man's memoryview.  Or you could bite the bullet and copy everything
once at the start to create a bytes object, and then never have to
worry about it again.



More information about the Python-list mailing list