[Python-ideas] A possible transition plan to bytes-based iteration and indexing for binary data

[Nathaniel Smith]

On 15 Jun 2014 16:25, "Steven D'Aprano" <steve at pearwood.info> wrote:
>
> On Sun, Jun 15, 2014 at 10:33:14PM +1000, Nick Coghlan wrote:
> > At PyCon earlier this year, Guido (and others) persuaded me that the
> > integer based indexing and iteration for bytes and bytearray in Python
> > 3 was a genuine design mistake based on the initial Python 3 design
> > which lacked an immutable bytes type entirely (so producing integers
> > was originally the only reasonable choice).
> [...]
> > The general principle involved would be to return an integer *subtype*
>
> Have you considered subclassing bytes, rather than int?

Isn't the obvious answer to subclass both? This would require a bit of
fiddling to ensure memory layout compatibility, but seems feasible to me
[1].

So b"abcd" would give a bytes object, and b"abcd"[0] would an inty_bytes
object, which acts like an int in int contexts and likes a bytes in bytes
contexts. E.g.,

  inty_bytes + int -> int (and warns)
  inty_bytes + bytes -> bytes

Bonus points if we can make isinstance(inty_bytes, int) warn too.

The main obstacle I see is that there are a small number of operations that
are well defined for both bytes and int objects with different semantics:

  inty_bytes * int -> ?
  inty_bytes + inty_bytes -> ?

I suspect these will be a major challenge for any transition scheme.

(Is it even viable to make bytes method behaviour dependent on a __future__
import? I guess this would require stack frame inspection?)

-n

[1] specifically I envision adding an unexposed base class that has the
struct fields required by int but no methods, making int and bytes both
inherit from it, and the inty_bytes would inherit from both. This wastes a
bit of memory in each bytes object, but only during the transition.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140615/0b8d6dfa/attachment-0001.html>