[Python-ideas] struct.unpack should support open files

[Paul Moore]

On Mon, 24 Dec 2018 at 13:39, Steven D'Aprano <steve at pearwood.info> wrote:
>
> > Files can be opened in text mode, what to do in this case? What
> > exception should be raised?
>
> That is easy to answer: the same exception you get if you pass text to
> unpack() when it is expecting bytes:
>
> py> struct.unpack(fmt, "a")
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: a bytes-like object is required, not 'str'
>
> There should be no difference whether the text comes from a literal, a
> variable, or is read from a file.

One difference is that with a file, it's (as far as I can see)
impossible to determine whether or not you're going to get bytes or
text without reading some data (and so potentially affecting the state
of the file object). This might be considered irrelevant (personally,
I don't see a problem with a function definition that says "parameter
fd must be an object that has a read(length) method that returns
bytes" - that's basically what duck typing is all about) but it *is* a
distinguishing feature of files over in-memory data.

There is also the fact that read() is only defined to return *at most*
the requested number of bytes. Non-blocking reads and objects like
pipes that can return additional data over time add extra complexity.
Again, not insoluble, and potentially simple enough to handle with
"read N bytes, if you got something other than bytes or fewer than N
of them, raise an error", but still enough that the special cases
start to accumulate.

The suggestion is a nice convenience method, and probably a useful
addition for the majority of cases where it would do exactly what was
needed, but still not completely trivial to actually implement and
document (if I were doing it, I'd go with the naive approach, and just
raise a ValueError when read(N) returns anything other than N bytes,
for what it's worth).

Paul