[Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

[Chris Barker]

On Thu, Oct 20, 2016 at 11:48 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> >>>>> len(get_builtin_methods())
> >>    230
> >
> > So what? No one looks in all the methods of builtins at once.
>
> Yes, Python implementation developers do, which is why it's a useful
> part of defining the overall "size" of Python and how that is growing
> over time.
>

sure -- but of course, the trick is that adding *one" new method is never a
big deal by itself.

I'm confused though -- IIUC, you are proposing adding a `iobuffers`
module to the std lib -- how is that not growing the "size" of Python?


I'm still confused about the "io" in "iobuffers" -- I've used buffers a lot
 -- for passing data around between various C libs -- numpy, image
processing, etc... I never really thought of it as IO though. which is why
a  simple frombuffer() seems to make a lot of sense to me, without any
other stuff. (to be honest, I reach for Cyton these days for that sort of
thing though)


>  and we

make it easier for educators to decide whether or not they should be
> introducing their students to the new capabilities.
>

advanced domain specific use cases (see
> http://learning-python.com/books/python-changes-2014-plus.html for one
> generalist author's perspective on the vast gulf that can arise
> between "What professional programmers want" and "What's relevant to
> new programmers")
>

thanks for the link -- I'll need to read the whole thing through -- though
from a glance, I have a slightly different perspective, as an educator as
well:

Python 3, in general, is harder to learn and less suited to scripting,
while potentially more suited to building larer systems.

I came to this conclusion last year when I converted my introductory class
to py3.

Some of it is the redundancy and whatnot talked about in that link --  yes,
those are issue or me. But more of it is real, maybe important change.
Interestingly, the biggest issue with the transition: Unicode, is one thing
that has made life much easier for newbies :-)

But the big ones are things like:

The more to be iterable focused rather than sequence focused -- iterables
really are harder to wrap one's head around when you are first learning.
And I was surprised at how often I had to wrap list() around stuff when
converting my examples and exercise solutions.

I've decided to teach the format() method for string formatting -- but it
is harder to wrap your head around as a newbie.

Even the extra parens in print() makes it a bit harder to script() well.

Use with: -- now I have to explain context managers before they can even
read a file.. (or gloss over it and jsut say " copy this code to open a
file"

Anyway, I've been meaning to write a Blog post about this, that would be
better formed, but you get the idea.

In short, I really appreciate the issues here -- though I really don't see
how adding one method to a fairily obscure builtin really applies -- this
is nothing like having three(!) ways to format strings.

Which is more comprehensible and discoverable, dict.setdefault(), or
> collections.defaultdict()?
>

Well, setdefault is Definitively more discoverable! not sure what your
point is.

As it happens, the homework for my intro class this week can greatly
benefit from setdefault() (or defaultdict() ) -- and in the last few years,
far fewer newbies have discovered defaultdict() for their solutions.
Empirical evidence for discoverability.

As for comprehensible -- I give a slight nod to .setdefault() - my solution
to the HW uses that. I can't say I have a strong argument as to why -- but
having (what looks like) a whole new class for this one extra feature seems
a bit odd, and makes one look carefully to see what else might be different
about it...


> Micro-optimisations like dict.setdefault() typically don't make sense
> in isolation - they only make sense in the context of a particular
> pattern of thought. Now, one approach to such patterns is to say "We
> just need to do a better job of teaching people to recognise and use
> the pattern!". This approach tends not to work very well - you're
> often better off extracting the entire pattern out to a higher level
> construct, giving that construct a name, and teaching that, and
> letting people worry about how it works internally later.
>

hmm -- maybe -- but to me, that example isn't really a pattern of thought
(to me) -- I actually remember my history of learning about setdefault(). I
found myself writing a bunch of code something like:

if key not in a_dict:
    a_dict[key] = something
a_dict['key'].somethign_or_other()

Once I had written that code a few times, I thought: "There has got to be a
cleaner way to do this", looked at the dict methods and eventually found
setdefault() (took an embarrassingly long time). I did think -- "this has
got to be a common enough pattern to be somehow supported" but I will say
that it never, ever dawned on me to think: "this is got to be a common
enough pattern that someone would have made a special kind of dictionary
for it" -- I thought of it as a feature you'd want with a dict -- into a
different kind of dict.

So in the end, if one were to ask ME whether Python should have a
.setdefault() method on dict, or an DefaultDict object -- I'd say the
method.

> bytes and bytarray are already low-level objects -- adding low-level
> > functionality to them makes perfect sense.
>
> They're not really that low level. They're *relatively* low level
> (especially for Python),


sure -- but that is the context here.


> > And no, this is not just for asycio at all -- it's potentially useful
> > for any byte manipulation.
>

Anyway, I think frombuffer() would be a very nice thing to have, and where
it goes is secondary.


> Yes, which is why I think the end goal should be a public `iobuffers`
> module in the standard library.


I guess I'd need to see a proposal for what would go in that module to have
any opinion on  whether it would be a good idea. Is anyone actually working
on such a proposal?

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20161021/e2ff7772/attachment.html>