[Python-Dev] PEP 455: TransformDict

[Raymond Hettinger]

On Sep 22, 2013, at 6:16 PM, Ethan Furman <ethan at stoneleaf.us> wrote:

> Are we close to asking for pronouncement? 

When you're ready, let me know.

In the meantime, I conducting usability tests on students in Python classes
and researching how well it substitutes for existing solutions for 
case insensitive dictionaries (the primary use case) and for other
existing cases such as dictionaries with unicode normalized keys.

If you want to participate in the research, I could also use help looking
at what other languages do.  Python is not the first language with
mappings or to encounter use cases for transforming keys prior
to insertion and lookup.   I would like to find out what work has
already been done on this problem.

Another consideration is whether the problem is more general
that just dictionaries.  Would you want similar functionality in
all mapping-like objects (i.e. a persistent dictionaries, os.environ, etc)?
Would you want similar functionality for other services
(i.e. case-insensitive filenames or other homomorphisms).

You can also add to the discussion by trying out your own usability
tests on people who haven't been exposed to this thread or the pep.

My early results indicate that the API still needs work.

* When shown code that uses a TransformDict, students don't seem 
to be able to deduce what the code does just from the context 
(this contrasts with something like OrderedDict and Counter where
the name says what it does).   

* When given a description of the mechanics of a TransformDict,
they don't seem to be able to figure-out what you would do with it
without being given an example.

* When given a example of using a TransformDict, they understand
the example but don't seem to be able to come-up with other examples
other than the one they were just shown.  And when shown multiple 
examples, they can't think of other use cases where they've ever 
needed this in their own code.

* This contrasts with the results when I show something less general
like a CaseInsensitiveDict.  People seem to get that right away.
As you might expect, the generalized solution is harder to wrap
your head around than a specific solution with a clear name.

* One student asked, "why give regular dicts a key-function like 
sorted(), min() and max()?"  I didn't have a good answer, but I 
haven't yet had time to read this whole thread.

* Another issue is that we're accumulating too many dictionary
variants and that is making it difficult to differentiate and choose
between them.  I haven't found anyone (even in advanced classes
with very experienced pythonistas) would knew about
all the variations:  dict, defaultdict, Mapping, MutableMapping,
mapping views, OrderedDict, Counter, ChainMap, andTransformDict.

David Beazley on twitter recently proposed that we add a 
MinDict and MaxDict.  There seems to be no shortage of ideas
of things that can be done with dictionaries.

Besides choosing among the dict variants, there is also confusion
about other mapping topics such as 1) when to subclass from dict
rather than inherit from MutableMapping, 2) the difference
between defaultdict(int) and Counter's use of __missing__ to return zero,
and 3) it seems that many experienced users can't even name all the
existing methods on dictionaries (they forget clear(), copy(), pop(), popitem(),
setdefault(), update() and the fromkeys() classmethod).

Overall, my impression at this point is that key transformations
are useful, but I'm not sure how to incorporate them without
taking Python further away from being a language "that just fits
in your head."


Raymond
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20131004/eae27968/attachment.html>