[Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered

[Nick Coghlan]

On 13 September 2016 at 08:25, Gregory P. Smith <greg at krypto.org> wrote:
> At this point I think coding up an example patch against beta1 offering a
> choice of disordered iteration capability that does not increase memory or
> iteration overhead in any significant way is needed.
>
> The problem is... I don't know how to express this as an API. Which sinks my
> whole though process and tables the idea.
>
> A parameter to .items(), .keys() and .values() is undesirable as it isn't
> backwards compatible [meaning it'll never be used] and .keys() needs to
> match __iter__ which can't have one anyways. A parameter on dict
> construction is similarly infeasible.
>
> Requiring the use of an orderdict like type in order to get the behavior is
> undesirable. Effectively I'm asking for some boolean state in each dict as
> to if it should iterate in order or not and a way to expose that to pure
> Python code in a way that namespace dicts iterate in order by default and
> others do not unless explicitly configured to do so.
>
> oh well.  end thought process on my end.  it was good while it lasted.

I think this is looking at the compatibility testing problem from the
wrong direction anyway, as rather than making it difficult for people
to implicitly depend on the default key ordering, the scenario we
would want to help with is this one:

1. Library developer inadvertently depends on the
dicts-are-ordered-by-default implementation detail
2. Library user reports "your library isn't working for me on <other
implementation without that behaviour>"
3. Library developer figures out the problem, and would like to update
their test suite to deliberately provoke the misbehaviour
4. ???

That is, it falls into the same category as folks depending on
CPython's reference counting for prompt resource cleanup, where we
offer ResourceWarning to detect such cases, and context managers to
clean them up more explicitly.

For dict ordering dependence, anyone regularly testing against CPython
2.7 and CPython 3.5 will already have a good chance of detecting key
order reliance just through hash randomisation (e.g. I hit an
"inconsistent key order in generated JSON makes line-based diffing
unreadable" one myself last week with a 3-entry dict for md5, sha1 and
sha256 hashes - it was relatively rare to get the same key order two
runs in a row)

That means the only problematic case is the one where the only CPython
version a project supports is 3.6+ *and* they want to support
alternate implementations that don't preserve order in their default
dict implementation.

Given that current alternate implementations are still in the process
of catching up to *3.5* (or Python 3 at all in the case of Jython and
IronPython), I think we still have a good few years to ponder the
question before this particular concern starts cropping up in practise
:)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia