Multi-dimensional list initialization

[Prasad, Ramit]

Andrew Robinson wrote:
> 
> On 11/06/2012 01:04 AM, Steven D'Aprano wrote:
> > On Mon, 05 Nov 2012 21:51:24 -0800, Andrew Robinson wrote:
> >
[snip]
> >      Q: What about other mutable objects like sets or dicts?
> >      A: No, the elements are never copied.
> They aren't list multiplication compatible in any event! It's a total
> nonsense objection.
> 
> If these are inconsistent in my idea -- OBVIOUSLY -- they are
> inconsistent in Python's present implementation.  You can't even
> reference duplicate them NOW.
> 
>  >>> { 1:'a', 2:'b', 3:'c' } * 2
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> TypeError: unsupported operand type(s) for *: 'dict' and 'int'

>>> z = [ {'a':1} ]*10
>>> z[0]['b'] = 4
>>> z
[{'a': 1, 'b': 4}, {'a': 1, 'b': 4}, {'a': 1, 'b': 4},{'a': 1, 'b': 4}, 
{'a': 1, 'b': 4}, {'a': 1, 'b': 4}, {'a': 1, 'b': 4}, {'a': 1, 'b': 4},
{'a': 1, 'b': 4}, {'a': 1, 'b': 4}]

Should that copy the dictionary? According to logical reasoning
it should copy the dictionary as well. How do you draw the line of 
what should be copied and what should not? 

> 
> >      Q: How about on Tuesdays? I bet they're copied on Tuesdays.
> >      A: No, the elements are never copied.
> That's really a stupid objection, and everyone knows it.

Agreed. [snip]

> >      Q: How about if I use delegation to proxy a list?
> >      A: Oh no, they definitely won't be copied.
> Give an example usage of why someone would want to do this.  Then we can
> discuss it.

IIRC, someone wanted to do something very similar for dictionaries to 
prevent editing of global variables.

> >      Q: What about other mutable objects like sets or dicts?
> >      A: No, definitely not. Unless people complain enough.
> now you're just repeating yourself to make your contrived list longer --
> but there's no new objections...

This is my main objection and one of the flaws of your argument.
You want to handle one type of mutable objects completely separately
than other mutable objects. Why is list any different than dictionary
in this respect? The only reason I can imagine is because lists
end up being used for 2d (or higher) "matrices".

> 
> > Losing consistency in favour of saving a few characters for something as
> > uncommon as list multiplication is a poor tradeoff. That's why this
> > proposal has been rejected again and again and again every time it has
> > been suggested.
> Please link to the objection being proposed to the developers, and their
> reasoning for rejecting it.
> I think you are exaggerating.

I reject (as a developer) it because it forces me to remember a very 
specific quirk versus a simple (logical) rule that applies to all objects. Not to mention that the quirk is not even that useful except for beginners.

> 
> > List multiplication [x]*n is conceptually equivalent to:
> > <snip>
> > This is nice and simple and efficient.
> No it isn't efficient. It's *slow* when done as in your example.
> 
> > Copying other objects is slow and inefficient. Keeping list
> > multiplication consistent, and fast, is MUCH more important than making
> > it work as expected for the rare case of 2D arrays:
> I don't think so -- again, look at range(); it was made to work
> inconsistent for a "common" case.
> 
> Besides, 2D arrays are *not* rare and people *have* to copy internals of
> them very often.
> The copy speed will be the same or *faster*, and the typing less -- and
> the psychological mistakes *less*, the elegance more.
> 
> It's hardly going to confuse anyone to say that lists are copied with
> list multiplication, but the elements are not.
> 
> Every time someone passes a list to a function, they *know* that the
> list is passed by value -- and the elements are passed by reference.
> People in Python are USED to lists being "the" way to weird behavior
> that other languages don't do.

I think you just lost 90% of your credibility (with me). When did lists 
get passed by value? Python uses call by sharing[0]. 

Terminology aside, lists are handled exactly the same way as all
other objects; the rules regarding their mutability in the callee
are the same as dictionaries, sets, or any mutable type (including
non-builtins). 
  

> 
> >
> > Copying those elements does not come for free.
> >
> > It is true that list multiplication can be much faster than a list comp.
> > But that's because the list multiply doesn't have to inspect the
> > elements, copy them, or engage the iteration machinery. Avoiding copying
> > gives you a big saving:
> >
> >
> > [steve at ando ~]$ python3.3 -m timeit -s "x = range(1000)"
> > "[x for _ in range(100)]"  # not copied
> > 100000 loops, best of 3: 11.9 usec per loop
> >
> > [steve at ando utilities]$ python3.3 -m timeit -s "x = range(1000)"
> > "[x[:] for _ in range(100)]"  # copied
> > 10000 loops, best of 3: 103 usec per loop
> >
> > So there's a factor of ten difference right there. If list multiplication
> > had to make copies, it would lose much of its speed advantage.
> And when multiplication doesn't make copies of *lists*, it's going
> "nowhere fast", because people don't want the results that gives.
> 
> So what difference does it make?  People won't make the construction
> unless they wanted to make the copies in the first place.  If they want
> the copies, well -- copies are *slow*.  Big deal.
> 
> >   For large
> > enough lists, or complicated enough objects, it would become slower than
> > a list comprehension.
> Huh? You're nuts.
> 
> > It would be even slower if list multiplication had to inspect each
> > element first and decide whether or not to copy.
> A single pointer comparison in a 'C' for loop takes less than 5 nano
> seconds on a 1Ghz machine.
> (I'll bet yours is faster than that...!)
> Consider: list objects have a pointer which points back to the generic
> list object -- that's all it takes to determine what the "type" is.
> 
> Your measured loop times, doing list comprehensions takes over 10
> microseconds *per loop*.
> Compared to what you're proposing -- The pointer compare is a mere 0.05%
> change;  You can't even measure that with "timeit!".  BUT: The increase
> in speed for not running tokenized "for" loops is *much* bigger than the
> loss for a single pointer compare; so it will *usually* be a *serious*
> net gain.
> 
> >> I really don't think doing a shallow copy of lists would break anyone's
> >> program.
> > Anyone who is currently using list multiplication with mutable objects is
> > expecting that they will be the same object, and relying on that fact.
> > Otherwise they wouldn't be using list multiplication.
> yes, and I'm not changing that -- except for lists; and *no* one is
> using that.
> Find two examples of it from existing non contrived web examples of
> Python code.
> *ask* around.

I am positive that majority of code is not examples--web or otherwise.

> 
> >
> > You're suggesting a semantic change. Therefore they will be expecting
> > something different from what actually happens. Result: broken code.
> Even if it was;  So are many semantic changes happening between python 2
> and python 3.
> Look at what python 2 did:
> 
>  >>> range(0,5)[0]
> 0
>  >>> range(0,5)[1:3]
> [1, 2]
> 
> That's a *semantic* change.
> Also; if you complain that xrange has been renamed range; then look:
> 
>  >>> xrange(0,5)[0]
> 0
>  >>> xrange(0,5)[1:3]
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> TypeError: sequence index must be integer, not 'slice'
> 
> WOW. WOW. WOW.  An even BIGGER semantic change.

So because one thing has a semantic change that gives license
for semantic changes everywhere? Bah, ridiculous!

[snip]
> 
> > to get the behaviour you want, and then in Python 3.5 it would become the
> > default. That's three years until it becomes the standard. Meanwhile,
> > there will still be millions of people using Python 2.7 or 3.2, and their
> > code will behave differently from your code.
> Uh, they aren't *using* the construction I am proposing now -- they are
> avoiding it like the plague.
> Hence, it will merely become a new ability in a few years -- not
> 'differently' behaving code.

How in the name of <insert deity (or religion)> do you have any clue 
about that? Granted, as an educated you *may* be right, but you
may not be. I have no idea how you could know this definitively
or with any great degree of certainty. [snip]


~Ramit

[0] http://en.wikipedia.org/wiki/Evaluation_strategy#Call_by_sharing


This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.