[Python-ideas] Where did we go wrong with negative stride?

[Oscar Benjamin]

On 1 November 2013 10:17, Steven D'Aprano <steve at pearwood.info> wrote:
> On Thu, Oct 31, 2013 at 03:37:36PM +0200, אלעזר wrote:
>
>> Perhaps we can get this End object by adding two tokens: ":-" and "[-". So
>>
>> a[-3:-5] == a[slice(End-3, End-5, None)]
>
> That's ambiguous. Consider:
>
> a[-2]
>
> Is that a dict lookup with key -2, or a list indexed with End-2? Or
> worse, a dict lookup with key len(a)-2.

I've been thinking about that. The End object would have to be
unhashable just like slice objects:

$ python3
Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:03:43) [MSC v.1600
32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> d = {}
>>> d[1:2]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'slice'

> To break the ambiguity, we'd need a rule that End objects can only occur
> inside slices with at least one colon.

This would defeat much of the point of having a new notation. The
problem with negative wraparound applies just as much to ordinary
indexing as to slicing.

> But, I think that means that the
> parser would have to look ahead to see whether it was within a slice or
> not, and that might not be possible with Python's parser.

If you've tried to write a parser for Python expressions you'll know
that it takes a lot of special casing to handle slices anyway. (It
would be simpler if slices were a valid expression in their own
right).

> Even if were,
> it's still a special case that -2 means something different inside a
> slice than outside a slice, and we know what the Zen of Python says
> about special cases.
>
>
>> although it will turn a[-3] into a[End-3]. I don't think it's a
>> problem if the latter will behave in the same way as the former (i.e
>> End-3 be a subtype of int).
>
> So what would (End-3)*10 return? How about End & 0xF ?

I would expect End to just behave like an integer in the index/slice expression.

[snip]
>
> All this seems very clever, but as far as I'm concerned, it's too
> clever. I don't like objects which are context-sensitive. Given:
>
>     x = End-2
>
> then in this context, x behaves like 4:
>
>     "abcdef"[x:]
>
> while in this context, x behaves like 0:
>
>     "abcd"[x:]
>
> I really don't like that. That makes it hard to reason about code.

I think that's a good argument for not having a magic object. Matlab
doesn't allow you to use the keyword 'end' outside of an index/slice
expression (well actually it does but it's used to signify the end of
a block rather than as a magic object). Note that it's currently hard
to reason about something like "abcde"[x:] because you need to know
the sign of x to understand what it does.

> End seems to me to be an extremely sophisticated object, far too
> sophisticated for slicing, which really ought to be conceptually and
> practically a simple operation. I would not have to like to explain this
> to beginners to Python. I especially would not like to explain how it
> works. (How would it work?) I think the only answer is, "It's magic". I
> think it is something that would be right at home in PHP or Perl, and I
> don't mean that as an insult, but only that it's not a good fit to
> Python.

I agree that the magic End object is a probably a bad idea on the
basis that it should never be used outside of a slice/index
expression. I'm still thinking about what would be a good,
backward-compatible way of implementing something that achieves the
desired semantics.


Oscar