[Python-Dev] What do we do about bad slicing and possible crashes (issue 27867)

[Nick Coghlan]

On 28 August 2016 at 08:25, Terry Reedy <tjreedy at udel.edu> wrote:
> Slicing can be made to malfunction and even crash with an 'evil' __index__
> method. https://bugs.python.org/issue27867
>
> The crux of the problem is this: PySlice_GetIndicesEx
> receives a slice object and a sequence length.  Calling __index__ on the
> start, stop, and step components can mutate the sequence and invalidate the
> length.  Adjusting the int values of start and stop according to an invalid
> length (in particular, one that is too long) will result in invalid results
> or a crash.
>
> Possible actions -- very briefly.  For more see end of
> https://bugs.python.org/issue27867?@ok_message=msg 273801
> 0. Do nothing.
> 1. Detect length change and raise.
> 2. Retrieve length after any possible changes and proceed as normal.
>
> Possible implementation strategies for 1. and 2.
> A. Change all functions that call PySlice_GetIndicesEx.
> B. Add PySlice_GetIndicesEx2 (or ExEx?), which would receive *collection
> instead of length, so the length could be retrieved after the __index__
> calls.  Change calls. Deprecate PySlice_GetIndicesEx.

Given Serhiy's clarification that this is primarily a thread safety
problem, I'm more supportive of the "PySlice_GetIndicesForObject"
approach (since that can call all the __index__ methods first, leaving
the final __len__ call as the only problematic case).

However, given the observation that __len__ can also release the GIL,
I'm not clear on how 2A is supposed to work - a poorly timed thread
switch means there's always going to be a risk of len(obj) returning
outdated information if a container implemented in Python is being
mutated concurrently from different threads, so what can be done
differently in the calling functions that couldn't be done in a new
API that accepted the container reference?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia