Unicode and Python - how often do you index strings?

[Dave Angel]

Chris Angelico <rosuav at gmail.com> Wrote in message:
> On Wed, Jun 4, 2014 at 8:10 PM, Peter Otten <__peter__ at web.de> wrote:
>> The indices used for slicing typically don't come out of nowhere. A simple
>> example would be
>>
>> def strip_prefix(text, prefix):
>>     if text.startswith(prefix):
>>         text = text[len(prefix):]
>>     return text
>>
>> If both prefix and text use UTF-8 internally the byte offset is already
>> known. The question is then how we can preserve that information.
> 
> Almost completely useless. First off, it solves only the problem of
> operating on the string at exactly some point where you just got an
> index; and secondly, you don't always get that index from a string
> method. Suppose, for instance, that you iterate over a string thus:
> 
> for i, ch in enumerate(string):
>     if ch=='{': start = i
>     elif ch=='}': return string[start:end+1]
> 
> Okay, so this could be done by searching, but for something more
> complicated, I can imagine it being better to enumerate. (But "I can
> imagine" is much weaker than "Here's code that we use in production",
> which is why I asked the question.)
> 
> Incidentally, the above code highlights the first problem too. With
> direct indexing, you can ask for inclusive or exclusive slicing by
> adding or subtracting one from the index. If you do that with a
> byte-position-retaining special integer, you lose the byte position.
> 
> ChrisA
> 

A string could have two extra fields in it that hold index and
 offset for the most recent substring reference.  Even though the
 string is immutable,  nothing prevents mutable elements that are
 externally visible only by performance measurement.
 

So a loop using a subscript of a string would tend to be faster
 even if written in a naive way.

It's also conceivable to build an array of such pairs in strings
 over a threshold size. So if you had a megabyte string, there
 might be 100 evenly spaced pairs, calculated when the string
 object is first created.

And naturally there can be flags indicating that the particular
 string is pure ASCII.

Clearly this breaks down if there are two alternating references
 at different offsets, but I think this would be exceeding
 rare.

-- 
DaveA