[Python-Dev] Cost-Free Slice into FromString constructors--Long

[Raymond Hettinger]

Runar Petursson wrote:

> We've been talking this week about ideas for speeding up the parsing 
> of Longs coming out of files or network.  The use case is having a 
> large string with embeded Long's and parsing them to real longs.  One 
> approach would be to use a simple slice:
> long(mystring[x:y])
>
> an expensive operation in a tight loop.  The proposed solution is to 
> add further keyword arguments to Long (such as):
>
> long(mystring, base=10, start=x, end=y)
>
> The start/end would allow for negative indexes, as slices do, but 
> otherwise simply limit the scope of the parsing.  There are other 
> solutions, using buffer-like objects and such, but this seems like a 
> simple win for anyone parsing a lot of text.  I implemented it in a 
> branch  runar-longslice-branch, but it would need to be updated with 
> Tim's latest improvements to long.  Then you may ask, why not do it 
> for everything else parsing from string--to which I say it should.  
> Thoughts?


-1 This is a somewhat specialized parsing application and should not be 
allowed to muck-up an otherwise simple, general-purpose API.   
Micro-optimizations do not warrant API changes.   Certainly, it should 
not be propogated to everything that can parse from a string.  Also, you 
are likely to find that the cost of varargs and kwargs will offset the 
slicing gains.

I think the whole notion is off base.  The int(mystring[x:y]) situation 
is only important when you're doing it many times.  Often, you'll be 
doing other conversions as well.  So, you would be better-off creating a 
text version of the struct module that would enable you to extract and 
convert many elements from a long record stored as text.   Alternately, 
you could expose a function styled after fscanf().  IOW, better to 
provide to general purpose text parsing tool than to muck-up the whole 
language for one specialized application.


Raymond