speeding up string.split()

[Chris Green]

scarblac at pino.selwerd.nl (Remco Gerlich) writes:

> Chris Green <cmg at uab.edu> wrote in comp.lang.python:
>> I've tried w/ python 2 and 1.5.2 and the differences between perl and
>> python remain huge ( about 5s:1s python:perl ).
> 
> "Huge". It's 4 seconds on 300,000 lines. Since this sort of thing is Perl's
> special expertise, I'll admit that I think that Perl has the edge
> here.

Values of huge depend o

> 
> But does that matter for your application? If it means a minute on each
> daily run, it's not worth a Usenet thread :-)

I'm currently implementing it in C with a python user interface to a
command line app ( unoptimized perl takes a bit too long ).

It's minutes on each run everytime I have to do a search ( usually
with a cgi ). There are 300,000 lines every 30 minutes so time adds up
quickly in an interactive mode, especially to do multiple day
investigations.  Theres only so many cups of coffee a day I can go get
while doing a search :).

I do appreciate a lot of the insights on this thread though that have
helped me learn more python :).

>  
>> The string is +'d together for usenet purposes
> 
> Hmmmmm. Actually, what happens when you remove the '+' and replace it with a 
> simple backslash? Two literals in a row may form one literal, but two
> strings with a '+' need to be added together every time. I think that this
> *may* make a (small) difference. Wouldn't expect it to be at all
> significant.

Yes, I could have saved a long thread had I written a \ - I've only
got a couple thousand lines of python under my belt yet and I use too
many languages and broke it up in a way to make it readable
(artificially).

-- 
Chris Green <cmg at uab.edu>