Huge performance gain compared to perl while loading a text file in a list ...!?

[Jon Stephens]

I happened to stumble across this.  I'll admit now that I've never touched
python.  But I was curious of your example.  Does python maintain the file
handle?  I don't see any explicit handling of the file like in your perl
script.  Since if you are opening and then maintaining a file handle in the
python script it will of course stomp your perl script.  Mainly because you
put the OPEN and CLOSE inside of the loop.  So it's constantly opening and
closing the file handle.  I've never seen anyone do that before, and I hope
I never see it again.  It should have looked like this:

 

#!/usr/bin/perl
 
open(DATA, "data.test");
for ($i = 0; $i < 20; $i++) {
    @Data = <DATA>;
}
close(DATA);

 

 

When I did a quick test with a 3MB file I had in my directory I got these
results by changing the perl script.

 

 

My test script:

 

      1 #!/usr/bin/perl

      2

      3 open DATA,"<export_data.txt";

      4 for (my $i = 0; $i < 20; $i++) {

      5     my @data = <DATA>;

      6 }

      7 close DATA;

      8

      9 1;

 

 

With the open/close outside of the loop.

[jstephens at pe66 (11:05:35) ~]$ time ./test.pl

 

real    0m1.293s

user    0m0.071s

sys     0m0.036s

 

With the open/close inside of the loop

[jstephens at pe66 (11:06:56) ~]$ time ./test.pl

 

real    0m1.854s

user    0m1.612s

sys     0m0.114s

 

 

 

So using a poorly written perl script for a performance comparison isn't
really fair.  If you fix your perl script I'm sure you'll see drastic
performance increases.

 

Jon

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20051017/b339cf74/attachment.html>