How to speed up reading bytes from file?

[Brad Clements]

I've received a number of suggestions. Current top speed is still about 14.2
seconds.


I received suggestions that getword() was too slow, or that multiple calls
to read() were causing problems, so I changed some code.

Old code:

                    if (mbv & CCWBYTE) != 0:
                        ccw = getbyte()
                    else:
                        ccw = getword()

                    if (mbv & CBCBYTE) != 0:
                        cbc = getbyte()
                    else:
                        cbc = getword()

New code:

                    sel = (mbv >> 4) & 0x3 # selector
                    if sel == 3:   # both bytes
                        ccw, cbc = unpack('BB',read(2))
                    elif sel == 2: # cbc is word, ccw is byte
                        ccw, cbc = unpack('!BH',read(3))
                    elif sel == 1: # cbc isbyte, ccw is word
                        ccw, cbc = unpack('!HB',read(3))
                    elif sel == 0: # cbc is word, ccw is word
                        ccw, cbc = unpack('!HH',read(4))

This is actually slightly slower.. 14.8 seconds vs 14.2 the old way.

new stats:

   138817   18.900    0.000   31.490    0.000
TokenIndex3.py:193(__loadIndexLevel)
   891347    8.690    0.000    8.690    0.000 TokenIndex3.py:99(getbyte)
   435948    3.900    0.000    3.900    0.000 TokenIndex3.py:203(read)

( had to make read a local function to get it to show up in the profile ..)

I'm thinking of going back to array, but using it __loadIndexLevel instead
of making a SeekableFileClass.

I'll let you know.. (also looking at Pyrex as the final step, but I want to
get the algorithms right first)







-----------== Posted via Newsfeed.Com - Uncensored Usenet News ==----------
   http://www.newsfeed.com       The #1 Newsgroup Service in the World!
-----= Over 100,000 Newsgroups - Unlimited Fast Downloads - 19 Servers =-----