Python update trouble (2.3 to 2.4): x<<y

[Gonzalo Monzón]

Thank you for all the suggestions! :-)

The C routine is almost -changing data type long for word- a copy of the 
function given by a hardware manufacturer, the same code used in their 
firmware to calc the checksum of every piece of data sent or received, 
and that data is somewhat special: it does not contain only plain ascii 
character data (only 0x01 - 0x09 as delimiters) and every data set ends 
with a 0x00 null byte so its fine to calculating checksum until "\x00" 
is reached. And I must get the same crc calculations. That C routine 
(and a very similar PHP one too) are tested for months and work smoothly.

And of course I should use integer not long!!... we aren't anymore in 
the 8 bit computer era... ;-) original code from hardware manufacturer 
use words by the way. And now really don't know why I used long. I 
thought I did that way 'cause see pyrex C generated glue code always 
using PyInt_FromLong so thought would use the same type.

I changed the C function to use integer, and it is performing a little 
bit slowly than using longs (best with long: 0.614us, best with int: 
0.629us), that is maybe as I said due pyrex glueing always the return 
values with PyObject* PyInt_FromLong for all kind of short int to long 
data types? Anyway both results are fine, and the timming gap is 
insignificant (int: 1580403 loops -> 0.973 secs, long: 1601902 loops -> 
1.01 secs) as i usually never have to call more than 100,000 times when 
loading a full file data backup.

As a note for the checksum algorithm used here, I thought the hardware 
manufacturer implemented this some years ago for serial data 
transmission, but later attached some ethernet ports and they wanted to 
have the protocol compatible. Anyway as I see in the link you reply, it 
seems to be based in CCITT CRC-16, but they adapted to their own 
requirements -performace issues for the hardware unit?- (never 0x00, etc.)

Regards,
Gonzalo.

John Machin escribió:

>Gonzalo wrote:
>"""
>I missed the point to add the *and* to workaround the long result
>issue!
>I think I understand it now.
>
>I am timing the code once translated, so here are the results for the
>crc calculation function.
>"""
>
>Yes, and both of us were missing the point that the "16" means 16 bits
>wide!! It is carrying an extra 16 redundant bits that don't contribute
>to the checking ability.
>
>If your software *needs* to generate/check the exact same 32-bit-wide
>CRC as calculated by that C routine, then what I gave you should be OK.
> Do you have any more test values? In particular test an empty string
>and say "\x00" * 8 -- sometimes a CRC specification will do something
>special with short strings e.g. pad them put with null bytes.
>
>That C routine is actually, on further reading,  mind-boggling!! It
>stops on a null byte, rather than working on a given length. A CRC is
>normally expected to work on any data (including binary data) whether
>it contains \x00 or not. What are you using it for??
>
>If you don't have to interface with other software, AND a 16-bit check
>is adequate, then at the end of this message is a 16-bit version which
>will use only ints and therefore may run faster.
>
>However you may wish to use a CRC32 which is supplied with Python (in
>the binascii module).
>
>A further note: what we have so far seems not to be the CCITT aka X.25
>standard CRC-16 , but a reflection. See
>http://www.joegeluso.com/software/articles/ccitt.htm
>and also if you have the time and the determination, the article by
>Ross Williams that it refers to.
>
>Cheers,
>John
>
>=== 16-bit version ===
>gCRC16Table = []
>
>def InitCRC16():
>    global gCRC16Table
>    for i in xrange(0,256):
>        crc = i << 8
>        for j in xrange(0,8):
>            if (crc & 0x8000) != 0:
>                tmp = 0x1021
>            else:
>                tmp = 0
>            crc = (crc << 1) ^ tmp
>            crc &= 0xFFFF
>        gCRC16Table.append(crc)
>
>def CalcCRC16(astr):
>    crc = 0xFFFF
>    for c in astr:
>        crc = gCRC16Table[((crc >> 8) & 255)] ^ ((crc & 0xFF) << 8) ^
>ord(c)
>        assert 0 <= crc <= 0xffff # remove when doing timings :-)
>    return crc
>
>test = "123456asdfg12345123"
>InitCRC16()
>result = CalcCRC16(test)
>print result, hex(result)
>======
>
>  
>