Python serial data aquisition

[Bengt Richter]

On 11 Jan 2005 07:51:35 -0800, fccoelho at gmail.com (Flavio codeco coelho) wrote:

>bokr at oz.net (Bengt Richter) wrote in message news:<41e384d0.552090624 at news.oz.net>...
>> On 9 Jan 2005 14:13:28 -0800, fccoelho at gmail.com (Flavio codeco coelho) wrote:
>> 
>> >Hi,
>> >
>> >I am using pyserial to acquire data from an A/D converter plugged to
>> >my serial port.
>> >
>> >my hardware represents analog voltages as 12bit numbers. So, according
>> >to the manufacturer, this number will be stored in two bytes like
>> >this;
>> > |-------------bits(1-8)-----------|
>> >Byte1: x x x n1 n2 n3 n4 n5
>> >Byte2: x n6 n7 n8 n9 n10 n11 n12
>> >
>> >where x is some other information, and nx are the digits of my number.
>> >
>> >My problem is to how to recover my reading from these bytes, since
>> >pyserial gives me a character (string) from each byte... I dont know
>> >how to throw away the unneeded bits and concatenate the remaining
>> >bits to form a number...
>> >
>> The others have shown how to recover a 12 bit positive value 0 through 4095,
>> but if the number is signed, and you want the signed value, you'll have to
>> find out how signed numbers are represented. An offset is common, in which
>> case you would subtract 2048 (2**11). If it's two's complement by some chance,
>> you'll want (I think -- untested ;-) to do num -= 2*(num&2048) instead of always num -= 2048
>> If you need speed in converting large strings of byte pairs, you could
>> convert pairs of bytes with the array module ('H' for unsigned 2-byte numbers)
>> and use the resulting 16-bit numbers as indices into another array of final values
>> prepared beforehand with redundant information that will accomplish the effect
>> of masking and shifting and adjusting sign.
>> If you need this speed, volunteers will magically appear. Maybe even if you don't ;-)
>> Regards,
>> Bengt Richter
>
>
>Hi Bengt,
>
>The Idea of using Array is realy cool Though I have to think about it
>since I would to plot the values as they are sampled...
>
>Anyway, how would you set up this array to do the shifting and
>masking?
See below.

>
>BTW, since this thread is generating quite a bit of attention let me
>post a complete description of my problem so that it may serve as
>reference to others:
>
>Hardware: DI-151RS from Dataq (2 analog channels, 2 digital input,
>single ended/differential recording, max sampling rate 240Hz) connects
>to the serial pro through a standard db9 plug.
>
>Encryption table:
>
>
>       B7      B6      B5      B4      B3      B2      B1      B0
>Byte1   A4      A3      A2      A1      A0      1       Din     0
>Byte2   A11     A10     A9      A8      A7      A6      A5      1
>Byte3   B4      B3      B2      B1      B0      1       Din     1
>Byte4   B11     B10     B9      B8      B7      B6      B5      1
>
>first two bytes are for analog ch 1 and remaining two are for ch 2.
>Din stands for digital in. AXX and BXX are the nth bits of each
>reading. A0 and B0 are the least significant bits.
>
>The latest and preferred solution on how to convert these bytes is,
>according to the suggestion of Chris Liechti (author of pyserial) is:
>(this is for the first channel only, repeat for the second)
>
>import struct
>
> l, h = struct.unpack(">BB", ser.read(2))
> n = (l >> 3) + ((h >> 1)<<5)
>
>struct.unpack returns a tuple of values represented by a string(the
>output of the read command) packed according to the format specified
>by ">BB"
>In this forma string, ">" stands for big Endian representation and "B"
>stands for unsigned char.
>
>If anyone has a better suggestion, speack up!
>
>oof! I started this thread knowing next to nothing about this stuff,
>It seem that I am finally getting the idea! :))
>
----< flavio.py >----------------------------------------------------
import array
# set up conversion table for all 8-bit little-endian byte pair possibilities
convert = array.array('l',[((ahi>>1)<<5)+(alo>>3) for ahi in xrange(256) for alo in xrange(256)])
    
def samptoval(samples):
    # convert many-sample string to array of 16-bit unnsigned integers
    data = array.array('H')
    data.fromstring(samples)
    # replace 16-bit byte pair values with 12-bit values
    for i, v in enumerate(data):
        data[i] = convert[v]
    return data

def fakedata(i12, b0=0, din=0):
    """convert unsigned 12-bit integer to little-endian byte string"""
    return chr(((i12&0x1f)<<3)+4+din*2+b0) + chr(((i12&0xfe0)>>4)+1) 

def test(v1=0, v2=256):
    assert 0 <= v1 <= v2 <=4096, "values must be in range(4096)"
    # default (0, 256) => fake a lot of A, B input pairs like ser.read(4*256)
    samples = ''.join([fakedata(i)+fakedata(i,1) for i in xrange(v1,v2)])
    data = samptoval(samples)
    for i, v in enumerate(xrange(v1,v2)):
        #print i, data[i*2:i*2+2], v
        assert data[i*2] == data[i*2+1] == v
    return samples, data
    
if __name__ == '__main__':
    import sys
    args = sys.argv[1:3]
    result = test(*map(int, args))
    print '%r\n%r' % result
---------------------------------------------------------------------

This creates pairs of A and B data for a sequence of 12-bit values and converts
them back for a test, e.g.,

[13:40] C:\pywk\clp>flavio.py 0 4
'\x04\x01\x05\x01\x0c\x01\r\x01\x14\x01\x15\x01\x1c\x01\x1d\x01'
array('H', [0, 0, 1, 1, 2, 2, 3, 3])

[13:45] C:\pywk\clp>flavio.py 30 34
'\xf4\x01\xf5\x01\xfc\x01\xfd\x01\x04\x03\x05\x03\x0c\x03\r\x03'
array('H', [30, 30, 31, 31, 32, 32, 33, 33])

[13:45] C:\pywk\clp>flavio.py 2046 2050
'\xf4\x7f\xf5\x7f\xfc\x7f\xfd\x7f\x04\x81\x05\x81\x0c\x81\r\x81'
array('H', [2046, 2046, 2047, 2047, 2048, 2048, 2049, 2049])

[13:45] C:\pywk\clp>flavio.py 4092 4096
'\xe4\xff\xe5\xff\xec\xff\xed\xff\xf4\xff\xf5\xff\xfc\xff\xfd\xff'
array('H', [4092, 4092, 4093, 4093, 4094, 4094, 4095, 4095])

>From your diagram it looks to me like your data is little-endian (least significant
data in the first byte) so I did it that way.

Actually at 240 hz, I doubt you'd have any problem with speed doing it any which way,
so this is premature optimization, and just to explain the technique, in case it's interesting.
But if you had a HUGE long string of sample data, samptoval should go fairly fast I would think.

You could also use struct to convert two bytes to an unsigned 16-bit number, and then use that number
to look up the value as flavio.convert[num] instead of making two numbers and doing the combination
yourself.





Regards,
Bengt Richter