Parsing String, Dictionary Lookups, Writing to Database Table

[Rich Shepard]

   I need to learn how to process a byte stream from a form reader where each
pair of bytes has meaning according to lookup dictionaries, then use the
values to build an array of rows inserted into a sqlite3 database table.

   Here's the context: The OMR card reader sends a stream of 69 bytes over
the serial line; the last byte is a carriage return ('\r') indicating the
end of record. Three pairs (in specific positions at the beginning of the
stream) represent blanks (no value); two other pairs represent character
strings; the values are determined from two dictionaries. The remaining 28
pairs represent values from a third dictionary.

   What I'm doing: I think that I have the first part correct; that is, the
three blanks and two string values. The first three dictionaries are:

     DATA_MAP_BLANK = {
       chr(32)+chr(32): 0       # SP + SP
       }
     DATA_MAP_2 = {
       chr(32)+chr(36): 'nat',  # SP + $
       chr(96)+chr(32): 'eco',  #   + SP
       chr(36)+chr(32): 'soc'   #  $ + SP
       }
     DATA_MAP_5 = {
       chr(32)+chr(36): 'pro',  # SP + $
       chr(96)+chr(32): 'neu',  #   + SP
       chr(36)+chr(32): 'con'   #  $ + SP
       }

   I read the data into a string and split that into byte tokens, then start
building the row to be inserted into a database table:

     line = ser.readline()
     split_line = line.split()

     # then pre-pend the record number to the front of the row
     row.join(', ', vote_id)

     # extract category choice (row 2, bytes 2 and 3); look up value in dictionary
     cat = split_line(2:4)
     row.join(', ', DATA_MAP_2(cat) # I'm not sure this is a correct lookup

     # extract position (row 5, bytes 8 and 9P; look up value in dictionary
       pos = split_line(8:10)
       row.join(', ', DATA_MAP_5(pos))

   Is the above the most 'correct' way of extracting specific byte pairs,
using them as dictionary keys to get values, then build a string of
comma-and-quote values for insertion in the database table?

   Then, I've no idea how to get the rest of the data parsed for use as keys
in the last data mapping dictionary. I do not see a skip value for slicing,
other than in Numeric Python, and I'm not yet building an array of data.

   Here's the last dictionary:

     DATA_MAP_7 = {
       chr(32)+chr(16): 1.000,  # SP + DLE
       chr(32)+chr(8): 2.000,   # SP + BS
       chr(32)+chr(4): 3.000,   # SP + EOT
       chr(32)+chr(2): 4.000,   # SP + STX
       chr(32)+chr(1): 5.000,   # SP + SOH
       chr(64)+chr(32): 6.000,  #  @ + SP
       chr(16)+chr(32): 7.000,  # DLE + SP
       chr(8)+chr(32): 8.000,   # BS + SP
       chr(4)+chr(32): 9.000,   # EOT + SP
       chr(34)+chr(8): 0.500,   #  " + BS
       chr(34)+chr(4): 0.333,   #  " + EOT
       chr(34)+chr(2): 0.025,   #  " + STX
       chr(34)+chr(1): 0.200,   #  " + SOH
       chr(66)+chr(32): 0.167,  #  B + SP
       chr(18)+chr(32): 0.143,  # DC2 + SP
       chr(10)+chr(32): 0.125,  # LF + SP
       chr(6)+chr(32): 0.111    # ACK + SP
       }

   I know how I'd do all this in C, but since I'm learning python I have not
found how best to accomplish this despite the books and online references
I've read.

TIA,

Rich

-- 
Richard B. Shepard, Ph.D.               |    The Environmental Permitting
Applied Ecosystem Services, Inc.(TM)    |            Accelerator
<http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863