Struggling with struct.unpack() and "p" format specifier

[Tim Peters]

[Geoffrey <geskerrett at hotmail.com>]
> I am trying to read data from a file binary file and then unpack the
> data into python variables.  Some of the data is store like this;
> 
> xbuffer: '\x00\x00\xb9\x02\x13EXCLUDE_CREDIT_CARD'
> # the above was printed using repr(xbuffer).
> # Note that int(0x13) = 19 which is exactly the length of the visible
> text
> #
> 
> In the code I have the following statement;
> x = st.unpack('>xxBBp',xbuffer)
> 
> This throws out the following error;
>
> x = st.unpack('>xxBBp',xbuffer)
> error: unpack str size does not match format
> 
> As I read the documentation the "p" format string seems to
> address this situation, where the number bytes of the string to
> read is the first byte of the stored value but I keep getting this
> error.

...

Well, the docs mean it when they say:

    Note that for unpack(), the "p" format character consumes count
    bytes

You don't have an explicit count in front of your "p" code, so count
defaults to 1, so only one byte of xbuffer will get consumed.

This works, telling struct that this particular p field consumes 20
bytes (including the string-length byte):

>>> struct.unpack('>xxBB20p',xbuffer)
(185, 2, 'EXCLUDE_CREDIT_CARD')

Or, a bit more generally, assuming your p field is always at the end,
and is preceded by 4 bytes:

>>> struct.unpack('>xxBB%dp' % (len(xbuffer) - 4), xbuffer)
(185, 2, 'EXCLUDE_CREDIT_CARD')

Note that there's no direct support for any kind of variable-width
data in struct.  The number of bytes involved has to be deducible from
the format string alone.