Library for parsing binary structures

[dieter]

Paul Moore <p.f.moore at gmail.com> writes:

> On Thu, 28 Mar 2019 at 08:15, dieter <dieter at handshake.de> wrote:
> ...
> My real interest is in whether any
> libraries exist to do this sort of thing (there are plenty of parser
> libraries for text, pyparsing being the obvious one, but far fewer for
> binary structures).

Sure. *BUT* the library must fit your specific binary structure.
How should a general libary know how to interpret your
specific "type byte"s or that "(" introduces a homogenous
sequence of given length which must be terminated by ")"?

On the other hand, if those specifics are known, then
the remaining is trivial (as shown in my previous message).


If the binary structure is not fixed (i.e.
you deserialize only things you yourself have serialized), then you
can use Python's "pickle" (and likely also "marshal").
It supports the structuring you need and (among
others) the Python elementary types.


There is also "asn1" (--> "https://pypi.org/project/asn1/")
for ASN.1 (BER/DER) binary formats.
ASN.1 is a widely used very flexible language to describe structured data
(which typically has a binary encoding) -- used e.g.
by LDAP and X.509. It supports (among others) an extremely rich set of
elementary types and structuring via "Sequence", "Set" and "Choice".

The elementary binary format is "tag value".
This is near to your "type_byte value". However, "tag" is
not a byte. Instead, it consists of a number (identifying the
type within its class), a class and an encoding indication.
This more general type specification is necessary as
in the general case, a byte is not sufficient to identify all
possible relevant types.