Designing socket messaging format

[David Bolen]

shriek at gmx.co.uk (Stephen) writes:

> Just read Dan Bernstein's description. Interesting note about
> security considerations using CRLF, which follows on from an
> earlier post in this thread.

Well, but of course the bug here wasn't really using the CRLF
terminator, but the laziness of the programmer in handling the general
case of a buffer overflow.  And a lazy programmer can cause a security
problem in oh so many ways.

It's just as easy to discard overflows while watching for a terminator
when you run out of buffer space as it is to discard invalid length
messages after you've received a length that is too long.  And in
fact, they have more in common than not, since even in the length
counted case, it's going to be up to the receiver to "absorb" the data
in the stream that it will be ignoring.  So it's really just a
question as to when you decide that the message is invalid and needs
to be skipped.

One could even argue that the delimited case is also a touch more
robust in the presence of data corruption.  If corruption in the
stream affects the delimiter than the receiver (assuming no timeout)
will wait indefinitely, but any eventual retransmission from the
client will eventually trigger a new recognition of the retransmitted
terminator.  And then probably throwing away the mess of two packets
:-) Whereas, in the length case, corruption of the length field will
not only affect the receivers proper reception of the data but will
also complicate its ability to resynchronize with the sender for any
retransmissions, since it has to "find" the length field again.

So in custom protocol formats it can also be helpful to have a magic
marker for start of packets that, if present, is quoted in the stream,
to aid in any resynchronization.  Adding a CRC can't hurt either.  Of
course, this assumes you aren't opening and tearing down connections
for each packet, which in most cases is a lot of overhead.

--
-- David
-- 
/-----------------------------------------------------------------------\
 \               David Bolen            \   E-mail: db3l at fitlinxx.com  /
  |             FitLinxx, Inc.            \  Phone: (203) 708-5192    |
 /  860 Canal Street, Stamford, CT  06902   \  Fax: (203) 316-5150     \
\-----------------------------------------------------------------------/