rencode -- Reduced length encodings

[Connelly Barnes]

A space-efficient serialization module based on bencode,
which can be used to decode strings from untrusted sources.

Works well for complex, heterogeneous data structures with
many small elements.  The encodings take considerably less
space than bencode, gherkin, and Recipe 415503 for that use
case, and this module is faster, too.

Source (MIT license) is available from:

  http://barnesc.blogspot.com/2006/01/
  rencode-reduced-length-encodings.html

---------------------------------------------------------------------
Comparison of rencode with similar modules
---------------------------------------------------------------------

Connelly Barnes, 2006-01-23.

The following data structures were used:

Case 1: range(100)
Case 2: range(10000)
Case 3: {192: [3.3492934923942394, 1.230030343242342, 9.4929349232349243,
               9.4939349349394931, 4.342992349295949, 0.4309953949239393],
         298: [6.3249093409490905, 2.439009234090990, 0.9139904909090434,
               5.8795437823407908, 3.529098080909880, 4.5899843289893433]}
        All modules were modified to use 64-bit float encoding.
Case 4: {'abc': 'abc0123', 'def': 'def0123', 'ghi': 'ghi0123',
         '0123456789': '0123456789'}
Case 5: ' '*10000

---------------------------------------------------------------------
Results
---------------------------------------------------------------------

The four modules rencode, bencode, gherkin and Recipe 415503 were
used to encode each of the preceding data structures.  The size of
the encoded string and the number of calls per second to the encoding
function (on a Windows 3.0 GHz Intel machine) are summarized:

            rencode      bencode      gherkin       recipe
Case 1:         170          392          511          505 bytes
          19315.318    10002.625     6080.862     5170.539 calls/sec
Case 2:       29842        58892        50011        50005 bytes
            148.454       88.767       59.236       52.619 calls/sec
Case 3:         117          N/A          149          143 bytes
          51254.721          N/A    15470.597    17869.132 calls/sec
Case 4:          59           70          121          115 bytes
         118209.471    64589.227    20216.850    22516.589 calls/sec
Case 5:       10006        10006        10011        10005 bytes
         100186.770   100663.271    75245.603   102635.125 calls/sec


The source code used to build this table is available at:

  http://oregonstate.edu/~barnesc/python/rencode_compare.py


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com