[issue16475] Support object instancing and recursion in marshal

[Kristján Valur Jónsson]

Kristján Valur Jónsson added the comment:

Serhiy, to answer your questions:

> Can you please measure the time of unmarshalling?
Sure. Normally, I wouldn't consider time to be important here because for code objects, unmarshaling is done only once per run  However, testing it is simple:
I added timeit(number=100) for unmarshaling of the different version code.
0.09614 0.09605 0.582672 0.553599
0.05590 0.05628 0.293370 0.285517
0.05703 0.05754 0.294601 0.292070
The lines are version 2, 3, and 4(*) respectively.  The columns are test_marshal.py, recode.intern(test_marshal.py), decimal.py and recode.intern(decimal.py)

As you see, loading time is almost halfed with version 3 and 4 compared to 2.  Version 3 is also slightly faster than 4
(*)verion 4 is a "special" version that only instances strings.


> If you can count the statistics about marshalled types
> (what percent of shared and non shared integers, strings, etc),
> it would also be very interesting.
That's more tricky, at least on a type=by-type basis, but I could do a global object count.  Later.

> There is at least one reason. This increases size of the refs table.
I checked this, by printing out the size of the instance list when loads() was done:
457 1571
297 1163
429 1539
The columns are test_marshal.py and decimal.py
the lines are version 3, version 4( only strings ) and special version5 which is like 3 but omits ints.
As you see, the ints correspond to roughly 6% and 2% of the instances respectively.  The bulk of the list is taken up by strings (65% and 74%)

This shows that adding instancing of all other types on top of the strings does not typically expand the instance list more than 50%

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16475>
_______________________________________