[scikit-learn] Model checksums

Stuart Reynolds stuart at stuartreynolds.net
Tue Dec 13 13:27:05 EST 2016


I'd like to cache some functions to avoid rebuilding models like so:

    @cached
    def train(model, dataparams): ...


model is an (untrained) scikit-learn object and dataparams is a dict.
The @cached annotation forms a SHA checksum out of the parameters of the
function it annotates and returns the previously calculated function result
if the parameters match.

The tricky part here is reliably generating a checksum from the parameters.
Scikit uses Python's pickle (
http://scikit-learn.org/stable/modules/model_persistence.html) but the
pickle library is non-deterministic (same inputs to pickle.dumps yields
differing output! -- *I know*).

So... any suggestions on how to generate checksums from models in python?

Thanks.
- Stuart
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20161213/8a7045de/attachment.html>


More information about the scikit-learn mailing list