pickle and module versioning

[Calvin Spealman]

This can definitely be a problem, but there are tried and true strategies to deal with it and the best course of action depends a lot on what works for you. Pickle deserialization is just an API instantiating your class instances. That means all the same rules and approaches to API changes apply like any other code changes.

So, if you have to add a field to a class it’ll need to be optional, so previous callers to that API can still use it and the same rule exists here. If you add a new field and its optional or has a default, then un-pickling from an older version should still work if those original fields are still valid.

It does mean the changes you can make are limited, so you need to think through the implications. One approach to mitigating that problem is to isolate the serializing to data classes intended specifically for pickling, so larger changes can happen outside those classes.


From: Neal Becker
Sent: Monday, December 17, 2018 7:02 AM
To: python-list at python.org
Subject: pickle and module versioning

I find pickle really handy for saving results from my (simulation) 
experiments.  But recently I realized there is an issue.  Reading the saved 
results requires loading the pickle, which in turn will load any referenced 
modules.  Problem is, what if the modules have changed?

For example, I just re-implemented a python module in C++, in a not quite 
compatible way.  AFAIK, my only choice to not break my setup is to choose a 
different name for the new module.

Has anyone else run into this issue and have any ideas?  I can imagine 
perhaps some kind of module versioning could be used (although haven't 
really thought through the details).

Thanks,
Neal

-- 
https://mail.python.org/mailman/listinfo/python-list