[Python-porting] pickle data

[Benjamin Peterson]

On Tue, Dec 16, 2014, at 20:26, Barry Warsaw wrote:
> Here's an interesting situation I am faced with as I port Mailman 3 to
> Python
> 3.  I haven't seen any other discussion of it, so I thought I'd post here
> for
> posterity.
> 
> Let's say you have a pickle created in Python 2 that is to be read in
> Python
> 3.  In Mailman 2, persistent mailing list data is stored in pickles.
> 
> It seems like both Python 2 types (unicode and str/bytes) get unpickled
> as
> Python 3 str types.

You can change this by passing encoding="bytes" to pickle.loads. See
https://docs.python.org/3/library/pickle.html#pickle.load

> 
> * Python 2 unicode should unpickle as Python 3 str
> * In Python 2, bytes are just an alias for str
> * There's no way to know the intent of whether Python 2 "bytes" should be
>   unpickled as Python 3 bytes or str.

Yeah, this is problematic. Probably the best you can do is unpickle
everything with bytes then manually decode actual string things into
str.