enhancement request: make py3 read/write py2 pickle format

[Robert Kern]

On 2015-06-10 12:04, Neal Becker wrote:
> Chris Warrick wrote:
>
>> On Tue, Jun 9, 2015 at 8:08 PM, Neal Becker <ndbecker2 at gmail.com> wrote:
>>> One of the most annoying problems with py2/3 interoperability is that the
>>> pickle formats are not compatible.  There must be many who, like myself,
>>> often use pickle format for data storage.
>>>
>>> It certainly would be a big help if py3 could read/write py2 pickle
>>> format. You know, backward compatibility?
>>
>> Don’t use pickle. It’s unsafe — it executes arbitrary code, which
>> means someone can give you a pickle file that will delete all your
>> files or eat your cat.
>>
>> Instead, use a safe format that has no ability to execute code, like
>> JSON. It will also work with other programming languages and
>> environments if you ever need to talk to anyone else.
>>
>> But, FYI: there is backwards compatibility if you ask for it, in the
>> form of protocol versions. That’s all you should know — again, don’t
>> use pickle.
>
> I believe a good native serialization system is essential for any modern
> programming language.  If pickle isn't it, we need something else that can
> serialize all language objects.  Or, are you saying, it's impossible to do
> this safely?

By the very nature of the stated problem: serializing all language objects. 
Being able to construct any object, including instances of arbitrary classes, 
means that arbitrary code can be executed. All I have to do is make a pickle 
file for an object that claims that its constructor is shutil.rmtree().

This is fine in some use cases (e.g. wire format for otherwise-secured 
communication between two endpoints under your complete control), but it is 
worrying in others, like your use case of data storage (and presumably sharing).

Python 2/3 is also the least of your compatibility worries there. Refactor a 
class to a different module, or did one of your third-party dependencies do 
this? Poof! Your pickle files no longer work.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco