[Python-Dev] Module renaming and pickle mechanisms

[Brett Cannon]

On Mon, May 19, 2008 at 12:26 PM, Brett Cannon <brett at python.org> wrote:
> On Mon, May 19, 2008 at 5:08 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>> On 2008-05-18 22:24, Brett Cannon wrote:
>>>
>>> On Sun, May 18, 2008 at 6:14 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>>>
>>>> M.-A. Lemburg wrote:
>>>>>
>>>>> Perhaps I have a misunderstanding of the reasoning behind
>>>>> doing the renaming in the 2.x branch, but it appears that
>>>>> the only reason is to get used to the new names. That's a
>>>>> rather low priority argument in comparison to the breakage
>>>>> the renaming will cause in the 2.x branch.
>>>>
>>>> I think this is the key point here. The possibility of breaking pickling
>>>> compatibility never came up during the PEP 3108 discussions, so wasn't
>>>> taken
>>>> into account in deciding whether or not backporting the name changes was
>>>> a
>>>> good idea.
>>>>
>>>> I think it's pretty clear that the code needs to be moved back into the
>>>> modules with the old names for 2.6. The only question is whether or not
>>>> we
>>>> put any effort into making the new stdlib organisation usable in 2.x, or
>>>> just rely on 2to3 to fix it (note that the "increasing the common subset"
>>>> argument doesn't really apply, since you can catch the import errors in
>>>> order to try both names).
>>>
>>> Problem with this is it makes forward-porting revisions to 3.0 a PITA.
>>> By keeping the module names consistent between the versions merging a
>>> revision is just a matter of ``svnmerge merge`` with the usual
>>> 3.0-specific changes. Reverting the modules back to the old name will
>>> make forward-porting much more difficult as I don't think svn keeps
>>> rename information around (and thus map the old name to the new name
>>> in terms of diffs).
>>
>> svnmerge is written in Python, so wouldn't it be possible to add
>> support for maintaining such renaming to that tool ?
>>
>
> Don't know, possibly. But I am not about to try to figure out.
>
>> I don't think that an administrative problem such as forward-
>> porting patches to 3.x warrants breakage in the 2.x branch.
>>
>
> That's why I suggested changing pickle to deal with the rename, but
> obviously I am in the minority in that idea.
>
>> After all, the renaming was approached for Python 3.0 and not
>> 2.6 *because* it introduces major breakage.
>>
>> AFAIR, the discussion on the stdlib-sig also didn't include the
>> plan to backport such changes to 2.6. Otherwise, we would have
>> hashed them out there.
>>
>
> Never came up.
>
>>> Alexandre's idea of teaching pickle the mapping of old names to new
>>> might be the best solution. We could have a flag to pickle that
>>> deactivates the renaming. Otherwise we could bump the pickle version
>>> number so that the new number doesn't do the mapping while the old
>>> versions to the implicit module mapping.
>>>
>>> And as Greg and Glpyh have pointed out, this is a problem that might
>>> need to be addressed in the future with some changes to our
>>> serialization method (I have no clue how since I don't deal with
>>> pickle very much).
>>
>> It is possible to make pickle aware of the module renames, but
>> that doesn't solve problems with other forms of serialization
>> or use of the .__module__ attribute in general.
>>
>> Why can't we just provide a "from __future__ import renamed_modules"
>> which then provides all the new name to old name mappings in
>> some form (e.g. module proxies or whatever) and leave the
>> existing modules in 2.x untouched ?
>
> I have started a discussion on the stdlib SIG on how to handle this,
> so I will defer this discussion to there.
>

The decision was made (by me) to just revert all of the renames in
2.6. A note will be in the docs stating the rename, but otherwise 2to3
will be relied upon for all transitions from old names to new names.

I have updated the PEP to note about which modules need to be reverted
and the new steps to rename a module, and added/re-opened the
appropriate issues (all attached to issue 2775).

-Brett