[Python-3000] [Python-Dev] inst_persistent_id

[Alexandre Vassalotti]

On Jan 22, 2008 10:30 AM, Jim Fulton <jim at zope.com> wrote:
>
> On Jan 20, 2008, at 10:39 PM, Alexandre Vassalotti wrote:
>
> > On Jan 20, 2008 4:14 PM, Jim Fulton <jim at zope.com> wrote:
> >>
> >> Interesting. Does that mean that cPickle can be extended through
> >> inheritance?
> >>
> >
> > Well, Pickler and Unpickler can be subclassed. I am not sure they can
> > be really "extended," since, unlike the Python version, the C version
> > of Pickler and Unpickler only exposes a minimal API.
>
> Then I don't understand what you mean by:
>
> "I removed the differences between the Python and C implementation of
> pickle, and thus allowing the C version to be transparently used,
> instead of the Python one, when it is available."
>
> People *do* routinely subclass pickle.Pickler and pickle.Unpickler and
> if this isn't possible with cPickle, then it can't be substituted for
> pickle.  If making cPickle.Pickler and cPickle.Unpickler subclassible
> in the same way as the pickle classes makes cPickle much slower, then
> I don't think the "transparency" is worth the tradeoff.
>

I am not sure what you mean by "cPickle.Pickler and cPickle.Unpickler
subclassible in the same way as the pickle classes." It is possible to
subclass the C implementation. However, the C implementation does not
expose pickle "private" methods, such as Pickler.save_int and
Unpickler.load_tuple. This is fine because these were never documented
as part of the interface of pickle. Only Pickler.dump,
Pickler.clear_memo and Unpickler.load are supported by the Python
implementation. All these are supported by the C implementation as
well.

So, what kind of "extensibility" are you looking for the C
implementation of pickle?

Quick note, one of the original motives for making the C
implementation of pickle transparent was to make it more "friendly"
for alternate implementations of Python (e.g., Jython, IronPython,
etc).

>
> > Well, I haven't changed cPickle much to make it subclassable. So, the
> > impact is minimal. Currently, the only performance change is due the
> > removal of the special-cases for PyFile and cStringIO "buffers."
>
> What is the benefit of removing this special casing? What is the cost?
>

As far as know, most of PyFile is gone in Py3k -- i.e., it been
castrated beyond recognition [1]. Therefore, the special case for
PyFile didn't improve the performance (in fact, it probably made
things go slower). I removed the special case for cStringIO for
similiar reason -- i.e., cStringIO will be replaced by the respective
C implementations of io.BytesIO and io.StringIO. I also did it because
it allowed me to keep the part of pickle that aren't speed critical
written in Python, thus cleaning out almost 1000 lines of code. The
performance cost of this is minimal and constant.

-- Alexandre

.. [1] http://svn.python.org/view/python/branches/py3k/Objects/fileobject.c?rev=59311&view=markup