[Python-checkins] r71089 - python/branches/py3k/Doc/library/pickle.rst
alexandre.vassalotti
python-checkins at python.org
Fri Apr 3 08:13:30 CEST 2009
Author: alexandre.vassalotti
Date: Fri Apr 3 08:13:29 2009
New Revision: 71089
Log:
Add more examples in pickle documentation.
Modified:
python/branches/py3k/Doc/library/pickle.rst
Modified: python/branches/py3k/Doc/library/pickle.rst
==============================================================================
--- python/branches/py3k/Doc/library/pickle.rst (original)
+++ python/branches/py3k/Doc/library/pickle.rst Fri Apr 3 08:13:29 2009
@@ -14,6 +14,7 @@
.. sectionauthor:: Jim Kerr <jbkerr at sr.hp.com>.
.. sectionauthor:: Barry Warsaw <barry at zope.com>
+
The :mod:`pickle` module implements a fundamental, but powerful algorithm for
serializing and de-serializing a Python object structure. "Pickling" is the
process whereby a Python object hierarchy is converted into a byte stream, and
@@ -282,23 +283,9 @@
.. attribute:: memo
- Dictionary holding previously pickled objects to allow shared or
- recursive objects to pickled by reference as opposed to by value.
-
-
-.. XXX Move these comments to somewhere more appropriate.
-
-It is possible to make multiple calls to the :meth:`dump` method of the same
-:class:`Pickler` instance. These must then be matched to the same number of
-calls to the :meth:`load` method of the corresponding :class:`Unpickler`
-instance. If the same object is pickled by multiple :meth:`dump` calls, the
-:meth:`load` will all yield references to the same object.
-
-Please note, this is intended for pickling multiple objects without intervening
-modifications to the objects or their parts. If you modify an object and then
-pickle it again using the same :class:`Pickler` instance, the object is not
-pickled again --- a reference to it is pickled and the :class:`Unpickler` will
-return the old value, not the modified one.
+ Dictionary-like object holding previously pickled objects to allow
+ shared or recursive objects to pickled by reference as opposed to
+ by value.
.. class:: Unpickler(file, [\*, encoding="ASCII", errors="strict"])
@@ -345,6 +332,11 @@
how they can be loaded, potentially reducing security risks. Refer to
:ref:`pickle-restrict` for details.
+ .. attribute:: memo
+
+ Dictionary holding previously unpickled objects to allow shared or
+ recursive objects to unpickled by reference as opposed to by value.
+
.. _pickle-picklable:
@@ -593,7 +585,7 @@
line = self.file.readline()
if not line:
return None
- if line.endswith("\n"):
+ if line.endswith('\n'):
line = line[:-1]
return "%i: %s" % (self.lineno, line)
@@ -707,25 +699,24 @@
As our examples shows, you have to be careful with what you allow to be
unpickled. Therefore if security is a concern, you may want to consider
-alternatives such as the marshalling API in :mod:`xmlrpc.client` or third-party
-solutions.
+alternatives such as the marshalling API in :mod:`xmlrpc.client` or
+third-party solutions.
.. _pickle-example:
-Usage Examples
---------------
+Examples
+--------
-For the simplest code, use the :func:`dump` and :func:`load` functions. Note
-that a self-referencing list is pickled and restored correctly. ::
+For the simplest code, use the :func:`dump` and :func:`load` functions. ::
import pickle
# An arbitrary collection of objects supported by pickle.
data = {
- 'a': [1, 2.0, 3, 4+6j],
- 'b': ("character string", b"byte string"),
- 'c': set([None, True, False])
+ 'a': [1, 2.0, 3, 4+6j],
+ 'b': ("character string", b"byte string"),
+ 'c': set([None, True, False])
}
with open('data.pickle', 'wb') as f:
@@ -743,11 +734,72 @@
data = pickle.load(f)
+Reusing Pickler Instances
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+It is possible to make multiple calls to the :meth:`dump` method of the same
+:class:`Pickler` instance. These must then be matched to the same number of
+calls to the :meth:`load` method of the corresponding :class:`Unpickler`
+instance. If the same object is pickled by multiple :meth:`dump` calls, the
+:meth:`load` will all yield references to the same object.
+
+Please note, this is intended for pickling multiple objects without intervening
+modifications to the objects or their parts. If you modify an object and then
+pickle it again using the same :class:`Pickler` instance, the object is not
+pickled again --- a reference to it is pickled and the :class:`Unpickler` will
+return the old value, not the modified one. ::
+
+ import io
+ import pickle
+
+ data = {"hello": 0, "spam": 1}
+
+ # Create a binary file-like object to which the Pickler instance will
+ # write the pickles.
+ f = io.BytesIO()
+ p = pickle.Pickler(f)
+ p.dump(data)
+
+ # This second call appends a new pickle to the file. The modification we
+ # make is lost because objects are pickled by reference when seen again.
+ data["eggs"] = 2
+ p.dump(data)
+
+ # Now, we load the pickles saved in our file-like object.
+ f.seek(0)
+ u = pickle.Unpickler(f)
+ data1 = u.load()
+ data2 = u.load()
+
+ if data1 is data2:
+ print("data1 and data2 are the same object")
+ else:
+ print("data1 and data2 are not the same object")
+
+ if "eggs" in data2:
+ print("The modification was pickled.")
+ else:
+ print("The modification was not pickled.")
+
+Reusing a :class:`Pickler` instance like we shown can be a useful
+optimization. For example, a multi-process application could use this feature
+to reduce the size of the pickles transmitted across processes over time
+(assuming the pickles exchanged are containers sharing common immutable
+objects). However, you should take special care to regularly clear
+:attr:``Pickler.memo`` and :attr:``Unpickler.memo`` to avoid memory-leaks.
+
+.. XXX: Add examples showing how to optimize pickles for size (like using
+.. pickletools.optimize() or the gzip module).
+
+
.. seealso::
Module :mod:`copyreg`
Pickle interface constructor registration for extension types.
+ Module :mod:`pickletools`
+ Tools for working with and analyzing pickled data.
+
Module :mod:`shelve`
Indexed databases of objects; uses :mod:`pickle`.
More information about the Python-checkins
mailing list