pickle performance on larger objects

Geoffrey Talvola gtalvola at nameconnector.com
Wed Jul 17 17:13:25 EDT 2002


Try using binary pickle format (pass in 1 as an extra third argument to
dump()).  It's quite a bit faster both dumping and loading.

- Geoff

> -----Original Message-----
> From: Sam Penrose [mailto:spenrose at intersight.com]
> Sent: Wednesday, July 17, 2002 5:09 PM
> To: python-list at python.org
> Subject: pickle performance on larger objects
> 
> 
> On a recent project we decided to use pickle for some quick-and-dirty
> object persistence. The object in question is a list of 3,000 
> dictionaries
> whose keys and values are short (< 100 character) strings--about 1.5
> megs worth of character data in total. Loading this object 
> from a pickle
> using cPickle took so long we assumed something was broken.
> 





More information about the Python-list mailing list