tarfile's tar.extractfile() file-like object incompatible with pickle.load()?
Tom B.
sbabbitt at commspeed.net
Fri Aug 27 16:53:05 EDT 2004
"Matt Doucleff" <mattduke at ugcs.caltech.edu> wrote in message
news:3e6d6c21.0408271000.6007a361 at posting.google.com...
> "Tom B." <sbabbitt at commspeed.net> wrote in message
news:<1093572564.733113 at news.commspeed.net>...
> > "Matt Doucleff" <mattduke at ugcs.caltech.edu> wrote in message
> > news:3e6d6c21.0408261043.264c3a2d at posting.google.com...
> > > Hi everyone! I must be doing something wrong here :) I have a
> > > tarball that contains a single file whose contents are a pickled
> > > object. I would like to unpickle the object directly from the tarball
> > > using the file-like object provided by extractfile(). Attempts to do
> > > this result in EOFError. However if I first extract to a temporary
> > > file, then unpickle from there, it works. The below code reproduces
> > > the problem (on my machine at least). I'm running Python 2.3.4,
> > > manually installed on Debian Woody (original python removed). Thanks!
> > >
> > > This sample code creates (and then removes) files in the tmp directory
> > > and in the current working directory.
> > >
> > > # demonstrates extractfile/unpickle failure (bug?)
> > >
> > > # pickle a dict to a temp file
> > > # create tar file, add temp file to it, close tar file
> > > # open tar file for reading
> > > # obtain file-like object for pickled file using extractfile()
> > > # attempt to unpickle dict from file-like object
> > > # fails with EOFError exception
> > >
> > > import tarfile
> > > import pickle
> > > import tempfile
> > > import os
Now that I take a closer look at your program it should read,
> > > if __name__ == '__main__':
> > > try:
> > > hashtopickle = { 'a' : 1, 'b' : 2 }
> > >
> > > # pickle to temp file
> > > (fd, tmpfilename) = tempfile.mkstemp()
> > > tmpfile = file(tmpfilename,'w+')
> > > pickle.dump(hashtopickle, tmpfile)
> > > tmpfile.close()
> > >
> > > # create tar; add temp file
> > > tar = tarfile.open('tarpickle.tar', 'w')
> > > tar.add(tmpfilename, 'pickledhash')
> > > tar.close()
> > >
> > > # remove temp file
> > > os.remove(tmpfilename)
> > >
> > > # open tarfile for reading, get filelike
> > > tar = tarfile.open('tarpickle.tar', 'r')
>
> > > filelike = tar.extractfile('pickledhash')
> > >
> > > # fails
> > > hashcopy = pickle.load(filelike)
> > >
> > > finally:
> > > # cleanup
> > > os.remove('tarpickle.tar')
> >
Your opening the tmpfilename in an odd way.
Tom
More information about the Python-list
mailing list