tarfile's tar.extractfile() file-like object incompatible with pickle.load()?

[Tom B.]

"Matt Doucleff" <mattduke at ugcs.caltech.edu> wrote in message
news:3e6d6c21.0408261043.264c3a2d at posting.google.com...
> Hi everyone!  I must be doing something wrong here :)  I have a
> tarball that contains a single file whose contents are a pickled
> object.  I would like to unpickle the object directly from the tarball
> using the file-like object provided by extractfile().  Attempts to do
> this result in EOFError.  However if I first extract to a temporary
> file, then unpickle from there, it works.  The below code reproduces
> the problem (on my machine at least).  I'm running Python 2.3.4,
> manually installed on Debian Woody (original python removed).  Thanks!
>
> This sample code creates (and then removes) files in the tmp directory
> and in the current working directory.
>
> # demonstrates extractfile/unpickle failure (bug?)
>
> # pickle a dict to a temp file
> # create tar file, add temp file to it, close tar file
> # open tar file for reading
> # obtain file-like object for pickled file using extractfile()
> # attempt to unpickle dict from file-like object
> # fails with EOFError exception
>
> import tarfile
> import pickle
> import tempfile
> import os
>
> if __name__ == '__main__':
>     try:
>         hashtopickle = { 'a' : 1, 'b' : 2 }
>
>         # pickle to temp file
>         (fd, tmpfilename) = tempfile.mkstemp()
>         tmpfile = os.fdopen(fd, 'w')
>         pickle.dump(hashtopickle, tmpfile)
>         tmpfile.close()
>
>         # create tar; add temp file
>         tar = tarfile.open('tarpickle.tar', 'w')
>         tar.add(tmpfilename, 'pickledhash')
>         tar.close()
>
>         # remove temp file
>         os.remove(tmpfilename)
>
>         # open tarfile for reading, get filelike
>         tar = tarfile.open('tarpickle.tar', 'r')
>         filelike = tar.extractfile('pickledhash')
>
>         # fails
>         hashcopy = pickle.load(filelike)
>
>     finally:
>         # cleanup
>         os.remove('tarpickle.tar')

It occurs to me that you need to do,

 hashcopy = pickle.loads(filelike)

if filelike is a string.

Tom
P.S. have a look at pickle.dumps()