Help: pickle module unable to load "rb" mode files in linux

[Peter Otten]

Surya Kasturi wrote:

> I am attaching the file which has to be read.. please take a look into it.
> The actual source code can be observed at
> 
https://github.com/scipy/SciPyCentral/blob/master/scipy_central/rest_comments/views.py#L235
> 
> 
> when we use "rb" mode in windows, its working. but its not working in
> linux system (particularly CentOS)
> 
> 
> 
> On Tue, Apr 2, 2013 at 8:50 PM, Peter Otten <__peter__ at web.de> wrote:
> 
>> Surya Kasturi wrote:
>>
>> > Hi, hope you can help me on it..
>> >
>> > with open(pickle_f, 'r') as fhand:
>> >         obj = pickle.load(fhand)
>> >
>> >
>> > This works on linux but not in windows until  I use "rb" mode while
>> > creating file object. Surprisingly, the "rb" mode is not working on
>> > Linux.. raising EOFError.
>> >
>> > Why is this happening?
>>
>> I don't know.
>>
>> Please give a complete self-contained example that uses "wb" to write the
>> file and "rb" to read it, and that fails on Linux. Don't forget to tell
>> us the version of Python you used to run that script.

I am using Linux, but I cannot reproduce an EOFError with the attached file:

Python 2.7.2+ (default, Jul 20 2012, 22:15:08) 
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> with open("index.fpickle", "rb") as f:
...     obj = pickle.load(f)
... 
>>> obj
{'body': u'<p>yahoo</p>\n', 'prev': None, 'display_toc': False, 'title': 
u'<no title>', 'sourcename': 'index.txt', 'customsidebar': None, 
'current_page_name': 'index', 'next': None, 'rellinks': [('genindex', 
u'General Index', 'I', u'index')], 'meta': {}, 'parents': [], 'toc': u'<ul 
class="simple">\n</ul>\n', 'sidebars': None, 'metatags': ''}

As the data has only built-in types it cannot be a library thing either.

Given that the file's location is determined dynamically

[views.py]
    call_sphinx_to_compile(settings.SPC['comment_compile_dir'])

    pickle_f = ''.join([settings.SPC['comment_compile_dir'], os.sep,
                        '_build', os.sep, 'pickle', os.sep, 
'index.fpickle'])
    with open(pickle_f, 'rb') as fhand:
        obj = pickle.load(fhand)

I'd add a

print pickle_f

statement just before the with-statement to ensure that you don't have two 
different pickle files, one of them being empty or otherwise broken.