xlrd and cPickle.dump

[patrick.waldo at gmail.com]

> How many megabytes is "extremely large"? How many seconds does it take
> to open it with xlrd.open_workbook?

The document is 15mb ad 50,000+ rows (for test purposes I will use a
smaller sample), but my computer hangs (ie it takes a long time) when
I try to do simple manipulations and the documentation leads me to
believe cPickle will be more efficient.  If this is not true, then I
don't have a problem (ie I just have to wait), but I still would like
to figure out how to pickle an xlrd object anyways.

> You only need one of the above imports at the best of times, and for
> what you are attempting to do, you don't need pyExcelerator at all.

Using pyExcelerator was a guess, because the traditional way didn't
work and I thought it may be because it's an Excel file.  Secondly, I
import it twice because sometimes, and I don't know why, PythonWin
does not import pyExcelerator the first time.  This has only been true
with pyExcelerator.

> > data_path = """C:\test.xls"""
>
> It is extremely unlikely that you have a file whose basename begins with
> a TAB ('\t') character. Please post the code that you actually ran.

you're right, I had just quickly erased my documents and settings
folder to make it smaller for an example.

>
> Please post the minimal pyExcelerator-free script that demonstrates your
> problem. Ensure that it includes the following line:
>      import sys; print sys.version; print xlrd.__VERSION__
> Also post the output and the traceback (in full).

As to copy_reg.py, I downloaded Activestate Python 2.4 and that was
it, so I have had no other version on my computer.

Here's the code:

import cPickle,xlrd, sys

print sys.version
print xlrd.__VERSION__

data_path = """C:\\test\\test.xls"""
pickle_path = """C:\\test\\pickle.pickle"""

book = xlrd.open_workbook(data_path)
Data_sheet = book.sheet_by_index(0)

pickle_file = open(pickle_path, 'w')
cPickle.dump(book, pickle_file)
pickle_file.close()

Here's the output:

2.4.3 (#69, Apr 11 2006, 15:32:42) [MSC v.1310 32 bit (Intel)]
0.6.1
Traceback (most recent call last):
  File "C:\Python24\Lib\site-packages\pythonwin\pywin\framework
\scriptutils.py", line 310, in RunScript
    exec codeObject in __main__.__dict__
  File "C:\text analysis\pickle_test2.py", line 13, in ?
    cPickle.dump(book, pickle_file)
  File "C:\Python24\lib\copy_reg.py", line 69, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle module objects


Thanks for the advice!