[SciPy-User] loadmat, savemat and strings

[Angus McMorland]

Hi all,

I'm running in to problems trying to save some metadata in a matlab file
variable written with savemat, and read in with loadmat. I've condensed the
problem to the following:

import numpy as np
from scipy import io
a = np.array(['HelloWorld', 'Foobar'])
io.savemat('tmp.mat', dict(a=a))
res = io.loadmat('tmp.mat')

print res['a']
->
array([u'HloolWw', u'elWrdo'],
      dtype='<U10')

i.e. the character order alternates incorrectly between the values. It
doesn't appear this is just a byte order issue (or quite possibly I have no
idea how to swap the byteorder around) since:

print res['a'].view(dtype=('>U10'))
->
array([u'\U48000000\U6c000000\U6f000000\U6f000000\U6c000000\U57000000\U77000000',
       u'\U65000000\U6c000000\U57000000\U72000000\U64000000\U6f000000'],
      dtype='>U10')

and

res['a'].byteswap() gives the same result.

Finally, I've tried coercing the input into a unicode type before saving...

print a.astype('U')
->array([u'HelloWorld', u'Wow'],
      dtype='<U10')

so far so good, but then

io.savemat('tmp.mat', dict(a=a.astype('U')))
res = io.loadmat('tmp.mat')

throws the error:

...
/usr/lib/python2.6/dist-packages/scipy/io/matlab/mio.pyc in
loadmat(file_name, mdict, appendmat, **kwargs)
    109     '''
    110     MR = mat_reader_factory(file_name, appendmat, **kwargs)
--> 111     matfile_dict = MR.get_variables()
    112     if mdict is not None:
    113         mdict.update(matfile_dict)

/usr/lib/python2.6/dist-packages/scipy/io/matlab/miobase.pyc in
get_variables(self, variable_names)
    359                 getter.to_next()
    360                 continue
--> 361             res = getter.get_array()
    362             mdict[name] = res
    363             if getter.is_global:

/usr/lib/python2.6/dist-packages/scipy/io/matlab/miobase.pyc in
get_array(self)
    400     def get_array(self):
    401         ''' Gets an array from matrix, and applies any necessary
processing '''
--> 402         arr = self.get_raw_array()
    403         return self.array_reader.processor_func(arr, self)
    404

/usr/lib/python2.6/dist-packages/scipy/io/matlab/mio5.pyc in
get_raw_array(self)
    442                           dtype=np.dtype('U1'),
    443                           buffer=np.array(res),
--> 444                           order='F').copy()
    445
    446

TypeError: buffer is too small for requested array

Is this a bug (I guess it shouldn't throw errors quite like this in any
case), and is there a successful method for saving string types into matlab
files and retrieving them?

Thanks,

Angus.

-- 
AJC McMorland
Post-doctoral research fellow
Neurobiology, University of Pittsburgh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20100503/90a952cc/attachment.html>