[SciPy-user] Possible bug in loadmat and savemat use of fread and fwrite

[Andrew Straw]

Hi Souheil,

I wrote the following email to scipy-dev last week. There is  
certainly something wrong with the low-level code at the moment  
(unless it's been fixed very recently).

Nevertheless, I got beyond the bug you posted (see the attached  
patch) only to find another...

     From:       strawman at astraw.com
     Subject:     [SciPy-dev] getting scipy.io.mio / numpyio working  
again
     Date:     December 15, 2005 2:22:22 PM PST
     To:       scipy-dev at scipy.net
     Reply-To:       scipy-dev at scipy.net

Hi,

I'm trying to get the following working:

import scipy

a={'a':scipy.arange(10)}
scipy.io.mio.savemat('test.mat',a)
b = scipy.io.mio.loadmat('test.mat')
print b


If I apply the following patch:

Index: mio.py
===================================================================
--- mio.py      (revision 1485)
+++ mio.py      (working copy)
@@ -870,7 +870,7 @@

          imagf = var.dtypechar in ['F', 'D']
          fid.fwrite([var.shape[1], var.shape[0], imagf,
len(variable)+1],'int')
-        fid.fwrite(variable+'\x00','char')
+        fid.fwrite(variable+'\x00','uchar')
          if imagf:
              fid.fwrite(var.real)
              fid.fwrite(var.imag)

I can get as far as the following traceback.  I don't have time right
now to delve deeper.

Traceback (most recent call last):
   File "numpyiotest.py", line 4, in ?
     scipy.io.mio.savemat('test.mat',a)
   File "/usr/lib/python2.3/site-packages/scipy/io/mio.py", line 873, in
savemat
     fid.fwrite(variable+'\x00','uchar')
   File "/usr/lib/python2.3/site-packages/scipy/io/mio.py", line 221, in
write
     numpyio.fwrite(self,count,data,mtype,bs)
numpyio.error: Does not support extended types.


On Dec 21, 2005, at 4:58 AM, Souheil Inati wrote:

> Hi all,
>
> I am new to the scipy (and python).  I have just gotten scipy working
> on a mac G4 OS X 10.4.4 with python 2.4.2, thanks to Chris
> Fonnesbeck's instructions.  Apologies in advance for the long post
> and if I am asking a question previously answered on the mailing
> list.  I can't figure out how to search the mailing list properly.
> Mailman itself (scipy.net) doesn't have a search and the plone
> interface to mailman on scipy.org does nothing for searching.  Google
> is not terrible.  Anyway....
>
>   I'm having problems with io functions loadmat and savemat. I've dug
> as far as my limited python knowledge can get me and my current
> working hypothesis is that there is a bug in the low-level fread and
> fwrite functions, or maybe the way in which the loadmat and savemat
> functions call them.  The code in io/examples works fine and the
> reading and writing of binary files via a combination of fopen and
> fwrite works fine as well.
>
> Any help or nudge in the proper direction would be greatly  
> appreciated.
>
> Thanks,
> Souheil
>
>
> ---------------------------------
>
> Souheil Inati, PhD
> Assistant Professor
> Center for Neural Science and Department of Psychology
> New York University
> 4 Washington Place, Room 809
> New York, N.Y., 10003-6621
> Office: (212) 998-3741
> Email: souheil.inati at nyu.edu
>
>
> ------------------------------------
> ------------------------------------
> Here's the behavior that i see for loadmat
>
> I created a file in Matlab 7.1
>
>>> A = randn(4)
>>> save -V6 foo A
>>>
>
> In python:
> from scipy import *
> b = io.loadmat('foo.mat')
> gives a bus error.
>
> Reading through io/mio.py at around line 744 in the the beginning of
> the loadmat function is this bit of code:
>      fid = fopen(full_name,'rb')
>      test_vals = fid.fread(4,'byte')
>
> If I try to run this directly from python, like this:
> from scipy import *
> fid = fopen('foo.mat','rb')
> test_vals = fid.fread(4,'byte')
> I get the same bus error.
>
> Using the lower level python file functions directly works:
> fid = file('foo.mat','rb')
> b = fid.fread(4)
> returns a string b = 'MATL'  as expected. So presumable there is an
> error in fread.
>
> ------------------------------------
> ------------------------------------
> For the savemat function the problem looks like this:
> from scipy import *
> io.savemat('foo2.mat',{'a': array([1.,2.,3.])})
>
> ---------------------------------------------------------------------- 
> --
> ---
> exceptions.TypeError                                 Traceback (most
> recent call last)
>
> /Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-
> packages/scipy/io/mio.py in savemat(filename, dict)
>      871         imagf = var.dtypechar in ['F', 'D']
>      872         fid.fwrite([var.shape[1], var.shape[0], imagf, len
> (variable)+1],'int')
> --> 873         fid.fwrite(variable+'\x00','char')
>      874         if imagf:
>      875             fid.fwrite(var.real)
>
> /Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-
> packages/scipy/io/mio.py in write(self, data, mtype, bs)
>      219         howmany,mtype = getsize_type(mtype)
>      220         count = product(data.shape)
> --> 221         numpyio.fwrite(self,count,data,mtype,bs)
>      222         return
>      223
>
> TypeError: argument 4 must be char, not None
>
>
>
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.net
> http://www.scipy.net/mailman/listinfo/scipy-user
>