[Numpy-discussion] numpy.lib.npyio.load

[Jean-Louis Durrieu]

Hi everyone,

I was just wondering something: lately, I had to use the load function, to load arrays stored in npz files. 

During one session, I need to read quite a few times several files (or even the same files), for some model training. I however just found out that the batch processing I ran failed because of a "too many open files" problem. 

After checking, with lsof, it seems that the use of np.load(filename), where filename is a string (= path to the file), worked an unexpected way. When I do the following, in a ipython 0.11 session, with the --pylab option :

In [1]: np.__version__
Out[1]: '1.6.1'

In [2]: np.load
Out[2]: <function numpy.lib.npyio.load>

In [3]: struc = np.load('path/to/file.npz')

In [4]: ar1 = struc['ar1']

I would expect to have opened a file, read the array in it, and closed it. However, 'lsof' proved me wrong, and I found out that I need to explicitly do 'struc.close()' in order to close the file. 

While this is not a big issue, since one has to do that anyway when opening/closing files, I was a bit surprised that I had to do it when using the load function. Maybe this behaviour should be made explicit in the documentation? (at least, in the help(load), nothing is said about that, as far as I could read). 

I just wanted to share that, just in case someone was crossing the same type of issues. I hope this is going to be useful for others!

Best regards,

Jean-Louis Durrieu