[AstroPy] "double free or corruption" crash: PyFITS + NumPy 1.6.0

Tue Jul 5 19:15:02 EDT 2011

I don't see this on my MacOS X 10.6 box. I can load the file just fine.

However, I've been getting memory problems with numpy and Python 2.7 as well. I've reluctantly decided to go back to Python 2.6 to release applications that others can reliably use. I spent some time trying to reproduce the problem today and found one case that reliably causes numpy.subtract to segfault on MacOS X 10.5 (pyfits is not involved), but so far no luck on 10.6. I submitted a ticket.

Regards,

-- Russell

On Jul 3, 2011, at 1:02 PM, Tom Aldcroft wrote:

> I'm seeing some sort of memory or allocation error (glibc detected)
> when trying to read a particular FITS file with the following setup:
> 
> OS: CentOS-5.6 (64-bit)
> Python: 2.7.1.4 (ActiveState distribution)
> NumPy: 1.6.0
> PyFits: 2.4.0
> 
> If I downgrade to NumPy 1.5.0 and re-build everything then I do not
> see this problem, while PyFITS 2.2.2 + NumPy 1.6.0 did show this
> crash.  I've put the troublesome file on:
> 
>   http://hea-www.harvard.edu/~aldcroft/tmp/glibc_detected.fits.gz
> 
> This particular file is the output of the Chandra X-ray Center
> automated processing which converts raw Chandra telemetry to "Level-0
> engineering data".  I've read hundreds of thousands of such files with
> Python 2.6, PyFits 2.2.2 and NumPy 1.3.0 without issue.  I recently
> upgraded Python, PyFits and NumPy and ran into this problem.  Other
> similar Level-0 files were read OK with the new versions, but not the
> one in question.  I did a significant amount of regression testing
> before doing this version upgrade and didn't see any differences, but
> my testing didn't hit this particular file type.  For now I'm back at
> NumPy 1.5.0.
> 
> Regards,
> Tom Aldcroft
> 
> -----------------
> 
> ccosmos% python
> ActivePython 2.7.1.4 (ActiveState Software Inc.) based on
> Python 2.7.1 (r271:86832, Feb  7 2011, 11:30:54)
> [GCC 4.0.2 20051125 (Red Hat 4.0.2-8)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import pyfits
>>>> hdus = pyfits.open('glibc_detected.fits.gz')
>>>> dat = hdus[1].data
>>>> print dat
> *** glibc detected ***
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python: double free
> or corruption (!prev): 0x0000000017aea970 ***
> ======= Backtrace: =========
> /lib64/libc.so.6[0x390d07245f]
> /lib64/libc.so.6(cfree+0x4b)[0x390d0728bb]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/lib/python2.7/site-packages/numpy/core/multiarray.so[0x2b97362a94f2]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/lib/python2.7/site-packages/numpy/core/umath.so[0x2b9736462061]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/lib/python2.7/site-packages/numpy/core/umath.so[0x2b9736475b27]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/lib/python2.7/site-packages/numpy/core/umath.so[0x2b973647739b]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyObject_Call+0x62)[0x41a032]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_EvalFrameEx+0x4fd4)[0x494954]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_EvalCodeEx+0x675)[0x496de5]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_EvalFrameEx+0x5bbd)[0x49553d]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_EvalCodeEx+0x675)[0x496de5]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_EvalFrameEx+0x5bbd)[0x49553d]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_EvalCodeEx+0x675)[0x496de5]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_EvalFrameEx+0x5bbd)[0x49553d]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_EvalCodeEx+0x675)[0x496de5]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python[0x4f7717]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyObject_Call+0x62)[0x41a032]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python[0x421f07]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyObject_Call+0x62)[0x41a032]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python[0x46e99f]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_EvalFrameEx+0x68bf)[0x49623f]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_EvalFrameEx+0x5c52)[0x4955d2]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_EvalCodeEx+0x675)[0x496de5]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_EvalFrameEx+0x5bbd)[0x49553d]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_EvalCodeEx+0x675)[0x496de5]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_EvalFrameEx+0x5bbd)[0x49553d]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_EvalCodeEx+0x675)[0x496de5]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python[0x4f7717]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyObject_Call+0x62)[0x41a032]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_CallObjectWithKeywords+0x55)[0x48ed15]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/lib/python2.7/site-packages/numpy/core/multiarray.so[0x2b97362873b4]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(_PyObject_Str+0x69)[0x449cc9]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyObject_Str+0x13)[0x449da3]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python[0x449f7d]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyFile_WriteObject+0x7f)[0x42e04f]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_EvalFrameEx+0x4b8d)[0x49450d]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_EvalCodeEx+0x675)[0x496de5]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyEval_EvalCode+0x32)[0x4970e2]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyRun_InteractiveOneFlags+0x190)[0x4b9e10]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyRun_InteractiveLoopFlags+0x4e)[0x4ba01e]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(PyRun_AnyFileExFlags+0x4c)[0x4ba12c]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python(Py_Main+0xa0e)[0x415bee]
> /lib64/libc.so.6(__libc_start_main+0xf4)[0x390d01d994]
> /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python[0x414df9]
> ======= Memory map: ========
> 00400000-0054b000 r-xp 00000000 08:04 106168389
>  /data/cosmos2/ska/arch/x86_64-linux_CentOS-5/bin/python
> 0064b000-0068f000 rw-p 0014b000 08:04 106168389
>  /data/cosmos2/ska/arcAbort
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy