[AstroPy] new pyfits version deletes NP_pyfits, breaking pickle

[Johann Cohen-Tanugi]

well there is ROOT : http://root.cern.ch
but it might represent too large a stretch.....

Johann

On 11/11/2010 11:57 PM, Joe Harrington wrote:
> Erik, thanks for the unpickler code.  We saw the note about those but
> don't offhand know how to use it to fix the problem.
>
> Perry, thanks for looking into it.
>
> Also, if you know of *any* other way to save an object, please say.
> It seems pretty clear to me that to load objects from any kind of save
> file, you have to import the classes of the object and any objects it
> contains that are not standard Python objects.  So even if we had
> other methods for saving, they would have the same problem as pickle.
> But we have to be able to save objects!  Perhaps saving the definitions
> of the types rather than importing them would be the way to go?  I bet
> there's a long thread about this somewhere.
>
> --jh--
>
> On Thu, 11 Nov 2010 14:38:38 -0500, Perry Greenfield<perry at stsci.edu>  wrote:
>
>    
>> We'll look into it. This is a general problem with pickles (and one
>> reason I've been hesitant to avoid using them like save files). I
>> wonder if there is a better solution than that. In this case we had to
>> clean out the previous numarray interface.
>>
>> Perry
>>
>> On Nov 10, 2010, at 7:24 PM, Joe Harrington wrote:
>>
>>      
>>> My research group uses Python pickles to save data as it goes through
>>> our pipeline (.npy and .npz do not save objects, and neither does HDF,
>>> etc.).  These need to be loadable forever, as we often compare work to
>>> work done much earlier.  Some of the objects we save contain pyfits
>>> header objects.  Pickles have to import all classes used in the
>>> pickled objects before they load, and we are getting an ImportError
>>> about NP_pyfits.  The file NP_pyfits.py existed in stsci_python 2.8
>>> but is gone in 2.10.  The pickles refer to this object explicitly:
>>>
>>> ....
>>> sS'photchan'
>>> p494
>>> I3
>>> sS'header'
>>> p495
>>> (ipyfits.NP_pyfits
>>> Header
>>> p496
>>> (dp497
>>> S'_hdutype'
>>> p498
>>> cpyfits.NP_pyfits
>>> PrimaryHDU
>>> p499
>>> sS'ascard'
>>> p500
>>> ccopy_reg
>>> _reconstructor
>>> p501
>>> (cpyfits.NP_pyfits
>>> CardList
>>> p502
>>> c__builtin__
>>> list
>>> p503
>>> (lp504
>>> g501
>>> (cpyfits.NP_pyfits
>>> Card
>>> p505
>>> c__builtin__
>>> object
>>> p506
>>> NtRp507
>>> (dp508
>>> S'_valuestring'
>>> p509
>>> S'T'
>>> ....
>>>
>>> Is there any way to make our pickles readable again, other than
>>> running the old version of pyfits forever?  Can you provide a pickle
>>> converter that replaces the old names in the file with whatever is
>>> new?
>>>
>>> Please (everyone, not just STScI) be aware of this issue going
>>> forward.  Pickles are the only way we know of to save objects.  You
>>> can add things to your classes, but if you change what they import (or
>>> otherwise break pickle), nobody can restore your class across
>>> releases.
>>>
>>> Thanks,
>>>
>>> --jh--
>>>        
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
>