[AstroPy] Image combine

[Perry Greenfield]

Hi Neil,

That's a bit misleading. The application of the masking operation is  
certainly done in C. The whole issue is how tightly integrated the  
masks are with native arrays. For large arrays, it's not likely a  
significant speed issue per se, but it is perhaps a memory issue (with  
consequent speed issues if memory caches are involved).

Perry

On Mar 7, 2012, at 8:45 AM, Neil Crighton wrote:

> Hi Wolfgang,
>
> The problem is that masked arrays in numpy are not implemented in c,
> so they are not c-speed.  There is a proposal to implement a masked
> array in c (https://github.com/numpy/numpy/blob/master/doc/neps/missing-data.rst 
> )
> but this hasn't been finalised.
>
> Cheers, Neil
>
>
> On 7 March 2012 14:41, Wolfgang Kerzendorf <wkerzendorf at gmail.com>  
> wrote:
>> Hello Jose,
>>
>> What you're trying to do is already half implemented in numpy and  
>> called
>> masked arrays:
>>
>> Let's say you want to stack images and you have put them together  
>> in a 3d
>> cube (first axis being the images):
>>
>> myimages = numpy.ma.MaskedArray(rand(5,100,100),
>> mask=zeros((5,100,100).astype(bool))
>>
>> let's do a simple sigma clipping algorithm:
>>
>> image_mean = mean(myimages)
>> image_std = std(myimages)
>>
>> #now let's adjust the mask to mask the pixels that are more than 1  
>> sigma out
>>
>> new_mask = abs(myimages - image_mean) > 1.
>>
>> myimages.mask = new_mask
>>
>> #if you do a mean now it will ignore the values where the mask = True
>>
>> mean(myimages, axis=0)
>>
>> This will be all near C-speeds (I guess a factor of 100 is easily  
>> in there).
>> ----
>>
>> As other's have pointed out using i,j in a loop is a very bad idea  
>> for numpy
>> arrays. Numpy arrays are much more easy to handle than in most other
>> languages, there is a bit of getting used to required though. I  
>> recommend
>> reviewing the different operations
>> on http://www.scipy.org/Tentative_NumPy_Tutorial.
>>
>> Hope that helps,
>>
>>     Wolfgang
>> On 2012-03-07, at 4:41 AM, Sergio Pascual wrote:
>>
>> Hello
>>
>> I have written a image combine implementation for the pipeline of
>> EMIR[1], a near infrared instrument for the 10m GTC Telescope. It  
>> does
>> scaling,
>> weighting and masking. It's written as a C/C++ extension. It's part  
>> of
>> a larger package, numina, used for other GTC instruments, but I may
>> split combine if there is interest in it. The package, called numina,
>> its under heavy development, but the combine part is fairly stable.
>>
>> You can see the code here https://guaix.fis.ucm.es/hg/numina/
>>
>> The combine part is the module numina.array.combine, fairly all the
>> C/C++ code is in src/
>>
>> Regards, Sergio
>>
>>
>> [1] http://www.gtc.iac.es/pages/instrumentacion/emir.php
>> [2] http://www.gtc.iac.es/
>>
>>
>>
>> El día 6 de marzo de 2012 12:00, Jose Miguel Ibáñez  
>> <ppmime at gmail.com>
>> escribió:
>>
>> Hello everyone,
>>
>>
>> does anyone know of an implementation of the iraf.imcombine task in
>>
>> python+numpy ? (of course, not using pyraf.imcombine call)
>>
>>
>>
>> Thanks !
>>
>> Jose
>>
>> _______________________________________________
>>
>> AstroPy mailing list
>>
>> AstroPy at scipy.org
>>
>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>>
>>
>>
>> --
>> Sergio Pascual     http://guaix.fis.ucm.es/~spr    +34 91 394 5018
>> gpg fingerprint: 5203 B42D 86A0 5649 410A F4AC A35F D465 F263 BCCC
>> Departamento de Astrofísica -- Universidad Complutense de Madrid  
>> (Spain)
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>>
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy