From ejensen1 at swarthmore.edu  Mon Aug  3 23:01:21 2020
From: ejensen1 at swarthmore.edu (Eric Jensen)
Date: Mon, 3 Aug 2020 23:01:21 -0400
Subject: [AstroPy] Rebin FITS images and preserve or recalculate WCS?
Message-ID: <37B76AF8-5114-4F2F-B9EB-ED99AB01069E@swarthmore.edu>

Hi all, 

We are looking at purchasing a new CMOS camera that has 4-micron pixels, which would significantly undersample our PSF. (The camera is otherwise excellent for our needs, e.g. excellent QE.)  So I?m looking at whether there would be a straightforward way in scripted post-processing to reduce the image file sizes while not losing information.  The existing camera driver doesn?t support more than 2x2 binning at this time. 

Is there available Python code that can take a FITS image with a valid WCS in the header and rebin it, and output a FITS image that still has a valid WCS for the rebinned image?  The rebinning should also preserve flux, though I think that?s easier than the WCS part.  Nothing obvious turns up in a little searching, but I could easily have missed it.  

It seems that Montage might be able to do this (mShrink) but if possible I?d prefer a pure-Python solution, as it would be easier to implement under Windows on our observatory control computer.  (A fallback would be to install a Linux distro on top of the Windows Subsystem for Linux, but if it?s possible to do it purely in Python that would be a lot simpler.) 

Thanks for any thoughts, 

Eric

Eric Jensen
Professor of Astronomy
Dept. of Physics and Astronomy
Swarthmore College
 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3946 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/astropy/attachments/20200803/2be277ff/attachment.bin>

From garyb at PHYSICS.UPENN.EDU  Tue Aug  4 05:44:19 2020
From: garyb at PHYSICS.UPENN.EDU (Bernstein, Gary M)
Date: Tue, 4 Aug 2020 09:44:19 +0000
Subject: [AstroPy] Rebin FITS images and preserve or recalculate WCS?
In-Reply-To: <37B76AF8-5114-4F2F-B9EB-ED99AB01069E@swarthmore.edu>
References: <37B76AF8-5114-4F2F-B9EB-ED99AB01069E@swarthmore.edu>
Message-ID: <CB15266E-2189-454A-AAC0-8A2D1DDEAA71@upenn.edu>

Hi Eric -
I don?t know whether this exists but in this case it might be faster to write than to do the Google search.  The block summing can be done in numpy by reshaping the array.  In the case where the original image shape is divisible by the shrinkage factor s, you?d do

m,n = img.shape
img = np.sum( img.reshape(m//s,s,n), axis=1)  # Contract along y
Img = np.sum( img.reshape(m//s,n//s,s),axis=2). # Contract along x

And to fix the WCS, you need to divide CRPIXn by s, and multiply CDELTn by s.  There is some slight complication to the first operation because FITS assumes 1-indexed pixels, and the integer marks the center of a pixel, so its really CRPIXn -> (CRPIXn?0.5)/s + 0.5

I think this should work universally because the linear transform is always the first step of a WCS mapping.

Cheers,
Gary

> On Aug 3, 2020, at 11:01 PM, Eric Jensen <ejensen1 at swarthmore.edu> wrote:
> 
> Hi all, 
> 
> We are looking at purchasing a new CMOS camera that has 4-micron pixels, which would significantly undersample our PSF. (The camera is otherwise excellent for our needs, e.g. excellent QE.)  So I?m looking at whether there would be a straightforward way in scripted post-processing to reduce the image file sizes while not losing information.  The existing camera driver doesn?t support more than 2x2 binning at this time. 
> 
> Is there available Python code that can take a FITS image with a valid WCS in the header and rebin it, and output a FITS image that still has a valid WCS for the rebinned image?  The rebinning should also preserve flux, though I think that?s easier than the WCS part.  Nothing obvious turns up in a little searching, but I could easily have missed it.  
> 
> It seems that Montage might be able to do this (mShrink) but if possible I?d prefer a pure-Python solution, as it would be easier to implement under Windows on our observatory control computer.  (A fallback would be to install a Linux distro on top of the Windows Subsystem for Linux, but if it?s possible to do it purely in Python that would be a lot simpler.) 
> 
> Thanks for any thoughts, 
> 
> Eric
> 
> Eric Jensen
> Professor of Astronomy
> Dept. of Physics and Astronomy
> Swarthmore College
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy


From petr at kubanek.net  Tue Aug  4 06:33:16 2020
From: petr at kubanek.net (=?utf-8?Q?Petr_Kub=C3=A1nek?=)
Date: Tue, 4 Aug 2020 12:33:16 +0200
Subject: [AstroPy] Rebin FITS images and preserve or recalculate WCS?
In-Reply-To: <CB15266E-2189-454A-AAC0-8A2D1DDEAA71@upenn.edu>
References: <37B76AF8-5114-4F2F-B9EB-ED99AB01069E@swarthmore.edu>
 <CB15266E-2189-454A-AAC0-8A2D1DDEAA71@upenn.edu>
Message-ID: <0B17AC35-7B73-4E7B-8D20-3AAFCEB24034@kubanek.net>

And if you happen to have WCS in CD_ matrix, divide it by 2 (scalar, eg. all members).

Petr

> 4. 8. 2020 v 11:44 dop., Bernstein, Gary M <garyb at PHYSICS.UPENN.EDU>:
> 
> Hi Eric -
> I don?t know whether this exists but in this case it might be faster to write than to do the Google search.  The block summing can be done in numpy by reshaping the array.  In the case where the original image shape is divisible by the shrinkage factor s, you?d do
> 
> m,n = img.shape
> img = np.sum( img.reshape(m//s,s,n), axis=1)  # Contract along y
> Img = np.sum( img.reshape(m//s,n//s,s),axis=2). # Contract along x
> 
> And to fix the WCS, you need to divide CRPIXn by s, and multiply CDELTn by s.  There is some slight complication to the first operation because FITS assumes 1-indexed pixels, and the integer marks the center of a pixel, so its really CRPIXn -> (CRPIXn?0.5)/s + 0.5
> 
> I think this should work universally because the linear transform is always the first step of a WCS mapping.
> 
> Cheers,
> Gary
> 
>> On Aug 3, 2020, at 11:01 PM, Eric Jensen <ejensen1 at swarthmore.edu> wrote:
>> 
>> Hi all, 
>> 
>> We are looking at purchasing a new CMOS camera that has 4-micron pixels, which would significantly undersample our PSF. (The camera is otherwise excellent for our needs, e.g. excellent QE.)  So I?m looking at whether there would be a straightforward way in scripted post-processing to reduce the image file sizes while not losing information.  The existing camera driver doesn?t support more than 2x2 binning at this time. 
>> 
>> Is there available Python code that can take a FITS image with a valid WCS in the header and rebin it, and output a FITS image that still has a valid WCS for the rebinned image?  The rebinning should also preserve flux, though I think that?s easier than the WCS part.  Nothing obvious turns up in a little searching, but I could easily have missed it.  
>> 
>> It seems that Montage might be able to do this (mShrink) but if possible I?d prefer a pure-Python solution, as it would be easier to implement under Windows on our observatory control computer.  (A fallback would be to install a Linux distro on top of the Windows Subsystem for Linux, but if it?s possible to do it purely in Python that would be a lot simpler.) 
>> 
>> Thanks for any thoughts, 
>> 
>> Eric
>> 
>> Eric Jensen
>> Professor of Astronomy
>> Dept. of Physics and Astronomy
>> Swarthmore College
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at python.org
>> https://mail.python.org/mailman/listinfo/astropy
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy


From ejensen1 at swarthmore.edu  Tue Aug  4 09:16:29 2020
From: ejensen1 at swarthmore.edu (Eric Jensen)
Date: Tue, 4 Aug 2020 09:16:29 -0400
Subject: [AstroPy] Rebin FITS images and preserve or recalculate WCS?
In-Reply-To: <0B17AC35-7B73-4E7B-8D20-3AAFCEB24034@kubanek.net>
References: <37B76AF8-5114-4F2F-B9EB-ED99AB01069E@swarthmore.edu>
 <CB15266E-2189-454A-AAC0-8A2D1DDEAA71@upenn.edu>
 <0B17AC35-7B73-4E7B-8D20-3AAFCEB24034@kubanek.net>
Message-ID: <4A90B8D1-DCF3-4BCB-8D57-90E20781CC86@swarthmore.edu>

Thanks, Gary and Petr!  I appreciate the answers so far.  A couple of additional notes:

I should have been clear that I realize that the basics of the binning part are hard with numpy operations, though I appreciate the clear code from Gary.   There are possibly edge-of-image issues, but those are probably best handled by just restricting the binning factor to numbers that divide evenly into the array dimensions.  

However, there are potential bit-depth issues - if I add together several pixels with 50,000 counts (previously stored as 16-bit unsigned integers) I?ll need to change the bit depth.  Maybe astropy?s FITS-handling routines make that automatic when writing an array to a new FITS file.  

That said, it?s mostly the subtleties of the WCS part that I?d like to handle with vetted tools if possible.  

For example, if I look at a FITS header from an image solved by astrometry.net, it has not just CRVAL1,2 + CRPIX1,2 + a CD matrix, but it also has SIP distortion coefficients.  It?s possible that those are negligible for our images (I haven?t yet checked them to work out the details of the standard to figure out how much difference they make) and they could just be ignored.  (And our images straight off the telescope, using MaximDL, have some kind of proprietary/undocumented distortion correction scheme that I?ll just need to ignore anyway, I think, so maybe the distortion issue isn?t important.  I?ll have to test that.) 

But it highlights the general issue that WCS has lots of permutations, and it seems better to handle it (if possible) with tested tools such as already exist in astropy.  The question is whether there?s a useful way to deploy the astropy (or other library) WCS functionality here, or whether it is indeed best just to do it manually with the linear coefficients following the solution outlined by Gary and Petr already. 

One specific question: 

> On Aug 4, 2020, at 6:33 AM, Petr Kub?nek <petr at kubanek.net> wrote:
> 
> And if you happen to have WCS in CD_ matrix, divide it by 2 (scalar, eg. all members).

These would need to be *multiplied* by the binning factor, yes? 

If there is no general solution already implemented and tested, I certainly can dig into it more myself - I think the basic outline (minus distortion) is already given, modulo subtleties I haven?t though of.  But in general I?ve encouraged my students to use tested libraries rather than re-inventing the wheel when possible, so I?m trying to take my own advice!  :-) 

Thanks for any additional thoughts,

Eric
 

> On Aug 4, 2020, at 6:33 AM, Petr Kub?nek <petr at kubanek.net> wrote:
> 
> And if you happen to have WCS in CD_ matrix, divide it by 2 (scalar, eg. all members).
> 
> Petr
> 
>> 4. 8. 2020 v 11:44 dop., Bernstein, Gary M <garyb at PHYSICS.UPENN.EDU>:
>> 
>> Hi Eric -
>> I don?t know whether this exists but in this case it might be faster to write than to do the Google search.  The block summing can be done in numpy by reshaping the array.  In the case where the original image shape is divisible by the shrinkage factor s, you?d do
>> 
>> m,n = img.shape
>> img = np.sum( img.reshape(m//s,s,n), axis=1)  # Contract along y
>> Img = np.sum( img.reshape(m//s,n//s,s),axis=2). # Contract along x
>> 
>> And to fix the WCS, you need to divide CRPIXn by s, and multiply CDELTn by s.  There is some slight complication to the first operation because FITS assumes 1-indexed pixels, and the integer marks the center of a pixel, so its really CRPIXn -> (CRPIXn?0.5)/s + 0.5
>> 
>> I think this should work universally because the linear transform is always the first step of a WCS mapping.
>> 
>> Cheers,
>> Gary
>> 
>>> On Aug 3, 2020, at 11:01 PM, Eric Jensen <ejensen1 at swarthmore.edu> wrote:
>>> 
>>> Hi all, 
>>> 
>>> We are looking at purchasing a new CMOS camera that has 4-micron pixels, which would significantly undersample our PSF. (The camera is otherwise excellent for our needs, e.g. excellent QE.)  So I?m looking at whether there would be a straightforward way in scripted post-processing to reduce the image file sizes while not losing information.  The existing camera driver doesn?t support more than 2x2 binning at this time. 
>>> 
>>> Is there available Python code that can take a FITS image with a valid WCS in the header and rebin it, and output a FITS image that still has a valid WCS for the rebinned image?  The rebinning should also preserve flux, though I think that?s easier than the WCS part.  Nothing obvious turns up in a little searching, but I could easily have missed it.  
>>> 
>>> It seems that Montage might be able to do this (mShrink) but if possible I?d prefer a pure-Python solution, as it would be easier to implement under Windows on our observatory control computer.  (A fallback would be to install a Linux distro on top of the Windows Subsystem for Linux, but if it?s possible to do it purely in Python that would be a lot simpler.) 
>>> 
>>> Thanks for any thoughts, 
>>> 
>>> Eric
>>> 
>>> Eric Jensen
>>> Professor of Astronomy
>>> Dept. of Physics and Astronomy
>>> Swarthmore College
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at python.org
>>> https://mail.python.org/mailman/listinfo/astropy
>> 
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at python.org
>> https://mail.python.org/mailman/listinfo/astropy
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3946 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/astropy/attachments/20200804/ba4e8cae/attachment.bin>

From ejensen1 at swarthmore.edu  Tue Aug  4 09:18:25 2020
From: ejensen1 at swarthmore.edu (Eric Jensen)
Date: Tue, 4 Aug 2020 09:18:25 -0400
Subject: [AstroPy] Rebin FITS images and preserve or recalculate WCS?
In-Reply-To: <4A90B8D1-DCF3-4BCB-8D57-90E20781CC86@swarthmore.edu>
References: <37B76AF8-5114-4F2F-B9EB-ED99AB01069E@swarthmore.edu>
 <CB15266E-2189-454A-AAC0-8A2D1DDEAA71@upenn.edu>
 <0B17AC35-7B73-4E7B-8D20-3AAFCEB24034@kubanek.net>
 <4A90B8D1-DCF3-4BCB-8D57-90E20781CC86@swarthmore.edu>
Message-ID: <0799D0A8-35B3-4BF3-9F28-61CF2DC10E76@swarthmore.edu>


> On Aug 4, 2020, at 9:16 AM, Eric Jensen <ejensen1 at swarthmore.edu> wrote:
> 
>  I realize that the basics of the binning part are hard with numpy operations, 

Sorry, this should have said ?are *not* hard?? 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20200804/29285de5/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3946 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/astropy/attachments/20200804/29285de5/attachment-0001.bin>

From garyb at PHYSICS.UPENN.EDU  Tue Aug  4 09:37:34 2020
From: garyb at PHYSICS.UPENN.EDU (Bernstein, Gary M)
Date: Tue, 4 Aug 2020 13:37:34 +0000
Subject: [AstroPy] Rebin FITS images and preserve or recalculate WCS?
In-Reply-To: <4A90B8D1-DCF3-4BCB-8D57-90E20781CC86@swarthmore.edu>
References: <37B76AF8-5114-4F2F-B9EB-ED99AB01069E@swarthmore.edu>
 <CB15266E-2189-454A-AAC0-8A2D1DDEAA71@upenn.edu>
 <0B17AC35-7B73-4E7B-8D20-3AAFCEB24034@kubanek.net>
 <4A90B8D1-DCF3-4BCB-8D57-90E20781CC86@swarthmore.edu>
Message-ID: <E3B7258B-E294-4387-B0F6-B6FA30EC7892@upenn.edu>

Hi Eric - I actually just faced the rebinning issue with a summer student and had produced this function, which is very general about the dimensions of the input array.  It also handles the not-an-even-multiple case by trimming the input image to be a multiple of the binning factor.  The overflow issue is a good point; if your inputs are ushort then you might need to convert the type, which could be done at the ?copy()? line below.

And I see that the SIP polynomials are applied to the pixel coordinates rather than the intermediate world coordinates, which I didn?t know about, so indeed that complicates the WCS a bit.  Looking at https://irsa.ipac.caltech.edu/data/SPITZER/docs/files/spitzer/shupeADASS.pdf
it seems like the polynomial arguments are still relative to CRPIXn, so the CRPIX transformation in the earlier message would still work and the CDp_q scaling still needed.  But you?d have to alter all the [AB]_p_q keywords by a factor s**(p+q-1) and the inverse [AB]P_p_q by s**(1-p-q).  So it?s still not too messy, but you do have to go hunting through the header for things now, & I understand your desired to find a previously vetted solution!

Gary


def blockavg(array,factor):
    '''Average the array in blocks of `factor` pixels on each axis.
    `factor` can be an integer, in which case it is applied to
    all axes, or it can be an array, in which case each axis
    is reduced by corresponding element of factor until either
    axes or factor array are exhausted.
    
    If array dimension is not divisible by factor, it will
    be clipped to make it so.
    
    Returns the reduced array.
    '''
    
    if type(factor)==int:
        # If factor is an int, apply to all dims
        ff = [factor]*array.ndim
    else:
        # Just copy each element of an iterable
        ff = [i for i in factor]
        
    naxes = min(array.ndim,len(ff))  # Number of axes to avg

    # First I'll truncate all axes to be divisible by factors:
    s = [ slice((array.shape[i]//ff[i])*ff[i]) for i in range(naxes)]
    tmp = array[tuple(s)].copy()

    # Now reduce each axis in turn
    for i in range(naxes):
        s = tmp.shape[:i] + (tmp.shape[i]//ff[i],ff[i]) + tmp.shape[i+1:]
        tmp = np.sum( tmp.reshape(s),axis=i+1)
    
    # Divide to turn into average - delete this line if you want to sum...
    tmp = tmp / np.prod(ff)
    return tmp


> On Aug 4, 2020, at 9:16 AM, Eric Jensen <ejensen1 at swarthmore.edu> wrote:
> 
> Thanks, Gary and Petr!  I appreciate the answers so far.  A couple of additional notes:
> 
> I should have been clear that I realize that the basics of the binning part are hard with numpy operations, though I appreciate the clear code from Gary.   There are possibly edge-of-image issues, but those are probably best handled by just restricting the binning factor to numbers that divide evenly into the array dimensions.  
> 
> However, there are potential bit-depth issues - if I add together several pixels with 50,000 counts (previously stored as 16-bit unsigned integers) I?ll need to change the bit depth.  Maybe astropy?s FITS-handling routines make that automatic when writing an array to a new FITS file.  
> 
> That said, it?s mostly the subtleties of the WCS part that I?d like to handle with vetted tools if possible.  
> 
> For example, if I look at a FITS header from an image solved by astrometry.net, it has not just CRVAL1,2 + CRPIX1,2 + a CD matrix, but it also has SIP distortion coefficients.  It?s possible that those are negligible for our images (I haven?t yet checked them to work out the details of the standard to figure out how much difference they make) and they could just be ignored.  (And our images straight off the telescope, using MaximDL, have some kind of proprietary/undocumented distortion correction scheme that I?ll just need to ignore anyway, I think, so maybe the distortion issue isn?t important.  I?ll have to test that.) 
> 
> But it highlights the general issue that WCS has lots of permutations, and it seems better to handle it (if possible) with tested tools such as already exist in astropy.  The question is whether there?s a useful way to deploy the astropy (or other library) WCS functionality here, or whether it is indeed best just to do it manually with the linear coefficients following the solution outlined by Gary and Petr already. 
> 
> One specific question: 
> 
>> On Aug 4, 2020, at 6:33 AM, Petr Kub?nek <petr at kubanek.net> wrote:
>> 
>> And if you happen to have WCS in CD_ matrix, divide it by 2 (scalar, eg. all members).
> 
> These would need to be *multiplied* by the binning factor, yes? 
> 
> If there is no general solution already implemented and tested, I certainly can dig into it more myself - I think the basic outline (minus distortion) is already given, modulo subtleties I haven?t though of.  But in general I?ve encouraged my students to use tested libraries rather than re-inventing the wheel when possible, so I?m trying to take my own advice!  :-) 
> 
> Thanks for any additional thoughts,
> 
> Eric
> 
> 
>> On Aug 4, 2020, at 6:33 AM, Petr Kub?nek <petr at kubanek.net> wrote:
>> 
>> And if you happen to have WCS in CD_ matrix, divide it by 2 (scalar, eg. all members).
>> 
>> Petr
>> 
>>> 4. 8. 2020 v 11:44 dop., Bernstein, Gary M <garyb at PHYSICS.UPENN.EDU>:
>>> 
>>> Hi Eric -
>>> I don?t know whether this exists but in this case it might be faster to write than to do the Google search.  The block summing can be done in numpy by reshaping the array.  In the case where the original image shape is divisible by the shrinkage factor s, you?d do
>>> 
>>> m,n = img.shape
>>> img = np.sum( img.reshape(m//s,s,n), axis=1)  # Contract along y
>>> Img = np.sum( img.reshape(m//s,n//s,s),axis=2). # Contract along x
>>> 
>>> And to fix the WCS, you need to divide CRPIXn by s, and multiply CDELTn by s.  There is some slight complication to the first operation because FITS assumes 1-indexed pixels, and the integer marks the center of a pixel, so its really CRPIXn -> (CRPIXn?0.5)/s + 0.5
>>> 
>>> I think this should work universally because the linear transform is always the first step of a WCS mapping.
>>> 
>>> Cheers,
>>> Gary
>>> 
>>>> On Aug 3, 2020, at 11:01 PM, Eric Jensen <ejensen1 at swarthmore.edu> wrote:
>>>> 
>>>> Hi all, 
>>>> 
>>>> We are looking at purchasing a new CMOS camera that has 4-micron pixels, which would significantly undersample our PSF. (The camera is otherwise excellent for our needs, e.g. excellent QE.)  So I?m looking at whether there would be a straightforward way in scripted post-processing to reduce the image file sizes while not losing information.  The existing camera driver doesn?t support more than 2x2 binning at this time. 
>>>> 
>>>> Is there available Python code that can take a FITS image with a valid WCS in the header and rebin it, and output a FITS image that still has a valid WCS for the rebinned image?  The rebinning should also preserve flux, though I think that?s easier than the WCS part.  Nothing obvious turns up in a little searching, but I could easily have missed it.  
>>>> 
>>>> It seems that Montage might be able to do this (mShrink) but if possible I?d prefer a pure-Python solution, as it would be easier to implement under Windows on our observatory control computer.  (A fallback would be to install a Linux distro on top of the Windows Subsystem for Linux, but if it?s possible to do it purely in Python that would be a lot simpler.) 
>>>> 
>>>> Thanks for any thoughts, 
>>>> 
>>>> Eric
>>>> 
>>>> Eric Jensen
>>>> Professor of Astronomy
>>>> Dept. of Physics and Astronomy
>>>> Swarthmore College
>>>> _______________________________________________
>>>> AstroPy mailing list
>>>> AstroPy at python.org
>>>> https://mail.python.org/mailman/listinfo/astropy
>>> 
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at python.org
>>> https://mail.python.org/mailman/listinfo/astropy
>> 
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at python.org
>> https://mail.python.org/mailman/listinfo/astropy
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy


From simon at sconseil.fr  Tue Aug  4 10:04:12 2020
From: simon at sconseil.fr (Simon Conseil)
Date: Tue, 4 Aug 2020 10:04:12 -0400
Subject: [AstroPy] Rebin FITS images and preserve or recalculate WCS?
In-Reply-To: <E3B7258B-E294-4387-B0F6-B6FA30EC7892@upenn.edu>
References: <37B76AF8-5114-4F2F-B9EB-ED99AB01069E@swarthmore.edu>
 <CB15266E-2189-454A-AAC0-8A2D1DDEAA71@upenn.edu>
 <0B17AC35-7B73-4E7B-8D20-3AAFCEB24034@kubanek.net>
 <4A90B8D1-DCF3-4BCB-8D57-90E20781CC86@swarthmore.edu>
 <E3B7258B-E294-4387-B0F6-B6FA30EC7892@upenn.edu>
Message-ID: <813bf3da-95d1-2465-b586-a3f80e99588d@sconseil.fr>

Hi,

There is a rebin method in MPDAF, with an option for the margins, and 
using the same reshape trick as Gary: Image.rebin, which calls the 
generic ND method on the parent class, and WCS.rebin (class based on 
Astropy one):
https://github.com/musevlt/mpdaf/blob/master/lib/mpdaf/obj/image.py#L2380
https://github.com/musevlt/mpdaf/blob/master/lib/mpdaf/obj/data.py#L1377
https://github.com/musevlt/mpdaf/blob/master/lib/mpdaf/obj/coords.py#L1579

Simon


On 8/4/20 9:37 AM, Bernstein, Gary M wrote:
> Hi Eric - I actually just faced the rebinning issue with a summer student and had produced this function, which is very general about the dimensions of the input array.  It also handles the not-an-even-multiple case by trimming the input image to be a multiple of the binning factor.  The overflow issue is a good point; if your inputs are ushort then you might need to convert the type, which could be done at the ?copy()? line below.
> 
> And I see that the SIP polynomials are applied to the pixel coordinates rather than the intermediate world coordinates, which I didn?t know about, so indeed that complicates the WCS a bit.  Looking at https://irsa.ipac.caltech.edu/data/SPITZER/docs/files/spitzer/shupeADASS.pdf
> it seems like the polynomial arguments are still relative to CRPIXn, so the CRPIX transformation in the earlier message would still work and the CDp_q scaling still needed.  But you?d have to alter all the [AB]_p_q keywords by a factor s**(p+q-1) and the inverse [AB]P_p_q by s**(1-p-q).  So it?s still not too messy, but you do have to go hunting through the header for things now, & I understand your desired to find a previously vetted solution!
> 
> Gary
> 
> 
> def blockavg(array,factor):
>      '''Average the array in blocks of `factor` pixels on each axis.
>      `factor` can be an integer, in which case it is applied to
>      all axes, or it can be an array, in which case each axis
>      is reduced by corresponding element of factor until either
>      axes or factor array are exhausted.
>      
>      If array dimension is not divisible by factor, it will
>      be clipped to make it so.
>      
>      Returns the reduced array.
>      '''
>      
>      if type(factor)==int:
>          # If factor is an int, apply to all dims
>          ff = [factor]*array.ndim
>      else:
>          # Just copy each element of an iterable
>          ff = [i for i in factor]
>          
>      naxes = min(array.ndim,len(ff))  # Number of axes to avg
> 
>      # First I'll truncate all axes to be divisible by factors:
>      s = [ slice((array.shape[i]//ff[i])*ff[i]) for i in range(naxes)]
>      tmp = array[tuple(s)].copy()
> 
>      # Now reduce each axis in turn
>      for i in range(naxes):
>          s = tmp.shape[:i] + (tmp.shape[i]//ff[i],ff[i]) + tmp.shape[i+1:]
>          tmp = np.sum( tmp.reshape(s),axis=i+1)
>      
>      # Divide to turn into average - delete this line if you want to sum...
>      tmp = tmp / np.prod(ff)
>      return tmp
> 
> 
>> On Aug 4, 2020, at 9:16 AM, Eric Jensen <ejensen1 at swarthmore.edu> wrote:
>>
>> Thanks, Gary and Petr!  I appreciate the answers so far.  A couple of additional notes:
>>
>> I should have been clear that I realize that the basics of the binning part are hard with numpy operations, though I appreciate the clear code from Gary.   There are possibly edge-of-image issues, but those are probably best handled by just restricting the binning factor to numbers that divide evenly into the array dimensions.
>>
>> However, there are potential bit-depth issues - if I add together several pixels with 50,000 counts (previously stored as 16-bit unsigned integers) I?ll need to change the bit depth.  Maybe astropy?s FITS-handling routines make that automatic when writing an array to a new FITS file.
>>
>> That said, it?s mostly the subtleties of the WCS part that I?d like to handle with vetted tools if possible.
>>
>> For example, if I look at a FITS header from an image solved by astrometry.net, it has not just CRVAL1,2 + CRPIX1,2 + a CD matrix, but it also has SIP distortion coefficients.  It?s possible that those are negligible for our images (I haven?t yet checked them to work out the details of the standard to figure out how much difference they make) and they could just be ignored.  (And our images straight off the telescope, using MaximDL, have some kind of proprietary/undocumented distortion correction scheme that I?ll just need to ignore anyway, I think, so maybe the distortion issue isn?t important.  I?ll have to test that.)
>>
>> But it highlights the general issue that WCS has lots of permutations, and it seems better to handle it (if possible) with tested tools such as already exist in astropy.  The question is whether there?s a useful way to deploy the astropy (or other library) WCS functionality here, or whether it is indeed best just to do it manually with the linear coefficients following the solution outlined by Gary and Petr already.
>>
>> One specific question:
>>
>>> On Aug 4, 2020, at 6:33 AM, Petr Kub?nek <petr at kubanek.net> wrote:
>>>
>>> And if you happen to have WCS in CD_ matrix, divide it by 2 (scalar, eg. all members).
>>
>> These would need to be *multiplied* by the binning factor, yes?
>>
>> If there is no general solution already implemented and tested, I certainly can dig into it more myself - I think the basic outline (minus distortion) is already given, modulo subtleties I haven?t though of.  But in general I?ve encouraged my students to use tested libraries rather than re-inventing the wheel when possible, so I?m trying to take my own advice!  :-)
>>
>> Thanks for any additional thoughts,
>>
>> Eric
>>
>>
>>> On Aug 4, 2020, at 6:33 AM, Petr Kub?nek <petr at kubanek.net> wrote:
>>>
>>> And if you happen to have WCS in CD_ matrix, divide it by 2 (scalar, eg. all members).
>>>
>>> Petr
>>>
>>>> 4. 8. 2020 v 11:44 dop., Bernstein, Gary M <garyb at PHYSICS.UPENN.EDU>:
>>>>
>>>> Hi Eric -
>>>> I don?t know whether this exists but in this case it might be faster to write than to do the Google search.  The block summing can be done in numpy by reshaping the array.  In the case where the original image shape is divisible by the shrinkage factor s, you?d do
>>>>
>>>> m,n = img.shape
>>>> img = np.sum( img.reshape(m//s,s,n), axis=1)  # Contract along y
>>>> Img = np.sum( img.reshape(m//s,n//s,s),axis=2). # Contract along x
>>>>
>>>> And to fix the WCS, you need to divide CRPIXn by s, and multiply CDELTn by s.  There is some slight complication to the first operation because FITS assumes 1-indexed pixels, and the integer marks the center of a pixel, so its really CRPIXn -> (CRPIXn?0.5)/s + 0.5
>>>>
>>>> I think this should work universally because the linear transform is always the first step of a WCS mapping.
>>>>
>>>> Cheers,
>>>> Gary
>>>>
>>>>> On Aug 3, 2020, at 11:01 PM, Eric Jensen <ejensen1 at swarthmore.edu> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> We are looking at purchasing a new CMOS camera that has 4-micron pixels, which would significantly undersample our PSF. (The camera is otherwise excellent for our needs, e.g. excellent QE.)  So I?m looking at whether there would be a straightforward way in scripted post-processing to reduce the image file sizes while not losing information.  The existing camera driver doesn?t support more than 2x2 binning at this time.
>>>>>
>>>>> Is there available Python code that can take a FITS image with a valid WCS in the header and rebin it, and output a FITS image that still has a valid WCS for the rebinned image?  The rebinning should also preserve flux, though I think that?s easier than the WCS part.  Nothing obvious turns up in a little searching, but I could easily have missed it.
>>>>>
>>>>> It seems that Montage might be able to do this (mShrink) but if possible I?d prefer a pure-Python solution, as it would be easier to implement under Windows on our observatory control computer.  (A fallback would be to install a Linux distro on top of the Windows Subsystem for Linux, but if it?s possible to do it purely in Python that would be a lot simpler.)
>>>>>
>>>>> Thanks for any thoughts,
>>>>>
>>>>> Eric
>>>>>
>>>>> Eric Jensen
>>>>> Professor of Astronomy
>>>>> Dept. of Physics and Astronomy
>>>>> Swarthmore College
>>>>> _______________________________________________
>>>>> AstroPy mailing list
>>>>> AstroPy at python.org
>>>>> https://mail.python.org/mailman/listinfo/astropy
>>>>
>>>> _______________________________________________
>>>> AstroPy mailing list
>>>> AstroPy at python.org
>>>> https://mail.python.org/mailman/listinfo/astropy
>>>
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at python.org
>>> https://mail.python.org/mailman/listinfo/astropy
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at python.org
>> https://mail.python.org/mailman/listinfo/astropy
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy
> 

From jslavin at cfa.harvard.edu  Tue Aug  4 11:42:59 2020
From: jslavin at cfa.harvard.edu (Slavin, Jonathan)
Date: Tue, 4 Aug 2020 11:42:59 -0400
Subject: [AstroPy] Rebin FITS images and preserve or recalculate WCS?
In-Reply-To: <mailman.5428.1596547110.9565.astropy@python.org>
References: <mailman.5428.1596547110.9565.astropy@python.org>
Message-ID: <CACcRS=eP1_dJKQU6ZcR+YT6ZeAosJk8icMRoaR2-1HEsBxP9LQ@mail.gmail.com>

Hi Eric,

I posted a very similar question to the Python users in Astronomy Facebook
group a few weeks ago (July 7). You might look up that discussion. There
were some interesting suggestions, though I wouldn't say any definitive
answers.

Regards,
Jon

On Tue, Aug 4, 2020 at 9:18 AM <astropy-request at python.org> wrote:

> Today's Topics:
>
>    1. Re: Rebin FITS images and preserve or recalculate WCS?
>       (Eric Jensen)
>    2. Re: Rebin FITS images and preserve or recalculate WCS?
>       (Eric Jensen)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 4 Aug 2020 09:16:29 -0400
> From: Eric Jensen <ejensen1 at swarthmore.edu>
> To: Astronomical Python mailing list <astropy at python.org>
> Subject: Re: [AstroPy] Rebin FITS images and preserve or recalculate
>         WCS?
> Message-ID: <4A90B8D1-DCF3-4BCB-8D57-90E20781CC86 at swarthmore.edu>
> Content-Type: text/plain; charset="utf-8"
>
> Thanks, Gary and Petr!  I appreciate the answers so far.  A couple of
> additional notes:
>
> I should have been clear that I realize that the basics of the binning
> part are hard with numpy operations, though I appreciate the clear code
> from Gary.   There are possibly edge-of-image issues, but those are
> probably best handled by just restricting the binning factor to numbers
> that divide evenly into the array dimensions.
>
> However, there are potential bit-depth issues - if I add together several
> pixels with 50,000 counts (previously stored as 16-bit unsigned integers)
> I?ll need to change the bit depth.  Maybe astropy?s FITS-handling routines
> make that automatic when writing an array to a new FITS file.
>
> That said, it?s mostly the subtleties of the WCS part that I?d like to
> handle with vetted tools if possible.
>
> For example, if I look at a FITS header from an image solved by
> astrometry.net, it has not just CRVAL1,2 + CRPIX1,2 + a CD matrix, but it
> also has SIP distortion coefficients.  It?s possible that those are
> negligible for our images (I haven?t yet checked them to work out the
> details of the standard to figure out how much difference they make) and
> they could just be ignored.  (And our images straight off the telescope,
> using MaximDL, have some kind of proprietary/undocumented distortion
> correction scheme that I?ll just need to ignore anyway, I think, so maybe
> the distortion issue isn?t important.  I?ll have to test that.)
>
> But it highlights the general issue that WCS has lots of permutations, and
> it seems better to handle it (if possible) with tested tools such as
> already exist in astropy.  The question is whether there?s a useful way to
> deploy the astropy (or other library) WCS functionality here, or whether it
> is indeed best just to do it manually with the linear coefficients
> following the solution outlined by Gary and Petr already.
>
> One specific question:
>
> > On Aug 4, 2020, at 6:33 AM, Petr Kub?nek <petr at kubanek.net> wrote:
> >
> > And if you happen to have WCS in CD_ matrix, divide it by 2 (scalar, eg.
> all members).
>
> These would need to be *multiplied* by the binning factor, yes?
>
> If there is no general solution already implemented and tested, I
> certainly can dig into it more myself - I think the basic outline (minus
> distortion) is already given, modulo subtleties I haven?t though of.  But
> in general I?ve encouraged my students to use tested libraries rather than
> re-inventing the wheel when possible, so I?m trying to take my own advice!
> :-)
>
> Thanks for any additional thoughts,
>
> Eric
>
>
> > On Aug 4, 2020, at 6:33 AM, Petr Kub?nek <petr at kubanek.net> wrote:
> >
> > And if you happen to have WCS in CD_ matrix, divide it by 2 (scalar, eg.
> all members).
> >
> > Petr
> >
> >> 4. 8. 2020 v 11:44 dop., Bernstein, Gary M <garyb at PHYSICS.UPENN.EDU>:
> >>
> >> Hi Eric -
> >> I don?t know whether this exists but in this case it might be faster to
> write than to do the Google search.  The block summing can be done in numpy
> by reshaping the array.  In the case where the original image shape is
> divisible by the shrinkage factor s, you?d do
> >>
> >> m,n = img.shape
> >> img = np.sum( img.reshape(m//s,s,n), axis=1)  # Contract along y
> >> Img = np.sum( img.reshape(m//s,n//s,s),axis=2). # Contract along x
> >>
> >> And to fix the WCS, you need to divide CRPIXn by s, and multiply CDELTn
> by s.  There is some slight complication to the first operation because
> FITS assumes 1-indexed pixels, and the integer marks the center of a pixel,
> so its really CRPIXn -> (CRPIXn?0.5)/s + 0.5
> >>
> >> I think this should work universally because the linear transform is
> always the first step of a WCS mapping.
> >>
> >> Cheers,
> >> Gary
> >>
> >>> On Aug 3, 2020, at 11:01 PM, Eric Jensen <ejensen1 at swarthmore.edu>
> wrote:
> >>>
> >>> Hi all,
> >>>
> >>> We are looking at purchasing a new CMOS camera that has 4-micron
> pixels, which would significantly undersample our PSF. (The camera is
> otherwise excellent for our needs, e.g. excellent QE.)  So I?m looking at
> whether there would be a straightforward way in scripted post-processing to
> reduce the image file sizes while not losing information.  The existing
> camera driver doesn?t support more than 2x2 binning at this time.
> >>>
> >>> Is there available Python code that can take a FITS image with a valid
> WCS in the header and rebin it, and output a FITS image that still has a
> valid WCS for the rebinned image?  The rebinning should also preserve flux,
> though I think that?s easier than the WCS part.  Nothing obvious turns up
> in a little searching, but I could easily have missed it.
> >>>
> >>> It seems that Montage might be able to do this (mShrink) but if
> possible I?d prefer a pure-Python solution, as it would be easier to
> implement under Windows on our observatory control computer.  (A fallback
> would be to install a Linux distro on top of the Windows Subsystem for
> Linux, but if it?s possible to do it purely in Python that would be a lot
> simpler.)
> >>>
> >>> Thanks for any thoughts,
> >>>
> >>> Eric
> >>>
> >>> Eric Jensen
> >>> Professor of Astronomy
> >>> Dept. of Physics and Astronomy
> >>> Swarthmore College
> >>> _______________________________________________
> >>> AstroPy mailing list
> >>> AstroPy at python.org
> >>> https://mail.python.org/mailman/listinfo/astropy
> >>
> >> _______________________________________________
> >> AstroPy mailing list
> >> AstroPy at python.org
> >> https://mail.python.org/mailman/listinfo/astropy
> >
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at python.org
> > https://mail.python.org/mailman/listinfo/astropy
>
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: smime.p7s
> Type: application/pkcs7-signature
> Size: 3946 bytes
> Desc: not available
> URL: <
> http://mail.python.org/pipermail/astropy/attachments/20200804/ba4e8cae/attachment-0001.bin
> >
>
> ------------------------------
>
> Message: 2
> Date: Tue, 4 Aug 2020 09:18:25 -0400
> From: Eric Jensen <ejensen1 at swarthmore.edu>
> To: Astronomical Python mailing list <astropy at python.org>
> Subject: Re: [AstroPy] Rebin FITS images and preserve or recalculate
>         WCS?
> Message-ID: <0799D0A8-35B3-4BF3-9F28-61CF2DC10E76 at swarthmore.edu>
> Content-Type: text/plain; charset="utf-8"
>
>
> > On Aug 4, 2020, at 9:16 AM, Eric Jensen <ejensen1 at swarthmore.edu> wrote:
> >
> >  I realize that the basics of the binning part are hard with numpy
> operations,
>
> Sorry, this should have said ?are *not* hard??
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mail.python.org/pipermail/astropy/attachments/20200804/29285de5/attachment.html
> >
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: smime.p7s
> Type: application/pkcs7-signature
> Size: 3946 bytes
> Desc: not available
> URL: <
> http://mail.python.org/pipermail/astropy/attachments/20200804/29285de5/attachment.bin
> >
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy
>
>
> ------------------------------
>
> End of AstroPy Digest, Vol 167, Issue 2
> ***************************************
>


-- 
Jonathan D. Slavin
Astrophysicist - High Energy Astrophysics Division
Center for Astrophysics | Harvard & Smithsonian
Office: (617) 496-7981 | Cell: (781) 363-0035
60 Garden Street | MS 83 | Cambridge, MA 02138
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20200804/cff099bb/attachment.html>

From kellecruz at gmail.com  Tue Aug  4 14:19:09 2020
From: kellecruz at gmail.com (Kelle Cruz)
Date: Tue, 4 Aug 2020 14:19:09 -0400
Subject: [AstroPy] Rebin FITS images and preserve or recalculate WCS?
In-Reply-To: <CACcRS=eP1_dJKQU6ZcR+YT6ZeAosJk8icMRoaR2-1HEsBxP9LQ@mail.gmail.com>
References: <mailman.5428.1596547110.9565.astropy@python.org>
 <CACcRS=eP1_dJKQU6ZcR+YT6ZeAosJk8icMRoaR2-1HEsBxP9LQ@mail.gmail.com>
Message-ID: <CAApiRV6MuKKc+8HqNnSg6r3dm=bF4=DnHUt_XrtAJ2r3D1BZQg@mail.gmail.com>

Link to FB discussion:
https://www.facebook.com/groups/astropython/permalink/2702447229999950/

and link to a printout of the discussion:
https://www.dropbox.com/s/zyy67kfemo03w66/%282%29%20Python%20users%20in%20Astronomy.pdf?dl=0

Kelle

--
Kelle Cruz, PhD (she/her, they/them)
Assoc. Professor, Physics and Astronomy, Hunter College
CEO, ScienceBetter Consulting, LLC
CFO, Startorialist, Inc. <http://shop.startorialist.com>
Editor-in-Chief, AstroBetter Blog and Wiki <http://astrobetter.com>
Director, McNulty Scholars Program, Hunter College
Research Associate, American Museum of Natural History
Visiting Scholar, Center for Computational Astrophysics


On Tue, Aug 4, 2020 at 11:43 AM Slavin, Jonathan <jslavin at cfa.harvard.edu>
wrote:

> Hi Eric,
>
> I posted a very similar question to the Python users in Astronomy Facebook
> group a few weeks ago (July 7). You might look up that discussion. There
> were some interesting suggestions, though I wouldn't say any definitive
> answers.
>
> Regards,
> Jon
>
> On Tue, Aug 4, 2020 at 9:18 AM <astropy-request at python.org> wrote:
>
>> Today's Topics:
>>
>>    1. Re: Rebin FITS images and preserve or recalculate WCS?
>>       (Eric Jensen)
>>    2. Re: Rebin FITS images and preserve or recalculate WCS?
>>       (Eric Jensen)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Tue, 4 Aug 2020 09:16:29 -0400
>> From: Eric Jensen <ejensen1 at swarthmore.edu>
>> To: Astronomical Python mailing list <astropy at python.org>
>> Subject: Re: [AstroPy] Rebin FITS images and preserve or recalculate
>>         WCS?
>> Message-ID: <4A90B8D1-DCF3-4BCB-8D57-90E20781CC86 at swarthmore.edu>
>> Content-Type: text/plain; charset="utf-8"
>>
>> Thanks, Gary and Petr!  I appreciate the answers so far.  A couple of
>> additional notes:
>>
>> I should have been clear that I realize that the basics of the binning
>> part are hard with numpy operations, though I appreciate the clear code
>> from Gary.   There are possibly edge-of-image issues, but those are
>> probably best handled by just restricting the binning factor to numbers
>> that divide evenly into the array dimensions.
>>
>> However, there are potential bit-depth issues - if I add together several
>> pixels with 50,000 counts (previously stored as 16-bit unsigned integers)
>> I?ll need to change the bit depth.  Maybe astropy?s FITS-handling routines
>> make that automatic when writing an array to a new FITS file.
>>
>> That said, it?s mostly the subtleties of the WCS part that I?d like to
>> handle with vetted tools if possible.
>>
>> For example, if I look at a FITS header from an image solved by
>> astrometry.net, it has not just CRVAL1,2 + CRPIX1,2 + a CD matrix, but
>> it also has SIP distortion coefficients.  It?s possible that those are
>> negligible for our images (I haven?t yet checked them to work out the
>> details of the standard to figure out how much difference they make) and
>> they could just be ignored.  (And our images straight off the telescope,
>> using MaximDL, have some kind of proprietary/undocumented distortion
>> correction scheme that I?ll just need to ignore anyway, I think, so maybe
>> the distortion issue isn?t important.  I?ll have to test that.)
>>
>> But it highlights the general issue that WCS has lots of permutations,
>> and it seems better to handle it (if possible) with tested tools such as
>> already exist in astropy.  The question is whether there?s a useful way to
>> deploy the astropy (or other library) WCS functionality here, or whether it
>> is indeed best just to do it manually with the linear coefficients
>> following the solution outlined by Gary and Petr already.
>>
>> One specific question:
>>
>> > On Aug 4, 2020, at 6:33 AM, Petr Kub?nek <petr at kubanek.net> wrote:
>> >
>> > And if you happen to have WCS in CD_ matrix, divide it by 2 (scalar,
>> eg. all members).
>>
>> These would need to be *multiplied* by the binning factor, yes?
>>
>> If there is no general solution already implemented and tested, I
>> certainly can dig into it more myself - I think the basic outline (minus
>> distortion) is already given, modulo subtleties I haven?t though of.  But
>> in general I?ve encouraged my students to use tested libraries rather than
>> re-inventing the wheel when possible, so I?m trying to take my own advice!
>> :-)
>>
>> Thanks for any additional thoughts,
>>
>> Eric
>>
>>
>> > On Aug 4, 2020, at 6:33 AM, Petr Kub?nek <petr at kubanek.net> wrote:
>> >
>> > And if you happen to have WCS in CD_ matrix, divide it by 2 (scalar,
>> eg. all members).
>> >
>> > Petr
>> >
>> >> 4. 8. 2020 v 11:44 dop., Bernstein, Gary M <garyb at PHYSICS.UPENN.EDU>:
>> >>
>> >> Hi Eric -
>> >> I don?t know whether this exists but in this case it might be faster
>> to write than to do the Google search.  The block summing can be done in
>> numpy by reshaping the array.  In the case where the original image shape
>> is divisible by the shrinkage factor s, you?d do
>> >>
>> >> m,n = img.shape
>> >> img = np.sum( img.reshape(m//s,s,n), axis=1)  # Contract along y
>> >> Img = np.sum( img.reshape(m//s,n//s,s),axis=2). # Contract along x
>> >>
>> >> And to fix the WCS, you need to divide CRPIXn by s, and multiply
>> CDELTn by s.  There is some slight complication to the first operation
>> because FITS assumes 1-indexed pixels, and the integer marks the center of
>> a pixel, so its really CRPIXn -> (CRPIXn?0.5)/s + 0.5
>> >>
>> >> I think this should work universally because the linear transform is
>> always the first step of a WCS mapping.
>> >>
>> >> Cheers,
>> >> Gary
>> >>
>> >>> On Aug 3, 2020, at 11:01 PM, Eric Jensen <ejensen1 at swarthmore.edu>
>> wrote:
>> >>>
>> >>> Hi all,
>> >>>
>> >>> We are looking at purchasing a new CMOS camera that has 4-micron
>> pixels, which would significantly undersample our PSF. (The camera is
>> otherwise excellent for our needs, e.g. excellent QE.)  So I?m looking at
>> whether there would be a straightforward way in scripted post-processing to
>> reduce the image file sizes while not losing information.  The existing
>> camera driver doesn?t support more than 2x2 binning at this time.
>> >>>
>> >>> Is there available Python code that can take a FITS image with a
>> valid WCS in the header and rebin it, and output a FITS image that still
>> has a valid WCS for the rebinned image?  The rebinning should also preserve
>> flux, though I think that?s easier than the WCS part.  Nothing obvious
>> turns up in a little searching, but I could easily have missed it.
>> >>>
>> >>> It seems that Montage might be able to do this (mShrink) but if
>> possible I?d prefer a pure-Python solution, as it would be easier to
>> implement under Windows on our observatory control computer.  (A fallback
>> would be to install a Linux distro on top of the Windows Subsystem for
>> Linux, but if it?s possible to do it purely in Python that would be a lot
>> simpler.)
>> >>>
>> >>> Thanks for any thoughts,
>> >>>
>> >>> Eric
>> >>>
>> >>> Eric Jensen
>> >>> Professor of Astronomy
>> >>> Dept. of Physics and Astronomy
>> >>> Swarthmore College
>> >>> _______________________________________________
>> >>> AstroPy mailing list
>> >>> AstroPy at python.org
>> >>> https://mail.python.org/mailman/listinfo/astropy
>> >>
>> >> _______________________________________________
>> >> AstroPy mailing list
>> >> AstroPy at python.org
>> >> https://mail.python.org/mailman/listinfo/astropy
>> >
>> > _______________________________________________
>> > AstroPy mailing list
>> > AstroPy at python.org
>> > https://mail.python.org/mailman/listinfo/astropy
>>
>> -------------- next part --------------
>> A non-text attachment was scrubbed...
>> Name: smime.p7s
>> Type: application/pkcs7-signature
>> Size: 3946 bytes
>> Desc: not available
>> URL: <
>> http://mail.python.org/pipermail/astropy/attachments/20200804/ba4e8cae/attachment-0001.bin
>> >
>>
>> ------------------------------
>>
>> Message: 2
>> Date: Tue, 4 Aug 2020 09:18:25 -0400
>> From: Eric Jensen <ejensen1 at swarthmore.edu>
>> To: Astronomical Python mailing list <astropy at python.org>
>> Subject: Re: [AstroPy] Rebin FITS images and preserve or recalculate
>>         WCS?
>> Message-ID: <0799D0A8-35B3-4BF3-9F28-61CF2DC10E76 at swarthmore.edu>
>> Content-Type: text/plain; charset="utf-8"
>>
>>
>> > On Aug 4, 2020, at 9:16 AM, Eric Jensen <ejensen1 at swarthmore.edu>
>> wrote:
>> >
>> >  I realize that the basics of the binning part are hard with numpy
>> operations,
>>
>> Sorry, this should have said ?are *not* hard??
>>
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: <
>> http://mail.python.org/pipermail/astropy/attachments/20200804/29285de5/attachment.html
>> >
>> -------------- next part --------------
>> A non-text attachment was scrubbed...
>> Name: smime.p7s
>> Type: application/pkcs7-signature
>> Size: 3946 bytes
>> Desc: not available
>> URL: <
>> http://mail.python.org/pipermail/astropy/attachments/20200804/29285de5/attachment.bin
>> >
>>
>> ------------------------------
>>
>> Subject: Digest Footer
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at python.org
>> https://mail.python.org/mailman/listinfo/astropy
>>
>>
>> ------------------------------
>>
>> End of AstroPy Digest, Vol 167, Issue 2
>> ***************************************
>>
>
>
> --
> Jonathan D. Slavin
> Astrophysicist - High Energy Astrophysics Division
> Center for Astrophysics | Harvard & Smithsonian
> Office: (617) 496-7981 | Cell: (781) 363-0035
> 60 Garden Street | MS 83 | Cambridge, MA 02138
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20200804/4b0fb326/attachment-0001.html>

From adam.g.ginsburg at gmail.com  Tue Aug  4 14:41:21 2020
From: adam.g.ginsburg at gmail.com (Adam Ginsburg)
Date: Tue, 4 Aug 2020 14:41:21 -0400
Subject: [AstroPy] Rebin FITS images and preserve or recalculate WCS?
In-Reply-To: <CAApiRV6MuKKc+8HqNnSg6r3dm=bF4=DnHUt_XrtAJ2r3D1BZQg@mail.gmail.com>
References: <mailman.5428.1596547110.9565.astropy@python.org>
 <CACcRS=eP1_dJKQU6ZcR+YT6ZeAosJk8icMRoaR2-1HEsBxP9LQ@mail.gmail.com>
 <CAApiRV6MuKKc+8HqNnSg6r3dm=bF4=DnHUt_XrtAJ2r3D1BZQg@mail.gmail.com>
Message-ID: <CAEBNSwZoUTaHbN44scH3GM51ib7OpjR+VPk5NYBO6Qq4Z=1p0A@mail.gmail.com>

I'm a bit late to this, but I think astropy handles a lot of this really
well internally with the wcs object.  You can resample any WCS object with:
wcs[::2, ::2]
for example, and it will do something at least close to correct (in other
words, check that it looks right!)

I had written downsampling code years ago in this standalone package, but
I'm sure it's outdated - I just can't find the astropy-specific tool that
supplanted it
https://fits-tools.readthedocs.io/en/latest/_modules/FITS_tools/downsample.html

I think the code I link there is just a more complicated version of Gary's
suggestion:
>> m,n = img.shape
>> img = np.sum( img.reshape(m//s,s,n), axis=1)  # Contract along y
>> Img = np.sum( img.reshape(m//s,n//s,s),axis=2). # Contract along x
but if you piece Gary's suggestion together with `wcs[::s, ::s]`, you
should be able to do this all with vetted tools.

On Tue, Aug 4, 2020 at 2:20 PM Kelle Cruz <kellecruz at gmail.com> wrote:

> Link to FB discussion:
> https://www.facebook.com/groups/astropython/permalink/2702447229999950/
>
> and link to a printout of the discussion:
>
> https://www.dropbox.com/s/zyy67kfemo03w66/%282%29%20Python%20users%20in%20Astronomy.pdf?dl=0
>
> Kelle
>
> --
> Kelle Cruz, PhD (she/her, they/them)
> Assoc. Professor, Physics and Astronomy, Hunter College
> CEO, ScienceBetter Consulting, LLC
> CFO, Startorialist, Inc. <http://shop.startorialist.com>
> Editor-in-Chief, AstroBetter Blog and Wiki <http://astrobetter.com>
> Director, McNulty Scholars Program, Hunter College
> Research Associate, American Museum of Natural History
> Visiting Scholar, Center for Computational Astrophysics
>
>
> On Tue, Aug 4, 2020 at 11:43 AM Slavin, Jonathan <jslavin at cfa.harvard.edu>
> wrote:
>
>> Hi Eric,
>>
>> I posted a very similar question to the Python users in Astronomy
>> Facebook group a few weeks ago (July 7). You might look up that discussion.
>> There were some interesting suggestions, though I wouldn't say any
>> definitive answers.
>>
>> Regards,
>> Jon
>>
>> On Tue, Aug 4, 2020 at 9:18 AM <astropy-request at python.org> wrote:
>>
>>> Today's Topics:
>>>
>>>    1. Re: Rebin FITS images and preserve or recalculate WCS?
>>>       (Eric Jensen)
>>>    2. Re: Rebin FITS images and preserve or recalculate WCS?
>>>       (Eric Jensen)
>>>
>>>
>>> ----------------------------------------------------------------------
>>>
>>> Message: 1
>>> Date: Tue, 4 Aug 2020 09:16:29 -0400
>>> From: Eric Jensen <ejensen1 at swarthmore.edu>
>>> To: Astronomical Python mailing list <astropy at python.org>
>>> Subject: Re: [AstroPy] Rebin FITS images and preserve or recalculate
>>>         WCS?
>>> Message-ID: <4A90B8D1-DCF3-4BCB-8D57-90E20781CC86 at swarthmore.edu>
>>> Content-Type: text/plain; charset="utf-8"
>>>
>>> Thanks, Gary and Petr!  I appreciate the answers so far.  A couple of
>>> additional notes:
>>>
>>> I should have been clear that I realize that the basics of the binning
>>> part are hard with numpy operations, though I appreciate the clear code
>>> from Gary.   There are possibly edge-of-image issues, but those are
>>> probably best handled by just restricting the binning factor to numbers
>>> that divide evenly into the array dimensions.
>>>
>>> However, there are potential bit-depth issues - if I add together
>>> several pixels with 50,000 counts (previously stored as 16-bit unsigned
>>> integers) I?ll need to change the bit depth.  Maybe astropy?s FITS-handling
>>> routines make that automatic when writing an array to a new FITS file.
>>>
>>> That said, it?s mostly the subtleties of the WCS part that I?d like to
>>> handle with vetted tools if possible.
>>>
>>> For example, if I look at a FITS header from an image solved by
>>> astrometry.net, it has not just CRVAL1,2 + CRPIX1,2 + a CD matrix, but
>>> it also has SIP distortion coefficients.  It?s possible that those are
>>> negligible for our images (I haven?t yet checked them to work out the
>>> details of the standard to figure out how much difference they make) and
>>> they could just be ignored.  (And our images straight off the telescope,
>>> using MaximDL, have some kind of proprietary/undocumented distortion
>>> correction scheme that I?ll just need to ignore anyway, I think, so maybe
>>> the distortion issue isn?t important.  I?ll have to test that.)
>>>
>>> But it highlights the general issue that WCS has lots of permutations,
>>> and it seems better to handle it (if possible) with tested tools such as
>>> already exist in astropy.  The question is whether there?s a useful way to
>>> deploy the astropy (or other library) WCS functionality here, or whether it
>>> is indeed best just to do it manually with the linear coefficients
>>> following the solution outlined by Gary and Petr already.
>>>
>>> One specific question:
>>>
>>> > On Aug 4, 2020, at 6:33 AM, Petr Kub?nek <petr at kubanek.net> wrote:
>>> >
>>> > And if you happen to have WCS in CD_ matrix, divide it by 2 (scalar,
>>> eg. all members).
>>>
>>> These would need to be *multiplied* by the binning factor, yes?
>>>
>>> If there is no general solution already implemented and tested, I
>>> certainly can dig into it more myself - I think the basic outline (minus
>>> distortion) is already given, modulo subtleties I haven?t though of.  But
>>> in general I?ve encouraged my students to use tested libraries rather than
>>> re-inventing the wheel when possible, so I?m trying to take my own advice!
>>> :-)
>>>
>>> Thanks for any additional thoughts,
>>>
>>> Eric
>>>
>>>
>>> > On Aug 4, 2020, at 6:33 AM, Petr Kub?nek <petr at kubanek.net> wrote:
>>> >
>>> > And if you happen to have WCS in CD_ matrix, divide it by 2 (scalar,
>>> eg. all members).
>>> >
>>> > Petr
>>> >
>>> >> 4. 8. 2020 v 11:44 dop., Bernstein, Gary M <garyb at PHYSICS.UPENN.EDU>:
>>> >>
>>> >> Hi Eric -
>>> >> I don?t know whether this exists but in this case it might be faster
>>> to write than to do the Google search.  The block summing can be done in
>>> numpy by reshaping the array.  In the case where the original image shape
>>> is divisible by the shrinkage factor s, you?d do
>>> >>
>>> >> m,n = img.shape
>>> >> img = np.sum( img.reshape(m//s,s,n), axis=1)  # Contract along y
>>> >> Img = np.sum( img.reshape(m//s,n//s,s),axis=2). # Contract along x
>>> >>
>>> >> And to fix the WCS, you need to divide CRPIXn by s, and multiply
>>> CDELTn by s.  There is some slight complication to the first operation
>>> because FITS assumes 1-indexed pixels, and the integer marks the center of
>>> a pixel, so its really CRPIXn -> (CRPIXn?0.5)/s + 0.5
>>> >>
>>> >> I think this should work universally because the linear transform is
>>> always the first step of a WCS mapping.
>>> >>
>>> >> Cheers,
>>> >> Gary
>>> >>
>>> >>> On Aug 3, 2020, at 11:01 PM, Eric Jensen <ejensen1 at swarthmore.edu>
>>> wrote:
>>> >>>
>>> >>> Hi all,
>>> >>>
>>> >>> We are looking at purchasing a new CMOS camera that has 4-micron
>>> pixels, which would significantly undersample our PSF. (The camera is
>>> otherwise excellent for our needs, e.g. excellent QE.)  So I?m looking at
>>> whether there would be a straightforward way in scripted post-processing to
>>> reduce the image file sizes while not losing information.  The existing
>>> camera driver doesn?t support more than 2x2 binning at this time.
>>> >>>
>>> >>> Is there available Python code that can take a FITS image with a
>>> valid WCS in the header and rebin it, and output a FITS image that still
>>> has a valid WCS for the rebinned image?  The rebinning should also preserve
>>> flux, though I think that?s easier than the WCS part.  Nothing obvious
>>> turns up in a little searching, but I could easily have missed it.
>>> >>>
>>> >>> It seems that Montage might be able to do this (mShrink) but if
>>> possible I?d prefer a pure-Python solution, as it would be easier to
>>> implement under Windows on our observatory control computer.  (A fallback
>>> would be to install a Linux distro on top of the Windows Subsystem for
>>> Linux, but if it?s possible to do it purely in Python that would be a lot
>>> simpler.)
>>> >>>
>>> >>> Thanks for any thoughts,
>>> >>>
>>> >>> Eric
>>> >>>
>>> >>> Eric Jensen
>>> >>> Professor of Astronomy
>>> >>> Dept. of Physics and Astronomy
>>> >>> Swarthmore College
>>> >>> _______________________________________________
>>> >>> AstroPy mailing list
>>> >>> AstroPy at python.org
>>> >>> https://mail.python.org/mailman/listinfo/astropy
>>> >>
>>> >> _______________________________________________
>>> >> AstroPy mailing list
>>> >> AstroPy at python.org
>>> >> https://mail.python.org/mailman/listinfo/astropy
>>> >
>>> > _______________________________________________
>>> > AstroPy mailing list
>>> > AstroPy at python.org
>>> > https://mail.python.org/mailman/listinfo/astropy
>>>
>>> -------------- next part --------------
>>> A non-text attachment was scrubbed...
>>> Name: smime.p7s
>>> Type: application/pkcs7-signature
>>> Size: 3946 bytes
>>> Desc: not available
>>> URL: <
>>> http://mail.python.org/pipermail/astropy/attachments/20200804/ba4e8cae/attachment-0001.bin
>>> >
>>>
>>> ------------------------------
>>>
>>> Message: 2
>>> Date: Tue, 4 Aug 2020 09:18:25 -0400
>>> From: Eric Jensen <ejensen1 at swarthmore.edu>
>>> To: Astronomical Python mailing list <astropy at python.org>
>>> Subject: Re: [AstroPy] Rebin FITS images and preserve or recalculate
>>>         WCS?
>>> Message-ID: <0799D0A8-35B3-4BF3-9F28-61CF2DC10E76 at swarthmore.edu>
>>> Content-Type: text/plain; charset="utf-8"
>>>
>>>
>>> > On Aug 4, 2020, at 9:16 AM, Eric Jensen <ejensen1 at swarthmore.edu>
>>> wrote:
>>> >
>>> >  I realize that the basics of the binning part are hard with numpy
>>> operations,
>>>
>>> Sorry, this should have said ?are *not* hard??
>>>
>>> -------------- next part --------------
>>> An HTML attachment was scrubbed...
>>> URL: <
>>> http://mail.python.org/pipermail/astropy/attachments/20200804/29285de5/attachment.html
>>> >
>>> -------------- next part --------------
>>> A non-text attachment was scrubbed...
>>> Name: smime.p7s
>>> Type: application/pkcs7-signature
>>> Size: 3946 bytes
>>> Desc: not available
>>> URL: <
>>> http://mail.python.org/pipermail/astropy/attachments/20200804/29285de5/attachment.bin
>>> >
>>>
>>> ------------------------------
>>>
>>> Subject: Digest Footer
>>>
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at python.org
>>> https://mail.python.org/mailman/listinfo/astropy
>>>
>>>
>>> ------------------------------
>>>
>>> End of AstroPy Digest, Vol 167, Issue 2
>>> ***************************************
>>>
>>
>>
>> --
>> Jonathan D. Slavin
>> Astrophysicist - High Energy Astrophysics Division
>> Center for Astrophysics | Harvard & Smithsonian
>> Office: (617) 496-7981 | Cell: (781) 363-0035
>> 60 Garden Street | MS 83 | Cambridge, MA 02138
>>
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at python.org
>> https://mail.python.org/mailman/listinfo/astropy
>>
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy
>


-- 
Adam Ginsburg
Assistant Professor, Department of Astronomy
University of Florida, Gainesville
http://www.adamgginsburg.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20200804/9c7344ee/attachment-0001.html>

From larry.bradley at gmail.com  Tue Aug  4 14:46:09 2020
From: larry.bradley at gmail.com (Larry Bradley)
Date: Tue, 4 Aug 2020 14:46:09 -0400
Subject: [AstroPy] Rebin FITS images and preserve or recalculate WCS?
In-Reply-To: <CAEBNSwZoUTaHbN44scH3GM51ib7OpjR+VPk5NYBO6Qq4Z=1p0A@mail.gmail.com>
References: <mailman.5428.1596547110.9565.astropy@python.org>
 <CACcRS=eP1_dJKQU6ZcR+YT6ZeAosJk8icMRoaR2-1HEsBxP9LQ@mail.gmail.com>
 <CAApiRV6MuKKc+8HqNnSg6r3dm=bF4=DnHUt_XrtAJ2r3D1BZQg@mail.gmail.com>
 <CAEBNSwZoUTaHbN44scH3GM51ib7OpjR+VPk5NYBO6Qq4Z=1p0A@mail.gmail.com>
Message-ID: <CAFfmSa0ur1EDQzF4Hbi4GW34AcjPd_68s-zW=gSDns6nZC0=pQ@mail.gmail.com>

The astropy function to downsample an array by applying a user-supplied
function (e.g., np.mean, np.sum) to local blocks is block_reduce (
https://docs.astropy.org/en/latest/api/astropy.nddata.block_reduce.html).
It does not handle the WCS.

On Tue, Aug 4, 2020 at 2:41 PM Adam Ginsburg <adam.g.ginsburg at gmail.com>
wrote:

> I'm a bit late to this, but I think astropy handles a lot of this really
> well internally with the wcs object.  You can resample any WCS object with:
> wcs[::2, ::2]
> for example, and it will do something at least close to correct (in other
> words, check that it looks right!)
>
> I had written downsampling code years ago in this standalone package, but
> I'm sure it's outdated - I just can't find the astropy-specific tool that
> supplanted it
>
> https://fits-tools.readthedocs.io/en/latest/_modules/FITS_tools/downsample.html
>
> I think the code I link there is just a more complicated version of Gary's
> suggestion:
> >> m,n = img.shape
> >> img = np.sum( img.reshape(m//s,s,n), axis=1)  # Contract along y
> >> Img = np.sum( img.reshape(m//s,n//s,s),axis=2). # Contract along x
> but if you piece Gary's suggestion together with `wcs[::s, ::s]`, you
> should be able to do this all with vetted tools.
>
> On Tue, Aug 4, 2020 at 2:20 PM Kelle Cruz <kellecruz at gmail.com> wrote:
>
>> Link to FB discussion:
>> https://www.facebook.com/groups/astropython/permalink/2702447229999950/
>>
>> and link to a printout of the discussion:
>>
>> https://www.dropbox.com/s/zyy67kfemo03w66/%282%29%20Python%20users%20in%20Astronomy.pdf?dl=0
>>
>> Kelle
>>
>> --
>> Kelle Cruz, PhD (she/her, they/them)
>> Assoc. Professor, Physics and Astronomy, Hunter College
>> CEO, ScienceBetter Consulting, LLC
>> CFO, Startorialist, Inc. <http://shop.startorialist.com>
>> Editor-in-Chief, AstroBetter Blog and Wiki <http://astrobetter.com>
>> Director, McNulty Scholars Program, Hunter College
>> Research Associate, American Museum of Natural History
>> Visiting Scholar, Center for Computational Astrophysics
>>
>>
>> On Tue, Aug 4, 2020 at 11:43 AM Slavin, Jonathan <jslavin at cfa.harvard.edu>
>> wrote:
>>
>>> Hi Eric,
>>>
>>> I posted a very similar question to the Python users in Astronomy
>>> Facebook group a few weeks ago (July 7). You might look up that discussion.
>>> There were some interesting suggestions, though I wouldn't say any
>>> definitive answers.
>>>
>>> Regards,
>>> Jon
>>>
>>> On Tue, Aug 4, 2020 at 9:18 AM <astropy-request at python.org> wrote:
>>>
>>>> Today's Topics:
>>>>
>>>>    1. Re: Rebin FITS images and preserve or recalculate WCS?
>>>>       (Eric Jensen)
>>>>    2. Re: Rebin FITS images and preserve or recalculate WCS?
>>>>       (Eric Jensen)
>>>>
>>>>
>>>> ----------------------------------------------------------------------
>>>>
>>>> Message: 1
>>>> Date: Tue, 4 Aug 2020 09:16:29 -0400
>>>> From: Eric Jensen <ejensen1 at swarthmore.edu>
>>>> To: Astronomical Python mailing list <astropy at python.org>
>>>> Subject: Re: [AstroPy] Rebin FITS images and preserve or recalculate
>>>>         WCS?
>>>> Message-ID: <4A90B8D1-DCF3-4BCB-8D57-90E20781CC86 at swarthmore.edu>
>>>> Content-Type: text/plain; charset="utf-8"
>>>>
>>>> Thanks, Gary and Petr!  I appreciate the answers so far.  A couple of
>>>> additional notes:
>>>>
>>>> I should have been clear that I realize that the basics of the binning
>>>> part are hard with numpy operations, though I appreciate the clear code
>>>> from Gary.   There are possibly edge-of-image issues, but those are
>>>> probably best handled by just restricting the binning factor to numbers
>>>> that divide evenly into the array dimensions.
>>>>
>>>> However, there are potential bit-depth issues - if I add together
>>>> several pixels with 50,000 counts (previously stored as 16-bit unsigned
>>>> integers) I?ll need to change the bit depth.  Maybe astropy?s FITS-handling
>>>> routines make that automatic when writing an array to a new FITS file.
>>>>
>>>> That said, it?s mostly the subtleties of the WCS part that I?d like to
>>>> handle with vetted tools if possible.
>>>>
>>>> For example, if I look at a FITS header from an image solved by
>>>> astrometry.net, it has not just CRVAL1,2 + CRPIX1,2 + a CD matrix, but
>>>> it also has SIP distortion coefficients.  It?s possible that those are
>>>> negligible for our images (I haven?t yet checked them to work out the
>>>> details of the standard to figure out how much difference they make) and
>>>> they could just be ignored.  (And our images straight off the telescope,
>>>> using MaximDL, have some kind of proprietary/undocumented distortion
>>>> correction scheme that I?ll just need to ignore anyway, I think, so maybe
>>>> the distortion issue isn?t important.  I?ll have to test that.)
>>>>
>>>> But it highlights the general issue that WCS has lots of permutations,
>>>> and it seems better to handle it (if possible) with tested tools such as
>>>> already exist in astropy.  The question is whether there?s a useful way to
>>>> deploy the astropy (or other library) WCS functionality here, or whether it
>>>> is indeed best just to do it manually with the linear coefficients
>>>> following the solution outlined by Gary and Petr already.
>>>>
>>>> One specific question:
>>>>
>>>> > On Aug 4, 2020, at 6:33 AM, Petr Kub?nek <petr at kubanek.net> wrote:
>>>> >
>>>> > And if you happen to have WCS in CD_ matrix, divide it by 2 (scalar,
>>>> eg. all members).
>>>>
>>>> These would need to be *multiplied* by the binning factor, yes?
>>>>
>>>> If there is no general solution already implemented and tested, I
>>>> certainly can dig into it more myself - I think the basic outline (minus
>>>> distortion) is already given, modulo subtleties I haven?t though of.  But
>>>> in general I?ve encouraged my students to use tested libraries rather than
>>>> re-inventing the wheel when possible, so I?m trying to take my own advice!
>>>> :-)
>>>>
>>>> Thanks for any additional thoughts,
>>>>
>>>> Eric
>>>>
>>>>
>>>> > On Aug 4, 2020, at 6:33 AM, Petr Kub?nek <petr at kubanek.net> wrote:
>>>> >
>>>> > And if you happen to have WCS in CD_ matrix, divide it by 2 (scalar,
>>>> eg. all members).
>>>> >
>>>> > Petr
>>>> >
>>>> >> 4. 8. 2020 v 11:44 dop., Bernstein, Gary M <garyb at PHYSICS.UPENN.EDU
>>>> >:
>>>> >>
>>>> >> Hi Eric -
>>>> >> I don?t know whether this exists but in this case it might be faster
>>>> to write than to do the Google search.  The block summing can be done in
>>>> numpy by reshaping the array.  In the case where the original image shape
>>>> is divisible by the shrinkage factor s, you?d do
>>>> >>
>>>> >> m,n = img.shape
>>>> >> img = np.sum( img.reshape(m//s,s,n), axis=1)  # Contract along y
>>>> >> Img = np.sum( img.reshape(m//s,n//s,s),axis=2). # Contract along x
>>>> >>
>>>> >> And to fix the WCS, you need to divide CRPIXn by s, and multiply
>>>> CDELTn by s.  There is some slight complication to the first operation
>>>> because FITS assumes 1-indexed pixels, and the integer marks the center of
>>>> a pixel, so its really CRPIXn -> (CRPIXn?0.5)/s + 0.5
>>>> >>
>>>> >> I think this should work universally because the linear transform is
>>>> always the first step of a WCS mapping.
>>>> >>
>>>> >> Cheers,
>>>> >> Gary
>>>> >>
>>>> >>> On Aug 3, 2020, at 11:01 PM, Eric Jensen <ejensen1 at swarthmore.edu>
>>>> wrote:
>>>> >>>
>>>> >>> Hi all,
>>>> >>>
>>>> >>> We are looking at purchasing a new CMOS camera that has 4-micron
>>>> pixels, which would significantly undersample our PSF. (The camera is
>>>> otherwise excellent for our needs, e.g. excellent QE.)  So I?m looking at
>>>> whether there would be a straightforward way in scripted post-processing to
>>>> reduce the image file sizes while not losing information.  The existing
>>>> camera driver doesn?t support more than 2x2 binning at this time.
>>>> >>>
>>>> >>> Is there available Python code that can take a FITS image with a
>>>> valid WCS in the header and rebin it, and output a FITS image that still
>>>> has a valid WCS for the rebinned image?  The rebinning should also preserve
>>>> flux, though I think that?s easier than the WCS part.  Nothing obvious
>>>> turns up in a little searching, but I could easily have missed it.
>>>> >>>
>>>> >>> It seems that Montage might be able to do this (mShrink) but if
>>>> possible I?d prefer a pure-Python solution, as it would be easier to
>>>> implement under Windows on our observatory control computer.  (A fallback
>>>> would be to install a Linux distro on top of the Windows Subsystem for
>>>> Linux, but if it?s possible to do it purely in Python that would be a lot
>>>> simpler.)
>>>> >>>
>>>> >>> Thanks for any thoughts,
>>>> >>>
>>>> >>> Eric
>>>> >>>
>>>> >>> Eric Jensen
>>>> >>> Professor of Astronomy
>>>> >>> Dept. of Physics and Astronomy
>>>> >>> Swarthmore College
>>>> >>> _______________________________________________
>>>> >>> AstroPy mailing list
>>>> >>> AstroPy at python.org
>>>> >>> https://mail.python.org/mailman/listinfo/astropy
>>>> >>
>>>> >> _______________________________________________
>>>> >> AstroPy mailing list
>>>> >> AstroPy at python.org
>>>> >> https://mail.python.org/mailman/listinfo/astropy
>>>> >
>>>> > _______________________________________________
>>>> > AstroPy mailing list
>>>> > AstroPy at python.org
>>>> > https://mail.python.org/mailman/listinfo/astropy
>>>>
>>>> -------------- next part --------------
>>>> A non-text attachment was scrubbed...
>>>> Name: smime.p7s
>>>> Type: application/pkcs7-signature
>>>> Size: 3946 bytes
>>>> Desc: not available
>>>> URL: <
>>>> http://mail.python.org/pipermail/astropy/attachments/20200804/ba4e8cae/attachment-0001.bin
>>>> >
>>>>
>>>> ------------------------------
>>>>
>>>> Message: 2
>>>> Date: Tue, 4 Aug 2020 09:18:25 -0400
>>>> From: Eric Jensen <ejensen1 at swarthmore.edu>
>>>> To: Astronomical Python mailing list <astropy at python.org>
>>>> Subject: Re: [AstroPy] Rebin FITS images and preserve or recalculate
>>>>         WCS?
>>>> Message-ID: <0799D0A8-35B3-4BF3-9F28-61CF2DC10E76 at swarthmore.edu>
>>>> Content-Type: text/plain; charset="utf-8"
>>>>
>>>>
>>>> > On Aug 4, 2020, at 9:16 AM, Eric Jensen <ejensen1 at swarthmore.edu>
>>>> wrote:
>>>> >
>>>> >  I realize that the basics of the binning part are hard with numpy
>>>> operations,
>>>>
>>>> Sorry, this should have said ?are *not* hard??
>>>>
>>>> -------------- next part --------------
>>>> An HTML attachment was scrubbed...
>>>> URL: <
>>>> http://mail.python.org/pipermail/astropy/attachments/20200804/29285de5/attachment.html
>>>> >
>>>> -------------- next part --------------
>>>> A non-text attachment was scrubbed...
>>>> Name: smime.p7s
>>>> Type: application/pkcs7-signature
>>>> Size: 3946 bytes
>>>> Desc: not available
>>>> URL: <
>>>> http://mail.python.org/pipermail/astropy/attachments/20200804/29285de5/attachment.bin
>>>> >
>>>>
>>>> ------------------------------
>>>>
>>>> Subject: Digest Footer
>>>>
>>>> _______________________________________________
>>>> AstroPy mailing list
>>>> AstroPy at python.org
>>>> https://mail.python.org/mailman/listinfo/astropy
>>>>
>>>>
>>>> ------------------------------
>>>>
>>>> End of AstroPy Digest, Vol 167, Issue 2
>>>> ***************************************
>>>>
>>>
>>>
>>> --
>>> Jonathan D. Slavin
>>> Astrophysicist - High Energy Astrophysics Division
>>> Center for Astrophysics | Harvard & Smithsonian
>>> Office: (617) 496-7981 | Cell: (781) 363-0035
>>> 60 Garden Street | MS 83 | Cambridge, MA 02138
>>>
>>>
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at python.org
>>> https://mail.python.org/mailman/listinfo/astropy
>>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at python.org
>> https://mail.python.org/mailman/listinfo/astropy
>>
>
>
> --
> Adam Ginsburg
> Assistant Professor, Department of Astronomy
> University of Florida, Gainesville
> http://www.adamgginsburg.com/
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20200804/213f537a/attachment-0001.html>

From ejensen1 at swarthmore.edu  Tue Aug  4 16:55:40 2020
From: ejensen1 at swarthmore.edu (Eric Jensen)
Date: Tue, 4 Aug 2020 16:55:40 -0400
Subject: [AstroPy] Rebin FITS images and preserve or recalculate WCS?
In-Reply-To: <CAFfmSa0ur1EDQzF4Hbi4GW34AcjPd_68s-zW=gSDns6nZC0=pQ@mail.gmail.com>
References: <mailman.5428.1596547110.9565.astropy@python.org>
 <CACcRS=eP1_dJKQU6ZcR+YT6ZeAosJk8icMRoaR2-1HEsBxP9LQ@mail.gmail.com>
 <CAApiRV6MuKKc+8HqNnSg6r3dm=bF4=DnHUt_XrtAJ2r3D1BZQg@mail.gmail.com>
 <CAEBNSwZoUTaHbN44scH3GM51ib7OpjR+VPk5NYBO6Qq4Z=1p0A@mail.gmail.com>
 <CAFfmSa0ur1EDQzF4Hbi4GW34AcjPd_68s-zW=gSDns6nZC0=pQ@mail.gmail.com>
Message-ID: <4058F87C-B8FE-43BC-B015-913F199111EA@swarthmore.edu>

Ah, excellent!  This is the kind of thing I was hoping for. 

I?ll play around with Larry?s suggestion of block_reduce combined with Adam?s idea of WCS resampling and see if I can come up with a compact solution.   It may be a little bit before I get to test it fully, but I?ll definitely post back here to share the final product. 

Thanks to all for the help!

Eric

> On Aug 4, 2020, at 2:46 PM, Larry Bradley <larry.bradley at gmail.com> wrote:
> 
> The astropy function to downsample an array by applying a user-supplied function (e.g., np.mean, np.sum) to local blocks is block_reduce (https://docs.astropy.org/en/latest/api/astropy.nddata.block_reduce.html <https://docs.astropy.org/en/latest/api/astropy.nddata.block_reduce.html>).  It does not handle the WCS.
> 
> On Tue, Aug 4, 2020 at 2:41 PM Adam Ginsburg <adam.g.ginsburg at gmail.com <mailto:adam.g.ginsburg at gmail.com>> wrote:
> I'm a bit late to this, but I think astropy handles a lot of this really well internally with the wcs object.  You can resample any WCS object with:
> wcs[::2, ::2]
> for example, and it will do something at least close to correct (in other words, check that it looks right!)
> 
> I had written downsampling code years ago in this standalone package, but I'm sure it's outdated - I just can't find the astropy-specific tool that supplanted it
> https://fits-tools.readthedocs.io/en/latest/_modules/FITS_tools/downsample.html <https://fits-tools.readthedocs.io/en/latest/_modules/FITS_tools/downsample.html>  
> I think the code I link there is just a more complicated version of Gary's suggestion:
> >> m,n = img.shape
> >> img = np.sum( img.reshape(m//s,s,n), axis=1)  # Contract along y
> >> Img = np.sum( img.reshape(m//s,n//s,s),axis=2). # Contract along x 
> but if you piece Gary's suggestion together with `wcs[::s, ::s]`, you should be able to do this all with vetted tools. 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20200804/fd44f1a2/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3946 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/astropy/attachments/20200804/fd44f1a2/attachment.bin>

From garyb at PHYSICS.UPENN.EDU  Wed Aug  5 16:59:00 2020
From: garyb at PHYSICS.UPENN.EDU (Bernstein, Gary M)
Date: Wed, 5 Aug 2020 20:59:00 +0000
Subject: [AstroPy] Rebin FITS images and preserve or recalculate WCS?
In-Reply-To: <CAEBNSwZoUTaHbN44scH3GM51ib7OpjR+VPk5NYBO6Qq4Z=1p0A@mail.gmail.com>
References: <mailman.5428.1596547110.9565.astropy@python.org>
 <CACcRS=eP1_dJKQU6ZcR+YT6ZeAosJk8icMRoaR2-1HEsBxP9LQ@mail.gmail.com>
 <CAApiRV6MuKKc+8HqNnSg6r3dm=bF4=DnHUt_XrtAJ2r3D1BZQg@mail.gmail.com>
 <CAEBNSwZoUTaHbN44scH3GM51ib7OpjR+VPk5NYBO6Qq4Z=1p0A@mail.gmail.com>
Message-ID: <E1C36211-5986-4D6C-BB8C-F65236DCB27E@physics.upenn.edu>

Thanks Adam, this idea of slicing WCS objects is really clever and useful, I didn?t know about it.  I tried it out:

w1 = WCS(header=pf.getheader(?someimage.fits?))
w2 = w1[::2,::2]

and I find that the CRPIXn values have been properly changed but the CDp_q values were not changed; and w1.pixel_to_world(100.5,100.5) is quite different from w2.pixel_to_world(50.5,50.5).  So maybe it?s not fully implemented?  The docs for the `slice` method of WCS at https://docs.astropy.org/en/stable/api/astropy.wcs.WCS.html#astropy.wcs.WCS.slice suggest this is the case: "The step method, the third argument to a slice, is not presently supported.? 

I didn?t see any open issues about this on Astropy but I?m not experienced in looking through them.

Gary

> On Aug 4, 2020, at 2:41 PM, Adam Ginsburg <adam.g.ginsburg at gmail.com> wrote:
> 
> I'm a bit late to this, but I think astropy handles a lot of this really well internally with the wcs object.  You can resample any WCS object with:
> wcs[::2, ::2]
> for example, and it will do something at least close to correct (in other words, check that it looks right!)
> 
> I had written downsampling code years ago in this standalone package, but I'm sure it's outdated - I just can't find the astropy-specific tool that supplanted it
> https://fits-tools.readthedocs.io/en/latest/_modules/FITS_tools/downsample.html  
> I think the code I link there is just a more complicated version of Gary's suggestion:
> >> m,n = img.shape
> >> img = np.sum( img.reshape(m//s,s,n), axis=1)  # Contract along y
> >> Img = np.sum( img.reshape(m//s,n//s,s),axis=2). # Contract along x 
> but if you piece Gary's suggestion together with `wcs[::s, ::s]`, you should be able to do this all with vetted tools. 
> 
> On Tue, Aug 4, 2020 at 2:20 PM Kelle Cruz <kellecruz at gmail.com> wrote:
> Link to FB discussion: https://www.facebook.com/groups/astropython/permalink/2702447229999950/
> 
> and link to a printout of the discussion: 
> https://www.dropbox.com/s/zyy67kfemo03w66/%282%29%20Python%20users%20in%20Astronomy.pdf?dl=0
> 
> Kelle
> 
> --
> Kelle Cruz, PhD (she/her, they/them)
> Assoc. Professor, Physics and Astronomy, Hunter College
> CEO, ScienceBetter Consulting, LLC
> CFO, Startorialist, Inc.
> Editor-in-Chief, AstroBetter Blog and Wiki
> Director, McNulty Scholars Program, Hunter College
> Research Associate, American Museum of Natural History
> Visiting Scholar, Center for Computational Astrophysics
> 


From adam.g.ginsburg at gmail.com  Wed Aug  5 17:12:06 2020
From: adam.g.ginsburg at gmail.com (Adam Ginsburg)
Date: Wed, 5 Aug 2020 17:12:06 -0400
Subject: [AstroPy] Rebin FITS images and preserve or recalculate WCS?
In-Reply-To: <E1C36211-5986-4D6C-BB8C-F65236DCB27E@physics.upenn.edu>
References: <mailman.5428.1596547110.9565.astropy@python.org>
 <CACcRS=eP1_dJKQU6ZcR+YT6ZeAosJk8icMRoaR2-1HEsBxP9LQ@mail.gmail.com>
 <CAApiRV6MuKKc+8HqNnSg6r3dm=bF4=DnHUt_XrtAJ2r3D1BZQg@mail.gmail.com>
 <CAEBNSwZoUTaHbN44scH3GM51ib7OpjR+VPk5NYBO6Qq4Z=1p0A@mail.gmail.com>
 <E1C36211-5986-4D6C-BB8C-F65236DCB27E@physics.upenn.edu>
Message-ID: <CAEBNSwaZvrC91W8953V2Du3n54S-BN8x+QzCDadYu+bZW1thpA@mail.gmail.com>

Oops.  Yeah, I guess we partly implemented that - if you have a WCS that's
aligned with the celestial axes, i.e., it has CDELT specified but no CROTA
and only an identity matrix for CD, then it will work.  For other cases,
it's less straightforward.  It looks like I'm to blame for the
partially-implemented state of this:
https://github.com/astropy/astropy/blame/master/astropy/wcs/wcs.py#L3010

It might be possible, but not trivial, to generalize this to the CD case.

On Wed, Aug 5, 2020 at 4:59 PM Bernstein, Gary M <garyb at physics.upenn.edu>
wrote:

> Thanks Adam, this idea of slicing WCS objects is really clever and useful,
> I didn?t know about it.  I tried it out:
>
> w1 = WCS(header=pf.getheader(?someimage.fits?))
> w2 = w1[::2,::2]
>
> and I find that the CRPIXn values have been properly changed but the CDp_q
> values were not changed; and w1.pixel_to_world(100.5,100.5) is quite
> different from w2.pixel_to_world(50.5,50.5).  So maybe it?s not fully
> implemented?  The docs for the `slice` method of WCS at
> https://docs.astropy.org/en/stable/api/astropy.wcs.WCS.html#astropy.wcs.WCS.slice
> suggest this is the case: "The step method, the third argument to a slice,
> is not presently supported.?
>
> I didn?t see any open issues about this on Astropy but I?m not experienced
> in looking through them.
>
> Gary
>
> > On Aug 4, 2020, at 2:41 PM, Adam Ginsburg <adam.g.ginsburg at gmail.com>
> wrote:
> >
> > I'm a bit late to this, but I think astropy handles a lot of this really
> well internally with the wcs object.  You can resample any WCS object with:
> > wcs[::2, ::2]
> > for example, and it will do something at least close to correct (in
> other words, check that it looks right!)
> >
> > I had written downsampling code years ago in this standalone package,
> but I'm sure it's outdated - I just can't find the astropy-specific tool
> that supplanted it
> >
> https://fits-tools.readthedocs.io/en/latest/_modules/FITS_tools/downsample.html
>
> > I think the code I link there is just a more complicated version of
> Gary's suggestion:
> > >> m,n = img.shape
> > >> img = np.sum( img.reshape(m//s,s,n), axis=1)  # Contract along y
> > >> Img = np.sum( img.reshape(m//s,n//s,s),axis=2). # Contract along x
> > but if you piece Gary's suggestion together with `wcs[::s, ::s]`, you
> should be able to do this all with vetted tools.
> >
> > On Tue, Aug 4, 2020 at 2:20 PM Kelle Cruz <kellecruz at gmail.com> wrote:
> > Link to FB discussion:
> https://www.facebook.com/groups/astropython/permalink/2702447229999950/
> >
> > and link to a printout of the discussion:
> >
> https://www.dropbox.com/s/zyy67kfemo03w66/%282%29%20Python%20users%20in%20Astronomy.pdf?dl=0
> >
> > Kelle
> >
> > --
> > Kelle Cruz, PhD (she/her, they/them)
> > Assoc. Professor, Physics and Astronomy, Hunter College
> > CEO, ScienceBetter Consulting, LLC
> > CFO, Startorialist, Inc.
> > Editor-in-Chief, AstroBetter Blog and Wiki
> > Director, McNulty Scholars Program, Hunter College
> > Research Associate, American Museum of Natural History
> > Visiting Scholar, Center for Computational Astrophysics
> >
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy
>


-- 
Adam Ginsburg
Assistant Professor, Department of Astronomy
University of Florida, Gainesville
http://www.adamgginsburg.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20200805/21bfc068/attachment.html>

From moritz.guenther at gmx.de  Fri Aug  7 10:32:17 2020
From: moritz.guenther at gmx.de (=?UTF-8?Q?Moritz_G=c3=bcnther?=)
Date: Fri, 7 Aug 2020 10:32:17 -0400
Subject: [AstroPy] Call for proposals: Inclusion, Diversity, and Empowerment
Message-ID: <c9759b07-f77c-f693-2b7e-6739b378bce1@gmx.de>

We recognize that contributing to open source projects is a privilege
not available to everyone and that the Astropy Project community is
highly white, predominantly male, and is homogeneous in several other
ways, too. In our code of conduct
(https://www.astropy.org/code_of_conduct.html), we committed to being an
inclusive and welcoming community, but clearly it?s not enough to write
down a commitment, there needs to be action too. Hence, we urge all
Astropy community members to take action to broaden participation in the
Project at all levels and we want to empower those actions.

Often, such actions require resources and thus the Project asks for
proposals from members of the community for up to $15,000 for any action
that helps with inclusion, support, and empowerment of underrepresented
or marginalized groups in the Astropy community. We are looking for
proposals that lead to concrete actions soon, preferably in the next six
months.

We envision an open and collaborative process for proposals and the
proposal review, where ideas are opened as pull requests in
https://github.com/astropy/project (see template below), similar to the
way APEs are suggested, discussed and improved. That way any community
member can comment on, flesh out, add details to, and improve proposals.
Similar to hack day pitches, proposers do not need to have the needed
team already assembled and we envision the proposal process as an
opportunity for teams to be recruited and formed.

We hope that an open discussion will encourage people with similar ideas
to join forces and that the community will converge on one or two
proposals to be supported. This relies on community members to provide
honest and constructive feedback and input to see which proposals can be
realistically executed with the resources we have and have the most
positive impact.

If there is no consensus by end of August, the following process will be
used to decide: The CoCo will select 4-5 proposals that are (i) fundable
through the Moore grant, (ii) fleshed out in enough detail that work and
funding can happen on a six-month time scale, and (iii) have the most
community buy-in (counting number of contributors and positive comments
in the PR from distinct people).  We will then invite the community to
comment on those proposals specifically and, should a tie-break be
necessary, the CoCo will make a decision on September 15th.

This initiative was inspired by recent events in the US, but we stress
that proposals may target any underrepresented or marginalized groups
worldwide and Project-specific challenges. We also want to emphasize
that proposals can be outward facing (e.g., focusing on recruitment) or
inward facing (e.g., working on existing practices in our community
which may be exclusionary).

Funding for this comes from the Moore grant to the Astropy Project,
which includes $15,000 for ?mentoring? in the budget year Nov 2019 to
Oct 2020, thus one or more proposals with a total cost of $15,000 can be
funded. For proposals that do not fit under the ?mentoring? proposed in
the Moore grant, the finance committee will try to find alternative
funds. Likely, that will require to shift other spending between budget
items and thus the total cap of $15,000 still applies.

Moritz Guenther
(for the Astropy Interim Finance Committee)


Template
Think of this like a hack day pitch or a github issue for discussion,
not like a funding  proposal to a government agency. Short and informal
is encouraged.
Post proposals as PRs in https://github.com/astropy/project

# Title

## Summary
(2-3 sentences)

## Team
(people responsible for execution or places where more hands are needed,
not necessarily everybody who contributed to the proposal)

## Plan
(1-2 paragraphs)
What actions are proposed? What needs to be done?

## Impact
(1 paragraph)
How does this help with diversity, inclusion, and empowerment of people
from underrepresented groups in the Astropy Project? Be as brief and
specific as possible.

## Budget
(a few lines of text)
What?s the total cost and how will the money be spent?

From moritz.guenther at gmx.de  Tue Aug 11 06:35:12 2020
From: moritz.guenther at gmx.de (=?UTF-8?Q?Moritz_G=c3=bcnther?=)
Date: Tue, 11 Aug 2020 06:35:12 -0400
Subject: [AstroPy] Call for proposals: Inclusion, Diversity,
 and Empowerment
In-Reply-To: <2fc75f18-e993-516e-29d8-13c191047c22@gmx.de>
References: <2fc75f18-e993-516e-29d8-13c191047c22@gmx.de>
Message-ID: <ce161fb8-53e6-80f0-2207-37099eeb755d@gmx.de>

Hi,

I want to clarify one point about this call for proposals:

Proposals can come from anywhere on the planet and can be used to fund
projects almost anywhere. The money is provided by a grant from the
private Moore foundation, which does not restrict the use of the money
to the US. However, the funds are hold in a US bank account, so we
cannot transfer them to a country where the US imposes financial
sanctions (e.g. Iran, North Korea).

The Astropy Project contributors are heavily concentrated in "Western"
countries and widening participation is a valid goal.

Please use this issue: https://github.com/astropy/project/issues/120 for
any further questions that come up and check that issue for any further
clarifications.

Yours,
Moritz

On 8/7/20 10:27 AM, Moritz G?nther wrote:
> We recognize that contributing to open source projects is a privilege
> not available to everyone and that the Astropy Project community is
> highly white, predominantly male, and is homogeneous in several other
> ways, too. In our code of conduct
> (https://www.astropy.org/code_of_conduct.html), we committed to being an
> inclusive and welcoming community, but clearly it?s not enough to write
> down a commitment, there needs to be action too. Hence, we urge all
> Astropy community members to take action to broaden participation in the
> Project at all levels and we want to empower those actions.
>
> Often, such actions require resources and thus the Project asks for
> proposals from members of the community for up to $15,000 for any action
> that helps with inclusion, support, and empowerment of underrepresented
> or marginalized groups in the Astropy community. We are looking for
> proposals that lead to concrete actions soon, preferably in the next six
> months.
>
> We envision an open and collaborative process for proposals and the
> proposal review, where ideas are opened as pull requests in
> https://github.com/astropy/project (see template below), similar to the
> way APEs are suggested, discussed and improved. That way any community
> member can comment on, flesh out, add details to, and improve proposals.
> Similar to hack day pitches, proposers do not need to have the needed
> team already assembled and we envision the proposal process as an
> opportunity for teams to be recruited and formed.
>
> We hope that an open discussion will encourage people with similar ideas
> to join forces and that the community will converge on one or two
> proposals to be supported. This relies on community members to provide
> honest and constructive feedback and input to see which proposals can be
> realistically executed with the resources we have and have the most
> positive impact.
>
> If there is no consensus by end of August, the following process will be
> used to decide: The CoCo will select 4-5 proposals that are (i) fundable
> through the Moore grant, (ii) fleshed out in enough detail that work and
> funding can happen on a six-month time scale, and (iii) have the most
> community buy-in (counting number of contributors and positive comments
> in the PR from distinct people).  We will then invite the community to
> comment on those proposals specifically and, should a tie-break be
> necessary, the CoCo will make a decision on September 15th.
>
> This initiative was inspired by recent events in the US, but we stress
> that proposals may target any underrepresented or marginalized groups
> worldwide and Project-specific challenges. We also want to emphasize
> that proposals can be outward facing (e.g., focusing on recruitment) or
> inward facing (e.g., working on existing practices in our community
> which may be exclusionary).
>
> Funding for this comes from the Moore grant to the Astropy Project,
> which includes $15,000 for ?mentoring? in the budget year Nov 2019 to
> Oct 2020, thus one or more proposals with a total cost of $15,000 can be
> funded. For proposals that do not fit under the ?mentoring? proposed in
> the Moore grant, the finance committee will try to find alternative
> funds. Likely, that will require to shift other spending between budget
> items and thus the total cap of $15,000 still applies.
>
> Moritz Guenther
> (for the Astropy Interim Finance Committee)
>
>
> Template
> Think of this like a hack day pitch or a github issue for discussion,
> not like a funding  proposal to a government agency. Short and informal
> is encouraged.
> Post proposals as PRs in https://github.com/astropy/project
>
> # Title
>
> ## Summary
> (2-3 sentences)
>
> ## Team
> (people responsible for execution or places where more hands are needed,
> not necessarily everybody who contributed to the proposal)
>
> ## Plan
> (1-2 paragraphs)
> What actions are proposed? What needs to be done?
>
> ## Impact
> (1 paragraph)
> How does this help with diversity, inclusion, and empowerment of people
> from underrepresented groups in the Astropy Project? Be as brief and
> specific as possible.
>
> ## Budget
> (a few lines of text)
> What?s the total cost and how will the money be spent?
>

From taldcroft at gmail.com  Tue Aug 25 12:40:42 2020
From: taldcroft at gmail.com (Tom Aldcroft)
Date: Tue, 25 Aug 2020 12:40:42 -0400
Subject: [AstroPy] Astropy Ombudsperson nominations and CoCo running notes
Message-ID: <CAGsQ-NtO7c6qcDPG_00tO+=5q91gWL-MNY_9_Xv96VEnJGG-sw@mail.gmail.com>

Dear Astropy community,

We, the Coordination Committee, would like to give you an update on two
things:

   - We are seeking nominations for a new ombudsperson for the Astropy
   Project.
   - The Astropy Coordination Committee running meeting notes are now
   available.


*Ombudsperson*
Steve Crawford has accepted a new position with NASA as the Science Mission
Directorate at NASA as the new executive program office for scientific data
and computing. This is fantastic news for Steve and for the scientific
computing community (including Astropy!). However, it means that Steve will
no longer be able to serve as the Astropy ombudsperson and he has stepped
down as of Monday, Aug 17.

In advance of the new Astropy governance document (APE0) coming into
effect, the Coordination Committee is seeking nominations for the role of
interim Astropy Ombudsperson. Self-nominations are very welcome. APE0
contains a description of the ombudsperson voting process, so when it
becomes the active governance document, that process will be undertaken to
replace or confirm the interim Ombudsperson.

The Ombudsperson role has the following responsibilities:


*Provide a point of contact for sensitive issues separate from the
coordinating committee, including:*

   - *Monitoring the confidential at astropy.org
   <confidential at astropy.org> email account*
   - *Solicit and provide anonymized feedback to the astropy coordination
   committee regarding coordination of the project*
   - *Assist the coordination committee and community engagement
   coordinator with violations of the code of conduct or other ethical
   concerns*


*Coordination committee running notes*
In order to improve transparency and communications, the Coordination
Committee is taking a page from the interim finance committee: the meeting
notes of the Coordination Committee are now available at the following URL:

https://docs.google.com/document/d/19jnnQsnCbpSU1yUVmSLnVfOhrHyxN1P1kYAqiG4Ic7k/edit#

This includes both past meetings (starting from Aug 11, 2020) and a
tentative agenda for the next scheduled meeting. Questions, comments,
feedback on the notes can be made as comments on the doc, emails to
coordinators at astropy.org, discussion in the #project Slack channel, or
issues in the project repo.

Cheers,
Tom A on behalf the Coordination Committee
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20200825/66ffe519/attachment.html>