From jslavin at cfa.harvard.edu  Thu Jan  5 10:38:31 2023
From: jslavin at cfa.harvard.edu (Slavin, Jonathan)
Date: Thu, 5 Jan 2023 10:38:31 -0500
Subject: [AstroPy] passing hdus as arguments
Message-ID: <CACcRS=eHsQQC9=XYk9Mhd7KSSUMRyCTXD-8pN+mkJ1+E8+rMfA@mail.gmail.com>

Hi,

I have a question related to astropy.io.fits.
I'm wondering about the ramifications of defining a function to require
hdus as arguments. So the user would do:
hdu = fits.open(fitsfilename)
result = fcn(hdu,...)
What I wonder about is the ramifications if the hdu isn't closed. Of course
the call could use a context manager:
with fits.open(fitsfilename) as hdu:
    result = fcn(hdu,...)
which assures the hdu will be closed, though I don't think there's any way
to enforce calling it in that way. I suppose that the worst that is likely
to happen is that memory will be used up. I guess this is not really
different from any function using file handles as arguments. Any thoughts?

Jon

-- 
Jonathan D. Slavin (he/him)
Astrophysicist - High Energy Astrophysics Division
Center for Astrophysics | Harvard & Smithsonian
Office: (617) 496-7981 | Cell: (781) 363-0035
60 Garden Street | MS 04 | Cambridge, MA 02138
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/astropy/attachments/20230105/a6461861/attachment.html>

From erik.m.bray at gmail.com  Thu Jan  5 12:35:21 2023
From: erik.m.bray at gmail.com (E. Madison Bray)
Date: Thu, 5 Jan 2023 18:35:21 +0100
Subject: [AstroPy] passing hdus as arguments
In-Reply-To: <CACcRS=eHsQQC9=XYk9Mhd7KSSUMRyCTXD-8pN+mkJ1+E8+rMfA@mail.gmail.com>
References: <CACcRS=eHsQQC9=XYk9Mhd7KSSUMRyCTXD-8pN+mkJ1+E8+rMfA@mail.gmail.com>
Message-ID: <CAOTD34ZFe9hOCeuWbORvqads7W61Zwuz0qumuV0nGeKSOD4_=g@mail.gmail.com>

Hi Jon,

First a quick clarification about nomenclature : what fits.open() returns
is not an HDU (Header Data Unit, of which there can be one or more in a
single FITS file. Rather, something that for historical reasons is called
HDUList, but is just an object representing the whole FITS file. As you
surmised it contains a wrapper around a file handle. Keeping that open
doesn't use any significant memory, but reading the actual data arrays in
the individual HDUs will of course map their data into "real" memory.

As for the ramifications it depends on what your function is doing and how
it's intended to be used. In particular if it's just reading the data (and
not syncing updates back to the original file) I wouldn't overly worry
about it.

As long as there aren't any reference leaks, once there are no more
references to the HDUList object it will eventually be garbage collected.

Of course, if you're actually noticing some memory leaks that's worth
bringing up. There have been some in the past IIRC.


On Thu, Jan 5, 2023, 16:39 Slavin, Jonathan <jslavin at cfa.harvard.edu> wrote:

> Hi,
>
> I have a question related to astropy.io.fits.
> I'm wondering about the ramifications of defining a function to require
> hdus as arguments. So the user would do:
> hdu = fits.open(fitsfilename)
> result = fcn(hdu,...)
> What I wonder about is the ramifications if the hdu isn't closed. Of
> course the call could use a context manager:
> with fits.open(fitsfilename) as hdu:
>     result = fcn(hdu,...)
> which assures the hdu will be closed, though I don't think there's any way
> to enforce calling it in that way. I suppose that the worst that is likely
> to happen is that memory will be used up. I guess this is not really
> different from any function using file handles as arguments. Any thoughts?
>
> Jon
>
> --
> Jonathan D. Slavin (he/him)
> Astrophysicist - High Energy Astrophysics Division
> Center for Astrophysics | Harvard & Smithsonian
> Office: (617) 496-7981 | Cell: (781) 363-0035
> 60 Garden Street | MS 04 | Cambridge, MA 02138
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/astropy/attachments/20230105/72a3f2a7/attachment.html>

From jslavin at cfa.harvard.edu  Fri Jan  6 14:42:42 2023
From: jslavin at cfa.harvard.edu (Slavin, Jonathan)
Date: Fri, 6 Jan 2023 14:42:42 -0500
Subject: [AstroPy] passing hdus as arguments
In-Reply-To: <mailman.18.1673024402.19675.astropy@python.org>
References: <mailman.18.1673024402.19675.astropy@python.org>
Message-ID: <CACcRS=dkHLhb9OPwnrnFTW0Kmdsx8A=D3hWm0pMoHxsbVmd7gw@mail.gmail.com>

Hi Madison,

Thanks for your response. I was aware that the HDUList returned by
fits.open() can contain more than one HDU. Your discussion of the
considerations was very helpful. We're just looking to read in the data
from a file on disk at this point, so it seems like we should be okay.
Though I'm still a bit undecided whether it's better to pass in the
HDULists or the file paths.

Regards,
Jon

On Fri, Jan 6, 2023 at 12:00 PM <astropy-request at python.org> wrote:

> Message: 1
> Date: Thu, 5 Jan 2023 18:35:21 +0100
> From: "E. Madison Bray" <erik.m.bray at gmail.com>
> To: Astronomical Python mailing list <astropy at python.org>
> Subject: Re: [AstroPy] passing hdus as arguments
> Message-ID:
>         <CAOTD34ZFe9hOCeuWbORvqads7W61Zwuz0qumuV0nGeKSOD4_=
> g at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi Jon,
>
> First a quick clarification about nomenclature : what fits.open() returns
> is not an HDU (Header Data Unit, of which there can be one or more in a
> single FITS file. Rather, something that for historical reasons is called
> HDUList, but is just an object representing the whole FITS file. As you
> surmised it contains a wrapper around a file handle. Keeping that open
> doesn't use any significant memory, but reading the actual data arrays in
> the individual HDUs will of course map their data into "real" memory.
>
> As for the ramifications it depends on what your function is doing and how
> it's intended to be used. In particular if it's just reading the data (and
> not syncing updates back to the original file) I wouldn't overly worry
> about it.
>
> As long as there aren't any reference leaks, once there are no more
> references to the HDUList object it will eventually be garbage collected.
>
> Of course, if you're actually noticing some memory leaks that's worth
> bringing up. There have been some in the past IIRC.
>
>
> On Thu, Jan 5, 2023, 16:39 Slavin, Jonathan <jslavin at cfa.harvard.edu>
> wrote:
>
> > Hi,
> >
> > I have a question related to astropy.io.fits.
> > I'm wondering about the ramifications of defining a function to require
> > hdus as arguments. So the user would do:
> > hdu = fits.open(fitsfilename)
> > result = fcn(hdu,...)
> > What I wonder about is the ramifications if the hdu isn't closed. Of
> > course the call could use a context manager:
> > with fits.open(fitsfilename) as hdu:
> >     result = fcn(hdu,...)
> > which assures the hdu will be closed, though I don't think there's any
> way
> > to enforce calling it in that way. I suppose that the worst that is
> likely
> > to happen is that memory will be used up. I guess this is not really
> > different from any function using file handles as arguments. Any
> thoughts?
> >
> > Jon
> >
> > --
> > Jonathan D. Slavin (he/him)
> > Astrophysicist - High Energy Astrophysics Division
> > Center for Astrophysics | Harvard & Smithsonian
> > Office: (617) 496-7981 | Cell: (781) 363-0035
> > 60 Garden Street | MS 04 | Cambridge, MA 02138
> >
> >
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at python.org
> > https://mail.python.org/mailman/listinfo/astropy
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> https://mail.python.org/pipermail/astropy/attachments/20230105/72a3f2a7/attachment-0001.html
> >
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy
>
>
> ------------------------------
>
> End of AstroPy Digest, Vol 194, Issue 2
> ***************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/astropy/attachments/20230106/fe2c172c/attachment.html>

From thomas.robitaille at gmail.com  Mon Jan  9 05:33:28 2023
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Mon, 9 Jan 2023 10:33:28 +0000
Subject: [AstroPy] ANN: astropy 5.2 released
Message-ID: <CAGMHX_3AnKXE=MFW_qySKJtTVCgaWNeXixmTD--HpPoLEJGveA@mail.gmail.com>

Dear colleagues,

We are very happy to announce the v5.2 release of astropy, a core Python
package for Astronomy (and a v5.2.1 release which fixes compatibility with
Numpy 1.24):

    http://www.astropy.org

The astropy core package is a community-driven Python package intended to
contain much of the core functionality and common tools needed for
astronomy and astrophysics. It is part of the Astropy Project, which aims
to foster an ecosystem of interoperable astronomy packages for Python.

New and improved major functionality in this release includes:

* Quantity data types
* Updates to astropy.cosmology
* Topocentric ITRS Frame
* Enhanced Fixed Width ASCII Tables
* Accessing cloud-hosted FITS files
* Drawing the instrument beam and a physical scale bar on celestial images
Interior ticks and tick labels
* Support for tilde-prefixed paths
* CCDData PSF Image representation

In addition, hundreds of smaller improvements and fixes have been made. An
overview of the changes is provided at:

     http://docs.astropy.org/en/stable/whatsnew/5.2.html

Instructions for installing astropy are provided on our website, and
extensive documentation can be found at:

     http://docs.astropy.org

If you usually use pip/vanilla Python, you can do:

    pip install astropy --upgrade

If you make use of the Anaconda Python Distribution, soon you will be able
update to Astropy v5.2 with:

    conda update astropy

Or if you cannot wait for Anaconda to update their default version, you can
use the conda-forge channel:

     conda update -c conda-forge astropy

Please report any issues, or request new features via our GitHub repository:

     https://github.com/astropy/astropy/issues

Over 430 people have contributed code to the core astropy package so far,
and you can find out more about the team here:

     https://www.astropy.org/team.html

If you use astropy directly for your work, or as a dependency to another
package, please remember to acknowledge it by citing the appropriate
Astropy paper. For the most up-to-date suggestions, see the acknowledgement
page:

    https://www.astropy.org/acknowledging.html

We hope that you enjoy using astropy as much as we enjoyed developing it!

Thomas Robitaille
v5.2 Release Coordinator
on behalf of The Astropy Project
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/astropy/attachments/20230109/e016d56e/attachment.html>