From mcraig at mnstate.edu Mon Dec 4 14:39:11 2017 From: mcraig at mnstate.edu (Matthew Craig) Date: Mon, 4 Dec 2017 19:39:11 +0000 Subject: [AstroPy] Reminder: Complete your application for Python in Astronomy 2018 this week! Message-ID: <188DFEBF-8D41-4178-85D1-9E263672A332@mnstate.edu> Hi, This is a friendly reminder that applications for Python in Astronomy 2018 close this week. The application form is open until 23:59UTC on Dec 9, 2017. Python in Astronomy 2018 will be held 30 Apr - 4 May 2018 at the Center for Computational Astrophysics at the Flatiron Institute in New York, NY. Thanks to generous support from the Flatiron Institute, there is no conference fee, and there will be limited travel support available. The application form is at: https://goo.gl/forms/AGuoU8mUFv3WJ8Qn2 More information about the conference, and links to past years, is available at: http://openastronomy.org/pyastro/2018/ Finally, a brief excerpt from the description of the conference: In addition to sharing information about state-of-the art Python Astronomy packages, the workshop will focus on improving interoperability between astronomical Python packages, mentoring open-source contributors, and developing educational materials for Python in Astronomy. The meeting is therefore not only aimed at current developers and users, but also educators or others who are interested in being involved in these efforts. Participant selection will be made with the goal of enhancing the Python in Astronomy community and we particularly encourage requests to attend from junior astronomers and people who are new to contributing to open source software. Effort will also be made to select participants who have a demonstrated potential to contribute meaningfully to the Python in Astronomy infrastructure via providing educational materials, documentation, and/or coding. This conference is neither intended to be an introduction to Python nor only for expert-level Python developers. Thanks, Matt Craig, on behalf of the SOC: Azalee Bostroem Daniela Huppenkothen Drew Leonard Duncan Macleod Brigitta Sipocz Erik Tollerud ?? Professor Department of Physics and Astronomy Minnesota State University Moorhead 1104 7th Ave S, Moorhead MN 56563 office: Hagen 307F phone: (218) 477-2439 fax: (218) 477-2290 -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnon.sela at gmail.com Mon Dec 4 16:02:03 2017 From: arnon.sela at gmail.com (Arnon Sela) Date: Mon, 4 Dec 2017 15:02:03 -0600 Subject: [AstroPy] Nested recarrays in FITS Message-ID: Dear Whom that Can Help, I have nested numpy recarray structure to be stored into Fits. The following code is a just a test I used to build a nested structure (data_for_fits variable in the last line of the code). Code start >>>>>> import numpy as np ''' The following two functions are adapted from: adopted from https://stackoverflow.com/questions/32328889/numpy-structured-array-from-arbitrary-level-nested-dictionary ''' def mkdtype(d): ''' Creates dtype for nested dictionary with numpy based type objects ''' result = [] for k, v in d.items(): if isinstance(v,np.ndarray): result.append((k, v.dtype, v.shape)) else: result.append((k, mkdtype(v))) return np.dtype(result) def dict2recarray(data, rec=None): ''' Creates numpy.recarray from data (dict) ''' def _dict2recarray(data, rec): if rec.dtype.names: for n in rec.dtype.names: _dict2recarray(data[n], rec[n]) else: rec[:] = data return rec dtype = mkdtype(data) if rec is None: rec = np.zeros(dtype.shape, dtype) return _dict2recarray(data, rec) datan_raw = {'DATA': {'D1': np.linspace( 0, 100, 8*4,).reshape(8, 4), 'D2': np.linspace( 0, 100, 10*5, ).reshape(10, 5), 'ND': {'D1': np.linspace( 0, 100, 10*5, ).reshape(10, 5), 'D2': np.linspace( 0, 100, 8*4,).reshape(8, 4), }}} dtype = mkdtype(datan_raw) *data_for_fits* = dict2recarray(datan_raw) >>>>>> Code ends I couldn't find documentation on how to build such a FITS structure (nested recarrays). One option is to build sub-recarrays into different BIN tables with a header that would correspond to a nested key in the recarray. But that would require creating another function to reconstruct the recarray structure after reading the BIN tables from the FITS file. The better option is to build FITS is such a manner that would retrieve the structure correctly on FITS load(). Thank you for your help, Best regards. -------------- next part -------------- An HTML attachment was scrubbed... URL: From npkuin at gmail.com Mon Dec 4 16:08:04 2017 From: npkuin at gmail.com (Paul Kuin) Date: Mon, 4 Dec 2017 21:08:04 +0000 Subject: [AstroPy] Nested recarrays in FITS In-Reply-To: References: Message-ID: I think that HDF does that for you. FIts is more flexible, but you have to do your own writes and retrievals. In the end you will be reinventing the wheel unless you check out how HDF does it, That's my opinion. Cheers, Paul On Mon, Dec 4, 2017 at 9:02 PM, Arnon Sela wrote: > Dear Whom that Can Help, > > I have nested numpy recarray structure to be stored into Fits. > The following code is a just a test I used to build a nested structure > (data_for_fits variable in the last line of the code). > > Code start >>>>>> > > import numpy as np > > ''' The following two functions are adapted from: > adopted from https://stackoverflow.com/questions/32328889/numpy- > structured-array-from-arbitrary-level-nested-dictionary > ''' > > def mkdtype(d): > ''' Creates dtype for nested dictionary with numpy based type objects > ''' > result = [] > for k, v in d.items(): > if isinstance(v,np.ndarray): > result.append((k, v.dtype, v.shape)) > else: > result.append((k, mkdtype(v))) > return np.dtype(result) > > def dict2recarray(data, rec=None): > ''' Creates numpy.recarray from data (dict) > ''' > def _dict2recarray(data, rec): > if rec.dtype.names: > for n in rec.dtype.names: > _dict2recarray(data[n], rec[n]) > else: > rec[:] = data > return rec > > dtype = mkdtype(data) > if rec is None: > rec = np.zeros(dtype.shape, dtype) > > return _dict2recarray(data, rec) > > datan_raw = {'DATA': {'D1': np.linspace( 0, 100, 8*4,).reshape(8, 4), > 'D2': np.linspace( 0, 100, 10*5, ).reshape(10, 5), > 'ND': {'D1': np.linspace( 0, 100, 10*5, > ).reshape(10, 5), > 'D2': np.linspace( 0, 100, 8*4,).reshape(8, > 4), }}} > > dtype = mkdtype(datan_raw) > *data_for_fits* = dict2recarray(datan_raw) > > > >>>>>> Code ends > > I couldn't find documentation on how to build such a FITS structure > (nested recarrays). > > One option is to build sub-recarrays into different BIN tables with a > header that would correspond to a nested key in the recarray. But that > would require creating another function to reconstruct the recarray > structure after reading the BIN tables from the FITS file. > > The better option is to build FITS is such a manner that would retrieve > the structure correctly on FITS load(). > > Thank you for your help, > > Best regards. > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > > -- * * * * * * * * http://www.mssl.ucl.ac.uk/~npmk/ * * * * Dr. N.P.M. Kuin (n.kuin at ucl.ac.uk) phone +44-(0)1483 (prefix) -204111 (work) mobile +44(0)7908715953 skype ID: npkuin Mullard Space Science Laboratory ? University College London ? Holmbury St Mary ? Dorking ? Surrey RH5 6NT? U.K. -------------- next part -------------- An HTML attachment was scrubbed... URL: From teuben at astro.umd.edu Mon Dec 4 16:46:27 2017 From: teuben at astro.umd.edu (Peter Teuben) Date: Mon, 4 Dec 2017 22:46:27 +0100 Subject: [AstroPy] Nested recarrays in FITS In-Reply-To: References: Message-ID: ?another thought on this: I think the original question was also limited in not explaining why fits was needed. I could argue for pickle. Paul is right, HDF might be a better match, especially if you have to switch to another language, HDF has a more native match to that. But does it have to be persistent data? otherwise using a python-c/fortran interface is far more efficient.? (I believe HDF is actually more flexible than the F in FITS). You can't beat a native pickle: ??? ??? ??? import pickle ??? ??? ??? pickle.dump(datan_raw,open("test.dat","wb")) ??? ??? ??? .. ??? ??? ??? new_raw = pickle.load(open("test.dat", "rb")) So perhaps we could return the question and ask in what situation you need this data structure (for). - peter On 12/04/2017 10:08 PM, Paul Kuin wrote: > I think that HDF does that for you. FIts is more flexible, but you have to do your own writes and retrievals. In the end you will be reinventing the wheel unless you check out how HDF does it, That's my opinion.? > > Cheers,? > > ? ?Paul > > On Mon, Dec 4, 2017 at 9:02 PM, Arnon Sela > wrote: > > Dear Whom that Can Help, > > I have nested numpy recarray structure to be stored into Fits. > The following code is a just a test I used to build a nested structure (data_for_fits variable in the last line of the code). > > Code start >>>>>> > > import numpy as np > > ''' The following two functions are adapted from:? > adopted from https://stackoverflow.com/questions/32328889/numpy-structured-array-from-arbitrary-level-nested-dictionary > ''' > > def mkdtype(d): > ? ? ''' Creates dtype for nested dictionary with numpy based type objects > ? ? ''' > ? ? result = [] > ? ? for k, v in d.items(): > ? ? ? ? if isinstance(v,np.ndarray): > ? ? ? ? ? ? result.append((k, v.dtype, v.shape)) > ? ? ? ? else: > ? ? ? ? ? ? result.append((k, mkdtype(v))) > ? ? return np.dtype(result) > > def dict2recarray(data, rec=None): > ? ? ''' Creates numpy.recarray from data (dict) > ? ? ''' > ? ? def _dict2recarray(data, rec): > ? ? ? ? if rec.dtype.names: > ? ? ? ? ? ? for n in rec.dtype.names: > ? ? ? ? ? ? ? ? _dict2recarray(data[n], rec[n]) > ? ? ? ? else: > ? ? ? ? ? ? rec[:] = data > ? ? ? ? return rec > ? ?? > ? ? dtype = mkdtype(data) > ? ? if rec is None: > ? ? ? ? rec = np.zeros(dtype.shape, dtype) > ? ? ? ?? > ? ? return _dict2recarray(data, rec) > > datan_raw = {'DATA': {'D1': np.linspace( 0, 100, 8*4,).reshape(8, 4), > ? ? ? ? ? ? ? ? ? ? ? 'D2': np.linspace( 0, 100, 10*5, ).reshape(10, 5),? > ? ? ? ? ? ? ? ? ? ? ? 'ND': {'D1': np.linspace( 0, 100, 10*5, ).reshape(10, 5),? > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?'D2': np.linspace( 0, 100, 8*4,).reshape(8, 4), }}} > > dtype = mkdtype(datan_raw) > */data_for_fits/* = dict2recarray(datan_raw) > > > >>>>>> Code ends > > I couldn't find documentation on how to build such a FITS structure (nested recarrays). > > One option is to build sub-recarrays into different BIN tables with a header that would correspond?to a nested key?in the recarray. But that would require creating another function to reconstruct the recarray structure after reading the BIN tables from the FITS file. > > The better option is to build FITS is such a manner that would retrieve the structure correctly on FITS load(). > > Thank you for your help, > > Best regards. > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > > > > > -- > > * * * * * * * * http://www.mssl.ucl.ac.uk/~npmk/ * * * * > Dr. N.P.M. Kuin ? ? ?(n.kuin at ucl.ac.uk ) ? ? ? > phone +44-(0)1483 (prefix) -204111 (work) > mobile +44(0)7908715953 ?skype ID: npkuin > Mullard Space Science Laboratory ?? University College London ?? > Holmbury St Mary ? Dorking ? Surrey RH5 6NT? ?U.K. > > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnon.sela at gmail.com Mon Dec 4 17:29:23 2017 From: arnon.sela at gmail.com (Arnon Sela) Date: Mon, 4 Dec 2017 16:29:23 -0600 Subject: [AstroPy] Nested recarrays in FITS In-Reply-To: References: Message-ID: Thank you. It may be that HDF would be a good fit. This is part of an effort to move ROTSE data from a proprietary format into a format that is more shareable. The original thought was to use FITS as other projects use it. But it may be that we would need to move to more flexible format as HDF. Thank you so much for the help. With this, I have already worked an example with h5py. Again, thank you!!! On Mon, Dec 4, 2017 at 3:46 PM, Peter Teuben wrote: > > another thought on this: > > I think the original question was also limited in not explaining why fits > was needed. I could argue for pickle. Paul is right, HDF might be a better > match, especially if you have to switch to another language, HDF has a more > native match to that. But does it have to be persistent data? otherwise > using a python-c/fortran interface is far more efficient. (I believe HDF > is actually more flexible than the F in FITS). > > You can't beat a native pickle: > > import pickle > pickle.dump(datan_raw,open("test.dat","wb")) > .. > new_raw = pickle.load(open("test.dat", "rb")) > > So perhaps we could return the question and ask in what situation you need > this data structure (for). > > - peter > > > On 12/04/2017 10:08 PM, Paul Kuin wrote: > > I think that HDF does that for you. FIts is more flexible, but you have to > do your own writes and retrievals. In the end you will be reinventing the > wheel unless you check out how HDF does it, That's my opinion. > > Cheers, > > Paul > > On Mon, Dec 4, 2017 at 9:02 PM, Arnon Sela wrote: > >> Dear Whom that Can Help, >> >> I have nested numpy recarray structure to be stored into Fits. >> The following code is a just a test I used to build a nested structure >> (data_for_fits variable in the last line of the code). >> >> Code start >>>>>> >> >> import numpy as np >> >> ''' The following two functions are adapted from: >> adopted from https://stackoverflow.com/questions/32328889/numpy-structure >> d-array-from-arbitrary-level-nested-dictionary >> ''' >> >> def mkdtype(d): >> ''' Creates dtype for nested dictionary with numpy based type objects >> ''' >> result = [] >> for k, v in d.items(): >> if isinstance(v,np.ndarray): >> result.append((k, v.dtype, v.shape)) >> else: >> result.append((k, mkdtype(v))) >> return np.dtype(result) >> >> def dict2recarray(data, rec=None): >> ''' Creates numpy.recarray from data (dict) >> ''' >> def _dict2recarray(data, rec): >> if rec.dtype.names: >> for n in rec.dtype.names: >> _dict2recarray(data[n], rec[n]) >> else: >> rec[:] = data >> return rec >> >> dtype = mkdtype(data) >> if rec is None: >> rec = np.zeros(dtype.shape, dtype) >> >> return _dict2recarray(data, rec) >> >> datan_raw = {'DATA': {'D1': np.linspace( 0, 100, 8*4,).reshape(8, 4), >> 'D2': np.linspace( 0, 100, 10*5, ).reshape(10, 5), >> 'ND': {'D1': np.linspace( 0, 100, 10*5, >> ).reshape(10, 5), >> 'D2': np.linspace( 0, 100, 8*4,).reshape(8, >> 4), }}} >> >> dtype = mkdtype(datan_raw) >> *data_for_fits* = dict2recarray(datan_raw) >> >> >> >>>>>> Code ends >> >> I couldn't find documentation on how to build such a FITS structure >> (nested recarrays). >> >> One option is to build sub-recarrays into different BIN tables with a >> header that would correspond to a nested key in the recarray. But that >> would require creating another function to reconstruct the recarray >> structure after reading the BIN tables from the FITS file. >> >> The better option is to build FITS is such a manner that would retrieve >> the structure correctly on FITS load(). >> >> Thank you for your help, >> >> Best regards. >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at python.org >> https://mail.python.org/mailman/listinfo/astropy >> >> > > > -- > > * * * * * * * * http://www.mssl.ucl.ac.uk/~npmk/ * * * * > Dr. N.P.M. Kuin (n.kuin at ucl.ac.uk) > phone +44-(0)1483 (prefix) -204111 (work) > mobile +44(0)7908715953 <+44%207908%20715953> skype ID: npkuin > Mullard Space Science Laboratory ? University College London ? > Holmbury St Mary ? Dorking ? Surrey RH5 6NT? U.K. > > > _______________________________________________ > AstroPy mailing listAstroPy at python.orghttps://mail.python.org/mailman/listinfo/astropy > > > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ddavella at stsci.edu Mon Dec 4 18:00:27 2017 From: ddavella at stsci.edu (Daniel D'avella) Date: Mon, 4 Dec 2017 23:00:27 +0000 Subject: [AstroPy] Consider ASDF for hierarchical numpy data In-Reply-To: References: Message-ID: There are some good suggestions in this thread. If you do in fact need to serialize your data to disk and if you're not tied to FITS for other reasons, you might consider using the Advanced Scientific Data Format (ASDF) which is designed specifically for this purpose. Here's an example of how to use ASDF to store the data set you described: >>> import asdf >>> import numpy as np >>> data = {'D1': np.linspace( 0, 100, 8*4,).reshape(8, 4), 'D2': np.linspace( 0, 100, 10*5, ).reshape(10, 5), 'ND': {'D1': np.linspace( 0, 100, 10*5,).reshape(10, 5), 'D2': np.linspace( 0, 100, 8*4,).reshape(8, 4), }}} # Writing data to file on disk >>> outfile = asdf.AsdfFile(data) >>> outfile.write_to('data.asdf') # Reading data from file on disk >>> infile = asdf.open('data.asdf') >>> infile.tree {'D1': , 'D2': , 'ND': {'D1': , 'D2': }} # Data arrays can be accessed hierarchically from the top-level tree: >>> infile.tree['D1'] array([[ 0. , 3.22580645, 6.4516129 , 9.67741935], [ 12.90322581, 16.12903226, 19.35483871, 22.58064516], [ 25.80645161, 29.03225806, 32.25806452, 35.48387097], [ 38.70967742, 41.93548387, 45.16129032, 48.38709677], [ 51.61290323, 54.83870968, 58.06451613, 61.29032258], [ 64.51612903, 67.74193548, 70.96774194, 74.19354839], [ 77.41935484, 80.64516129, 83.87096774, 87.09677419], [ 90.32258065, 93.5483871 , 96.77419355, 100. ]]) >>> infile.tree['ND'] {'D1': , 'D2': } The metadata contents of the ASDF file are human-readable: #ASDF 1.0.0 #ASDF_STANDARD 1.1.0 %YAML 1.1 %TAG ! tag:stsci.edu:asdf/ --- !core/asdf-1.0.0 D1: !core/ndarray-1.0.0 source: 0 datatype: float64 byteorder: little shape: [8, 4] D2: !core/ndarray-1.0.0 source: 1 datatype: float64 byteorder: little shape: [10, 5] ND: D1: !core/ndarray-1.0.0 source: 2 datatype: float64 byteorder: little shape: [10, 5] D2: !core/ndarray-1.0.0 source: 3 datatype: float64 byteorder: little shape: [8, 4] asdf_library: !core/software-1.0.0 {author: Space Telescope Science Institute, homepage: 'http://github.com/spacetelescope/asdf', name: asdf, version: 1.3.2.dev1044} The data arrays themselves are stored efficiently, and can even be compressed. ASDF is also capable of serializing various types from Astropy including tables, Time objects, units and quantities, and some transforms and coordinates. ASDF can be installed using pip: $ pip install asdf Basic documentation can be found here: http://asdf.readthedocs.io/en/latest/ If you have any questions feel free to open an issue in our github repo: https://github.com/spacetelescope/asdf [https://avatars0.githubusercontent.com/u/2751928?s=400&v=4] GitHub - spacetelescope/asdf: ASDF (Advanced Scientific Data Format) is a next generation interchange format for scientific data github.com asdf - ASDF (Advanced Scientific Data Format) is a next generation interchange format for scientific data ________________________________ From: AstroPy on behalf of astropy-request at python.org Sent: Monday, December 4, 2017 4:51 PM To: astropy at python.org Subject: AstroPy Digest, Vol 135, Issue 2 Send AstroPy mailing list submissions to astropy at python.org To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/astropy or, via email, send a message with subject or body 'help' to astropy-request at python.org You can reach the person managing the list at astropy-owner at python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of AstroPy digest..." Today's Topics: 1. Re: Nested recarrays in FITS (Paul Kuin) 2. Re: Nested recarrays in FITS (Peter Teuben) ---------------------------------------------------------------------- Message: 1 Date: Mon, 4 Dec 2017 21:08:04 +0000 From: Paul Kuin To: Astronomical Python mailing list Cc: Daniel Sela Subject: Re: [AstroPy] Nested recarrays in FITS Message-ID: Content-Type: text/plain; charset="utf-8" I think that HDF does that for you. FIts is more flexible, but you have to do your own writes and retrievals. In the end you will be reinventing the wheel unless you check out how HDF does it, That's my opinion. Cheers, Paul On Mon, Dec 4, 2017 at 9:02 PM, Arnon Sela wrote: > Dear Whom that Can Help, > > I have nested numpy recarray structure to be stored into Fits. > The following code is a just a test I used to build a nested structure > (data_for_fits variable in the last line of the code). > > Code start >>>>>> > > import numpy as np > > ''' The following two functions are adapted from: > adopted from https://stackoverflow.com/questions/32328889/numpy- > structured-array-from-arbitrary-level-nested-dictionary > ''' > > def mkdtype(d): > ''' Creates dtype for nested dictionary with numpy based type objects > ''' > result = [] > for k, v in d.items(): > if isinstance(v,np.ndarray): > result.append((k, v.dtype, v.shape)) > else: > result.append((k, mkdtype(v))) > return np.dtype(result) > > def dict2recarray(data, rec=None): > ''' Creates numpy.recarray from data (dict) > ''' > def _dict2recarray(data, rec): > if rec.dtype.names: > for n in rec.dtype.names: > _dict2recarray(data[n], rec[n]) > else: > rec[:] = data > return rec > > dtype = mkdtype(data) > if rec is None: > rec = np.zeros(dtype.shape, dtype) > > return _dict2recarray(data, rec) > > datan_raw = {'DATA': {'D1': np.linspace( 0, 100, 8*4,).reshape(8, 4), > 'D2': np.linspace( 0, 100, 10*5, ).reshape(10, 5), > 'ND': {'D1': np.linspace( 0, 100, 10*5, > ).reshape(10, 5), > 'D2': np.linspace( 0, 100, 8*4,).reshape(8, > 4), }}} > > dtype = mkdtype(datan_raw) > *data_for_fits* = dict2recarray(datan_raw) > > > >>>>>> Code ends > > I couldn't find documentation on how to build such a FITS structure > (nested recarrays). > > One option is to build sub-recarrays into different BIN tables with a > header that would correspond to a nested key in the recarray. But that > would require creating another function to reconstruct the recarray > structure after reading the BIN tables from the FITS file. > > The better option is to build FITS is such a manner that would retrieve > the structure correctly on FITS load(). > > Thank you for your help, > > Best regards. > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > > -- * * * * * * * * http://www.mssl.ucl.ac.uk/~npmk/ * * * * Paul Kuin, Mullard Space Science Laboratory, UCL www.mssl.ucl.ac.uk Space Science, Supernovae, Novae, Gamma Ray Bursts, Solar Flares, Coronal Mass Ejections, Stellar Winds and Coronae, N. Paul M. Kuin Dr. N.P.M. Kuin (n.kuin at ucl.ac.uk) phone +44-(0)1483 (prefix) -204111 (work) mobile +44(0)7908715953 skype ID: npkuin Mullard Space Science Laboratory ? University College London ? Holmbury St Mary ? Dorking ? Surrey RH5 6NT? U.K. -------------- next part -------------- An HTML attachment was scrubbed... URL: ------------------------------ Message: 2 Date: Mon, 4 Dec 2017 22:46:27 +0100 From: Peter Teuben To: astropy at python.org Subject: Re: [AstroPy] Nested recarrays in FITS Message-ID: Content-Type: text/plain; charset="utf-8" ?another thought on this: I think the original question was also limited in not explaining why fits was needed. I could argue for pickle. Paul is right, HDF might be a better match, especially if you have to switch to another language, HDF has a more native match to that. But does it have to be persistent data? otherwise using a python-c/fortran interface is far more efficient.? (I believe HDF is actually more flexible than the F in FITS). You can't beat a native pickle: ??? ??? ??? import pickle ??? ??? ??? pickle.dump(datan_raw,open("test.dat","wb")) ??? ??? ??? .. ??? ??? ??? new_raw = pickle.load(open("test.dat", "rb")) So perhaps we could return the question and ask in what situation you need this data structure (for). - peter On 12/04/2017 10:08 PM, Paul Kuin wrote: > I think that HDF does that for you. FIts is more flexible, but you have to do your own writes and retrievals. In the end you will be reinventing the wheel unless you check out how HDF does it, That's my opinion.? > > Cheers,? > > ? ?Paul > > On Mon, Dec 4, 2017 at 9:02 PM, Arnon Sela > wrote: > > Dear Whom that Can Help, > > I have nested numpy recarray structure to be stored into Fits. > The following code is a just a test I used to build a nested structure (data_for_fits variable in the last line of the code). > > Code start >>>>>> > > import numpy as np > > ''' The following two functions are adapted from:? > adopted from https://stackoverflow.com/questions/32328889/numpy-structured-array-from-arbitrary-level-nested-dictionary > ''' > > def mkdtype(d): > ? ? ''' Creates dtype for nested dictionary with numpy based type objects > ? ? ''' > ? ? result = [] > ? ? for k, v in d.items(): > ? ? ? ? if isinstance(v,np.ndarray): > ? ? ? ? ? ? result.append((k, v.dtype, v.shape)) > ? ? ? ? else: > ? ? ? ? ? ? result.append((k, mkdtype(v))) > ? ? return np.dtype(result) > > def dict2recarray(data, rec=None): > ? ? ''' Creates numpy.recarray from data (dict) > ? ? ''' > ? ? def _dict2recarray(data, rec): > ? ? ? ? if rec.dtype.names: > ? ? ? ? ? ? for n in rec.dtype.names: > ? ? ? ? ? ? ? ? _dict2recarray(data[n], rec[n]) > ? ? ? ? else: > ? ? ? ? ? ? rec[:] = data > ? ? ? ? return rec > ? ?? > ? ? dtype = mkdtype(data) > ? ? if rec is None: > ? ? ? ? rec = np.zeros(dtype.shape, dtype) > ? ? ? ?? > ? ? return _dict2recarray(data, rec) > > datan_raw = {'DATA': {'D1': np.linspace( 0, 100, 8*4,).reshape(8, 4), > ? ? ? ? ? ? ? ? ? ? ? 'D2': np.linspace( 0, 100, 10*5, ).reshape(10, 5),? > ? ? ? ? ? ? ? ? ? ? ? 'ND': {'D1': np.linspace( 0, 100, 10*5, ).reshape(10, 5),? > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?'D2': np.linspace( 0, 100, 8*4,).reshape(8, 4), }}} > > dtype = mkdtype(datan_raw) > */data_for_fits/* = dict2recarray(datan_raw) > > > >>>>>> Code ends > > I couldn't find documentation on how to build such a FITS structure (nested recarrays). > > One option is to build sub-recarrays into different BIN tables with a header that would correspond?to a nested key?in the recarray. But that would require creating another function to reconstruct the recarray structure after reading the BIN tables from the FITS file. > > The better option is to build FITS is such a manner that would retrieve the structure correctly on FITS load(). > > Thank you for your help, > > Best regards. > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > > > > > -- > > * * * * * * * * http://www.mssl.ucl.ac.uk/~npmk/ * * * * > Dr. N.P.M. Kuin ? ? ?(n.kuin at ucl.ac.uk ) ? ? ? > phone +44-(0)1483 (prefix) -204111 (work) > mobile +44(0)7908715953 ?skype ID: npkuin > Mullard Space Science Laboratory ?? University College London ?? > Holmbury St Mary ? Dorking ? Surrey RH5 6NT? ?U.K. > > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: ------------------------------ Subject: Digest Footer _______________________________________________ AstroPy mailing list AstroPy at python.org https://mail.python.org/mailman/listinfo/astropy ------------------------------ End of AstroPy Digest, Vol 135, Issue 2 *************************************** -------------- next part -------------- An HTML attachment was scrubbed... URL: From npkuin at gmail.com Mon Dec 4 20:10:06 2017 From: npkuin at gmail.com (Paul Kuin) Date: Tue, 5 Dec 2017 01:10:06 +0000 Subject: [AstroPy] Nested recarrays in FITS In-Reply-To: References: Message-ID: What have I done? ROTSE to HDF ? I suppose there is some emotional connection I have remaining to fits, but if hdf fits better, OK. I have used HDF before, for an Earth Science project, and it is kind of nice to use. Just never needed that kind of metadata hierarchy for my astronomy stuff. Good luck! On Mon, Dec 4, 2017 at 10:29 PM, Arnon Sela wrote: > Thank you. It may be that HDF would be a good fit. > > This is part of an effort to move ROTSE data from a proprietary format > into a format that is more shareable. The original thought was to use FITS > as other projects use it. > > But it may be that we would need to move to more flexible format as HDF. > > Thank you so much for the help. With this, I have already worked an > example with h5py. > > Again, thank you!!! > > On Mon, Dec 4, 2017 at 3:46 PM, Peter Teuben wrote: > >> >> another thought on this: >> >> I think the original question was also limited in not explaining why fits >> was needed. I could argue for pickle. Paul is right, HDF might be a better >> match, especially if you have to switch to another language, HDF has a more >> native match to that. But does it have to be persistent data? otherwise >> using a python-c/fortran interface is far more efficient. (I believe HDF >> is actually more flexible than the F in FITS). >> >> You can't beat a native pickle: >> >> import pickle >> pickle.dump(datan_raw,open("test.dat","wb")) >> .. >> new_raw = pickle.load(open("test.dat", "rb")) >> >> So perhaps we could return the question and ask in what situation you >> need this data structure (for). >> >> - peter >> >> >> On 12/04/2017 10:08 PM, Paul Kuin wrote: >> >> I think that HDF does that for you. FIts is more flexible, but you have >> to do your own writes and retrievals. In the end you will be reinventing >> the wheel unless you check out how HDF does it, That's my opinion. >> >> Cheers, >> >> Paul >> >> On Mon, Dec 4, 2017 at 9:02 PM, Arnon Sela wrote: >> >>> Dear Whom that Can Help, >>> >>> I have nested numpy recarray structure to be stored into Fits. >>> The following code is a just a test I used to build a nested structure >>> (data_for_fits variable in the last line of the code). >>> >>> Code start >>>>>> >>> >>> import numpy as np >>> >>> ''' The following two functions are adapted from: >>> adopted from https://stackoverflow.com/ques >>> tions/32328889/numpy-structured-array-from-arbitrary-level-n >>> ested-dictionary >>> ''' >>> >>> def mkdtype(d): >>> ''' Creates dtype for nested dictionary with numpy based type objects >>> ''' >>> result = [] >>> for k, v in d.items(): >>> if isinstance(v,np.ndarray): >>> result.append((k, v.dtype, v.shape)) >>> else: >>> result.append((k, mkdtype(v))) >>> return np.dtype(result) >>> >>> def dict2recarray(data, rec=None): >>> ''' Creates numpy.recarray from data (dict) >>> ''' >>> def _dict2recarray(data, rec): >>> if rec.dtype.names: >>> for n in rec.dtype.names: >>> _dict2recarray(data[n], rec[n]) >>> else: >>> rec[:] = data >>> return rec >>> >>> dtype = mkdtype(data) >>> if rec is None: >>> rec = np.zeros(dtype.shape, dtype) >>> >>> return _dict2recarray(data, rec) >>> >>> datan_raw = {'DATA': {'D1': np.linspace( 0, 100, 8*4,).reshape(8, 4), >>> 'D2': np.linspace( 0, 100, 10*5, ).reshape(10, 5), >>> 'ND': {'D1': np.linspace( 0, 100, 10*5, >>> ).reshape(10, 5), >>> 'D2': np.linspace( 0, 100, 8*4,).reshape(8, >>> 4), }}} >>> >>> dtype = mkdtype(datan_raw) >>> *data_for_fits* = dict2recarray(datan_raw) >>> >>> >>> >>>>>> Code ends >>> >>> I couldn't find documentation on how to build such a FITS structure >>> (nested recarrays). >>> >>> One option is to build sub-recarrays into different BIN tables with a >>> header that would correspond to a nested key in the recarray. But that >>> would require creating another function to reconstruct the recarray >>> structure after reading the BIN tables from the FITS file. >>> >>> The better option is to build FITS is such a manner that would retrieve >>> the structure correctly on FITS load(). >>> >>> Thank you for your help, >>> >>> Best regards. >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at python.org >>> https://mail.python.org/mailman/listinfo/astropy >>> >>> >> >> >> -- >> >> * * * * * * * * http://www.mssl.ucl.ac.uk/~npmk/ * * * * >> Dr. N.P.M. Kuin (n.kuin at ucl.ac.uk) >> phone +44-(0)1483 (prefix) -204111 (work) >> mobile +44(0)7908715953 <+44%207908%20715953> skype ID: npkuin >> Mullard Space Science Laboratory ? University College London ? >> Holmbury St Mary ? Dorking ? Surrey RH5 6NT? U.K. >> >> >> _______________________________________________ >> AstroPy mailing listAstroPy at python.orghttps://mail.python.org/mailman/listinfo/astropy >> >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at python.org >> https://mail.python.org/mailman/listinfo/astropy >> >> > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > > -- * * * * * * * * http://www.mssl.ucl.ac.uk/~npmk/ * * * * Dr. N.P.M. Kuin (n.kuin at ucl.ac.uk) phone +44-(0)1483 (prefix) -204111 (work) mobile +44(0)7908715953 skype ID: npkuin Mullard Space Science Laboratory ? University College London ? Holmbury St Mary ? Dorking ? Surrey RH5 6NT? U.K. -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnon.sela at gmail.com Mon Dec 4 21:44:09 2017 From: arnon.sela at gmail.com (Arnon Sela) Date: Mon, 4 Dec 2017 20:44:09 -0600 Subject: [AstroPy] Consider ASDF for hierarchical numpy data In-Reply-To: References: Message-ID: Thank you. I do like that ADSF has text header like FITS has. >From the example you sent (thank you very much), ADSF handles hierarchical dictionary dataset. But: 1. When I tried a numpy.recarray dataset (as in the original example), it fails. I understand It would be easy to convert recarrays to dictionaries, but I was wondering if there is an inherent way to store data as recarrays. Is there an option to store recarrays directly? 2. Also, elements are loaded (ndarrays) as asdf.tags.core.ndarray.NDArrayType, is there a way to tell asdf to load as numpy.ndarrays? Thanks, On Mon, Dec 4, 2017 at 5:00 PM, Daniel D'avella wrote: > There are some good suggestions in this thread. If you do in fact need to > serialize your data to disk and if you're not tied to FITS for other > reasons, you might consider using the Advanced Scientific Data Format > (ASDF) which is designed specifically for this purpose. Here's an example > of how to use ASDF to store the data set you described: > > > >>> import asdf > > >>> import numpy as np > > > >>> data = {'D1': np.linspace( 0, 100, 8*4,).reshape(8, 4), > > 'D2': np.linspace( 0, 100, 10*5, ).reshape(10, 5), > 'ND': {'D1': np.linspace( 0, 100, 10*5,).reshape(10, 5), > > 'D2': np.linspace( 0, 100, 8*4,).reshape(8, > 4), }}} > > > # Writing data to file on disk > > >>> outfile = asdf.AsdfFile(data) > > >>> outfile.write_to('data.asdf') > > > # Reading data from file on disk > > >>> infile = asdf.open('data.asdf') > > >>> infile.tree > > {'D1': , > > 'D2': , > 'ND': {'D1': , 'D2': > }} > # Data arrays can be accessed hierarchically from the top-level tree: > >>> infile.tree['D1'] > array([[ 0. , 3.22580645, 6.4516129 , 9.67741935], > [ 12.90322581, 16.12903226, 19.35483871, 22.58064516], > [ 25.80645161, 29.03225806, 32.25806452, 35.48387097], > [ 38.70967742, 41.93548387, 45.16129032, 48.38709677], > [ 51.61290323, 54.83870968, 58.06451613, 61.29032258], > [ 64.51612903, 67.74193548, 70.96774194, 74.19354839], > [ 77.41935484, 80.64516129, 83.87096774, 87.09677419], > [ 90.32258065, 93.5483871 , 96.77419355, 100. ]]) > >>> infile.tree['ND'] > {'D1': , > 'D2': } > > The metadata contents of the ASDF file are human-readable: > > #ASDF 1.0.0 > #ASDF_STANDARD 1.1.0 > %YAML 1.1 > %TAG ! tag:stsci.edu:asdf/ > --- !core/asdf-1.0.0 > D1: !core/ndarray-1.0.0 > source: 0 > datatype: float64 > byteorder: little > shape: [8, 4] > D2: !core/ndarray-1.0.0 > source: 1 > datatype: float64 > byteorder: little > shape: [10, 5] > ND: > D1: !core/ndarray-1.0.0 > source: 2 > datatype: float64 > byteorder: little > shape: [10, 5] > D2: !core/ndarray-1.0.0 > source: 3 > datatype: float64 > byteorder: little > shape: [8, 4] > asdf_library: !core/software-1.0.0 {author: Space Telescope Science > Institute, homepage: 'http://github.com/spacetelescope/asdf', > name: asdf, version: 1.3.2.dev1044} > > The data arrays themselves are stored efficiently, and can even be > compressed. > > ASDF is also capable of serializing various types from Astropy including > tables, Time objects, units and quantities, and some transforms and > coordinates. > > ASDF can be installed using pip: > $ pip install asdf > > Basic documentation can be found here: > > http://asdf.readthedocs.io/en/latest/ > > > If you have any questions feel free to open an issue in our github repo: > > https://github.com/spacetelescope/asdf > > GitHub - spacetelescope/asdf: ASDF (Advanced Scientific Data Format) is a > next generation interchange format for scientific data > > github.com > asdf - ASDF (Advanced Scientific Data Format) is a next generation > interchange format for scientific data > > > > ------------------------------ > *From:* AstroPy on behalf > of astropy-request at python.org > *Sent:* Monday, December 4, 2017 4:51 PM > *To:* astropy at python.org > *Subject:* AstroPy Digest, Vol 135, Issue 2 > > Send AstroPy mailing list submissions to > astropy at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/astropy > or, via email, send a message with subject or body 'help' to > astropy-request at python.org > > You can reach the person managing the list at > astropy-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of AstroPy digest..." > > > Today's Topics: > > 1. Re: Nested recarrays in FITS (Paul Kuin) > 2. Re: Nested recarrays in FITS (Peter Teuben) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 4 Dec 2017 21:08:04 +0000 > From: Paul Kuin > To: Astronomical Python mailing list > Cc: Daniel Sela > Subject: Re: [AstroPy] Nested recarrays in FITS > Message-ID: > gmail.com> > Content-Type: text/plain; charset="utf-8" > > I think that HDF does that for you. FIts is more flexible, but you have to > do your own writes and retrievals. In the end you will be reinventing the > wheel unless you check out how HDF does it, That's my opinion. > > Cheers, > > Paul > > On Mon, Dec 4, 2017 at 9:02 PM, Arnon Sela wrote: > > > Dear Whom that Can Help, > > > > I have nested numpy recarray structure to be stored into Fits. > > The following code is a just a test I used to build a nested structure > > (data_for_fits variable in the last line of the code). > > > > Code start >>>>>> > > > > import numpy as np > > > > ''' The following two functions are adapted from: > > adopted from https://stackoverflow.com/questions/32328889/numpy- > > structured-array-from-arbitrary-level-nested-dictionary > > ''' > > > > def mkdtype(d): > > ''' Creates dtype for nested dictionary with numpy based type objects > > ''' > > result = [] > > for k, v in d.items(): > > if isinstance(v,np.ndarray): > > result.append((k, v.dtype, v.shape)) > > else: > > result.append((k, mkdtype(v))) > > return np.dtype(result) > > > > def dict2recarray(data, rec=None): > > ''' Creates numpy.recarray from data (dict) > > ''' > > def _dict2recarray(data, rec): > > if rec.dtype.names: > > for n in rec.dtype.names: > > _dict2recarray(data[n], rec[n]) > > else: > > rec[:] = data > > return rec > > > > dtype = mkdtype(data) > > if rec is None: > > rec = np.zeros(dtype.shape, dtype) > > > > return _dict2recarray(data, rec) > > > > datan_raw = {'DATA': {'D1': np.linspace( 0, 100, 8*4,).reshape(8, 4), > > 'D2': np.linspace( 0, 100, 10*5, ).reshape(10, 5), > > 'ND': {'D1': np.linspace( 0, 100, 10*5, > > ).reshape(10, 5), > > 'D2': np.linspace( 0, 100, 8*4,).reshape(8, > > 4), }}} > > > > dtype = mkdtype(datan_raw) > > *data_for_fits* = dict2recarray(datan_raw) > > > > > > >>>>>> Code ends > > > > I couldn't find documentation on how to build such a FITS structure > > (nested recarrays). > > > > One option is to build sub-recarrays into different BIN tables with a > > header that would correspond to a nested key in the recarray. But that > > would require creating another function to reconstruct the recarray > > structure after reading the BIN tables from the FITS file. > > > > The better option is to build FITS is such a manner that would retrieve > > the structure correctly on FITS load(). > > > > Thank you for your help, > > > > Best regards. > > > > _______________________________________________ > > AstroPy mailing list > > AstroPy at python.org > > https://mail.python.org/mailman/listinfo/astropy > > > > > > > -- > > * * * * * * * * http://www.mssl.ucl.ac.uk/~npmk/ * * * * > Paul Kuin, Mullard Space Science Laboratory, UCL > > www.mssl.ucl.ac.uk > Space Science, Supernovae, Novae, Gamma Ray Bursts, Solar Flares, Coronal > Mass Ejections, Stellar Winds and Coronae, N. Paul M. Kuin > > Dr. N.P.M. Kuin (n.kuin at ucl.ac.uk) > phone +44-(0)1483 (prefix) -204111 (work) > mobile +44(0)7908715953 <+44%207908%20715953> skype ID: npkuin > Mullard Space Science Laboratory ? University College London ? > Holmbury St Mary ? Dorking ? Surrey RH5 6NT? U.K. > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: 20171204/8577ee9c/attachment-0001.html> > > ------------------------------ > > Message: 2 > Date: Mon, 4 Dec 2017 22:46:27 +0100 > From: Peter Teuben > To: astropy at python.org > Subject: Re: [AstroPy] Nested recarrays in FITS > Message-ID: > Content-Type: text/plain; charset="utf-8" > > > ?another thought on this: > > I think the original question was also limited in not explaining why fits > was needed. I could argue for pickle. Paul is right, HDF might be a better > match, especially if you have to switch to another language, HDF has a more > native match to that. But does it have to be persistent data? otherwise > using a python-c/fortran interface is far more efficient.? (I believe HDF > is actually more flexible than the F in FITS). > > You can't beat a native pickle: > > ??? ??? ??? import pickle > ??? ??? ??? pickle.dump(datan_raw,open("test.dat","wb")) > ??? ??? ??? .. > ??? ??? ??? new_raw = pickle.load(open("test.dat", "rb")) > > So perhaps we could return the question and ask in what situation you need > this data structure (for). > > - peter > > On 12/04/2017 10:08 PM, Paul Kuin wrote: > > I think that HDF does that for you. FIts is more flexible, but you have > to do your own writes and retrievals. In the end you will be reinventing > the wheel unless you check out how HDF does it, That's my opinion.? > > > > Cheers,? > > > > ? ?Paul > > > > On Mon, Dec 4, 2017 at 9:02 PM, Arnon Sela mailto:arnon.sela at gmail.com >> wrote: > > > > Dear Whom that Can Help, > > > > I have nested numpy recarray structure to be stored into Fits. > > The following code is a just a test I used to build a nested > structure (data_for_fits variable in the last line of the code). > > > > Code start >>>>>> > > > > import numpy as np > > > > ''' The following two functions are adapted from:? > > adopted from https://stackoverflow.com/questions/32328889/numpy- > structured-array-from-arbitrary-level-nested-dictionary < > https://stackoverflow.com/questions/32328889/numpy-structured-array-from- > arbitrary-level-nested-dictionary> > > ''' > > > > def mkdtype(d): > > ? ? ''' Creates dtype for nested dictionary with numpy based > type objects > > ? ? ''' > > ? ? result = [] > > ? ? for k, v in d.items(): > > ? ? ? ? if isinstance(v,np.ndarray): > > ? ? ? ? ? ? result.append((k, v.dtype, v.shape)) > > ? ? ? ? else: > > ? ? ? ? ? ? result.append((k, mkdtype(v))) > > ? ? return np.dtype(result) > > > > def dict2recarray(data, rec=None): > > ? ? ''' Creates numpy.recarray from data (dict) > > ? ? ''' > > ? ? def _dict2recarray(data, rec): > > ? ? ? ? if rec.dtype.names: > > ? ? ? ? ? ? for n in rec.dtype.names: > > ? ? ? ? ? ? ? ? _dict2recarray(data[n], rec[n]) > > ? ? ? ? else: > > ? ? ? ? ? ? rec[:] = data > > ? ? ? ? return rec > > ? ?? > > ? ? dtype = mkdtype(data) > > ? ? if rec is None: > > ? ? ? ? rec = np.zeros(dtype.shape, dtype) > > ? ? ? ?? > > ? ? return _dict2recarray(data, rec) > > > > datan_raw = {'DATA': {'D1': np.linspace( 0, 100, > 8*4,).reshape(8, 4), > > ? ? ? ? ? ? ? ? ? ? ? 'D2': np.linspace( 0, 100, 10*5, > ).reshape(10, 5),? > > ? ? ? ? ? ? ? ? ? ? ? 'ND': {'D1': np.linspace( 0, 100, 10*5, > ).reshape(10, 5),? > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?'D2': np.linspace( 0, 100, > 8*4,).reshape(8, 4), }}} > > > > dtype = mkdtype(datan_raw) > > */data_for_fits/* = dict2recarray(datan_raw) > > > > > > >>>>>> Code ends > > > > I couldn't find documentation on how to build such a FITS structure > (nested recarrays). > > > > One option is to build sub-recarrays into different BIN tables with > a header that would correspond?to a nested key?in the recarray. But that > would require creating another function to reconstruct the recarray > structure after reading the BIN tables from the FITS file. > > > > The better option is to build FITS is such a manner that would > retrieve the structure correctly on FITS load(). > > > > Thank you for your help, > > > > Best regards. > > > > _______________________________________________ > > AstroPy mailing list > > AstroPy at python.org > > > https://mail.python.org/mailman/listinfo/astropy < > https://mail.python.org/mailman/listinfo/astropy> > > > > > > > > > > -- > > > > * * * * * * * * http://www.mssl.ucl.ac.uk/~npmk/ < > http://www.mssl.ucl.ac.uk/%7Enpmk/> * * * * > > Dr. N.P.M. Kuin ? ? ?(n.kuin at ucl.ac.uk >) ? ? ? > > phone +44-(0)1483 (prefix) -204111 (work) > > mobile +44(0)7908715953 <+44%207908%20715953> ?skype ID: npkuin > > Mullard Space Science Laboratory ?? University College London ?? > > Holmbury St Mary ? Dorking ? Surrey RH5 6NT? ?U.K. > > > > > > _______________________________________________ > > AstroPy mailing list > > AstroPy at python.org > > https://mail.python.org/mailman/listinfo/astropy > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: 20171204/1e4458bb/attachment.html> > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > > > ------------------------------ > > End of AstroPy Digest, Vol 135, Issue 2 > *************************************** > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mcraig at mnstate.edu Wed Dec 13 21:20:09 2017 From: mcraig at mnstate.edu (Matthew Craig) Date: Thu, 14 Dec 2017 02:20:09 +0000 Subject: [AstroPy] Brief re-opening of Python in Astronomy 2018 applications Message-ID: <8018DFB5-3E5E-4D63-A871-030D48CD49EF@mnstate.edu> Hi, It came to the attention of the Python in Astronomy 2018 SOC that there were several people who received permission from their workplace to apply only after applications closed. In the interests of ensuring the broadest possible applicant pool, we will be re-opening the application form at 03:00UTC on Dec 14, 2018 and closing them again at roughly 15:00UTC on Dec 16 (i.e. it'll be open Thursday and Friday). We'll also ask future SOCs to emphasize a message we think is important to the vitality of the astronomical python community: If you want to attend, but are not sure you will be able to (because of a need for funding, permission to travel, a visa,...) please apply! If it turns out you cannot attend, you can withdraw your application and your spot will simply go to someone on the waitlist. Thanks, Matt Craig, on behalf of the SOC Azalee Bostroem Daniela Huppenkothen Drew Leonard Duncan Macleod Brigitta Sipocz Erik Tollerud -------------- next part -------------- An HTML attachment was scrubbed... URL: From hjyanghj at gmail.com Fri Dec 15 12:20:45 2017 From: hjyanghj at gmail.com (Yang Hon-Jang) Date: Sat, 16 Dec 2017 01:20:45 +0800 Subject: [AstroPy] what does it mean get_body('moon',t,location=loc)? Message-ID: I am a real beginner in astronomy. It seems relation between get_body('moon',t,location=loc) and get_body('moon',t) is not a translation. So it raise a stupid question in my mind, what does it mean get_body('moon',t,location=loc)? Following is my snippet: import astropy.units as u from astropy.coordinates import get_body,EarthLocation from astropy.time import Time import numpy as np t=Time('2017-12-16') moon=get_body('moon',t) loc=EarthLocation(x=6000,y=6000,z=6000,unit='km') moon2=get_body('moon',t,location=loc) print(np.array(loc.value)) print(np.array(moon2.cartesian.xyz.value)) print(np.array(moon.cartesian.xyz.value)) # wonder why np.array(moon.cartesian.xyz.value)-(np.array(moon2.cartesian.xyz.value)+np.array(loc.value)) is not zero? np.array(moon.cartesian.xyz.value)-(np.array(moon2.cartesian.xyz.value)+np.array(loc.value)) Hope some one can guide me, Thanks! From bsipocz at gmail.com Wed Dec 20 17:32:22 2017 From: bsipocz at gmail.com (Brigitta Sipocz) Date: Wed, 20 Dec 2017 22:32:22 +0000 Subject: [AstroPy] ANN: astropy bugfix release v2.0.3 Message-ID: Dear all, The latest astropy bugfix version has been released, and now available either on PyPI or on the 'default' or 'conda-forge' conda channels. Among other fixes, it adds backward compatibility for the standalone pytest-astropy plugins. Maintainers using the astropy testing framework may find this useful if they plan to support both astropy LTS and 3.0+ in their packages. The full list of fixes can be found in the changelog: https://github.com/astropy/astropy/blob/v2.0.3/CHANGES.rst Thank you for everyone who contributed for this release! Cheers, Brigitta -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.robitaille at gmail.com Thu Dec 21 06:27:01 2017 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Thu, 21 Dec 2017 12:27:01 +0100 Subject: [AstroPy] ANN: Release of pywwt v0.3.0 Message-ID: Hi everyone, We are pleased to announce the v0.3.0 release of pywwt, a Python package that makes it easy to use/control WorldWide Telescope from Python and works on Linux, MacOS X, and Windows. This is a major release of the package that includes the ability to embed the web version of WorldWide Telescope as a widget inside Jupyter notebooks, and also includes a way to use WorldWide Telescope in a standalone Qt-based viewer and inside Qt applications. WorldWide Telescope (http://worldwidetelescope.org/home) is a free and powerful viewer currently developed by the American Astronomical Society that can display astronomical and planetary data. The two main versions of WorldWide Telescope are a web client, which works across all platforms, and a Windows desktop application. WorldWide Telescope is designed to be useful to a wide audience, including researchers, educators, and the general public. The documentation for pywwt can be found at: http://pywwt.readthedocs.io If you want to try out pywwt inside a Jupyter notebook without installing anything, you can try it out online ? see: http://pywwt.readthedocs.io/en/latest/#quick-start for more details. You can also install it using conda: conda install -c wwt pywwt or pip: pip install pywwt jupyter nbextension enable --py --sys-prefix pywwt This package is at an early stage of development, and we would appreciate feedback - in particular feel free to report issues or request features using the GitHub issue tracker: https://github.com/WorldWideTelescope/pywwt/issues Please don't hesitate to let us know if you have any questions! Thomas Robitaille, O. Justin Otor, John ZuHone, and Phil Rosenfield From hjyanghj at gmail.com Fri Dec 22 19:37:16 2017 From: hjyanghj at gmail.com (Yang Hon-Jang) Date: Sat, 23 Dec 2017 08:37:16 +0800 Subject: [AstroPy] What should I know in advance to understand get_body(planet, time, location)? Message-ID: As far as I know get_body('mars',time,location) return GCRS coordinate of mars observed at location on earth. So difference vector between get_body('mars',time,loc1) and get_body('mars',time,loc2) should be the same as differce vector between loc1 and doc2. Therefore difference vector between get_body('mars',time,loc1) and get_body('mars',time,loc2) should be the same as difference vector between get_body('jupiter',time,loc1) and get_body('jupiter',time,loc2). But use following snippet, from astropy.coordinates import > EarthLocation,solar_system_ephemeris,get_body > from astropy.time import Time > import astropy.units as u > planets=solar_system_ephemeris.bodies > t=Time('2017-1-7') > loc1=EarthLocation.of_site('ALMA') > loc2=EarthLocation.of_site('BAO') > with solar_system_ephemeris.set('jpl'): > poses1=[] > poses2=[] > for i in range(len(planets)): > poses1.append(get_body(planets[i],t,location=loc1).cartesian.xyz) > poses2.append(get_body(planets[i],t,location=loc2).cartesian.xyz) > print(poses2[i]-poses1[i]) get result: [ 8063.95816843 7068.00850269 -6606.53423256] km > [ 8224.04320343 7065.45624797 -6482.55365168] km > [ 8064.42921932 7067.72318247 -6607.26674529] km > [ 8266.19750362 7036.04863887 -6534.63230393] km > [ 8025.04757395 6956.33442213 -6558.59139917] km > [ 8064.08233562 7068.37313613 -6606.78203702] km > [ 7979.54002401 6710.05480098 -6540.78743029] km > [ 8766.62041795 5208.77094686 -6699.57502227] km > [ 11237.56838802 6188.62185216 -5658.59566617] km > [ 10447.75206614 304.28817022 -6981.0367673 ] km > [ 6385.49008512 1293.04231977 -5061.7454344 ] km The result is a list of very different vectors, it confused me very much. Hope some one can help me? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: