From aldcroft at head.cfa.harvard.edu Thu Mar 2 14:06:18 2017 From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas) Date: Thu, 2 Mar 2017 14:06:18 -0500 Subject: [AstroPy] OpenAstronomy in Google Summer of Code 2017 Message-ID: On behalf of the OpenAstronomy organization I'm pleased to announce that we have again been accepted as a mentoring organization in the Google Summer of Code program. Thus far we have potential project ideas from Astropy, SunPy, and Glue. *Students* If you or someone you know is interested in participating in GSoC with OpenAstronomy, please visit the following links to get started: GSoC OpenAstronomy page OpenAstronomy student guidelines OpenAstronomy project ideas *More project ideas* It is late, but not impossibly late, to add ideas for GSoC projects for this year. If you'd like to do so please add an issue to https://github.com/OpenAstronomy/openastronomy.github.io and the admins there will take it from there. Note that one should definitely have at least two committed mentors in mind for any project. Cheers, Tom A, on behalf of OpenAstronomy -------------- next part -------------- An HTML attachment was scrubbed... URL: From s.littlefair at sheffield.ac.uk Thu Mar 2 18:01:29 2017 From: s.littlefair at sheffield.ac.uk (Stuart P Littlefair) Date: Thu, 2 Mar 2017 23:01:29 +0000 Subject: [AstroPy] super-high-speed parsing of large FITS files Message-ID: Perhaps not the right place to ask this, but the official FITS mailing lists are overrun with spam. We are choosing a file format for a high speed photometer. One proposal is a MEF file. Frame rates are kHz, and there are five detectors so each FITS file may have many thousands of HDUs. One of our requirements is to be able to reduce the data in real time, whilst the data is being written. This means retrieving a given HDU from such a file in a msec or less. It would be OK if finding an initial HDU corresponding to a given frame number is slower than this, if we can grab *subsequent* blocks of 5 HDUs (1 per detector) in < 1 msec. So I am wondering: a) is this even possible in principle, with low-level C-code? b) Can it be done via some un-sanctioned use of the private functions in the astropy FITs library? Many thanks in advance to anyone who can help -- Stuart Littlefair ------------------------------------------------------- Dept. of Physics & Astronomy, Univ. of Sheffield, Sheffield, S3 7RH. email: S.Littlefair at sheffield.ac.uk phone: +44 114 2224525 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dkirkby at uci.edu Thu Mar 2 18:25:17 2017 From: dkirkby at uci.edu (David Kirkby) Date: Thu, 02 Mar 2017 23:25:17 +0000 Subject: [AstroPy] super-high-speed parsing of large FITS files In-Reply-To: References: Message-ID: Hi Stuart, I was trying to use astropy.io.fits to read and write files with 45K HDUs a while ago and eventually gave up and used the fitsio package, which was much faster in this case. You can see the relevant thread in the astropy mailing list at https://mail.scipy.org/pipermail/astropy/2015-January/thread.html David On Thu, Mar 2, 2017 at 3:07 PM Stuart P Littlefair < s.littlefair at sheffield.ac.uk> wrote: > Perhaps not the right place to ask this, but the official FITS mailing > lists are overrun with spam. > > We are choosing a file format for a high speed photometer. One proposal is > a MEF file. Frame rates are kHz, and there are five detectors so each FITS > file may have many thousands of HDUs. > > One of our requirements is to be able to reduce the data in real time, > whilst the data is being written. This means retrieving a given HDU from > such a file in a msec or less. It would be OK if finding an initial HDU > corresponding to a given frame number is slower than this, if we can grab > *subsequent* blocks of 5 HDUs (1 per detector) in < 1 msec. > > So I am wondering: > > a) is this even possible in principle, with low-level C-code? > b) Can it be done via some un-sanctioned use of the private functions in > the astropy FITs library? > > Many thanks in advance to anyone who can help > > -- > Stuart Littlefair > > ------------------------------------------------------- > Dept. of Physics & Astronomy, > Univ. of Sheffield, Sheffield, S3 7RH. > > email: S.Littlefair at sheffield.ac.uk > phone: +44 114 2224525 <+44%20114%20222%204525> > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > https://mail.scipy.org/mailman/listinfo/astropy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From phirst at gemini.edu Thu Mar 2 18:36:17 2017 From: phirst at gemini.edu (Paul Hirst) Date: Thu, 2 Mar 2017 13:36:17 -1000 Subject: [AstroPy] super-high-speed parsing of large FITS files In-Reply-To: References: Message-ID: Hi Stuart, A couple of quick thoughts: I'd consider storing the data as a cube per detector with time as the third axis, so one FITS extension per detector rather than one per 2D frame. Then you can just mmap the cube (or potentially chunks of it in the time axis) and I think that would be about as efficient as it gets. The disadvantage of that is that you can't just append data to the file as it comes in, you'd need to know a-priori how much data was going to arrive so that you could allocate the appropriate size blocks of data for each extension. Or you could go with a separate file for each detector which seems less elegant, but solves that problem. Just a point of sanity check here - I don't know how many pixels you have, but it sounds like it would require some thought be put into the I/O and storage hardware, especially if you want to read it back to a data processing machine while it is being written. If you process it on the same machine as you are writing it from, you can probably arrange for the kernel filesystem cache to help out significantly with that, especially if you mmap the data. Regards, Paul On Thu, Mar 2, 2017 at 1:01 PM, Stuart P Littlefair < s.littlefair at sheffield.ac.uk> wrote: > Perhaps not the right place to ask this, but the official FITS mailing > lists are overrun with spam. > > We are choosing a file format for a high speed photometer. One proposal is > a MEF file. Frame rates are kHz, and there are five detectors so each FITS > file may have many thousands of HDUs. > > One of our requirements is to be able to reduce the data in real time, > whilst the data is being written. This means retrieving a given HDU from > such a file in a msec or less. It would be OK if finding an initial HDU > corresponding to a given frame number is slower than this, if we can grab > *subsequent* blocks of 5 HDUs (1 per detector) in < 1 msec. > > So I am wondering: > > a) is this even possible in principle, with low-level C-code? > b) Can it be done via some un-sanctioned use of the private functions in > the astropy FITs library? > > Many thanks in advance to anyone who can help > > -- > Stuart Littlefair > > ------------------------------------------------------- > Dept. of Physics & Astronomy, > Univ. of Sheffield, Sheffield, S3 7RH. > > email: S.Littlefair at sheffield.ac.uk > phone: +44 114 2224525 <+44%20114%20222%204525> > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > https://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From s.littlefair at sheffield.ac.uk Thu Mar 2 18:49:58 2017 From: s.littlefair at sheffield.ac.uk (Stuart P Littlefair) Date: Thu, 2 Mar 2017 23:49:58 +0000 Subject: [AstroPy] super-high-speed parsing of large FITS files In-Reply-To: References: Message-ID: Thanks for the tips Paul, David; fitsio is certainly adequately fast for grabbing a given extension once an initial HDU has been read in. However, the high-level APIs don't cope well with finding HDUs that have been inserted after the initial read. Maybe it's possible if I dig into the library a bit more... Paul - unfortunately we won't know a-priori how many frames will be taken. At these frame rates we are only writing ~50 000 pixels, so I/O and storage are not a serious issue. For similar instruments we have used a custom binary blob as a data format with no issues. However in this case the person in charge of writing the DAS wants to use a MEF file, largely because this is easiest for them. On 2 March 2017 at 23:36, Paul Hirst wrote: > Hi Stuart, > > A couple of quick thoughts: > > I'd consider storing the data as a cube per detector with time as the > third axis, so one FITS extension per detector rather than one per 2D > frame. Then you can just mmap the cube (or potentially chunks of it in the > time axis) and I think that would be about as efficient as it gets. > > The disadvantage of that is that you can't just append data to the file as > it comes in, you'd need to know a-priori how much data was going to arrive > so that you could allocate the appropriate size blocks of data for each > extension. Or you could go with a separate file for each detector which > seems less elegant, but solves that problem. > > Just a point of sanity check here - I don't know how many pixels you have, > but it sounds like it would require some thought be put into the I/O and > storage hardware, especially if you want to read it back to a data > processing machine while it is being written. If you process it on the same > machine as you are writing it from, you can probably arrange for the kernel > filesystem cache to help out significantly with that, especially if you > mmap the data. > > Regards, > Paul > > > On Thu, Mar 2, 2017 at 1:01 PM, Stuart P Littlefair < > s.littlefair at sheffield.ac.uk> wrote: > >> Perhaps not the right place to ask this, but the official FITS mailing >> lists are overrun with spam. >> >> We are choosing a file format for a high speed photometer. One proposal >> is a MEF file. Frame rates are kHz, and there are five detectors so each >> FITS file may have many thousands of HDUs. >> >> One of our requirements is to be able to reduce the data in real time, >> whilst the data is being written. This means retrieving a given HDU from >> such a file in a msec or less. It would be OK if finding an initial HDU >> corresponding to a given frame number is slower than this, if we can grab >> *subsequent* blocks of 5 HDUs (1 per detector) in < 1 msec. >> >> So I am wondering: >> >> a) is this even possible in principle, with low-level C-code? >> b) Can it be done via some un-sanctioned use of the private functions in >> the astropy FITs library? >> >> Many thanks in advance to anyone who can help >> >> -- >> Stuart Littlefair >> >> ------------------------------------------------------- >> Dept. of Physics & Astronomy, >> Univ. of Sheffield, Sheffield, S3 7RH. >> >> email: S.Littlefair at sheffield.ac.uk >> phone: +44 114 2224525 <+44%20114%20222%204525> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> https://mail.scipy.org/mailman/listinfo/astropy >> >> > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > https://mail.scipy.org/mailman/listinfo/astropy > > -- Stuart Littlefair ------------------------------------------------------- Dept. of Physics & Astronomy, Univ. of Sheffield, Sheffield, S3 7RH. email: S.Littlefair at sheffield.ac.uk phone: +44 114 2224525 -------------- next part -------------- An HTML attachment was scrubbed... URL: From derek at astro.physik.uni-goettingen.de Thu Mar 2 18:59:05 2017 From: derek at astro.physik.uni-goettingen.de (Derek Homeier) Date: Fri, 3 Mar 2017 00:59:05 +0100 Subject: [AstroPy] super-high-speed parsing of large FITS files In-Reply-To: References: Message-ID: <11F8588F-7009-4D1A-A5E3-F7B2C953D811@astro.physik.uni-goettingen.de> Hi Stuart, > > We are choosing a file format for a high speed photometer. One proposal is a MEF file. Frame rates are kHz, and there are five detectors so each FITS file may have many thousands of HDUs. > > One of our requirements is to be able to reduce the data in real time, whilst the data is being written. This means retrieving a given HDU from such a file in a msec or less. It would be OK if finding an initial HDU corresponding to a given frame number is slower than this, if we can grab *subsequent* blocks of 5 HDUs (1 per detector) in < 1 msec. > > So I am wondering: > > a) is this even possible in principle, with low-level C-code? > b) Can it be done via some un-sanctioned use of the private functions in the astropy FITs library? > if I understood right that you are not fully settled on using FITS as a format at all at this point, it might be worth looking into HDF5 instead. You may find some ideas about potential performance and optimisation possibilities in the PyTables documentation: http://www.pytables.org/usersguide/optimization.html Unfortunately not all of these (in particular the Blosc compression library, afaik) are available with the h5py library used in astropy, but perhaps writing a direct interface to pytables could still be an option. Cheers, Derek From nathan12343 at gmail.com Thu Mar 2 19:50:17 2017 From: nathan12343 at gmail.com (Nathan Goldbaum) Date: Fri, 03 Mar 2017 00:50:17 +0000 Subject: [AstroPy] super-high-speed parsing of large FITS files In-Reply-To: <11F8588F-7009-4D1A-A5E3-F7B2C953D811@astro.physik.uni-goettingen.de> References: <11F8588F-7009-4D1A-A5E3-F7B2C953D811@astro.physik.uni-goettingen.de> Message-ID: See https://github.com/h5py/h5py/issues/611 On Thu, Mar 2, 2017 at 5:59 PM Derek Homeier < derek at astro.physik.uni-goettingen.de> wrote: Hi Stuart, > > We are choosing a file format for a high speed photometer. One proposal is a MEF file. Frame rates are kHz, and there are five detectors so each FITS file may have many thousands of HDUs. > > One of our requirements is to be able to reduce the data in real time, whilst the data is being written. This means retrieving a given HDU from such a file in a msec or less. It would be OK if finding an initial HDU corresponding to a given frame number is slower than this, if we can grab *subsequent* blocks of 5 HDUs (1 per detector) in < 1 msec. > > So I am wondering: > > a) is this even possible in principle, with low-level C-code? > b) Can it be done via some un-sanctioned use of the private functions in the astropy FITs library? > if I understood right that you are not fully settled on using FITS as a format at all at this point, it might be worth looking into HDF5 instead. You may find some ideas about potential performance and optimisation possibilities in the PyTables documentation: http://www.pytables.org/usersguide/optimization.html Unfortunately not all of these (in particular the Blosc compression library, afaik) are available with the h5py library used in astropy, but perhaps writing a direct interface to pytables could still be an option. Cheers, Derek _______________________________________________ AstroPy mailing list AstroPy at scipy.org https://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From fitz at noao.edu Sat Mar 4 03:54:51 2017 From: fitz at noao.edu (Mike Fitzpatrick) Date: Sat, 4 Mar 2017 01:54:51 -0700 Subject: [AstroPy] super-high-speed parsing of large FITS files In-Reply-To: References: Message-ID: I had to build a similar system a few years back (a 4K IR Mosaic camera w/ burst speeds of 5-10 hz) and faced some of the same issues. Your question presumes that the data coming from the instrument have already been serialized into some format which adds overhead to the readout speed; in our solution we simply collected the pixels and metadata as fast as possible with a separate process and stuffed them into shared memory pages on the processing machine. 'Processing' here meant flip/flopping the arrays due to the amplifier locations in the corners but also meant an (optional) real-time display of the image to the console, the creation of the science image (that produced computed header values, assembled MEF extensions, etc) was slower and left to a separate process working at a different rate than the image readout by churning through the shared memory pages and so offloading some of the speed requirement. The shared memory pages allowed the processing and science-image serialization to run in parallel and at rates different than the instrument readout. For your 50K pixels at khz rates you're looking at ~100MB/s so saving these to memory is reasonable for short-ish periods, or longer if you can buffer/cycle appropriately. Since you're not building a generalized task for reading extensions at msec rates you can also take advantage of knowing what the file structure will be for a particular sequence, i.e. if you do use MEF and have a fixed focal plane you will always known that frame is some fixed multiple of a 2880-byte offset in the file. Whether this is a file i/o offset or a memory offset of a shared/mmap image doesn't matter and you can decide whether the memory image is a series of pixel rasters or a structured format like MEF and how best to handle the metadata. You shouldn't limit yourself only to what's possible using exclusively python/astropy if speed is your primary requirement. It's perfectly reasonable to separate the instrument readout as low-level C code, data processing, and science-image creation as multiple/parallel processes in differing environments if these can also be partitioned into separate time-domain requirements where each domain has a best-solution. I can send you a reference paper for our system in you're interested, good luck on the project. -Mike On Thu, Mar 2, 2017 at 4:01 PM, Stuart P Littlefair < s.littlefair at sheffield.ac.uk> wrote: > Perhaps not the right place to ask this, but the official FITS mailing > lists are overrun with spam. > > We are choosing a file format for a high speed photometer. One proposal is > a MEF file. Frame rates are kHz, and there are five detectors so each FITS > file may have many thousands of HDUs. > > One of our requirements is to be able to reduce the data in real time, > whilst the data is being written. This means retrieving a given HDU from > such a file in a msec or less. It would be OK if finding an initial HDU > corresponding to a given frame number is slower than this, if we can grab > *subsequent* blocks of 5 HDUs (1 per detector) in < 1 msec. > > So I am wondering: > > a) is this even possible in principle, with low-level C-code? > b) Can it be done via some un-sanctioned use of the private functions in > the astropy FITs library? > > Many thanks in advance to anyone who can help > > -- > Stuart Littlefair > > ------------------------------------------------------- > Dept. of Physics & Astronomy, > Univ. of Sheffield, Sheffield, S3 7RH. > > email: S.Littlefair at sheffield.ac.uk > phone: +44 114 2224525 <+44%20114%20222%204525> > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > https://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erin.sheldon at gmail.com Sat Mar 4 10:07:35 2017 From: erin.sheldon at gmail.com (Erin Sheldon) Date: Sat, 4 Mar 2017 10:07:35 -0500 Subject: [AstroPy] super-high-speed parsing of large FITS files In-Reply-To: References: Message-ID: On 3/2/17, Stuart P Littlefair wrote: > Thanks for the tips Paul, David; > > fitsio is certainly adequately fast for grabbing a given extension once an > initial HDU has been read in. However, the high-level APIs don't cope well > with finding HDUs that have been inserted after the initial read. Maybe > it's possible if I dig into the library a bit more... Hi Stuart - Can you please open an issue for fitsio concerning this poor performance when adding new extensions? https://github.com/esheldon/fitsio thanks, -e -- Erin Scott Sheldon Brookhaven National Laboratory erin dot sheldon at gmail dot com From hessman at astro.physik.uni-goettingen.de Wed Mar 8 03:26:05 2017 From: hessman at astro.physik.uni-goettingen.de (Frederic V. Hessman) Date: Wed, 8 Mar 2017 09:26:05 +0100 Subject: [AstroPy] TIme.now() problem Message-ID: I?ve probably got a mental block after reading lots of documentation: how does one handle Time.now() and Time.sidereal_time() ? The examples given are always things like from astropy.time import Time from astropy.coordinates import EarthLocation loc = EarthLocation.of_site ('McDonald Observatory?) t = Time ('2016-03-08T12:34:56.7', format='isot', location=loc) lst = t.sidereal_time (?mean?) but if I want to use the current time, the logical thing to do would be tnow = Time(None,location=loc).now() # USE None BECAUSE I HAVE TO USE SOMETHING BUT DON?T KNOW THE TIME but this doesn?t work: Traceback (most recent call last): tnow = Time(None,location=loc).now() File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/astropy/time/core.py", line 253, in __init__ precision, in_subfmt, out_subfmt) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/astropy/time/core.py", line 303, in _init_from_vals precision, in_subfmt, out_subfmt) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/astropy/time/core.py", line 347, in _get_time_fmt raise ValueError('Input values did not match {0}'.format(err_msg)) ValueError: Input values did not match any of the formats where the format keyword is optional ['datetime', 'iso', 'isot', 'yday', 'fits', 'byear_str', 'jyear_str?] and it doesn?t help to give a format. Ok, so Time() wants some concrete, if dummy value/object. If I simply use the old t to create a tnow tnow = t.now() things are fine but when I use it to get the LST, lstnow = tnow.sideral_time (?mean?) I get a location error: Traceback (most recent call last): lstnow = tnow.sidereal_time ('mean') File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/astropy/time/core.py", line 715, in sidereal_time raise ValueError('No longitude is given but the location for ' ValueError: No longitude is given but the location for the Time object is not set. which is wrong, since t had the location and tnow was derived directly from t. Changing things to tnow = Time(t,location=loc).now() doesn?t help and something like tnow = t.now (location=loc) which would have passed on the location to the dumb time object isn?t supported. Any hints? python 3.4.5, astropy 1.3, OSX Rick From s.littlefair at sheffield.ac.uk Wed Mar 8 03:34:37 2017 From: s.littlefair at sheffield.ac.uk (Stuart P Littlefair) Date: Wed, 8 Mar 2017 08:34:37 +0000 Subject: [AstroPy] TIme.now() problem In-Reply-To: References: Message-ID: <1723036912291344020@unknownmsgid> Hi Rick You've two options; either pass the longitude to the sidereal_time call, e.g from astropy.time import Time from astropy import units as u now = Time.now() now.sidereal_time('mean', longitude=20*u.deg) Or you can add the location post-hoc, e.g now.location = loc now.sidereal_time('mean') Stuart Littlefair Dept. of Physics & Astronomy Univ. of Sheffield, Sheffield, S3 7RH email: s.littlefair at shef.ac.uk Phone: +44 114 2224525 Sent from my iPhone > On 8 Mar 2017, at 08:26, Frederic V. Hessman wrote: > > I?ve probably got a mental block after reading lots of documentation: how does one handle Time.now() and Time.sidereal_time() ? > > The examples given are always things like > > from astropy.time import Time > from astropy.coordinates import EarthLocation > loc = EarthLocation.of_site ('McDonald Observatory?) > t = Time ('2016-03-08T12:34:56.7', format='isot', location=loc) > lst = t.sidereal_time (?mean?) > > but if I want to use the current time, the logical thing to do would be > > tnow = Time(None,location=loc).now() # USE None BECAUSE I HAVE TO USE SOMETHING BUT DON?T KNOW THE TIME > > but this doesn?t work: > > Traceback (most recent call last): > tnow = Time(None,location=loc).now() > File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/astropy/time/core.py", line 253, in __init__ > precision, in_subfmt, out_subfmt) > File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/astropy/time/core.py", line 303, in _init_from_vals > precision, in_subfmt, out_subfmt) > File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/astropy/time/core.py", line 347, in _get_time_fmt > raise ValueError('Input values did not match {0}'.format(err_msg)) > ValueError: Input values did not match any of the formats where the format keyword is optional ['datetime', 'iso', 'isot', 'yday', 'fits', 'byear_str', 'jyear_str?] > > and it doesn?t help to give a format. Ok, so Time() wants some concrete, if dummy value/object. If I simply use the old t to create a tnow > > tnow = t.now() > > things are fine but when I use it to get the LST, > > lstnow = tnow.sideral_time (?mean?) > > I get a location error: > > Traceback (most recent call last): > lstnow = tnow.sidereal_time ('mean') > File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/astropy/time/core.py", line 715, in sidereal_time > raise ValueError('No longitude is given but the location for ' > ValueError: No longitude is given but the location for the Time object is not set. > > which is wrong, since t had the location and tnow was derived directly from t. Changing things to > > tnow = Time(t,location=loc).now() > > doesn?t help and something like > > tnow = t.now (location=loc) > > which would have passed on the location to the dumb time object isn?t supported. > > Any hints? python 3.4.5, astropy 1.3, OSX > > Rick > > > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > https://mail.scipy.org/mailman/listinfo/astropy From jakevdp at cs.washington.edu Thu Mar 9 10:15:09 2017 From: jakevdp at cs.washington.edu (Jacob Vanderplas) Date: Thu, 9 Mar 2017 07:15:09 -0800 Subject: [AstroPy] Astronomy symposium at SciPy 2017: talk proposals due March 27 Message-ID: Dear AstroPy users & developers, The 2017 *Scientific Computing with Python* conference (SciPy; July 10-16, 2017 in Austin TX) will include an Astronomy & Astrophysics symposium, with the theme *Open-Sourcing Astronomy*. Python has emerged in recent years as an important research tool in the astronomy and astrophysics community, and this symposium is a chance for researchers to share and discuss their Python-driven astronomy work. Do you have research results that have made use of Python?s open source tools? A new Python-driven method or open source package to share? A short tutorial on how to get the most out of an existing Python tool? We would love to hear from you. Talk proposals are due by March 27th; there is some financial aid available, particularly for students, with applications due April 21st. More information is available at http://scipy2017.scipy.org. Please forward this to anybody you think might be interested, and we hope to see you this summer! Pauline Barmby & Jake Vanderplas, co-chairs -------------- next part -------------- An HTML attachment was scrubbed... URL: From derek at astro.physik.uni-goettingen.de Thu Mar 9 12:25:56 2017 From: derek at astro.physik.uni-goettingen.de (Derek Homeier) Date: Thu, 9 Mar 2017 18:25:56 +0100 Subject: [AstroPy] super-high-speed parsing of large FITS files In-Reply-To: References: <11F8588F-7009-4D1A-A5E3-F7B2C953D811@astro.physik.uni-goettingen.de> Message-ID: <501AD208-70B9-4532-A688-561C8C0A24CE@astro.physik.uni-goettingen.de> On 3 Mar 2017, at 1:50 am, Nathan Goldbaum wrote: > > See https://github.com/h5py/h5py/issues/611 > Thanks for the link, that?s very useful information! I had been looking into the loadable-filter functionality of HDF5, but never pursued it for lack of time. Would be great if Blosc support finally could be implemented at the h5py level after all. Derek > > On Thu, Mar 2, 2017 at 5:59 PM Derek Homeier wrote: > Hi Stuart, > > > > We are choosing a file format for a high speed photometer. One proposal is a MEF file. Frame rates are kHz, and there are five detectors so each FITS file may have many thousands of HDUs. > > > > One of our requirements is to be able to reduce the data in real time, whilst the data is being written. This means retrieving a given HDU from such a file in a msec or less. It would be OK if finding an initial HDU corresponding to a given frame number is slower than this, if we can grab *subsequent* blocks of 5 HDUs (1 per detector) in < 1 msec. > > > > So I am wondering: > > > > a) is this even possible in principle, with low-level C-code? > > b) Can it be done via some un-sanctioned use of the private functions in the astropy FITs library? > > > if I understood right that you are not fully settled on using FITS as a format at all at this point, it might be > worth looking into HDF5 instead. You may find some ideas about potential performance and optimisation > possibilities in the PyTables documentation: > http://www.pytables.org/usersguide/optimization.html > > Unfortunately not all of these (in particular the Blosc compression library, afaik) are available with the > h5py library used in astropy, but perhaps writing a direct interface to pytables could still be an option. From s.littlefair at sheffield.ac.uk Mon Mar 13 06:09:44 2017 From: s.littlefair at sheffield.ac.uk (Stuart P Littlefair) Date: Mon, 13 Mar 2017 10:09:44 +0000 Subject: [AstroPy] super-high-speed parsing of large FITS files In-Reply-To: <501AD208-70B9-4532-A688-561C8C0A24CE@astro.physik.uni-goettingen.de> References: <11F8588F-7009-4D1A-A5E3-F7B2C953D811@astro.physik.uni-goettingen.de> <501AD208-70B9-4532-A688-561C8C0A24CE@astro.physik.uni-goettingen.de> Message-ID: Over on the GitHub repo for FITSIO, Erin Sheldon has shown that update_hdu_list is performant for large fits files as long as the rebuild=False option is enabled On 9 March 2017 at 17:25, Derek Homeier < derek at astro.physik.uni-goettingen.de> wrote: > On 3 Mar 2017, at 1:50 am, Nathan Goldbaum wrote: > > > > See https://github.com/h5py/h5py/issues/611 > > > Thanks for the link, that?s very useful information! I had been looking > into the loadable-filter > functionality of HDF5, but never pursued it for lack of time. Would be > great if Blosc support > finally could be implemented at the h5py level after all. > > Derek > > > > On Thu, Mar 2, 2017 at 5:59 PM Derek Homeier goettingen.de> wrote: > > Hi Stuart, > > > > > > We are choosing a file format for a high speed photometer. One > proposal is a MEF file. Frame rates are kHz, and there are five detectors > so each FITS file may have many thousands of HDUs. > > > > > > One of our requirements is to be able to reduce the data in real time, > whilst the data is being written. This means retrieving a given HDU from > such a file in a msec or less. It would be OK if finding an initial HDU > corresponding to a given frame number is slower than this, if we can grab > *subsequent* blocks of 5 HDUs (1 per detector) in < 1 msec. > > > > > > So I am wondering: > > > > > > a) is this even possible in principle, with low-level C-code? > > > b) Can it be done via some un-sanctioned use of the private functions > in the astropy FITs library? > > > > > if I understood right that you are not fully settled on using FITS as a > format at all at this point, it might be > > worth looking into HDF5 instead. You may find some ideas about potential > performance and optimisation > > possibilities in the PyTables documentation: > > http://www.pytables.org/usersguide/optimization.html > > > > Unfortunately not all of these (in particular the Blosc compression > library, afaik) are available with the > > h5py library used in astropy, but perhaps writing a direct interface to > pytables could still be an option. > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > https://mail.scipy.org/mailman/listinfo/astropy > -- Stuart Littlefair ------------------------------------------------------- Dept. of Physics & Astronomy, Univ. of Sheffield, Sheffield, S3 7RH. email: S.Littlefair at sheffield.ac.uk phone: +44 114 2224525 -------------- next part -------------- An HTML attachment was scrubbed... URL: From stuart at cadair.com Tue Mar 14 10:48:57 2017 From: stuart at cadair.com (Stuart Mumford) Date: Tue, 14 Mar 2017 14:48:57 +0000 Subject: [AstroPy] NAM 2017: Python in Astronomy Session Message-ID: <5fe655d5-1791-4a55-5d8b-2d133616fc04@cadair.com> Hello all, I wanted to bring to your attention that at the UK National Astronomy Meeting (NAM 2017) there will be a Python in Astronomy session, for which abstract submission is now open. We are soliciting abstracts that showcase the Python tools used to accomplish your scientific objectives as well as talks on projects that might be of interest to the wide astronomy community. Further information on the conference and abstract submission can be found on the conference website: https://nam2017.org/ Thanks, Stuart Mumford From p3y1i4n at gmail.com Fri Mar 17 22:27:07 2017 From: p3y1i4n at gmail.com (Pey Lian Lim) Date: Fri, 17 Mar 2017 22:27:07 -0400 Subject: [AstroPy] Moving Cone Search from Astropy to Astroquery Message-ID: This is going to happen. Speak now or forever hold your peace. Ideally, I would like to start issuing deprecation warnings in Astropy v2 and remove it completely in Astropy v3. And hopefully will be in Astroquery 0.3.6. https://github.com/astropy/astroquery/pull/859 -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsipocz at gmail.com Sun Mar 19 13:53:32 2017 From: bsipocz at gmail.com (Brigitta Sipocz) Date: Sun, 19 Mar 2017 17:53:32 +0000 Subject: [AstroPy] ANN: Astropy bugfix v1.3.1 and v1.0.12 released Message-ID: Dear all, I'm happy to announce that Astropy v1.3.1, a bugfix to the latest version, was released yesterday. It is available either on PyPI or on the default and conda-forge conda channels. The latest release on the Long Term Support branch is v1.0.12. There were 60+ fixes or improvements in these releases, you can read the full list in the changelog: https://github.com/astropy/astropy/blob/v1.3.1/CHANGES.rst Thanks everyone who contributed to these releases! Cheers, Brigitta -------------- next part -------------- An HTML attachment was scrubbed... URL: From kellecruz at gmail.com Mon Mar 20 13:13:07 2017 From: kellecruz at gmail.com (Kelle Cruz) Date: Mon, 20 Mar 2017 13:13:07 -0400 Subject: [AstroPy] Migrating to Google Groups Message-ID: Hi Everyone, The Astropy Coordination Committee is in the process of migrating this group from Mailman to Google Groups. Shortly, I?ll send around an invite link. The intention is for nothing to change about the group except the address and management options. Any topic related to using Python for Astronomy is welcome and need not be relevant to the ?Astropy Project.? We plan for the migration to be complete in two weeks and this address will be deactivated on March 31. The archive will stay up indefinitely but no new messages will be accepted after that date. We?ll send another reminder next week. We would like to take this opportunity to remind everyone that this group pre-dates the Astropy Project and indeed, was the inspiration for the project name and not the other way around. Thanks so much for making this an amazing community! Kelle Astropy Community Engagement Coordinator -- Kelle Cruz, PhD ? http://kellecruz.com/ 917.725.1334 ? Hunter: x16486 ? AMNH: x3404 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan.zolotukhin at gmail.com Mon Mar 20 17:28:29 2017 From: ivan.zolotukhin at gmail.com (Ivan Zolotukhin) Date: Mon, 20 Mar 2017 22:28:29 +0100 Subject: [AstroPy] invalidate download cache items Message-ID: Hi, In some of my web applications I use astropy's fits.open(url) calls which transparently cache downloaded files providing significant speed up of the service for repeated calls. However, I'd like to have more fine grained control over this cache. In particular, this cache assumes that the fetched URL content never changes which is not the case for my setup (it happens though rarely). I just had to drop few tens of thousands cached files because I could not find the right one which changed, and this causes measurable slow down of the web app I'd like to avoid in future. Implementing own conditional cache=False logics does not seem worth it. Is there any advice on how can one invalidate some cached URLs? -- With best regards, Ivan From stuart at cadair.com Tue Mar 21 06:53:24 2017 From: stuart at cadair.com (Stuart Mumford) Date: Tue, 21 Mar 2017 10:53:24 +0000 Subject: [AstroPy] invalidate download cache items In-Reply-To: References: Message-ID: <5dc1606b-813a-d4ea-5ca6-92a34cb2c68e@cadair.com> Just a quick note to say I ran into this the other week, so would like to see a cache invalidation function as well. Stuart On 20/03/17 21:28, Ivan Zolotukhin wrote: > Hi, > > In some of my web applications I use astropy's fits.open(url) calls > which transparently cache downloaded files providing significant speed > up of the service for repeated calls. However, I'd like to have more > fine grained control over this cache. In particular, this cache > assumes that the fetched URL content never changes which is not the > case for my setup (it happens though rarely). I just had to drop few > tens of thousands cached files because I could not find the right one > which changed, and this causes measurable slow down of the web app I'd > like to avoid in future. Implementing own conditional cache=False > logics does not seem worth it. > > Is there any advice on how can one invalidate some cached URLs? > > -- > With best regards, > Ivan > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > https://mail.scipy.org/mailman/listinfo/astropy From simon at sconseil.fr Wed Mar 22 05:46:10 2017 From: simon at sconseil.fr (Simon Conseil) Date: Wed, 22 Mar 2017 10:46:10 +0100 Subject: [AstroPy] Migrating to Google Groups In-Reply-To: References: Message-ID: <732d05a6-8ce3-cb8d-5b66-0ee58271f03f@sconseil.fr> Hi, On related news, a migration of scipy.org mailing lists to python.org is planned "likely within a day or two" (and including Astropy): https://mail.scipy.org/pipermail/numpy-discussion/2017-March/076613.html Simon Le 20/03/2017 ? 18:13, Kelle Cruz a ?crit : > Hi Everyone, > > The Astropy Coordination Committee is in the process of migrating > this group from Mailman to Google Groups. Shortly, I?ll send around > an invite link. The intention is for nothing to change about the > group except the address and management options. Any topic related to > using Python for Astronomy is welcome and need not be relevant to the > ?Astropy Project.? > > We plan for the migration to be complete in two weeks and this > address will be deactivated on March 31. The archive will stay up > indefinitely but no new messages will be accepted after that date. > We?ll send another reminder next week. > > We would like to take this opportunity to remind everyone that this > group pre-dates the Astropy Project and indeed, was the inspiration > for the project name and not the other way around. > > Thanks so much for making this an amazing community! > > Kelle Astropy Community Engagement Coordinator > > -- Kelle Cruz, PhD ? http://kellecruz.com/ 917.725.1334 ? Hunter: > x16486 ? AMNH: x3404 > > > _______________________________________________ AstroPy mailing list > AstroPy at scipy.org https://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From erwin at mpe.mpg.de Wed Mar 22 06:13:06 2017 From: erwin at mpe.mpg.de (Peter Erwin) Date: Wed, 22 Mar 2017 11:13:06 +0100 Subject: [AstroPy] Migrating to Google Groups In-Reply-To: <732d05a6-8ce3-cb8d-5b66-0ee58271f03f@sconseil.fr> References: <732d05a6-8ce3-cb8d-5b66-0ee58271f03f@sconseil.fr> Message-ID: So is the AstroPy mailing list being migrated to Google Groups, or to python.org? Those aren?t, as far as I know, the same thing. ? Peter > On Mar 22, 2017, at 10:46 AM, Simon Conseil wrote: > > Hi, > > On related news, a migration of scipy.org mailing lists to python.org is planned "likely within a day or two" (and including Astropy): > https://mail.scipy.org/pipermail/numpy-discussion/2017-March/076613.html > > Simon > > > Le 20/03/2017 ? 18:13, Kelle Cruz a ?crit : > > Hi Everyone, > > > > The Astropy Coordination Committee is in the process of migrating > > this group from Mailman to Google Groups. Shortly, I?ll send around > > an invite link. The intention is for nothing to change about the > > group except the address and management options. Any topic related to > > using Python for Astronomy is welcome and need not be relevant to the > > ?Astropy Project.? > > > > We plan for the migration to be complete in two weeks and this > > address will be deactivated on March 31. The archive will stay up > > indefinitely but no new messages will be accepted after that date. > > We?ll send another reminder next week. > > > > We would like to take this opportunity to remind everyone that this > > group pre-dates the Astropy Project and indeed, was the inspiration > > for the project name and not the other way around. > > > > Thanks so much for making this an amazing community! > > > > Kelle Astropy Community Engagement Coordinator > > > > -- Kelle Cruz, PhD ? http://kellecruz.com/ 917.725.1334 ? Hunter: > > x16486 ? AMNH: x3404 > > > > > > _______________________________________________ AstroPy mailing list > > AstroPy at scipy.org https://mail.scipy.org/mailman/listinfo/astropy > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > https://mail.scipy.org/mailman/listinfo/astropy ============================================================= Peter Erwin Max-Planck-Insitute for Extraterrestrial erwin at mpe.mpg.de Physics, Giessenbachstrasse tel. +49 (0)176 2481 7713 85748 Garching, Germany fax +49 (0)89 30000 3495 http://www.mpe.mpg.de/~erwin From hessman at astro.physik.uni-goettingen.de Wed Mar 22 08:53:34 2017 From: hessman at astro.physik.uni-goettingen.de (Frederic V. Hessman) Date: Wed, 22 Mar 2017 13:53:34 +0100 Subject: [AstroPy] Migrating to Google Groups Message-ID: <6473A5A9-1A41-4661-A79C-7A57EA86DEDE@astro.physik.uni-goettingen.de> I understand why many would like to migrate, but this does have the undesirable side-effect that subscribers will be forced to have a Google account rather than simply receiving and responding to emails. I have an account, but I choose to ignore it as much as possible on principle. Rick From thomas.robitaille at gmail.com Wed Mar 22 10:09:21 2017 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Wed, 22 Mar 2017 14:09:21 +0000 Subject: [AstroPy] Migrating to Google Groups In-Reply-To: <6473A5A9-1A41-4661-A79C-7A57EA86DEDE@astro.physik.uni-goettingen.de> References: <6473A5A9-1A41-4661-A79C-7A57EA86DEDE@astro.physik.uni-goettingen.de> Message-ID: Hi Rick, Just to be clear, there is no requirement to have a Google account to join a Google group mailing list. Cheers, Tom On 22 March 2017 at 12:53, Frederic V. Hessman wrote: > I understand why many would like to migrate, but this does have the > undesirable side-effect that subscribers will be forced to have a Google > account rather than simply receiving and responding to emails. I have an > account, but I choose to ignore it as much as possible on principle. > > Rick > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > https://mail.scipy.org/mailman/listinfo/astropy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dkirkby at uci.edu Wed Mar 22 12:17:05 2017 From: dkirkby at uci.edu (David Kirkby) Date: Wed, 22 Mar 2017 16:17:05 +0000 Subject: [AstroPy] Migrating to Google Groups In-Reply-To: References: <6473A5A9-1A41-4661-A79C-7A57EA86DEDE@astro.physik.uni-goettingen.de> Message-ID: I have a google account but subscribe to the astropy list with a different email account. As a result, clicking on the "Accept this invitation" link I received fails with "You are using a different account than the intended recipient of the invitation." Perhaps send an email to the list letting them know they can sign themselves up with a different account by visiting the group page directly (although this does seem to require a google account): https://groups.google.com/forum/astropy David On Wed, Mar 22, 2017 at 7:12 AM Thomas Robitaille < thomas.robitaille at gmail.com> wrote: Hi Rick, Just to be clear, there is no requirement to have a Google account to join a Google group mailing list. Cheers, Tom On 22 March 2017 at 12:53, Frederic V. Hessman < hessman at astro.physik.uni-goettingen.de> wrote: I understand why many would like to migrate, but this does have the undesirable side-effect that subscribers will be forced to have a Google account rather than simply receiving and responding to emails. I have an account, but I choose to ignore it as much as possible on principle. Rick _______________________________________________ AstroPy mailing list AstroPy at scipy.org https://mail.scipy.org/mailman/listinfo/astropy _______________________________________________ AstroPy mailing list AstroPy at scipy.org https://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.robitaille at gmail.com Wed Mar 22 14:19:08 2017 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Wed, 22 Mar 2017 18:19:08 +0000 Subject: [AstroPy] Migrating to Google Groups In-Reply-To: References: Message-ID: Hi everyone, Just to answer some concerns from people who have asked about whether a Google account, the answer is no - you can join the new mailing list as a normal email mailing list by sending an email to astropy+subscribe at googlegroups.com, then replying to the email that you receive back to confirm subscription. I've added these instructions to the welcome page for the group. Let me know (off-list) if you have any issues with joining using this method! Cheers, Tom On 20 March 2017 at 17:13, Kelle Cruz wrote: > Hi Everyone, > > The Astropy Coordination Committee is in the process of migrating this > group from Mailman to Google Groups. Shortly, I?ll send around an invite > link. The intention is for nothing to change about the group except the > address and management options. Any topic related to using Python for > Astronomy is welcome and need not be relevant to the ?Astropy Project.? > > We plan for the migration to be complete in two weeks and this address > will be deactivated on March 31. The archive will stay up indefinitely but > no new messages will be accepted after that date. We?ll send another > reminder next week. > > We would like to take this opportunity to remind everyone that this group > pre-dates the Astropy Project and indeed, was the inspiration for the > project name and not the other way around. > > Thanks so much for making this an amazing community! > > Kelle > Astropy Community Engagement Coordinator > > -- > Kelle Cruz, PhD ? http://kellecruz.com/ > 917.725.1334 <(917)%20725-1334> ? Hunter: x16486 ? AMNH: x3404 > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > https://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pweilbacher at aip.de Thu Mar 23 04:51:25 2017 From: pweilbacher at aip.de (Peter Weilbacher) Date: Thu, 23 Mar 2017 09:51:25 +0100 (CET) Subject: [AstroPy] Migrating to Google Groups In-Reply-To: References: Message-ID: So you are actually asking everyone to resubscribe? That was not clear in the message. Why are you not using the same non-Google system that apparently the rest of scipy/numpy switches to (see announcement by Ralf Gommers, )? I understand that then you could just migrate all subscribers directly. Peter. On Wed, 22 Mar 2017, Thomas Robitaille wrote: > Hi everyone, > > Just to answer some concerns from people who have asked about whether a > Google account, the answer is no - you can join the new mailing list as a > normal email mailing list by sending an email to > astropy+subscribe at googlegroups.com, then replying to the email that you > receive back to confirm subscription. I've added these instructions to the > welcome page for the group. > > Let me know (off-list) if you have any issues with joining using this > method! > > Cheers, > Tom > > > > On 20 March 2017 at 17:13, Kelle Cruz wrote: > > > Hi Everyone, > > > > The Astropy Coordination Committee is in the process of migrating this > > group from Mailman to Google Groups. Shortly, I?ll send around an invite > > link. The intention is for nothing to change about the group except the > > address and management options. Any topic related to using Python for > > Astronomy is welcome and need not be relevant to the ?Astropy Project.? > > > > We plan for the migration to be complete in two weeks and this address > > will be deactivated on March 31. The archive will stay up indefinitely but > > no new messages will be accepted after that date. We?ll send another > > reminder next week. > > > > We would like to take this opportunity to remind everyone that this group > > pre-dates the Astropy Project and indeed, was the inspiration for the > > project name and not the other way around. > > > > Thanks so much for making this an amazing community! > > > > Kelle > > Astropy Community Engagement Coordinator -- Dr. Peter M. Weilbacher http://www.aip.de/People/PWeilbacher Phone +49 331 74 99-667 encryption key ID 7D6B4AA0 ------------------------------------------------------------------------ Leibniz-Institut f?r Astrophysik Potsdam (AIP) An der Sternwarte 16, D-14482 Potsdam Vorstand: Prof. Dr. Matthias Steinmetz, Matthias Winker Stiftung b?rgerlichen Rechts, Stiftungsverz. Brandenburg: 26 742-00/7026 From kellecruz at gmail.com Thu Mar 23 14:46:51 2017 From: kellecruz at gmail.com (Kelle Cruz) Date: Thu, 23 Mar 2017 14:46:51 -0400 Subject: [AstroPy] Migrating to Google Groups In-Reply-To: References: Message-ID: Hello Everyone, My sincere apologies: The migration to Google Groups is "paused". We are going to look into the migration of the lists to python.org and re-evaluate. Thanks to Simon for bringing that key piece of info to our attention! Kelle On Thu, Mar 23, 2017 at 4:51 AM, Peter Weilbacher wrote: > So you are actually asking everyone to resubscribe? That was not clear > in the message. > > Why are you not using the same non-Google system that apparently the > rest of scipy/numpy switches to (see announcement by Ralf Gommers, > )? > I understand that then you could just migrate all subscribers directly. > > Peter. > > On Wed, 22 Mar 2017, Thomas Robitaille wrote: > > > Hi everyone, > > > > Just to answer some concerns from people who have asked about whether a > > Google account, the answer is no - you can join the new mailing list as a > > normal email mailing list by sending an email to > > astropy+subscribe at googlegroups.com, then replying to the email that you > > receive back to confirm subscription. I've added these instructions to > the > > welcome page for the group. > > > > Let me know (off-list) if you have any issues with joining using this > > method! > > > > Cheers, > > Tom > > > > > > > > On 20 March 2017 at 17:13, Kelle Cruz wrote: > > > > > Hi Everyone, > > > > > > The Astropy Coordination Committee is in the process of migrating this > > > group from Mailman to Google Groups. Shortly, I?ll send around an > invite > > > link. The intention is for nothing to change about the group except the > > > address and management options. Any topic related to using Python for > > > Astronomy is welcome and need not be relevant to the ?Astropy Project.? > > > > > > We plan for the migration to be complete in two weeks and this address > > > will be deactivated on March 31. The archive will stay up indefinitely > but > > > no new messages will be accepted after that date. We?ll send another > > > reminder next week. > > > > > > We would like to take this opportunity to remind everyone that this > group > > > pre-dates the Astropy Project and indeed, was the inspiration for the > > > project name and not the other way around. > > > > > > Thanks so much for making this an amazing community! > > > > > > Kelle > > > Astropy Community Engagement Coordinator > > -- > Dr. Peter M. Weilbacher http://www.aip.de/People/PWeilbacher > Phone +49 331 74 99-667 encryption key ID 7D6B4AA0 > ------------------------------------------------------------------------ > Leibniz-Institut f?r Astrophysik Potsdam (AIP) > An der Sternwarte 16, D-14482 Potsdam > > Vorstand: Prof. Dr. Matthias Steinmetz, Matthias Winker > Stiftung b?rgerlichen Rechts, Stiftungsverz. Brandenburg: 26 742-00/7026 > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > https://mail.scipy.org/mailman/listinfo/astropy > -- Kelle Cruz, PhD ? http://kellecruz.com/ 917.725.1334 ? Hunter: x16486 ? AMNH: x3404 -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon at sconseil.fr Fri Mar 24 05:26:55 2017 From: simon at sconseil.fr (Simon Conseil) Date: Fri, 24 Mar 2017 10:26:55 +0100 Subject: [AstroPy] Migrating to Google Groups In-Reply-To: References: Message-ID: <41f72a18-3545-41e1-1477-9e56545b1b34@sconseil.fr> Hi, The mailing lists moved (https://mail.python.org/pipermail/numpy-discussion/2017-March/076592.html). So the new address is now astropy at python.org (https://mail.python.org/mailman/listinfo/astropy) Simon On 03/23/17 19:46, Kelle Cruz wrote: > Hello Everyone, > > My sincere apologies: The migration to Google Groups is "paused". We > are going to look into the migration of the lists to python.org > and re-evaluate. > > Thanks to Simon for bringing that key piece of info to our attention! > > Kelle > > On Thu, Mar 23, 2017 at 4:51 AM, Peter Weilbacher > wrote: > > So you are actually asking everyone to resubscribe? That was not clear > > in the message. > > > > Why are you not using the same non-Google system that apparently the > > rest of scipy/numpy switches to (see announcement by Ralf Gommers, > > >)? > > I understand that then you could just migrate all subscribers > directly. > > > > Peter. > > > > On Wed, 22 Mar 2017, Thomas Robitaille wrote: > > > > > Hi everyone, > > > > > > Just to answer some concerns from people who have asked about > whether a > > > Google account, the answer is no - you can join the new mailing > list as a > > > normal email mailing list by sending an email to > > > astropy+subscribe at googlegroups.com > , then replying to > the email that you > > > receive back to confirm subscription. I've added these > instructions to the > > > welcome page for the group. > > > > > > Let me know (off-list) if you have any issues with joining using > this > > > method! > > > > > > Cheers, > > > Tom > > > > > > > > > > > > On 20 March 2017 at 17:13, Kelle Cruz > wrote: > > > > > > > Hi Everyone, > > > > > > > > The Astropy Coordination Committee is in the process of > migrating this > > > > group from Mailman to Google Groups. Shortly, I?ll send around > an invite > > > > link. The intention is for nothing to change about the group > except the > > > > address and management options. Any topic related to using > Python for > > > > Astronomy is welcome and need not be relevant to the ?Astropy > Project.? > > > > > > > > We plan for the migration to be complete in two weeks and this > address > > > > will be deactivated on March 31. The archive will stay up > indefinitely but > > > > no new messages will be accepted after that date. We?ll send > another > > > > reminder next week. > > > > > > > > We would like to take this opportunity to remind everyone that > this group > > > > pre-dates the Astropy Project and indeed, was the inspiration > for the > > > > project name and not the other way around. > > > > > > > > Thanks so much for making this an amazing community! > > > > > > > > Kelle > > > > Astropy Community Engagement Coordinator > > > > -- > > Dr. Peter M. Weilbacher http://www.aip.de/People/PWeilbacher > > > Phone +49 331 74 99-667 > encryption key ID 7D6B4AA0 > > ------------------------------------------------------------------------ > > Leibniz-Institut f?r Astrophysik Potsdam (AIP) > > An der Sternwarte 16, D-14482 Potsdam > > > > Vorstand: Prof. Dr. Matthias Steinmetz, Matthias Winker > > Stiftung b?rgerlichen Rechts, Stiftungsverz. Brandenburg: 26 > 742-00/7026 > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > https://mail.scipy.org/mailman/listinfo/astropy > > > > > > -- > Kelle Cruz, PhD ? http://kellecruz.com/ > 917.725.1334 ? Hunter: x16486 ? AMNH: x3404 > > > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > https://mail.scipy.org/mailman/listinfo/astropy > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gabrielperren at gmail.com Fri Mar 24 09:33:26 2017 From: gabrielperren at gmail.com (Gabriel Perren) Date: Fri, 24 Mar 2017 13:33:26 +0000 Subject: [AstroPy] Migrating to Google Groups In-Reply-To: <41f72a18-3545-41e1-1477-9e56545b1b34@sconseil.fr> References: <41f72a18-3545-41e1-1477-9e56545b1b34@sconseil.fr> Message-ID: So, just to be clear: everyone needs to re-suscribe to the mailing list here https://mail.python.org/mailman/listinfo/astropy? El vie., 24 de mar. de 2017 a la(s) 06:27, Simon Conseil escribi?: > Hi, > > The mailing lists moved ( > https://mail.python.org/pipermail/numpy-discussion/2017-March/076592.html > ). > > So the new address is now astropy at python.org ( > https://mail.python.org/mailman/listinfo/astropy) > Simon > > > > On 03/23/17 19:46, Kelle Cruz wrote: > > Hello Everyone, > > My sincere apologies: The migration to Google Groups is "paused". We are > going to look into the migration of the lists to python.org and > re-evaluate. > > Thanks to Simon for bringing that key piece of info to our attention! > > Kelle > > On Thu, Mar 23, 2017 at 4:51 AM, Peter Weilbacher > wrote: > > So you are actually asking everyone to resubscribe? That was not clear > > in the message. > > > > Why are you not using the same non-Google system that apparently the > > rest of scipy/numpy switches to (see announcement by Ralf Gommers, > > )? > > I understand that then you could just migrate all subscribers directly. > > > > Peter. > > > > On Wed, 22 Mar 2017, Thomas Robitaille wrote: > > > > > Hi everyone, > > > > > > Just to answer some concerns from people who have asked about whether a > > > Google account, the answer is no - you can join the new mailing list as a > > > normal email mailing list by sending an email to > > > astropy+subscribe at googlegroups.com, then replying to the email that you > > > receive back to confirm subscription. I've added these instructions to > the > > > welcome page for the group. > > > > > > Let me know (off-list) if you have any issues with joining using this > > > method! > > > > > > Cheers, > > > Tom > > > > > > > > > > > > On 20 March 2017 at 17:13, Kelle Cruz wrote: > > > > > > > Hi Everyone, > > > > > > > > The Astropy Coordination Committee is in the process of migrating this > > > > group from Mailman to Google Groups. Shortly, I?ll send around an > invite > > > > link. The intention is for nothing to change about the group except the > > > > address and management options. Any topic related to using Python for > > > > Astronomy is welcome and need not be relevant to the ?Astropy Project.? > > > > > > > > We plan for the migration to be complete in two weeks and this address > > > > will be deactivated on March 31. The archive will stay up indefinitely > but > > > > no new messages will be accepted after that date. We?ll send another > > > > reminder next week. > > > > > > > > We would like to take this opportunity to remind everyone that this > group > > > > pre-dates the Astropy Project and indeed, was the inspiration for the > > > > project name and not the other way around. > > > > > > > > Thanks so much for making this an amazing community! > > > > > > > > Kelle > > > > Astropy Community Engagement Coordinator > > > > -- > > Dr. Peter M. Weilbacher http://www.aip.de/People/PWeilbacher > > Phone +49 331 74 99-667 <%2B49%20331%2074%2099-667> > encryption key ID 7D6B4AA0 > > ------------------------------------------------------------------------ > > Leibniz-Institut f?r Astrophysik Potsdam (AIP) > > An der Sternwarte 16, D-14482 Potsdam > > > > Vorstand: Prof. Dr. Matthias Steinmetz, Matthias Winker > > Stiftung b?rgerlichen Rechts, Stiftungsverz. Brandenburg: 26 742-00/7026 > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > https://mail.scipy.org/mailman/listinfo/astropy > > > > > -- > Kelle Cruz, PhD ? http://kellecruz.com/ > 917.725.1334 ? Hunter: x16486 ? AMNH: x3404 > > > > > > _______________________________________________ > AstroPy mailing listAstroPy at scipy.orghttps://mail.scipy.org/mailman/listinfo/astropy > > > > > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jturner at gemini.edu Fri Mar 24 09:37:25 2017 From: jturner at gemini.edu (James E.H. Turner) Date: Fri, 24 Mar 2017 10:37:25 -0300 Subject: [AstroPy] Migrating to Google Groups In-Reply-To: References: <41f72a18-3545-41e1-1477-9e56545b1b34@sconseil.fr> Message-ID: Apparently not, since it delivered your question... On 24/03/17 10:33, Gabriel Perren wrote: > So, just to be clear: everyone needs to re-suscribe to the mailing list > here https://mail.python.org/mailman/listinfo/astropy? From franz.bergesund at gmail.com Fri Mar 24 09:41:10 2017 From: franz.bergesund at gmail.com (Francesco Montesano) Date: Fri, 24 Mar 2017 13:41:10 +0000 Subject: [AstroPy] Migrating to Google Groups In-Reply-To: References: <41f72a18-3545-41e1-1477-9e56545b1b34@sconseil.fr> Message-ID: Hi Gabriel, the list has already been migrated. This mail is already from the new python.org server. @Simon: what about the old list? Has it been removed or we need to unsubscribe? Ciao, Fra Il giorno ven 24 mar 2017 alle ore 14:34 Gabriel Perren < gabrielperren at gmail.com> ha scritto: > So, just to be clear: everyone needs to re-suscribe to the mailing list > here https://mail.python.org/mailman/listinfo/astropy? > > El vie., 24 de mar. de 2017 a la(s) 06:27, Simon Conseil < > simon at sconseil.fr> escribi?: > > Hi, > > The mailing lists moved ( > https://mail.python.org/pipermail/numpy-discussion/2017-March/076592.html > ). > > So the new address is now astropy at python.org ( > https://mail.python.org/mailman/listinfo/astropy) > Simon > > > > On 03/23/17 19:46, Kelle Cruz wrote: > > Hello Everyone, > > My sincere apologies: The migration to Google Groups is "paused". We are > going to look into the migration of the lists to python.org and > re-evaluate. > > Thanks to Simon for bringing that key piece of info to our attention! > > Kelle > > On Thu, Mar 23, 2017 at 4:51 AM, Peter Weilbacher > wrote: > > So you are actually asking everyone to resubscribe? That was not clear > > in the message. > > > > Why are you not using the same non-Google system that apparently the > > rest of scipy/numpy switches to (see announcement by Ralf Gommers, > > )? > > I understand that then you could just migrate all subscribers directly. > > > > Peter. > > > > On Wed, 22 Mar 2017, Thomas Robitaille wrote: > > > > > Hi everyone, > > > > > > Just to answer some concerns from people who have asked about whether a > > > Google account, the answer is no - you can join the new mailing list as a > > > normal email mailing list by sending an email to > > > astropy+subscribe at googlegroups.com, then replying to the email that you > > > receive back to confirm subscription. I've added these instructions to > the > > > welcome page for the group. > > > > > > Let me know (off-list) if you have any issues with joining using this > > > method! > > > > > > Cheers, > > > Tom > > > > > > > > > > > > On 20 March 2017 at 17:13, Kelle Cruz wrote: > > > > > > > Hi Everyone, > > > > > > > > The Astropy Coordination Committee is in the process of migrating this > > > > group from Mailman to Google Groups. Shortly, I?ll send around an > invite > > > > link. The intention is for nothing to change about the group except the > > > > address and management options. Any topic related to using Python for > > > > Astronomy is welcome and need not be relevant to the ?Astropy Project.? > > > > > > > > We plan for the migration to be complete in two weeks and this address > > > > will be deactivated on March 31. The archive will stay up indefinitely > but > > > > no new messages will be accepted after that date. We?ll send another > > > > reminder next week. > > > > > > > > We would like to take this opportunity to remind everyone that this > group > > > > pre-dates the Astropy Project and indeed, was the inspiration for the > > > > project name and not the other way around. > > > > > > > > Thanks so much for making this an amazing community! > > > > > > > > Kelle > > > > Astropy Community Engagement Coordinator > > > > -- > > Dr. Peter M. Weilbacher http://www.aip.de/People/PWeilbacher > > Phone +49 331 74 99-667 <%2B49%20331%2074%2099-667> > encryption key ID 7D6B4AA0 > > ------------------------------------------------------------------------ > > Leibniz-Institut f?r Astrophysik Potsdam (AIP) > > An der Sternwarte 16, D-14482 Potsdam > > > > Vorstand: Prof. Dr. Matthias Steinmetz, Matthias Winker > > Stiftung b?rgerlichen Rechts, Stiftungsverz. Brandenburg: 26 742-00/7026 > > _______________________________________________ > > AstroPy mailing list > > AstroPy at scipy.org > > https://mail.scipy.org/mailman/listinfo/astropy > > > > > -- > Kelle Cruz, PhD ? http://kellecruz.com/ > 917.725.1334 ? Hunter: x16486 ? AMNH: x3404 > > > > > > _______________________________________________ > AstroPy mailing listAstroPy at scipy.orghttps://mail.scipy.org/mailman/listinfo/astropy > > > > > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jslavin at cfa.harvard.edu Fri Mar 24 10:04:38 2017 From: jslavin at cfa.harvard.edu (Slavin, Jonathan) Date: Fri, 24 Mar 2017 10:04:38 -0400 Subject: [AstroPy] So whence the google group? Message-ID: In the meantime I was asked to join the google group. Is that now defunct? I for one was looking forward to the nicer formatting that you get with e-mail lists for google groups. Jon -- ________________________________________________________ Jonathan D. Slavin Harvard-Smithsonian CfA jslavin at cfa.harvard.edu 60 Garden Street, MS 83 phone: (617) 496-7981 Cambridge, MA 02138-1516 cell: (781) 363-0035 USA ________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon at sconseil.fr Fri Mar 24 10:09:45 2017 From: simon at sconseil.fr (Simon Conseil) Date: Fri, 24 Mar 2017 15:09:45 +0100 Subject: [AstroPy] Migrating to Google Groups In-Reply-To: References: <41f72a18-3545-41e1-1477-9e56545b1b34@sconseil.fr> Message-ID: Hi, As python.org uses the same software as scipy.org (Mailman), I think the migration is completely transparent. So no need to re-suscribe, or unsubscribe (the old list at scipy.org will probably be archived or deleted, I don't know, but it's probably already in read-only mode). Simon On 03/24/17 14:41, Francesco Montesano wrote: > Hi Gabriel, > > the list has already been migrated. This mail is already from the new > python.org server. > > @Simon: what about the old list? Has it been removed or we need to > unsubscribe? > > Ciao, > > Fra > > Il giorno ven 24 mar 2017 alle ore 14:34 Gabriel Perren > > ha scritto: > > So, just to be clear: everyone needs to re-suscribe to the mailing > list here https://mail.python.org/mailman/listinfo/astropy? > > El vie., 24 de mar. de 2017 a la(s) 06:27, Simon Conseil > > escribi?: > > Hi, > > The mailing lists moved > (https://mail.python.org/pipermail/numpy-discussion/2017-March/076592.html). > > So the new address is now astropy at python.org > > (https://mail.python.org/mailman/listinfo/astropy) > > Simon > > > > On 03/23/17 19:46, Kelle Cruz wrote: >> Hello Everyone, >> >> My sincere apologies: The migration to Google Groups is >> "paused". We are going to look into the migration of the >> lists to python.org and re-evaluate. >> >> Thanks to Simon for bringing that key piece of info to our >> attention! >> >> Kelle >> >> On Thu, Mar 23, 2017 at 4:51 AM, Peter Weilbacher >> > wrote: >> >> So you are actually asking everyone to resubscribe? That >> was not clear >> >> in the message. >> >> >> >> Why are you not using the same non-Google system that >> apparently the >> >> rest of scipy/numpy switches to (see announcement by Ralf >> Gommers, >> >> )? >> >> I understand that then you could just migrate all >> subscribers directly. >> >> >> >> Peter. >> >> >> >> On Wed, 22 Mar 2017, Thomas Robitaille wrote: >> >> >> >> > Hi everyone, >> >> > >> >> > Just to answer some concerns from people who have asked >> about whether a >> >> > Google account, the answer is no - you can join the new >> mailing list as a >> >> > normal email mailing list by sending an email to >> >> > astropy+subscribe at googlegroups.com >> , then >> replying to the email that you >> >> > receive back to confirm subscription. I've added these >> instructions to the >> >> > welcome page for the group. >> >> > >> >> > Let me know (off-list) if you have any issues with >> joining using this >> >> > method! >> >> > >> >> > Cheers, >> >> > Tom >> >> > >> >> > >> >> > >> >> > On 20 March 2017 at 17:13, Kelle Cruz >> > wrote: >> >> > >> >> > > Hi Everyone, >> >> > > >> >> > > The Astropy Coordination Committee is in the process >> of migrating this >> >> > > group from Mailman to Google Groups. Shortly, I?ll >> send around an invite >> >> > > link. The intention is for nothing to change about >> the group except the >> >> > > address and management options. Any topic related to >> using Python for >> >> > > Astronomy is welcome and need not be relevant to the >> ?Astropy Project.? >> >> > > >> >> > > We plan for the migration to be complete in two weeks >> and this address >> >> > > will be deactivated on March 31. The archive will >> stay up indefinitely but >> >> > > no new messages will be accepted after that date. >> We?ll send another >> >> > > reminder next week. >> >> > > >> >> > > We would like to take this opportunity to remind >> everyone that this group >> >> > > pre-dates the Astropy Project and indeed, was the >> inspiration for the >> >> > > project name and not the other way around. >> >> > > >> >> > > Thanks so much for making this an amazing community! >> >> > > >> >> > > Kelle >> >> > > Astropy Community Engagement Coordinator >> >> >> >> -- >> >> Dr. Peter M. Weilbacher http://www.aip.de/People/PWeilbacher >> >> Phone +49 331 74 99-667 >> encryption key ID 7D6B4AA0 >> >> ------------------------------------------------------------------------ >> >> Leibniz-Institut f?r Astrophysik Potsdam (AIP) >> >> An der Sternwarte 16, D-14482 Potsdam >> >> >> >> Vorstand: Prof. Dr. Matthias Steinmetz, Matthias Winker >> >> Stiftung b?rgerlichen Rechts, Stiftungsverz. Brandenburg: >> 26 742-00/7026 >> >> _______________________________________________ >> >> AstroPy mailing list >> >> AstroPy at scipy.org >> >> https://mail.scipy.org/mailman/listinfo/astropy >> >> >> >> >> -- >> Kelle Cruz, PhD ? http://kellecruz.com/ >> 917.725.1334 ? Hunter: x16486 ? AMNH: x3404 >> >> >> >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> https://mail.scipy.org/mailman/listinfo/astropy >> >> > > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > > > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From kellecruz at gmail.com Fri Mar 24 10:52:22 2017 From: kellecruz at gmail.com (Kelle Cruz) Date: Fri, 24 Mar 2017 14:52:22 +0000 Subject: [AstroPy] So whence the google group? In-Reply-To: References: Message-ID: We're in the process of reevaluating the move to Google Groups. On Fri, Mar 24, 2017 at 10:05 AM Slavin, Jonathan wrote: > In the meantime I was asked to join the google group. Is that now defunct? > > I for one was looking forward to the nicer formatting that you get with > e-mail lists for google groups. > > Jon > > -- > ________________________________________________________ > Jonathan D. Slavin Harvard-Smithsonian CfA > jslavin at cfa.harvard.edu 60 Garden Street, MS 83 > phone: (617) 496-7981 Cambridge, MA 02138-1516 > cell: (781) 363-0035 USA > ________________________________________________________ > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > -- kelle (via iOS) -------------- next part -------------- An HTML attachment was scrubbed... URL: From evert.rol at gmail.com Sun Mar 26 02:17:07 2017 From: evert.rol at gmail.com (Evert Rol) Date: Sun, 26 Mar 2017 17:17:07 +1100 Subject: [AstroPy] astropy Table to table in a multi-extension fits file In-Reply-To: <6BB70AA4-37F9-482C-A614-D91FB39B07A8@gmail.com> References: <6BB70AA4-37F9-482C-A614-D91FB39B07A8@gmail.com> Message-ID: > > I have several related astropy Tables (inside a python script) which I want to write to a multi-extension fits file. Is there a simple way to do this using astropy.io.fits? Hi Knox, astropy.io.fits has a function `table_to_hdu` which may make things easier for you. Here's an example: from astropy.io.fits import table_to_hdu, HDUList, PrimaryHDU from astropy.table import Table HDUList([ PrimaryHDU(), table_to_hdu( Table({'a': [1,2]})), table_to_hdu( Table({'b': [1,2,3]})) ]).writeto('test.fits') (possibly appending to a HDU list inside a loop in your case.) Evert From knoxlong at gmail.com Sun Mar 26 13:27:25 2017 From: knoxlong at gmail.com (Knox Long) Date: Sun, 26 Mar 2017 17:27:25 +0000 Subject: [AstroPy] astropy Table to table in a multi-extension fits file In-Reply-To: References: <6BB70AA4-37F9-482C-A614-D91FB39B07A8@gmail.com> Message-ID: Yes, Evert. Thank you. That did work. Knox On Sun, Mar 26, 2017 at 1:17 AM Evert Rol wrote: > > > > I have several related astropy Tables (inside a python script) which I > want to write to a multi-extension fits file. Is there a simple way to do > this using astropy.io.fits? > > Hi Knox, > > astropy.io.fits has a function `table_to_hdu` which may make things easier > for you. > Here's an example: > > from astropy.io.fits import table_to_hdu, HDUList, PrimaryHDU > from astropy.table import Table > > HDUList([ > PrimaryHDU(), > table_to_hdu( Table({'a': [1,2]})), > table_to_hdu( Table({'b': [1,2,3]})) > ]).writeto('test.fits') > > (possibly appending to a HDU list inside a loop in your case.) > > > Evert > > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy > -- Knox S. Long 410-338-4862 (office) -------------- next part -------------- An HTML attachment was scrubbed... URL: From evert.rol at gmail.com Mon Mar 27 03:05:22 2017 From: evert.rol at gmail.com (Evert Rol) Date: Mon, 27 Mar 2017 18:05:22 +1100 Subject: [AstroPy] Astropy logging configuration Message-ID: <9A72FED5-403C-4E13-B5DA-91F6DEFABC8D@gmail.com> Is there a way to configure the astropy logger using the standard Python logging configuration? If I try the code below, I actually get double output, with only one of the two lines using the format I specified. More specifically, I'd like to be able to use a config dict, or a yaml file that may contain the configuration for several loggers: some may require a a debug level (for specific packages I'm debugging), while I'd like to silence others (to a warning or error level); so a global (root) logging configuration doesn't really work. (I realise that the amount of logging in astropy is low to non-existent, even at the debug level.) Practically, this issue came up when using a Django application, where I tried to set the various loggers in the Django settings file. That doesn't work (and caused some headaches, because astropy needs to be imported before Django sets up its loggers, which means it needs to be imported outside the whole Django project). Evert import logging import astropy logger = logging.getLogger('astropy') formatter = logging.Formatter(fmt="%(asctime)s [%(levelname)s] %(message)s") handler = logging.StreamHandler() handler.setLevel(logging.DEBUG) handler.setFormatter(formatter) logger.addHandler(handler) logger.setLevel(logging.INFO) logger.info('Hello') astropy.log.info('Bye') output: INFO: Hello [__main__] 2017-03-27 14:05:28,330 [INFO] Hello INFO: Bye [__main__] 2017-03-27 14:05:29,341 [INFO] Bye From adrianmpw at gmail.com Mon Mar 27 09:44:06 2017 From: adrianmpw at gmail.com (Adrian Price-Whelan) Date: Mon, 27 Mar 2017 09:44:06 -0400 Subject: [AstroPy] Astropy logging configuration In-Reply-To: <9A72FED5-403C-4E13-B5DA-91F6DEFABC8D@gmail.com> References: <9A72FED5-403C-4E13-B5DA-91F6DEFABC8D@gmail.com> Message-ID: Interested to hear if there is a better solution (I run in to this too). The hack I've used to suppress the duplicates is to remove all handlers from the logger after importing astropy but before adding my own. For your example: import logging import astropy logger = logging.getLogger('astropy') for handler in logger.handlers: logger.removeHandler(handler) formatter = logging.Formatter(fmt="%(asctime)s [%(levelname)s] %(message)s") handler = logging.StreamHandler() handler.setLevel(logging.DEBUG) handler.setFormatter(formatter) logger.addHandler(handler) logger.setLevel(logging.INFO) logger.info('Hello') astropy.log.info('Bye') output is then: 2017-03-27 09:43:34,319 [INFO] Hello 2017-03-27 09:43:34,321 [INFO] Bye On Mon, Mar 27, 2017 at 3:05 AM, Evert Rol wrote: > Is there a way to configure the astropy logger using the standard Python logging configuration? > If I try the code below, I actually get double output, with only one of the two lines using the format I specified. > More specifically, I'd like to be able to use a config dict, or a yaml file that may contain the configuration for several loggers: some may require a a debug level (for specific packages I'm debugging), while I'd like to silence others (to a warning or error level); so a global (root) logging configuration doesn't really work. > (I realise that the amount of logging in astropy is low to non-existent, even at the debug level.) > > Practically, this issue came up when using a Django application, where I tried to set the various loggers in the Django settings file. That doesn't work (and caused some headaches, because astropy needs to be imported before Django sets up its loggers, which means it needs to be imported outside the whole Django project). > > > Evert > > > import logging > import astropy > logger = logging.getLogger('astropy') > formatter = logging.Formatter(fmt="%(asctime)s [%(levelname)s] %(message)s") > handler = logging.StreamHandler() > handler.setLevel(logging.DEBUG) > handler.setFormatter(formatter) > logger.addHandler(handler) > logger.setLevel(logging.INFO) > > logger.info('Hello') > astropy.log.info('Bye') > > > output: > INFO: Hello [__main__] > 2017-03-27 14:05:28,330 [INFO] Hello > INFO: Bye [__main__] > 2017-03-27 14:05:29,341 [INFO] Bye > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy -- Adrian M. Price-Whelan Lyman Spitzer, Jr. Postdoctoral Fellow Princeton University http://adrian.pw From evert.rol at gmail.com Mon Mar 27 23:46:31 2017 From: evert.rol at gmail.com (Evert Rol) Date: Tue, 28 Mar 2017 14:46:31 +1100 Subject: [AstroPy] Astropy logging configuration In-Reply-To: References: <9A72FED5-403C-4E13-B5DA-91F6DEFABC8D@gmail.com> Message-ID: <819098DD-A303-4E34-911F-E2E949D5AF22@gmail.com> Adrian, thanks. I actually realised just shortly after I sent my question that there were of course two handlers. So your solution below will definitely help customising the astropy logger. I'm not sure if it can fully solve the underlying issues I had (that is, configure the astropy logger even before astropy is imported), but it may help towards that, or help circumvent that. > Interested to hear if there is a better solution (I run in to this > too). The hack I've used to suppress the duplicates is to remove all > handlers from the logger after importing astropy but before adding my > own. For your example: > > import logging > import astropy > logger = logging.getLogger('astropy') > for handler in logger.handlers: > logger.removeHandler(handler) > > formatter = logging.Formatter(fmt="%(asctime)s [%(levelname)s] %(message)s") > handler = logging.StreamHandler() > handler.setLevel(logging.DEBUG) > handler.setFormatter(formatter) > logger.addHandler(handler) > logger.setLevel(logging.INFO) > > logger.info('Hello') > astropy.log.info('Bye') > > output is then: > 2017-03-27 09:43:34,319 [INFO] Hello > 2017-03-27 09:43:34,321 [INFO] Bye > > On Mon, Mar 27, 2017 at 3:05 AM, Evert Rol wrote: >> Is there a way to configure the astropy logger using the standard Python logging configuration? >> If I try the code below, I actually get double output, with only one of the two lines using the format I specified. >> More specifically, I'd like to be able to use a config dict, or a yaml file that may contain the configuration for several loggers: some may require a a debug level (for specific packages I'm debugging), while I'd like to silence others (to a warning or error level); so a global (root) logging configuration doesn't really work. >> (I realise that the amount of logging in astropy is low to non-existent, even at the debug level.) >> >> Practically, this issue came up when using a Django application, where I tried to set the various loggers in the Django settings file. That doesn't work (and caused some headaches, because astropy needs to be imported before Django sets up its loggers, which means it needs to be imported outside the whole Django project). >> >> >> Evert >> >> >> import logging >> import astropy >> logger = logging.getLogger('astropy') >> formatter = logging.Formatter(fmt="%(asctime)s [%(levelname)s] %(message)s") >> handler = logging.StreamHandler() >> handler.setLevel(logging.DEBUG) >> handler.setFormatter(formatter) >> logger.addHandler(handler) >> logger.setLevel(logging.INFO) >> >> logger.info('Hello') >> astropy.log.info('Bye') >> >> >> output: >> INFO: Hello [__main__] >> 2017-03-27 14:05:28,330 [INFO] Hello >> INFO: Bye [__main__] >> 2017-03-27 14:05:29,341 [INFO] Bye >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at python.org >> https://mail.python.org/mailman/listinfo/astropy > > > > -- > Adrian M. Price-Whelan > Lyman Spitzer, Jr. Postdoctoral Fellow > Princeton University > http://adrian.pw > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy From adam.g.ginsburg at gmail.com Thu Mar 30 15:34:30 2017 From: adam.g.ginsburg at gmail.com (Adam Ginsburg) Date: Thu, 30 Mar 2017 13:34:30 -0600 Subject: [AstroPy] Astroquery 0.3.5 release Message-ID: Hi all, Astroquery v0.3.5 has been released. It contains a few bugfixes and minor enhancements that are, as always, cataloged in the changelog. A notable new feature is the ability to resume partially-completed downloads if the archive supports 'range' queries. Pey Lian Lim ported the astropy.vo conesearch into astroquery.vo_conesearch. A new module for querying GAIA databases has been added, which was contributed by Juan Segovia from ESA: http://astroquery.readthedocs.io/en/latest/gaia/gaia.html In v0.3.3, the ESASky module was contributed by Henrik Norman, also from ESA (see https://arxiv.org/abs/1610.09826): http://astroquery.readthedocs.io/en/latest/esasky/esasky.html The changelog for v0.3.5 is here: https://github.com/astropy/astroquery/blob/v0.3.5/CHANGES Astroquery continues to grow! As always, external contributions are welcome. -- Adam Ginsburg & Brigitta Sipcoz -------------- next part -------------- An HTML attachment was scrubbed... URL: