From demitri.muna at gmail.com Thu Aug 1 15:27:58 2013 From: demitri.muna at gmail.com (Demitri Muna) Date: Thu, 1 Aug 2013 15:27:58 -0400 Subject: [AstroPy] A replacement for the FITS format. In-Reply-To: <51ED40ED.4020103@stsci.edu> References: <13D245F5-9F4D-44DD-90A9-82E84B8594AD@gmail.com> <51ED40ED.4020103@stsci.edu> Message-ID: <87A8564D-FE2E-446C-8B48-6860BCF69885@gmail.com> Hi, On Jul 22, 2013, at 10:25 AM, Erik Bray wrote: > I think that the last bullet point could use clarification. The way I might > word this is that it should be possible to order table data in row-order or > column-order depending on how it is most likely to be accessed. This is of > course a determination that would have to be made by the user, and sometimes > there is no right answer (but where there is a right answer it should be > possible to do). Is this that important anymore? First, it's hard (if not sometimes impossible) so say whether a user will want to read a file by columns or rows. It depends on the use case. Second, if offsets to the start of each row were cached, I'd argue that with SSD (we're thinking of the future here) drives and proper random access to the file, reading a column with offsets shouldn't be too big of a speed hit. The real problem is that FITS tables can have variable length rows. I think this is the real problem to address - not allowing that. Cheers, Demitri _________________________________________ Demitri Muna Department of Astronomy Ohio State University http://scicoder.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ghang.naoc at gmail.com Tue Aug 6 10:45:25 2013 From: ghang.naoc at gmail.com (gonghang.naoc) Date: Tue, 6 Aug 2013 22:45:25 +0800 Subject: [AstroPy] an image fitting problem Message-ID: Hi everybody, If a star is in the outskirt of a galaxy,how to estimate its background reasonably?The final target is to do photometry correctly.The background is sort of bright and the star is difficult to be seen visually. I plan to to remove the region around the star ,then make a local thin-plate background fitting. Does anybody have any suggestions?Fitting methods or existing python scripts ,both are welcome. Best regards hang -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.tollerud at gmail.com Tue Aug 6 16:35:00 2013 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Tue, 6 Aug 2013 16:35:00 -0400 Subject: [AstroPy] Astropy Coordination Meeting Oct 7-8 @ Yale University Message-ID: Hello all, Based on the results of the doodle poll from last month, there's enough interest that we will hold an Astropy Coordination meeting on Oct 7-8 (Mon/Tues) at Yale University in New Haven, CT, USA. This year we will aim for a code-oriented meeting, with no more than a half-day of updates/talks, and the remainder of the time to be focused on breakout group planning discussions and code sprints. A more specific agenda (and more detailed logistics) will be posted on the wiki page as the meeting time approaches. If you wish to attend the meeting (either in person or remotely), please add your name to the github wiki page at https://github.com/astropy/astropy/wiki/Yalemeeting2013 by September 1. This list will be used as a headcount to determine how much coffee and lunch to order (which will be generously provided by the Yale Astronomy Department). Additionally, if enough people sign up, we may be able to reserve a block at a nearby hotel. -- Erik T -------------- next part -------------- An HTML attachment was scrubbed... URL: From lsinger at caltech.edu Wed Aug 7 12:28:28 2013 From: lsinger at caltech.edu (Leo Singer) Date: Wed, 7 Aug 2013 09:28:28 -0700 Subject: [AstroPy] Astropy bootcamp at iPTF workshop Message-ID: <01BEFA2A-EA3A-4207-9A23-C16C05DC7705@caltech.edu> Hello, Next week, I am supposed to give a tutorial on Astropy during a 'bootcamp' session of the iPTF workshop (http://ptf.caltech.edu/iptf/iptf_workshop/srk_agenda.html). I am interpreting this as an introduction to Python itself as well. I have half an hour, but I am asking the organizers to extend that to a full hour. Are there any tutorial resources on Astropy that I should know about? My idea was to put together a presentation as an IPython Notebook and go through a few different common data analysis tasks. Thanks, Leo Singer Graduate Student @ LIGO-Caltech From demitri.muna at gmail.com Wed Aug 7 12:56:26 2013 From: demitri.muna at gmail.com (Demitri Muna) Date: Wed, 7 Aug 2013 12:56:26 -0400 Subject: [AstroPy] Astropy demo presentations. In-Reply-To: <01BEFA2A-EA3A-4207-9A23-C16C05DC7705@caltech.edu> References: <01BEFA2A-EA3A-4207-9A23-C16C05DC7705@caltech.edu> Message-ID: <868798CF-FFEF-4918-9CB4-D26F530716AF@gmail.com> Hi, On Aug 7, 2013, at 12:28 PM, Leo Singer wrote: > Next week, I am supposed to give a tutorial on Astropy during a 'bootcamp' session of the iPTF workshop (http://ptf.caltech.edu/iptf/iptf_workshop/srk_agenda.html). I am interpreting this as an introduction to Python itself as well. I have half an hour, but I am asking the organizers to extend that to a full hour. Are there any tutorial resources on Astropy that I should know about? My idea was to put together a presentation as an IPython Notebook and go through a few different common data analysis tasks. This raises a request I was going to bring up. For each major release, can we (as a group) put together an Astropy demonstration for each major release? This way when there is a new release, people at any institution would have something to demo for their department, e.g. at their morning coffee. I think many more people will give such a presentation if it exists versus sitting down to write one and give it. I'd recommend a five minute version and a half-hour version. These should be available coincident with the releases, and highlight the major functionality of Astropy. If one has to choose between a full introduction and a "what is new since the last release", I'd opt for the former, but both would be ideal. Cheers, Demitri _________________________________________ Demitri Muna Department of Astronomy Ohio State University http://scicoder.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From moritz.guenther at gmx.de Wed Aug 7 13:19:32 2013 From: moritz.guenther at gmx.de (Hans Moritz Guenther) Date: Wed, 07 Aug 2013 13:19:32 -0400 Subject: [AstroPy] Astropy bootcamp at iPTF workshop In-Reply-To: References: Message-ID: <520281A4.4070608@gmx.de> Hi, astropy specific tutorials are hosted at: https://github.com/astropy/astropy-tutorials and https://github.com/python4astronomers (see rendered version here: http://python4astronomers.github.io/ ) The last two chapters are astro-py specific, the others teach python in general. However, you'll have to make a careful selection what you can reasonably show in 1 hour (or even only 30 min). Moritz From klabrie at gemini.edu Wed Aug 7 15:37:58 2013 From: klabrie at gemini.edu (Kathleen Labrie) Date: Wed, 7 Aug 2013 19:37:58 +0000 Subject: [AstroPy] Astropy demo presentations. In-Reply-To: <868798CF-FFEF-4918-9CB4-D26F530716AF@gmail.com> References: <01BEFA2A-EA3A-4207-9A23-C16C05DC7705@caltech.edu> <868798CF-FFEF-4918-9CB4-D26F530716AF@gmail.com> Message-ID: This is a brilliant idea Demitri! That would be so useful. Kathleen On Aug 7, 2013, at 6:56 AM, Demitri Muna > wrote: Hi, On Aug 7, 2013, at 12:28 PM, Leo Singer > wrote: Next week, I am supposed to give a tutorial on Astropy during a 'bootcamp' session of the iPTF workshop (http://ptf.caltech.edu/iptf/iptf_workshop/srk_agenda.html). I am interpreting this as an introduction to Python itself as well. I have half an hour, but I am asking the organizers to extend that to a full hour. Are there any tutorial resources on Astropy that I should know about? My idea was to put together a presentation as an IPython Notebook and go through a few different common data analysis tasks. This raises a request I was going to bring up. For each major release, can we (as a group) put together an Astropy demonstration for each major release? This way when there is a new release, people at any institution would have something to demo for their department, e.g. at their morning coffee. I think many more people will give such a presentation if it exists versus sitting down to write one and give it. I'd recommend a five minute version and a half-hour version. These should be available coincident with the releases, and highlight the major functionality of Astropy. If one has to choose between a full introduction and a "what is new since the last release", I'd opt for the former, but both would be ideal. Cheers, Demitri _________________________________________ Demitri Muna Department of Astronomy Ohio State University http://scicoder.org/ _______________________________________________ AstroPy mailing list AstroPy at scipy.org http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From fabricio at ferrari.pro.br Thu Aug 8 09:24:39 2013 From: fabricio at ferrari.pro.br (Fabricio Ferrari) Date: Thu, 8 Aug 2013 10:24:39 -0300 Subject: [AstroPy] SDSS Cutout image in FITS format Message-ID: Hello all I'd like to retrieve cutouts of SDSS fields in FITS format, essentially in the same way to what CASJOBS ImgCutoutDR7 does, except the output format. In ImgCutoutDR7 you can specify the RA/DEC, SCALE and W/H directly in the URL specification, like the example below. http://casjobs.sdss.org/ImgCutoutDR7/getjpeg.aspx?ra=184.9511&dec=-0.8754&scale=1.5020920&width=293&height=293 So, it is well suited to bath retrieve a list of galaxies. I'd like the same for FITS. Do you know some tool/way to do that? thanks in advance ..-. ..-. Fabricio Ferrari [www.ferrari.pro.br] IMEF - FURG Rio Grande, RS Brasil -------------- next part -------------- An HTML attachment was scrubbed... URL: From erwin at mpe.mpg.de Thu Aug 8 09:51:16 2013 From: erwin at mpe.mpg.de (Peter Erwin) Date: Thu, 8 Aug 2013 15:51:16 +0200 Subject: [AstroPy] SDSS Cutout image in FITS format In-Reply-To: References: Message-ID: <200D0B02-AA88-455B-9074-9EEB70545A51@mpe.mpg.de> Hi Fabricio, My fetchsdss.py script might do some of what you want: http://www.mpe.mpg.de/~erwin/code/ It retrieves the full FITS images for a given SDSS field, given either the name of an object (with coordinate lookup via Simbad) or the coordinates (or the field specification itself, if you happen to know that). The current version works with DR7, but I have a beta version that can also retrieve DR9 images. There's no ability to specify cutouts, though. cheers, Peter On Aug 8, 2013, at 3:24 PM, Fabricio Ferrari wrote: > Hello all > > I'd like to retrieve cutouts of SDSS fields in FITS format, essentially in the same way to what CASJOBS ImgCutoutDR7 does, except the output format. In > ImgCutoutDR7 you can specify the RA/DEC, SCALE and W/H directly in the URL specification, like the example below. > > http://casjobs.sdss.org/ImgCutoutDR7/getjpeg.aspx?ra=184.9511&dec=-0.8754&scale=1.5020920&width=293&height=293 > > > So, it is well suited to bath retrieve a list of galaxies. > I'd like the same for FITS. Do you know some tool/way to do that? > > thanks in advance > ..-. ..-. > Fabricio Ferrari [www.ferrari.pro.br] > IMEF - FURG > Rio Grande, RS > Brasil > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy ============================================================= Peter Erwin Max-Planck-Insitute for Extraterrestrial erwin at mpe.mpg.de Physics, Giessenbachstrasse tel. +49 (0)176 2481 7713 85748 Garching, Germany fax +49 (0)89 30000 3495 http://www.mpe.mpg.de/~erwin From benjamin.weaver at nyu.edu Thu Aug 8 10:01:31 2013 From: benjamin.weaver at nyu.edu (Benjamin Alan Weaver) Date: Thu, 08 Aug 2013 10:01:31 -0400 Subject: [AstroPy] SDSS Cutout image in FITS format In-Reply-To: <200D0B02-AA88-455B-9074-9EEB70545A51@mpe.mpg.de> References: <200D0B02-AA88-455B-9074-9EEB70545A51@mpe.mpg.de> Message-ID: <5203A4BB.10106@nyu.edu> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello Fabricio, I think you might want to try the mosaic service at http://data.sdss3.org/mosaics. Kia ora koe, Benjamin Alan Weaver On 2013-08-08 09:51, Peter Erwin wrote: > Hi Fabricio, > > My fetchsdss.py script might do some of what you want: > > http://www.mpe.mpg.de/~erwin/code/ > > It retrieves the full FITS images for a given SDSS field, given > either the name of an object (with coordinate lookup via Simbad) or > the coordinates (or the field specification itself, if you happen > to know that). The current version works with DR7, but I have a > beta version that can also retrieve DR9 images. > > There's no ability to specify cutouts, though. > > cheers, > > Peter > > On Aug 8, 2013, at 3:24 PM, Fabricio Ferrari > wrote: > >> Hello all >> >> I'd like to retrieve cutouts of SDSS fields in FITS format, >> essentially in the same way to what CASJOBS ImgCutoutDR7 does, >> except the output format. In ImgCutoutDR7 you can specify the >> RA/DEC, SCALE and W/H directly in the URL specification, like the >> example below. >> >> http://casjobs.sdss.org/ImgCutoutDR7/getjpeg.aspx?ra=184.9511&dec=-0.8754&scale=1.5020920&width=293&height=293 >> >> >> >> So, it is well suited to bath retrieve a list of galaxies. I'd >> like the same for FITS. Do you know some tool/way to do that? >> >> thanks in advance ..-. ..-. Fabricio Ferrari >> [www.ferrari.pro.br] IMEF - FURG Rio Grande, RS Brasil >> _______________________________________________ AstroPy mailing >> list AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > ============================================================= Peter > Erwin Max-Planck-Insitute for Extraterrestrial > erwin at mpe.mpg.de Physics, Giessenbachstrasse tel. +49 > (0)176 2481 7713 85748 Garching, Germany fax +49 (0)89 30000 > 3495 http://www.mpe.mpg.de/~erwin > > > > _______________________________________________ AstroPy mailing > list AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > - -- a.k.a. The Dream Weaver There are always at least four choices: one, the other, neither, or both. -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.18 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJSA6S7AAoJEDULpP7FtgwTtwQH/R1E41h+TWkfMGS9Z4CyUPoT f3IUgqm2EcfXsMyDnMMjxZ4/nlHvBgkj/OYXK1ro4NzDFxkbIzSXk1LnpZUr651f ps494T14Huh/WZ+iebqzSDMqFmz4+RffKk3Gr71k6v7DM3cG/g3hEWHz+/0s5ss3 NvNtEAVz6GJ29jmdjSJNOS4J1qGHWBXZWVXFzRwVIapKd0DoxXS5d+m8fqCJ4k6o sBptuB7o84wDXlDAPBBmTmr1kq3hvDFVTo4Ub3kEmxJ3wHBzL3hS/eDXKnfbd0ca uNfgRxzz/btvzHpP7dmiL7eYvBXP5PmZvgDEfUdXXU/lK0JTkZGVsZ8/mWDo1hM= =nV2r -----END PGP SIGNATURE----- From ghang.naoc at gmail.com Thu Aug 8 10:42:15 2013 From: ghang.naoc at gmail.com (gonghang.naoc) Date: Thu, 8 Aug 2013 22:42:15 +0800 Subject: [AstroPy] image interpolation Message-ID: hi everybody, I dig a hole in a fits image,then I want to fill the hole via its neighbouring pixels. Is there any simple pythonic way to do image interpolation? Thank you. hang -------------- next part -------------- An HTML attachment was scrubbed... URL: From thoger.emil at gmail.com Thu Aug 8 10:53:00 2013 From: thoger.emil at gmail.com (=?ISO-8859-1?Q?Th=F8ger_Rivera-Thorsen?=) Date: Thu, 08 Aug 2013 16:53:00 +0200 Subject: [AstroPy] image interpolation In-Reply-To: References: Message-ID: <5203B0CC.9030900@gmail.com> HI, Do you know about scipy.interpolate? http://docs.scipy.org/doc/scipy/reference/interpolate.html Cheers; Emil On 2013-08-08 16:42, gonghang.naoc wrote: > hi everybody, > I dig a hole in a fits image,then I want to fill the hole via its > neighbouring pixels. > Is there any simple pythonic way to do image interpolation? > > Thank you. > hang > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From thoger.emil at gmail.com Thu Aug 8 10:55:34 2013 From: thoger.emil at gmail.com (=?ISO-8859-1?Q?Th=F8ger_Rivera-Thorsen?=) Date: Thu, 08 Aug 2013 16:55:34 +0200 Subject: [AstroPy] image interpolation In-Reply-To: References: Message-ID: <5203B166.7040805@gmail.com> Or maybe scikit-image is what you need... http://scikit-image.org/docs/dev/auto_examples/plot_holes_and_peaks.html#example-plot-holes-and-peaks-py On 2013-08-08 16:42, gonghang.naoc wrote: > hi everybody, > I dig a hole in a fits image,then I want to fill the hole via its > neighbouring pixels. > Is there any simple pythonic way to do image interpolation? > > Thank you. > hang > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From ghang.naoc at gmail.com Thu Aug 8 11:18:38 2013 From: ghang.naoc at gmail.com (gonghang.naoc) Date: Thu, 8 Aug 2013 23:18:38 +0800 Subject: [AstroPy] image interpolation In-Reply-To: <5203B4A9.1040203@gmail.com> References: <5203B0CC.9030900@gmail.com> <5203B4A9.1040203@gmail.com> Message-ID: Yes.It seems we can not change the parameters.We do not now how to erode a hole in details,say,what kind of function was used. On Thu, Aug 8, 2013 at 11:09 PM, Th?ger Rivera-Thorsen < thoger.emil at gmail.com> wrote: > I don't know which of the methods on the page would be best, but yes, > the way to go would be to read your FITS image into a numpy array and then > perform your interpolation with whatever method seems appropriate for you. > > Did you see the link I sent to scikit-image? Depending on what exactly you > want to achieve, that may be a simpler way to do it. > > Emil > > > On 2013-08-08 17:04, gonghang.naoc wrote: > > hi, > You mean interp2d(),right? > Please forgive the rookie.I should readout the fits into an array,then > use interp2d() to do interpolation,and at last make a new fits,right? > > hang > > > On Thu, Aug 8, 2013 at 10:53 PM, Th?ger Rivera-Thorsen < > thoger.emil at gmail.com> wrote: > >> HI, >> >> Do you know about scipy.interpolate? >> http://docs.scipy.org/doc/scipy/reference/interpolate.html >> >> Cheers; >> Emil >> >> >> On 2013-08-08 16:42, gonghang.naoc wrote: >> >> hi everybody, >> I dig a hole in a fits image,then I want to fill the hole via its >> neighbouring pixels. >> Is there any simple pythonic way to do image interpolation? >> >> Thank you. >> hang >> >> >> _______________________________________________ >> AstroPy mailing listAstroPy at scipy.orghttp://mail.scipy.org/mailman/listinfo/astropy >> >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From embray at stsci.edu Thu Aug 8 11:50:10 2013 From: embray at stsci.edu (Erik Bray) Date: Thu, 8 Aug 2013 11:50:10 -0400 Subject: [AstroPy] Astropy demo presentations. In-Reply-To: <868798CF-FFEF-4918-9CB4-D26F530716AF@gmail.com> References: <01BEFA2A-EA3A-4207-9A23-C16C05DC7705@caltech.edu> <868798CF-FFEF-4918-9CB4-D26F530716AF@gmail.com> Message-ID: <5203BE32.5090700@stsci.edu> On 08/07/2013 12:56 PM, Demitri Muna wrote: > Hi, > > On Aug 7, 2013, at 12:28 PM, Leo Singer > wrote: > >> Next week, I am supposed to give a tutorial on Astropy during a 'bootcamp' >> session of the iPTF workshop >> (http://ptf.caltech.edu/iptf/iptf_workshop/srk_agenda.html). I am interpreting >> this as an introduction to Python itself as well. I have half an hour, but I >> am asking the organizers to extend that to a full hour. Are there any tutorial >> resources on Astropy that I should know about? My idea was to put together a >> presentation as an IPython Notebook and go through a few different common data >> analysis tasks. > > This raises a request I was going to bring up. For each major release, can we > (as a group) put together an Astropy demonstration for each major release? This > way when there is a new release, people at any institution would have something > to demo for their department, e.g. at their morning coffee. I think many more > people will give such a presentation if it exists versus sitting down to write > one and give it. I'd recommend a five minute version and a half-hour version. > These should be available coincident with the releases, and highlight the major > functionality of Astropy. If one has to choose between a full introduction and a > "what is new since the last release", I'd opt for the former, but both would be > ideal. Each new feature release already includes the latter: http://docs.astropy.org/en/stable/whatsnew/0.2.html Though I could see a cumulative guide that's kept up to date (and that incorporates examples from the "what's new" page) being useful too. Erik From indiajoe at gmail.com Thu Aug 8 14:15:56 2013 From: indiajoe at gmail.com (Joe Philip Ninan) Date: Thu, 8 Aug 2013 23:45:56 +0530 Subject: [AstroPy] an image fitting problem In-Reply-To: References: Message-ID: Hi Hang, If the background is smooth enough, may be the following approach will work. Once you have loaded a rectangular region around the star as a numpy matrix. First mask out some radius (say 1 times FWHM) around the star using the 'ma' module in numpy. You can then create three 1D array containing X, Y and flux count Z from the remaining unmasked pixels. For interpolating 2d mesh with gaps I usually use 2D Radial Basis Function linear interpolation using scipy.interpolate.Rbf(). For example rbfi = scipy.interpolate.Rbf(X,Y, Z, smooth=2,epsilon=2,function='linear') You can generate a mesh on which we have to do interpolation using np.meshgrid() Say XI, YI = np.meshgrid(Xvector,Yvector) And you can use your rbfi() to obtain the interpolated values at those X and Y coordinate matrix. ZI = rbfi(YI, XI) Which you can put back into your fits image to get the interpolated background. I will be eager to know if anybody has other better methods. -cheers joe On 6 August 2013 20:15, gonghang.naoc wrote: > Hi everybody, > > If a star is in the outskirt of a galaxy,how to estimate its background > reasonably?The final target is to do photometry correctly.The background > is sort of bright and the star is difficult to be seen visually. > I plan to to remove the region around the star ,then make a local > thin-plate background fitting. > Does anybody have any suggestions?Fitting methods or existing python > scripts ,both are welcome. > > Best regards > > hang > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -- /--------------------------------------------------------------- "GNU/Linux: because a PC is a terrible thing to waste" - GNU Generation ************************************************ Joe Philip Ninan http://sites.google.com/site/jpninan/ Research Scholar /________________\ DAA, | Vadakeparambil | TIFR, | Pullad P.O. | Mumbai-05, India. | Kerala, India | Ph: +917738438212 | PIN:689548 | ------------------------------\_______________/-------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.robitaille at gmail.com Thu Aug 8 16:55:50 2013 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Thu, 8 Aug 2013 22:55:50 +0200 Subject: [AstroPy] Astropy demo presentations. In-Reply-To: <5203BE32.5090700@stsci.edu> References: <01BEFA2A-EA3A-4207-9A23-C16C05DC7705@caltech.edu> <868798CF-FFEF-4918-9CB4-D26F530716AF@gmail.com> <5203BE32.5090700@stsci.edu> Message-ID: On 8 August 2013 17:50, Erik Bray wrote: > On 08/07/2013 12:56 PM, Demitri Muna wrote: >> Hi, >> >> On Aug 7, 2013, at 12:28 PM, Leo Singer > > wrote: >> >>> Next week, I am supposed to give a tutorial on Astropy during a 'bootcamp' >>> session of the iPTF workshop >>> (http://ptf.caltech.edu/iptf/iptf_workshop/srk_agenda.html). I am interpreting >>> this as an introduction to Python itself as well. I have half an hour, but I >>> am asking the organizers to extend that to a full hour. Are there any tutorial >>> resources on Astropy that I should know about? My idea was to put together a >>> presentation as an IPython Notebook and go through a few different common data >>> analysis tasks. >> >> This raises a request I was going to bring up. For each major release, can we >> (as a group) put together an Astropy demonstration for each major release? This >> way when there is a new release, people at any institution would have something >> to demo for their department, e.g. at their morning coffee. I think many more >> people will give such a presentation if it exists versus sitting down to write >> one and give it. I'd recommend a five minute version and a half-hour version. >> These should be available coincident with the releases, and highlight the major >> functionality of Astropy. If one has to choose between a full introduction and a >> "what is new since the last release", I'd opt for the former, but both would be >> ideal. > > Each new feature release already includes the latter: > http://docs.astropy.org/en/stable/whatsnew/0.2.html > > Though I could see a cumulative guide that's kept up to date (and that > incorporates examples from the "what's new" page) being useful too. How about we make a reveal.js slideshow version of the What's new page, or something similar? Tom > > Erik > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From lsinger at caltech.edu Thu Aug 8 23:56:23 2013 From: lsinger at caltech.edu (Leo Singer) Date: Thu, 8 Aug 2013 20:56:23 -0700 Subject: [AstroPy] Astropy demo presentations In-Reply-To: References: Message-ID: On Aug 8, 2013, at 10:00 AM, astropy-request at scipy.org wrote: > From: Erik Bray > Subject: Re: [AstroPy] Astropy demo presentations. > Date: August 8, 2013 8:50:10 AM PDT > To: > > > On 08/07/2013 12:56 PM, Demitri Muna wrote: >> Hi, >> >> On Aug 7, 2013, at 12:28 PM, Leo Singer > > wrote: >> >>> Next week, I am supposed to give a tutorial on Astropy during a 'bootcamp' >>> session of the iPTF workshop >>> (http://ptf.caltech.edu/iptf/iptf_workshop/srk_agenda.html). I am interpreting >>> this as an introduction to Python itself as well. I have half an hour, but I >>> am asking the organizers to extend that to a full hour. Are there any tutorial >>> resources on Astropy that I should know about? My idea was to put together a >>> presentation as an IPython Notebook and go through a few different common data >>> analysis tasks. >> >> This raises a request I was going to bring up. For each major release, can we >> (as a group) put together an Astropy demonstration for each major release? This >> way when there is a new release, people at any institution would have something >> to demo for their department, e.g. at their morning coffee. I think many more >> people will give such a presentation if it exists versus sitting down to write >> one and give it. I'd recommend a five minute version and a half-hour version. >> These should be available coincident with the releases, and highlight the major >> functionality of Astropy. If one has to choose between a full introduction and a >> "what is new since the last release", I'd opt for the former, but both would be >> ideal. > > Each new feature release already includes the latter: http://docs.astropy.org/en/stable/whatsnew/0.2.html > > Though I could see a cumulative guide that's kept up to date (and that incorporates examples from the "what's new" page) being useful too. I'm a little torn on which version of Astropy I should use for this workshop. Personally, I always use the bleeding edge from Git. Is it a very bad idea to teach from Git rather than from the stable release? FYI, some of the tasks in the tutorial will include Astroquery and Photutils. Thanks, Leo -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.robitaille at gmail.com Fri Aug 9 03:37:41 2013 From: thomas.robitaille at gmail.com (Thomas Robitaille) Date: Fri, 9 Aug 2013 09:37:41 +0200 Subject: [AstroPy] Astropy demo presentations. In-Reply-To: References: <01BEFA2A-EA3A-4207-9A23-C16C05DC7705@caltech.edu> <868798CF-FFEF-4918-9CB4-D26F530716AF@gmail.com> <5203BE32.5090700@stsci.edu> Message-ID: On 8 August 2013 22:55, Thomas Robitaille wrote: > On 8 August 2013 17:50, Erik Bray wrote: >> On 08/07/2013 12:56 PM, Demitri Muna wrote: >>> Hi, >>> >>> On Aug 7, 2013, at 12:28 PM, Leo Singer >> > wrote: >>> >>>> Next week, I am supposed to give a tutorial on Astropy during a 'bootcamp' >>>> session of the iPTF workshop >>>> (http://ptf.caltech.edu/iptf/iptf_workshop/srk_agenda.html). I am interpreting >>>> this as an introduction to Python itself as well. I have half an hour, but I >>>> am asking the organizers to extend that to a full hour. Are there any tutorial >>>> resources on Astropy that I should know about? My idea was to put together a >>>> presentation as an IPython Notebook and go through a few different common data >>>> analysis tasks. >>> >>> This raises a request I was going to bring up. For each major release, can we >>> (as a group) put together an Astropy demonstration for each major release? This >>> way when there is a new release, people at any institution would have something >>> to demo for their department, e.g. at their morning coffee. I think many more >>> people will give such a presentation if it exists versus sitting down to write >>> one and give it. I'd recommend a five minute version and a half-hour version. >>> These should be available coincident with the releases, and highlight the major >>> functionality of Astropy. If one has to choose between a full introduction and a >>> "what is new since the last release", I'd opt for the former, but both would be >>> ideal. >> >> Each new feature release already includes the latter: >> http://docs.astropy.org/en/stable/whatsnew/0.2.html >> >> Though I could see a cumulative guide that's kept up to date (and that >> incorporates examples from the "what's new" page) being useful too. > > How about we make a reveal.js slideshow version of the What's new > page, or something similar? Just to clarify, I meant based on the IPython notebook, of course :) Tom > > Tom > > >> >> Erik >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy From ejsaiet at alaska.edu Fri Aug 9 05:33:48 2013 From: ejsaiet at alaska.edu (Eyal Saiet) Date: Fri, 9 Aug 2013 01:33:48 -0800 Subject: [AstroPy] python and webcameras. Message-ID: Hello friends, I would like to create a python script to actively look at the stars. The idea is using a good webcamera, and the script takes every 40 seconds a bunch of pictures and stacks them to get good intensities of the stars. It seems that some astronomers do that by running the camera on a video mode and then adding the frames?! But video is inherently low quality so stacking low quality frames is not my preferred path. Have you done anything similar with python and a webcamera? Thanks -- Eyal Saiet -------------- next part -------------- An HTML attachment was scrubbed... URL: From aldcroft at head.cfa.harvard.edu Fri Aug 9 07:58:32 2013 From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas) Date: Fri, 9 Aug 2013 07:58:32 -0400 Subject: [AstroPy] Astropy demo presentations In-Reply-To: References: Message-ID: On Thu, Aug 8, 2013 at 11:56 PM, Leo Singer wrote: > On Aug 8, 2013, at 10:00 AM, astropy-request at scipy.org wrote: > > From: Erik Bray > Subject: Re: [AstroPy] Astropy demo presentations. > Date: August 8, 2013 8:50:10 AM PDT > To: > > > On 08/07/2013 12:56 PM, Demitri Muna wrote: > > Hi, > > On Aug 7, 2013, at 12:28 PM, Leo Singer > wrote: > > Next week, I am supposed to give a tutorial on Astropy during a 'bootcamp' > session of the iPTF workshop > (http://ptf.caltech.edu/iptf/iptf_workshop/srk_agenda.html). I am > interpreting > this as an introduction to Python itself as well. I have half an hour, but I > am asking the organizers to extend that to a full hour. Are there any > tutorial > resources on Astropy that I should know about? My idea was to put together a > presentation as an IPython Notebook and go through a few different common > data > analysis tasks. > > > This raises a request I was going to bring up. For each major release, can > we > (as a group) put together an Astropy demonstration for each major release? > This > way when there is a new release, people at any institution would have > something > to demo for their department, e.g. at their morning coffee. I think many > more > people will give such a presentation if it exists versus sitting down to > write > one and give it. I'd recommend a five minute version and a half-hour > version. > These should be available coincident with the releases, and highlight the > major > functionality of Astropy. If one has to choose between a full introduction > and a > "what is new since the last release", I'd opt for the former, but both would > be > ideal. > > > Each new feature release already includes the latter: > http://docs.astropy.org/en/stable/whatsnew/0.2.html > > Though I could see a cumulative guide that's kept up to date (and that > incorporates examples from the "what's new" page) being useful too. > > > I'm a little torn on which version of Astropy I should use for this > workshop. Personally, I always use the bleeding edge from Git. Is it a very > bad idea to teach from Git rather than from the stable release? > > FYI, some of the tasks in the tutorial will include Astroquery and > Photutils. In general I would advise against giving demos using the dev version. Especially for people that may be new to Python, this adds another layer of complexity / confusion and makes the whole installation issue even harder. In our tutorials we have had people just install Anaconda, which takes 5-10 minutes and already includes the stable astropy. There is plenty to demonstrate in astropy stable for an hour. As a side note, one thing we've run into with installing Anaconda at a workshop is wireless bandwidth problems when you have 20 people trying to download a 200 Mb file. Thumb drives are a good solution. - Tom > > Thanks, > Leo > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > From adam.g.ginsburg at gmail.com Fri Aug 9 10:59:25 2013 From: adam.g.ginsburg at gmail.com (Adam Ginsburg) Date: Fri, 9 Aug 2013 08:59:25 -0600 Subject: [AstroPy] Astropy demo presentations In-Reply-To: References: Message-ID: If you're going to demo astroquery, though, note that it will only work with astropy >=0.3, i.e. it requires the dev version of astropy right now. astroquery relies on astropy.coordinates, which changed API between 0.2.x and 0.3. Following Tom Aldcroft's suggestions seems reasonable, but for the next month or two that probably means astroquery has to be left out. On Fri, Aug 9, 2013 at 5:58 AM, Aldcroft, Thomas wrote: > On Thu, Aug 8, 2013 at 11:56 PM, Leo Singer wrote: >> On Aug 8, 2013, at 10:00 AM, astropy-request at scipy.org wrote: >> >> From: Erik Bray >> Subject: Re: [AstroPy] Astropy demo presentations. >> Date: August 8, 2013 8:50:10 AM PDT >> To: >> >> >> On 08/07/2013 12:56 PM, Demitri Muna wrote: >> >> Hi, >> >> On Aug 7, 2013, at 12:28 PM, Leo Singer > > wrote: >> >> Next week, I am supposed to give a tutorial on Astropy during a 'bootcamp' >> session of the iPTF workshop >> (http://ptf.caltech.edu/iptf/iptf_workshop/srk_agenda.html). I am >> interpreting >> this as an introduction to Python itself as well. I have half an hour, but I >> am asking the organizers to extend that to a full hour. Are there any >> tutorial >> resources on Astropy that I should know about? My idea was to put together a >> presentation as an IPython Notebook and go through a few different common >> data >> analysis tasks. >> >> >> This raises a request I was going to bring up. For each major release, can >> we >> (as a group) put together an Astropy demonstration for each major release? >> This >> way when there is a new release, people at any institution would have >> something >> to demo for their department, e.g. at their morning coffee. I think many >> more >> people will give such a presentation if it exists versus sitting down to >> write >> one and give it. I'd recommend a five minute version and a half-hour >> version. >> These should be available coincident with the releases, and highlight the >> major >> functionality of Astropy. If one has to choose between a full introduction >> and a >> "what is new since the last release", I'd opt for the former, but both would >> be >> ideal. >> >> >> Each new feature release already includes the latter: >> http://docs.astropy.org/en/stable/whatsnew/0.2.html >> >> Though I could see a cumulative guide that's kept up to date (and that >> incorporates examples from the "what's new" page) being useful too. >> >> >> I'm a little torn on which version of Astropy I should use for this >> workshop. Personally, I always use the bleeding edge from Git. Is it a very >> bad idea to teach from Git rather than from the stable release? >> >> FYI, some of the tasks in the tutorial will include Astroquery and >> Photutils. > > In general I would advise against giving demos using the dev version. > Especially for people that may be new to Python, this adds another > layer of complexity / confusion and makes the whole installation issue > even harder. In our tutorials we have had people just install > Anaconda, which takes 5-10 minutes and already includes the stable > astropy. There is plenty to demonstrate in astropy stable for an > hour. > > As a side note, one thing we've run into with installing Anaconda at a > workshop is wireless bandwidth problems when you have 20 people trying > to download a 200 Mb file. Thumb drives are a good solution. > > - Tom > >> >> Thanks, >> Leo >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -- Adam Ginsburg Center for Astrophysics and Space Astronomy University of Colorado at Boulder http://www.adamgginsburg.com/ From aldcroft at head.cfa.harvard.edu Fri Aug 9 11:07:04 2013 From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas) Date: Fri, 9 Aug 2013 11:07:04 -0400 Subject: [AstroPy] Astropy demo presentations In-Reply-To: References: Message-ID: On Fri, Aug 9, 2013 at 10:59 AM, Adam Ginsburg wrote: > If you're going to demo astroquery, though, note that it will only > work with astropy >=0.3, i.e. it requires the dev version of astropy > right now. astroquery relies on astropy.coordinates, which changed > API between 0.2.x and 0.3. > > Following Tom Aldcroft's suggestions seems reasonable, but for the > next month or two that probably means astroquery has to be left out. Of course if astroquery is what you really want to highlight based on your audience, then you should probably just do that part of the demo with astropy dev + astroquery, with a caveat up front that this functionality is "coming soon" to astropy stable. - Tom > > On Fri, Aug 9, 2013 at 5:58 AM, Aldcroft, Thomas > wrote: >> On Thu, Aug 8, 2013 at 11:56 PM, Leo Singer wrote: >>> On Aug 8, 2013, at 10:00 AM, astropy-request at scipy.org wrote: >>> >>> From: Erik Bray >>> Subject: Re: [AstroPy] Astropy demo presentations. >>> Date: August 8, 2013 8:50:10 AM PDT >>> To: >>> >>> >>> On 08/07/2013 12:56 PM, Demitri Muna wrote: >>> >>> Hi, >>> >>> On Aug 7, 2013, at 12:28 PM, Leo Singer >> > wrote: >>> >>> Next week, I am supposed to give a tutorial on Astropy during a 'bootcamp' >>> session of the iPTF workshop >>> (http://ptf.caltech.edu/iptf/iptf_workshop/srk_agenda.html). I am >>> interpreting >>> this as an introduction to Python itself as well. I have half an hour, but I >>> am asking the organizers to extend that to a full hour. Are there any >>> tutorial >>> resources on Astropy that I should know about? My idea was to put together a >>> presentation as an IPython Notebook and go through a few different common >>> data >>> analysis tasks. >>> >>> >>> This raises a request I was going to bring up. For each major release, can >>> we >>> (as a group) put together an Astropy demonstration for each major release? >>> This >>> way when there is a new release, people at any institution would have >>> something >>> to demo for their department, e.g. at their morning coffee. I think many >>> more >>> people will give such a presentation if it exists versus sitting down to >>> write >>> one and give it. I'd recommend a five minute version and a half-hour >>> version. >>> These should be available coincident with the releases, and highlight the >>> major >>> functionality of Astropy. If one has to choose between a full introduction >>> and a >>> "what is new since the last release", I'd opt for the former, but both would >>> be >>> ideal. >>> >>> >>> Each new feature release already includes the latter: >>> http://docs.astropy.org/en/stable/whatsnew/0.2.html >>> >>> Though I could see a cumulative guide that's kept up to date (and that >>> incorporates examples from the "what's new" page) being useful too. >>> >>> >>> I'm a little torn on which version of Astropy I should use for this >>> workshop. Personally, I always use the bleeding edge from Git. Is it a very >>> bad idea to teach from Git rather than from the stable release? >>> >>> FYI, some of the tasks in the tutorial will include Astroquery and >>> Photutils. >> >> In general I would advise against giving demos using the dev version. >> Especially for people that may be new to Python, this adds another >> layer of complexity / confusion and makes the whole installation issue >> even harder. In our tutorials we have had people just install >> Anaconda, which takes 5-10 minutes and already includes the stable >> astropy. There is plenty to demonstrate in astropy stable for an >> hour. >> >> As a side note, one thing we've run into with installing Anaconda at a >> workshop is wireless bandwidth problems when you have 20 people trying >> to download a 200 Mb file. Thumb drives are a good solution. >> >> - Tom >> >>> >>> Thanks, >>> Leo >>> >>> _______________________________________________ >>> AstroPy mailing list >>> AstroPy at scipy.org >>> http://mail.scipy.org/mailman/listinfo/astropy >>> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy > > > > -- > Adam Ginsburg > Center for Astrophysics and Space Astronomy > University of Colorado at Boulder > http://www.adamgginsburg.com/ From beaumont at hawaii.edu Fri Aug 9 11:13:36 2013 From: beaumont at hawaii.edu (Chris Beaumont) Date: Fri, 9 Aug 2013 11:13:36 -0400 Subject: [AstroPy] Astropy demo presentations In-Reply-To: References: Message-ID: On a related note, I just received an email advertising the ability to create and distribute custom environments for Wakari (which is basically a cloud-hosted python environment you use from the browser: http://continuum.io/blog/introducing-bundles). I haven't yet found Wakari compelling for personal use, but I could see it being a nice way to run demos, especially those that rely on non-trivial installs. cheers, chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From dencheva at stsci.edu Fri Aug 9 11:24:00 2013 From: dencheva at stsci.edu (Nadezhda Dencheva) Date: Fri, 9 Aug 2013 15:24:00 +0000 Subject: [AstroPy] Astropy demo presentations In-Reply-To: References: , Message-ID: <0153364C9F56D944B0A795780ECF88DF465781A1@EXCHMAIL2.stsci.edu> I have demoed modeling using wakari and ipython notebook. It had (probably still has) astropy 0.2 but I uploaded modeling as a standalone package. It was very easy and fast to set up. Nadia ________________________________ From: astropy-bounces at scipy.org [astropy-bounces at scipy.org] on behalf of Chris Beaumont [beaumont at hawaii.edu] Sent: Friday, August 09, 2013 11:13 AM To: Tom Aldcroft Cc: Leo Singer; Adam Ginsburg; astropy Subject: Re: [AstroPy] Astropy demo presentations On a related note, I just received an email advertising the ability to create and distribute custom environments for Wakari (which is basically a cloud-hosted python environment you use from the browser: http://continuum.io/blog/introducing-bundles). I haven't yet found Wakari compelling for personal use, but I could see it being a nice way to run demos, especially those that rely on non-trivial installs. cheers, chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From bkloppenborg at gmail.com Thu Aug 15 04:16:02 2013 From: bkloppenborg at gmail.com (Brian Kloppenborg) Date: Thu, 15 Aug 2013 10:16:02 +0200 Subject: [AstroPy] Photometric magnitude to flux conversion code? In-Reply-To: References: Message-ID: <520C8E42.2030606@gmail.com> Greetings, Has anyone written or is anyone aware of some Python code to convert magnitudes into flux for a wide variety of photometric filters? Say V-band magnitude into Jy? I've seen several websites that can do this (i.e. http://www.gemini.edu/?q=node/11119, assuming some SED and/or extrapolation method), but I haven't seen anything like this in Python. I could code up something, but I'd rather not if something suitable already exists. Thanks! Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From stsci.perry at gmail.com Thu Aug 15 09:28:49 2013 From: stsci.perry at gmail.com (Perry Greenfield) Date: Thu, 15 Aug 2013 09:28:49 -0400 Subject: [AstroPy] Photometric magnitude to flux conversion code? In-Reply-To: <520C8E42.2030606@gmail.com> References: <520C8E42.2030606@gmail.com> Message-ID: <151508DF-94E9-430A-98B5-F23C7702F7E5@gmail.com> Are you familiar with pysynphot? That uses a synthetic photometry approach to such conversions, but I wasn't sure what kind of conversion you were thinking of. We are working now on porting pysynphot to astropy, but is is currently available as part of stsci_python (but not well documented). Perry On Aug 15, 2013, at 4:16 AM, Brian Kloppenborg wrote: > Greetings, > > Has anyone written or is anyone aware of some Python code to convert magnitudes into flux for a wide variety of photometric filters? Say V-band magnitude into Jy? I've seen several websites that can do this (i.e. http://www.gemini.edu/?q=node/11119, assuming some SED and/or extrapolation method), but I haven't seen anything like this in Python. I could code up something, but I'd rather not if something suitable already exists. > > Thanks! > Brian > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From jmwalawender at gmail.com Thu Aug 15 21:45:08 2013 From: jmwalawender at gmail.com (Josh Walawender) Date: Thu, 15 Aug 2013 15:45:08 -1000 Subject: [AstroPy] table string length truncated after reading table Message-ID: Hi all, I'm having a problem working with astropy.table and astropy.io.ascii and I can't tell if the behavior I'm encountering is a feature or a bug. I'm hoping someone can guide me to a good solution. Here's the situation: I have code which loops though a series of input data files, does analysis, stores the results in an astropy.table, and writes the table to a text file using astropy.io.ascii. One of the fields in the row is the input filename. As the code loops through the input files, it reads the previous table as output by io.ascii, appends a row to the table object, then overwrites the old file with a new one based on the new table which contains the new row. The symptom is that the all subsequent times through, the length of all strings written to the file name field are now truncated to whatever length the first file name was. Here's a quick test case (copied and pasted from iPython) demonstrating the problem: ``` In [1]: import astropy.table as table In [2]: import astropy.io.ascii as ascii In [4]: firstTable = table.Table(names=('col1', 'col2'), dtypes=('S50', 'S50')) In [5]: firstTable.add_row(('abcdefghijk', 'short_string')) In [6]: firstTable.add_row(('abcdefghijklmnop', 'long_string')) In [7]: print(firstTable) col1 col2 ---------------- ------------ abcdefghijk short_string abcdefghijklmnop long_string In [8]: ascii.write(firstTable, "mytable.txt") In [9]: secondTable = ascii.read("mytable.txt") In [10]: print(secondTable) col1 col2 ---------------- ------------ abcdefghijk short_string abcdefghijklmnop long_string In [11]: secondTable.add_row(('abcdefghijklmnopqrst', 'longer_string')) In [12]: print(secondTable) col1 col2 ---------------- ------------ abcdefghijk short_string abcdefghijklmnop long_string abcdefghijklmnop longer_strin ``` Any suggestions on how to avoid this behavior? thanks! Josh P.S. Based on the recent discussion on astropy-dev about where to get help, I've also posted this on astrobabel: http://www.astrobabel.com/v/discussion/77/astropy-question-table-string-length-truncated-after-reading-table#Item_1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mcraig at mnstate.edu Thu Aug 15 22:51:20 2013 From: mcraig at mnstate.edu (Matthew Craig) Date: Thu, 15 Aug 2013 21:51:20 -0500 Subject: [AstroPy] table string length truncated after reading table In-Reply-To: References: Message-ID: <7762F343-2856-4BAC-9045-F971E03C50AF@mnstate.edu> I ran into this once upon a time too (in atPy). The type for the columns when you read from a table is being guessed at by ascii; in the absence of any other guidance it assumes the dtype of each column is the length of the longest string it finds in the column. Doesn't look like there is a way to specify type in ascii.read (though I just did a quick skim of the docs), but this would work: ``` In [35]: thirdTable = table.Table(np.array(secondTable), names=('col1', 'col2'), dtypes=('S50', 'S50')) In [36]: thirdTable.add_row(('abcdefghijklmnopqrst', 'longer_string')) In [37]: print thirdTable col1 col2 -------------------- ------------- abcdefghijk short_string abcdefghijklmnop long_string abcdefghijklmnopqrst longer_string ``` Matt Craig PS Would have been happy to answer on astrobabel but waiting for my membership to be approved :) Office hours/schedule at: http://physics.mnstate.edu/craig ---- Professor Department of Physics and Astronomy Minnesota State University Moorhead 1104 7th Ave S, Moorhead MN 56563 phone: (218) 477-2439 fax: (218) 477-2290 On Aug 15, 2013, at 8:45 PM, Josh Walawender wrote: > Hi all, > > I'm having a problem working with astropy.table and astropy.io.ascii and I can't tell if the behavior I'm encountering is a feature or a bug. I'm hoping someone can guide me to a good solution. Here's the situation: > > I have code which loops though a series of input data files, does analysis, stores the results in an astropy.table, and writes the table to a text file using astropy.io.ascii. One of the fields in the row is the input filename. As the code loops through the input files, it reads the previous table as output by io.ascii, appends a row to the table object, then overwrites the old file with a new one based on the new table which contains the new row. > > The symptom is that the all subsequent times through, the length of all strings written to the file name field are now truncated to whatever length the first file name was. > > Here's a quick test case (copied and pasted from iPython) demonstrating the problem: > > ``` > In [1]: import astropy.table as table > > In [2]: import astropy.io.ascii as ascii > > In [4]: firstTable = table.Table(names=('col1', 'col2'), dtypes=('S50', 'S50')) > > In [5]: firstTable.add_row(('abcdefghijk', 'short_string')) > > In [6]: firstTable.add_row(('abcdefghijklmnop', 'long_string')) > > In [7]: print(firstTable) > col1 col2 > ---------------- ------------ > abcdefghijk short_string > abcdefghijklmnop long_string > > In [8]: ascii.write(firstTable, "mytable.txt") > > In [9]: secondTable = ascii.read("mytable.txt") > > In [10]: print(secondTable) > col1 col2 > ---------------- ------------ > abcdefghijk short_string > abcdefghijklmnop long_string > > In [11]: secondTable.add_row(('abcdefghijklmnopqrst', 'longer_string')) > > In [12]: print(secondTable) > col1 col2 > ---------------- ------------ > abcdefghijk short_string > abcdefghijklmnop long_string > abcdefghijklmnop longer_strin > ``` > > Any suggestions on how to avoid this behavior? > > thanks! > Josh > > P.S. Based on the recent discussion on astropy-dev about where to get help, I've also posted this on astrobabel: > http://www.astrobabel.com/v/discussion/77/astropy-question-table-string-length-truncated-after-reading-table#Item_1 > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy -------------- next part -------------- An HTML attachment was scrubbed... URL: From aldcroft at head.cfa.harvard.edu Fri Aug 16 08:05:14 2013 From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas) Date: Fri, 16 Aug 2013 08:05:14 -0400 Subject: [AstroPy] table string length truncated after reading table In-Reply-To: <7762F343-2856-4BAC-9045-F971E03C50AF@mnstate.edu> References: <7762F343-2856-4BAC-9045-F971E03C50AF@mnstate.edu> Message-ID: (Sent this last night but accidentally just to Josh). - Tom Hi Josh, The problem is that when you read the file back with ascii.read() it has no memory of the original 'S50' data type, so it just makes the string type as long as the longest one in the ASCII table file. Then when you add a row it is not able to fit the new longer string into the existing column so it just truncates. (This is the behavior of the underlying numpy array which is used by Table). Fortunately there is a way to force the data type when reading with ascii.read() with the converters argument. In your example below you would do the read with: >>> secondTable = ascii.read("mytable.txt", converters={'col1': [ascii.convert_numpy('S50')], 'col2': [ascii.convert_numpy('S50')]}) The idea here is that you are forcing the column to be converted from a Python list to a numpy array with a dtype of 'S50' instead of the normal default of guessing float, int, str. See http://astropy.readthedocs.org/en/latest/io/ascii/read.html#converters for the docs. - Tom On Thu, Aug 15, 2013 at 10:51 PM, Matthew Craig wrote: > I ran into this once upon a time too (in atPy). > > The type for the columns when you read from a table is being guessed at by > ascii; in the absence of any other guidance it assumes the dtype of each > column is the length of the longest string it finds in the column. > > Doesn't look like there is a way to specify type in ascii.read (though I > just did a quick skim of the docs), but this would work: > > ``` > In [35]: thirdTable = table.Table(np.array(secondTable), names=('col1', > 'col2'), dtypes=('S50', 'S50')) > > In [36]: thirdTable.add_row(('abcdefghijklmnopqrst', 'longer_string')) > > In [37]: print thirdTable > col1 col2 > -------------------- ------------- > abcdefghijk short_string > abcdefghijklmnop long_string > abcdefghijklmnopqrst longer_string > ``` > > Matt Craig > PS Would have been happy to answer on astrobabel but waiting for my > membership to be approved :) > > > Office hours/schedule at: http://physics.mnstate.edu/craig > ---- > Professor > Department of Physics and Astronomy > Minnesota State University Moorhead > 1104 7th Ave S, Moorhead MN 56563 > > phone: (218) 477-2439 > fax: (218) 477-2290 > > On Aug 15, 2013, at 8:45 PM, Josh Walawender wrote: > > Hi all, > > I'm having a problem working with astropy.table and astropy.io.ascii and I > can't tell if the behavior I'm encountering is a feature or a bug. I'm > hoping someone can guide me to a good solution. Here's the situation: > > I have code which loops though a series of input data files, does analysis, > stores the results in an astropy.table, and writes the table to a text file > using astropy.io.ascii. One of the fields in the row is the input filename. > As the code loops through the input files, it reads the previous table as > output by io.ascii, appends a row to the table object, then overwrites the > old file with a new one based on the new table which contains the new row. > > The symptom is that the all subsequent times through, the length of all > strings written to the file name field are now truncated to whatever length > the first file name was. > > Here's a quick test case (copied and pasted from iPython) demonstrating the > problem: > > ``` > In [1]: import astropy.table as table > > In [2]: import astropy.io.ascii as ascii > > In [4]: firstTable = table.Table(names=('col1', 'col2'), dtypes=('S50', > 'S50')) > > In [5]: firstTable.add_row(('abcdefghijk', 'short_string')) > > In [6]: firstTable.add_row(('abcdefghijklmnop', 'long_string')) > > In [7]: print(firstTable) > col1 col2 > ---------------- ------------ > abcdefghijk short_string > abcdefghijklmnop long_string > > In [8]: ascii.write(firstTable, "mytable.txt") > > In [9]: secondTable = ascii.read("mytable.txt") > > In [10]: print(secondTable) > col1 col2 > ---------------- ------------ > abcdefghijk short_string > abcdefghijklmnop long_string > > In [11]: secondTable.add_row(('abcdefghijklmnopqrst', 'longer_string')) > > In [12]: print(secondTable) > col1 col2 > ---------------- ------------ > abcdefghijk short_string > abcdefghijklmnop long_string > abcdefghijklmnop longer_strin > ``` > > Any suggestions on how to avoid this behavior? > > thanks! > Josh > > P.S. Based on the recent discussion on astropy-dev about where to get help, > I've also posted this on astrobabel: > http://www.astrobabel.com/v/discussion/77/astropy-question-table-string-length-truncated-after-reading-table#Item_1 > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > From jmwalawender at gmail.com Mon Aug 19 22:53:29 2013 From: jmwalawender at gmail.com (Josh Walawender) Date: Mon, 19 Aug 2013 16:53:29 -1000 Subject: [AstroPy] table string length truncated after reading table In-Reply-To: References: <7762F343-2856-4BAC-9045-F971E03C50AF@mnstate.edu> Message-ID: <562CEBC8-D8A4-4195-A6CD-050EA2AEB28C@gmail.com> Hi Tom, Matt, Thanks for the tips. The converters worked, I should have realized they would be the way to do it. thanks! Josh On Aug 16, 2013, at 2:05 AM, "Aldcroft, Thomas" wrote: > (Sent this last night but accidentally just to Josh). - Tom > > > Hi Josh, > > The problem is that when you read the file back with ascii.read() it > has no memory of the original 'S50' data type, so it just makes the > string type as long as the longest one in the ASCII table file. Then > when you add a row it is not able to fit the new longer string into > the existing column so it just truncates. (This is the behavior of > the underlying numpy array which is used by Table). > > Fortunately there is a way to force the data type when reading with > ascii.read() with the converters argument. In your example below you > would do the read with: > >>>> secondTable = ascii.read("mytable.txt", converters={'col1': [ascii.convert_numpy('S50')], 'col2': [ascii.convert_numpy('S50')]}) > > The idea here is that you are forcing the column to be converted from > a Python list to a numpy array with a dtype of 'S50' instead of the > normal default of guessing float, int, str. > > See http://astropy.readthedocs.org/en/latest/io/ascii/read.html#converters > for the docs. > > - Tom > > On Thu, Aug 15, 2013 at 10:51 PM, Matthew Craig wrote: >> I ran into this once upon a time too (in atPy). >> >> The type for the columns when you read from a table is being guessed at by >> ascii; in the absence of any other guidance it assumes the dtype of each >> column is the length of the longest string it finds in the column. >> >> Doesn't look like there is a way to specify type in ascii.read (though I >> just did a quick skim of the docs), but this would work: >> >> ``` >> In [35]: thirdTable = table.Table(np.array(secondTable), names=('col1', >> 'col2'), dtypes=('S50', 'S50')) >> >> In [36]: thirdTable.add_row(('abcdefghijklmnopqrst', 'longer_string')) >> >> In [37]: print thirdTable >> col1 col2 >> -------------------- ------------- >> abcdefghijk short_string >> abcdefghijklmnop long_string >> abcdefghijklmnopqrst longer_string >> ``` >> >> Matt Craig >> PS Would have been happy to answer on astrobabel but waiting for my >> membership to be approved :) >> >> >> Office hours/schedule at: http://physics.mnstate.edu/craig >> ---- >> Professor >> Department of Physics and Astronomy >> Minnesota State University Moorhead >> 1104 7th Ave S, Moorhead MN 56563 >> >> phone: (218) 477-2439 >> fax: (218) 477-2290 >> >> On Aug 15, 2013, at 8:45 PM, Josh Walawender wrote: >> >> Hi all, >> >> I'm having a problem working with astropy.table and astropy.io.ascii and I >> can't tell if the behavior I'm encountering is a feature or a bug. I'm >> hoping someone can guide me to a good solution. Here's the situation: >> >> I have code which loops though a series of input data files, does analysis, >> stores the results in an astropy.table, and writes the table to a text file >> using astropy.io.ascii. One of the fields in the row is the input filename. >> As the code loops through the input files, it reads the previous table as >> output by io.ascii, appends a row to the table object, then overwrites the >> old file with a new one based on the new table which contains the new row. >> >> The symptom is that the all subsequent times through, the length of all >> strings written to the file name field are now truncated to whatever length >> the first file name was. >> >> Here's a quick test case (copied and pasted from iPython) demonstrating the >> problem: >> >> ``` >> In [1]: import astropy.table as table >> >> In [2]: import astropy.io.ascii as ascii >> >> In [4]: firstTable = table.Table(names=('col1', 'col2'), dtypes=('S50', >> 'S50')) >> >> In [5]: firstTable.add_row(('abcdefghijk', 'short_string')) >> >> In [6]: firstTable.add_row(('abcdefghijklmnop', 'long_string')) >> >> In [7]: print(firstTable) >> col1 col2 >> ---------------- ------------ >> abcdefghijk short_string >> abcdefghijklmnop long_string >> >> In [8]: ascii.write(firstTable, "mytable.txt") >> >> In [9]: secondTable = ascii.read("mytable.txt") >> >> In [10]: print(secondTable) >> col1 col2 >> ---------------- ------------ >> abcdefghijk short_string >> abcdefghijklmnop long_string >> >> In [11]: secondTable.add_row(('abcdefghijklmnopqrst', 'longer_string')) >> >> In [12]: print(secondTable) >> col1 col2 >> ---------------- ------------ >> abcdefghijk short_string >> abcdefghijklmnop long_string >> abcdefghijklmnop longer_strin >> ``` >> >> Any suggestions on how to avoid this behavior? >> >> thanks! >> Josh >> >> P.S. Based on the recent discussion on astropy-dev about where to get help, >> I've also posted this on astrobabel: >> http://www.astrobabel.com/v/discussion/77/astropy-question-table-string-length-truncated-after-reading-table#Item_1 >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> >> >> >> _______________________________________________ >> AstroPy mailing list >> AstroPy at scipy.org >> http://mail.scipy.org/mailman/listinfo/astropy >> From erik.tollerud at gmail.com Mon Aug 19 22:53:20 2013 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Mon, 19 Aug 2013 22:53:20 -0400 Subject: [AstroPy] Photometric magnitude to flux conversion code? In-Reply-To: <520C8E42.2030606@gmail.com> References: <520C8E42.2030606@gmail.com> Message-ID: You might also take a look at the tools in astropysics.phot ( http://pythonhosted.org/Astropysics/coremods/phot.html). I wrote those for relatively simple tasks just like this. Hopefully that will also end up in astropy in the future (or perhaps just an interface like this that is backed by something full-featured like pysynphot), but this should work for now. On Thu, Aug 15, 2013 at 4:16 AM, Brian Kloppenborg wrote: > Greetings, > > Has anyone written or is anyone aware of some Python code to convert > magnitudes into flux for a wide variety of photometric filters? Say V-band > magnitude into Jy? I've seen several websites that can do this (i.e. > http://www.gemini.edu/?q=node/11119, assuming some SED and/or > extrapolation method), but I haven't seen anything like this in Python. I > could code up something, but I'd rather not if something suitable already > exists. > > Thanks! > Brian > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > > -- Erik -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonysmith80 at gmail.com Wed Aug 21 08:13:24 2013 From: anthonysmith80 at gmail.com (Anthony Smith) Date: Wed, 21 Aug 2013 13:13:24 +0100 Subject: [AstroPy] List page down: how to subscribe? Message-ID: Hi all, I was going to recommend this list to someone, but the list page seems to be down: http://mail.scipy.org/mailman/listinfo/astropy I couldn't find any obvious contact details on scipy.org (for the webmaster, say), hence this message... Anyone know who to contact about this? Or an alternative way to subscribe? For what it's worth, it seems that all the subscribe links are (currently) broken on http://scipy.org/scipylib/mailing-lists.html, as well as http://planet.scipy.org/ http://wiki.scipy.org/ and probably more. Cheers, Anthony -------------- next part -------------- An HTML attachment was scrubbed... URL: From embray at stsci.edu Wed Aug 21 10:32:01 2013 From: embray at stsci.edu (Erik Bray) Date: Wed, 21 Aug 2013 10:32:01 -0400 Subject: [AstroPy] List page down: how to subscribe? In-Reply-To: References: Message-ID: <5214CF61.9020607@stsci.edu> On 08/21/2013 08:13 AM, Anthony Smith wrote: > Hi all, > > I was going to recommend this list to someone, but the list page seems to be down: > > http://mail.scipy.org/mailman/listinfo/astropy > > I couldn't find any obvious contact details on scipy.org (for > the webmaster, say), hence this message... > > Anyone know who to contact about this? > > Or an alternative way to subscribe? > > For what it's worth, it seems that all the subscribe links are (currently) > broken on http://scipy.org/scipylib/mailing-lists.html, as well as > http://planet.scipy.org/ http://wiki.scipy.org/ and probably more. > > Cheers, > > Anthony I'm pretty sure the SciPy site is hosted on somebody's wrist watch. All the site's subdomains are down so I wonder if it's a DNS configuration issue or something. I asked on IRC if anyone knows anything, so we'll see. Erik From rothe at carleton.edu Thu Aug 22 15:19:30 2013 From: rothe at carleton.edu (Erin Roth) Date: Thu, 22 Aug 2013 14:19:30 -0500 (CDT) Subject: [AstroPy] Running X-ray Astronomy Software Using Python? Message-ID: <471619752.46003986.1377199170364.JavaMail.root@carleton.edu> Hi everyone, I am an undergraduate researcher working in the field of x-ray astronomy this summer. Specifically, I am analyzing data from the Swift XRT telescope. I don't have much programming experience, but have been using python this summer to try to automate the creation of a light curve. My adviser is a big supporter of the python movement for astronomers, but doesn't have much experience herself, and as a result I am looking for some guidance. In particular, I am trying to run the tool xselect from python. However, since this is rather specific, a more general description of what I am trying to do is I am hoping to run the ftools/HEAsoft software from python (currently I run ftools/HEAsoft through the terminal window). I'm not exactly sure where to start and as a result I have spent a lot of time searching the web for examples of how to do this, but haven't been able to find many. Do any of you happen to have any example codes related to this? Any examples and/or suggestions of where to start would be much appreciated! Thanks, Erin From aldcroft at head.cfa.harvard.edu Thu Aug 22 15:40:06 2013 From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas) Date: Thu, 22 Aug 2013 15:40:06 -0400 Subject: [AstroPy] Running X-ray Astronomy Software Using Python? In-Reply-To: <471619752.46003986.1377199170364.JavaMail.root@carleton.edu> References: <471619752.46003986.1377199170364.JavaMail.root@carleton.edu> Message-ID: Hi Erin, Just to get you started, here is some info that the CIAO (Chandra) team put together about running analysis tools from Python using the standard library subprocess module. The same idea applies for running CIAO tools and HEAsoft tools: http://cxc.harvard.edu/ciao/scripting/runtool.html#subprocess - Tom On Thu, Aug 22, 2013 at 3:19 PM, Erin Roth wrote: > Hi everyone, > > I am an undergraduate researcher working in the field of x-ray astronomy this summer. Specifically, I am analyzing data from the Swift XRT telescope. I don't have much programming experience, but have been using python this summer to try to automate the creation of a light curve. My adviser is a big supporter of the python movement for astronomers, but doesn't have much experience herself, and as a result I am looking for some guidance. In particular, I am trying to run the tool xselect from python. However, since this is rather specific, a more general description of what I am trying to do is I am hoping to run the ftools/HEAsoft software from python (currently I run ftools/HEAsoft through the terminal window). I'm not exactly sure where to start and as a result I have spent a lot of time searching the web for examples of how to do this, but haven't been able to find many. Do any of you happen to have any example codes related to this? Any examples and/or suggestions of wher > e to start would be much appreciated! > > Thanks, > Erin > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy From ma.stranger at o2.co.uk Thu Aug 22 19:42:08 2013 From: ma.stranger at o2.co.uk (Morris Stranger) Date: Fri, 23 Aug 2013 00:42:08 +0100 Subject: [AstroPy] help with I2py Message-ID: Dear astropy, My name is Morris Stranger. I have been translating IDL to Python for a short astronomy program at the university of Oxford and I came across the I2py translator which was the exact thing I was looking for to assist my translation however it doesn't seem to work for me. Many of the times I enter a piece of code. The error message "too many ()'s comes up" when I tried opening the I2py file on another computer I could not execute the code at all. Could you please help? Thanks, Morris Stranger. By the way my email is ma.stranger at o2.co.uk From john.parejko at yale.edu Fri Aug 23 01:49:35 2013 From: john.parejko at yale.edu (John K. Parejko) Date: Thu, 22 Aug 2013 23:49:35 -0600 Subject: [AstroPy] pyfits does not write to gzip ab+ objects Message-ID: <1B3E32CA-9C5D-4B1B-A915-9F2965050B06@yale.edu> Hello, On python2.7/pyfits3.1.2 , when I try to write to a gzip object in append mode, 'ab+', I receive an IOError: [Errno 9] read() on write-only GzipFile object. Whereas when I use mode 'ab', I receive no error. The former worked in pyfits 2.4-ish (what we were previously using). We use NamedTemporaryFiles to prevent accidental filename collision, and pass that through a gzip object to select the compression level, before writing with hdu.writeto(tempFile,checksum=True). I'm not sure whether pyfits or gzip is really the culprit here. I've attached an example. Thanks in advance, John # Example: import gzip import pyfits import numpy as np hdu=pyfits.PrimaryHDU(np.random.random(10)) #IOError: [Errno 9] read() on write-only GzipFile object f=gzip.GzipFile(filename='foo.fits.gz',mode='ab+',compresslevel=4) hdu.writeto(f) #works, resulting file is gzipped. f=gzip.GzipFile(filename='foo.fits.gz',mode='ab',compresslevel=4) hdu.writeto(f) From ejensen1 at swarthmore.edu Fri Aug 23 10:13:42 2013 From: ejensen1 at swarthmore.edu (Eric L. N. Jensen) Date: Fri, 23 Aug 2013 10:13:42 -0400 Subject: [AstroPy] Importing packages across distributions? In-Reply-To: References: Message-ID: <97F84E76-73F2-4C4C-AFE7-7B4145ABBDBC@swarthmore.edu> Hi all, This is an old thread, but I'm resurrecting it because now I'm encountering a twist on what I asked about previously. In May, I asked about how to install astropy into the python distribution inside CASA, and Tom helpfully provided the script referenced below, which worked great. Now, I'd like to do the same thing, but in the Linux version of CASA. Tom's script has paths that are specific to the Mac version. Has anyone succeeded in doing this (installing astropy into CASA) on linux? Thanks in advance for your help with this, Eric On May 6, 2013, at 6:35 PM, Thomas Robitaille wrote: > On 6 May 2013 23:30, Nathan Goldbaum wrote: >> Hi Eric, >> >> I think you need to recursively add all the subfolders in the anaconda >> folder to your sys.path. You can also manage all this from the command line >> by setting the PYTHONPATH. > > I strongly recommend against this - see my previous email. For a > start, CASA and Anaconda may be using different Python versions (in > fact they are) and some of the Anaconda packages may conflict with > CASA. > > The correct way to do it is to install the packages directly into CASA: > > https://github.com/astrofrog/casa-python > > Eric confirmed off-list that this worked. > > Cheers, > Tom > >> >> I'm not familiar with CASA, but presumably it should be possible to install >> it in a way that uses anaconda's python instead of relying on its own, that >> way you won't need to worry about this PYTHONPATH mess. Instructions on how >> to build CASA from source are available here: >> https://safe.nrao.edu/wiki/bin/view/Main/ScottRankinCasaBuildNotes >> >> Cheers, >> >> Nathan >> >> >> From embray at stsci.edu Tue Aug 27 11:49:08 2013 From: embray at stsci.edu (Erik Bray) Date: Tue, 27 Aug 2013 11:49:08 -0400 Subject: [AstroPy] pyfits does not write to gzip ab+ objects In-Reply-To: <1B3E32CA-9C5D-4B1B-A915-9F2965050B06@yale.edu> References: <1B3E32CA-9C5D-4B1B-A915-9F2965050B06@yale.edu> Message-ID: <521CCA74.3020607@stsci.edu> On 08/23/2013 01:49 AM, John K. Parejko wrote: > Hello, > > On python2.7/pyfits3.1.2 , when I try to write to a gzip object in append mode, 'ab+', I receive an IOError: [Errno 9] read() on write-only GzipFile object. Whereas when I use mode 'ab', I receive no error. > > The former worked in pyfits 2.4-ish (what we were previously using). We use NamedTemporaryFiles to prevent accidental filename collision, and pass that through a gzip object to select the compression level, before writing with hdu.writeto(tempFile,checksum=True). > > I'm not sure whether pyfits or gzip is really the culprit here. I've attached an example. > > Thanks in advance, > John > > # Example: > > import gzip > import pyfits > import numpy as np > > hdu=pyfits.PrimaryHDU(np.random.random(10)) > > #IOError: [Errno 9] read() on write-only GzipFile object > f=gzip.GzipFile(filename='foo.fits.gz',mode='ab+',compresslevel=4) > hdu.writeto(f) > > #works, resulting file is gzipped. > f=gzip.GzipFile(filename='foo.fits.gz',mode='ab',compresslevel=4) > hdu.writeto(f) I think that the gzip module does not actually support 'ab+' mode, and that if you pass that in it gets confused though it doesn't actually give you any useful errors up front. Essentially 'ab' is a write-only mode, but 'ab+' is read/write, so when you pass in 'ab+' it opens the underlying file object in a read/write mode but the GzipFile itself is write-only. In think somewhere this incongruity causes problems... In this case if you're just writing a new file 'wb' should suffice too. Erik From john.parejko at yale.edu Wed Aug 28 16:24:18 2013 From: john.parejko at yale.edu (John K. Parejko) Date: Wed, 28 Aug 2013 16:24:18 -0400 Subject: [AstroPy] pyfits does not write to gzip ab+ objects In-Reply-To: <521CCA74.3020607@stsci.edu> References: <1B3E32CA-9C5D-4B1B-A915-9F2965050B06@yale.edu> <521CCA74.3020607@stsci.edu> Message-ID: <2B44F5CE-AD46-4EA2-BBF9-C8A90913FD19@yale.edu> On 27Aug 2013, at 11:49, Erik Bray wrote: > I think that the gzip module does not actually support 'ab+' mode, and that if > you pass that in it gets confused though it doesn't actually give you any useful > errors up front. Essentially 'ab' is a write-only mode, but 'ab+' is > read/write, so when you pass in 'ab+' it opens the underlying file object in a > read/write mode but the GzipFile itself is write-only. In think somewhere this > incongruity causes problems... Except that 'ab+' worked just fine with pyfits 2.4 (I don't know which pyfits version first behaved differently). That's why I suspect it's a pyfits change, not a gzip change. > In this case if you're just writing a new file 'wb' should suffice too. I'll check up on whether 'ab+' was really necessary for our purposes. John -- ************************* John Parejko john.parejko at yale.edu http://www.astro.yale.edu/~jp727/ 203 432-9759 JWG 465 Department of Physics Yale University New Haven, CT ************************** From embray at stsci.edu Thu Aug 29 10:31:55 2013 From: embray at stsci.edu (Erik Bray) Date: Thu, 29 Aug 2013 10:31:55 -0400 Subject: [AstroPy] pyfits does not write to gzip ab+ objects In-Reply-To: <2B44F5CE-AD46-4EA2-BBF9-C8A90913FD19@yale.edu> References: <1B3E32CA-9C5D-4B1B-A915-9F2965050B06@yale.edu> <521CCA74.3020607@stsci.edu> <2B44F5CE-AD46-4EA2-BBF9-C8A90913FD19@yale.edu> Message-ID: <521F5B5B.3000504@stsci.edu> On 08/28/2013 04:24 PM, John K. Parejko wrote: > On 27Aug 2013, at 11:49, Erik Bray wrote: > >> I think that the gzip module does not actually support 'ab+' mode, and that if >> you pass that in it gets confused though it doesn't actually give you any useful >> errors up front. Essentially 'ab' is a write-only mode, but 'ab+' is >> read/write, so when you pass in 'ab+' it opens the underlying file object in a >> read/write mode but the GzipFile itself is write-only. In think somewhere this >> incongruity causes problems... > > Except that 'ab+' worked just fine with pyfits 2.4 (I don't know which pyfits version first behaved differently). That's why I suspect it's a pyfits change, not a gzip change. I see now, you are correct that there's some change in pyfits with respect to this. Actually, it's related to a fix meant to handle inconsistency between platforms in the behavior of "a+" mode. Different OS's and even different Python versions handle it differently, so it gets normalized to r+ by default (though pyfits still seeks to the end of the file for writing which is what the behavior of a+ is *supposed* to provide). >> In this case if you're just writing a new file 'wb' should suffice too. > > > I'll check up on whether 'ab+' was really necessary for our purposes. Unless you're appending to an existing file it isn't necessary. Though in that case 'rb+' would still suffice for now. The problem is that when you use ab+ the GzipFile object is write-only. As such PyFITS should not try to do any read calls on it. But its internal bookkeeping is a little messed up by the a+ mode normalization and so it ends up thinking the file might be writeable after all. Definitely something to fix. > John > > -- > ************************* > John Parejko > john.parejko at yale.edu > http://www.astro.yale.edu/~jp727/ > 203 432-9759 > JWG 465 > Department of Physics > Yale University > New Haven, CT > ************************** > > > > > > From wkerzendorf at gmail.com Thu Aug 29 11:11:44 2013 From: wkerzendorf at gmail.com (Wolfgang Kerzendorf) Date: Thu, 29 Aug 2013 17:11:44 +0200 Subject: [AstroPy] ESA Summer of Code in Space is starting Message-ID: Hi all, Despite the fact that summer is already fading, ESA Summer of Code in Space is just starting. Jeff Taylor was just selected to work with astropy-specutils for three months. First of all: Congratulations! His initial timeline will have to be shifted as ESA moved the deadline, but here is his grand plan: ----- 1st phase: Method 1 (2013-08-10 to 2013-09-13): Weeks 1-2: Background reading and familiarization with wavelength calibration, plus pseudocode design for Method 1. Week 3: Design the code in Python and begin peak finding. Week 4: Finish peak finding, begin peak centering. Week 5: Finish peak centering, fit mapping of pixel coordinates to wavelength coordinates using astropy.modelling. Mid-term review: 2013-09-14 to 2013-09-20 2nd phase: Method 2 (2013-09-21 to 2013-10-11): I've set aside 3 weeks for this phase, but I'm not sure how realistic that is as I need more information about what exactly needs to be done. 3rd phase: Wavelength mappings and wrapping the project up (2013-10-12 to 2013-11-01): Week 10: Creating wavelength models using astropy.modelling and creating a link to the fitted wavelengths. Week 11: Store the calibrated wavelength on disk using a variety of formats (e.g. ASCII, FITS, etc.) and also read those formats back into a python model again. Week 12: Final debugging and submission of project. ---------- Adam G. and I will be his primary supervisors, but I'm sure that if he has trouble the community is more than happy to help him. Welcome Jeff! Cheers, Wolfgang -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4145 bytes Desc: not available URL: From embray at stsci.edu Thu Aug 29 12:39:59 2013 From: embray at stsci.edu (Erik Bray) Date: Thu, 29 Aug 2013 12:39:59 -0400 Subject: [AstroPy] pyfits does not write to gzip ab+ objects In-Reply-To: <521F5B5B.3000504@stsci.edu> References: <1B3E32CA-9C5D-4B1B-A915-9F2965050B06@yale.edu> <521CCA74.3020607@stsci.edu> <2B44F5CE-AD46-4EA2-BBF9-C8A90913FD19@yale.edu> <521F5B5B.3000504@stsci.edu> Message-ID: <521F795F.6040000@stsci.edu> On 08/29/2013 10:31 AM, Erik Bray wrote: > On 08/28/2013 04:24 PM, John K. Parejko wrote: >> On 27Aug 2013, at 11:49, Erik Bray wrote: >> >>> I think that the gzip module does not actually support 'ab+' mode, and that if >>> you pass that in it gets confused though it doesn't actually give you any useful >>> errors up front. Essentially 'ab' is a write-only mode, but 'ab+' is >>> read/write, so when you pass in 'ab+' it opens the underlying file object in a >>> read/write mode but the GzipFile itself is write-only. In think somewhere this >>> incongruity causes problems... >> >> Except that 'ab+' worked just fine with pyfits 2.4 (I don't know which pyfits version first behaved differently). That's why I suspect it's a pyfits change, not a gzip change. > > I see now, you are correct that there's some change in pyfits with respect to > this. Actually, it's related to a fix meant to handle inconsistency between > platforms in the behavior of "a+" mode. Different OS's and even different > Python versions handle it differently, so it gets normalized to r+ by default > (though pyfits still seeks to the end of the file for writing which is what the > behavior of a+ is *supposed* to provide). Further exploration of this issue has revealed some "fun" bugs in the handling of append mode in Python's io module. I think what confused me about this before is that when you open a file in Python 3 with, say, 'ab+' mode (same issue applies to any mode with 'a' in it), Python completely discards the 'a' but *does* still open the raw file descriptor with O_APPEND flag. The behavior of this flag, on supported systems, is to seek to the end of the file before any write operation (even if the file is otherwise opened in a random-access mode). On systems where O_APPEND is not supported the fact that the file was opened in 'append' mode is completely lost; Python has no internal bookkeeping on this. In fact, the only way to test if it was opened with O_APPEND is to use fcntl. And again on systems without O_APPEND the 'a' mode is just completely dropped, leading to subtly different behavior. Furthermore, when using the IO module and opening a file in 'ab+' (or 'rb+' or 'wb+') you get back what's called a BufferedRandom object by default, and this is all kinds of buggy with respect to append: # Write a simple test file >>> f = open('test', 'wb') >>> f.write(b'test') 4 >>> f.close() # Reopen it an ab+ mode >>> f = open('test', 'ab+') >>> f.write(b'A') 1 >>> f.seek(0) 0 >>> f.read() # This returns what we would expect: the 'A' was appended b'testA' >>> f.seek(0) 0 >>> f.read(1) b't' >>> f.write(b'B') 1 >>> f.tell() # Tell should show that we're at the end of the file but... 2 >>> f.seek(0) 0 >>> f.read() # 'B' should have been appended but this shows it wasn't b'tBstA' >>> f.flush() >>> f.seek(0) 0 >>> f.read() # Ah, but actually it was; read() was showing us Python's buffer b'testAB' The point of all this is the BufferedRandom object doesn't even know the file was opened in append mode and does not behave correctly. The BufferedRandom has no idea that the raw file was opened in append mode, and gets itself all manner of confused. Anyways, this is no longer an astronomy issue so I'll stop talking about it here. Erik >>> In this case if you're just writing a new file 'wb' should suffice too. >> >> >> I'll check up on whether 'ab+' was really necessary for our purposes. > > Unless you're appending to an existing file it isn't necessary. Though in that > case 'rb+' would still suffice for now. The problem is that when you use ab+ > the GzipFile object is write-only. As such PyFITS should not try to do any read > calls on it. But its internal bookkeeping is a little messed up by the a+ mode > normalization and so it ends up thinking the file might be writeable after all. > Definitely something to fix. > >> John >> >> -- >> ************************* >> John Parejko >> john.parejko at yale.edu >> http://www.astro.yale.edu/~jp727/ >> 203 432-9759 >> JWG 465 >> Department of Physics >> Yale University >> New Haven, CT >> ************************** >> >> >> >> >> >> > > _______________________________________________ > AstroPy mailing list > AstroPy at scipy.org > http://mail.scipy.org/mailman/listinfo/astropy > From stuart at mumford.me.uk Fri Aug 30 11:38:44 2013 From: stuart at mumford.me.uk (Stuart Mumford) Date: Fri, 30 Aug 2013 16:38:44 +0100 Subject: [AstroPy] Announcing SunPy 0.3.0 Message-ID: Hello, It gives me great pleasure to announce the release of a new version of SunPy. This version has been rather too long in the making, but is here at last! SunPy 0.3 is now available through PyPI and pip: https://pypi.python.org/pypi/sunpy and via GitHub: https://github.com/sunpy/sunpy/releases/tag/v0.3.0 The biggest change in 0.3 is a shift away from our Map and Spectra datatypes inheriting numpy.ndarray to having their array in a `.data` attribute. This was done to make development of these objects easier and more flexible and also to improve our compatibility with Astropy. In the process of doing this the map submodule has undergone a massive refactor to streamline the creation and inheritance structure of the module. For more details about the changes in 0.3 see here: http://www.sunpy.org/2013/08/30/announcing-sunpy-0-3/ SunPy 0.3 consits of 9 months of work from 15 people and over 300 commits to the git repository. The people who have contributed to this release are (in commit order): Stuart Mumford Russell Hewett Florian Mayer Steven Christe Albert Shih Simon Liedtke Ankit Angrawal Jack Ireland Matt Bates Nabil Freij Keith Hughitt David Perez-Suarez Tomas Meszaros Benjamin Mampaey Andrew Leonard On behalf of all the SunPy developers, we hope you enjoy 0.3! Stuart Mumford -------------- next part -------------- An HTML attachment was scrubbed... URL: